Get the essential data observability guide
Download this guide to learn:
What is data observability?
4 pillars of data observability
How to evaluate platforms
Common mistakes to avoid
The ROI of data observability
Unlock now
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Sign up for a free data observability workshop today.
Assess your company's data health and learn how to start monitoring your entire data stack.
Book free workshop
Sign up for news, updates, and events
Subscribe for free
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
Getting started with Data Observability Guide

Make a plan to implement data observability across your company’s entire data stack

Download for free
Book a data observability workshop with an expert.

Assess your company's data health and learn how to start monitoring your entire data stack.

Book free workshop

Two custom SQL monitor upgrades

You can now choose the machine learning model that you want your custom SQL monitor to use when making predictions, plus you can now choose whether the results are displayed as integers, decimals, percents, or units of time.

and
October 13, 2023

Product, Design

October 13, 2023
Two custom SQL monitor upgrades

Metaplane’s custom SQL monitors have always been a powerful companion to Metaplane’s out-of-the-box monitors (like row count, freshness, and nullness, among others). Custom SQL monitors let you apply machine learning-based anomaly detection to any query you can express with SQL. Even better, they’re flexible enough to monitor a single value, or to monitor a group of values, with individual models built for each.

Sometimes, custom SQL monitors are the best way to capture complex logic that out-of-the-box monitors cannot. For example, maybe you want to write a row count monitor that groups your product activity by customer—but the logic that defines a customer can’t be captured in a single column, and you have to do it with SQL. Or imagine you want to create a freshness monitor, but there’s one of several columns that could indicate how fresh the data is.

And while our default custom SQL models work for anomaly detection of all shapes and sizes, the  models for our out-of-the-box monitors are custom-built to spot the specific sorts of issues that tend to pop up when monitoring specific types of metadata—like unusual row counts, or weird freshness patterns.

In light of that, we’re thrilled to announce that our custom SQL monitors are now just a little bit more powerful. In addition to the default custom SQL model, you can now apply any of Metaplane’s out-of-the-box models to your custom SQL monitor. So in the example above, where you’re building a row count monitor with advanced grouping logic? Apply the row count model in lieu of the default model for more finely-tuned predictions. And if you’re creating a freshness monitor that’s evaluating several columns at once? Apply the freshness model so that you can take advantage of Metaplane’s proprietary freshness modeling.

If you’re not sure which model to add to your custom SQL monitor, we’d recommend sticking with the default, which works well for a wide variety of monitoring use cases, from business metrics to sanity checks and everything in between.

Alongside this feature comes a small, quality-of-life upgrade for custom SQL monitors. Previously, if you had a monitor that returned a percent, decimal, or number of seconds, those values would always show up as integers on the monitor page in the table and the graph. Now, if your custom SQL monitor returns one of these types of values, you can select which format you want to see that number displayed as. Choosing “Percent” will render those values as a percent, choosing “Time (in seconds)” will render those values in human-readable minute, hour, day, week, and year increments, and so on.

We hope these updates make your custom SQL monitors a bit more flexible and easier to use.

Table of contents

    Tags

    We’re hard at work helping you improve trust in your data in less time than ever. We promise to send a maximum of 1 update email per week.

    Your email
    No items found.
    Ensure trust in data

    Start monitoring your data in minutes.

    Connect your warehouse and start generating a baseline in less than 10 minutes. Start for free, no credit-card required.