While her official title at Mux is Senior Analytics Engineer, she is known as the company’s data swiss army knife. 

Since joining the team in April 2021, Marion has been responsible for the full spectrum of data engineering from warehousing data in Snowflake to transforming it in dbt into analytics-ready data models. The assets she creates power Looker dashboards used by her company’s marketing, sales, customer success, finance, and product teams. 

The only problem? Marion felt blind in her early days on the job.

In her past role, Marion had more peace of mind. Leveraging Airflow, Datadog, and some simple dashboarding, she had built a custom data observability tool that notified her when something broke. That gave her some visibility into the quality of her company’s data. But, although it took her weeks to conceptualize and build, it only provided her with about 60% coverage.

When she got to Mux, Marion tried to replicate core tests using tools already at her disposal. While dbt was an irreplaceable tool for transforming data, dbt tests were best suited for quality checks on a small number of tables with manual thresholds. Extending coverage across the warehouse, especially on data that changes, required time that she didn't have.

Searching for a best-in-class data observability tool 

“Can we detect corrupt data before it is consumed?” 

That was the question that led Marion to search for an end-to-end data observability tool. 

“If your data is corrupt, you won’t know until your stakeholders tell you, which is an issue because you didn’t catch it beforehand,” Marion said. “And if you lose the trust of your data consumers, you’re basically out of luck because people start using Excel and Google Sheets.” 

Marion also wanted an easy way to view her data lineage. Without it, she struggled to trace the cause of issues and had no idea which dashboards were impacted by them. 

To start, Marion did proof-of-concepts with other vendors.

“The other options were good but overly complex and tedious to implement,” Marion said.

They included a large surface area of functionality, but not necessarily in a good way for Mux.

“For what I needed, it would have been boiling the ocean,” Marion said. “And paying for it too.” 

Marion would have to pay for features that she didn’t use, on top more Snowflake spend from less efficient queries that didn’t seem to leverage existing metadata.

Competitor sales processes were also a bit much for her, along with the amount of custom business logic she had to write. 

“I felt locked in by their sales processes,” Marion said. “It’s hard to escape vendor lock-in once you’ve invested so much time into something.”

Marion didn’t have a defined use case or specific KPIs to track going into the proof-of-concepts. Without those, the tools seemed complicated. As a starting point, she just wanted a baseline of test coverage for all tables.

While searching for the optimal solution, Marion stumbled across a reference to Metaplane in dbt’s Slack community. She was instantly excited by the self-serve setup process.

“In just a few clicks, I was able to connect my data stack and set up monitoring that was much more comprehensive than what I had built at my previous company,” Marion said. 

Reclaiming peace of mind from accurate alerts

“Within a couple of days, I was alerted to an anomaly—one that mattered.”

Marion explained that, during her evaluations of other data observability tools, the alerts she received didn’t identify real issues, which quickly resulted in alert fatigue.

“With Metaplane, more than 80% of alerts are legitimate issues that require my attention,” Marion said. “The product speaks for itself. If something breaks, you’ll know.”

For example, Metaplane recently detected a table that suddenly changed in granularity, growing from millions of rows to billions overnight.

“If Metaplane hadn’t alerted me, I wouldn’t have known until it was too late—until my stakeholders flagged it,” Marion said. “As a result, Snowflake would have been slow to run queries and Looker dashboards would have experienced downtime.”

Marion estimates that Metaplane, which notifies her about anomalies three times per week on average, saved her eight hours that day.

“We were able to identify which table was impacted right away, so we knew where to look,” Marion said. “Otherwise, we would have had to go from a downstream table that was corrupt and work backwards from there.”

More importantly, the Metaplane alert empowered Marion to preserve stakeholder trust in the company’s data.

Gaining visibility into data quality and performance

Between Metaplane’s anomaly detection, lineage, and usage analytics features, Marion feels way more confident these days.

“Metaplane has everything I need in one place,” Marion said. “It gives me full visibility into the quality and performance of my data.”

Her advice for others?

“Start small,” Marion said. “Get a baseline of testing established, then build on that to meet your other needs.”