We chatted with Anthony Khoudary, Senior Analytics Engineer about how and why they chose Metaplane to improve their data quality and got this insider tip on Parachute Home’s offerings:

❝ The mattress is actually amazing. You can tell right away the moment you sit on it. If you live close to one of our locations, you’ll be able to tell right away. I switched over from Casper and would never go back.

You don’t need to take our word for it. At the date of publishing, Parachute Home is in the midst of a 20% off all items sale, which I’m taking advantage of to double down on new bed sheets - head over to their website to see if the sale is still on!

And key to understanding the success of this sale will be Anthony and the data team. After all, they’ve set up self service analytics to support the entire company - with a lofty vision.

❝ We will never be done with self service analytics until business users run out of questions. Our goal is to make it possible for the business to answer any question through Looker.

Building Self Service Analytics at Parachute Home

When it comes to self service analytics, many have tried, but few have succeeded. Of note is the fact that the data team only consisted of two people, including Anthony, when he first began. Some of the questions unique to companies in Parachute’s industry, with both an brick-and-mortar and e-commerce presence, that their team have answered, include: retail analytics, store foot traffic, order fulfillment, and of course, raising customer satisfaction.

The idea of self-service analytics often lead to heated debates, with detractors citing objections such as “complexity of data”, “incomplete guardrails”, and others. Here were Anthony and the data team’s solutions to these common objections:

Key to this effort was the addition of Kalie Pawlik to the data team:

❝ Most people don’t like doing documentation. We’re lucky to have an amazing data analyst in Kalie who did a phenomenal job setting up much of our self-service analytics program, and also has been amazing with Looker Documentation and training too.

For a team that wants to enable users to explore anything and everything on their own, data quality rises to the forefront. For them, it wasn’t just a matter of retaining the trust that they’ve built with their stakeholders, but also the desire to take more ownership of their data to fix issues before any negative impacts came up.

Tackling data quality issues with Metaplane

‍

Business stakeholders would point out issues in the dashboards that they were seeing, some of which were easily solvable, but others were less immediately recognizable.

❝ We had a bug where tax amount was included in the item_refund_revenue from Netsuite...causing our refunded revenue to appear higher. Because we caught this anomalous figure...we were able to update our stakeholders that nothing had actually changed with refunds and could fix things extremely quickly. Metaplane was key to alerting us of our higher than expected refunded revenue.

As dbt users, Anthony and his team already had the forethought to put guardrails in place through dbt tests, but explained how a few technical constraints with their data stack and scope of dbt tests led to their evaluation of Metaplane.

❝Every dbt user gets to the point where tests aren’t enough (to capture all incidents). We have Fivetran loading data multiple times a day, but our dbt jobs only run overnight, once daily. These different cadences mean that it’d be hard to capture production issues throughout the day by solely relying on dbt tests. We also needed to go beyond what dbt tests could look for. We wanted to know not only whether every row had a unique primary key, but also whether our order volume was anomalous. We wanted to know about issues that we wouldn’t have necessarily known to check for.

Now, more than a year into their use of Metaplane, they’ve been able to capture those order integrity issues along with finding so much more.

Key features to improve data quality

‍

Anthony and the Parachute Home data team use a combination of Metaplane’s machine-learning created thresholds and manual definitions to receive alerts on anomalous data ranging from:

• Freshness: Missed pipeline loads caused by situations such as a 3rd party API outage
• Row Count: These help with order volume, but also are early indicators of potential data quality issues, so these are deployed across the entire warehouse
• Sum + Custom SQL: These monitors cover order volume associated with different sources, to ensure that different distribution channels are functioning as expected.

Parachute Home uses NetSuite as their ERP, which allows them to track orders.

❝ We want to know if order volume ever declines below what’s expected because that’s key to the business. In one instance, someone had mistakenly updated permissions for the NetSuite role being used by Fivetran to sync data, which blocked access to those orders being synced. Because they could catch the problem early, they were able to resolve it before any order reporting was affected.

Schema change tracking

Beyond issues found by tracking data quality metrics, Anthony also specifically called out Metaplane’s schema evolution alerts as being useful. The Parachute Home team uses Hubspot for marketing needs and again uses Fivetran to sync the data to Snowflake. Two things to note about how the Fivetran connector works:

1. Each property in HubSpot gets written to its own column. A typical HubSpot “contact” table will often have tens of properties, leading to tens of columns in the warehouse target.
2. Schema drift is handled automatically by default. This means that any custom field name changes or new custom fields are automatically synced to the corresponding column in the data warehouse target. 

❝ We made a dbt model off of HubSpot data that relies on several properties. One of the teams had gone into HubSpot and changed the field name to represent a change to the business process. Seeing that change through Metaplane let us know that we had to go in and change the code. It’s even more interesting when we see deleted columns, because Fivetran doesn’t delete columns by default, so this typically indicates some dramatic unexpected behavior for us to look into.

Lineage

‍

When a data quality incident is surfaced, even for someone like Anthony, who built ~80% of the dbt code, Metaplane’s automatically generated lineage has been useful to connect the dots between issue to impact.

❝It’s really simple to look at the graph and identify all of the dashboards that are affected by the upstream model, so that I can quickly just alert stakeholders that we’re already investigating a fix.

Data CI/CD for Incident Prevention

Anthony and the team are also looking forward to using Metaplane’s Github app to datadiff their dbt pull requests (PRs) for model updates. This feature, which we call Data CI/CD, was built for teams to prevent issues caused by these model updates. Users can see which tables, dashboards, and models are downstream and how fields within each of them are affected by an update directly within their PR workflow.

Recap

• Parachute Home safeguards order integrity to retain customer satisfaction levels with the use of Metaplane’s monitors
• Schema change notifications help bridge the gap between upstream system administrators and downstream analytics changes 
• Lineage graphs generated for the data stack help with impact analysis and stakeholder notification when issues arise

❝ In summary, we use Metaplane to run tests on things that dbt can’t catch, either because we wouldn’t have known to catch it before the issue came or just due to the binary nature of dbt test setup. We want to fix all of these issues before the business or customers ever notice.

‍