Data Engineer Persona + LEDOW v1

Overview Demo

Bigeye for Data Engineers

All Sources | Bigeye

Pipeline Reliability, Autometrics and Autothresholds

Part 1: Deploying Data Quality Metrics at Scale

TOOY_DEMO_V2.PROD_REPL | Bigeye

Get started with monitoring. Automatically deploy Nulls, Duplicates, Freshness, Volume, Min, Max, and Average metrics on all upstream connected columns.

TOOY_DEMO.PROD_REPL | Bigeye

Freshness | Bigeye

ORDERS | Bigeye

Grouped Metrics, Autothresholds, Backfills, Windowing

Part 2: Tracking Data Anomalies Automatically Across Dimensions

No schedule
5 minutes
15 minutes
30 minutes
60 minutes
3 hours
6 hours
12 hours
24 hours

The lookback window is calculated from the latest timestamp in the row creation time column. Data is then aggregated into buckets based on bucket size.

None
Autothreshold
Relative
Standard deviation
Constant

Select column
CUSTOMER_ID
ORDER_ID
ORDER_TIMESTAMP
PAYMENT_METHOD_ID
PRICE_PER_UNIT
PRODUCT_TYPE
QUANTITY
SHIPPING_ADDRESS_1
SHIPPING_ADDRESS_2
SHIPPING_CITY
SHIPPING_STATE
SHIPPING_ZIP

Conditions

Add conditions using SQL (optional)

Average | Bigeye

Issue Management, Integrations, Root Cause Analysis

Part 3: Data Incident Management

demo_kyle-tooy-analytics (Channel) - Bigeye - Slack

The average value is too low in 1 of 17 groups on PRICE_PER_UNIT | Bigeye

Has issues — View 10 issues for PRICE_PER_UNIT
PRICE_PER_UNIT
Has issues — View 1 issue for ORDER_TIMESTAMP
ORDER_TIMESTAMP
orders
Has issues — View 2 issues for orders
Issues: 2
Expand node orders
orders
Root cause — View 13 issues for orders
Root causes: 2
Expand node orders
ORDERS
Has issues — View 11 issues for ORDERS
Issues: 11
View options for ORDERS
Collapse node ORDERS
PRICE_REPORTING
At risk - View issues for PRICE_REPORTING
At risk
Expand node PRICE_REPORTING
SALES_DASHBOARD
Has issues — View 3 issues for SALES_DASHBOARD
Issues: 3
Expand node SALES_DASHBOARD
ORDER_ANALYTICS
Has issues — View 3 issues for ORDER_ANALYTICS
Issues: 3
Expand node ORDER_ANALYTICS
s3_lake
AwsDataCatalog.tooy_demo_db_iceberg
Tables: 1
Has issues — View 2 issues for AwsDataCatalog.tooy_demo_db_iceberg
Issues: 2
Collapse node AwsDataCatalog.tooy_demo_db_iceberg
pg_webapp
tooy_demo_db.prod
Tables: 1
Root cause — View 13 issues for tooy_demo_db.prod
Root cause: 1
Collapse node tooy_demo_db.prod
snowflake_warehouse
TOOY_DEMO.PROD_REPL
Tables: 1
Has issues — View 11 issues for TOOY_DEMO.PROD_REPL
Issues: 11
Collapse node TOOY_DEMO.PROD_REPL
snowflake_warehouse
TOOY_DEMO.CONFORMED
Tables: 3
Has issues — View 6 issues for TOOY_DEMO.CONFORMED
Issues: 6
Collapse node TOOY_DEMO.CONFORMED
Demo-demo_cloud
Retail Analytics Workbook
Nodes: 4
At risk
Expand node Retail Analytics Workbook

price_per_unit | Bigeye

Min is alerting on price_per_unit | Bigeye

Root Cause Issue Filtering

Part 4: Reduce Alert Fatigue

Issues | Bigeye

Scorecards, Dashboard

Part 5: Data Quality Reporting

snowflake_warehouse | Bigeye

Topics covered: pipeline reliability, autometrics, autothresholds, issues, integrations, reporting. Please book a custom demo to see how we can help your team reduce data issue MTTR, increase data uptime and reduce manual effort spent on data quality.