Manual health monitoring of pipeline failures, data discrepancies and load issues leads to the reduced efficiency of the operations monitoring team. This process is resource, cost, and time intensive as the team uses multiple tools to identify the issues.
Problem statement
Operations monitoring dashboard features
Operations monitoring dashboard potential benefits
Operations monitoring dashboard: User personas
| Designation | Business user | Operations support & engineering leadership | Platform operations engineer |
|---|---|---|---|
| Task | Consumes sales data daily, to generate reports and take business decisions | Responsible for data observability to manage data quality, cost, and pipeline concerns | Monitors pipelines and resolves pipeline failure issues raised by analyst or business team |
| Pain points | Due to pipeline failures, he is unable to see complete data and take business decisions | Inability to timely detect failures and lack of clear visibility into failures | The process of identifying issues with data pipeline is manual and time consuming, and requires the use of multiple tools |
| Ideal experiences | Less turnaround time for resolution of pipeline issues, so that he can take decisions timely thereby not having major business impacts | Holistic view of pipeline health for timely and accurate detection of failures | Easy-to-use capability that will handle monitoring and reporting. It should give precise results at required frequency and reasonable costs |