Once an incident has been given a status, we then want to understand the median time from that moment until the status is updated to fixed

Risk is a combination of frequency and severity. You can’t understand the severity of a data incident if you don’t also understand how important the underlying table is. Importance score, measured by the number of read/writes as well as downstream consumption at the BI level, is an important part of enabling data engineers to triage their incident response.

It is important to place the total number of incidents within the larger context of table uptime or the percentage of tables without an incident. This can be filtered by type of incident, by custom data freshness rules for example, to get a broad look at SLA adherence.

This metric measures the number of errors or anomalies across all of your data pipelines. A high number of incidents indicates areas where more resources need to be dedicated towards optimizing the data systems and processes.

If the importance score is the severity, the table health, or number of incidents a table has experienced over a certain time period provides the frequency end of the risk equation.

This metric measures the median time from an incident being created until a member of the data engineering team updates it with a status (which would typically be “investigating,” but could also be expected, no action needed, or false positive).

Crossover

Population of solutions

Fitness evaluation

Mutation

Select according to fitness

Нанесення елементів у зразок випадковим чином

Вибірка, яка спотворює генеральну сукупність

Більший набір даних або ідея набору даних

Вибірка, яка є результатом випадкової вибірки без стратифікації
населення

Однорідна підгрупа сукупності зі спільними ознаками

Поділ сукупності на страти та випадкова вибірка з кожної страти