When missing values come out, certain model performance is decreased

Even for models that can handle missing values, they might be sensitive to it

Missing values are generated by collection errors or missing observations

Missing values are the most common problem in data analytics

Feature reduction

Data cleaning and/or statistical preprocessing

Building machine learning models on the data

Feature selection

Data lacks attributes or contains missing values

Data has thousands of attributes

Data contains incorrect records such as outliers

Data contains conflicting records or discrepancies

By calculating similarity among data samples with annotations

By calculating the overall agreement and reliability of a data set

By calculating the accuracy of an annotator compared to ground-truth data

By measuring inter-annotator agreement on a per-task basis to generate a confidence score for each training data label

Repair: Let the Bad Data Find You and Then Fix Things

Detection: Proactively Look for Bad Data Already Entered

Allocating Resources: How Much for Prevention, Detection, and Repair

Prevention: Keep Bad Data Out of the Database/List