van der Schaar Lab

Data monitoring

Identification and appropriate handling of inconsistencies in data at deployment time is crucial to using machine learning models reliably. Prior works have focused on out-of-distribution data. Instead, with data monitoring at test time, we develop ML methods to characterise incongruous and inconsistent in-distribution (ID) data regions, which may arise from feature space heterogeneity. Identifying such samples can guide understanding of why a model might fail on a specific data point and also guide future data collection. We have addressed this both from a representation learning, as well as, a causal structure perspective.

Read more:

Watch our Open House 2022 talk on Data Monitoring:

Watch our video on Data-Suite: