Project: Uncertainty Quantification via Test Time Augmentation
Description
Test-Time Augmentation (TTA) refers to applying data augmentations not only during training, but also during inference. For a single input, such as an image, the model is evaluated on multiple augmented versions of that input, for example rotated or cropped variants. The final prediction is then obtained by averaging the predictions across these augmentations.
TTA is often used to improve predictive performance, but it may also provide useful information about the model's uncertainty. Intuitively, if a model gives similar predictions across a range of (label-preserving) augmentations, this suggests that its prediction is stable and, hence, reliable. In contrast, strong variation across predictions may indicate high uncertainty. This makes TTA an interesting tool from the perspective of uncertainty quantification.
In this thesis, you will study TTA as a method for improving and analyzing uncertainty estimates in neural networks. The goal is not only to evaluate whether TTA helps, but also to better understand why it helps, when it fails, and how it can be improved.
Possible research directions include:
- investigating under which conditions TTA improves uncertainty estimation or when it fails
- studying how prediction variability across augmentations relates to internal feature representations in different layers of the network;
- designing or learning augmentation strategies that induce informative and diverse predictions, with the aim of improving uncertainty estimation.
Depending on the direction taken, the thesis may involve both empirical experiments and theoretical analysis.
Interesting Reading:
- Deep Ensembles: https://arxiv.org/pdf/1612.01474
- TTA: https://openaccess.thecvf.com/content/ICCV2021/papers/Shanmugam_Better_Aggregation_in_Test-Time_Augmentation_ICCV_2021_paper.pdf
- TTA & Uncertainty: https://arxiv.org/pdf/1807.0735
- Randomized Smoothing: https://arxiv.org/pdf/1902.02918
Details
- Supervisor
-
Sibylle Hess
- Secondary supervisor
-
Fabian Denoodt
- Interested?
- Get in contact