EREN

Motivation

Retraining a large vision backbone for robustness is expensive, and it has to be redone every time the backbone changes. Worse, the gains are narrow: a model augmented against a fixed corruption list learns those corruptions, as our work on MUFIA showed, rather than the underlying sensitivity.

A pre-processing front-end is a different bargain. It sits in front of any classifier, it is small enough to train quickly, and it can be swapped out without touching the deployed model. The question is whether a front-end can actually recover the signal a corruption destroyed, rather than just smoothing the image until the classifier stops complaining.

Approach

EREN is a differentiable normalisation and restoration stage placed between the input and a frozen classifier. Because the whole path is differentiable, it can be trained through the classifier's loss: the front-end learns to produce inputs that the specific downstream model reads correctly, rather than inputs that merely look clean to a human.

• Frozen backbone. The classifier is never updated, so existing accuracy on clean data is preserved by construction and the front-end transfers across deployments.
• Adaptive normalisation. Corruption statistics vary per image; the normalisation is conditioned on the input rather than fixed at training time.
• Real-time budget. The stage is deliberately small: the point is a front-end you can afford to run on every frame.

Evaluation

We evaluate against common natural corruptions and against frequency-domain attacks, which are the harder test: a front-end that only learned to undo Gaussian blur will not survive a corruption it has never seen. Alongside accuracy we report clean-accuracy retention, since a defence that quietly costs several points on uncorrupted data is not a defence anyone will deploy.

Status

Under review, 2024. Part of my doctoral work at EPFL on robust vision models.