Volltext-Downloads (blau) und Frontdoor-Views (grau)

Improving Native CNN Robustness with Filter Frequency Regularization

  • Neural networks tend to overfit the training distribution and perform poorly on out-ofdistribution data. A conceptually simple solution lies in adversarial training, which introduces worst-case perturbations into the training data and thus improves model generalization to some extent. However, it is only one ingredient towards generally more robust models and requires knowledge about the potentialNeural networks tend to overfit the training distribution and perform poorly on out-ofdistribution data. A conceptually simple solution lies in adversarial training, which introduces worst-case perturbations into the training data and thus improves model generalization to some extent. However, it is only one ingredient towards generally more robust models and requires knowledge about the potential attacks or inference time data corruptions during model training. This paper focuses on the native robustness of models that can learn robust behavior directly from conventional training data without out-of-distribution examples. To this end, we study the frequencies in learned convolution filters. Clean-trained models often prioritize high-frequency information, whereas adversarial training enforces models to shift the focus to low-frequency details during training. By mimicking this behavior through frequency regularization in learned convolution weights, we achieve improved native robustness to adversarial attacks, common corruptions, and other out-of-distribution tests. Additionally, this method leads to more favorable shifts in decision-making towards low-frequency information, such as shapes, which inherently aligns more closely with human vision.show moreshow less

Export metadata

Additional Services

Search Google Scholar

Statistics

frontdoor_oas
Metadaten
Document Type:Article (unreviewed)
Zitierlink: https://opus.hs-offenburg.de/8236
Bibliografische Angaben
Title (English):Improving Native CNN Robustness with Filter Frequency Regularization
Author:Jovita Lukasik, Paul GavrikovStaff MemberORCiDGND, Janis KeuperStaff MemberORCiDGND, Margret Keuper
Year of Publication:2023
First Page:1
Last Page:36
Parent Title (English):Transactions on Machine Learning Research
Issue:12/2023
ISSN:2835-8856
URL:https://openreview.net/pdf?id=2wecNCpZ7Y
Language:English
Inhaltliche Informationen
Institutes:Fakultät Elektrotechnik, Medizintechnik und Informatik (EMI) (ab 04/2019)
Forschung / IMLA - Institute for Machine Learning and Analytics
Institutes:Bibliografie
Tag:Deep Leaning
Formale Angaben
Relevance:Wiss. Zeitschriftenartikel unreviewed
Open Access: Open Access 
 Bronze 
Licence (German):License LogoCreative Commons - CC BY - Namensnennung 4.0 International