Combining Transformer Generators with Convolutional Discriminators
- Transformer models have recently attracted much interest from computer vision researchers and have since been successfully employed for several problems traditionally addressed with convolutional neural networks. At the same time, image synthesis using generative adversarial networks (GANs) has drastically improved over the last few years. The recently proposed TransGAN is the first GAN using onlyTransformer models have recently attracted much interest from computer vision researchers and have since been successfully employed for several problems traditionally addressed with convolutional neural networks. At the same time, image synthesis using generative adversarial networks (GANs) has drastically improved over the last few years. The recently proposed TransGAN is the first GAN using only transformer-based architectures and achieves competitive results when compared to convolutional GANs. However, since transformers are data-hungry architectures, TransGAN requires data augmentation, an auxiliary super-resolution task during training, and a masking prior to guide the self-attention mechanism. In this paper, we study the combination of a transformer-based generator and convolutional discriminator and successfully remove the need of the aforementioned required design choices. We evaluate our approach by conducting a benchmark of well-known CNN discriminators, ablate the size of the transformer-based generator, and show that combining both architectural elements into a hybrid model leads to better results. Furthermore, we investigate the frequency spectrum properties of generated images and observe that our model retains the benefits of an attention based generator.…
Document Type: | Conference Proceeding |
---|---|
Conference Type: | Konferenzartikel |
Zitierlink: | https://opus.hs-offenburg.de/5292 | Bibliografische Angaben |
Title (English): | Combining Transformer Generators with Convolutional Discriminators |
Conference: | German Conference on AI (44. : September 27-October 1 2021 : Virtual Event) |
Author: | Ricard Durall Lopez, Stanislav Frolov, Jörn Hees, Federico Raue, Franz-Josef Pfreundt, Andreas Dengel, Janis KeuperStaff MemberORCiDGND |
Year of Publication: | 2021 |
Place of publication: | Cham |
Publisher: | Springer |
First Page: | 67 |
Last Page: | 79 |
Parent Title (English): | KI 2021: Advances in Artificial Intelligence |
Editor: | Stefan Edelkamp, Ralf Möller, Elmar Rueckert |
Volume: | LNCS 12873 |
ISBN: | 978-3-030-87625-8 (Print) |
ISBN: | 978-3-030-87626-5 (Online) |
DOI: | https://doi.org/10.1007/978-3-030-87626-5_6 |
URL: | https://link.springer.com/chapter/10.1007/978-3-030-87626-5_6 |
Language: | English | Inhaltliche Informationen |
Institutes: | Fakultät Elektrotechnik, Medizintechnik und Informatik (EMI) (ab 04/2019) |
Forschung / IMLA - Institute for Machine Learning and Analytics | |
Institutes: | Bibliografie | Formale Angaben |
Open Access: | Closed Access |
Licence (German): | Urheberrechtlich geschützt |