Refine
Document Type
- Conference Proceeding (46)
- Article (unreviewed) (16)
- Article (reviewed) (10)
- Book (2)
- Part of a Book (2)
- Contribution to a Periodical (1)
- Doctoral Thesis (1)
- Report (1)
Conference Type
- Konferenzartikel (44)
- Konferenz-Abstract (1)
- Sonstiges (1)
Is part of the Bibliography
- yes (79)
Keywords
- Deep Leaning (11)
- Machine Learning (9)
- Robustness (4)
- Data Science (3)
- Generative Adversarial Network (3)
- image classification (3)
- Aliasing (2)
- Benutzererlebnis (2)
- CNNs (2)
- Computer Vision (2)
Institute
- IMLA - Institute for Machine Learning and Analytics (79) (remove)
Open Access
- Open Access (56)
- Bronze (16)
- Closed Access (11)
- Closed (10)
- Diamond (9)
- Gold (4)
- Hybrid (3)
- Grün (2)
Following their success in visual recognition tasks, Vision Transformers(ViTs) are being increasingly employed for image restoration. As a few recent works claim that ViTs for image classification also have better robustness properties, we investigate whether the improved adversarial robustness of ViTs extends to image restoration. We consider the recently proposed Restormer model, as well as NAFNet and the "Baseline network" which are both simplified versions of a Restormer. We use Projected Gradient Descent (PGD) and CosPGD for our robustness evaluation. Our experiments are performed on real-world images from the GoPro dataset for image deblurring. Our analysis indicates that contrary to as advocated by ViTs in image classification works, these models are highly susceptible to adversarial attacks. We attempt to find an easy fix and improve their robustness through adversarial training. While this yields a significant increase in robustness for Restormer, results on other networks are less promising. Interestingly, we find that the design choices in NAFNet and Baselines, which were based on iid performance, and not on robust generalization, seem to be at odds with the model robustness.
State-of-the-art models for pixel-wise prediction tasks such as image restoration, image segmentation, or disparity estimation, involve several stages of data resampling, in which the resolution of feature maps is first reduced to aggregate information and then sequentially increased to generate a high-resolution output. Several previous works have investigated the effect of artifacts that are invoked during downsampling and diverse cures have been proposed that facilitate to improve prediction stability and even robustness for image classification. However, equally relevant, artifacts that arise during upsampling have been less discussed. This is significantly relevant as upsampling and downsampling approaches face fundamentally different challenges. While during downsampling, aliases and artifacts can be reduced by blurring feature maps, the emergence of fine details is crucial during upsampling. Blurring is therefore not an option and dedicated operations need to be considered. In this work, we are the first to explore the relevance of context during upsampling by employing convolutional upsampling operations with increasing kernel size while keeping the encoder unchanged. We find that increased kernel sizes can in general improve the prediction stability in tasks such as image restoration or image segmentation, while a block that allows for a combination of small-size kernels for fine details and large-size kernels for artifact removal and increased context yields the best results.
Team description papers of magmaOffenburg are incremental in the sense that each year we address a different topic of our team and the tools around our team. In this year’s team description paper we focus on the architecture of the software. It is a main factor for being able to keep the code maintainable even after 15 years of development. We also describe how we make sure that the code follows this architecture.
This paper presents the new Deep Reinforcement Learning (DRL) library RL-X and its application to the RoboCup Soccer Simulation 3D League and classic DRL benchmarks. RL-X provides a flexible and easy-to-extend codebase with self-contained single directory algorithms. Through the fast JAX-based implementations, RL-X can reach up to 4.5x speedups compared to well-known frameworks like Stable-Baselines3.
In this paper we present the concept of the "KI-Labor Südbaden" to support regional companies in the use of AI technologies. The approach is based on the "Periodic Table of AI" and is extended with both new dimensions for sustainability, and the impact of AI on the working environment. It is illustrated on the basis of three real-world use cases: 1. The detection of humans with lowresolution infrared (IR) images for collaborative robotics; 2. The use of machine data from specifically designed vehicles; 3. State-of-the-art Large Language Models (LLMs) applied to internal company documents. We explain the use cases, thereby demonstrating how to apply the Periodic Table of AI to structure AI applications.
Online grocery shopping (OGS) has significantly risen due to accelerated retail digitization and reshaped consumer shopping behaviors over the last years. Despite this trend, the German online grocery market lags behind its international counterparts. Notably, with almost half of the German population aged over 50 and the 55–64 age group emerging as the largest user segment in e-commerce, the over-50 demographic presents an attractive yet relatively overlooked audience for the expansion of the online grocery market. However, research on OGS behavior among German over-50s is scarce. This study addresses this gap, empirically investigating OGS adoption factors within this demographic through an online survey with 179 respondents. Our findings reveal that over a third of the over-50 demographic has embraced OGS, indicating a growing receptivity for OGS among the over-50s. Notably, home delivery, product variety, convenience, and curiosity emerged as primary drivers for OGS adoption among this demographic. Surprisingly, most adopters did not increase online grocery orders since 2020 and a not inconsiderable proportion have even stopped buying groceries online again. For potential OGS adopters, regional product availability turned out as a motivator, signaling substantial growth potential and providing online grocers with strategic opportunities to target this demographic. In light of our research, we offer practical suggestions to online grocery retailers, aiming to overcome barriers and capitalize on key drivers identified in our study for sustained growth in the over-50 market segment.
Der Online-Handel verzeichnet seit Jahren ein stetiges Wachstum. Durch die COVID-19-Pandemie kaufen nun auch Nutzende, die zuvor physische Kanäle bevorzugten, vermehrt online ein. Der Anbietererfolg hängt dabei wesentlich von der Kenntnis über die Kund*innen ab. Allerdings dominieren einige große Anbieter den Markt, während kleinere Online-Shops Schwierigkeiten haben, ihre Angebote zu personalisieren. Eine Lösung bietet der Ansatz selbstbestimmter Identitäten. Dieser ermöglicht Kund*innen, ihre eigenen Shoppingdaten zu kontrollieren und sie selektiv mit Online-Shops zu teilen. Dadurch können individuelle Wünsche und Anforderungen der Kund*innen in Online-Shops berücksichtigt und ein personalisiertes Angebot sowie eine gute Nutzungserfahrung geboten werden. Trotz des großen Potenzials selbstbestimmter Identitäten ist der Ansatz in Deutschland kaum verbreitet. Dieser Beitrag beleuchtet den Einsatz selbstbestimmter Identitäten im Online-Handel. Mithilfe eines menschenzentrierten Gestaltungsprozesses wurden Personas und Ist-Szenarien erstellt, sowie daraus resultierend Anforderungen erhoben und Potenziale identifiziert. Auf Basis dessen konnte ein Daten- und Architekturmodell zur Integration von selbstbestimmten Identitäten im Online-Handel entwickelt werden.
Due to its performance, the field of deep learning has gained a lot of attention, with neural networks succeeding in areas like Computer Vision (CV), Neural Language Processing (NLP), and Reinforcement Learning (RL). However, high accuracy comes at a computational cost as larger networks require longer training time and no longer fit onto a single GPU. To reduce training costs, researchers are looking into the dynamics of different optimizers, in order to find ways to make training more efficient. Resource requirements can be limited by reducing model size during training or designing more efficient models that improve accuracy without increasing network size.
This thesis combines eigenvalue computation and high-dimensional loss surface visualization to study different optimizers and deep neural network models. Eigenvectors of different eigenvalues are computed, and the loss landscape and optimizer trajectory are projected onto the plane spanned by those eigenvectors. A new parallelization method for the stochastic Lanczos method is introduced, resulting in faster computation and thus enabling high-resolution videos of the trajectory and secondorder information during neural network training. Additionally, the thesis presents the loss landscape between two minima along with the eigenvalue density spectrum at intermediate points for the first time.
Secondly, this thesis presents a regularization method for Generative Adversarial Networks (GANs) that uses second-order information. The gradient during training is modified by subtracting the eigenvector direction of the biggest eigenvalue, preventing the network from falling into the steepest minima and avoiding mode collapse. The thesis also shows the full eigenvalue density spectra of GANs during training.
Thirdly, this thesis introduces ProxSGD, a proximal algorithm for neural network training that guarantees convergence to a stationary point and unifies multiple popular optimizers. Proximal gradients are used to find a closed-form solution to the problem of training neural networks with smooth and non-smooth regularizations, resulting in better sparsity and more efficient optimization. Experiments show that ProxSGD can find sparser networks while reaching the same accuracy as popular optimizers.
Lastly, this thesis unifies sparsity and neural architecture search (NAS) through the framework of group sparsity. Group sparsity is achieved through ℓ2,1-regularization during training, allowing for filter and operation pruning to reduce model size with minimal sacrifice in accuracy. By grouping multiple operations together, group sparsity can be used for NAS as well. This approach is shown to be more robust while still achieving competitive accuracies compared to state-of-the-art methods
In this paper, we propose a unified approach for network pruning and one-shot neural architecture search (NAS) via group sparsity. We first show that group sparsity via the recent Proximal Stochastic Gradient Descent (ProxSGD) algorithm achieves new state-of-the-art results for filter pruning. Then, we extend this approach to operation pruning, directly yielding a gradient-based NAS method based on group sparsity. Compared to existing gradient-based algorithms such as DARTS, the advantages of this new group sparsity approach are threefold. Firstly, instead of a costly bilevel optimization problem, we formulate the NAS problem as a single-level optimization problem, which can be optimally and efficiently solved using ProxSGD with convergence guarantees. Secondly, due to the operation-level sparsity, discretizing the network architecture by pruning less important operations can be safely done without any performance degradation. Thirdly, the proposed approach finds architectures that are both stable and well-performing on a variety of search spaces and datasets.
We demonstrate how to exploit group sparsity in order to bridge the areas of network pruning and neural architecture search (NAS). This results in a new one-shot NAS optimizer that casts the problem as a single-level optimization problem and does not suffer any performance degradation from discretizating the architecture.