Refine
Year of publication
Document Type
- Conference Proceeding (52)
- Contribution to a Periodical (4)
- Report (2)
- Article (unreviewed) (1)
Conference Type
- Konferenzartikel (21)
- Sonstiges (19)
- Konferenz-Abstract (12)
Is part of the Bibliography
- yes (59)
Keywords
- RoboCup (31)
- Roboter (6)
- Humanoider Roboter (2)
- Machine Learning (2)
- Agentbasierter Transport (1)
- Deep Learning (1)
- Deep Reinforcement Learning (1)
- Entscheidungstheorie (1)
- Fußball (1)
- Humanoid Robots (1)
Institute
- Fakultät Elektrotechnik und Informationstechnik (E+I) (bis 03/2019) (35)
- Fakultät Elektrotechnik, Medizintechnik und Informatik (EMI) (ab 04/2019) (21)
- Fakultät Maschinenbau und Verfahrenstechnik (M+V) (13)
- IMLA - Institute for Machine Learning and Analytics (5)
- Fakultät Medien und Informationswesen (M+I) (bis 21.04.2021) (4)
- Fakultät Wirtschaft (W) (3)
- INES - Institut für nachhaltige Energiesysteme (2)
- Zentrale Einrichtungen (2)
Open Access
- Open Access (52)
- Bronze (34)
- Closed Access (5)
- Closed (2)
- Grün (2)
- Diamond (1)
In many application areas, Deep Reinforcement Learning (DRL) has led to breakthroughs. In Curriculum Learning, the Machine Learning algorithm is not randomly presented with examples, but in a meaningful order of increasing difficulty. This has been used in many application areas to further improve the results of learning systems or to reduce their learning time. Such approaches range from learning plans created manually by domain experts to those created automatically. The automated creation of learning plans is one of the biggest challenges.In this work, we investigate an approach in which a trainer learns in parallel and analogously to the student to automatically create a learning plan for the student during this Double Deep Reinforcement Learning (DDRL). Three Reward functions, Friendly, Adversarial, and Dynamic based on the learner’s reward are compared. The domain for evaluation is kicking with variable distance, direction and relative ball position in the SimSpark simulated soccer environment.As a result, Statistic Curriculum Learning (SCL) performs better than a random curriculum with respect to training time and result quality. DDRL reaches a comparable quality as the baseline and outperforms it significantly in shorter trainings in the distance-direction subdomain reducing the number of required training cycles by almost 50%.
Team description papers of magmaOffenburg are incremental in the sense that each year we address a different topic of our team and the tools around our team. In this year’s team description paper we focus on the architecture of the software. It is a main factor for being able to keep the code maintainable even after 15 years of development. We also describe how we make sure that the code follows this architecture.
This paper presents the new Deep Reinforcement Learning (DRL) library RL-X and its application to the RoboCup Soccer Simulation 3D League and classic DRL benchmarks. RL-X provides a flexible and easy-to-extend codebase with self-contained single directory algorithms. Through the fast JAX-based implementations, RL-X can reach up to 4.5x speedups compared to well-known frameworks like Stable-Baselines3.
Sweaty has already participated several times in RoboCup soccer competitions (Adult Size). Now the work is focused coordinating the play of two robots. Moreover, we are working on stabilizing the gait by adding additional sensor information. An ongoing work is the optimization of the control strategy by balancing between impedance and position control. By minimizing the jerk, gait and overall gameplay should improve significantly.
Sweaty has already participated several times in RoboCup soccer competitions (Adult Size). Now the work is focused on stabilizing the gait. Moreover, we would like to overcome the constraints of a ZMP-algorithm that has a horizontal footplate as precondition for the simplification of the equations. In addition we would like to switch between impedance and position control with a fuzzy-like algorithm that might help to minimize jerks when Sweaty’s feet touch the ground.
Due to the Covid-19 pandemic, the RoboCup WorldCup 2021 was held completely remotely. For this competition the Webots simulator (https://cyberbotics.com/) was used, so all teams needed to transfer their robot to the simulation. This paper describes our experiences during this process as well as a genetic learning approach to improve our walk engine to allow a more stable and faster movement in the simulation. Therefore we used a docker setup to scale easily. The resulting movement was one of the outstanding features that finally led to the championship title.
The present work ties in with the problem of bicycle road assessment that is currently done using expensive special measuring vehicles. Our alternative approach for road condition assessment is to mount a sensor device on a bicycle which sends accelerometer and gyroscope data via WiFi to a classification server. There, a prediction model determines road type and condition based on the sensor data. For the classification task, we compare different machine learning methods with each other, whereby validation accuracies of 99% can be achieved with deep residual networks such as InceptionTime. The main contribution of this work with respect to comparable work is that we achieve excellent accuracies on a realistic dataset classifying road conditions into nine distinct classes that are highly relevant for practice.
This paper describes a thorough analysis of using PPO to learn kick behaviors with simulated NAO robots in the simspark environment. The analysis includes an investigation of the influence of PPO hyperparameters, network size, training setups and performance in real games. We believe to improve the state of the art mainly in four points: first, the kicks are learned with a toed version of the NAO robot, second, we improve the reliability with respect to kickable area and avoidance of falls, third, the kick can be parameterized with desired distance and direction as input to the deep network and fourth, the approach allows to integrate the learned behavior seamlessly into soccer games. The result is a significant improvement of the general level of play.
Machine learning (ML) has become highly relevant in applications across all industries, and specialists in the field are sought urgently. As it is a highly interdisciplinary field, requiring knowledge in computer science, statistics and the relevant application domain, experts are hard to find. Large corporations can sweep the job market by offering high salaries, which makes the situation for small and medium enterprises (SME) even worse, as they usually lack the capacities both for attracting specialists and for qualifying their own personnel. In order to meet the enormous demand in ML specialists, universities now teach ML in specifically designed degree programs as well as within established programs in science and engineering. While the teaching almost always uses practical examples, these are somewhat artificial or outdated, as real data from real companies is usually not available. The approach reported in this contribution aims to tackle the above challenges in an integrated course, combining three independent aspects: first, teaching key ML concepts to graduate students from a variety of existing degree programs; second, qualifying working professionals from SME for ML; and third, applying ML to real-world problems faced by those SME. The course was carried out in two trial periods within a government-funded project at a university of applied sciences in south-west Germany. The region is dominated by SME many of which are world leaders in their industries. Participants were students from different graduate programs as well as working professionals from several SME based in the region. The first phase of the course (one semester) consists of the fundamental concepts of ML, such as exploratory data analysis, regression, classification, clustering, and deep learning. In this phase, student participants and working professionals were taught in separate tracks. Students attended regular classes and lab sessions (but were also given access to e-learning materials), whereas the professionals learned exclusively in a flipped classroom scenario: they were given access to e-learning units (video lectures and accompanying quizzes) for preparation, while face-to-face sessions were dominated by lab experiments applying the concepts. Prior to the start of the second phase, participating companies were invited to submit real-world problems that they wanted to solve with the help of ML. The second phase consisted of practical ML projects, each tackling one of the problems and worked on by a mixed team of both students and professionals for the period of one semester. The teams were self-organized in the ways they preferred to work (e.g. remote vs. face-to-face collaboration), but also coached by one of the teaching staff. In several plenary meetings, the teams reported on their status as well as challenges and solutions. In both periods, the course was monitored and extensive surveys were carried out. We report on the findings as well as the lessons learned. For instance, while the program was very well-received, professional participants wished for more detailed coverage of theoretical concepts. A challenge faced by several teams during the second phase was a dropout of student members due to upcoming exams in other subjects.
Learning to Walk With Toes
(2020)
This paper explains how a model-free (with respect to the robot model and the behavior to learn) approach can facilitate learning to walk from scratch. It is applied to a simulated Nao robot with toes. Results show an improvement of 30% in speed compared to a model without toes and also compared to our model-based approach, but with less stability.
Sweaty has already participated four times in RoboCup soccer competitions (Adult Size) and came second three times. While 2016 Sweaty needed a lot of luck to be finalist, 2017 Sweaty was a serious adversary in the preliminary rounds. In 2018 Sweaty showed up in the final with some lack of experience and room for improvements, but not without any chance. This paper describes the intended improvements of the humanoid adult size robot Sweaty in order to qualify for the RoboCup 2019 adult size competition.
One of the challenges for autonomous driving in general is to detect objects in the car's camera images. In the Audi Autonomous Driving Cup (AADC), among those objects are other cars, adult and child pedestrians and emergency vehicle lighting. We show that with recent deep learning networks we are able to detect these objects reliably on the limited Hardware of the model cars. Also, the same deep network is used to detect road features like mid lines, stop lines and even complete crossings. Best results are achieved using Faster R-CNN with Inception v2 showing an overall accuracy of 0.84 at 7 Hz.
This paper describes the concept and some results of the project "Menschen Lernen Maschinelles Lernen" (Humans Learn Machine Learning, ML2) of the University of Applied Sciences Offenburg. It brings together students of different courses of study and practitioners from companies on the subject of Machine Learning. A mixture of blended learning and practical projects ensures a tight coupling of machine learning theory and application. The paper details the phases of ML2 and mentions two successful example projects.
This paper describes the Sweaty II humanoid adult size robot trying to qualify for the RoboCup 2018 adult size humanoid competition. Sweaty came 2nd in RoboCup 2017 adult size league. The main characteristics of Sweaty are described in the Team Description Paper 2017. The improvements that have been made or are planned to be implemented for RoboCup 2018 are described in this paper.
In this TDP we describe a new tool created for testing the strategy layer of our soccer playing agents. It is a complete 2D simulator that simulates the games based on the decisions of 22 agents. With this tool, debugging the decision and strategy layer of our agents is much more efficient than before due to various interaction methods and complete control over the simulation.
In the future, the tool could also serve as a measure to run simulations of game series much faster than with the 3D simulator. This way, the impact of different play strategies could be evaluated much faster than before.
In this paper we show that a model-free approach to learn behaviors in joint space can be successfully used to utilize toes of a humanoid robot. Keeping the approach model-free makes it applicable to any kind of humanoid robot, or robot in general. Here we focus on the benefit on robots with toes which is otherwise more difficult to exploit. The task has been to learn different kick behaviors on simulated Nao robots with toes in the RoboCup 3D soccer simulator. As a result, the robot learned to step on its toe for a kick that performs 30% better than learning the same kick without toes.
For the RoboCup Soccer AdultSize League the humanoid robot Sweaty uses a single fully convolutional neural network to detect and localize the ball, opponents and other features on the field of play. This neural network can be trained from scratch in a few hours and is able to perform in real-time within the constraints of computational resources available on the robot. The time it takes to precess an image is approximately 11 ms. Balls and goal posts are recalled in 99 % of all cases (94.5 % for all objects) accompanied by a false detection rate of 1.2 % (5.2 % for all). The object detection and localization helped Sweaty to become finalist for the RoboCup 2017 in Nagoya.
One of the challenges in humanoid robotics is motion control. Interacting with humans requires impedance control algorithms, as well as tackling the problem of the closed kinematic chains which occur when both feet touch the ground. However, pure impedance control for totally autonomous robots is difficult to realize, as this algorithm needs very precise sensors for force and speed of the actuated parts, as well as very high sampling rates for the controller input signals. Both requirements lead to a complex and heavy weight design, which makes up for heavy machines unusable in RoboCup Soccer competitions.
A lightweight motor controller was developed that can be used for admittance and impedance control as well as for model predictive control algorithms to further improve the gait of the robot.
This paper describes the Sweaty II humanoid adult size robot trying to qualify for the RoboCup 2017 adult size humanoid competition. Sweaty came 2nd in RoboCup 2016 adult size league. The paper describes the main characteristics of Sweaty that made this success possible, and improvements that have been made or are planned to be implemented for RoboCup 2017.
After having described many different aspects of our team software in previous years, in this paper we take the freedom to describe the magmaChallenge framework provided by the magmaOffenburg team. The framework is used as a benchmark tool to run different challenges like the running challenge in 2014 or the kick accuracy challenge in 2015. This description should serve as a documentation to simplify the maintenance by the community and to add new benchmarks in the future.
This paper describes the new Sweaty II humanoid adult size robot trying to qualify for the RoboCup 2016 adult size humanoid competition. Based on experiences during RoboCup 2014, the Sweaty robot has been completely redesigned to a new robot Sweaty II. A major change is the use of linear actuators for the legs. Another characteristic is its indirect actuation by means of rods. This allows a variable transmission ratio depending on the angle of a joint.
The humanoid Sweaty was the finalist in this year’s robocup soccer championship(adult size). For the optimization of the gait and the stability, data concerning forces and torques in the ankle joints would be helpful. In the following paper the development of a six-axis force and torque sensor for the humanoid robot Sweaty is described. Since commercial sensors do not meet the demands for the sensors in Sweatys ankle joints, a new sensor was developed. As a measuring devices we used strain gauges and custom electronics based on an acam PS09. The geometry was analyzed with the FEM program ANSYS to get optimal dimensions for the measuring beams. In addition ANSYS was used to optimize the position for the strain gauges on the beam.
Autonomous humanoid robots require light weight, high torque and high speed actuators to be able to walk and run. For conventional gears with a fixed gear ratio the product of torque and velocity is constant. On the other hand desired motions require maximum torque and speed. In this paper it is shown that with a variable gear ratio it is possible to vary the relation between torque and velocity. This is achieved by introducing systems of rods and levers to move the joints of our humanoid robot ”Sweaty II”. On the basis of a variable gear ratio low speed and high torque can be achieved for those joint angles, which require this motion mode, whereas high speed and low torque can be realized for those joint angles, where it is favorable for the desired motion.
Soccer simulation league is one of the founding leagues of RoboCup. In this paper we discuss the past, present and planned future achievements and changes. Also we summarize the connections and inter-league achievements of this league and provide an overview of the community contributions that made this league successful.
This paper describes the new Sweaty humanoid adult size robot trying to qualify for the RoboCup 2014 adult size humanoid competition. The robot is built from scratch to eventually allow it to run. One characteristic is that to prevent the motors from overheating, water evaporation is used for cooling. The robot is literally sweating which has given it its name. Another characteristic is, that the motors are not directly connected to the frame but by means of beams. This allows a variable transmission ratio depending on the angle.
This paper describes the magmaOffenburg 3D simulation team trying to qualify for RoboCup 2013. While last year’s TDP focused on different ways how robot behavior can be defined in the magmaOffenburg framework this year we focus on how we statistically evaluate new features on distributed systems. We also show some results gained through such analysis.
In this paper we propose a motion framework forbipedal robots that decouples motion definitions from stabilizingthe robot. This simplifies motion definitions yet allows dynamicmotion adaptations. Two applications, walking and stopping onone leg, demonstrate the power of the framework. We show thatour framework is able to perform walking and stopping on one legeven under extreme conditions and improves walking benchmarkssignificantly in the RoboCup 3D soccer simulation domain.
Autonomous humanoid robots need high torque actuators to be able to walk and run. One problem in this context is the heat generated. In this paper we propose to use water evaporation to improve cooling of the motors. Simulations based on thermodynamic calculations as well as measurements on real actuators show that, under the assumption of the load of a soccer game, cooling can be considerably improved with relatively small amounts of water.
This paper describes the magmaOffenburg 3D simulation team trying to qualify for RoboCup 2012. While last year’s TDP focused on the tool set created for 3D simulation and the support for heterogeneous robot models, this year we focus on the different ways how robot behavior can be defined in the magmaOffenburg framework and how those behaviors can be improved by learning.
Seit 1997 finden jährlich Weltmeisterschaften im Roboterfußball statt. Das Ziel ist es dabei, bis 2050 eine Mannschaft aus Robotern zu stellen, die gegen den menschlichen Fußballweltmeister gewinnt. Dazu müssen die Roboter in der Lage sein, das Verhalten ihrer menschlichen Gegner einzuschätzen und ihre Entscheidungen vorauszuahnen. Während die gängigen Verfahren zur Entscheidungsfindung in unsicheren Umgebungen in der Regel auf rationalen Entscheidungen nach der Entscheidungstheorie basieren, zeigt sich, dass menschliches Entscheiden teilweise nicht dieser Rationalität folgt. Daniel Kahneman und Amos Tversky zeigten das in vielen Studien und entwickelten daraus die bekannte Prospect Theory für die Kahneman 2002 den Wirtschaftsnobelpreis erhielt. In diesem Artikel wird beschrieben, wie Extended Behavior Networks (EBNs) auf einfache Weise erweitert werden können, um menschliches Entscheidungsverhalten auch in Situationen reproduzieren zu können, die von der rationalen Entscheidungstheorie abweichen.
Große Logistikunternehmen stehen in den letzten Jahren zunehmend vor neuen Herausforderungen. Zum einen steigt die Menge zu transportierender Güter jährlich, zum anderen entstanden durch Verschmelzungen großer Logistikunternehmen, wie z. B. Deutsche Post, Danzas und Exel oder UPS und Fritz, riesige Fahrzeugflotten, deren effiziente Planung die Unternehmen vor enorme Probleme stellt. Die einzige Möglichkeit, diese meist heterogenen, also aus vielen verschiedenen Verkehrsmitteln bestehenden Flotten mit herkömmlichen Mitteln effizient zu planen, ist die Aufteilung in (regionale) Geschäftsbereiche. Dadurch können viele Synergieeffekte nicht genutzt werden, was unter anderem zu unnötig hohen Transportkilometerleistungen und Leerfahrten führt. Im Rahmen des Forschungsprojekts Attractive (Programm IngenieurNachwuchs) wurden neue Algorithmen entwickelt, mit deren Hilfe dieOptimierung von Transportaufträgen unter realen Be-dingungen und in realistischen Größenordnungen möglich wird. In diesem Artikel wird kurz auf die Optimierung eingegangen, und dann werden die ersten gewonnenen Ergebnisse zusammengefasst.
This paper describes the magmaOffenburg 3D simulation team trying to qualify for RoboCup 2011. While last year’s TDP focused on the tool set created for 3D simulation in this year we describe the further improvement in this tools as well as some new features we implemented focusing on heterogeneous robot models which seem to be used in RoboCup 2012.
An additional tool was written to simply generate situation-dependent strategies. Furthermore some tools, described last year, are now integrated in one single GUI to easy things up.
Existing approaches solving multi-vehicle pickup and delivery problems with soft time windows typically use common benchmark sets to verify their performance. However, there is a gap from these benchmark sets to real world problems with respect to instance size and problem complexity. In this paper we show that a combination of existing approaches together with improved heuristics is able to deal with the instance sizes and complexity of real world problems. The cost savings potential of the heuristics is compared to human dispatching plans generated from the data of a European carrier.
Deutschland hat nicht zuletzt durch seine zentrale Lage eine führende Rolle im Bereich der Transportlogistik in Europa übernommen. Allerdings stehen die großen Logistikunternehmen in den letzten Jahren zunehmend vor neuen Herausforderungen. Zum einen steigt die Menge zu transportierender Güter jährlich, zum anderen entstanden durch Verschmelzungen großer Logistikunternehmen wie z. B. Deutsche Post, Danzas und Exel, UPS und Fritz riesige Fahrzeugflotten, deren effiziente Planung die Unternehmen vor enorme Probleme stellt. Die einzige Möglichkeit, diese meist heterogenen, also aus vielen verschiedenen Verkehrsmitteln bestehenden Flotten mit herkömmlichen Mitteln effizient zu planen, ist die Aufteilung in (regionale) Geschäftsbereiche. Dadurch können viele Synergieeffekte nicht genutzt werden, was unter anderem zu unnötig hohen Transportkilometerleistungen und Leerfahrten führt. Mit Hilfe agentenbasierter Systeme können heute schon Kosteneinsparungen von 3 – 6 % bei homogenen Verkehrsmitteln erzielt werden. Das Einsparpotenzial dürfte bei heterogenen Flotten ähnlich hoch, wenn nicht noch etwas höher sein. Allerdings liefern derzeit agentenbasierte Systeme für heterogene Flotten noch keine zufriedenstellenden Ergebnisse. Durch die Kombination der beiden vorrangig für die Transportoptimierung eingesetzten Techniken agentenbasierte (bottomup) Optimierung und der klassischen (topdown)Optimierung soll auch das Einsparpotenzial von heterogenen Flotten realisiert werden. Diese Optimierung ist Gegenstand des Attractive Forschungsprojekts, das von August 2009 bis Juli 2012 im Rahmen des Programms ingenieurNachwuchs gefördert wird.
Seit 1997 finden jährlich Weltmeisterschaften im Roboterfußball statt. Dabei wird in verschiedenen Ligen teils mit echten, teils mit simulierten Robotern Fußball gespielt. In der small size league spielen fünf gegen fünf Roboter auf einem 5x4,5 m großen Feld. Die Steuerung der Roboter wird von einem externen Rechner übernommen, der seine Information von einer über dem Feld angebrachten Kamera erhält. In der middle size league spielen vier gegen vier Roboter auf einem 8x12 m großen Feld. Hier müssen im Unterschied zur small size league die Roboter vollständig autonom sein, d.h., alle Sensoren und auch die Entscheidungslogik muss auf dem Roboter selbst untergebracht sein. Dasselbe gilt für die four legged robot league, bei der jeweils vier Sony Aibo Roboter gegeneinander antreten (Abbildung 1.11-1), sowie für die Königsklasse, der humanoid league, bei der jeweils drei zweibeinige Roboter gegeneinander spielen. Daneben existieren zwei Simulationsligen: die seit 1997 existierende 2D simulation league, bei der elf gegen elf gespielt wird und die seit 2005 im Programm befindliche 3D simulation league, bei der im Gegensatz zur 2D league tatsächlich existierende zweibeinige Nao-Roboter simuliert werden. In dieser Liga hat sich erstmals eine Mannschaft der Hochschule Offenburg für die Weltmeisterschaft 2009 qualifiziert. Neben Fußballrobotern gibt es auch Ligen für Hausroboter (RoboCup@Home) und Rettungsroboter (RoboCup Rescue). Inzwischen ist die RoboCup WM mit der zugehörigen Konferenz zum größten Robotik-Event weltweit avanciert.
In previous work we [1] and other authors (e.g. [2]) have shown that agent-based systems are successful in optimizing delivery plans of single logistics companies and are meanwhile successfully productive in industry. In this paper we show that agent-based systems are particularly useful to also optimize transport across logistics companies. In intercompany optimization, privacy is of major importance between the otherwise competing companies. Some data has to be treated strictly private like the cost model or the constraint model. Other data like order information has to be shared. However, typically the amount of orders released to other companies has also to be limited. We show that our agent-based approach can be easily fine tuned to trade off privacy against the benefit of cooperation.
Die Einhaltung der innerhalb der Designphase festgelegten Architektur eines Softwareprojektes muss w ̈ahrend der Entwicklungsphase sichergestellt werden. Dieses Papier beschreibt eine Erweiterung des Eclipse-Plugins JDepend4Eclipse, die die Verwaltung von Regels ̈atzen erlaubt und die Pr ̈ufung auf in einem Projekt vorhandene, unerlaubte Abh ̈angigkeiten auf Knopfdruck innerhalb der Entwicklungsumgebung vornimmt. Die Erweiterung des Plugins wird bereits erfolgreich in internen Projekten der Hochschule Offenburg eingesetzt und soll demn ̈achst ̈offentlich verf ̈ugbar sein.
This paper describes the magmaOffenburg 3D simulation team trying to qualify for RoboCup 2010. While last year’s TDP focused on decisions making using extended behavior networks and on its software architecture and implementation in this year we describe the tool set that was created for RoboCup 3D. It contians a GUI for agent- and world state visualization, for evaluation of localization algorithms and benchmarks in general, a visual editor for Extended Behavior Networks creation and debugging, a live movement tool to interact with the joints and finally a tool for editing behavior motor files.
In their famous work on prospect theory Kahneman and Tversky have presented a couple of examples where human decision making deviates from rational decision making as defined by decision theory. This paper describes the use of extended behavior networks to model human decision making in the sense of prospect theory. We show that the experimental findings of non-rational decision making described by Kahneman and Tversky can be reproduced using a slight variation of extended behavior networks.
This paper describes the magmaOffenburg 3D simulation team trying to qualify for RoboCup 2009. It focuses on two distinctive features of the team: decisions making using extended behavior networks and its software architecture and implementation in Java to open the simulation for the Java community.
This paper discusses a technological solution to real-time road transportation optimization using a commercial multi-agent based system, LS/ATN, which has been proven through real-world deployment to reduce transportation costs for both small and large fleets in the full and part load business. Subsequent to describing the real-time optimization approach, we discuss how the platform is currently evolving to accept live data from vehicles in the fleet in order to improve optimization accuracy. A selection of the predominant pervasive technologies available today for enhancing intelligent route optimization is described.