TY - CHAP U1 - Konferenzveröffentlichung A1 - Spitznagel, Martin A1 - Weiler, David A1 - Dorer, Klaus ED - Santos, Vitor ED - Lau, Nuno ED - Neto, Pedro ED - Lopes, Ana Cristina T1 - Deep Reinforcement Multi-Directional Kick-Learning of a Simulated Robot with Toes T2 - 2021 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC) N2 - This paper describes a thorough analysis of using PPO to learn kick behaviors with simulated NAO robots in the simspark environment. The analysis includes an investigation of the influence of PPO hyperparameters, network size, training setups and performance in real games. We believe to improve the state of the art mainly in four points: first, the kicks are learned with a toed version of the NAO robot, second, we improve the reliability with respect to kickable area and avoidance of falls, third, the kick can be parameterized with desired distance and direction as input to the deep network and fourth, the approach allows to integrate the learned behavior seamlessly into soccer games. The result is a significant improvement of the general level of play. Y1 - 2021 UR - https://magma.hs-offenburg.de/fileadmin/default_upload/2021ICARSCWithCopyright.pdf SN - 978-1-6654-3198-9 (elektronisch) SB - 978-1-6654-3198-9 (elektronisch) SN - 978-1-6654-3199-6 (Print) SB - 978-1-6654-3199-6 (Print) U6 - https://doi.org/10.1109/ICARSC52212.2021.9429811 DO - https://doi.org/10.1109/ICARSC52212.2021.9429811 SP - 104 EP - 110 PB - IEEE ER -