CV

Awards and Honors:

• Nominated (1 of 4) for the New Technology Foundation Award for Entertainment Robots and Systems for our paper „Towards learning of safety knowledge from human demonstrations“ at IROS 12 (1801 submissions, 812 accepted papers).
• Lista-Innovationspreis / Lista Award for Innovation, 2007, for my diploma thesis: „Entwicklung eines lernenden Laufroboters“, Stiftung Lista-Innovationspreis, Switzerland
• Outstanding graduate award, 2006, Verein der Freunde und Förderer der Hochschule Ravensburg-Weingarten

Academic career:

• 2008-2013 – Dr. rer. nat.: Doctoral thesis @ Institute of Neural Information Processing – Ulm University. Thesis title: „Reinforcement Learning mit adaptiver Steuerung von Exploration und Exploitatoin“ (engl.: Reinforcement Learning with Adaptive Control of Exploration and Exploitation). Adviser / Thesis Reader: Prof. Dr. Günther Palm and Prof. Dr. Martin Riedmiller. [ Download ]
• 2006-2008 – M.Sc.: Master of Science (in Computer Science) @ University of Applied Sciences Ravensburg-Weingarten.
• 2002-2006 – Dipl.-Inform.(FH): Diploma in Applied Computer Science @ University of Applied Sciences Ravensburg-Weingarten.

Academic Service:

• Moderator of the Reinforcement Learning Mailing List [rl-list] (currently ~2500 members worldwide)
• Reviewer, IEEE Transactions on Neural Networks and Learning Systems, 2018
• Reviewer, Neurocomputing, 2016
• Reviewer, Pattern Recognition Letters, 2016
• Reviewer, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2015
• Reviewer, Computational Statistics, 2015
• Reviewer, 9th IFAC Symposium on Fault Detection, Supervision and Safety for Technical Processes (SAFEPROCESS 2015)
• Program Committee Member, 24th International Conference on Artificial Neural Networks (ICANN 2014)
• Reviewer, Pattern Recognition Letters, Special Issue on Partially Supervised Learning 2012/2013
• Reviewer, 5th INNS IAPR TC3 GIRPR International Workshop on Artificial Neural Networks in Pattern Recognition (ANNPR’12)
• Reviewer, 41st ASEE/IEEE Frontiers in Education Conference (FIE’11)

Work experience:

• since 05/2014 – Senior Research Scientist for „Industrial Reinforcement Learning“ at Siemens AG Munich, Department: Technology / Data Analytics & Artificial Intelligence / Learning Systems. Currently im involved in various Machine Learning projects:
  • Production machines / intralogistic applications:
    • Reinforcement learning control of conveyor belts in
      • Production machines: Optimal product placement for bag packaging processes
      • Intralogistic applications: Singulating incoming packets from multiple conveyor lines on a single conveyor belt
    • Training of control policies is based on digital twins => application to real machine afterwards
    • Deployment of reinforcement learning policies to Simatic S7 PLCs for real-time control
    • Siemens news article: https://new.siemens.com/global/en/company/stories/research-technologies/artificial-intelligence/ai-and-the-chocolate-factory.html
  • Traffic Systems:
    • Greenlight prediction using recurrent neural networks
    • Development of AWS Cloud architecture
    • Siemens ingenuity article: https://ingenuity.siemens.com/2021/11/ai-makes-cities-livable-by-forecasting-traffic-light-phases/
  • Air Quality:
    • Forecasting of air quality (NOx, PM2.5, PM10) using recurrent neural networks
    • Development of AWS Cloud architecture
  • Gasturbines:
    • System identification (emissions, vibration dynamics) using recurrent neural networks
    • Forecasting of bearing vibrations using neural networks
    • Training of reinforcement learning policies towards emission reduction
    • Development of a reinforcement learning industrial benchmark
  • Big Data Analytics
    • Using Hadoop/Hive/Pig for analysing industrial time-series data
    • Predictive Maintenance
• 2006-2014 – Research Associate at University of Applied Sciences Ravensburg-Weingarten:
  • Research Center for autonomous mobile servicerobots
  • BMBF sponsored project Regelung und Automatisierung von autonomen unbemannten Fluggeräten(Auto-UAV)
  • Faculty of Electrical Engineering and Computer Sciences
  • Electronic data processing center, E-Learning
• 2004/2005 – Internship at ND SatCom AG – Department: Embedded Systems
• 2002 – International Internship at Miroglio S.p.A., Alba/Italy – Department: Network Management
• 2002-2008 Freelancer in software development, server administration …

Teaching experience:

• SS2018, SS2019, WS2019/2020, SS2021, SS2022, SS2023 – Lecturer „Praktikum Reinforcement Learning“ @ Ludwig Maximilians University Munich (LMU)
• WS2019/2020 – Lecturer „Machine Learning“ @ University of Applied Sciences Furtwangen
• SS2014, SS2015, SS2016, SS2017 – Guest lecture „Introduction to Reinforcement Learning“ @ Ludwig Maximilians University Munich (LMU)
• WS2013/2014 – Lecturer of the course „Angewandte Neuroinformatik“ (Applied Neuroinformatics) – University of Applied Sciences Ravensburg-Weingarten
• WS2009/2010 until SS2013 – Lecturer of the course „Systemadministration / Linux Workshop“ – University of Applied Sciences Ravensburg-Weingarten