The Crawling Robot

On the laboratory carpet:

On a foam mat:

On a desk:

On a wet desk with slippery surface:


On a linoleum surface:

The Crawling Robot Simulator

Check out the Python 2D Crawling robot simulation on Github developed by a group of my students from the lecture „Applied Reinforcement Learning“ @ LMU:

The package contains:

  • OpenAI gym environment for discrete and continuous actions
  • Example ipynb notebook with:
    • Manual control
    • Q-Learning example (discrete actions)
    • PPO2 (continuous actions) example using stable-baselines

Relevant Literature

  • M. Tokic and H. Bou Ammar.
    Teaching reinforcement learning using a physical robot.
    In Proceedings of the Workshop on Teaching Machine Learning at
    the 29th International Conference on Machine Learning
    , Edinburgh, UK,
    (to appear).
  • W. Ertel. Introduction to Artificial Intelligence. Springer London, 2011.
  • S. Montresor, J. Kay, M. Tokic, and J. Summerton.
    Work in progress: Programming in a confined space – a case study in
    porting modern robot software to an antique platform.
    In Proceedings of the 41st ASEE/IEEE Frontiers in Education
    , pages T3H-1-T3H-3, Rapid City, SD, USA, 2011. IEEE
  • M. Tokic, A. Usadel, J. Fessler, and W. Ertel.
    On an educational approach to behavior learning for robots.
    In Proceedings of the 1st International Conference on Robotics
    in Education
    , pages 171-176, Bratislava, Slovak Republic, 2010. Slovak
    University of Technology in Bratislava.
  • M. Tokic, J. Fessler, and W. Ertel.
    The crawler, a class room demonstrator for reinforcement learning.
    In C. Lane and H. Guesgen, editors, Proceedings of the 22th
    International Florida Artificial Intelligence Research Society Conference
    , pages 160-165, Menlo Park, California, USA, 2009. AAAI
  • W. Ertel, M. Schneider, R. Cubek, and M. Tokic.
    The Teaching-Box: a universal robot learning framework.
    In Proceedings of the 14th International Conference on Advanced
    Robotics ICAR’09.
    , pages 1-6, 2009.
  • H. Kimura, K. Miyazaki, and S. Kobayashi.
    Reinforcement learning in POMDPs with function approximation.
    In Proceedings of the 14th International Conference on Machine
    Learning (ICML’97)
    , pages 152-160, San Francisco, CA, USA, 1997.
    Morgan Kaufmann Publishers Inc.