Brainstormers 2D
The robotic soccer team Brainstormers 2D was a team participating in the simulation league of the RoboCup competition from 1998 to 2008. Brainstormers' research focused on neural reinforcement learning methods for large parts of agent behaviour, from basic skills like moving and dribbling to high-level team cooperation. The team was founded by Martin Riedmiller in 1998 at the University of Karlsruhe and co-lead by Thomas Gabel from 2005 onwards. Besides several European titles, the team won three World Championships in 2005, 2007 and 2008.
Key facts
founded in 1998
three times world champion: 2005, 2007, and 2008
six consecutive European-level titles: 2005 through 2010
research focus: use of artificial intelligence-based and reinforcement learning approaches to the largest degree possible
more than 20 publications on the use of machine learning techniques in the context of robotic soccer
RoboCup 2D league 2007 final: Brainstromers vs. Wright Eagle
Neural Reinforcement Learning agents learn to play sophisticated attack moves: running to open positions, playing passes, dribbling and scoring.
The learning agents are only rewarded if they score.
About the Brainstormers
The Brainstormers are a team of 11 computer programs, that each control a simulated robot on a virtual playfield. The simulated robots share a lot of features with their real hardware colleagues (see the Brainstormers Tribots' site) - for example restricted view, restricted moving capabilities, restricted resources - but they have the big advantage to be much cheaper and much less sensible to damages. Therefore, the simulator league teams can concentrate on the development of control software.
Robotic soccer represents an excellent testbed for machine learning and, particularly, for reinforcement learning tasks. The scientific goal of our team is to use machine learning techniques - in particular reinforcement learning methods - to develop the players' control programs. So, from a learning point of view it is also our long-term goal to realize an agent that obtains its behavior by entirely employing a reinforcement learning methodology.
You can find more details about that focus in our research and publications sections below.
What is robotic soccer?
RoboCup is an international research initiative intending to expedite AI and intelligent robotics research by defining a set of standard problems where various technologies can and ought to be combined solving them. Annually, there are championship tournaments in several leagues - ranging from rescue tasks over real soccer-playing robots to simulated ones.
The Brainstormers 2D participate in RoboCup's 2D Simulation League, where two teams of simulated soccer-playing agents compete against one another using the Soccer Server, a real-time soccer simulation system.
The Soccer Server allows autonomous software agents written in an arbitrary programming language to play soccer in a client/server-based style: The server simulates the playing field, communication, the environment and its dynamics, while the clients - eleven agents per team - are permitted to send their intended actions (e.g. a parameterised kick or dash command) once per simulation cycle to the server via UDP. Then, the server takes all agents' actions into account, computes the subsequent world state and provides all agents with (partial) information about their environment via appropriate messages over UDP. The course of action during a match can be visualised using an additional program, the Soccer Monitor.
If you are interested in finding out, what progress has been made in RoboCup throughout the previous years, watch the video to the left or visit the RoboCup website:
People
Prof. Dr. Thomas Gabel
Prof. Dr. Martin Riedmiller
Tobias Springenberg
Thomas Huber
Haiko Schol
Sascha Ludwig
Florian Trost
Hannes Schulz
Anton Fluegge
Kilian Klimek
Johannes Knabe
Hauke Strasdat
Manuel Nikschas
David Meier
Daniel Withopf
Artur Merke
Publications
T. Gabel and M. Riedmiller. On Progress in RoboCup: The Simulation League Showcase. RoboCup 2010: Robot Soccer World Cup XIV, LNCS. Springer, Singapore, 2010.
M. Riedmiller, T. Gabel, R. Hafner and S. Lange. Reinforcement Learning for Robot Soccer. Autonomous Robots, 27(1):55–74, Springer, 2009.
T. Gabel, M. Riedmiller and F. Trost. A Case Study on Improving Defense Behavior in Soccer Simulation 2D: The NeuroHassle Approach. RoboCup 2008: Robot Soccer World Cup XII, LNCS. Springer, Shouzou, China, 2008.
T. Gabel and M. Riedmiller. On Experiences in a Complex and Competitive Gaming Domain: Reinforcement Learning Meets RoboCup. Proceedings of the IEEE Symposium on Computational Intelligence and Games (CIG 2007). Honolulu, USA, 2007.
T. Gabel, R. Hafner, S. Lange, M. Lauer and M. Riedmiller. Bridging the Gap: Learning in the RoboCup Simulation and Midsize League. Proceedings of the 7th Portuguese Conference on Automatic Control (Controlo 2006). Portuguese Society of Automatic Control, Porto, Portugal, 2006.
T. Gabel and M. Riedmiller. Learning a Partial Behavior for a competitive Soccer Agent. Künstliche Intelligenz, 2:18-23, Springer, 2006.
M. Riedmiller, T. Gabel, R. Hafner, S. Lange and M. Lauer. Die Brainstormers: Entwurfsprinzipien lernfähiger autonomer Roboter. Informatik-Spektrum, 20(3):175-190, Springer, 2006.
T. Gabel and M. Riedmiller. CBR for State Value Function Approximation in Reinforcement Learning. Proceedings of the International Conference on Case Based Reasoning 2005 (ICCBR 2005). Springer, Chicago, USA, August 2005.
D. Withopf and M. Riedmiller. Comparing Different Methods to Speed-Up Reinforcement Learning in a Complex Domain. Proc. of the Int. Conference on Systems, Man and Cybernetics, 2005. Big Island, USA, October 2005.
D. Withopf and M. Riedmiller. Effective Methods for Reinforcement Learning in Large Multi-Agent Domains. it - Information Technology Journal, 5(47):241-249, Oldenbourg, 2005.
M. Riedmiller and A. Merke. Using machine learning techniques in complex multi-agent domains. In I. Stamatescu and W. Menzel and M. Richter and U. Ratsch, editor, Adaptivity and Learning, Springer, 2003.