[project deprecated in 2015]

CLS2 (pronounced: clsquare) provides a standardized framework for testing (not only) Reinforcement Learning controllers on a (growing) number of different plants.


  • Easy combination of different plants and controllers by definition in a configuration file.

  • Lean interfaces for plants and controllers for a quick migration to and from CLS2 from and to other software environments.

  • Optional graphical interface and visualization.

  • Many demos for easy starting.



    • Cart Pole (single, double and parallel), underactuated pole

    • Maze

    • Mountain Car

Hardware Interfaces


  • Various static standard controllers (linear, PID…)

  • Table-based Q Reinforcement Learning

  • Neural Q and policy controllers

  • Gaussian Processes

  • Kernel-based batch controller (Ormoneit and Sen, 2002)

  • Fitted Q Iteration with Extra Trees (Ernst, Geurts and Wehenkel, 2005)

Get the last version! (2015)

Supported Platforms

The software is developed and tested under:

  • Ubuntu Linux 12.04

  • Mac OSX 10.8

Basic requirements

  • a C++ compiler (build-essential for Ubuntu, xcode for OSX)

  • CMake 2.6.3 (2.8.2 is required for libgp)

Each module that has special requirements will report them during installation if they are missing.


  1. Get a fresh copy of CLS2

  2. Unpack the source code

    • tar xzf clsquare.tgz

  3. Compile the program

    • mkdir build

    • cd build

    • cmake ..

    • make install



The most recent manual is included in the source package. For convenience reasons, you can find the manual here, too: CLSquare Manual v4.0

In addition, documentation for the available modules can be generated using Doxygen by typing make doc in the program's build folder.