Spinning Up

What is pandemonium?

pandemonium is a library that provides implementations for RL agents that seek to learn about their environment by predicting multiple signals from a single stream of experience.

The name of the project is inspired from the architecture originally developed by Oliver Selfridge in the late 1950s. His computational model is composed of different groups of “demons” working independently to process the visual stimulus, hence the name – pandemonium.

The pandemonium framework has inspired some of the more recent work such as Horde by Sutton et. al 2011. The authors designed a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction. Since then, Horde was further developed and formalized in Adam White’s Doctoral thesis, from which this library borrows most of the definitions and notation.

The goal of this project is to further develop the computational framework established by the creators of Horde and express some of the common algorithms in RL in terms of hierarchy of “demons”.

Installation

pandemonium requires Python 3.6+ and is available on MacOS and Linux. To begin experimenting clone the repo and install using pip:

git clone https://github.com/konichuvak/pandemonium.git
pip install pandemonium

Note

While technically optional, we highly recommend that you use virtualenv to create an isolated python environment in which to install pandemonium.