ER¶

class ER(size: int, batch_size: int)¶

Bases: pandemonium.experience.buffers.ReplayBuffer

Experience Replay buffer.

Was originally designed as a means to make RL more data efficient [1]. Later on adapted in DQN architecture to make the data distribution more stationary [2].

References

[1] “Self-Improving Reactive Agents Based On RL, Planning and Teaching”: by Lin. http://www.incompleteideas.net/lin-92.pdf
[2] “Playing Atari with Deep Reinforcement Learning”: https://arxiv.org/pdf/1312.5602.pdf

Ray RLLib v9.0 implementation

Attributes Summary

`capacity`
`is_empty`
`is_full`

Methods Summary

`add`(self, transition, weight)
`add_batch`(self, transitions, weights)
`sample`(self, batch_size, contiguous)	Randomly draws a batch of transitions

Attributes Documentation

capacity¶

is_empty¶

is_full¶

Methods Documentation

add(self, transition: pandemonium.experience.experience.Transition, weight: float = None) → None¶

add_batch(self, transitions: List[ForwardRef(‘Transition’)], weights: List[float] = None)¶

sample(self, batch_size: int = None, contiguous: bool = True) → List[pandemonium.experience.experience.Transition]¶

Randomly draws a batch of transitions

Parameters

batch_size – Number of transitions to sample from the buffer.
contiguous – Whether transitions should be contiguous or not. This is particularly useful when using \(n\)-step methods.