SegmentedER

class SegmentedER(size: int, batch_size: int, segments: int, criterion: Callable[[pandemonium.experience.experience.Transition], int], dist: torch.distributions.categorical.Categorical = None)

Bases: pandemonium.experience.buffers.ER

Segmented Experience Replay buffer.

Allows for partitioning the ER into multiple segments, specifying a sampling distribution over segments.

Notes

This ER is not suitable for sampling sequences of experiences unless the type of experience is a Trajectory, which would be much more memory intensive compared to Transition and is thus not recommended.

Current implementation assumes segments of fixed size and, thus, it might take a while to fill up the buffer completely in case the criterion or distribution of experiences is highly skewed.

Methods Summary

add(self, transition, weight)
sample(self, batch_size, contiguous)	Randomly draws a batch of transitions

Methods Documentation

add(self, transition: pandemonium.experience.experience.Transition, weight: float = None) → None

sample(self, batch_size: int = None, contiguous: bool = True) → List[pandemonium.experience.experience.Transition]

Randomly draws a batch of transitions

Parameters

• batch_size – Number of transitions to sample from the buffer.

• contiguous – Whether transitions should be contiguous or not. This is particularly useful when using \(n\)-step methods.