Episode#

class hybrid_aif.train.Episode[source]#

Bases: NamedTuple

One episode’s per-step columns — the format the whole training side speaks.

Leaves are (T, ...) for one episode, (B, T, ...) for a batch. Row t pairs obs[t] with the action then drawn at it, so reward[t] is the reward of action[t−1] and every flag is a predicate on obs[t]. T applied actions make T + 1 rows: a reset row (is_first, reward = 0) and a terminal row with the action zeroed.

obs: Array#

The observations, (T, obs_dim) float32.

action: Array#

a_t, the raw draw taken at obs[t], (T, action_dim) float32.

reward: Array#

The reward that produced obs[t], that is a_{t-1}’s, (T,) float32.

cont: Array#

1 - terminal, a predicate on obs[t] as recorded by the collector, (T,) float32.

is_first: Array#

Whether obs[t] starts the episode (the reset row), (T,) bool.

is_last: Array#

Whether obs[t] ends the episode, (T,) bool.

extras: dict[str, Array]#

The task’s own per-step columns, (T, ...) each, carried through untouched.

prev_actions()[source]#

The actions re-indexed so entry t is a_{t−1}; step 0 gets zeros.

fit_world()’s convention.

static __new__(_cls, obs, action, reward, cont, is_first, is_last, extras={})#

Create new instance of Episode(obs, action, reward, cont, is_first, is_last, extras)