hybrid_aif.reference.raif.seed

hybrid_aif.reference.raif.seed#

Seeding M⁺: episode collection in their driver’s row convention, and the bang-bang fallback controller for fixture-less runs. Seed statistics are hybrid_aif.train.buffers.seed_stats().

Design: docs/design/amortization.md A12 item 6.

Module Attributes

Policy

A policy for collect_episode(): (key, obs) -> action of shape (1,).

Functions

bang_bang(key, obs)

a = sign(velocity): full throttle in the direction of travel, deterministic.

bang_bang_episodes(key[, n_episodes, env])

n_episodes seed episodes from bang_bang(), flagged expert.

collect_episode(env, policy, key, *[, ...])

Run one episode eagerly and record it as an Episode.

raif_episode(*, obs, action, reward, ...[, ...])

An Episode from their recorded columns: cont = 1 − is_success, the four task columns in extras (ρ zero until revise.label()).

Classes

Array

Array base class for JAX

Callable

Episode

One episode's per-step columns — the format the whole training side speaks.

MountainCar

Sparse-reward continuous Mountain Car.

hybrid_aif.reference.raif.seed.Policy#

A policy for collect_episode(): (key, obs) -> action of shape (1,).

alias of Callable[[Array, Array], Array]

hybrid_aif.reference.raif.seed.bang_bang(key, obs)[source]#

a = sign(velocity): full throttle in the direction of travel, deterministic.

sign(0) = 0 at the reset state costs one step and is not tie-broken.

hybrid_aif.reference.raif.seed.collect_episode(env, policy, key, *, can_self_imitate=False, steps=None)[source]#

Run one episode eagerly and record it as an Episode.

steps (default: the env’s time_limit) bounds the applied actions, so the episode is at most steps + 1 rows and stops early at the goal. Rows follow their driver, as hybrid_aif.reference.raif.run.make_collect_fn() does: row t is (obs_t, reward_t, flags_t, a_t) with reward_t caused by a_{t−1}, opened by a reset row at reward = 0 and closed by a terminal row whose action is zeroed and whose can_self_imitate is cleared. is_success is the goal predicate on the row’s observation, has_succeeded its running OR, and can_self_imitate is the caller’s flag for the policy (True for expert data).

Design: docs/experiments/raif-mountain-car/execution-ledger-acting.md D1–D3.

hybrid_aif.reference.raif.seed.bang_bang_episodes(key, n_episodes=22, env=None)[source]#

n_episodes seed episodes from bang_bang(), flagged expert.

The default 22 is their Mountain Car seed budget (≈1,000 agent steps). Every row but the terminal one has can_self_imitate=True.