hybrid_aif.reference.raif.metrics#

R-AIF’s three metrics: ACR, SR and R-S, folded from recorded training episodes.

They are online training statistics over the last WINDOW episodes; nothing here steps an environment. ACR is the mean raw episode score, SR the fraction of episodes that reached the goal, and R-S (lower is better) how far the rolling mean of C sits below the best episode’s. R-S depends on the task’s per-step offset (MC_OFFSET).

Design: docs/design/amortization-p0-addendum.md §6.

Module Attributes

MC_OFFSET

r_reward_offset per task (configs.yaml:116,126,137).

WINDOW

Their window: "the last 100 recorded agent training episodes" (paper §4).

Functions

NamedTuple(typename[, fields])

Typed version of namedtuple.

acr(returns)

Average cumulative reward: the mean episode score, raw and unoffset.

episode_returns(rewards[, offset])

Per-episode summed reward, with offset added per step.

evaluate(rewards, successes, *[, offset, window])

The three metrics over the last window episodes of a training run.

metrics_over_time(rewards, successes, *[, ...])

(episode index, Metrics) every every episodes: the curve, not the endpoint.

relative_stability(returns[, offset, window])

R-S over per-episode C: how far the rolling mean sits below the best.

sr(successes)

Success rate: the fraction of episodes that reached the goal at least once.

stability_c(rewards[, offset])

R-S's C for one episode: Σ_{t<T} (reward[t] + offset) over its T+1 rows.

Classes

Metrics

(acr, sr, rs) over one window, plus n, the number of episodes in it.

hybrid_aif.reference.raif.metrics.MC_OFFSET = 1.1#

r_reward_offset per task (configs.yaml:116,126,137). R-S only.

hybrid_aif.reference.raif.metrics.WINDOW = 100#

Their window: “the last 100 recorded agent training episodes” (paper §4).

class hybrid_aif.reference.raif.metrics.Metrics[source]#

Bases: NamedTuple

(acr, sr, rs) over one window, plus n, the number of episodes in it.

A window shorter than WINDOW is a different statistic from a full one.

acr: float#

Alias for field number 0

sr: float#

Alias for field number 1

rs: float#

Alias for field number 2

n: int#

Alias for field number 3

static __new__(_cls, acr, sr, rs, n)#

Create new instance of Metrics(acr, sr, rs, n)

hybrid_aif.reference.raif.metrics.acr(returns)[source]#

Average cumulative reward: the mean episode score, raw and unoffset.

hybrid_aif.reference.raif.metrics.sr(successes)[source]#

Success rate: the fraction of episodes that reached the goal at least once.

hybrid_aif.reference.raif.metrics.relative_stability(returns, offset=1.1, window=100)[source]#

R-S over per-episode C: how far the rolling mean sits below the best. Lower is better.

R-S = mean_k |rolling_mean_window(C)_k − max(C)| / |max(C)| over the last window rolling means, with max(C) over the whole history. With fewer than window episodes the rolling mean is taken over what there is, and the result is not comparable to a full run’s. Raises ValueError on no episodes or on max(C) == 0.

Design: docs/experiments/raif-mountain-car/execution-ledger-acting.md D9.

hybrid_aif.reference.raif.metrics.episode_returns(rewards, offset=0.0)[source]#

Per-episode summed reward, with offset added per step.

rewards is (episodes, steps) or a ragged sequence of reward columns. At offset=0 this is the score ACR averages. R-S’s C is stability_c(), not this.

hybrid_aif.reference.raif.metrics.stability_c(rewards, offset=1.1)[source]#

R-S’s C for one episode: Σ_{t<T} (reward[t] + offset) over its T+1 rows.

The closing is_last row contributes nothing and the reset row contributes one offset, as their MeasureRStability accumulates. rewards is one episode’s (T+1,) reward column in row order.

Design: docs/experiments/raif-mountain-car/execution-ledger-acting.md D9.

hybrid_aif.reference.raif.metrics.evaluate(rewards, successes, *, offset=1.1, window=100)[source]#

The three metrics over the last window episodes of a training run.

rewards is (episodes, steps) or a sequence of per-episode reward columns, one whole unpadded episode each; successes (episodes,) is each episode’s has_succeeded[-1]. ACR and SR are over the window. R-S is over the window’s rolling means against the best episode in the whole history.

hybrid_aif.reference.raif.metrics.metrics_over_time(rewards, successes, *, offset=1.1, window=100, every=10)[source]#

(episode index, Metrics) every every episodes: the curve, not the endpoint.