ImagineMetrics#

class hybrid_aif.train.ImagineMetrics[source]#

Bases: NamedTuple

Plain arrays, one scalar each — what a training loop logs and a test asserts.

actor_loss: Array#

The actor’s loss.

critic_loss: Array#

The critic’s loss.

value: Array#

The mean per-step G as a value (the negated expected free energy).

ret: Array#

The mean λ-return.

critic_value: Array#

The critic’s mean at the same states.

advantage: Array#

The mean advantage: the normalised return less the normalised critic value.

policy_entropy: Array#

The mean policy entropy H[pi(a | s)] over the rollouts.

prior_entropy: Array#

The mean prior entropy H[p(stoch | deter)] over the rollouts.

bc_logprob: Array#

mean(log pi(a_start | s_0)) over every start, weighted or not: the actor’s raw density at the replay draws, a distribution-shift diagnostic. It is not the BC term’s contribution — a product of means is not the mean of a product — and bc_term is the quantity to compare against a BC loss.

bc_weight: Array#

mean(bc_weights) (zero when no BC inputs were passed).

bc_term: Array#

The BC term’s actual contribution to the actor loss: bc_scale · mean(−log pi(a_start | s_0) · bc_weights), weight and scale folded in.

retnorm_low: Array#

The low percentile moving average of the return normalization, after this batch.

retnorm_high: Array#

The high percentile moving average of the return normalization, after this batch.

rollout_weight_mean: Array#

The mean per-step rollout weight (rollout_weight()): how much of the horizon the continuation head is currently truncating — 1.0 means “nothing”, and a value near 1/horizon means the rollouts are being cut at their first step. It is also the direct read on start_cont’s effect.

per_term: dict[str, Array]#

The value’s decomposition, keyed by term name.

static __new__(
_cls,
actor_loss,
critic_loss,
value,
ret,
critic_value,
advantage,
policy_entropy,
prior_entropy,
bc_logprob,
bc_weight,
bc_term,
retnorm_low,
retnorm_high,
rollout_weight_mean,
per_term,
)#

Create new instance of ImagineMetrics(actor_loss, critic_loss, value, ret, critic_value, advantage, policy_entropy, prior_entropy, bc_logprob, bc_weight, bc_term, retnorm_low, retnorm_high, rollout_weight_mean, per_term)