hybrid_aif.reference.raif.crspp#

The CRSPP (prior-preference model): its recognition scan, its loss and its scoring row.

The loss, with rho from hybrid_aif.reference.raif.revise.pref_labels():

L = mean_t[ gate_t · KL_balanced(q2_t ‖ p2_t) ]
    + contrast · mean_t[ −rho_t · cos_sim(C_post_t, sg(post_t)) ]
gate_t = max(0, sgn rho_t)          contrast = 1.0

The CRSPP is a declared System with no decoder and no action parent. Its posterior conditions on ("deter", "external:embed"), the agent’s encoder output handed over as a vector (crspp_recognize()), and the resulting beliefs reach the scoring row as aux_beliefs["cosine_sim_to_pref_rollout"], keyed by the CRSPP’s state factor names.

Design: docs/design/amortization-p0-addendum.md §4.

Module Attributes

DEFAULT_LR

Their preference-model optimizer is the model group's: adam 1e-4, clip 1000 (the CRSPP nets are trained inside the world-model loss's optimizer scope, configs.yaml:76).

DEFAULT_CONTRAST

loss_scales.contrast (configs.yaml; addendum §4).

ROW

The row whose parameters this trainer steps, and the address that follows.

PART_ORDER

The CRSPP's parts, in canonical order — the order its one global-norm clip sees, stated once for the reason hybrid_aif.inference.rssm.PART_FIELDS is.

CRSPP_BLOCK

The similarity leaf's block key in the term tree.

Functions

NamedTuple(typename[, fields])

Typed version of namedtuple.

assemble(system, params[, roles])

The world-model assembly behind a declared System: (core, slot addresses).

cosine_sim_to_pref_rollout(st)

Their sim: cosine similarity to the CRSPP's imagined preferred state.

crspp_loss(parts, key, embeds, rho, ...[, ...])

The CRSPP loss on one batch: (total, CRSPPLosses).

crspp_recognize(system, params, embeds, *, ...)

The CRSPP posterior over a batch's embeddings, as belief tuples.

embeddings(system, params, obs)

The agent's encoder output on a batch of observations: (..., embed_dim).

features(beliefs)

concat([deter, flat stoch, …]) — the feature vector every head reads.

fit_crspp(system, params, learn_state, ...)

One gradient step of preference learning: (crspp_params, learn_state, losses).

img_step_no_action(parts, key, deter, code)

One imagined CRSPP step: their img_step_no_action.

kl_balance(post_logits, prior_logits, *[, ...])

KL balancing with free bits: the complexity term of a world-model loss.

observe_no_action(parts, key, embeds[, is_first])

Their observe_no_action: filter one embedding sequence, no actions.

positive_gate(rho)

max(0, sgn ρ): the CRSPP KL's hard mask to positive states (paper Eq.

register_term(name, sign)

Register a term name with its sign as a contribution to G: +1.0 for a cost, -1.0 for a value.

resolve(system, params)

The CRSPP's parts and slot addresses, validated against A12 item 2's shape.

straight_through_onehot(key, logits)

Sample a one-hot per variable, with the probabilities' gradient attached.

Classes

Array

Array base class for JAX

CRSPPLosses

The preference model's loss, decomposed.

CRSPPParts

The CRSPP's declarations, resolved: the parts its scan runs and their slot addresses.

Deterministic

A deterministic node: a pure map, optionally with trainable parameters.

Interval

A closed interval [lower, upper], broadcastable like its member arrays.

LearnState

Optimizer state for an agent's gradient-learnable parts: learn's explicit carry.

MLPCategorical

MLP over concatenated parents → a factored categorical child.

Mapping

A Mapping is a generic container for associating key/value pairs.

System

Static structure of a System: discrete, continuous or mixed.

SystemParams

All numeric state of a System: generative tensors, designations and the closed-loop factors.

hybrid_aif.reference.raif.crspp.DEFAULT_LR = 0.0001#

Their preference-model optimizer is the model group’s: adam 1e-4, clip 1000 (the CRSPP nets are trained inside the world-model loss’s optimizer scope, configs.yaml:76).

hybrid_aif.reference.raif.crspp.DEFAULT_CONTRAST = 1.0#

loss_scales.contrast (configs.yaml; addendum §4).

hybrid_aif.reference.raif.crspp.ROW = 'cosine_sim_to_pref_rollout'#

The row whose parameters this trainer steps, and the address that follows.

hybrid_aif.reference.raif.crspp.PART_ORDER: tuple[str, ...] = ('gru', 'prior', 'posterior', 'initial')#

The CRSPP’s parts, in canonical order — the order its one global-norm clip sees, stated once for the reason hybrid_aif.inference.rssm.PART_FIELDS is.

class hybrid_aif.reference.raif.crspp.CRSPPParts[source]#

Bases: NamedTuple

The CRSPP’s declarations, resolved: the parts its scan runs and their slot addresses.

gru: Deterministic#

Alias for field number 0

prior: MLPCategorical#

Alias for field number 1

posterior: MLPCategorical#

Alias for field number 2

initial: object#

Alias for field number 3

embed_name: str#

Alias for field number 4

slots: tuple[str, ...]#

Alias for field number 5

property parts: tuple[object, ...]#

The four trainable parts, in the order slots addresses them.

with_parts(replaced)[source]#

A copy with the parts at those positions replaced (the gradient’s swap).

static __new__(_cls, gru, prior, posterior, initial, embed_name, slots)#

Create new instance of CRSPPParts(gru, prior, posterior, initial, embed_name, slots)

hybrid_aif.reference.raif.crspp.resolve(system, params)[source]#

The CRSPP’s parts and slot addresses, validated against A12 item 2’s shape.

The declaration must be one uncontrolled carry, one code whose parent is that carry, no modality, and one recognition entry over (carry, external embedding); anything else raises.

hybrid_aif.reference.raif.crspp.observe_no_action(parts, key, embeds, is_first=None)[source]#

Their observe_no_action: filter one embedding sequence, no actions.

embeds is (T, embed_dim). is_first, optional (T,) bool, marks episode starts inside the sequence: there the carry is replaced by the CRSPP’s own initial state before the step, as their obs_step_no_action does. The scan always starts from that state. Returns (deters, codes, post_logits, prior_logits) stacked over time; the carried posterior code is a straight-through sample. Unbatched.

hybrid_aif.reference.raif.crspp.img_step_no_action(parts, key, deter, code)[source]#

One imagined CRSPP step: their img_step_no_action. (deter', code').

hybrid_aif.reference.raif.crspp.crspp_recognize(system, params, embeds, *, key, is_first)[source]#

The CRSPP posterior over a batch’s embeddings, as belief tuples.

embeds is (B, T, embed_dim) (embeddings()). is_first (B, T) restarts the filter at episode starts (observe_no_action()); it is required, and None declares that every window is one episode. Returns the positional belief tuple in the CRSPP’s declaration order with leaves (B, T, ...); aux_beliefs takes it keyed: system.from_positional("beliefs", ...).

class hybrid_aif.reference.raif.crspp.CRSPPLosses[source]#

Bases: NamedTuple

The preference model’s loss, decomposed.

total is what the gradient is taken of. kl is the gated, averaged balanced KL and kl_raw the same before the gate. contrast is the cosine term before its scale. positive is the fraction of steps the gate admitted.

total: Array#

Alias for field number 0

kl: Array#

Alias for field number 1

kl_raw: Array#

Alias for field number 2

contrast: Array#

Alias for field number 3

positive: Array#

Alias for field number 4

static __new__(_cls, total, kl, kl_raw, contrast, positive)#

Create new instance of CRSPPLosses(total, kl, kl_raw, contrast, positive)

hybrid_aif.reference.raif.crspp.crspp_loss(
parts,
key,
embeds,
rho,
agent_features,
*,
is_first,
contrast=1.0,
dyn_scale=0.5,
rep_scale=0.1,
free_nats=1.0,
return_states=False,
)[source]#

The CRSPP loss on one batch: (total, CRSPPLosses). Differentiable in parts.

embeds (B, T, embed), rho (B, T), agent_features (B, T, feature_dim): the agent’s [deter, flat stoch] at the same steps, compared under a stop-gradient. is_first (B, T) restarts the filter at episode starts (observe_no_action()); it is required, and None declares that every window is one episode. With return_states=True the auxiliary is (CRSPPLosses, (deters, codes)), the pass’s own (B, T, ...) carries and codes, not gradient-stopped.

hybrid_aif.reference.raif.crspp.fit_crspp(
system,
params,
learn_state,
embeds,
rho,
agent_features,
*,
key,
is_first,
lr=0.0001,
lr_by_slot=None,
clip=1000.0,
return_states=False,
**loss_kwargs,
)[source]#

One gradient step of preference learning: (crspp_params, learn_state, losses).

system/params are the CRSPP’s own pair (System.scoring[ROW], SystemParams.scoring[ROW]) and learn_state is the nested carry at scoring:cosine_sim_to_pref_rollout, so lr_by_slot addresses the CRSPP’s slots. One global clip over the whole model precedes the per-slot steps. With return_states=True a fourth element follows: the pass’s posterior as the belief tuple crspp_recognize() returns, gradient stopped. is_first and **loss_kwargs go to crspp_loss().

hybrid_aif.reference.raif.crspp.CRSPP_BLOCK = 'crspp'#

The similarity leaf’s block key in the term tree.

hybrid_aif.reference.raif.crspp.cosine_sim_to_pref_rollout(st)[source]#

Their sim: cosine similarity to the CRSPP’s imagined preferred state.

The CRSPP is the System in this row’s scoring slot. Its rollout is update_empirical_prior with no action parent, so the preferred trajectory is computed once in prepare, for the st.horizon steps of the plan being scored: sampled with st.key, expected without. Comparison is on [deter, flat stoch]. The rollout starts from aux_beliefs[<this row's name>] (crspp_recognize(), passed through score_policies or imagine); with none it starts from the CRSPP’s declared episode start, a generic trajectory.

Design: docs/design/amortization-debt-ledger.md entry 5.

hybrid_aif.reference.raif.crspp.embeddings(system, params, obs)[source]#

The agent’s encoder output on a batch of observations: (..., embed_dim).

obs may carry any number of leading axes.