hybrid_aif.reference.raif.crspp#
The CRSPP (prior-preference model): its recognition scan, its loss and its scoring row.
The loss, with rho from hybrid_aif.reference.raif.revise.pref_labels():
L = mean_t[ gate_t · KL_balanced(q2_t ‖ p2_t) ]
+ contrast · mean_t[ −rho_t · cos_sim(C_post_t, sg(post_t)) ]
gate_t = max(0, sgn rho_t) contrast = 1.0
The CRSPP is a declared System with no decoder and no action parent. Its posterior
conditions on ("deter", "external:embed"), the agent’s encoder output handed over as a
vector (crspp_recognize()), and the resulting beliefs reach the scoring row as
aux_beliefs["cosine_sim_to_pref_rollout"], keyed by the CRSPP’s state factor names.
Design: docs/design/amortization-p0-addendum.md §4.
Module Attributes
Their preference-model optimizer is the model group's: |
|
|
|
The row whose parameters this trainer steps, and the address that follows. |
|
The CRSPP's parts, in canonical order — the order its one global-norm clip sees, stated once for the reason |
|
The similarity leaf's block key in the term tree. |
Functions
|
Typed version of namedtuple. |
|
The world-model assembly behind a declared System: |
Their |
|
|
The CRSPP loss on one batch: |
|
The CRSPP posterior over a batch's embeddings, as belief tuples. |
|
The agent's encoder output on a batch of observations: |
|
|
|
One gradient step of preference learning: |
|
One imagined CRSPP step: their |
|
KL balancing with free bits: the complexity term of a world-model loss. |
|
Their |
|
|
|
Register a term name with its sign as a contribution to G: |
|
The CRSPP's parts and slot addresses, validated against A12 item 2's shape. |
|
Sample a one-hot per variable, with the probabilities' gradient attached. |
Classes
|
Array base class for JAX |
The preference model's loss, decomposed. |
|
The CRSPP's declarations, resolved: the parts its scan runs and their slot addresses. |
|
|
A deterministic node: a pure map, optionally with trainable parameters. |
|
A closed interval [lower, upper], broadcastable like its member arrays. |
|
Optimizer state for an agent's gradient-learnable parts: |
|
MLP over concatenated parents → a factored categorical child. |
|
A Mapping is a generic container for associating key/value pairs. |
|
Static structure of a System: discrete, continuous or mixed. |
|
All numeric state of a System: generative tensors, designations and the closed-loop factors. |
- hybrid_aif.reference.raif.crspp.DEFAULT_LR = 0.0001#
Their preference-model optimizer is the model group’s:
adam 1e-4,clip 1000(the CRSPP nets are trained inside the world-model loss’s optimizer scope,configs.yaml:76).
- hybrid_aif.reference.raif.crspp.DEFAULT_CONTRAST = 1.0#
loss_scales.contrast(configs.yaml; addendum §4).
- hybrid_aif.reference.raif.crspp.ROW = 'cosine_sim_to_pref_rollout'#
The row whose parameters this trainer steps, and the address that follows.
- hybrid_aif.reference.raif.crspp.PART_ORDER: tuple[str, ...] = ('gru', 'prior', 'posterior', 'initial')#
The CRSPP’s parts, in canonical order — the order its one global-norm clip sees, stated once for the reason
hybrid_aif.inference.rssm.PART_FIELDSis.
- class hybrid_aif.reference.raif.crspp.CRSPPParts[source]#
Bases:
NamedTupleThe CRSPP’s declarations, resolved: the parts its scan runs and their slot addresses.
- gru: Deterministic#
Alias for field number 0
- prior: MLPCategorical#
Alias for field number 1
- posterior: MLPCategorical#
Alias for field number 2
- with_parts(replaced)[source]#
A copy with the parts at those positions replaced (the gradient’s swap).
- static __new__(_cls, gru, prior, posterior, initial, embed_name, slots)#
Create new instance of CRSPPParts(gru, prior, posterior, initial, embed_name, slots)
- hybrid_aif.reference.raif.crspp.resolve(system, params)[source]#
The CRSPP’s parts and slot addresses, validated against A12 item 2’s shape.
The declaration must be one uncontrolled carry, one code whose parent is that carry, no modality, and one recognition entry over
(carry, external embedding); anything else raises.
- hybrid_aif.reference.raif.crspp.observe_no_action(parts, key, embeds, is_first=None)[source]#
Their
observe_no_action: filter one embedding sequence, no actions.embedsis(T, embed_dim).is_first, optional(T,)bool, marks episode starts inside the sequence: there the carry is replaced by the CRSPP’s own initial state before the step, as theirobs_step_no_actiondoes. The scan always starts from that state. Returns(deters, codes, post_logits, prior_logits)stacked over time; the carried posterior code is a straight-through sample. Unbatched.
- hybrid_aif.reference.raif.crspp.img_step_no_action(parts, key, deter, code)[source]#
One imagined CRSPP step: their
img_step_no_action.(deter', code').
- hybrid_aif.reference.raif.crspp.crspp_recognize(system, params, embeds, *, key, is_first)[source]#
The CRSPP posterior over a batch’s embeddings, as belief tuples.
embedsis(B, T, embed_dim)(embeddings()).is_first(B, T)restarts the filter at episode starts (observe_no_action()); it is required, andNonedeclares that every window is one episode. Returns the positional belief tuple in the CRSPP’s declaration order with leaves(B, T, ...);aux_beliefstakes it keyed:system.from_positional("beliefs", ...).
- class hybrid_aif.reference.raif.crspp.CRSPPLosses[source]#
Bases:
NamedTupleThe preference model’s loss, decomposed.
totalis what the gradient is taken of.klis the gated, averaged balanced KL andkl_rawthe same before the gate.contrastis the cosine term before its scale.positiveis the fraction of steps the gate admitted.- static __new__(_cls, total, kl, kl_raw, contrast, positive)#
Create new instance of CRSPPLosses(total, kl, kl_raw, contrast, positive)
- hybrid_aif.reference.raif.crspp.crspp_loss(
- parts,
- key,
- embeds,
- rho,
- agent_features,
- *,
- is_first,
- contrast=1.0,
- dyn_scale=0.5,
- rep_scale=0.1,
- free_nats=1.0,
- return_states=False,
The CRSPP loss on one batch:
(total, CRSPPLosses). Differentiable inparts.embeds(B, T, embed),rho(B, T),agent_features(B, T, feature_dim): the agent’s[deter, flat stoch]at the same steps, compared under a stop-gradient.is_first(B, T)restarts the filter at episode starts (observe_no_action()); it is required, andNonedeclares that every window is one episode. Withreturn_states=Truethe auxiliary is(CRSPPLosses, (deters, codes)), the pass’s own(B, T, ...)carries and codes, not gradient-stopped.
- hybrid_aif.reference.raif.crspp.fit_crspp(
- system,
- params,
- learn_state,
- embeds,
- rho,
- agent_features,
- *,
- key,
- is_first,
- lr=0.0001,
- lr_by_slot=None,
- clip=1000.0,
- return_states=False,
- **loss_kwargs,
One gradient step of preference learning:
(crspp_params, learn_state, losses).system/paramsare the CRSPP’s own pair (System.scoring[ROW],SystemParams.scoring[ROW]) andlearn_stateis the nested carry atscoring:cosine_sim_to_pref_rollout, solr_by_slotaddresses the CRSPP’s slots. One global clip over the whole model precedes the per-slot steps. Withreturn_states=Truea fourth element follows: the pass’s posterior as the belief tuplecrspp_recognize()returns, gradient stopped.is_firstand**loss_kwargsgo tocrspp_loss().
- hybrid_aif.reference.raif.crspp.CRSPP_BLOCK = 'crspp'#
The similarity leaf’s block key in the term tree.
- hybrid_aif.reference.raif.crspp.cosine_sim_to_pref_rollout(st)[source]#
Their
sim: cosine similarity to the CRSPP’s imagined preferred state.The CRSPP is the System in this row’s scoring slot. Its rollout is
update_empirical_priorwith no action parent, so the preferred trajectory is computed once inprepare, for thest.horizonsteps of the plan being scored: sampled withst.key, expected without. Comparison is on[deter, flat stoch]. The rollout starts fromaux_beliefs[<this row's name>](crspp_recognize(), passed throughscore_policiesorimagine); with none it starts from the CRSPP’s declared episode start, a generic trajectory.Design:
docs/design/amortization-debt-ledger.mdentry 5.