MLPConditional#

class hybrid_aif.MLPConditional[source]#

Bases: Module

MLP over concatenated parents → diagonal-Gaussian child (see module docstring).

The family contract is log_prob(), sample() and moments(), all through condition_on(), the plain Gaussian the parents select. stats() (one tick’s gradient of the NLL) and absorb() (the optax step) learn it. It is not a Learnable (there is no posterior over the weights): isinstance(decl, Learnable) is False, so the conjugate paths leave it alone.

Construct with from_shapes(); the field constructor stays the dataclass default so dataclasses.replace and eqx.tree_at work. Freeze with dataclasses.replace(decl, frozen=True).

The final layer emits 2 * child_dim numbers, the mean and then the scale pre-activation, read as head says.

layers: tuple[Linear, ...]#

The linear layers; the trainable leaves live here.

parent_dims: tuple[int, ...]#

The width of each parent vector, in parent order.

child_dim: int#

The width of the child, a diagonal Gaussian.

norms: tuple[LayerNorm, ...] = ()#

One layer norm per hidden layer, or () for none.

activation: str = 'tanh'#

The hidden layers’ activation.

min_scale: float = 0.001#

The softplus head’s additive floor on the scale.

frozen: bool = False#

The trainable marking: True keeps the weights fixed.

head: str = 'softplus'#

How the final layer is read. "softplus": mean raw, scale min_scale + softplus(raw). "bounded": mean tanh(raw), scale minstd + (maxstd - minstd) * sigmoid(raw + 2); minstd == maxstd is a fixed-scale head whose scale parameters get zero gradient. See hybrid_aif.distributions.neural.

minstd: float = 0.1#

The bounded head’s smallest scale.

maxstd: float = 1.0#

The bounded head’s largest scale.

propagation: str = 'first_order'#

How a Gaussian belief on the one continuous parent is read through the network when the conditional is a likelihood on the analytic path: "first_order", a tangent at the mean, or "unscented", the statistical linearisation over the sigma points (linearise()).

classmethod from_shapes(
parent_dims,
child_dim,
*,
key,
width=64,
depth=2,
activation='tanh',
min_scale=0.001,
init_scale=1.0,
frozen=False,
norm=False,
head='softplus',
minstd=0.1,
maxstd=1.0,
init='default',
propagation='first_order',
)[source]#

A randomly-initialized net: depth hidden layers of width units.

init_scale centres a softplus head’s initial standard deviation (the scale-half bias) and must exceed min_scale; a bounded head ignores it. Options, all off by default:

  • norm=True: a LayerNorm between each hidden linear and its activation, hidden biases dropped;

  • head="bounded" with minstd/maxstd (minstd == maxstd is the fixed-σ head);

  • init="trunc_normal_avg": trunc_normal_avg() on the hidden layers, uniform_avg() on the fused head layer (each half at its own (in, child_dim) fan), every bias zero. init="default" is eqx.nn.Linear’s initializer throughout.

propagation sets the static field of that name: "first_order" (default) or "unscented".

property parent_dim: int#

Width of the concatenated parent vector.

property learnable: bool#

Capability: a non-frozen net’s weights are a trainable block.

condition_on(parents)[source]#

The conditional as a plain Gaussian with a diagonal Cholesky.

log_prob(child, parents)[source]#

log p(child | parents) (scalar), computed on the diagonal (diag_nll()).

sample(key, parents)[source]#

One ancestral draw of the child given its parents, shape (child_dim,).

moments(parents)[source]#

(mean, cov) given the parents; cov is the full, diagonal-valued (child_dim, child_dim) matrix.

diag_moments(parents)[source]#

(mean, scale): the head’s diagonal view, with no covariance built.

diag_nll(child, parents)[source]#

Negative log-density on the diagonal: 0.5·Σ((y−μ)/σ)² + Σ log σ + 0.5·d·log 2π.

Exactly −log_prob for this head, without building or solving a (d, d) Cholesky.

Design: docs/design/amortization.md chunk 2c note §(e) item 1

nll(child, parents)[source]#

Negative conditional log-density: the gradient loss on one evidence pair.

stats(child, parents)[source]#

The per-tick statistic ∇ nll(child, parents), as a same-structured pytree.

None sits at every non-trainable position. Raises ValueError on a frozen declaration.

init_opt_state()[source]#

Fresh optimizer state for this declaration — one entry of LearnState.

absorb(stats, opt_state, lr=1.0, decay=1.0)[source]#

One optax step: returns (declaration, opt_state).

The optimizer state is passed in and out. lr scales the step and may be traced; decay other than 1.0 raises NotImplementedError.

__init__(
layers,
parent_dims,
child_dim,
norms=(),
activation='tanh',
min_scale=0.001,
frozen=False,
head='softplus',
minstd=0.1,
maxstd=1.0,
propagation='first_order',
)#