PredictorHead#

class hybrid_aif.PredictorHead[source]#

Bases: Module

An MLPPredictor plus the name of the loss that reads its output.

What a likelihood slot holds for a head without a density family: symlog_mse reconstructs a vector, symlog_disc predicts a scalar through num_bins two-hot bins over [bin_low, bin_high] in symlog space, binary predicts a Bernoulli from one logit. nll() is the training loss and predict() the decoded prediction, in the value’s own units. Not a Distribution or a Conditional, and role-free. The three loss names are a closed list; an unknown name raises.

Design: docs/design/amortization-p0-addendum.md §2

trunk: MLPPredictor#

The network whose output the loss reads.

loss: str#

The loss name: "symlog_mse", "symlog_disc" or "binary".

num_bins: int = 255#

The number of two-hot bins (symlog_disc only).

bin_low: float = -20.0#

The lowest bin, in symlog space (symlog_disc only).

bin_high: float = 20.0#

The highest bin, in symlog space (symlog_disc only).

classmethod from_shapes(
parent_dims,
*,
loss,
key,
out_dim=None,
num_bins=255,
bin_low=-20.0,
bin_high=20.0,
width,
depth=2,
activation='silu',
norm=True,
out_scale=1.0,
frozen=False,
symlog_inputs=False,
init='default',
)[source]#

A randomly-initialized head. out_dim is required for symlog_mse and rejected by the other two losses, which fix the trunk’s width (one logit per bin; one logit). width is required; the trunk arguments are MLPPredictor.from_shapes()’s.

property out_dim: int#

Width of the trunk’s raw output (logits, for the two classifier losses).

property target_dim: int#

Width of the value this head predicts: the vector’s, or 1 for a scalar.

property bins: Array#

A symlog_disc head’s bin centres (a constant; XLA folds it).

nll(target, parents)[source]#

This head’s loss on one target: the training half of the contract.

predict(parents)[source]#

The decoded prediction, in the predicted value’s own units.

symexp of a symlog_mse output, the expected bin of a symlog_disc head, σ(logit) for a binary head.

__init__(trunk, loss, num_bins=255, bin_low=-20.0, bin_high=20.0)#