MLPCategorical#

class hybrid_aif.MLPCategorical[source]#

Bases: Module

MLP over concatenated parents → a factored categorical child.

num_vars independent categoricals of num_classes classes each, unimix-mixed and sampled as a straight-through one-hot. The child value is the one-hot code, shape (num_vars, num_classes); flat_dim is its width as a downstream parent. There is no moments: probs() is this family’s moment view.

trunk: MLPPredictor#

The network from the concatenated parents to the num_vars * num_classes logits.

num_vars: int#

The number of independent categoricals.

num_classes: int#

The number of classes of each.

unimix: float = 0.01#

The weight of the uniform distribution mixed into each categorical.

classmethod from_shapes(
parent_dims,
num_vars,
num_classes,
*,
key,
width,
depth=1,
activation='silu',
norm=True,
unimix=0.01,
frozen=False,
init='default',
)[source]#

A randomly-initialized head: depth normed, activated hidden layers of width units (required), then a linear to the num_vars·num_classes logits. init is as in MLPPredictor.from_shapes().

property flat_dim: int#

Width of the flattened one-hot code — what it contributes as a parent.

logits(parents)[source]#

Unimix-mixed logits, shape (num_vars, num_classes).

condition_on(parents)[source]#

The conditional as a Categorical whose batch axis is the variable axis, so entropies and KLs come out per variable.

probs(parents)[source]#

The (num_vars, num_classes) probability tensor: this family’s moment view.

log_prob(child, parents)[source]#

log q(child | parents) for a one-hot code, summed over variables.

Σ_v Σ_c child[v,c]·log p[v,c]: differentiable through a straight-through sample, and finite for every code (unimix_logits()).

sample(key, parents)[source]#

One straight-through one-hot draw, shape (num_vars, num_classes).

mode(parents)[source]#

The argmax code, same shape as sample().

__init__(trunk, num_vars, num_classes, unimix=0.01)#