GRUCell#

class hybrid_aif.distributions.GRUCell[source]#

Bases: Module

A GRU carry update (deter, *inputs) -> deter': a pure map, not a conditional.

Declared through Deterministic. DreamerV3’s GRU variant: project the non-carry inputs to hidden units, normalize, activate; concatenate the carry; one linear to 3·deter (normalized) giving (reset, candidate, update); then

deter' = u·tanh(σ(reset)·candidate) + (1−u)·deter, u = σ(update − 1)

inputs are unbatched vectors of widths input_dims, in declaration order; the carry is passed separately.

input_proj: Linear#

The projection of the non-carry inputs to hidden units.

gate: Linear#

The linear map from [deter, hidden] to 3 * deter: reset, candidate and update.

norms: tuple[LayerNorm, ...]#

The (input, gate) layer norms, or () for none.

deter_dim: int#

The width of the carry deter.

input_dims: tuple[int, ...]#

The widths of the non-carry inputs, in declaration order.

activation: str = 'silu'#

The activation applied after the input projection.

classmethod from_shapes(
deter_dim,
input_dims,
*,
key,
hidden,
activation='silu',
norm=True,
init='default',
)[source]#

A randomly-initialized cell; hidden (required) is the width the non-carry inputs are projected to.

norm=True puts a LayerNorm (eps 1e-3) after each of the two linears. init="default" keeps both biases and eqx.nn.Linear’s draw; init="trunc_normal_avg" draws trunc_normal_avg() weights on both linears, drops their biases when normed and starts any kept bias at zero.

__init__(input_proj, gate, norms, deter_dim, input_dims, activation='silu')#