GRUCell#
- class hybrid_aif.distributions.GRUCell[source]#
Bases:
ModuleA GRU carry update
(deter, *inputs) -> deter': a pure map, not a conditional.Declared through
Deterministic. DreamerV3’s GRU variant: project the non-carry inputs tohiddenunits, normalize, activate; concatenate the carry; one linear to3·deter(normalized) giving(reset, candidate, update); thendeter' = u·tanh(σ(reset)·candidate) + (1−u)·deter,u = σ(update − 1)inputsare unbatched vectors of widthsinput_dims, in declaration order; the carry is passed separately.- classmethod from_shapes(
- deter_dim,
- input_dims,
- *,
- key,
- hidden,
- activation='silu',
- norm=True,
- init='default',
A randomly-initialized cell;
hidden(required) is the width the non-carry inputs are projected to.norm=Trueputs a LayerNorm (eps 1e-3) after each of the two linears.init="default"keeps both biases andeqx.nn.Linear’s draw;init="trunc_normal_avg"drawstrunc_normal_avg()weights on both linears, drops their biases when normed and starts any kept bias at zero.
- __init__(input_proj, gate, norms, deter_dim, input_dims, activation='silu')#