Planning#

A System holds the name of a planner, never a planner. Planners are resolved through a registry, and the built-in ones are enumeration over a policy grid, the cross-entropy method, gradient descent, a mixed discrete and continuous search, a habit policy, and the sophisticated planner that branches on observations. Each has a ..._plan function, a ..._planner object that fits the registry and a make_... constructor that binds options.

The registry#

Planner

Policy inference over a PlannerContext.

PlannerContext

Everything a System offers a planner for a single infer_policies call.

register_planner

Decorator registering a planner under name for resolution by Systems.

get_planner

The planner registered under name (register_planner()).

planner_needs_key

Whether planner declared itself stochastic (see register_planner()).

planner_branches

Whether planner (a planner, or a registered name) declared itself branching (see register_planner()); an unregistered name is not.

planner_legal_depth

Whether planner declared that it masks illegal actions at every depth (see register_planner()).

planner_takes_init

Whether planner takes a warm start: the mark takes_init and the keyword init (see Planner).

Enumeration and policy scoring#

enumeration_planner

The Planner face of this module.

enumerate_policies

Exhaustive policy matrix, int32 (num_policies, policy_len, len(num_controls)).

evaluate_policies

Score every policy: scan beliefs over the horizon, accumulate step TermTrees.

evaluate_policy_tree

Score every policy of the grid by summing the step term trees along its path.

evaluate_policy_tree_total

Score every policy of the grid straight to its G, one Interval per node.

expand_level

Push every node through every action: one level of the policy tree.

policy_posterior

Policy posterior q_pi ∝ softmax(-gamma * G + log E + log 1[legal]).

action_marginals

First-step marginal posterior over each policy column's levels (pymdp get_marginals); num_controls gives each column's level count.

sample_action

Select an action from a policy posterior: a control level per control node, or a vector.

control_steps

Clamped steps over every control node from the discrete levels (..., policy_len, K_discrete) and the continuous plan (..., policy_len, action_dim): a ControlStep with each node's entry, a level array (..., policy_len) or a float block (..., policy_len, width).

Cross-entropy and gradient#

cem_planner

The Planner face of this module.

make_cem_planner

A CEM Planner with kwargs bound.

cem_plan

Run CEM against ctx; see the module docstring for the algorithm.

CEMResult

One CEM plan: the posterior, the metrics, and the proposal it ended on.

gradient_planner

The Planner face of this module.

make_gradient_planner

A gradient Planner with kwargs bound (those of gradient_plan()), e.g. make_gradient_planner(posterior="laplace", steps=400).

gradient_plan

Descend G over the float plan against ctx; see the module docstring.

GradientResult

One gradient plan.

PlanPosterior

A continuous plan and the Laplace posterior around it.

mean_action

The receding-horizon action: the final proposal mean's first step, candidates[0, 0].

Mixed, habit and sophisticated#

mixed_planner

The Planner face of this module.

make_mixed_planner

A mixed Planner with kwargs bound (those of mixed_plan()), e.g. make_mixed_planner(steps=100, discrete_policies=held).

mixed_plan

Search a mixed control space against ctx; see the module docstring.

MixedResult

One mixed plan, over P discrete policies, each with optimised continuous controls.

habit_planner

The Planner face of this module.

habit_action

One action from a policy module: the closure a System hands this planner.

sophisticated_planner

The Planner face; metrics=True returns {}.

make_sophisticated_planner

A sophisticated Planner with the memory options of sophisticated_plan() bound, e.g. make_sophisticated_planner(batch_size=64) or make_sophisticated_planner(memory_budget=2**30).

sophisticated_plan

The sophisticated planner with its memory options.

evaluate_branching_tree

The root's per-action G under sophisticated (closed-loop) planning.

branching_tree_memory

The memory of system's sophisticated tree at beliefs, estimated, and what memory_budget (bytes) would choose; nothing is evaluated.

BranchingTreeMemory

The memory of the sophisticated planner's tree, estimated.