Planning#
A System holds the name of a planner, never a planner. Planners are resolved through a registry,
and the built-in ones are enumeration over a policy grid, the cross-entropy method, gradient
descent, a mixed discrete and continuous search, a habit policy, and the sophisticated planner that
branches on observations. Each has a ..._plan function, a ..._planner object that fits the
registry and a make_... constructor that binds options.
The registry#
Policy inference over a |
|
Everything a System offers a planner for a single |
|
Decorator registering a planner under |
|
The planner registered under |
|
Whether |
|
Whether |
|
Whether |
|
Whether |
Enumeration and policy scoring#
The |
|
Exhaustive policy matrix, int32 |
|
Score every policy: scan beliefs over the horizon, accumulate step TermTrees. |
|
Score every policy of the grid by summing the step term trees along its path. |
|
Score every policy of the grid straight to its G, one Interval per node. |
|
Push every node through every action: one level of the policy tree. |
|
Policy posterior |
|
First-step marginal posterior over each policy column's levels (pymdp |
|
Select an action from a policy posterior: a control level per control node, or a vector. |
|
Clamped steps over every control node from the discrete levels |
Cross-entropy and gradient#
The |
|
A CEM |
|
Run CEM against |
|
One CEM plan: the posterior, the metrics, and the proposal it ended on. |
|
The |
|
A gradient |
|
Descend |
|
One gradient plan. |
|
A continuous plan and the Laplace posterior around it. |
|
The receding-horizon action: the final proposal mean's first step, |
Mixed, habit and sophisticated#
The |
|
A mixed |
|
Search a mixed control space against |
|
One mixed plan, over |
|
The |
|
One action from a policy module: the closure a System hands this planner. |
|
The |
|
A sophisticated |
|
The sophisticated planner with its memory options. |
|
The root's per-action G under sophisticated (closed-loop) planning. |
|
The memory of |
|
The memory of the sophisticated planner's tree, estimated. |