Policy Gradient Landscape

Reward Regime

Binary mode exposes the good/bad split, common-refinement cells, and the induced switching graph.

Display

Direction-Space Geometry

Cone backgroundDraw direction-space sectors behind the main parameter plot.

Good conesShow the normal cones attached to vertices of the good hull. Bad conesShow the normal cones attached to vertices of the bad hull. Cell intersectionsShow the common-refinement cells where a good cone and bad cone overlap.

Difference coneShow boundary rays for .

Cell graphDraw the finite transition graph induced by the cell drift directions. Only self-consistent cellsRestrict attention to cells whose drift points back into the same cell. Only selected outgoing edgeIn the graph, keep only the transition leaving the hovered or selected cell. Show ambiguous cellsInclude boundary or tie cases instead of suppressing them.

Drift vectors

Dynamics In Parameter Space

Gradient fieldSample the vector field across the parameter plane. PG trajectoryDraw the discrete policy-gradient iterate path .

Feature-Space Overlays

Barycentric optimal pathTrack the induced feature-space point supported on optimal actions.i Barycentric bad pathTrack the induced feature-space point supported on non-optimal actions.i Optimal hullOutline in feature space. Suboptimal hullOutline . Orthogonal projectionsProject non-optimal points onto the edges of the optimal hull.

Drift vectors are the cell-wise directions that define the switching map.

Cells and the induced graph require binary good/bad rewards.

Simulation

0.01

50k

White circle = (draggable)