Dual-Arm Stacking with Neural Dynamical Systems

Classical motion controllers for manipulation are either too rigid (scripted trajectories) or too fragile (open-loop). This project explores a middle ground: using learned neural dynamical systems (DS) to govern free-space motions, while switching to precise scripted IK for constrained contact phases (grasp, lift, place).

The task is dual-arm cube stacking in simulation: two robot arms must coordinate to pick up cubes and stack them in the correct order without interfering with each other's workspace.

The architecture:

• Neural DS controllers learn joint-space primitives for reach and transport motions. Each primitive is trained from demonstration data and constrained to be Lyapunov-stable — guaranteeing convergence to the goal regardless of starting configuration.
• Lyapunov safety projection reshapes the DS output at runtime to enforce stability, preventing the arms from entering divergent states even under perturbation.
• Cartesian workspace modulation enables the two arms to share the workspace safely, dynamically adjusting each arm's DS to avoid the other's trajectory.
• Lula IK handles constrained contact phases (grasp, lift, place) where geometric precision matters more than smooth flow.

With coordinate modulation enabled, the system achieved 100% task completion — stacking every cube in the correct order across all test trials. The project highlighted that data quality (demonstration consistency) had a larger effect on final performance than architecture complexity.