Switching density networks for hybrid control systems

Hybrid system identification can be particularly challenging, particularly in the context of visuomotor control. We introduce switching density networks (SDNs), which can be used to identify switching control systems in an end-to-end learning fashion from demonstration data.

We show that SDNs, when paired with a general purpose family of proportional-integral-derivative control laws, can identify the pump, spin and balance controllers required to keep an inverted pendulum upright (see header). We also use them to identify the joint angle goals that make up an inspection task on a PR2 robot, and those needed to open a suitcase.

Switching density networks are particularly useful for options learning, as the controllers identified using these can be re-used elsewhere. Importantly, by embedding structure into the network, SDNs become more interpretable, and allow for hierarchical learning that is not possible with their closely related counterparts, mixture density networks.

M Burke, Y Hristov, S Ramamoorthy, Switching Density Networks for Hybrid System Identification,  Conference on Robot Learning (CoRL) 2019. (arxiv link)

Inducing explainable robot programs

End-to-end learning is able to solve a wide range of control problems in robotics. Unfortunately, these systems lack interpretability and are difficult to reconfigure if there is a minor task change. For example, a robot inspecting a range of objects needs to be retrained if the order of inspection changes.

We address this by inducing a program from an end-to-end model using a generative model consisting of multiple proportional controllers. Inference under this model is challenging, so we use sensitivity analysis to extract controller goals and gains from the original model. The inferred controller trace (a sequence of controller goal states) is then simplified and controller specific grounding networks trained to predict controller goals for visual inputs, producing an interpretable and reconfigurable program describing the original learned behaviour.

Michael Burke, Svetlin Penkov, Subramanian Ramamoorthy, From explanation to synthesis: Compositional program induction for learning from demonstrationRobotics: Science and Systems (R:SS), 2019. arXiv link