Gesture recognition

Gesture recognition can be a valuable interface for human-robot interaction, but typical approaches to gesture recognition use pre-defined dictionaries of signs for recognition. This is often not particularly intuitive, so we explored the pantomimic gestures for human robot interaction.

Pantomimic gestures are those that mimic a desired behaviour or action, and can be used to improve gesture recognition. By mapping human actions to robot behaviours using matrix factorisation, we can successfully classify gestures. In fact, recognising human gestures using recordings of robot behaviours turns out to be easier than recognising these using recordings of gestures from other users.


Burke, M. G. (2015). Fast upper body pose estimation for human-robot interaction (doctoral thesis).

M. Burke and J. Lasenby, “Pantomimic Gestures for Human–Robot Interaction,” in IEEE Transactions on Robotics, vol. 31, no. 5, pp. 1225-1237, Oct. 2015.

M. Burke and J. Lasenby, “Multilinear function factorisation for time series feature extraction,” 2013 18th International Conference on Digital Signal Processing (DSP), Fira, 2013, pp. 1-8.