Real-Time Gesture Recognition System and Application: a motion-based approach

We challenge the problem of human behaviours understanding and imitation in computer vision and cognitive robotics applications. In this thesis we show how to design and develop a complete real-time arm-hand gesture recognition system: starting from videos we want to define descriptors able to capture human gestures and complex actions. We used a motion based approach without any prior knowledge of subjects in the scene. Our aim is to define most general gesture primitives regardless of the object that performs the action. Using primitives we can divide the gesture recognition problem into two level: low and high level. In the first one we recognize gestures primitives using Mixture of Gaussians for each primitive, the high level system combines sequences of primitives using Deterministic Finite Automata (DFA). We provide general gestures descriptors, we model them and we show how to learn new gestures just from one-shot demonstration. Finally we show how to implement two different applications based on the above system. The whole thesis has been developed and implemented at Imperial College of London, within the BioART laboratory under the supervision of Dr. Yiannis Demiris.