Arm Gesture Recognition for Human-Computer Interaction
Where: Université de Liège
When: 2010, 6 months
What: Master Thesis: Development and implementation of an algorithm for controlling a computer equipped with a stereo camera
Keywords: Image processing, computer vision, human-computer interaction, Stereo Vision, C++, OpenCV, CMake
This work presents an approach for human-computer interaction using a computer vision system based on a stereo camera. It is based on the depth information stereo heads are able to provide in order to track the user’s hand in space. The presented system is able to recover the position of the hand in the image plane as well as its distance from the camera. It is also able to identify if the user’s hand is closed or if the user is presenting his palm to the camera, in which case the system can recover the palm tilt. Those inputs are recovered in real-time and enable any user to control a typical mouse pointer simply by moving his/her fist in front of the camera, let him/her perform simple gestures that the system recognizes as clicks, page up/down and scroll up/down operations.
The actual system is using a stereo camera from Videre Designs (STH-MDCS2-9cm) and has been developed for Linux in C++ using the OpenCV as a back-end library to perform classic and well know tasks such as depth map computation or Kalman filtering.
The center core of the project is divided into two parts:
Hand tracking: Two methods has been developed for this. An adaptive algorithm giving the user some freedom about where to stand in order to operate the system, and a simpler method, removing this freedom but giving better results and robustness.
Gestures analysis: Analyzing the position and the shape of the hand in order to generate events to be sent to the X server.