source: liacs/mms/project/README@ 422

MotionMouse, Rick van der Zwet - 0433373 <hvdzwet@liacs.nl>



= Abstract =

'Enhanced' presentation utility. Mouse interaction device using gestures or a

laser pointer and a WiiRemote controllor.



= Introduction =

Current presentation devices only have a few forward/backward buttons and a

laser pointer, this is not suffient when presenting interactive media (e.g.

switching between many diffent input sources.). As now-a-days almost all

laptops has a high quality webcam included and are powerful devices. Using

commodity tools there needs to make a much better use of the resources

available.



= Background =

Alternative available devices 'emulate' the mouse using gyrosensors or a

trackball. The combination with laser tracking has been tried already in a

different form with a more fixed setup [1]. Laser pointer tracking has also

been done, but most of the times not in lively envirionments. Gesture tracking

has been done in a fixed setup with a easy distinguable object like a dark

background [2], or a well coloured glove [3] or in a more advanced but static

setup [4]. Also using high-tech equipment [5].



= Implementation =

The system is build on a Macbook Pro (model:4,1) running Ubuntu 9.10 amd64. The

Webcam used for tracking is an USB Trust WB-6250X Webcam (145f:0142). The

OpenCV package (2.0.0) is compiled from source. The WiiRemote is an model from

late 2008. The laser pointer used is a green laser pointer with high density

(50mW, WARNING: use in protected envirionments with IR safety eye wear).



The Beamer used had an resolution of 800x600 and was projecting on a white

stone wall. The webcam was placed on top of the beamer.



= Description =

The system consists of 3 components. The buttons and it's handeling are done

using the WiiRemote which connects to the laptop using bluetooth and the Open

Source program cwiid [6].



The second component is build on the OpenCV [7] package. It detect the laser

pointer using a difference-algoritm. 1) Blurring is applied to get rid of the

webcam detections errors (.e.g wrong pictures). 2) The current frame values are

subtracted by the previous frame. 3) The spot of X times Y with the highest Red

and Green density is detected as the current laser pointer position, if the

threshold is above a certain value. The C source code is found at [8].



The mouse location is updated to it's right location taking into count

transformation from the virtual to the real screen. The UNIX X11 implementation

is done using parts of the xdotool [9] package.



= Discussion and Conclusions =

The status of the project could be marked as an early alpha. Mouse updates and

clicking works fine, altough the initialisation phase (the so called pairing)

could use some improvement. The tracking works if the image is static and there

is not many distorting with regards to movement.



The laser to mouse pointer caliberation is not yet working has has to be done

by hand, the checkerboard image (in the source [8]) also needs to be done

automatically.



Brightness control is a part to focus on further research. The Webcam

now tries to optimize for the whole screen, causing the laser pointer to

disappear in a bright light blob.



The tracking algoritm now has trouble using moving images. As the picture send

to the output could be 'snooped' as well. There should be some more smarter

detection posible of the laser pointer.



= References =

[1] Christopher Armenio, Design of a Laser Controlled Mouse Using OpenCV, 2008



[2] Cristina Manresa, Javier Varona, Ramon Mas and Francisco J. Perales, Real

    –Time Hand Tracking and Gesture Recognition for Human-Computer Interaction,

    Electronic Letters on Computer Vision and Image Analysis 0(0):1-7, 2000



[3] Robert Y. Wan, Jovan Popovi, Real-Time Hand-Tracking with a Color Glove,

    2009



[4] Shahzad Malik, Real-time Hand Tracking and Finger Tracking for Interaction,

    2003 



[5] Daniel Kurz, et al., Laser Pointer Tracking in Projector-Augmented

    Architectural Environments, 2007



[6] cwiid, http://github.com/abstrakraft/cwiid



[7] OpenCV, http://opencv.willowgarage.com



[8] Detection software, http://rickvanderzwet.nl/svn/personal/liacs/mms/project/



[9] xdotool, http://www.semicomplete.com/projects/xdotool/