Opera Face Gestures Beta

Earlier this month was April Fools’ Day and there is no better place to be than the Internet. Many different companies like to release fake products and services on that day. Opera was one of the many companies to release such a product. This year they introduced Opera Face Gestures to revolutionize web browsing…or not. Even the guy making the faces can’t keep his face straight. Out of all of the jokes that I saw that day this one really stuck with me. I ended up sharing the link with a few of my friends and one of them commented on how hilarious it would be to see a whole class of people using it. Imagine someone unknowing walking in and seeing all those people making ridiculous faces.

The idea there inspired me to make my own program similar to Opera Face Gestures. I was hoping to throw this together quickly (quicker than releasing this on the 23rd) so the idea would be fresh. It did take a bit longer than I wanted but at least it is in a somewhat working order. I started looking for libraries that would assist in detecting faces, eventually I found OpenCV. I was hoping to find something with Python bindings and sure enough OpenCV has them. I found example code from a few websites but it wasn’t really needed since included in the library are a bunch of sample programs. This was enough to get me started but how would I get it to recognize facial moments?

When I started this I knew nothing about facial detection and how I would end up getting this to work. I’m still pretty ignorant on the subject but I’d always care to know more. One of the first example videos I saw was of a face that had many different points over key areas. I could use the movement of the points to make an algorithm. As you can see in that last video the points were easily moved and during my experiments I couldn’t get the points to stay reliably. I’m sure there are better ways to do what I’ve created but I was trying to get this done quickly. Lucky for me there was a motion example and a face detection example included with OpenCV.

Using those two examples I combined facial detection and motion. Then I started tracking motion only inside the face. I eventually added a simple calibration function to isolate the eyes, eyebrows and mouth. Which normalizes the rectangles found inside the detected face since the detected face size can change. Then we find the mode of the detected rectangles and generate an area based on the standard deviation of the collected points. Rectangles found in that area are saved and we find the one with the largest area. Then we prompt the user to accept the detected areas or to restart the process. Once that was completed I ignored all motion outside those areas. The motions inside the calibrated areas are checked to see if they match a pattern such as a eye blinking during a specified time period (to prevent false positives of normal movements while you sit in front of your computer). Sadly lighting plays the biggest role in how much movement is detected. Rooms with too much lighting might not be able to detect eye movements reliably. Eye movement is the hardest one to match since the conditions must be perfect.

To set this up on your computer you’re going to need Python 2.5 and OpenCV 1.0 (not sure what web cams are actually supported). I couldn’t get OpenCV 1.1 pre1 working so I’ve stuck with the 1.0 release. Since it is the 1.0 release and not 1.1 you will have to use Python 2.5 since I couldn’t get the bindings to compile correctly. I’m using an older Logitech QuickCam Pro for Notebooks which doesn’t say that it is supported on the OpenCV wiki but works well. By older I have one that only works with Logitech QuickCam 1.1.0.0. You’re going to have to copy the OpenCV Python bindings folder found in C:\Program Files\OpenCV\interfaces\swig\python\build\lib.win32-2.5 and put it in C:\Python25\Lib\site-packages. You can find my code here and the haar classification cascade here.

Assuming everything is setup correctly just run the program and follow the directions in this video. You will need to turn on closed captions since annotations weren’t working for this video.

There is one issue with high memory usage after using this program for an extended period of time. I’m not sure who’s really at fault for the apparent memory leak. I’ve tried using the OpenCV API to release the memory allocated by the images/frame but it makes no difference or results in the program crashing. I’d appreciate any feedback about my code and if there are any errors or ways to make it more pythonic.

In case you missed out on April Fools’ Day here is some of what you missed.