Face Recognition in Video

Will computers (man-made perception systems) be ever as efficient in recognizing what they see as are humans?

This was the question which drove us to look more closely into the way perception is done in biological systems. It was realized then that not only the results from Computer Vision and Pattern Recognition, commonly used by computer scientists working in the area, should be used for this problem, but also the results obtained in Neurobiology.

We are still far from modeling the entire brain (or visual cortex), which will unlikely ever happen, but some advances have been already made, most important of which are

1. Setting a distributed, non Von-Neumann, neural-network based paradigm for video processing and memorization, which allows one to incorporate the visual attention and other visual perception mechanisms performed in the human brain , and
2. Establishing the framework for testing the performance of the thus developed "mini models of brains" - the way we call our Facial Video Memory system.

One of the major advantages of the first result is that it allows one to deal with the continuous flow of video images, as they are in video. - Rather than storing an unlimited number of individual video frames, the approach uses the incoming flow of data to continuously tune the synaptic connections of a multi-connected neural network. This is analogous to the way associative memorization is performed in visual cortex. Then, when a new video stimulus is presented to the retina, the neural network converges to state which is best described by the past seen/learnt experience.

The second result is related to using video sequences, rather than individual video frames, which is in accordance with the developed paradigm. As claimed in our papers, high resolution is not needed for video-based recognition, as it is not the way it is in biological systems and may, in fact, result in overloading the problem. Hence, our video sequences are 160x120, taken with an off-the-shelf webcam, the face of person in video occupying 1/16 - 1/4 of an image.

Direct applications of this technology are in:

• Face Recognition for Security: Surveillance, Tracking and Backtracking, Biometrics
• Face Recognition for Computer-Human Interaction: Perceptual User Interfaces, Nouse
• Face Recognition for Video annotation and Games
• General object memorization and recognition in low-quality video data

Where to go from here

• Publications: to learn more about this technology
• Associative Neural Networks: the maintained within this project OpenSource library
• Face Database: dataset of facial video sequences which can be used to evaluate face-recognition-in-video technology.