A US startup out of Bell Labs has developed a way of producing realistic animation of a face at low bit rates from an ordinary video stream, opening up the possibility of having talking heads on Web sites and mobile phones with today's low speed technology.

The technology from Face2face generates a 2kbit/s stream of plain text that can be used by a standard MPEG4 player to animate a face based on an audio stream. Face2Face has developed its own player, but the stream can be used to animate any face as long as the player conforms to the MPEG4 face and body animation (FBA) format.
The technology is heading into the retail market at the end of the year, as the company has developed an encoder box that combines a CMOS camera chip with the BSP-15 multimedia chip from Equator Technologies. The unit will produce the 2kbit/s stream in real time, possible linked to a PC via Bluetooth or other wireless technology, where it can be streamed over the Internet. A player on a PC or mobile phone at the other end would then use a model of any face synchronised with the audio stream.
Face2face currently uses face models with 300 triangles, and these can be generated from any image, including the original video stream.
Higher resolution images could be used depending on the processing power of the terminal. While photorealistic animation isn't possible yet, the models can be corrected using tools such as Photoshop, says Dr Eric Petajan, founder and chief scientist of Face2Face. "We can make people look better than they are," he said. "It's virtual plastic surgery."
The technology is aimed at the ARM9 processor core being used in next generation mobile phones and does not need a 3D accelerator co-processor, although that would help. "ARM9 is the sweet spot and because of the model independence we don't need graphics primitives," he said.
It is being used by animators to produce realistic faces via a Website. Face2Face encodes the video and sends it hack to the animators within 24 hours The technology tracks specific points on the face such as eyebrows, eyelids, nostrils, inner lip contours and direction of gaze. This also allows the animation to accurately lip sync to music for realistic singing.
The company also sells different face models, which can be in two or three dimensions and could be of anyone, including celebrities.