|
|
Face, Speech, and Acoustics |
|
| Introduction · Program · Local Information · Registration · Contact |
In this talk we propose a method to measure face motion from standard video recordings by deforming the surface of an ellipsoidal mesh fit to the face. The mesh is initialized manually for a reference frame and then projected onto subsequent video frames. Location changes (between successive frames) for each mesh node are determined adaptively within a well-defined area around each mesh node, using a two-dimensional cross-correlation analysis for a two-dimensional wavelet transform of the frames. Position parameters are propagated in several steps from a coarser mesh and a correspondingly higher scale of the wavelet transform to the final fine mesh and lower scale of the wavelet transform. The sequential changes in position of the mesh nodes represent the facial motion.
The method takes advantage of inherent constraints of the facial surfaces. This distinguishes it clearly from more general image motion tracking methods, on the one hand, and from feature-based methods, on the other hand
Tracking results from a large set of video sequences (83 sentences uttered by a single speaker) show that different parts of the face are tracked equally well, despite differences in image gradient - e.g., strong image gradients for the lips, and weaker gradients for the cheeks or the chin. This was confirmed by comparing directly a set of manually tracked points in a subset of 8 sentences from a different experiment with the automatic tracking.
For additional validation a Principal Component Analysis (PCA) was applied to the larger data set. The behavior picked up by the first components was then compared to a similar analysis of motion tracking results acquired with a marker-based method known to be able to perform precise tracking (OPTOTRAK).