Face Extraction Using EigenSnakes

A rigid template, such as Hough Transform, allows global deformation only. Deformable template or active contour model (snake) allows also local deformation, but would be too "floppy" for face extraction if insufficient prior knowledge about expected shapes is incorporated. In EigenSnake,

Prior knowledge about expected shapes is incorporated by a statistical model,
An affine invariant shape model is integrated for accounting for global deformation due to rigid motion.
The ability of normal snake models to account for local deformation is retained.
A Bayesian scheme is employed to integrate general knowledge about shapes and observation.

METHOD

Bayesian Formulation and Energy Functions: Using Bayesian framework, the problem to extract a contour with unknown deformation from a noise image can be posed as a problem of maximum a posterior (MAP) estimation

where f denotes the contour, d is the input image, u is the translation of f from the target object. The MAP estimates are then obtained by

Eigensnakes for face extraction: Our model for face extraction consists of 7 EigenSnakes, each corresponding to a facial feature.

EXPERIMENT

This experiment demonstrates the EigenSnake applied to extract a full face with 7 facial features. The training set consists of 120 frontal face images from the face database of Bern university. There are two testing sets: (i) The first is a normal testing set. It consists other (which are not used to train) 60 frontal face images also from the Bern face database. (ii) The second is a difficult testing set chosen from the face database of Carnegie Mellon University; this set contains samples with much more noise, occlusion and other structures than those previously used in most successful face extraction experiments.

A total of 108 landmark points is used to represent a full face, among which 10 for left brow, 10 for right brow, 10 for left eye, 10 for right eye, 17 for nose, 16 for mouth and 35 for the face boundary. The means of the training contours are used as initial prototype contours of 7 g-snakes and an EigenSnake respectively.

Contour initialization is done manually, as a common practice for snakes. It has the same shape as the mean contour but it is re-scaled randomly to between 70% and 120% of the true mean scale, shifted away randomly from the true position by between +- 20 pixels, and rotated randomly from the true orientation in the range of +- 15 degrees. All the methods have the same initial contour for each test image.

Overall, it has been demonstrated by experimetns that the EigenSnakes

can correctly locate the full face contours if the initial distance between the initial contour and the target object is in the scope of one fourth of the initial contour,
is scale and rotation insensitive to a great extent,
can resist the misguide due to other structure or noise.

The following are some experimental results (initial contour on the left and final contour on the right):