Source code
github.com/
Contributors
Latest commits

Live Online Demo

Face Alignment Demo

Abstract

Within a facial image, the task of accurately localizing the set of landmark points that define the shape of the face is called face alignment. This paper presents an approach to robustly align a facial shape to a face image of any unknown pose even in the presence of partial occlusions. Most face alignment approaches are vulnerable to partial occlusions of the face, since it is assumed, either explicitly or implicitly, that faces are not significantly occluded. To deal with the multi-modal nature of unknown facial pose and unknown occlusion, our approach explicitly searches over a discrete representation of pose and binary occlusion labels. Towards this end, to handle unknown pose, we first adopt a discrete shape representation as well as a discrete representation for the multi-modal nature of landmark detector responses arising from facial appearance variation with pose. Secondly, to handle partial occlusions, we propose an efficient randomized hypothesize-and-test search over sampled subsets of landmark detector response modes. This method of explicit search over discretized pose and randomized subset based shape fitting can effectively handle unknown pose even in the presence of partial occlusions and recover the correct facial shape. We evaluate the proposed method on a large number of face images, occluded and unoccluded, spanning a wide range of facial poses. Our results demonstrate that facial alignment is accurate and stable over a wide spectrum of occlusion and pose variations demonstrating excellent performance on many real-world face datasets.

Overview

alt text

An overview of our face alignment algorithm. Given an image, faces are detected using a face detector. For face in the image, we fit N different shape models corresponding to different facial poses and expressions. For each model we first get candidate landmark estimates from the local landmark appearance models. We then estimate the geometric transformation parameters and the shape parameters via a hypothesize-and-evaluate procedure. We finally select the best facial shape alignment hypothesis and refine it to get the final facial shape alignment result.

Contributions

  • Face alignment across facial pose robust to occlusions.
  • Combinatorial search over the space of facial pose and facial occlusions which we span using multiple shape models and multiple local feature detectors for each local feature.
  • Efficient hypothesize-and-test approach to search and find the optimal solution over this space of possible solutions.
  • Unlike most previous face alignment approaches, our proposed algorithm also estimates and provides occlusion labels for all landmarks.

Results

alt text

References