Poses and Grimaces: Challenges for automated face identification algorithms?

Forensic image identification is based on the assumption that images can convey information about person's identifying characteristics. While any aspect of physical appearance (motion, body build, stature, clothing) can be processed, facial appearance is the most common identifying feature. Nowadays, image identification tasks are gradually being automated and are therefore the subject of many machine learning algorithms, with convolutional neural networks (CNNs) being the leading strategy. From the perspective of everyday forensic expertise, automation offers several advantages. It reduces the time required to process a large number of images, increases accuracy, and eliminates the human factor. Its quantitative nature provides the necessary scientific basis, testability, and a quantifiable probability of error - all critical requirements for a method to be considered applicable in forensics.

However, the performance of automated forensic image identification is susceptible to many factors. The constantly changing behavioral characteristics of the captured subjects (e.g., pose, expression, disguise) have been repeatedly cited as major challenges. In addition, the complicated and hidden nature of automated algorithms creates the black-box problem that prevents a complete understanding of the algorithm and its outcomes unless various real-world conditions are thoroughly tested.

This study tests two state-of-the-art face identification algorithms - ArcFace [1] and SphereFace [2], and examines two factors known to complicate face processing - facial expressions and head pose.