Reconstructing face representations: a psychological and computational modelling approach to explore the effect of familiarity on human face recognition

Abstract

When thinking about finding the face of a friend in a crowd, albeit challenging, most of us would be relatively certain we would be successful in finding them. However, if the same task were applied to an unfamiliar face, e.g. based on a photo, this would seem like an impossible task. Although the field of face recognition is very active across multiple disciplines, and there is a wealth of research on face recognition and the difference between familiar and unfamiliar faces, it still remains unclear what internal face representations we use. The aim of this thesis is to investigate what internal representations of faces we have and how these are different between familiar and unfamiliar faces. A better understanding of this internal representation will allow for the development of tasks that measure face processing abilities in more detail and increase our understanding of the visual information needed to identify a face, either familiar or unfamiliar. In the behavioural part of this thesis, I focused on exploring the relationship between image-based and perceptual similarity. In the computational part, I explore the concept of familiarity in deep neural networks. This thesis can be summed up in four major findings: there is a linear relationship between image-based spaces and perceived similarity that can be used to explore the effect of face image transformations; familiar faces are perceived as more similar overall compared to unfamiliar faces; the increase in sensitivity for greater changes between pairs is larger for familiar faces than for unfamiliar faces; and the effect of familiarity can be found at the representational level (in DNNs). Although many questions remain open, it should be evident from this thesis that a computational cognitive approach to face recognition, although technically challenging and on many occasions laborious, will progress our understanding of human face recognition.