Interactions among the Effects of Head Orientation, Emotional Expression, and Physical Attractiveness on Face Preferences

Previous studies have shown that preferences for direct versus averted gaze are modulated by emotional expressions and physical attractiveness. For example, preferences for direct gaze are stronger when judging happy or physically attractive faces than when judging disgusted or physically unattractive faces. Here we show that preferences for front versus three-quarter views of faces, in which gaze direction was always congruent with head orientation, are also modulated by emotional expressions and physical attractiveness; participants demonstrated preferences for front views of faces over three-quarter views of faces when judging the attractiveness of happy, physically attractive individuals, but not when judging the attractiveness of relatively unattractive individuals or those with disgusted expressions. Moreover, further analyses indicated that these interactions did not simply reflect differential perceptions of the intensity of the emotional expressions shown in each condition. Collectively, these findings present novel evidence that the effect of the direction of the attention of others on attractiveness judgments is modulated by cues to the physical attractiveness and emotional state of the depicted individual, potentially reflecting psychological adaptations for efficient allocation of social effort. These data also present the first behavioural evidence that the effect of the direction of the attention of others on attractiveness judgments reflects viewer-referenced, rather than face-referenced, coding and/or processing of gaze direction.


Introduction
Studies of facial attractiveness have typically investigated how physical cues, such as symmetry, averageness, and sexual dimorphism of face shape, influence face preferences (see Rhodes 2006 for a recent meta-analytic review). By contrast with this emphasis on the effects that physical cues have on facial attractiveness, recent studies have also investigated the ways in which the effects of these invariant physical cues and those of changeable social signals, such as gaze direction and expression, might interact when judging the attractiveness of others (eg Conway et al 2008a;Jones et al 2006;Kampe et al 2001;O'Doherty et al 2003).
Neuroimaging studies have shown that both facial expressions and gaze direction modulate the extent to which physically attractive faces elicit greater activation in neural mechanisms implicated in the processing of rewards than do relatively unattractive faces (Kampe et al 2001;O'Doherty et al 2003). For example, direct gaze (compared with averted gaze) increases the reward value of physically attractive faces, but decreases the reward value of physically unattractive faces (Kampe et al 2001). Similarly, the effect of physical attractiveness on the reward value of faces is significantly greater when the faces shown are smiling at the viewer than when the faces are shown with a neutral expression (O'Doherty et al 2003). More recent behavioural studies have also presented evidence that the effects of physical attractiveness, emotional expressions, and gaze direction interact when subjects are asked to judge the facial attractiveness of others. For example, Jones et al (2006) demonstrated that preferences for physically attractive colour and texture cues in faces are greater when the faces presented are smiling at Interactions among the effects of head orientation, emotional expression, and physical attractiveness on face preferences the viewer than when the faces presented are smiling away from the viewer or are shown with a neutral expression. In a similar vein, studies have shown that participants generally prefer faces with direct gaze to those with averted gaze (Mason et al 2005;Conway et al 2008a), but that these preferences are stronger when judging oppositesex than own-sex faces (Mason et al 2005;Conway et al 2008a), physically attractive faces than physically unattractive faces (Conway et al 2008b), and faces with happy expressions than faces with disgusted expressions (Conway et al 2008a). Integrating information about the sex and physical attractiveness of others with information about the direction and valence of their social interest in this way may serve an important function during social interactions by promoting efficient and effective allocation of social effort (ie by promoting allocation of more social effort to the most attractive individuals who appear willing to reciprocate öConway et al Jones et al 2006;Mason et al 2005;O'Doherty et al 2003).
While several recent studies have demonstrated interactions among various physical cues in faces that are relatively invariant over time (eg sex of face, physical attractiveness) and various social signals that can change rapidly during social interactions (eg gaze direction, emotional expressions), surprisingly little is known about the psychological processes that underpin these interactions. For example, although many researchers have suggested that the effects of gaze direction on attractiveness judgments occur because direct gaze signals social interest in the viewer (Conway et al 2008a(Conway et al , 2008bJones et al 2006;Kampe et al 2001;Mason et al 2005), this interpretation assumes that gaze direction is coded and processed relative to the viewer (viewerreferenced coding of gaze, ie the person is looking at me versus not looking at me), rather than being coded and processed relative to the individual being judged (face-referenced coding of gaze, ie the person shown is looking straight ahead versus looking left or right). While interpretations of previous findings for gaze preferences have assumed that viewer-referenced coding and/or processing of gaze direction plays an important role in attractiveness judgments, it is equally plausible that these findings reflect face-referenced, rather than viewer-referenced, coding and/or processing of gaze direction. For example, face-referenced gaze direction may be a cue to the depicted individual's emotional state, since people tend to shift their gaze away from a neutral, direct position when they experience strong negative emotions (eg disdainöGivens 1978). In their fMRI study, Kampe et al (2001) showed that the effects of gaze direction on the reward value of faces occurred irrespective of whether gaze direction and head orientation were congruent or incongruent, consistent with the proposal that viewer-referenced coding and/or processing of gaze direction is critical for the effects of gaze direction on attraction. However, there is no behavioural evidence that viewer-referenced, rather than face-referenced, coding of gaze direction is critical for attractiveness judgments. Previous behavioural studies of gaze preferences have, without exception, tested perceptions of front views of faces only (Conway et al 2008a(Conway et al , 2008bJones et al 2006;Mason et al 2005), meaning that viewer-referenced and face-referenced gaze directions were fully confounded in the face stimuli employed. Thus, testing whether gaze preferences are best explained by viewer-referenced or face-referenced coding and/or processing of gaze direction would test a critical assumption of previous explanations of the effect of the direction of the attention of others on attraction.
A second psychological process that may be important for interactions between the effects of physical and social cues when judging the attractiveness of others is the extent to which physical attractiveness may modulate the perceived intensity of facial expressions. It is possible that attractiveness-contingent preferences for perceiver-directed smiles (eg Conway et al 2008b; see also Conway et al 2008a for analogous sex-contingent preferences for perceiver-directed smiles) simply occur because positive emotions are perceived as more intense in physically attractive faces. For example, people may perceive attractive smiling individuals to be happier than unattractive smiling individuals because of the well-established tendency to automatically ascribe more positive traits to attractive people (ie the halo effectöFeingold 1992). Although there is evidence that physical aspects of faces (eg structural cues to maturity) can modulate the perceived intensity of emotional expressions (Sacco and Hugenberg 2009), it is not known whether physical attractiveness affects the perceived intensity of facial expressions in a similar way. Thus, establishing whether attractiveness-contingent preferences for perceiver-directed smiles can be explained simply by attractiveness-contingent perceptions of emotional intensity would test a possible psychological mechanism that contributes to preferences for direct gaze.
In light of the above, we investigated viewers' preferences for front versus threequarter views of faces in which gaze direction was congruent with head orientation (ie the individuals depicted were always looking directly ahead relative to their own face) under four different conditions: when the faces presented were physically attractive with smiling expressions, physically attractive with disgusted expressions, relatively unattractive with smiling expressions, and relatively unattractive with disgusted expressions. Because our direct and averted gaze stimuli differ in viewer-referenced gaze direction, but are matched in terms of face-referenced gaze direction (ie the individuals depicted were always looking directly ahead relative to their own face), our design allows us to test for effects of viewer-referenced gaze direction that cannot be due to face-referenced coding and/or processing of gaze cues. If viewer-referenced coding and/or processing of gaze direction is critical for the effects of gaze on attractiveness, as many researchers have assumed (eg Conway et al 2008aConway et al , 2008bJones et al 2006;Mason et al 2005), one would expect interactions among head orientation, expression and physical attractiveness that are similar to those observed in studies where gaze direction was manipulated in front views of faces (eg Conway et al 2008aConway et al , 2008bJones et al 2006). Specifically, front views (ie views in which social attention is directed towards the viewer) should be perceived as more attractive than three-quarter views (ie views in which social attention is directed away from the viewer), particularly when judging physically attractive faces with smiling expressions. However, if face-referenced coding and/or processing of gaze direction is critical for the effects of gaze direction on attractiveness, one would expect to observe no differences in participants' preferences for front versus three-quarter views, since both views display direct gaze relative to the stimulus face.
In addition to the tests described in the previous paragraph, we also investigated the manner in which head orientation, expression, and physical attractiveness interact when viewers judge the intensity of the emotions shown. If the attractiveness-contingent preferences for perceiver-directed smiles reported in previous studies (eg Conway et al 2008b; see also Conway et al 2008a andJones et al 2006) occur simply because viewers perceive attractive smiling individuals to be particularly happy, we would expect to observe the same pattern of results for both judgments of emotional intensity and attractiveness. However, if the effects of physical attractiveness on preferences for perceiver-directed smiles are not simply due to attractiveness-contingent perceptions of emotional intensity, we would expect to observe different patterns of results for judgments of emotional intensity and attractiveness.
Many of the previous studies that have demonstrated interactions between invariant physical cues and changeable social signals when judging the attractiveness of others have demonstrated these effects using only female face images (Conway et al 2008b;Jones et al 2006;Mason et al 2005). By contrast with this emphasis on female attractiveness in previous research, in the current study we tested for comparable interactions when participants judged the attractiveness of male faces.

Stimuli
First, we randomly selected 30 men from the Karolinska Directed Emotional Faces (KDEF) image set (Lundqvist and Litton 1998). Twenty women (mean age 23.02 years, SD 7X60 years) rated the physical attractiveness of each of these 30 men using a 1 (very unattractive) to 7 (very attractive) scale. All men were shown with neutral expressions and in front view. The order in which images were presented was fully randomised. Since inter-rater agreement for these ratings was high (Cronbach's a 0X90), we calculated the average attractiveness rating given to each male identity. These mean attractiveness ratings were used to identify the 6 most attractive males (mean attractiveness rating 3.99, SD 0X38) and the 6 least attractive males (mean attractiveness rating 1.70, SD 0X17).
Next, we selected front-view images with happy and disgusted expressions for each of the 6 most attractive males and the 6 least attractive males. We also selected three-quarter left view images with happy and disgusted expressions for each of these 12 males. Corresponding three-quarter right view images with happy and disgusted expressions were manufactured by mirror flipping each of the three-quarter left view images around their central vertical axis. It was these images of physically attractive and relatively physically unattractive male faces shown in front and three-quarter views with happy and disgusted expressions that were used as stimuli in our experiment. Examples of these stimuli are shown in figure 1.

Procedure
The method we used to assess perceptions of front versus three-quarter views is adapted from the methods used to assess face preferences in many previous studies (eg Conway et al 2008a;DeBruine et al 2006;Jones et al 2005Jones et al , 2006Jones et al , 2007Welling et al 2007Welling et al , 2008aWelling et al , 2008b. Participants (N 292, 148 female, 144 male; mean age 24.53 years, SD 10X61 years) viewed pairs of faces that differed in head orientation (front versus three-quarter view), and were matched in terms of identity, attractiveness, and expression. Participants were randomly allocated to one of two conditions. In one condition (attractiveness judgments), participants (N 147, 73 female, 74 male) were instructed to choose the face that they thought was the more attractive in each pair. In the other condition (emotional intensity judgments), participants (N 145, 75 female, 70 male) were instructed to choose the face that was showing the most intense (ie strongest) emotional expression. The pairs of faces remained on screen until a response was made. Both the order in which the pairs of faces were presented and side of the screen on which any particular image was shown were fully randomised.

Initial processing of data
For each participant who made attractiveness judgments, we calculated the proportion of trials on which they chose the front view as the more attractive face when judging attractive faces with happy expressions, unattractive faces with happy expressions, attractive faces with disgusted expressions, and unattractive faces with disgusted expressions. Thus, four scores in total were calculated for each participant who had made attractiveness judgments.
For each participant who judged the emotional intensity of the facial expressions shown, we calculated the proportion of trials on which they chose the front view as showing the more intense emotional expression when judging attractive faces with happy expressions, unattractive faces with happy expressions, attractive faces with disgusted expressions, and unattractive faces with disgusted expressions. Thus, four scores in total were calculated for each participant who had made intensity judgments.

Main analysis
Responses were first analysed with a mixed-design ANOVA [dependent variable: proportion of trials on which front view was chosen; within-subjects factors: attractiveness of face judged (attractive, unattractive), expression of face judged (happy, disgusted); between-subjects factors: judgment type (attractiveness, intensity of expression), sex of judge (female, male)]. This analysis revealed a significant main effect of attractiveness of face judged (F 1 288 37X86, p 5 0X001, partial Z 2 0X116), whereby participants were more likely to choose front views when judging attractive faces (M 0X60, SEM 0X11) than when judging relatively unattractive faces (M 0X55, SEM 0X10). There was also a significant main effect of expression of face judged (F 1 288 42X48, p 5 0X001, partial Z 2 0X129), whereby participants were more likely to choose front views when judging happy faces (M 0X60, SEM 0X11) than when judging , , disgusted faces (M 0X54, SEM 0X11). Additionally, the main effect of judgment type was significant (F 1 288 107X02, p 5 0X001, partial Z 2 0X271), indicating that participants were more likely to choose the front views when judging the emotional intensity of the faces (M 0X67, SEM 0X14) than when judging the attractiveness of the faces (M 0X47, SEM 0X13). A main effect of sex of judge, whereby women tended to choose front views of faces more often than men did (women: M 0X59, SEM 0X13; men: M 0X55, SEM 0X14), approached significance (F 1 288 3X67, p 0X056, partial Z 2 0X013). The mixed-design ANOVA also revealed significant interactions between attractiveness of face judged and judgment type (F 1 288 46X42, p 5 0X001, partial Z 2 0X139) and between expression of face judged and judgment type (F 1 288 6X86, p 0X009, partial Z 2 0X023).
All of these main effects and interactions, apart from the main effect of sex of judge, were qualified by a three-way interaction among attractiveness of face judged, expression of face judged, and judgment type (F 1 288 7X93, p 0X005, partial Z 2 0X027ösee figure 2). There were no other significant effects (all Fs 5 1X60, all ps 4 0X20).
To interpret the significant three-way interaction among attractiveness of face judged, expression of face judged, and judgment type, we conducted separate mixed design ANOVAs for attractiveness judgments and judgments of emotional intensity.

Attractiveness judgments
Analysing participants' attractiveness judgments with a mixed-design ANOVA [dependent variable: proportion of trials on which front view was chosen as the more attractive; within-subjects factors: attractiveness of face judged (attractive, unattractive), expression of face judged (happy, disgusted); between-subjects factor: sex of judge (female, male)] revealed significant main effects of expression of face judged (F 1 145 32X07, p 4 0X001, partial Z 2 0X181), and attractiveness of face judged (F 1 145 58X93, p 4 0X001, partial Z 2 0X289). These main effects were qualified by the predicted interaction between expression of face judged and attractiveness of face judged (F 1 145 6X97, p 0X009, partial Z 2 0X046, see figure 2a).
Planned comparisons with paired-samples t-tests showed that participants were more likely to choose the front views when judging the attractiveness of attractive happy faces (M 0X58, SEM 0X19) than when judging the attractiveness of unattractive happy faces (M 0X45, SEM 0X17; t 146 7X22, p 5 0X001, Cohen's d 0X60) and were also more likely to choose front views when judging the attractiveness of attractive disgusted faces (M 0X46, SEM 0X19) than when judging the attractiveness of unattractive disgusted faces (M 0X38, SEM 0X17; t 146 4X90, p 5 0X001, Cohen's d 0X35). Note that the interaction between expression of face judged and attractiveness of face judged indicates that the effect of attractiveness on preferences for front view versus three-quarter views was significantly greater for happy than disgusted faces (see figure 2a). One-sample t-tests, comparing the proportion of trials where participants chose the front view as the more attractive with the chance value of 0.5, showed that participants preferred front views to three-quarter views when judging attractive happy faces (t 146 4X43, p 5 0X001, Cohen's d 0X37, M 0X58, SEM 0X19), but preferred three-quarter to front views when judging unattractive happy faces (t 146 À2X81, p 0X006, Cohen's d 0X23, M 0X45, SEM 0X17). Corresponding analyses for judgments of disgusted faces showed that participants preferred three-quarter views to front views when judging unattractive disgusted faces (t 146 À6X68, p 5 0X001, Cohen's d 0X55, M 0X38, SEM 0X17), and also showed that this preference for three-quarter views over front views approached significance when judging attractive disgusted faces (t 146 À1X92, p 0X056, Cohen's d 0X16, M 0X46, SEM 0X19).

Emotional intensity judgments
Judgments of emotional intensity were analysed in the same way as attractiveness judgments [dependent variable: proportion of trials on which front view was chosen as showing the more intense emotion; within-subjects factors: attractiveness of face judged (attractive, unattractive), expression of face judged (happy, disgusted); between-subjects factor: sex of judge (female, male)]. This analysis revealed a significant main effect of sex of judge (F 1 143 5X39, p 0X022, partial Z 2 0X036), whereby women were more likely to choose front views as showing the more intense emotion than men were (women: M 0X70, SEM 0X18; men: M 0X64, SEM 0X19). There were no other significant effects (all Fs 5 2X55, all ps 4 0X11).
One-sample t-tests, comparing the proportion of trials where participants chose the front view as displaying the more intense emotion with the chance value of 0.5, showed that participants perceived the front views as displaying more intense emotions than the three-quarter views in all four conditions (all ts 4 9X04, all ps 5 0X001, all Cohen's ds 4 0X75).

Discussion
As in previous research (eg Conway et al 2008a;Jones et al 2006), physical attractiveness and facial expression modulated participants' preferences for perceived-directed versus other-directed gaze. Participants preferred front views of faces (ie perceiverdirected social interest) to three-quarter views (ie other-directed social interest) when judging happy physically attractive faces, but preferred three-quarter views to front views when judging happy physically unattractive faces and when judging faces with disgusted expressions. Collectively, these findings show that preferences for perceiverdirected attention, as indicated by head orientation, are sensitive to both the emotional state and physical attractiveness of the depicted individual. As in previous research on gaze preferences (eg Conway et al 2008aConway et al , 2008b, preferences for perceiver-directed attention were strongest when judging smiling, physically attractive individuals. In the current study, stimulus faces differed in head orientation but were always depicted with direct gaze relative to their own face, meaning that viewer-referenced gaze direction differed between face pairs while face-referenced gaze direction did not. By contrast, previous behavioural studies of gaze direction on face preferences (eg Conway et al 2008aConway et al , 2008bJones et al 2006;Mason et al 2005) used front views of faces only and, therefore, confounded face-referenced and viewer-referenced gaze direction. If the previously reported effects of attractiveness and expression on preferences for direct versus averted gaze in front views of faces were to generalise to preferences for front versus three-quarter views of faces, this would suggest that preferences , for perceiver-directed versus other-directed cues of social interest primarily reflect viewer-referenced, rather than face-referenced, coding and/or processing of gaze. Consistent with this proposal, we found that preferences for front versus three-quarter views of faces were sensitive to expression and attractiveness in the same way as preferences for direct versus averted gaze in front views of faces are (see eg Conway et al 2008b). Thus, our results support an important role for viewer-referenced coding of the direction of the attention of others in social perception of faces. These findings are consistent with those of fMRI studies that tested for independent effects of gaze direction and head orientation on the reward value of attractive faces, which found that direct gaze increased the reward value of attractive, but not unattractive, faces irrespective of head orientation (Kampe et al 2001), and represent the first behavioural evidence that the effect of the direction of the attention of others on face preferences primarily reflects viewer-referenced, rather than face-referenced, coding and/or processing of gaze cues.
In addition to the above, we also investigated whether attractiveness-contingent preferences for perceiver-directed smiles simply reflect a tendency to perceive positive emotional expressions in attractive faces as being more intense than the same expressions in relatively unattractive faces. To do so, we compared the effect of attractiveness on preferences for perceiver-versus other-directed smiles with the effect of attractiveness on the perceived intensity of perceiver-versus other-directed smiles. While participants expressed greater preferences for perceiver-directed smiles from attractive individuals than from unattractive individuals, physical attractiveness had no effect on the perceived intensity of perceiver-versus other-directed smiles. That attractiveness modulates preferences for perceiver-directed smiles, but not judgments of emotional intensity, shows, for the first time, that the effect of physical attractiveness on preferences for perceiverdirected smiles is not simply a byproduct of attractiveness-contingent perceptions of emotional intensity. Adams and Kleck (2003) have previously reported that direct gaze (compared to averted gaze) facilitates classification of the approach-oriented emotional expressions anger and happiness, but impairs classification of the avoidance-oriented emotional expressions fear and sadness (see also Kleck 2005 andHess et al 2007). In light of these findings, Adams and Kleck (2005) proposed a theory of the role of gaze direction in expression perception whereby direct gaze increases attributions of approach-oriented emotions (eg anger, happiness) while averted gaze increases attributions of avoidance-oriented emotions (eg sadness, fear). More recently, however, Bindemann et al (2008) were unable to replicate Adams and Kleck's findings. Our findings for the effects of head orientation on the perceived intensity of facial expressions of emotion are also difficult to reconcile with Adams and Kleck's proposal; we found that front views of faces were perceived as showing more intense emotions than three-quarter views of faces regardless of whether subjects were judging how happy faces displaying the approach-oriented emotion happiness appeared to be or were judging how disgusted faces showing the avoidance-oriented emotion disgust appeared to be. Like Bindemann et al (2008), we found little evidence that gaze direction has different effects on perceptions of approach-and avoidance-oriented emotions.
As in some previous research on preferences for perceiver-versus other-directed social interest (Conway et al 2008b;Jones et al 2006), preferences for perceiver-directed social interest were equivalent for judgments of own-and opposite-sex individuals (see also Kampe et al 2001;O'Doherty et al 2003). However, other studies have found that preferences for perceiver-directed social attention were stronger when judging opposite-sex faces (Conway et al 2008a;Mason et al 2005). Individual differences in preferences for facial cues associated with positive social interest have been reported that are related to variation in observers' anxiety levels (Conway et al 2008b) and partnership status (Conway et al 2009). Thus, it is possible that such individual differences mask opposite-sex biases in attractiveness judgments. Further research is needed to investigate this issue.
Our findings show that attractiveness and emotional expression modulate preferences for the direction of the social attention of others, such that preferences for perceiverdirected social attention are stronger when judging the attractiveness of smiling, physically attractive individuals than when judging the attractiveness of smiling, physically unattractive individuals or individuals displaying disgusted expressions. Moreover, our findings show, for the first time, that attractiveness-contingent preferences for perceiver-directed smiles reflect viewer-referenced coding and/or processing of the gaze direction of others and are not simply a byproduct of a possible tendency to perceive smiles in attractive individuals as more intense than those in relatively unattractive individuals. Collectively, these findings support the proposal that attractiveness-and expression-contingent preferences for perceiver-directed social attention may reflect mechanisms that promote efficient allocation of social effort (ie allocation of more social effort to attractive individuals who appear willing to reciprocate öConway et al Jones et al 2006;Mason et al 2005;O'Doherty et al 2003). Moreover, the interaction among judgment type, facial expression, physical attractiveness, and direction of attention observed in our study highlights the complex integrative processes that underpin social perception of faces.