Cross-modal and synaesthetic perception in music and vision

Abstract

This thesis is concerned with the cross-modal and synaesthetic perception of musical and visual stimuli. Each of these types of perception has been researched separately, and a hypothesis is presented here that accounts for both cross-modal matching and the development of synaesthesia. This hypothesis claims that sensory information can be evaluated in another modality by using a scale of comparison in that modality. The first set of experiments examines normal subjects performing cross-modal matching with coloured circles and auditory stimuli that vary in complexity. It is shown that subjects use a variety of scales of comparison from both visual and auditory modalities to form matches. As the stimuli increase in complexity, the individual variation in cross-modal matching also increases. The second set of experiments examines matching performance using higher order stimuli, by having subjects evaluate fragments of melodies and complete melodies on affective and descriptive adjective scales. Melodies were also matched with landscape scenes to examine if subjects could form matches between two highly complex sets of stimuli. The final experiments examine synaesthetic associations with colour, evoked from music, letters, numbers, and other categorical information. Common features of synaesthesia from a population of synaesthetes are identified, and experiments performed to test the interference of the synaesthetic associations. Additional experiments are presented that explore the superior short-term memory of one synaesthete, and the role of his associations as a mnemonic device.