barry_c_smith's picture
Professor & Director, Institute of Philosophy School of Advanced Study University of London
The Senses and the Multi-Sensory

For far too long we have laboured under a faulty conception of the senses. Ask anyone you know how many senses we have and they will probably say five; unless they start talking to you about a sixth sense. But why pick five? What of the sense of balance provided by the vestibular system, telling you whether you are going up or down in a lift, forwards or backwards on a train, or side to side on a boat? What about proprioception that gives you a firm sense of where your limbs are when you close your eyes? What about feeling pain, hot and cold? Are these just part of touch, like feeling velvet or silk? And why think of sensory experiences like seeing, hearing, tasting, touching and smelling as being produced by a single sense?

Contemporary neuroscientists have postulated two visual systems — one responsible for how things look to us, the other for controlling action — that operate independently of one another. The eye may fall for visual illusions but the hand does not, reaching smoothly for a shape that looks larger than it is to the observer.

And it doesn't stop here. There is good reason to think that we have two senses of smell: an external sense of smell, orthonasal olfaction, produced by inhaling, that enables us to detect things in the environment such food, predators or smoke; and internal sense, retronasal olfaction, produced by exhaling, that enables us to detect the quality of what we have just eaten, allowing us to decide whether we want any more or should expel it.

Associated with each sense of smell is a distinct hedonic response. Orthonasal olfaction gives rise to the pleasure of anticipation. Retronasal olfaction gives rise to the pleasure of reward. Anticipation is not always matched by reward. Have you ever noticed how the enticing aromas of freshly brewed coffee are never quite matched by the taste? There is always a little disappointment. Interestingly, the one food where the intensity of orthonsally and retronasally judged aromas match perfectly is chocolate. We get just what we expected, which may explain why chocolate is such a powerful stimulus.

Besides the proliferation of the senses in contemporary neuroscience, another major change is taking place. We used to study the senses in isolation, with the greatest majority of researchers focusing on vision. Things are rapidly changing. We now know that the senses do not operate in isolation, but combine at both early and late stages of processing to produce our rich perceptual experiences of our surroundings. It is almost never the case that our experience presents us with just sights or sounds. We are always enjoying conscious experiences made up of sights and sounds, smells, the feel of our body, the taste in our mouths; and yet these are not presented as separate sensory parcels. We simply take in the rich and complex scene without giving much thought to how the different contributors produce the whole experience.

We give little thought to how smell provides a background to every conscious waking moment. People who lose their sense of smell can be plunged into depression and show less sign of recovery a year later than people who lose their sight. This is because familiar places no longer smell the same, and people no longer have their reassuring olfactory signature. Also, patients who lose their smell believe they have lost their sense of taste. When tested, they acknowledge that that can taste sweet, sour, salt, bitter savoury, and metallic. But everything else, missing from the taste of what they are eating, is due to retronasal smell.

What we call taste is one of the most fascinating case studies for how inaccurate our view of our senses is: it is not produced by the tongue alone but is always an amalgam of taste, touch and smell. Touch contributes to sauces tasting creamy, and other foods tasting chewy, crisp, or stale. The only difference between potato chips, which "taste" fresh or stale, is a difference in texture. The largest part of what we call "taste" is in fact smell in the form of retronasal olfaction, which is why people who lose their ability to smell say they can no longer taste anything. Taste, touch and smell are not merely combined to produce experiences of foods of liquids, rather the information from the separate sensory channels is fused into a unified experience of that we call taste and food scientists call flavour.

Flavour perception is the result of multi-sensory integration of gustatory, olfactory and oral somatosenory information into a single experience whose components we are unable to distinguish. It is one of the most multi-sensory experiences we have and can be influenced by both sight and sound. The colours of wines and the sounds food make when we bite or chew them can have large impacts on our resulting appreciation and assessment, and irritation of the trigeminal nerve in the face will make chillies feel "hot" and menthol feel "cool" in the mouth without any actual change in temperature.

In sensory perception, multi-sensory integration is the rule not the exception. In audition, we don't just hear with our ears, we use our eyes to locate the apparent sources of sounds in the cinema where we "hear" the voices coming from the actors' mouths on the screen although the sounds are coming from the sides of the theatre. This is known as the ventriloquism effect. Similarly, retronasal odours detected by olfactory receptors in the nose are experienced as tastes in the mouth. The sensations get re-located to the mouth because oral sensations of chewing or swallowing capture our attention, making us think these olfactory experiences are occurring in the same place.

Other surprising collaboration among the senses are due to cross-modal effects, where stimulation of one sense boosts activity in another. Looking at someone's lips across a crowded room can improve our ability to hear what they are saying, and the smell of vanilla can make a liquid we sip "taste" sweeter, and less sour. This is why we say vanilla is sweet smelling, although sweet is a taste, and pure vanilla is not sweet at all. Industrial manufacturers know about these effects and exploit them. Certain aromas in shampoos, for example, can make the hair "feel" softer; and red coloured drinks "taste" sweet, while drinks with a light green colour "taste" sour. In many of these interactions vision will dominate; but not in every case

. For anyone unlucky enough to have disturbance in their vestibular system they will feel the world is spinning although cues from the eyes and the body should be telling them everything is still. Instead, the brain goes with the combined picture and vision and proprioception fall in line. Luckily, our senses cooperate and we get us around the world, and the world we inhabit is not a sensory, but a multisensory world.