Physical gestures may be intrinsically linked to human expression, but linguistics hasn’t always recognized their vital role in producing meaning, seeing them as mere adornments to speech.
Jennifer Hinnell
Linguist Jennifer Hinnell is determined to change that and she’s off to a brilliant start. Her PhD thesis on the topic won this year’s Governor General’s Gold Medal, the University of Alberta’s top doctoral award.
Beyond a better understanding of how gesturing contributes to human conversation, her research has applications in fields from neuroscience to designing animated avatars in video games.
The gist of her argument is that language is “inherently embodied.”
“When people are talking face to face, you can’t strip the body out of the linguistic signal,” she said. “Body actions are linguistic, rather than just a sideshow or add-on to the meaning produced in words.”
Human gesturing is known in linguistics as “co-speech behaviour.” It includes manual gestures, head movements, shoulder shrugs, postural shifts, eye gaze and brow movements linked to certain verbal expressions.
When people converse, they really can’t help getting their body involved, she said, however slightly, and in ways that sometimes “almost fly under the radar.”
An obvious example is a person weighing two options. They might lift one hand, palm out, then the other, but they might just tilt their head slightly from side to side, as if expressing “the internal experience of the contrast,” said Hinnell.
To do her research, Hinnell turned to the Red Hen Lab, a huge database of American broadcast television clips – mostly from news programming and talk shows. All of the clips are closed-captioned, which made machine searching for phrases and expressions far easier, she said.
She explained that because our gesturing to some extent implies our experience of physical forces, it can likely be understood across cultures, regardless of the verbal language spoken.
“There is a fair amount that’s universal, because we all live in a physical embodied world in these human bodies.
“When people express an ongoing event, for example, they draw a big timeline in front of them with their hands, a cyclical movement to show it’s repeated over time, or an abrupt gesture when it ends suddenly.”
Apart from learning more for the sake of science about how gesturing complements language, there are practical applications for Hinnell’s research, such as its connection to neuroscience.
Parkinson’s patients, for example, often lose some control over body movements, so their expression of emotion through hand and facial gestures is reduced.
“If we understand better how gesturing works in the normative population, we can educate and assist both the speaker and the listener when language and the body are not functioning normally,” she said.
Those on the autism spectrum could similarly benefit from her research.
A lexicon of body language could also aid in fleshing out technologies such as virtual agents and improving dialogue for animation in films and video games.
“Labs developing embodied agents are really focusing on the linguistic part, but they’re leaving out what the body needs to be doing for people interacting with the agent to accept the communication as relatively normal.”
As a result, animated characters can be “stiff or doing really random things humans would not do in a given context,” she said.
“On all of these levels – the technological as well as the clinical – my research can support efforts to enable authentic communication.”
In her post-doctoral research at the University of British Columbia, Hinnell plans to look at how post-truth discourse – the tendency to base communication on appeals to emotion rather than fact – and fake news are reinforced by multiple modes of communication, including the physical.
When inquiring about her research, said Hinnell, people often bring up Lie to Me, the popular television series starring Tim Roth that aired between 2009 and 2011. Roth’s character is a “deception expert” who reads facial expressions and body language to determine whether criminal suspects are lying.
It’s an intriguing idea for a police drama but it’s hardly based on science, said Hinnell. There has been relatively little linguistic analysis of body language, so the premise remains fictional, at least for now.
“I think there’s a lot of conjecture; I don’t know that there’s a lot of hard research.”
| By Geoff McMaster
This article was submitted by the University of Alberta’s online publication Folio, a Troy Media content provider partner.
The views, opinions and positions expressed by columnists and contributors are the author’s alone. They do not inherently or expressly reflect the views, opinions and/or positions of our publication.
