We spoke to Zoë Firth – an Early Career Researcher working on Five Minute Speech Samples and ‘expressed emotion’ – about her background, her project, and how she is disseminating her findings in an engaging and informative way
Tell us about yourself and your journey as an Early Career Researcher – how did you end up working on this project?
Zoë: “I was hired to work on this project in May 2023, where my role would be to transcribe and evaluate the speech data from our project, and work with the rest of the team as they processed this data in the analysis. I’ve worked in mental health before and have an MSc in Clinical Linguistics, which means that I already had experience in working with language data within the context of clinical research questions, which is what I needed to do on the project. I feel lucky that this role came up when it did, as it aligns with my past experience so well!”
What was the background to the project, and what is the challenge that you and your team are trying to address?
Zoë: “We were studying something called the Five Minute Speech Sample, which is when an interviewer asks a parent to talk about their child for five minutes. We know from previous research that information in this speech sample such as ‘expressed emotion’ – which is the type and intensity of emotions a parent expresses about their child – can be good at predicting that child’s mental health in the future. 1 However, having someone transcribe and rate parents’ Five Minute Speech Samples as part of routine children’s health assessments wouldn’t be feasible, as this process is very resource intensive.
"So, in this project, we wanted to assess whether we could build an AI-driven tool that could transcribe and rate parents’ speech in the Five Minute Speech Sample as well as a human can. If so, this type of automated tool could potentially be used by clinicians as one source of data to help identify children and families who may benefit from mental health support.”
We worked with a near-future science fiction writer, Stephen Oram, to write two stories about possible futures of our project - what might it look like if this tool were actually implemented in children's mental health assessment?
Zoë Firth
Early Career Researcher
How did you approach this element of the project?
Zoë: “We were presenting two aspects of our project on this poster. First, we presented how we built the AI-driven speech analysis tool. We were looking at data from a large longitudinal study of twins called the E-Risk study, where Five Minute Speech Samples had been collected when the twins were 10 years old. I transcribed a subset of the Five Minute Speech Sample recordings, which had also been rated for ‘expressed emotion’ by trained researchers. The speech analysis team built and tested different ways of analysing the recordings, looking at different combinations of the things that parents said, and the way they said them. These were used to generate an ‘expressed emotion’ score. The team also constructed Automatic Speech Recognition models to automate the transcription of the audio recordings.
"Second, we also presented the engagement work we did with young people and parents for the project. Given that our tool is at an early stage of development, its applications in a clinical setting are still quite abstract. So, we worked with a near-future science fiction writer Stephen Oram to write two stories about possible futures of our project – what might it look like if this tool were actually implemented into children’s mental health assessments? We then read these two stories in workshops with young people and parents, which stimulated discussions on our project overall.”
And what did you find?
Zoë: “The AI-driven tool yielded some promising results, with decent accuracy at classifying the level of 'expressed emotion' in the speech samples - perhaps not as high accuracy as we had hoped but the figures indicate that it would be productive to develop this tool further in future research projects. We were working with audio recorded on cassette tapes 20 years ago, so researchers using more modern data may be able to improve on these results.
"In terms of the workshops, we found that the parents in particular didn’t necessarily object outright to the use of AI in mental healthcare. Instead, young people and parents were more keen to highlight concerns about how this tool might be implemented in a healthcare system where not everyone gets access to the care they need, and this gap is widening. They also expressed concerns about the ability of one short Five Minute Speech Sample from a parent to accurately capture the nuance and complexity of a child’s personality and the parent-child relationship. Some stressed the importance of ensuring timely access to assessment and treatment for young people over the need to predict mental health outcomes.”
Thinking about how to present these findings, how did you approach producing your poster?
Zoë: “Having learned a lot of new concepts on this project as someone without a computer science background, I could relate to our poster audience in terms of what might be confusing! While the details of how the AI-driven speech analysis tools were developed might have been useful for a more technology-focussed setting, here we tried to present the results on the performance of the tool by highlighting the concepts that people are already familiar with – namely the accuracy, sensitivity, and specificity of diagnostic tools. And of course, it was important to remember that posters – even academic ones – are a visual medium, so I tried to make the poster as visually engaging as possible.”
Was there one key message that you wanted people to remember after looking at your poster?
Zoë: “We often shy away from engaging in discourse around areas of research that we think we don’t fully understand, so I wanted this to make people feel comfortable at least starting to engage with research on AI-driven tools in healthcare, as the use of these types of tools is likely to increase in the coming years. As such, we also wanted people to come away feeling positive that the development of these tools can be regulated in empowering and accountable ways, with constant – and at times surprising! – input and feedback from the public.”
Do you have any final tips for colleagues wanting to share messages from their adolescent mental health research in an engaging and memorable way?
Zoë: “Don’t assume what people will think or like about your research, or that they will find it boring: there are so many aspects of young people’s mental health that resonate with the public. I was surprised with the results of our engagement work; I think I had assumed that people’s views on our tool would be largely negative. Instead, young people and parents brought up so many aspects of the premise of the Five Minute Speech Sample assessment and the utility of predicting mental health outcomes that expanded my understanding of what needs to be done to ensure these assessments are carried out fairly: whether done by a human or an AI-driven tool.
"I’d also encourage other researchers to use creative methods as part of their research. Working with Stephen was incredibly valuable not only for deepening our engagement with the young people and parents we worked with, but also for us as researchers; reading the stories helped us think and talk about our own research, and place it into the broader sociotechnical context of its implementation. Whether through fiction or other disciplines, creative practices are a powerful way of stimulating discussion on complex scientific topics.”
Reference list
Rutter, M. & Brown, G.W., The Reliability and Validity of Measures of Family Life and Relationships in Families Containing a Psychiatric Patient. Social Psychiatry 1, 38–53 (1966).