Innovation - The technologies we use to drive meaningful progress
Designing a data collection app for citizen science
In order to obtain the 600 samples we would need to test and train the app, we needed to design and develop an app that could be easily deployed on the AppStore and PlayStore and downloaded by parents around the country.
The app needed to be able to work offline but to upload the data when online later and work on a range of mobile phones and tablets so that we could have participants from all backgrounds involved.
Instructions needed to be clear for parents and for children and we needed to be able to quickly explain the purpose of the project and how their participation would make a difference!
Designing a data collection app for use by children
We worked with children, parents and clinicians to design an app that could be used to elicit a story retelling sample from children remotely. We needed to make sure that the theme was universally recognised and that boys and girls would be engaged.
​
After lots of work with children, a treasure quest was decided to be fun and internationally appropriate. The main characters of a boy and a dog are familiar and the supporting cast of a tortoise and birds were also identifiable for children.
​
We tested the voices and the sound effects, the rewards and the colours - to make sure the app would be motivating for children starting at four and going right up to 8!
Speech Recognition Technology
Speech recognition technology is used in all aspects of our day to day life now. It is one of the core technologies in use in this project as the child records their story retell into the app and it needs to be transcribed so that the analysis of language can take place.
Transcription improvement tool
After the automated transcription is presented to the clinician, they can edit any errors the speech recogniser has made and make sure it is sorted into separate utterances.
The user interface is designed to make this as quick a task as possible and allows you to listen back to the recording as you go through. Marking up any mispronunciations and mazes also improves the accuracy of the grammatical analysis that the tool will undertake.
Language analysis
Using the confirmed transcription, the project looked at the analysis of linguistic features in the sample.
Acoustic analysis
In addition to looking at the language features of the samples, we also looked at the acoustic properties of the recording. That includes looking at the nature and duration of pauses as well as the balance of talking between the child and the adult.
We can also look at the pitch, loudness and other variables of the child's spoken sample.
Reporting
It will be important to consider how information from the app is presented to users. Feedback will be sought in the clinical evaluation.
Remote integration
Our project started in November 2019 and by the time we were midway through our citizen science work package we knew we needed to make sure that this language analysis tool could be used in remote speech and language therapy sessions using technology to not just mirror a screen, but to really take the therapist into the child's app.
Our clinical evaluation will now include testing the app in both face-to-face and remote sessions, which will be evermore important as we go forward.
Research tools
A group of UCL researchers in the Neurotherapeutics group piloted the transcription improvement tool to support the transcription and analysis of 66 x 2-minute picture description samples. The team calculated that using our software tools would save them 20% of their time when compared to conducting the work manually. The researchers also identified the benefits of the tool beyond time-saving:
* reducing the differences between scorers
* having access to the output of the analysis in a file ready to copy into a group document
* having access to measuring features that take considerable time to do by hand - such as counting the occurrence of verbs or nouns.