ľ�ϸ���Ӱ��

Predicting whether a child is experiencing internalising or externalising problems based on a video. Niilo Valtakari is working on this: "My research shows how machine learning combined with psychology can provide new insights into behaviour." With a background in psychology, he made the switch to the Faculty of Science. In his research, he is training a classification system that can recognise anxiety symptoms based on video.

What is your research about?

"The aim of my project was to automatically assess children for problem behaviour based on videos. This includes externalising symptoms, which are behaviours that are more outwardly focused, such as angry outbursts. But it also includes internalising symptoms, which involve more inward-focused behaviours such as worrying or feeling down. I worked with a YOUth dataset in which children and parents communicated with each other via screens in the same room. I had video recordings of their faces, audio recordings, and recordings of their gaze behaviour (eye-tracking data). The children and their parents were asked to discuss something they had recently disagreed about. Alternatively, to have a conversation that encourages cooperation, such as planning a birthday party.

How exactly do you analyse those videos?

"You can often tell whether someone is angry, happy or sad by looking at their face. Researchers aimed to understand these expressions more precisely and created a taxonomy system to describe every small facial action in the 1970s. With facial behaviour analysis software like PyAFAR and OpenFace, it was possible to not only detect facial actions but also head movements directly from the video recordings.

We considered a number of behaviours to be important, such as how much the children and parents moved their heads, how much they looked at each other, and how much they smiled throughout the video. We also examined some more complex behaviours, such as whether children looked away after smiling. We then combined these behavioural measures together with symptom scores from a questionnaire and fed this information into a machine learning classifier. The questionnaire focused on the internalising and externalising problem behaviours exhibited by the child, such as anxiety symptoms and expressions of aggression, respectively. The aim of our research was to train the classifier to predict problem behaviour as measured by the questionnaire.

We first used three-quarters of the data to train the system. With the remaining quarter, it was possible to test how well the classification system was able to predict the children’s symptom scores.”

What did you find?

“We found that this did not work for all the symptom scores we tried to predict, but it did work for anxiety scores. Once we knew that anxiety scores were predicted best, we zoomed in on the specific behaviours that contributed most to classifier performance. We found a set of specific behaviours that were highly important for predicting anxiety scores. The behaviour that contributed most was sincere smiles made by the parent while their child did not smile back. Other important behaviours were child head shakes, looks to the face of the child made by the parent, and looks to the face of the parent made by the child."

Why is it important to conduct research into this?

Because it shows how you can combine these machine learning methods with psychology to gain a better understanding of behaviour. We are learning to better understand the interactions between parents and children. Assessing a diagnosis takes a lot of time, and this could potentially help the process. We cannot yet use it to classify anxious children, but it is a step in the right direction. There is also still a lot of uncertainty about mental health issues in children: the symptoms they display are often different from those of adults. We hope to gain more information about how anxiety manifests itself in children's behaviour."