ESR12 - Irene Ballester Campos

This project has conducted a state-of-the-art review of AI technologies for dementia care and has established two areas of work in which behavioural modelling using AI-based technologies is applied. The first area of work focused on the development of an algorithm for assistance in daily activities for people with dementia, with a focus on the use of the algorithm. For this, a prototype was developed that compares the activities the user is performing on the toilet using a depth sensor and provides guidance in case deviations from a predefined model are detected. From the very beginning, a collaboration with the DIANA project  was established. Adapting the interaction for users with dementia is a key part of the success of this work, so a literature review in combination with focus groups with healthcare professionals has been carried out in collaboration with ESR15. The outcome of this was published in Pervasive Health 2021. A second phase of this project included experiments to validate the performance of the system, the results of which were published at the AHFE 2022 conference. This work was further validated through extensive real-world data evaluation, which confirmed both the high accuracy of the action recognition models (published in PerCom workshops 2024) and the effectiveness of the developed interaction modalities (published in JAIHC 2024), demonstrating the practical relevance and robustness of the system in real-life settings with people with dementia.

The second area of work was dedicated to the detection of long-term behavioural changes in people with dementia using data from depth sensors as input data. As a first step, a review of the state of the art was carried out together with a preliminary study to explore the possibilities of 3D coordinates as input data for the detection of behavioural changes. Secondly, a taxonomy of behaviours related to cognitive impairment to be measured was defined and a collaboration with the AlgoCare project  was initiated. 

In addition to using depth images as input data, the project explored point clouds derived from depth maps to measure behaviour. State-of-the-art methods were evaluated with real-world data on daily activities performed by people with dementia (published in ICCHP 2024). In this direction, we further advanced the field by proposing a novel method for 3D Human Pose Estimation from point cloud videos (published in ICPR 2024) and explored action recognition from skeleton data extracted from depth videos of daily activities.

The final contributions before submission focused on 4D scene understanding to locate open-set actions in 4D space and exploring continual learning from point cloud videos to handle data imbalances in real-world scenarios, allowing for model retraining while preventing catastrophic forgetting.