Scenario 1: Healing with multiple times

Research questions addressed: How can multi-time EnTimeMent technologies improve everyday life communicative and prosocial skills in severely disabled persons with low or no verbal capabilities?

Narrative description: Giorgio is a 4 year-old child with a severe case of dystonic cerebral palsy. As a consequence, he is not able to control his movements, feed himself or effectively communicate with others, which threatens learning and participation. His mom Francesca struggles with her child’s condition which she hasn’t still fully accepted. Therapists know the situation is extremely complex, with the family living far from the hospital, in an area where social assistance and services are less effective.

In cases as severe as Giorgio’s, in which a complex rehabilitation project takes place as the person is still growing up, the healing process becomes a life project, with multiple aims and goals unfolding in multiple temporal dimensions. The rehabilitation team has leveraged a “home-care-unit” that affords the family to live autonomously while having Giorgio monitored by the EnTimeMent technology , and allows to read Giorgio’s body communicative signals at multiple temporal scales, in order to:

at a lower temporal scale, enhance movement control by interpreting bodily sensorimotor and muscular signals to regulate and better tune care and assistance; Improve emotion regulation by decoding affective bodily signals and providing most appropriate responses and care.

at a middle temporal scale, foster attention and motivation by interpreting bodily signals of vigilance, focus and interest; Improve control of bodily movements while learning daily-life processes such as drinking; Support emotion regulation by interpreting residual bodily intentional communication

at a higher temporal, foster planning and control of complex activities such as play and social interaction by decoding participative signals; Enhance metacognitive processes by reading resilience signals.

Scenario 2: Chronic Musculoskeletal Pain Management with multiple times

Research question addressed: Can the modeling and enhancement of diverse temporal-scales of movement  in chronic musculoskeletal pain facilitate self-management and functional activity?

Narrative description: Since developing chronic back pain, Paul’s attention is constantly on pain and on fearing increased pain or damage. He no longer understands what his body can and cannot do, or learn from experiencing movement. When he tackles physical tasks, his concern is about pain so he rushes them, feels no satisfaction at completing them and often feels defeated by increased pain. His central nervous system is ‘stuck’ on pain. The newly released EnTimeMent motion capture system captures his movements at different temporal-scales through the day and sonifies it. Each temporal-scale of movement carries particular meaning. Low-level temporal dynamics of Paul’s taking a book from a shelf tells him how far he stretches and how much more he could move, constraining anxiety. Longer temporal aspects of his movement, outside his awareness, represent task achievement (e.g. washing his car) and remind him to take regular breaks before pain builds up. Even longer temporal dynamics, over weeks and months, show him progress in his capabilities, while increased fluidity of his movement reflects reduced anxiety and increased confidence. This progress sustains him during setbacks and higher pain days, as it reminds him how to build activity again, and his central nervous system is processing normal messages about movement. We will investigate new computational models that capture these different temporal dynamics of movement and the multiple temporal phenomena characterising them and their interactions (kinematics, muscle activity, respirations, brain activation) and transform them into sound enriched by metaphorical musical structures that through embodiment enhances the awareness of movement capability, and provides a path for taking back control from pain. Such computational models will also aim to model even lower temporal dynamics, capturing protective and avoidant movement responses (habits outside awareness) at physiological and neural levels that maintain pain. We will aim at developing new metrics of progress (e.g. individual motor signature as in Słowiński et al. 2016) based on these different temporal-scales.  Such computational models can inform the design of rehabilitation for chronic pain self-management supported by the novel multi-temporal multi-modal EnTimeMent motion capture technology. We will also explore how it can be used to facilitate communication between the person with chronic pain and her home-carer and healthcare staff to facilitate learning of management skills.

Scenario 3:  Dancing with multiples times

Research questions addressed: How are individual and group motor signatures of dancers characterized at multiple time scales during improvisation among them and with non-anthropomorphic partners?

Narrative description: A group of dancers engaged in a performance with non-anthropomorphic dancing partners perform joint improvisation sequences, playing with social roles (leader, follower, improvising), emotional gesture qualities (hesitant, aggressive etc.) and physical/ghost presence of their dancing partners. Dancers perform in an architectural space, which can be interactively manipulated by sound (interactive movement sonification, to enhance the communication at multiple temporal scales) or light (e.g., in the light or in the dark), and which can even be structurally transformed by means of interactive mobile scenery (e.g. a theatre stage technologically equipped simulating a future living architecture). Using the EnTimeMent motion analysis system, biological and social signals are recorded at multiples time scales, from milliseconds (e.g., EMG) to seconds (e.g., MOCAP) to minutes (e.g., breathing), allowing the computation of Individual Motor Signatures of each dancer (IMS), Group Motor Signature (GMS), together with their dynamics at their respective temporal scales, as well as entrainment, saliency, and prediction of qualities. Applying the new Curiosity-based EnTimeMent Learning Algorithm (CELA), relevant individual and social performance qualities are extracted simultaneously. Concrete instances of this group interaction scenario will be developed, from dance to sports, fitness, wellness, and entertainment group activities.

EnTimeMent project objectives...

EnTimeMent project objectives description...

EnTimeMent measurable objectives can be summarised as follows:

EnTimeMent project scientific objectives...

EnTimeMent project technological objectives...

EnTimeMent project community-building objectives...

EnTimeMent project scenarios...