Skip to main content
The European High Performance Computing Joint Undertaking (EuroHPC JU)

Voice for purpose - Phase 2: Enabling expressive conversations with patients with permanent voice loss and movement paralysis

50,000
Awarded Resources (in node hours)
Leonardo BOOSTER
System Partition
November 2024 - November 2025
Allocation period

AI Technology: Audio (speech recognition, speech synthesis, etc.) - Generative Language Modeling

The "Voice for Purpose" project represents an innovative approach to enabling communication for individuals with voice and movement impairments, in all European languages. 

Phase 2 of the project focuses on two fronts:

1) enable such vocally impaired patients to communicate using synthetic expressive voices created from short recordings of their own voice or donors' voices, thereby enhancing, in a number of languages, personal identity and fostering emotional connection

2) reduce the conversational effort, which can be very high for patients who lost the control of their movements, by employing sophisticated interactions with state-of-the-art Large Language Models (LLMs) that leverage the context of the conversation in a number of languages.

The project's objectives include the development of scalable, multilingual voice cloning technology, with a strong focus on computational research on the optimal performance for all European languages, and an accurate study of the input simplification methods based on the interaction with an optimized LLM devoted to input prediction. 

Results from Phase 1 of the Voice for Purpose project indicate that the impact of such innovations includes: retention of identity, enhanced social inclusion, improved quality of life for those affected by permanent voice loss and movement impairment. 

This project addresses critical needs in the healthcare and assistive technology sectors, for people affected by ALS and treated with voice box removal due to cancer at the larynx, with potential applications extending to various domains that require voice synthesis and enhanced human-computer interactions.