Skip to main content
Logo
The European High Performance Computing Joint Undertaking (EuroHPC JU)

Holistic Open-Vocabulary Video Anticipation and Forecasting

90,000
Awarded Resources (in node hours)
Leonardo Booster
System Partition
29 July 2024 - 28 July 2025
Allocation Period

This project is conducted as part of the European Research Council, ERC Consolidator Grant 2022 – 2027, "Forecasting and Preventing Human Errors (FORHUE)". 

It aims to advance AI to enhance human performance and prevent errors across various sectors such as automotive, surgical, and manufacturing, where human errors lead to significant financial losses and safety concerns.

Led by the Computer Vision Group (CVG) in Bonn, the project employs a holistic approach involving the development of new architectures for video recognition and anticipation, scaling methods to larger datasets, and leveraging multimodal data for real-world use. 

The objectives include developing tailored architectures for accurate forecasting based on partial observation, optimizing efficiency for real-time applications, scaling methods to larger datasets such as EGO4D, and establishing novel protocols for assessing video forecasting techniques within an open-vocabulary context. 

The expected outcomes include revolutionizing AI tools to reduce incidents and financial losses in various settings, with key innovations including an open vocabulary protocol for error prediction and a specialized architecture for video anticipation optimized for large-scale datasets.

Success in this endeavor could standardize error categorization, improve forecasting accuracy, and pave the way for transformative applications across different fields.