eFlows4HPC

Today, developers lack tools that enable the development of complex workflows involving HPC simulation and modelling with data analytics (DA) and machine learning (ML). The eFlows4HPC project aims to deliver a workflow software stack and an additional set of services to enable the integration of HPC simulation and modelling with big data analytics and machine learning in scientific and industrial applications.   

The software stack will allow to develop innovative adaptive workflows that efficiently use the computing resources and also innovative data management solutions. In order to attract first-time users of HPC systems, the project will provide HPC Workflows as a Service (HPCWaaS), an environment for sharing, reusing, deploying and executing existing workflows on HPC systems. The workflow technologies, which use with machine learning and big data libraries, leverages previous open-source European initiatives. Specific optimizations of application for the use of accelerators (FPGAs, GPUs) and the processors developed by the European Processor Initiative (EPI) will be performed.

To demonstrate the workflow software stack, use cases from three thematic pillars have been selected:

• Pillar I focuses on the construction of DigitalTwins for the prototyping of complex manufactured objects integrating state-of-the-art adaptive solvers with machine learning and data-mining, contributing to the Industry 4.0 vision. 
• Pillar II develops innovative adaptive workflows for climate and the study of Tropical Cyclones (TC) in the context of the CMIP6 experiment, including in-situ analytics. 
• Pillar III explores the modelling of natural catastrophes - in particular, earthquakes and their associated tsunamis shortly after such an event is recorded. Leveraging two existing workflows, Pillar will work on integrating them with the eFlows4HPC software stack and on producing policies for urgent access to supercomputers. The results from the three Pillars will be demonstrated to the target community CoEs to foster adoption and receive feedback.

Cordis Factsheet