DEEP-SEA: Software for Exascale Architectures

DEEP stands for Dynamical Exascale Entry Platform. Since 2011, the DEEP series of projects (DEEP, DEEP-ER, DEEP-EST) has pioneered an innovative concept known as the Modular Supercomputer Architecture (MSA). This novel HPC architecture paves the way for the next milestone in supercomputing: exascale systems with the power to perform 10^18 Floating Point Operations per second.
The fourth project in the row is DEEP-SEA. It started on April 1, 2021 with a term of 3 years. The project will deliver the programming environment for future European exascale systems, adapting all levels of the software stack to support highly heterogeneous compute and memory configurations. It will also allow code optimisation across existing and future architectures and systems. The software stack includes low-level drivers, computation and communication libraries, resource management, and programming abstractions with associated runtime systems and tools.
@ Copyright: Unsplash / Markus Spiske

Mission and Vision

With DEEP-EST, a prototype was established that leverages the benefits of a Modular Supercomputing Architecture (MSA). Here, different components like standard CPUs, accelerators like GPUs, among others, build a complex mesh of heterogeneous technologies. Each technology is best suited for specific workloads, applications can leverage the parts of the MSA that allow for optimal energy/performance ratio. Large-scale simulations, data analytics, machine- and deep learning – different jobs have different requirements to run at the equilibrium of energy usage and compute performance.
The downside: the complexity on system and node level makes it hard to allocate all resources in the best way. Parallelization is not trivial in traditional HPC environment – with the rising complexity of heterogeneous resources is will be hardly manageable by the users and the application developers. They need to understand their codes and the underlying hardware very well in order to decide on what components parts of the code should run. Also, the need grows to port codes to different technologies such as accelerators or even to completely new platforms. Not to mention the burden of optimization for a variety of HPC systems. DEEP-SEA addresses this challenge.

The main goals of this project are:

  • Better manage and program compute and memory heterogeneity.
  • Targets easier programming for Modular Supercomputers.
To achieve these goals, DEEP-SEA looks at all relevant parts of the system: nodes, clusters, applications, system software and tools. To provide solutions that are usable on as many HPC systems as possible and that scale to exascale and beyond, DEEP-SEA uses the co-design approach that has already proven effective in earlier DEEP projects.

The DEEP projects

From DEEP to DEEP-SEA – a history of the DEEP projects

01

DEEP

The first DEEP project started on December 1, 2011 and came to a successful conclusion on May 31, 2015. The consortium consisted of sixteen partners from eight European countries, with Forschungszentrum Jülich acting as the coordinator.

  • Name: DEEP (Dynamical Exascale Entry Platform)
  • Contract no.: ICT-287530
  • Project type: IP/STREP
  • Start date: 01/12/2011
  • Duration: 45 months
  • Budget: 18,500,000 €
See project details
02

DEEP-ER

The second project, DEEP-ER, started on October 1, 2013 and came to a successful conclusion on March 31, 2017. The consortium consisted of fourteen partners from seven European countries, with Forschungszentrum Jülich acting as the coordinator.

  • Name: DEEP-ER (Dynamical Exascale Entry Platform – Extended Reach)
  • Contract no.: ICT-610476
  • Start date: 01/10/2013
  • Duration: 42 months
  • Total budget: 9,823,190€
See project details
03

DEEP-EST

The DEEP-EST project started on July 1, 2017, and came to a successful conclusion on March 31, 2021. The consortium consisted of sixteen partners from nine European countries, with Forschungszentrum Jülich acting as the coordinator.

  • Name: DEEP-EST (Dynamical Exascale Entry Platform – Extreme Scale Technologies)
  • Contract n°: ICT-754304
  • Start Date: 01/07/2017
  • Duration: 45 months
  • Total Budget: 15.365.546 €
See project details
04

DEEP-SEA

The current project, DEEP-SEA, started on 1 April, 2021, with a term of 3 years.

  • Name: DEEP-SEA (Dynamical Exascale Entry Platform – Software for Exascale Architectures)
  • Contract n°: ICT-955606
  • Start Date: 01/04/2021
  • Duration: 36 months
  • Total Budget: 15,069,476.25 €
See project details