The DEEP-EST project developed and validated the first multi-module Modular Supercomputing Architecture system (the Cluster/Booster systems of  DEEP and DEEP-ER had just two modules): a 50-node dual-processor Intel® Xeon® Cluster module using InfiniBand EDR, a 75-node Booster with one NVIDIA V100 GPU each coupled to a small x86 host CPU and interconnected by the ASIC implementation of the EXTOLL network, a data analytics module with 16 high-end dual-processor Intel® Xeon® nodes with an Intel® Stratix ® 10 FPGA attached and 3 Terabytes of nonvolatile Intel® OptaneTM Byte-addressable memory. A conventional, disk-based storage module and an innovative, NVMe flash module support low-latency, high-bandwidth parallel I/O via the BeeGFS parallel file system. The Cluster and Booster modules use Megware’s blade and warm water cooling technologies, and Megware-developed power meters report the overall power and energy use of each node with a time resolution of a few milliseconds. The EXTOLL and InfiniBand networks were connected by a gateway-based network federation, and ParaStation MPI provided seamless and transparent communication across the whole prototype.

As exploratory research activities, a next generation prototype of an EXTOLL-attached memory device (NAM) and a novel acceleration device for collective MPI communication (GCE) were created and their functionality was validated. The DEEP-EST prototype continues to be used by the DEEP-SEA projects, and EXTOLL GmbH changed its business model to developing network-related silicon IP blocks.

DEEP-EST project: architecture diagram
DEEP-EST: applications

Together with the lessons learnt from the previous DEEP projects, six complex, real-world HPC applications from the fields of brain simulation , high energy physics event reconstruction , molecular dynamics for life sciences ,  radio astronomy , satellite image analysis  and space weather simulation , provided co-design input for the DEEP-EST protype. The satellite image analysis and the space weather application introduced AI/ML components. The DEEP software stack was adapted and extended to cover he new, multi-module MSA system, and a extgeneration networked attached memory plus an innovative MPI collective communications offload engine were integrated with the resource manager and ParaStation MPI.

DEEP-EST: applications

The DEEP-EST project started on July 1, 2017 and came to a successful conclusion on March 31, 2021. The consortium consisted of sixteen partners from nine European countries, with Forschungszentrum Jülich acting as the coordinator. The project budget was 15.4M €, and the European Union contribution was
15 M€.

The DEEP-EST project has received funding from the European Community’s Horizon 2020 Programme under grant agreement No. 754304.

Project stats

  • Project Acronym: DEEP-EST (Dynamical Exascale Entry Platform – Extreme Scale Technologies)
  • Contract n°: ICT-754304
  • Start Date: 01/07/2017
  • Duration: 45 months
  • Total Budget: 15.365.546 €
  • EC Funding: 14.998.342 €

The DEEP-EST project partners

  • Barcelona Supercomputing Centre (BSC)
  • CERN
  • EPCC
  • Fraunhofer Institute for Industrial Mathematics (ITWM)
  • Forschungszentrum Jülich
  • KULeuven
  • Leibniz Supercomputing Centre
  • Megware
  • National Center for Supercomputing Applications (NCSA)
  • Norwegian University of Life Sciences (NMBU)
  • ParTec AG
  • University of Heidelberg
  • University of Iceland