About the HPC system Raven

Lenovo system with Intel Xeon Icelake CPUs and Nvidia A100 GPUs

Supercomputer Raven was installed in September 2020 and was upgraded in two phases during the first half of 2021. The initial configuration served as an interim system and comprised 516 compute nodes based on the Intel Xeon CascadeLake-AP processor (Xeon Platinum 9242), providing 49,536 CPU-cores, 193 TB RAM, and 3.5 PFlop/s theoretical peak (FP64) in total. 

In April and June 2021 the interim system was replaced in two steps by the final system. The new machine finally comprises 1592 compute nodes with the new Intel Xeon IceLake-SP processor (Platinum 8360Y with 72 cores per node and 256 GB RAM). The theoretical peak performance is 8.8 Pflop/s (FP64). A subset of 64 nodes is equipped with 512 GB RAM and 4 nodes with 2048 GB RAM. In addition, Raven provides 192 GPU-accelerated compute nodes, each with 4 Nvidia A100 GPUs (4 x 40 GB HBM2 memory per node and Nvlink). The theoretical peak performance of the GPU-accelerated system is 16 Pflop/s (FP64, including tensor cores and host CPUs). The  nodes are interconnected with a Mellanox HDR InfiniBand network (100 Gbit/s) using a pruned fat-tree topology with four non-blocking islands (720 CPU nodes with 256 GB RAM, 660 CPU nodes with 256 GB RAM, 192 GPU nodes plus 64 CPU nodes with 512 GB RAM and 4 CPU nodes with 2 TB RAM, 144 CPU nodes with 256 GB RAM). The GPU nodes are interconnected with at least 200 Gbit/s.

In addition there are 4 nodes for login and interactive tests, and an I/O subsystem that serves ca. 7 Peta Bytes of disk storage with direct HSM access (via GHI).

Overall compute-node configuration:

• 1524 CPU compute nodes (Intel Xeon IceLake-SP 8360Y with 72 cores per node and 256 GB RAM per node)
• 64 CPU compute nodes (Intel Xeon IceLake-SP 8360Y with 72 cores per node and 512 GB RAM per node)
• 4 CPU compute nodes (Intel Xeon IceLake-SP 8360Y with 72 cores per node and 2048 GB RAM per node)
• 192 GPU-accelerated compute nodes (2x Intel Xeon IceLake-SP 8360Y with 512 GB RAM per node, 4x Nvidia A100-SXM4 with 160 GB HBM2 per node)