Hardware Overview

The FPGA infrastructure consists of FPGA cards included into the Otus and Noctua 2 supercomputers.

A technical description of the FPGA partitions with more details can be found in the Otus paper and Noctua 2 paper.

Below you can find overview tables for both systems

Otus Pilot Phase

	Otus Pilot Phase	Noctua 2

	Otus Pilot Phase	Noctua 2
FPGA Cards per Node	1x AMD/Xilinx Alveo V80	3x AMD/Xilinx Alveo U280	2x BittWare 520N
Card Installation Year	2025	2022	2018
Number of Nodes	3 out of 32¹	16	16
¹ 32 nodes are available for the FPGA partition of Otus. Currently 3 are equipped with FPGA cards.
Host Configuration
CPUs	2x AMD EPYC 9655 (Turin)	2x AMD EPYC 7713 (Milan)
Main Memory	768 GiB	512 GiB
Local Storage	960 GB NVMe	480 GB SSD

Card Specifications
Host Interface¹	2x PCIe Gen5 x8	PCIe Gen3 x16	PCIe Gen3 x8
Host Interface Bandwidth	31.5 GB/s	15.8 GB/s	7.9 GB/s
High-Bandwidth Memory	32 GB HBM2E (2x 16 GB stacks)	8 GB HBM2 (2x 4 GB stacks)	-
HBM Bandwidth	820 GB/s	460 GB/s	-
DDR Memory	1x 32 GB DDR4	2x 16 GB DDR4	4x 8 GB DDR4
DDR Bandwidth	25.6 GB/s	38.4 GB/s	76.8 GB/s
Network Interfaces¹	4x QSFP56 (200 Gbit/s)²	2x QSFP28 (100 Gbit/s)	4x QSFP+ (40 Gbit/s)
Thermal Design Power³	190 W	225 W	225 W
¹ Effectively usable values, implemented by the FPGA shell and used in our system. ² Each port supports 2x 100 Gbit/s or 4x 10/25/40/50 Gbit/s. ³ The TDP does not reflect the actual power consumption for typical workloads (see Section 7.1 for details).

Noctua 2

The Noctua 2 FPGA infrastructure consists of 36 nodes in the fpga partition and 3 heterogeneous accelerator nodes in the hacc partition. Note that not only the dedicated hacc-nodes, but all nodes with AMD/Xilinx FPGAs are accessible as part of the Heterogeneous Accelerated Compute Clusters (HACC) program via a small project proposal for FPGA researchers worldwide.

	Xilinx Alveo U280 Nodes	Intel Stratix 10 Nodes	Custom Configuration Nodes	HACC Nodes

	Xilinx Alveo U280 Nodes	Intel Stratix 10 Nodes	Custom Configuration Nodes	HACC Nodes
Number of Nodes	16	16	4	3

Accelerator Cards	3x Xilinx Alveo U280 cards	2x Bittware 520N cards		2x Xilinx Alveo U55C cards 2x Xilinx VCK5000 Versal development cards 4x AMD Instinct MI210 GPU cards
FPGA Types	Xilinx UltraScale+ FPGA (XCU280, 3 SLRs)	Intel Stratix 10 GX 2800 FPGA		Xilinx UltraScale+ FPGA (3 SLRs) Xilinx Versal FPGA
Main Memory per Card	32 GiB DDR	32 GiB DDR		- 16 GiB DDR
High-Bandwidth Memory per Card	8 GiB HBM2			8 GiB HBM2 -
Network Interfaces per Card	2x QSFP28 (100G) links	4x QSFP+ (40G) serial point-to-point links		2x QSFP28 (100G) links (U55C) 2x QSFP28 (100G) links (VCK5000)
Topology of System	Topology Alveo Node	Topology Stratix Node		Topology HACC Node

CPUs	2x AMD Milan 7713, 2.0 GHz, each with 64 cores			2x AMD Milan 7V13, 2.45 GHz, each with 64 cores
Main Memory	512 GiB			512 GiB
Storage	480 GB local SSD in `/tmp/`, full access to the Noctua 2 shared file systems			full access to the Noctua 2 shared file systems

Application-specific interconnect	Connected via CALIENT S320 Optical Circuit Switch (OCS), configurable point-to-point connections to any other FPGA or to a 100G Ethernet switch, more details see FPGA-to-FPGA Networking.