FPGA Usage

Status

  • traffic light/corporate design to indicate status

  • reference to main Noctua 2 status page

News and Updates

Below you can find the most recent software and hardware updates regarding the FPGA partition. More news and updates can be found in the FPGA news and updates archive.

2023

FPGA Infrastructure

Hardware Overview

The software environment is setup using modules, see section Software Overview. For hardware execution, FPGA nodes with the correct configuration and driver version (so-called board support package or BSP) need to be allocated, see section FPGA System Access to FPGA Partition.

Software Overview

The software environment is setup using modules. Depending on the user requirements, different development flows for the FPGA cards are supported.

  • Xilinx Alveo U280

    • Vitis Design Flow (recommended)

    • Vivado Design Flow

  • Intel Stratix 10

    • oneAPI (recommended)

    • OpenCL (recommended for projects with exisiting OpenCL code base or for usage of serial channels)

    • DSP Builder

The development (including emulation to check functional correctness, report generation to get indications of expected performance, and synthesis with bittstream generation) can be done on any Noctua 2 node. Just load the corresponding modules for the target FPGA platform and Development Tool flow.

To execute designs on actual FPGAs, the same modules are required and additionally an FPGA node needs to be allocated with a fitting constraint, to get FPGAs with the expected configuration and drivers.

The Table of FPGA Software and Firmware Stacks provides an overview of the interplay of tools, modules and constraints for the three recommended development flows. Additionally, we have created Quick Start Guides to walk you through the six steps with examples using the latest tools.

System Access to FPGA Partition

To use Noctua 2 nodes with FPGAs, along with your Slurm command, you need to select the FPGA partition and provide a constraint to specify the configuration of FPGAs (shell, driver, board support package (BSP)) that your designs have been built for.

For Xilinx Alveo U280 cards you can use

srun --partition=fpga --constraint=xilinx_u280_xrt2.12 -A [YOUR_PROJECT_ACCOUNT] -t 2:00:00 --pty bash

For Bittware 520n cards with Intel Stratix 10 FPGAs you can use

srun --partition=fpga --constraint=bittware_520n_20.4.0_max -A [YOUR_PROJECT_ACCOUNT] -t 2:00:00 --pty bash

Constraints can be used together with srun, sbatch and salloc, however under some conditions salloc will fail, for details click the expansion box below. We recommend to always use srun or sbatch.

A problem occurs when one of the nodes to be allocated is configured for a different constraint and is currently in use. Then salloc will fail with the following error message.

salloc: error: Job submit/allocate failed: Requested node configuration is not available

Workaround: use an allocation without requesting specific node names.

A problem also occurs when one of the nodes to be allocated is configured for a different constraint and is currently free. The allocation succeeds while the nodes are still reconfigured, programs or scripts starting during this time will fail, actual errors encountered differ.

A list of available matching versions of the BSPs and SDKs can be found in the Software Overview Details.

Serial Channels between FPGAs

When configured with a correct BSP, all FPGA boards offer 4 point-to-point connections to other FPGA boards. FPGA Serial Channels have an own documentation page with various examples and a graphical input tool (see figure below).

Clique topology
Clique topology between two FPGA nodes.

Sanity Checks and Troubleshooting

Contact and Support

For problems with the FPGA infrastructure (software and/or hardware) use our main support mail address pc2-support@uni-paderborn.de.

In order to help you as quickly as possible, please follow these guidelines

  • Use [Noctua2-FPGA] as a prefix in your email subject line.

  • Where did the problem occur and what did you expect to happen?

  • If possible, how can we reproduce the error in a systematic manner?

  • Did you attempt to fix/troubleshoot the problem? If you attempted to debug the problem, provide us the steps you already took and the intermediate results.

For general questions, acceleration support or project ideas with regarding FPGAs, please reach out to the FPGA domain-expects.

Getting Started

Available FPGA Libraries and Applications

We and others have developed several ready to use libraries and applications that use FPGAs to accelerate the computation. Please use the links to get examples and the documentation. You can contact us, if you need guidance to accelerate your target code with our FPGAs libraries.

Application

Toolchain

Type of Support

Application

Toolchain

Type of Support

CP2K for DFT with FPGA Acceleration of the Submatrix Method

OpenCL

Ready-to-use module files and bitstreams deployed on Noctua 2

CP2K for DFT with FPGA Acceleration of 3D FFTs

OpenCL

FPGA support in CP2K main repository + extra repository with FPGA designs fitting the Bittware 520N cards in Noctua 2

HPCC FPGA: HPC Challenge Benchmark Suite for FPGAs

OpenCL

Repository with benchmark suite targeting FPGAs from Intel (including the Bittware 520N with Stratix 10 cards in Noctua 2) and Xilinx

Cannon Matrix Multiplication on FPGAs

OpenCL

Repository with implementation of Cannon matrix multiplication as building block for GEMM on FPGAs fitting the Bittware 520N cards in Noctua 2

StencilStream Stencil Simulation Library for FPGAs

oneAPI

Repository with library and examples build with oneAPI for stencil simulations on FPGAs fitting the Bittware 520N cards in Noctua 2

Related pages