We are currently in the pilot phase for the FPGA partition. In this stage, we have acquired a small number of FPGA boards and are evaluating their suitability for our requirements. More FPGAs of different vendors will be added in the near future.

Hardware Setup

Currently we have

• 3x Alveo V80 FPGAs (see official product page)

in our FPGA pilot phase of Otus.

An overview of the current hardware setup is given in the following table:

Please note:

• you only need to allocate one of the nodes for hardware execution. You can compile, emulate, simulate and synthesize on any other Otus node.

• the Nvidia A40 GPUs were used for thermal evaluations of the nodes and will be removed in future.

Software Stacks

Currently, we test the following software stacks in our pilot phase.

SLASH VRT (V80 Runtime)

The V80 cards can be used via VRT (similar abstraction to Xilinx XRT)

• see technical details at official repository: https://github.com/Xilinx/SLASH

• we maintain a RHEL9/Rocky 9 fork of SLASH/VRT at: https://github.com/pc2/SLASH

The VRT tool flow can be used with modules on any nodes of Otus:

Please note:

• the Lmod warning regarding ncurses is not critical.

Example Applications

VRT comes with example applications (see https://github.com/Xilinx/SLASH/tree/dev/examples) to test the basic functionality.

In order to run them you can do the following:

Load required VRT module

• loads the main VRT tool flow and all required dependencies

• sets up the required environment variables (mainly AMI_HOME)

Get example repository

• uses tested VRT/SLASH version maintained by PC2

Build examples for emulation

• 00_axilite: example to test linking and AXI-Lite control

• emulation: everything runs on the CPU. See below for simulation and hardware build.

• 61:00.0: Usually the PCIe ID (BDF) parameter of the V80 card to use. Can be any reasonable value for emulation.

Hardware Execution

If you want to execute a design in hardware, you need to allocate an FPGA node that has at least one V80 card attached.

Allocate node with V80 card

You can use this command to get node fpga1612 for one hour:

Load the required modules

Check FPGA status

Get the current status of the FPGAs via ami_tool. For example on fpga1612 with two V80 cards:

• the most important output is the State column. Only FPGAs in READY state should be used.

• you can use the BDF 61:00.0 or 91:00.0 to identify one of the cards

• Use ami_tool -h for all options

Execute example in hardware

To speed-up the process and save resources for unnecessary synthesis we have pre-synthesized the design for example 00_axilite. The vrtbin file with the hardware design is located at:

You can run the design in hardware with

• uses device 91:00.0 and the pre-synthesized design 00_axilite_hw.vrtbin

• if you do not have the host code, see the emulation build and execution

If you want to synthesize a design yourself, see the description below.

(Optional) Repeat steps for simulation and hardware build

Hardware simulation can be performed with

In order to synthesize a design instead of using the pre-synthesized version, you can use these steps:

Then, we submit the synthesis_script.sh to the slurm workload manager:

Afterwards you can use the generated 00_axilite_hw.vrtbin as described above for hardware execution.

Early Access and Troubleshooting

If you are interested in getting early access to the FPGA partition in the pilot phase or have issues/questions with the current setup, please contact us via Email.