FPGA Linktest - OpenCL

Owned by Heinrich Riebler
Last updated: Oct 30, 2024
15 min read

In order to try and check the FPGA-to-FPGA serial channel connections of the Intel Stratix 10 cards with the OpenCL tool flow, we have prepared ready-to-use modules with compiled host code and synthesized device code. To run the tests, please follow the steps.

Get Node and Prepare Environment

  1. Get a node with Stratix 10 cards in the right BSP (all BSPs with _max suffix support serial channels)

srun --partition=fpga -A YOUR-PROJECT-NAME --constraint=bittware_520n_20.4.0_max -t 1:00:00 --pty bash

This command allocates one node with the correct BSP for one hour. You get a shell (bash) on the node.

  1. Load the ready-to-use modules to run the linktests

module reset module load fpga module load intel/opencl_sdk/21.4.0 module load bittware/520n/20.4.0_max module load changeFPGAlinks module load intel/testFPGAlinks-opencl

This commands

  • (intel/opencl_sdk/21.4.0 and bittware/520n/20.4.0_max): load the correct BSP version and the latest OpenCL SDK version that is supported on our systems

  • (changeFPGAlinks): loads the utility to change the topology how the FPGA-to-FPGA network is configured, see FPGA-to-FPGA Networking

  • (intel/testFPGAlinks-opencl): loads the OpenCL linktest suite

Configure Topology and Run Test

Before running a test, the correct topology on the FPGA-to-FPGA network needs to be set. This is done with the changeFPGAlinks utility. The following example describes the required steps for the test testFPGAlinks_throughput_nodes_1_cards_1 (using one FPGA node and one FPGA card). The test has the following parameters

Usage : testFPGAlinks_throughput_nodes_1_cards_1 [<options>] -d <device> : OpenCL Device : 0 - acl0 (default) : 1 - acl1 : 2 - acl2 -s <sequence> : Data Sequence : 0 - Random (default) : 1 - Count (with channel in top nibble) : 2 - Inverse of 1 -n <numerrs> : Num of errors per channel to display. Default 1 -v <verbosity> : How verbose to display. Default 0 -t <tx_gap> : Default = 0 -r <rx_gap> : Default = 0 -l <loops> : Default = 1 -m <testmode> : Test connection mode. : 0 - QSFP0 to QSFP1, QSFP2 to QSFP3 (default) : 1 - QSFP0 to QSFP2, QSFP1 to QSFP3 : 2 - QSFP0 to QSFP3, QSFP1 to QSPF2 : 3 - each QSFP linked to itself (loopback)

The important parameters are

  • -d to select the FPGA card (= 0 for the first FPGA card or = 1 for the second FPGA card)

  • -m to set the testmode, how the FPGA links are set. The image below shows a visualization of the different testmodes

 

grafik-20241030-145659.png

If we want to run testmode 0, we need to setup the FPGA-to-FPGA network accordingly. This is done with the changeFPGAlinks utility:

Testmode 0, see FPGA GUI

The other topologies for the testmodes can be set accordingly, expand to see details.

Testmode 1

Testmode 2

Testmode 3

Now we can run the test. To execute the test on the first FPGA card and the first testmode, we use

The test should end with Serial channel test PASSED.