changeFPGALinks

Owned by Nils Winnwa
Last updated: Oct 19, 2023 by Tobias Kenter
3 min read

Change Serial Channels during a running job

For the general usage of FPGA Serial Channels visit the main documentation page.

Sometimes a previously configured topology needs to be reconfigured, either because the topology was specified incorrectly or a different topology is required. This would entail saving the state of the job, canceling the running job and submitting a new job. By using the tool changeFPGAlinks mid-job you can change the topology on-the-fly without canceling the running allocation. This tool can also be used as a quick and handy way to visualize the current configuration with the FPGA-Link-GUI.

Usage

  • The tool currently only works directly on the FPGA-Nodes during a running job.

  • When the tool returns, the topology is changed to the requested one.

  • After changing the topology, make sure to reset or reprogram your FPGA designs. Otherwise the network interfaces may get stuck in an undefined state.

  • You can run the tool as often as you need topology changes.

Preparation

First submit a job like normal. You can start without any intial links set:

fpga-tester@n2login1:~ $ srun -A pc2-mitarbeiter --constraint=xilinx_u280_xrt2.15 -N 1 -p fpga -t 2:00:00 --pty zsh

or for the Bittware 520N partition you can also start with a specific initial topology:

fpga-tester@n2login1:~ $ srun -A pc2-mitarbeiter --constraint=bittware_520n_20.4.0_max -N 1 -p fpga -t 2:00:00  --fpgalink=ringO --pty zsh

In both cases, next load the module:

fpga-tester@n2fpga17:~ $ module load fpga/changeFPGAlinks

Then you can run the commandchangeFPGAlinksXilinxfor the Alveo U280 partition or the commandchangeFPGAlinks for the Bittware 520N partition. The command names might be subject to changes / unification in the near future. You can run the command to specify a new topology, or run it without topology arguments to check the current status.

Examples for running the command without a topology

This will only generate a overview and not change anything, but can be useful for documenting the current status:

Visualize here: Click!

Examples for running the command to specify a topology

with the --fpgalink= syntax to set the links:

Connect for 3 Xilinx Alveo U280 cards within a single node the respective first port to the Ethernet switch as shown here:

Create a point-to-point loopback configuration for 2 Bittware 520N cards within one node as shown here:

Create a customized point-to-point ring for 3 Xilinx Alveo cards within one node as shown here: