Versions Compared

Old Version 7

changes.mady.by.user Tobias Kenter

Saved on

compared with

New Version 8

changes.mady.by.user Michael Lass

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

As it can be tedious and error-prone to define each connection manually, we also provide a set of predefined topologies to be requested. The following table summarizes the available options.

Topology type

Invocation

Min-Max number of nodes

Brief description

pair

--fpgalink="pair"

1-N

Pairwise connect the 2 FPGAs within each node

clique

--fpgalink="clique"

2

All-to-all connection for 2 nodes, 4 FPGAs

ring

--fpgalink="ringO"

1-N

Ring with two links per direction, acl0 down, acl1 up

--fpgalink="ringN"

1-N

Ring with two links per direction, acl0 down, acl1 down

--fpgalink="ringZ"

1-N

Ring with two links per direction, acl0 and acl1 neighbors

torus

--fpgalink="torus2"

1-N

Torus with 2 FPGAs per row

--fpgalink="torus3"

2-N

Torus with 3 FPGAs per row

--fpgalink="torus4"

2-N

Torus with 4 FPGAs per row

--fpgalink="torus5"

3-N

Torus with 5 FPGAs per row

--fpgalink="torus6"

3-N

Torus with 6 FPGAs per row

Pair topology

...

Within each node, all channels of one FPGA board are connected to the respective channel of the other FPGA board. No connections between nodes are made.

...

Expand
titleExpected output
Code Block
Summarizing most recent topology information and exporting FPGALINK variables:
Host list
fpga-0001
fpga-0002
fpga-0003
fpga-0004
fpga-0005
fpga-0006
fpga-0007
fpga-0008
Torus topology with width 4 and height 4
Torus topology information: columns from north to south, rows from west to east, end connected back to start
fpga-0001:acl0 - fpga-0001:acl1 - fpga-0002:acl0 - fpga-0002:acl1
fpga-0003:acl0 - fpga-0003:acl1 - fpga-0004:acl0 - fpga-0004:acl1
fpga-0005:acl0 - fpga-0005:acl1 - fpga-0006:acl0 - fpga-0006:acl1
fpga-0007:acl0 - fpga-0007:acl1 - fpga-0008:acl0 - fpga-0008:acl1
Generated connections
FPGALINK0=fpga-0001:acl0:ch1-fpga-0003:acl0:ch0
FPGALINK1=fpga-0001:acl0:ch3-fpga-0001:acl1:ch2
FPGALINK2=fpga-0001:acl1:ch1-fpga-0003:acl1:ch0
FPGALINK3=fpga-0001:acl1:ch3-fpga-0002:acl0:ch2
FPGALINK4=fpga-0002:acl0:ch1-fpga-0004:acl0:ch0
FPGALINK5=fpga-0002:acl0:ch3-fpga-0002:acl1:ch2
FPGALINK6=fpga-0002:acl1:ch1-fpga-0004:acl1:ch0
FPGALINK7=fpga-0002:acl1:ch3-fpga-0001:acl0:ch2
FPGALINK8=fpga-0003:acl0:ch1-fpga-0005:acl0:ch0
FPGALINK9=fpga-0003:acl0:ch3-fpga-0003:acl1:ch2
FPGALINK10=fpga-0003:acl1:ch1-fpga-0005:acl1:ch0
FPGALINK11=fpga-0003:acl1:ch3-fpga-0004:acl0:ch2
FPGALINK12=fpga-0004:acl0:ch1-fpga-0006:acl0:ch0
FPGALINK13=fpga-0004:acl0:ch3-fpga-0004:acl1:ch2
FPGALINK14=fpga-0004:acl1:ch1-fpga-0006:acl1:ch0
FPGALINK15=fpga-0004:acl1:ch3-fpga-0003:acl0:ch2
FPGALINK16=fpga-0005:acl0:ch1-fpga-0007:acl0:ch0
FPGALINK17=fpga-0005:acl0:ch3-fpga-0005:acl1:ch2
FPGALINK18=fpga-0005:acl1:ch1-fpga-0007:acl1:ch0
FPGALINK19=fpga-0005:acl1:ch3-fpga-0006:acl0:ch2
FPGALINK20=fpga-0006:acl0:ch1-fpga-0008:acl0:ch0
FPGALINK21=fpga-0006:acl0:ch3-fpga-0006:acl1:ch2
FPGALINK22=fpga-0006:acl1:ch1-fpga-0008:acl1:ch0
FPGALINK23=fpga-0006:acl1:ch3-fpga-0005:acl0:ch2
FPGALINK24=fpga-0007:acl0:ch1-fpga-0001:acl0:ch0
FPGALINK25=fpga-0007:acl0:ch3-fpga-0007:acl1:ch2
FPGALINK26=fpga-0007:acl1:ch1-fpga-0001:acl1:ch0
FPGALINK27=fpga-0007:acl1:ch3-fpga-0008:acl0:ch2
FPGALINK28=fpga-0008:acl0:ch1-fpga-0002:acl0:ch0
FPGALINK29=fpga-0008:acl0:ch3-fpga-0008:acl1:ch2
FPGALINK30=fpga-0008:acl1:ch1-fpga-0002:acl1:ch0
FPGALINK31=fpga-0008:acl1:ch3-fpga-0007:acl0:ch2

...

Using Serial Channels in Design Flows

Xilinx Alveo U280

Refer to our Aurora_HLS project for an example design implementing serial communication channels on the Alveo U280. Alternatively, you can use Xilinx ACCL for an Ethernet-based communication scheme.

Intel Stratix 10

All Intel Stratix 10 boards on Noctua 2 offer 4 point-to-point connections to other FPGA boards when the node is configured with a p520_max_sg280l BSP. Their use differs based on the used development flow.

OneAPI

Refer to the documentation on I/O Pipes for details on how to use the external serial channels. The channel IDs are mapped as follows:

  • Port 0: Channels 0 (read) and 1 (write)

  • Port 1: Channels 2 (read) and 3 (write)

  • Port 2: Channels 4 (read) and 5 (write)

  • Port 3: Channels 6 (read) and 7 (write)

The pipes need to be configured for a data type of width 256 bits. This could, for example, be a std::array<int, 8>. You may use a small C++ header like the following to bundle read and write pipes into a single channel class that has both read and write operations:

Code Block
languagecpp
#include <sycl/ext/intel/fpga_extensions.hpp>
  
template <int portnum>
struct read_channel_id {
    static constexpr unsigned id = portnum * 2;
};

template <int portnum>
struct write_channel_id {
    static constexpr unsigned id = portnum * 2 + 1;
};

template <int portnum, class T, std::size_t min_capacity = 0>
// requires((portnum >= 0) && (portnum < 4) && (sizeof(T) == 32)) // C++20 only
struct external_channel
    : private sycl::ext::intel::kernel_readable_io_pipe<read_channel_id<portnum>, T, min_capacity>,
      private sycl::ext::intel::kernel_writeable_io_pipe<write_channel_id<portnum>, T, min_capacity> {

    using read_pipe = sycl::ext::intel::kernel_readable_io_pipe<read_channel_id<portnum>, T, min_capacity>;
    using write_pipe = sycl::ext::intel::kernel_writeable_io_pipe<write_channel_id<portnum>, T, min_capacity>;

    using read_pipe::read;
    using write_pipe::write;
};

OpenCL

From the OpenCL environment, these links are used as external serial channels. A status reg value of 0xfff11ff1 in the diagnose indicates an active connection.