Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Table of Contents

Xilinx Alveo U280

...

The Xilinx xrt driver allows to measure power using a standalone command line tool. The specific tool invocation and output format depends on the xrt version (module xilinx/xrt and configuration constraint xilinx_u280_xrt...).

, Xilinx Alveo U55C, Xilinx Versal VCK5000

The Xilinx xrt driver provides power measurements via a hwmon interface. It can be queried by simply reading from a corresponding file, which returns the current power consumption in µW:

Code Block
# Location: /sys/bus/pci/devices/$BDF/hwmon/hwmon*/power1_input,
# where BDF is one of 0000:a1:00.1, 0000:81:00.1 or 0000:01:00.1

[tester@n2fpga01 ~]$ cat /sys/bus/pci/devices/0000\:01\:00.1/hwmon/hwmon*/power1_input 
58360411

For a more convenient interface, we provide a dedicated tool named xilinx_power which is available on all Xilinx FPGA and Xilinx HACC nodes:

Code Block
#
# Xilinx FPGA Nodes
#
[tester@n2fpga01 ~]$ xilinx_power 
0000:a1:00.1: 41.35W
0000:81:00.1: 39.22W
0000:01:00.1: 58.28W

[tester@n2fpga01 ~]$ xilinx_power -c2
0000:01:00.1: 58.22W

[tester@n2fpga01 ~]$ xilinx_power -c 0000:01:00.1
0000:01:00.1: 57.95W


#
# Xilinx HACC Nodes
#
[tester@n2hacc03 ~]$ xbutil examine
# [...]
Devices present
BDF             :  Shell                              Logic UUID                            Device ID       Device Ready*  
---------------------------------------------------------------------------------------------------------------------------
[0000:81:00.1]  :  xilinx_u55c_gen3x16_xdma_base_3    97088961-FEAE-DA91-52A2-1D9DFD63CCEF  user(inst=134)  Yes            
[0000:a1:00.1]  :  xilinx_vck5000_gen4x8_qdma_base_2  05DCA096-76CB-730B-8D19-EC1192FBAE3F  user(inst=135)  Yes            
[0000:c1:00.1]  :  xilinx_u55c_gen3x16_xdma_base_3    97088961-FEAE-DA91-52A2-1D9DFD63CCEF  user(inst=133)  Yes            
[0000:e1:00.1]  :  xilinx_vck5000_gen4x8_qdma_base_2  05DCA096-76CB-730B-8D19-EC1192FBAE3F  user(inst=132)  Yes   

[tester@n2hacc03 ~]$ xilinx_power                
0000:e1:00.1: 34.00W
0000:c1:00.1: 15.04W
0000:a1:00.1: 41.00W
0000:81:00.1: 14.50W

Run with --help to get a list of command line arguments. -c allows specifying a specific card by either index or BDF.

You can also use the XRT API to query electrical information, including the current power consumption, as JSON. The following example uses Boost to parse that JSON data:

Code Block
#include <boost/property_tree/json_parser.hpp>
#include <xrt/xrt_device.h>

[...]

// Assuming `device` being an instance of or a reference to a valid xrt::device

auto json = std::stringstream{};
json << device.get_info<xrt::info::device::electrical>();

// parse JSON into a property tree
auto props = boost::property_tree::ptree{};
boost::property_tree::read_json(json, props);

auto watts = props.get<float>("power_consumption_watts", 0.0f);
std::cout << watts << "W\n";

Lastly, you can also use xbutil to query those electrical information. Usage example and sample output for xrt 2.12, after selecting querying the first card. During this example, the card is idle and configured with the validation bitstream. Total card power Power consumption is shown in line 17.:

Code Block
[tester@n2fpga02 ~]$ ml fpga
[tester@n2fpga02 ~]$ ml xilinx/xrt/2.12
[tester@n2fpga02 ~]$ xbutil examine
...
Devices present
  [0000:a1:00.1] : xilinx_u280_xdma_201920_3 user(inst=129) 
  [0000:81:00.1] : xilinx_u280_xdma_201920_3 user(inst=130) 
  [0000:01:00.1] : xilinx_u280_xdma_201920_3 user(inst=128) 
  
[tester@n2fpga02 ~]$ xbutil examine -d 0000:a1:00.1 --report electrical

-----------------------------------------------
1/1 [0000:a1:00.1] : xilinx_u280_xdma_201920_3
-----------------------------------------------
Electrical
  Max Power              : 225 Watts
  Power                  : 33.793573 Watts
  Power Warning          : false

  Power Rails            : Voltage   Current
  12 Volts Auxillary     : 12.199 V,  1.363 A
  12 Volts PCI Express   : 12.192 V,  1.408 A
  3.3 Volts PCI Express  :  3.286 V
  3.3 Volts Auxillary    :  3.292 V
  Internal FPGA Vcc      :  0.851 V,  5.076 A
  DDR Vpp Bottom         :  2.500 V
  DDR Vpp Top            :  2.500 V
  5.5 Volts System       :  5.488 V
  Vcc 1.2 Volts Top      :  1.212 V
  Vcc 1.2 Volts Bottom   :  1.204 V
  1.8 Volts Top          :  1.808 V
  0.9 Volts Vcc          :  0.901 V
  12 Volts SW            : 12.235 V
  Mgt Vtt                :  1.203 V

General notes:

  • We recommend to run kernels for at least ~1s and automatically perform repeated invocations to the command line tool concurrently in order to get reasonably accurate results. Further effects of increased power consumption due to thermal effects can be observed after multiple minutes of load on the cards, but are comparably minor.

  • Power consumption values are only updated once per second. Querying with a higher frequency therefore does not provide any additional data.

Bittware 520N

The Bittware driver allows to measure power using a standalone command line tool that queries the board power via the i2c bus. It's available on all FPGA nodes with Bittware 520N cards (irrespective of the constraint).

  • This tool currently requires special permissions. You can request those via email to pc2-support@uni-paderborn.de.

  • We recommend to run kernels for at least ~1s and automatically perform repeated invocations to the command line tool concurrently in order to get reasonably accurate results. Further effects of increased power consumption due to thermal effects can be observed after multiple minutes of load on the cards, but are comparably minor.

Usage example and sample output. During this example, the card is executing a small sample kernel. Total card power in last row.:

Code Block
[
tester@fpga-0001
tester@n2fpga18 
tmp
~]$ 
newgrp dialout [tester@fpga-0001 tmp]$ /usr/share/nallatech/520n/bist/utilities/nalla_serial_cardmon/bin/nalla_serial_cardmon ################################### 520 nalla_serial card monitor utility version 2.3 ################################### System manager version 0x142 FPGA fabric over temperature limit: 100 degC FPGA tile over temperature limit: 100 degC Over temperature: NOT TRIPPED Voltages -------- PCIE_12V0 (V) : 11.98 EXT2_12V0 (V) : 12.10 EXT1_12V0 (V) : 12.14 VCC_1V2 (V) : 1.20 VCC_0V9 (V) : 0.86 VID_VOUT (V) : 0.86 VCCR_1V0 (V) : 1.13 VCCT_1V0 (V) : 1.12 VCC_2V5 (V) : 2.50 VCC_1V8 (V) : 1.86 VCC_3V3 (V) : 3.28 VCC_1V8A (V) : 1.79 VCC_ERAM (V) : 0.92 Power up watchdog OK PSU health : 0x8fff PCIE_12V0 : UP EXT1_12V0 : UP EXT2_12V0 : UP VCC_0V9 : UP VCCR_1V0 : UP VCCT_1V0 : UP VCC_ERAM : UP VCC_1V8 : UP VCC_1V8A : UP VCC_1V2 : UP VCC_2V5 : UP VCC_3V3 : UP Temperatures ------------ Front PCB (C) : 39.6 Centre PCB (C) : 35.5 Rear PCB (C) : 34.0 FPGA fabric temperature (deg C): 40.44 FPGA tile temperature (deg C): 35.19 Core PH12: controller temperature (C) : 38.4 Core PH12: phase 0 temperature (C) : 39.5 Core PH12: phase 1 temperature (C) : 40.1 Core PH34: controller temperature (C) : 37.7 Core PH34: phase 0 temperature (C) : 40.3 Core PH34: phase 1 temperature (C) : 40.3 Core PH56: controller temperature (C) : 33.1 Core PH56: phase 0 temperature (C) : 34.1 Core PH56: phase 1 temperature (C) : 34.4 VCCR_1V8A: controller temperature (C) : 44.1 VCCR_1V8A: phase 0 temperature (C) : 45.6 VCCR_1V8A: phase 1 temperature (C) : 45.6 1V2_3V3: controller temperature (C) : 42.9 1V2_3V3: phase 0 temperature (C) : 45.9 1V2_3V3: phase 1 temperature (C) : 45.7 Power Measurements ------------------ VCCR iout (A) : 3.19 1V8A iout (A) : 2.64 1V2 iout (A) : 2.02 3V3 iout (A) : 3.88 Core PH12 iout (A) : 8.88 Core PH34 iout (A) : 9.78 Core PH56 iout (A) : 1.94 Total Core iout (A) : 20.60 TI INA219 current monitor, PCIe 12V bus voltage (V) : 12.03 TI INA219 current monitor, PCIe 12V current (A) : 2.28 TI INA219 current monitor, Ext1 12V bus voltage (V) : 12.10 TI INA219 current monitor, Ext1 12V current (A) : 1.98 TI INA219 current monitor, Ext2 12V bus voltage (V) : 12.11 TI INA219 current monitor, Ext2 12V current (A) : 0.32 Total board power (W): 60.25

...

bittware_power
acl0: 64.86W
acl1: 65.99W

[tester@n2fpga18 ~]$ bittware_power -c0
acl0: 64.91W

[tester@n2fpga18 ~]$ bittware_power -c1
acl1: 65.99W

Run with --help to get a list of command line arguments. For example, with -c the 520N card of interest can be selected.