NVIDIA A100 40GB with NVLINK

Specification

Property	gpu
GPUs	4x NVIDIA A100 TENSOR CORE GPU
Chip	GA100
Transistors	54 Billion
Manufacturing	7 nm
Chip Size	826 mm²
FP32-ALUs	6.912
INT32-ALUs	6.912
SMs	108
Tensor Cores	432
INT4-Performance	1.248 TOPS / 2.496 TOPS*
INT8-Performance	624 TOPS / 1.248 TOPS*
FP16-Performance	312 TFLOPS / 624 TFLOPS*
Bfloat16-Performance	312 TFLOPS / 624 TFLOPS*
FP32-Performance	19,5 TFLOPS
TF32-TensorCore-Performance	156 TFLOPS / 312 TFLOPS*
FP64-Performance	9,7 TFLOPS
FP64-TensorCore-Performance	19,5 TFLOPS
GPU Memory	40 GB
GPU Memory Type	HBM2
Memory Interface	5.120 Bit
Memory	1.555 GB/s
Max Thermal Design Power (TDP)	400 W

*With sparsity

Topology

GPU topology is important to efficient GPU usage for some applications, e.g. Gaussian. The Table below shows the GPUs in one GPU compute node, the CPU affinity, and the NUMA affinity.

GPUs	CPU affinity	NUMA affinity

GPUs	CPU affinity	NUMA affinity
NVIDIA A100 GPU 0	CPU cores 48-63	NUMA node 3
NVIDIA A100 GPU 1	CPU cores 16-31	NUMA node 1
NVIDIA A100 GPU 2	CPU cores 112-127	NUMA node 7
NVIDIA A100 GPU 3	CPU cores 80-95	NUMA node 5

The figure below shows the GPUs and their NUMA affinity.

CUDA Capability Major/Minor version number:    8.0
Total amount of global memory:                 40441 MBytes (42405855232 bytes)
(108) Multiprocessors, ( 64) CUDA Cores/MP:     6912 CUDA Cores
GPU Max Clock rate:                            1410 MHz (1.41 GHz)
Memory Clock rate:                             1215 Mhz
Memory Bus Width:                              5120-bit
L2 Cache Size:                                 41943040 bytes
Maximum Texture Dimension Size (x,y,z)         1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers  1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers  2D=(32768, 32768), 2048 layers
Total amount of constant memory:               65536 bytes
Total amount of shared memory per block:       49152 bytes
Total number of registers available per block: 65536
Warp size:                                     32
Maximum number of threads per multiprocessor:  2048
Maximum number of threads per block:           1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch:                          2147483647 bytes
Texture alignment:                             512 bytes
Concurrent copy and kernel execution:          Yes with 5 copy engine(s)
Run time limit on kernels:                     No
Integrated GPU sharing Host Memory:            No
Support host page-locked memory mapping:       Yes
Alignment requirement for Surfaces:            Yes
Device has ECC support:                        Enabled
Device supports Unified Addressing (UVA):      Yes
Device supports Compute Preemption:            Yes
Supports Cooperative Kernel Launch:            Yes
Supports MultiDevice Co-op Kernel Launch:      Yes