Skip to content

Reference

A plain-language glossary of the hardware, specs, and terms we work with every day.

120V vs 240V power
A 15 A, 120 V circuit sustains about 1,440 W. A 240 V circuit passes twice the watts per amp and lets large power supplies deliver more.
12VHPWR vs 12V-2x6
12VHPWR is the original 16-pin, 600 W GPU power connector. 12V-2x6 is the revision with changed PCB header pin lengths.
1DPC vs 2DPC
1DPC puts one DIMM in each memory channel. 2DPC puts two for more capacity, and the board may rate that at a lower speed.
3DS RDIMM
A 3DS RDIMM is a server memory module whose DRAM packages stack several dies, joined by through-silicon vias, to reach 256 GB per module.
AMD Instinct
AMD Instinct is AMD's line of data center GPU accelerators for AI and HPC, built on the CDNA architecture and programmed through ROCm.
AVX-512
AVX-512 is an x86 vector instruction set with 512-bit registers. Xeon 6 P-cores and AMD EPYC since Zen 4 support it, Xeon 6 E-cores do not.
Arm vs x86 server
Arm servers run CPUs built on the Arm instruction set, such as NVIDIA Grace. x86 servers run Intel Xeon or AMD EPYC. Each needs its own builds.
B200 vs B300
HGX B200 uses Blackwell GPUs with 180 GB of HBM3e each. HGX B300 uses Blackwell Ultra GPUs with 288 GB each, and cuts FP64 to 10 TFLOPS.
BF16
BF16 is a 16-bit floating point format with 8 exponent and 7 mantissa bits, matching FP32's range at lower precision.
BMC
A BMC (baseboard management controller) is the small dedicated chip on a server motherboard that runs IPMI or Redfish for remote management.
Barebone server
A server chassis shipped with the motherboard, backplane, power supplies, and cooling installed, but no CPU, memory, storage, or GPUs.
Blower vs open-air GPU cooler
A blower cooler exhausts a GPU's heat out the card's rear bracket. An open-air cooler vents it into the case for the chassis fans to remove.
Burn-in testing
Burn-in testing runs new hardware at sustained full load before deployment to catch early failures before a customer sees them.
C13 vs C19
C13 is an IEC 60320 coupler rated 10 A at 250 V. C19 is the larger IEC 60320 coupler rated 16 A at 250 V.
CAMM2
CAMM2 is a compression-attached memory module. LPCAMM2 puts LPDDR5X on it as a detachable, upgradeable alternative to soldered memory.
CRPS
CRPS, Common Redundant Power Supply, is a hot-swap server power module standard. Intel specifies 73.5 mm wide modules, 185 or 265 mm long.
CUDA compute capability
CUDA compute capability is the major.minor version that sets which hardware features an NVIDIA GPU supports, such as 8.0 for the A100.
CUDA cores vs Tensor cores
CUDA cores run general parallel math one operation at a time. Tensor cores complete a small matrix multiply per clock for AI math.
CUDA vs ROCm
CUDA is NVIDIA's parallel computing platform for its GPUs. ROCm is AMD's open source software platform for GPU computing on AMD hardware.
CXL (Compute Express Link)
Compute Express Link (CXL) is an open interconnect on the PCIe physical layer that gives CPUs coherent access to device and expander memory.
Cluster computing
Cluster computing links multiple servers together so they operate as one system for a single workload.
Colocation
Colocation is renting rack space, power, and network connectivity in a third-party data center for hardware you own and manage yourself.
Context window
A context window is the maximum number of tokens a language model can hold at once, covering the prompt, the history, and its own reply.
Continuous batching
Continuous batching is an LLM serving method that adds and drops requests at every decode step instead of waiting for the whole batch to finish.
Coolant distribution unit (CDU)
A coolant distribution unit (CDU) moves heat from a liquid-cooled rack's coolant loop into facility water while keeping the two fluids apart.
Correctable vs uncorrectable ECC errors
A correctable error (CE) is a bit flip that ECC fixed. An uncorrectable error (UE) is damage past what the code can repair.
DDR4 vs DDR5
DDR4 puts one 64-bit channel on each module, up to 3,200 MT/s. DDR5 splits each module into two 32-bit subchannels, from 4,800 MT/s.
DDR5 on-die ECC
DDR5 on-die ECC corrects single-bit errors inside each DRAM chip. Side-band ECC adds check bits across the module and is what makes a DIMM ECC memory.
DGX
DGX is NVIDIA's line of complete, factory-built AI servers and workstations, sold with GPUs, CPU, memory, and software preinstalled as one unit.
DGX Spark
DGX Spark is NVIDIA's compact desktop AI computer, built on the GB10 Grace Blackwell Superchip with 128 GB of unified memory.
DGX SuperPOD
DGX SuperPOD is NVIDIA's turnkey reference architecture for clusters of DGX systems, built from repeatable blocks called scalable units.
DWPD vs TBW
DWPD is how many times a day the whole drive can be rewritten. TBW is the total data written over the rating period. Both measure SSD endurance.
Deep learning
Deep learning is machine learning using neural networks with many stacked layers, trained on data to approximate a function.
Direct-to-chip liquid cooling
Direct-to-chip liquid cooling pumps coolant through cold plates mounted on the CPU and GPU. A CDU circulates it through rack manifolds.
DisplayPort vs HDMI
DisplayPort and HDMI are the two video outputs on workstation GPUs, differing mainly in bandwidth per version and licensing.
Drive form factors (U.2, M.2, EDSFF)
U.2, M.2, and EDSFF are the three physical shapes an SSD ships in. None of them interchange without an adapter.
E1.S vs E3.S
E1.S is the short, narrow EDSFF SSD for dense 1U servers. E3.S is the wider EDSFF SSD, 76 mm across in Kioxia's 7.5 mm CD8P-R.
ECC memory
ECC memory detects and corrects single-bit errors in RAM automatically, using an extra chip per module to store parity data.
EPS vs PCIe 8-pin
EPS is the 8-pin 12 V CPU power connector with four 12 V pins. PCIe 8-pin is the 150 W graphics card connector, keyed differently.
EPYC vs Xeon
EPYC is AMD's x86 server CPU line. Xeon is Intel's. They differ in cores per socket, memory speed, PCIe lanes and maximum socket count.
FP16
FP16 (half precision) is a 16-bit floating point format, 1 sign bit, 5 exponent bits, 10 mantissa bits, used in mixed precision training.
FP32
FP32 (single precision) is a 32-bit floating point format, 1 sign bit, 8 exponent bits, 23 mantissa bits, the default type for GPU compute.
FP4
FP4 is a 4-bit floating point format used in NVIDIA Blackwell GPUs for AI inference, trading precision for lower memory use and higher throughput.
FP6
FP6 is a 6-bit floating point format in two layouts, E2M3 and E3M2, that the OCP Microscaling (MX) specification uses for MXFP6.
FP64
FP64 (double precision) is a 64-bit floating point format, 1 sign bit, 11 exponent bits, 52 mantissa bits, used in HPC and simulation.
FP8
FP8 is an 8-bit floating point format used for neural network training and inference, storing each value in 1 byte instead of 2 or 4.
Fine-tuning
Fine-tuning continues training a pretrained model on a smaller, focused dataset so it adopts a specific style, domain, or task.
GDDR
GDDR is graphics-oriented DDR memory built for bandwidth over capacity. GDDR6 and GDDR7 are the current versions.
GDDR6 vs GDDR7
GDDR6 uses two-level NRZ signaling, rated up to 24 Gbps per pin by Samsung. GDDR7 uses three-level PAM3, up to 32 Gb/s at Micron.
GGUF
GGUF is a single-file format that packs a language model's weights, tokenizer, and settings together, built for llama.cpp.
GPU junction temperature
GPU junction temperature, or hotspot, is a GPU sensor reading AMD SMI reports with slowdown and shutdown limits, next to edge temperature.
GPU partitioning
GPU partitioning splits one physical GPU into hardware partitions so each VM gets a dedicated fraction, through SR-IOV on Windows Server 2025 Hyper-V.
GPU passthrough
GPU passthrough assigns one physical GPU directly to a single virtual machine, bypassing the hypervisor for near-native performance.
GPU server
A GPU server is a rack-mounted computer built to hold multiple GPUs, with the power and cooling to run them all at full load.
GPU time-slicing
GPU time-slicing shares one GPU by letting workloads take turns on its engines. Kubernetes replicas get no memory or fault isolation.
GPU-accelerated RAID
GPU-accelerated RAID offloads NVMe RAID processing to an NVIDIA GPU instead of a RAID card or the host CPU. Graid's SupremeRAID is the example.
GPUDirect Storage
GPUDirect Storage (GDS) sets up a DMA path between NVMe or network storage and GPU memory, skipping the bounce buffer in CPU memory.
Grace Blackwell superchip
A Grace Blackwell superchip is an NVIDIA processor that pairs an Arm CPU with Blackwell GPUs so both share one coherent memory pool.
Gray market hardware
Gray market hardware is a genuine product sold outside a manufacturer's authorized channel, usually without transferable warranty or support.
HBM
HBM is memory stacked in vertical layers beside a processor and connected through a silicon interposer for very high bandwidth.
HBM3E vs HBM4
HBM3E stacks use a 1,024-bit interface. HBM4 doubles it to 2,048 bits and lifts per-stack bandwidth to 3,300 GB/s in Samsung's rating.
HGX
HGX is NVIDIA's SXM GPU baseboard that server makers build into their own systems, not a complete server on its own.
Hardware RAID vs software RAID
Hardware RAID runs on a controller card with its own processor and protected cache. Software RAID runs on the host CPU inside the OS.
Hopper vs Blackwell
Hopper is NVIDIA's single-die H100 and H200 GPU generation. Blackwell is the dual-die B200 and B300 generation, and it adds FP4.
Hybrid storage
Hybrid storage combines fast NVMe or SSD drives with high capacity HDDs in one system, using each tier for what it does best.
INT4
INT4 is a 4-bit integer format holding 16 values, used mainly to quantize neural network weights for inference.
INT8
INT8 is an 8-bit integer format that stores whole numbers from -128 to 127, paired with a scale factor for quantized inference.
IOMMU
An IOMMU translates device DMA addresses and restricts the memory a device can reach, which lets a PCIe device be assigned to a virtual machine.
IPMI
IPMI is the standard protocol for monitoring and controlling a server remotely, independent of its operating system or power state.
ISV certification
ISV certification is a software vendor's formal validation that a specific GPU, driver version, and OS combination runs its application correctly.
Immersion cooling
Immersion cooling is a method that submerges servers in a dielectric liquid, which carries heat away without conducting electricity.
Inference
Inference is the stage where a trained model answers a prompt, producing output from fixed weights rather than adjusting them.
InfiniBand vs Ethernet
InfiniBand is a purpose-built lossless fabric for GPU clusters. Ethernet is the general-purpose standard that now competes through RoCE.
KV cache
The KV cache stores each previous token's key and value tensors so a transformer doesn't recompute them at every generation step.
LPDDR5X
LPDDR5X is low-power DRAM, usually soldered to the board, with speed grades up to 10.7 Gbps per pin. It trades upgradability for efficiency.
LRDIMM vs RDIMM
RDIMM buffers only command and address signals. LRDIMM adds data buffers that cut bus load, for higher capacity per channel.
Lead time
Lead time is the span from placing an order to the goods arriving. It runs from days for in-stock parts to months for allocated ones.
Liquid cooling vs air cooling
Liquid cooling carries chip heat away in a coolant loop instead of airflow, handling far higher power density than air at higher cost.
LoRA
LoRA (low-rank adaptation) fine-tunes a model by training small adapter matrices instead of all its weights.
Local LLM
A local LLM runs inference on hardware you own, keeping data on premises instead of sending prompts to a cloud API.
Low profile vs full height GPU
A low-profile card is about 69 mm tall. A full-height card is 111 mm tall or more and needs a full-height slot opening.
MCIO vs SlimSAS
MCIO (SFF-TA-1016) carries PCIe 5.0 x8 links on server boards. SlimSAS (SFF-8654) carries SAS and NVMe on storage adapters.
MGX
MGX is NVIDIA's modular server reference architecture, letting OEMs mix CPU, GPU, and DPU trays on shared chassis designs.
MOQ
MOQ, or minimum order quantity, is the smallest number of units a supplier will sell on a single purchase order for a given part number.
MRDIMM
MRDIMM is a DDR5 server module that runs two ranks at once through a data buffer to reach 8,800 MT/s or more. Only CPUs that list it can use it.
Memory bandwidth
Memory bandwidth is the rate data moves between memory and a processor, set by bus width times data rate and measured in GB/s or TB/s.
Memory bus width
Memory bus width is the number of data bits a GPU moves to memory per transfer, in bits. Bandwidth equals bus width times data rate divided by 8.
Memory channels
A memory channel is an independent data path between a CPU's memory controller and its DIMM slots. Bandwidth scales with populated channels.
Memory-bound vs compute-bound
A memory-bound workload waits on data from memory. A compute-bound workload waits on the processor's math units.
Mixed precision
Mixed precision runs most neural network math in 16-bit formats while keeping FP32 for sensitive steps like weight updates.
Mixture of experts (MoE)
Mixture of experts splits a model into parallel sub-networks and routes each token through only a few of them, not the whole model.
Multi-GPU workstation
A multi-GPU workstation runs two or more GPUs in one machine, usually on separate parallel jobs rather than one combined render.
Multi-Instance GPU (MIG)
Multi-Instance GPU (MIG) is NVIDIA's hardware partitioning of one GPU into up to seven isolated instances with dedicated memory and compute.
NPU
An NPU is a dedicated chip block that runs AI inference at low power, rated in TOPS, separate from the CPU and the GPU.
NUMA
NUMA is a memory design where each CPU socket or node reaches its own local memory faster than memory attached to other nodes.
NVIDIA AI Enterprise
NVIDIA AI Enterprise is NVIDIA's supported software platform for production AI, licensed per GPU and released in branches with fixed support windows.
NVIDIA GPU architectures
NVIDIA GPU architectures are the chip design generations, such as Ampere, Hopper, Blackwell, and Rubin, that define a GPU's era and features.
NVIDIA Quadro
Quadro was NVIDIA's workstation GPU brand. It became Quadro RTX, then RTX A-series, RTX Ada Generation and RTX PRO.
NVIDIA vGPU
NVIDIA vGPU is software that shares one physical GPU among several virtual machines using time-sliced or MIG-backed virtual GPUs.
NVIDIA-Certified Systems
NVIDIA-Certified Systems is NVIDIA's program that tests partner servers, workstations and edge systems with specific GPUs and network adapters.
NVL72
NVL72 is NVIDIA's liquid-cooled rack design that joins 72 GPUs and 36 CPUs in one NVLink domain, as in the GB200 NVL72 and GB300 NVL72.
NVLink and NVSwitch
NVLink is Nvidia's direct GPU-to-GPU interconnect. NVSwitch is the crossbar chip that lets many GPUs share it as one any-to-any fabric.
NVLink bridge
An NVLink bridge is a physical connector that links two or four PCIe GPUs directly over NVLink, without a baseboard or NVSwitch.
NVLink-C2C
NVLink-C2C is NVIDIA's chip-to-chip interconnect that gives a Grace CPU and a GPU one coherent memory space at 900 GB/s on GH200.
OAM vs SXM
OAM is the Open Compute Project's open module format for AI accelerators. SXM is NVIDIA's own module format for its data center GPUs.
OCP NIC 3.0
OCP NIC 3.0 is the Open Compute Project spec for server network cards. It has its own slot and card sizes, unlike a standard PCIe add-in card.
OCuLink
OCuLink is a PCIe-based cable interface built on SFF-8611 plugs and SFF-8612 receptacles, in x4 and x8 sizes, for internal and external links.
OEM vs ODM
ODM designs and builds a product from scratch. OEM puts its own brand on that product and sells it.
OSFP vs QSFP
OSFP is an eight-lane module format with its own cage. QSFP-DD is an eight-lane format in a cage that also takes QSFP modules.
Open Compute Project (OCP)
The Open Compute Project (OCP) is a foundation, started by Facebook in 2011, that publishes open data center hardware specs such as Open Rack V3.
Open weights
Open weights are a model's trained parameters released for anyone to download and run, instead of access only through a hosted API.
Out-of-band management
Out-of-band management controls a server through a separate controller and network port, so it works when the host OS is down.
PCIe 4.0 vs PCIe 5.0 vs PCIe 6.0
PCIe 4.0 runs 16 GT/s per lane. PCIe 5.0 doubles that to 32 GT/s. PCIe 6.0 doubles it again to 64 GT/s with PAM4 signaling.
PCIe bifurcation
PCIe bifurcation splits one x16 slot into smaller links such as x8x8 or x4x4x4x4. It is a BIOS setting and adds no lanes.
PCIe lanes
PCIe lanes are the individual data paths linking the CPU to GPUs, drives, and cards. Lane count sets the bandwidth ceiling on a slot.
PCIe power connectors
PCIe power connectors are supplemental cables, 6-pin, 8-pin, and 12V-2x6, that deliver power to a GPU beyond the 75 W the slot itself supplies.
PCIe switch
A PCIe switch is a chip that fans one upstream PCIe link out to several downstream ports, so more devices share a CPU's lanes.
PSU efficiency (80 PLUS)
PSU efficiency is the share of AC input a supply delivers as DC output. 80 PLUS certifies it in tiers from Bronze to Titanium.
Passive vs active GPU cooling
A passive GPU cooler is a fanless heatsink cooled by the server's fans. An active GPU cooler carries its own fan on the card.
Prefill vs decode
Prefill reads an LLM prompt in parallel and is compute-bound. Decode emits one output token per step and is memory-bound.
Production Branch vs New Feature Branch
Production Branch is NVIDIA's long-support RTX Enterprise driver with ISV certification. New Feature Branch trades support length for newer features.
QLC vs TLC
TLC NAND stores 3 bits per cell. QLC stores 4, which adds capacity per cell and cuts endurance and performance.
Quantization
Quantization reduces a model's numerical precision, from 16-bit to 8-bit or 4-bit, shrinking memory use with some loss of accuracy.
RAID
RAID combines multiple physical drives into one logical volume for redundancy, performance, or both, depending on the level used.
RDIMM vs UDIMM
RDIMM adds a register chip that buffers command and address signals for higher capacity. UDIMM skips that chip for lower cost and latency.
RMA
RMA (return merchandise authorization) is the number a vendor issues before it accepts a defective part back for repair, replacement, or credit.
RTX PRO
RTX PRO is Nvidia's brand for workstation and server GPUs, successor to Quadro, with certified drivers and ECC memory.
RTX PRO 6000 Workstation vs Max-Q vs Server Edition
The Workstation Edition runs at 600 W. The Max-Q Edition caps at 300 W. The passive Server Edition is configurable from 400 W to 600 W.
Rack PDU
A rack PDU is the rack-mounted power strip that feeds every server and switch in one cabinet from one or two upstream circuits.
Rack power density
Rack power density is the electrical load of one rack in kW. It equals the heat the room must remove, so it sets the power feed and the cooling.
Rack unit
A rack unit (U) is 1.75 inches of vertical rack height. Server chassis are sized in whole U, most commonly 1U, 2U, and 4U.
Rear door heat exchanger
A rear door heat exchanger is a chilled water coil on a rack's back door. Passive doors use server fans. Active doors add their own.
Redfish
Redfish is the DMTF standard for managing servers through the BMC over an HTTPS REST API with JSON payloads.
Redundant power supply
Two or more power supplies in one server, each able to carry the full load, so the system stays up when one fails or loses input power.
Resizable BAR
Resizable BAR is a PCI Express capability that lets the CPU address a GPU's whole memory through one large base address register.
Retrieval-augmented generation (RAG)
RAG pairs a language model with a document search step, so it answers using retrieved passages instead of only what it learned in training.
SAS vs SATA vs NVMe
SAS, SATA, and NVMe are the three interfaces a drive connects through, and they differ in protocol, speed, and cost.
SKU
SKU (stock keeping unit) is the code identifying one specific, orderable version of a product, distinct from others sharing its name.
SR-IOV
SR-IOV is a PCI Express feature that makes one physical device appear as many virtual devices, each assignable to a different virtual machine.
SXM vs PCIe
SXM is a socketed GPU module built for full NVLink bandwidth. PCIe is the standard add-in card that fits an ordinary server slot.
Single rank vs dual rank
A single-rank module has one chip-select row of DRAM. A dual-rank module has two, and they cannot be accessed at the same time.
Single-slot vs dual-slot GPU
A single-slot GPU fills one expansion slot opening. A dual-slot GPU fills two, which limits how many cards fit in one chassis.
Single-socket vs dual-socket server
A single-socket server has one CPU. A dual-socket server has two, joined by a coherent link, for more cores and memory channels.
Speculative decoding
Speculative decoding is an LLM speedup where a draft proposes tokens and the full model verifies them in parallel, keeping its output distribution.
Storage server
A storage server is built to hold and serve data, spending its budget on drive bays and network throughput instead of CPU or GPU power.
Strix Halo
Strix Halo is AMD's codename for the Ryzen AI Max 300 series, such as the Max+ 395 with a 40-unit GPU and up to 128 GB of shared memory.
Structured sparsity
Structured sparsity is a pruning method, such as NVIDIA's 2:4 pattern, that lets tensor hardware skip zeroed weights for up to 2x matrix throughput.
Studio vs Game Ready driver
Game Ready drivers target day-one game support. Studio drivers are tested against creative apps and ship on a slower cadence.
TAA compliance
TAA compliance means a product was made or substantially transformed in the U.S. or a country designated under the Trade Agreements Act.
TDP
TDP is the heat, in watts, a chip's cooling system must remove at sustained rated clocks. It's a cooling target, not a power draw cap.
TF32
TF32 is an NVIDIA tensor core math mode that runs FP32 matrix multiplies with a 10-bit mantissa for faster AI training.
TFLOPS vs TOPS
TFLOPS counts trillions of floating-point operations per second. TOPS counts trillions of operations per second, usually INT8 or FP4.
Tensor parallelism
Tensor parallelism splits individual weight matrices across multiple GPUs, so each one computes a slice of every layer in a model.
Thermal throttling
Thermal throttling is a GPU or CPU lowering its own clock speed when it hits a temperature or power limit, so work runs slower.
Threadripper PRO vs Xeon W
Threadripper PRO and Xeon W are AMD's and Intel's single-socket workstation CPU platforms, differing in cores, PCIe lanes, and ISV support.
Threadripper vs Threadripper PRO
Threadripper runs on TRX50 boards with four memory channels and 80 PCIe 5.0 lanes. Threadripper PRO runs on WRX90 with eight channels and 128 lanes.
Three-phase power
Three-phase power delivers AC on three lines offset by 120 degrees. Rack PDUs use it to carry more kW per circuit than a single-phase feed.
Time to first token (TTFT)
Time to first token (TTFT) is the delay from sending a request to receiving the first output token, covering queueing and prompt prefill.
Tokens per second
Tokens per second measures how fast a model generates output during inference. It's the standard throughput metric for LLM serving.
Tower vs rack server
A tower server stands upright on a floor or desk. A rack server mounts flat in a 19-inch rack and is sized in rack units.
Transformer
A transformer is the attention-based neural network architecture behind most large language models, including GPT and BERT.
Transformer Engine
Transformer Engine is NVIDIA's open-source library that runs Transformer layers in FP8 and FP4, tracking the scaling factors for you.
Tri-mode controller
A tri-mode controller runs SAS, SATA and NVMe drives from the same ports, so one card and one backplane serve all three.
Unified memory
Unified memory is a single memory pool shared by the CPU and GPU, instead of separate system RAM and dedicated VRAM on a graphics card.
VLA model
A VLA (vision-language-action) model takes camera images and a language instruction and outputs robot actions directly.
VRAM
VRAM is the dedicated memory on a graphics card, separate from system RAM, holding textures, frame buffers, or model weights.
Workstation
A workstation is a desktop or tower built for sustained professional compute work such as CAD, rendering, or AI development.
Workstation GPU vs gaming GPU
Workstation GPUs use the same silicon as gaming cards but add ECC memory, more VRAM, and drivers certified against professional software.
Xeon 6 P-core vs E-core
Xeon 6 P-core parts, Granite Rapids, are the high-clock line with AVX-512 and AMX. E-core parts, Sierra Forest, trade those for more cores per socket.
vLLM vs llama.cpp
vLLM is a Python engine for high-throughput, multi-user LLM serving. llama.cpp is a C/C++ engine for running quantized GGUF models locally.