Tag: LLM inference

Quantized AI Models Explained: What FP16, FP8, FP4 (and 4-bit) Really Mean for Your GPU, VRAM, and Unified Memory

Quantized AI models store weights (and sometimes activations/cache) using fewer bits—like FP8 or FP4 instead of FP16—cutting VRAM/unified memory requirements…