In the previous post, we examined how differences in Number Formats affect hardware area and power consumption. We established that FP32 is excessively expensive and heavy from a hardware standpoint. Today, we will discuss the technology that puts this heavy data on a diet: Quantization.<\/p>\n\n\n\n
Many people think of quantization simply as a \"compression technique to reduce model file size.\" While reduced file size is true, the real reason System Architects are obsessed with quantization lies elsewhere. It is because of 'Bandwidth'<\/strong> and 'Data Movement.'<\/strong><\/p>\n\n\n\n In this article, we will coldly analyze the physical benefits that occur inside the chip when FP32 (32-bit Floating Point) converts to INT8 (8-bit Integer), and what we lose in return (the Trade-off).<\/p>\n\n\n