模型量化与边缘推理：从 FP32 到 INT4 的精度-速度权衡

左手厨刀右手茼蒿

976人浏览 · 2026-06-09 11:58:02

左手厨刀右手茼蒿 · 2026-06-09 11:58:02 发布

模型量化与边缘推理：从 FP32 到 INT4 的精度-速度权衡

cover

一、边缘推理的"算力天花板"：模型太大，设备太弱

大模型在云端跑得很好，但部署到边缘设备（手机、IoT、车载）就遇到瓶颈：一个 7B 参数的 LLM，FP32 精度下需要 28GB 内存，而手机只有 8-12GB 可用内存（还要分给操作系统和其他应用）。即使勉强装下，推理速度也只有每秒 1-2 个 Token，用户体验极差。

模型量化的核心思想是"用更少的位表示权重和激活值"。FP32→FP16→INT8→INT4，每次量化都减少内存占用和计算量，但代价是精度损失。量化的关键不是"能不能量化"，而是"量化后精度损失是否可接受"——不同任务对精度的容忍度不同，分类任务可以容忍更多损失，生成任务对精度更敏感。

二、量化方法对比

graph TB
    subgraph 量化方法
        A[训练后量化 PTQ<br/>无需重训练]
        B[量化感知训练 QAT<br/>训练时模拟量化]
        C[混合精度量化<br/>敏感层FP16+其他INT8]
    end

    subgraph 精度影响
        A --> D[INT8: 精度损失1-3%<br/>INT4: 精度损失5-15%]
        B --> E[INT8: 精度损失<1%<br/>INT4: 精度损失2-5%]
        C --> F[接近原始精度<br/>部分层保持高精度]
    end

    subgraph 适用场景
        D --> G[快速部署<br/>资源有限]
        E --> H[精度要求高<br/>有训练资源]
        F --> I[精度敏感任务<br/>如生成式模型]
    end

PTQ（Post-Training Quantization）最简单——训练完成后直接量化，不需要重训练。QAT（Quantization-Aware Training）在训练时模拟量化误差，让模型适应低精度表示，精度更好但需要训练资源。混合精度对敏感层保持高精度，非敏感层使用低精度，兼顾精度和速度。

三、量化实现

3.1 训练后量化（PTQ）

import numpy as np
from typing import Tuple

class PostTrainingQuantizer:
    """训练后量化器"""

    @staticmethod
    def quantize_tensor(
        tensor: np.ndarray, bits: int = 8
    ) -> Tuple[np.ndarray, float, float]:
        """
        对称量化：将 FP32 张量量化为 N-bit 整数。
        返回量化后的张量、缩放因子和零点。
        """
        # 计算量化范围
        abs_max = np.max(np.abs(tensor))
        qmax = (2 ** (bits - 1)) - 1  # 对称量化范围

        # 缩放因子
        scale = abs_max / qmax if abs_max > 0 else 1.0

        # 量化
        quantized = np.clip(
            np.round(tensor / scale),
            -qmax - 1, qmax
        ).astype(np.int8 if bits == 8 else np.int32)

        return quantized, scale, 0.0  # 对称量化零点为 0

    @staticmethod
    def dequantize_tensor(
        quantized: np.ndarray, scale: float, zero_point: float
    ) -> np.ndarray:
        """反量化：将整数张量还原为浮点"""
        return (quantized.astype(np.float32) - zero_point) * scale

    def quantize_model(
        self, weights: dict, bits: int = 8
    ) -> dict:
        """量化模型权重"""
        quantized_weights = {}
        quantization_params = {}

        for name, param in weights.items():
            q_param, scale, zp = self.quantize_tensor(
                param, bits
            )
            quantized_weights[name] = q_param
            quantization_params[name] = {
                'scale': scale,
                'zero_point': zp,
                'bits': bits,
            }

        # 计算压缩率
        original_size = sum(
            p.nbytes for p in weights.values()
        )
        quantized_size = sum(
            p.nbytes for p in quantized_weights.values()
        )
        compression_ratio = original_size / quantized_size

        return {
            'weights': quantized_weights,
            'params': quantization_params,
            'original_size_mb': original_size / 1024 / 1024,
            'quantized_size_mb': quantized_size / 1024 / 1024,
            'compression_ratio': f'{compression_ratio:.1f}x',
        }

3.2 精度评估

class QuantizationEvaluator:
    """量化精度评估器"""

    def evaluate(
        self,
        original_outputs: np.ndarray,
        quantized_outputs: np.ndarray,
        task_type: str = "classification",
    ) -> dict:
        """评估量化前后的精度差异"""
        # 均方误差
        mse = np.mean(
            (original_outputs - quantized_outputs) ** 2
        )

        # 余弦相似度
        cos_sim = self._cosine_similarity(
            original_outputs.flatten(),
            quantized_outputs.flatten(),
        )

        # 信噪比
        signal_power = np.mean(original_outputs ** 2)
        noise_power = mse
        snr = 10 * np.log10(
            signal_power / noise_power if noise_power > 0 else float('inf')
        )

        results = {
            'mse': float(mse),
            'cosine_similarity': float(cos_sim),
            'snr_db': float(snr),
        }

        # 分类任务：准确率对比
        if task_type == "classification":
            orig_preds = np.argmax(original_outputs, axis=-1)
            quant_preds = np.argmax(quantized_outputs, axis=-1)
            accuracy_drop = 1 - np.mean(
                orig_preds == quant_preds
            )
            results['accuracy_drop'] = float(accuracy_drop)

        # 生成任务：Token 一致率
        elif task_type == "generation":
            orig_tokens = np.argmax(original_outputs, axis=-1)
            quant_tokens = np.argmax(quantized_outputs, axis=-1)
            token_agreement = np.mean(
                orig_tokens == quant_tokens
            )
            results['token_agreement'] = float(token_agreement)

        return results

    @staticmethod
    def _cosine_similarity(a: np.ndarray, b: np.ndarray) -> float:
        norm_a = np.linalg.norm(a)
        norm_b = np.linalg.norm(b)
        if norm_a == 0 or norm_b == 0:
            return 0.0
        return float(np.dot(a, b) / (norm_a * norm_b))

3.3 混合精度量化

class MixedPrecisionQuantizer:
    """混合精度量化器：敏感层保持高精度"""

    def analyze_sensitivity(
        self,
        model_weights: dict,
        calibration_data: np.ndarray,
    ) -> dict:
        """分析每层对量化的敏感度"""
        sensitivity = {}

        for name, weight in model_weights.items():
            # 对每层分别量化，测量输出偏差
            fp32_output = self._forward_single_layer(
                weight, calibration_data
            )

            # INT8 量化
            q8_weight, scale, zp = (
                PostTrainingQuantizer.quantize_tensor(weight, 8)
            )
            dq8_weight = PostTrainingQuantizer.dequantize_tensor(
                q8_weight, scale, zp
            )
            int8_output = self._forward_single_layer(
                dq8_weight, calibration_data
            )

            # 计算输出偏差
            output_diff = np.mean(
                np.abs(fp32_output - int8_output)
            )
            sensitivity[name] = {
                'output_diff': float(output_diff),
                'recommended_bits': (
                    16 if output_diff > 0.1 else
                    8 if output_diff > 0.01 else 4
                ),
            }

        return sensitivity

    def _forward_single_layer(
        self, weight: np.ndarray, data: np.ndarray
    ) -> np.ndarray:
        """单层前向传播（简化）"""
        return data @ weight.T