Challenge: Client needed to optimize the noise reduction model using int8 and float16 quantization, while ensuring the minimum drop in SDR noise reduction quality. The physical size of the model and the value of BOPS (basic operations per second) were chosen as target performance metrics.
Solution: Quantization (post-training, aware-training, learned side-step) with different initialization strategies (min-max, 99th percentile), Knowledge Distillation (KL loss, STFT Loss, Blockwise, etc.) and their combinations lets us to beat the target metrics
Result:
Results of float32βint8 quantisation
β Reducing the size of models: x25-x60
β Reduction BOPS: x150-x250
β SDR drop, dB: -0.2 - 1.6
Results of float32βfloat16 quantisation
β Reducing the size of models: x10 - x35
β Reduction BOPS: ~x30-80
β SDR drop, dB: -0.1 - 1.4