Optimizing Stable Diffusion Models for Edge Devices

Optimize Stable Diffusion for edge devices through pruning, distillation, quantization, model format conversion, and deployment on Orange Pi.

As a Machine Learning Engineer on this capstone project at Amazon.com, Inc., I worked on a pipeline spanning pruning, distillation, quantization, model format conversion, and deployment on Orange Pi. I also reproduced Q-Diffusion and implemented timestep-aware calibration.

The optimization workflow began with pruning and distillation, followed by quantization before model conversion and deployment.

Applied mixed quantization to reduce U-Net model size by 50% while maintaining image quality.

Converted PyTorch models to ONNX with static quantization and deployed them on Orange Pi using the RK3588 NPU.

Reduced inference latency by 55% through Stable Diffusion model quantization, alongside a 50% reduction in U-Net model size while maintaining image quality.