Project: Sustainable Diffusion Models
Description
Project
description:
Diffusion
Models are deep-learning models that achieve state-of-the-art performance in
many image synthesis tasks. They are typically parameterized with UNets and
consist of billions of weights. Expressing their weights in float32 leads to
models that cannot be easily deployed on
edge devices (e.g., smartphones) since they take too much space and consume too
much energy. Thus, the problem is to develop and/or apply techniques like quantization-aware
training (QAT), post-training quantization (PTQ), or neural architecture search
for obtaining sustainable diffusion models.
In this
thesis: (a) you will study the techniques for formulating more sustainable diffusion
models, (b) you will formulate and code your own sustainable diffusion models,
(c) you will design and carry out evaluations for your sustainable diffusion
models.
Literature (examples):
- Vasu et al., "FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization", https://arxiv.org/abs/2303.14189
- Shang et al., “Post-training Quantization on Diffusion Models”, https://arxiv.org/abs/2211.15736
- Li et al., “Q-Diffusion: Quantizing Diffusion Models”, https://arxiv.org/abs/2302.04304
- Park et al., “Quadapter: Adapter for GPT-2 Quantization”, https://arxiv.org/abs/2211.16912
- Nagel et al., “Up or Down? Adaptive Rounding for Post-Training Quantization”, https://arxiv.org/abs/2004.10568
- “Optimizing Stable Diffusion for Intel CPUs with NNCF and 🤗 Optimum”, https://huggingface.co/blog/train-optimize-sd-intel
- Tomczak, “Deep Generative Modeling”, https://link.springer.com/book/10.1007/978-3-030-93158-2
Prerequisites:
- reading
and understanding scientific literature
- very good
coding skills in Python using PyTorch and other ML libraries
- good knowledge of Deep Learning and the basics of Generative AI
- curious
attitude, independence, thinking out-of-the box
Details
- Supervisor
-
Jakub Tomczak