# DeCo

**Repository Path**: monkeycc/DeCo

## Basic Information

- **Project Name**: DeCo
- **Description**: No description available
- **Primary Language**: Python
- **License**: Not specified
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-12-03
- **Last Updated**: 2025-12-03

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation

<div class="is-size-5 publication-authors", align="center",>
  <span class="author-block">
    <a href="https://zehong-ma.github.io/" target="_blank">Zehong Ma</a><sup>1,3</sup><sup>†</sup>,&nbsp;
  </span>
  <span class="author-block">
    <a href="https://joinwei-pku.github.io/longhuiwei.github.io/" target="_blank">Longhui Wei</a><sup>3</sup><sup>‡</sup><sup>*</sup>,&nbsp;
  </span>
  <span class="author-block">
    <a href="https://scholar.google.com/citations?user=us4prRUAAAAJ&hl=zh-CN" target="_blank">Shuai Wang</a><sup>2</sup>,&nbsp;
  </span>
  <span class="author-block">
    <a href="https://www.pkuvmc.com/" target="_blank">Shiliang Zhang</a><sup>1</sup><sup>*</sup>,&nbsp;
  </span>
  <span class="author-block">
    <a href="https://www.qitian1987.com/" target="_blank">Qi Tian</a><sup>3</sup>
  </span>
</div>

<div class="is-size-5 publication-authors", align="center",>
  <span class="author-block"><sup>1</sup>State Key Laboratory of Multimedia Information Processing, <br>School of Computer Science, Peking University,&nbsp;</span>
  <span class="author-block"><sup>2</sup>Nanjing University,&nbsp;</span>
  <span class="author-block"><sup>3</sup>Huawei Inc.</span>
</div>
<div class="is-size-5 publication-authors", align="center",>
  († Work was done during internship at Huawei., ‡ Project Leader. * Corresponding author.)
</div>

<h5 align="center">

[![hf_paper](https://img.shields.io/badge/🤗-Paper%20In%20HF-red.svg)](https://huggingface.co/papers/2511.19365)
[![arXiv](https://img.shields.io/badge/Arxiv-2511.19365-b31b1b.svg?logo=arXiv)](https://arxiv.org/abs/2511.19365) 
[![Home Page](https://img.shields.io/badge/Project-<Website>-blue.svg)](https://zehong-ma.github.io/DeCo/) 
[![Huggingface](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Online_Demo-green)](https://14467288703cf06a3c.gradio.live) 
[![github](https://img.shields.io/github/stars/Zehong-Ma/DeCo.svg?style=social)](https://github.com/Zehong-Ma/DeCo/)
</h5>

<div class="content has-text-centered">
          <img src="./docs/static/images/t2i_visualization.jpg" style="width: 100%;"><br>
          <span style="font-size: 0.8em; width: 100%; display: inline-block;">Figure 1: Visualization of 512x512 images generated by our DeCo.</span>
</div>


## 🫖 Introduction 
We introduce a novel frequency-decoupled framework DeCo for pixel diffusion, where a lightweight pixel decoder is proposed to model high-frequency signals, freeing the DiT to specialize in low-frequency semantic modeling. 

<div class="content">
            <img src="./docs/static/images/intro.jpg" style="width: 100%;"><br>
</div>

- We achieve **1.62 FID** on ImageNet256x256 Benchmark with DeCo-XL/16.
- We achieve **2.22 FID** on ImageNet512x512 Benchmark with DeCo-XL/16.
- We achieve **0.86 overall score** on GenEval Benchmark with DeCo-XXL/16.
- We achieve **81.4 avergae score** on DPG Benchmark with DeCo-XXL/16.
- **If you like our project, please kindly give us a star ⭐ on GitHub.**

## <img src="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0iIzg4OCI+PHBhdGggZD0iTTEyLDJBNy45LDcuOSwwLDAsMCw0LjEsOS45LDcuOCw3LjgsMCwwLDAsMiwxN2gzYTEsMSwwLDAsMCwxLDFoM3YyYTEsMSwwLDAsMCwxLDFoNHYwYTEsMSwwLDAsMCwxLTFWMThoM2ExLDEsMCwwLDAsMS0xaDNjMC0zLjgtMS42LTcuNS01LjgtOS4yQTguMSw4LjEsMCwwLDAsMTIsMlpNMTQsMTMuNWwtLjUuOGExLDEsMCwwLDEtLjkuNWgtMS4yYTEsMSwwLDAsMS0uOS0uNWwtLjUtLjhDOS4zLDEzLDksMTIuNyw5LDEyLjRzLjMtLjYsLjktMWwuNS0uOGExLDEsMCwwLDEsLjktLjVoMS4yYTEsMSwwLDAsMSwuOS41bC41LjhjLjYuNC45LjcuOSwxUzE0LjYsMTMuMSwxNCwxMy41WiIvPjwvc3ZnPg==" width="20" style="vertical-align: middle;"/> DCT Spectral Analysis 

<div class="content">
            <img src="./docs/static/images/dct_and_FID_comparison.jpg"  style="width: 95%;"><br>
            <span style="font-size: 0.8em; width: 100%; display: inline-block;">DCT energy distribution of DiT outputs and predicted pixel velocities. Compared with baseline, DeCo suppresses high-frequency signals in DiT outputs while preserving strong high-frequency energy in pixel velocity, confirming effective frequency decoupling. The distribution is computed on 10K images across all diffusion steps using DCT transform with 8x8 block size. (b) FID comparison between our DeCo and baseline. DeCo reaches 2.57 FID in 400k iterations, 10× faster than the baseline.
            </span>
          </div>

## 🧩 Visualizations
+ Visualization of more images generated by our text-to-image DeCo.
<div class="content">
            <img src="./docs/static/images/appendix_t2i_figures.jpg" style="width: 100%;"><br>
</div>

+ Visualization of 256*256 images generated by our class-to-image DeCo.
<div class="content">
            <img src="./docs/static/images/c2i_imagenet256.jpg" style="width: 100%;"><br>
</div>

## 🎉 Checkpoints

| Dataset       | Epoch | Model         | Params | FID   | HuggingFace                           |
|---------------|-------|---------------|--------|-------|---------------------------------------|
| ImageNet256    | 320   |   DeCo-XL/16 | 682M   | 1.90  | [🤗](https://huggingface.co/zehongma/DeCo/blob/main/imagenet256_epoch320.ckpt) |
| ImageNet256    | 600   |   DeCo-XL/16 | 682M   | 1.78  | [🤗](https://huggingface.co/zehongma/DeCo/blob/main/imagenet256_epoch600.ckpt) |
| ImageNet256    | 800   |   DeCo-XL/16 | 682M   | 1.62  | [🤗](https://huggingface.co/zehongma/DeCo/blob/main/imagenet256_epoch800.ckpt) |
| ImageNet512 |  340  | DeCo-XL/16 | 682M   | 2.22  | [🤗](https://huggingface.co/zehongma/DeCo/blob/main/imagenet512_epoch340.ckpt) |

| Dataset       | Model         | Params | GenEval | DPG  | HuggingFace                                              |
|---------------|---------------|--------|------|------|----------------------------------------------------------|
| Text-to-Image | DeCo-XXL/16| 1.1B | 0.86 | 81.4| [🤗](https://huggingface.co/zehongma/DeCo/blob/main/t2i_DeCo.ckpt) |
## 🔥 Online Demos
![](./docs/static/images/demo.jpg)
We provide online demos for DeCo-XXL/16(text-to-image) on HuggingFace Spaces.

HF spaces: [https://14467288703cf06a3c.gradio.live](https://14467288703cf06a3c.gradio.live)

To host the local gradio demo, run the following command:
```bash
# for text-to-image applications
python app.py --config ./configs_t2i/sft_res512.yaml --ckpt_path=./ckpts/t2i_DeCo.ckpt
```

## 🤖 Usages
In class-to-image(ImageNet) experiments, We use [ADM evaluation suite](https://github.com/openai/guided-diffusion/tree/main/evaluations) to report FID. 
In text-to-image experiments, we use BLIP3o dataset as training set and utilize GenEval and DPG to collect metrics.

+ Environments
```bash
# for installation (recommend python 3.10)
pip install -r requirements.txt
```

+ Inference
```bash
# for inference
python main.py predict -c ./configs_c2i/DeCo_XL.yaml --ckpt_path=XXX.ckpt
```

+ Train
```bash
# for c2i training
# Please modify the ImageNet1k path in the config file before training.
python main.py fit -c ./configs_c2i/DeCo_XL.yaml

# for 512*512 continuing pretraining
python main.py fit -c ./configs_c2i/DeCo_XL_512.yaml --ckpt_path=/path/to/256/checkpoint/at/320/epochs
```

```bash
# for t2i training
python main.py fit -c ./configs_t2i/pretraining_res256.yaml
python main.py fit -c ./configs_t2i/pretraining_res512.yaml --ckpt_path=./ckpts/pretrain256.ckpt
python main.py fit -c ./configs_t2i/sft_res512.yaml  --ckpt_path=./ckpts/pretrain512.ckpt
```

## 💐 Acknowledgement 

This repository is built based on [PixNerd](https://github.com/MCG-NJU/PixNerd) and [DDT](https://github.com/MCG-NJU/DDT). Thanks for their contributions and [Shuai Wang](https://github.com/WANGSSSSSSS)'s support!

## 📖 Citation 
If you find DeCo is useful in your research or applications, please consider giving us a star ⭐ and citing it by the following BibTeX entry.

```
@misc{ma2025decofrequencydecoupledpixeldiffusion,
      title={DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation}, 
      author={Zehong Ma and Longhui Wei and Shuai Wang and Shiliang Zhang and Qi Tian},
      year={2025},
      eprint={2511.19365},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2511.19365}, 
}
```