# PID

**Repository Path**: xdd0404/pid

## Basic Information

- **Project Name**: PID
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: MIT
- **Default Branch**: main
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2025-04-14
- **Last Updated**: 2025-04-14

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# PID: Physics-Informed Diffusion Model for Infrared Image Generation

<img src="PID.png" alt="PID" style="zoom:50%;" />

## Paper

The paper is under review: https://arxiv.org/abs/2407.09299

## Environment

We recommend you to install the environment with environment.yaml. 

```bash
conda env create --file=environment.yaml
```

## Datasets

Download KAIST dataset from https://github.com/SoonminHwang/rgbt-ped-detection.

Download FLIRv1 dataset from https://www.flir.com/oem/adas/adas-dataset-form/.

We adopt the official dataset split in our experiments.

## Checkpoint

VQGAN can be downloaded from https://ommer-lab.com/files/latent-diffusion/vq-f8.zip (Other GAN models can be downloaded from https://github.com/CompVis/latent-diffusion).

|  Name  |                   Note                   |                             Link                             |
| :----: | :--------------------------------------: | :----------------------------------------------------------: |
| TeVNet | TeVNet checkpoint for KAIST, epoch=0.95k | [TeVNet_KAIST.zip](https://1drv.ms/u/s!AiBcDdKjVk0ubr49dgvt-R5s6jM?e=7hpRLS) |
| TeVNet |   TeVNet checkpoint for FLIR, epoch=1k   | [TeVNet_FLIR.zip](https://1drv.ms/u/s!AiBcDdKjVk0ubV_nT3EIDnzJ10Y?e=bQeXc4) |
|  PID   |  PID checkpoint for KAIST, k1=50, k2=5   | [PID_KAIST.zip](https://1drv.ms/u/s!AiBcDdKjVk0ub0BI3aQGHxxQf-g?e=6LlzBE) |
|  PID   |  PID checkpoint for KAIST, k1=50, k2=5, resume   | [PID_KAIST_resume.zip](https://1drv.ms/u/c/2e4d56a3d20d5c20/EbqLPzlm4JNBosiswq7uEf4BG-ea-XW-0ERQGM6drM5YHQ?e=ifrEBb) |
|  PID   |    PID checkpoint for FLIR, k1=k2=50     | [PID_FLIR.zip](https://1drv.ms/u/s!AiBcDdKjVk0ucJF2M7FwuVhJoiU?e=Hfm2B9) |

## Evaluation

Use the shellscript to evaluate. `indir` is the input directory of visible RGB images, `outdir` is the output directory of translated infrared images, `config` is the chosen config in `configs/latent-diffusion/config.yaml`.  We prepare some RGB images in `dataset/KAIST` for quick evaluation.

```sh
bash run_test_kaist512_vqf8.sh
```

## Train

### Dataset preparation

Prepare corresponding RGB and infrared images with same names in two directories.

### Stage 1: Train TeVNet

```bash
cd TeVNet
bash shell/train.sh
```

### Stage 2: Train PID

Use the shellscript to train. It is recommended to use our pretrained model to accelerate the train process.

```bash
bash shell/run_train_kaist512_vqf8.sh
```

## Acknowledgements

Our code is built upon [LDM](https://github.com/CompVis/latent-diffusion) and [HADAR](https://github.com/FanglinBao/HADAR). We thank the authors for their excellent work.

## Citation

If you find this work helpful in your research, please consider citing our paper:

```
@inproceedings{Mao2024PIDPD,
  title={PID: Physics-Informed Diffusion Model for Infrared Image Generation},
  author={Fangyuan Mao and Jilin Mei and Shun Lu and Fuyang Liu and Liang Chen and Fangzhou Zhao and Yu Hu},
  year={2024},
  url={https://doi.org/10.48550/arXiv.2407.09299}
}
```

If you have any question, feel free to contact maofangyuan23s@ict.ac.cn .