RL-Restore: A Reinforcement-Learning Toolchain for Image Restoration
I reimplemented a CVPR 2018 paper where a deep-RL agent repairs photos by chaining small specialist CNNs, then extended it with baselines, the paper's unreleased fine-tuning, a generative finisher, and a live web app that shows every step.
12
Specialist CNN tools
+2.93 dB
Agent PSNR gain
~90%
Of oracle ceiling
Yes
Live demo
Hypothesis
When a photo is damaged by an unknown mix of blur, noise, and JPEG artifacts, can a reinforcement-learning agent pick a short, ordered chain of small specialist filters that beats any single filter, and is the result worth the extra machinery compared to one end-to-end network?
- Problem
- Real photos carry a mix of blur, noise and JPEG damage at unknown strengths, and no single fixed filter handles all of them. The original paper never released its joint fine-tuning step and never compared against a single network of the same size.
- My role
- Solo course project: reimplemented Yu et al. (CVPR 2018) end to end and extended it.
- Result
- The agent repairs test patches by +2.93 dB, 0.59 dB (25%) more than the best single tool and 90% of the oracle-with-stop ceiling; the paper's unreleased joint fine-tuning lifts it to +3.67 dB.
Honest limits
- A single CNN of the same size repairs more (+4.03 dB); the toolchain's appeal is that each step is visible (not measured), not peak fidelity.
- PSNR is measured on 63×63 test patches drawn from 50 held-out images, from one training seed.
- The agent's dispatcher was trained on moderate damage only but is scored on all three strengths, while the single CNN trained on all three, which favours the CNN.

Runs right here on this page. It loads from Hugging Face, which may set its own cookies, and a sleeping demo takes 30 to 60 seconds to wake up.
No single filter fixes mixed damage
A denoiser blurs away real edges. A deblurrer amplifies noise. A JPEG-artifact remover does nothing for either. Real photos carry all three at once, at strengths you do not know in advance, so the right fix is a sequence of small corrections rather than one big one.
This is a reimplementation of Yu, Dong, Lin, and Loy, "Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning" (CVPR 2018). I built it end to end for a computer-vision course: the 12 tools, the agent, the baselines, the fine-tuning, and the web app.
Twelve specialist tools, one agent that chains them
The toolbox is 12 small CNNs. Each is trained to fix one kind of damage at one strength, so each is good at exactly one thing and useless at the rest. On its own that is a limitation. Chained by a controller, it becomes a vocabulary.
The controller is a DQN with an LSTM. It looks at a 63x63 crop of the damaged photo, picks one tool, applies it to the full image, looks again, and either picks the next tool or stops. Chains run up to three tools. The LSTM is what lets the agent remember what it has already done and adapt the rest of the chain to the image in front of it. Reported scores are on 63x63 test patches; the full-image path is the demo.
Where the agent lands, honestly
Held-out DIV2K test split. Every number is PSNR gain in decibels over the damaged input, averaged across mild, moderate, and severe degradation. Higher is better.
| Method | Gain | Note |
|---|---|---|
| Best single tool | +2.34 | The strongest one-shot specialist. The floor any chain should beat. |
| RL agent (DQN+LSTM, up to 3 tools) | +2.93 | +0.59 dB (25%) more than the best single tool. Reaches about 90% of the oracle-with-stop ceiling. |
| Oracle with stop (best possible chain) | +3.24 | The ceiling for this toolbox if you could always pick the perfect chain. |
| Agent after joint fine-tuning | +3.67 | Retraining tools on the mid-chain images they actually see. A +0.74 dB lift, about 3x the paper's reported 0.25. |
| Single CNN baseline (448k params) | +4.03 | One end-to-end network of the same size. Higher raw PSNR than the toolchain. |
The honest takeaway: the agent reaches about 90% of the oracle-with-stop ceiling and gains 0.59 dB (25%) more than the best single tool, but a single end-to-end CNN of the same size scores higher on raw PSNR. So the toolchain's appeal is not peak fidelity: it is that each step is visible (this was not measured). The comparison also favours the CNN, which trained on all three damage strengths while the agent's dispatcher trained on moderate damage only. Pixel-loss models also come out slightly soft from regression to the mean, which is why the generative finisher exists.
What I added
Single-CNN baseline
One end-to-end network of the same parameter budget, so the toolchain is measured against an honest alternative, not just against itself.
Joint fine-tuning (the paper's Algorithm 1)
Never released by the authors. It retrains the tools on the actual mid-chain images they encounter, not clean degradations. Lifts the same agent from +2.93 to +3.67.
Generative Enhance & Upscale 4x
A Real-ESRGAN finisher that invents plausible detail to look crisp. Shown without a PSNR number, because that detail is synthesized, not recovered.
Live web app
A FastAPI + React demo that plays back the agent's decisions one tool at a time, with the quality meter climbing as it goes.
The agent adapts the chain to the image
Two held-out test crops, left to right: the damaged input, each tool the agent applied in order (with the running PSNR), and the ground truth. The agent picks a different chain for each image.


Top: here the agent should have stopped after the first tool (+3.16 dB); the two later steps lowered the score to +2.20 dB. Bottom: a heavily degraded crop gets a single strong denoise and a +4.45 dB gain. Same agent, different decision per image.
Watch the agent restore a photo, one tool at a time
A self-contained FastAPI + React app, no Gradio. Pick a method, upload or snap a photo, and watch the agent restore it step by step with the quality meter climbing. The optional Enhance & Upscale 4x step runs Real-ESRGAN.
Loads RL-Restore from huggingface.co, which may set its own cookies. A sleeping demo takes 30 to 60 seconds to wake up.
Frameworks and infrastructure
PyTorch for the 12 tools, the DQN+LSTM agent, the baselines, and joint fine-tuning. FastAPI + React + TypeScript + Docker for the web app, deployed on a free HuggingFace CPU Space. About 200 tests. Source on GitHub.
Based on Yu, Dong, Lin, Loy, "Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning," CVPR 2018 (arXiv:1804.03312). The Enhance step uses Real-ESRGAN (Wang et al., 2021). My own code is MIT-licensed.