Skip to content
All projects
2026

RL-Restore: A Reinforcement-Learning Toolchain for Image Restoration

I reimplemented a CVPR 2018 paper where a deep-RL agent repairs photos by chaining small specialist CNNs, then extended it with baselines, the paper's unreleased fine-tuning, a generative finisher, and a live web app that shows every step.

12

Specialist CNN tools

+2.93 dB

Agent PSNR gain

~90%

Of oracle ceiling

Yes

Live demo

Hypothesis

When a photo is damaged by an unknown mix of blur, noise, and JPEG artifacts, can a reinforcement-learning agent pick a short, ordered chain of small specialist filters that beats any single filter, and is the result worth the extra machinery compared to one end-to-end network?

Problem
Real photos carry a mix of blur, noise and JPEG damage at unknown strengths, and no single fixed filter handles all of them. The original paper never released its joint fine-tuning step and never compared against a single network of the same size.
My role
Solo course project: reimplemented Yu et al. (CVPR 2018) end to end and extended it.
Result
The agent repairs test patches by +2.93 dB, 0.59 dB (25%) more than the best single tool and 90% of the oracle-with-stop ceiling; the paper's unreleased joint fine-tuning lifts it to +3.67 dB.

Honest limits

  • A single CNN of the same size repairs more (+4.03 dB); the toolchain's appeal is that each step is visible (not measured), not peak fidelity.
  • PSNR is measured on 63×63 test patches drawn from 50 held-out images, from one training seed.
  • The agent's dispatcher was trained on moderate damage only but is scored on all three strengths, while the single CNN trained on all three, which favours the CNN.
Screenshot of the live demo showing the agent's repair steps
Try it live

Runs right here on this page. It loads from Hugging Face, which may set its own cookies, and a sleeping demo takes 30 to 60 seconds to wake up.

The problem

No single filter fixes mixed damage

A denoiser blurs away real edges. A deblurrer amplifies noise. A JPEG-artifact remover does nothing for either. Real photos carry all three at once, at strengths you do not know in advance, so the right fix is a sequence of small corrections rather than one big one.

This is a reimplementation of Yu, Dong, Lin, and Loy, "Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning" (CVPR 2018). I built it end to end for a computer-vision course: the 12 tools, the agent, the baselines, the fine-tuning, and the web app.

How it works

Twelve specialist tools, one agent that chains them

The toolbox is 12 small CNNs. Each is trained to fix one kind of damage at one strength, so each is good at exactly one thing and useless at the rest. On its own that is a limitation. Chained by a controller, it becomes a vocabulary.

The controller is a DQN with an LSTM. It looks at a 63x63 crop of the damaged photo, picks one tool, applies it to the full image, looks again, and either picks the next tool or stops. Chains run up to three tools. The LSTM is what lets the agent remember what it has already done and adapt the rest of the chain to the image in front of it. Reported scores are on 63x63 test patches; the full-image path is the demo.

Results

Where the agent lands, honestly

Held-out DIV2K test split. Every number is PSNR gain in decibels over the damaged input, averaged across mild, moderate, and severe degradation. Higher is better.

MethodGainNote
Best single tool+2.34The strongest one-shot specialist. The floor any chain should beat.
RL agent (DQN+LSTM, up to 3 tools)+2.93+0.59 dB (25%) more than the best single tool. Reaches about 90% of the oracle-with-stop ceiling.
Oracle with stop (best possible chain)+3.24The ceiling for this toolbox if you could always pick the perfect chain.
Agent after joint fine-tuning+3.67Retraining tools on the mid-chain images they actually see. A +0.74 dB lift, about 3x the paper's reported 0.25.
Single CNN baseline (448k params)+4.03One end-to-end network of the same size. Higher raw PSNR than the toolchain.

The honest takeaway: the agent reaches about 90% of the oracle-with-stop ceiling and gains 0.59 dB (25%) more than the best single tool, but a single end-to-end CNN of the same size scores higher on raw PSNR. So the toolchain's appeal is not peak fidelity: it is that each step is visible (this was not measured). The comparison also favours the CNN, which trained on all three damage strengths while the agent's dispatcher trained on moderate damage only. Pixel-loss models also come out slightly soft from regression to the mean, which is why the generative finisher exists.

Beyond the paper

What I added

Single-CNN baseline

One end-to-end network of the same parameter budget, so the toolchain is measured against an honest alternative, not just against itself.

Joint fine-tuning (the paper's Algorithm 1)

Never released by the authors. It retrains the tools on the actual mid-chain images they encounter, not clean degradations. Lifts the same agent from +2.93 to +3.67.

Generative Enhance & Upscale 4x

A Real-ESRGAN finisher that invents plausible detail to look crisp. Shown without a PSNR number, because that detail is synthesized, not recovered.

Live web app

A FastAPI + React demo that plays back the agent's decisions one tool at a time, with the quality meter climbing as it goes.

See it work

The agent adapts the chain to the image

Two held-out test crops, left to right: the damaged input, each tool the agent applied in order (with the running PSNR), and the ground truth. The agent picks a different chain for each image.

Mild degradation restored in three steps: degraded 19.93 dB, denoise4, deblur3, denoise3, then ground truth, for a +2.20 dB agent gain; stopping after the first tool would have scored +3.16 dB
Severe degradation restored in one step: degraded 24.61 dB, denoise3, then ground truth, for a +4.45 dB agent gain

Top: here the agent should have stopped after the first tool (+3.16 dB); the two later steps lowered the score to +2.20 dB. Bottom: a heavily degraded crop gets a single strong denoise and a +4.45 dB gain. Same agent, different decision per image.

Live demo

Watch the agent restore a photo, one tool at a time

A self-contained FastAPI + React app, no Gradio. Pick a method, upload or snap a photo, and watch the agent restore it step by step with the quality meter climbing. The optional Enhance & Upscale 4x step runs Real-ESRGAN.

Loads RL-Restore from huggingface.co, which may set its own cookies. A sleeping demo takes 30 to 60 seconds to wake up.

Tech stack

Frameworks and infrastructure

PythonPyTorchDQN + LSTMCNNReal-ESRGANFastAPIReact + TypeScriptDocker

PyTorch for the 12 tools, the DQN+LSTM agent, the baselines, and joint fine-tuning. FastAPI + React + TypeScript + Docker for the web app, deployed on a free HuggingFace CPU Space. About 200 tests. Source on GitHub.

Based on Yu, Dong, Lin, Loy, "Crafting a Toolchain for Image Restoration by Deep Reinforcement Learning," CVPR 2018 (arXiv:1804.03312). The Enhance step uses Real-ESRGAN (Wang et al., 2021). My own code is MIT-licensed.