RSI-PDEbench: A Benchmark for Self‑Improving Agents in PDE Solving
RSI-PDEbench gives a self-improvement harness one foundation model, a fixed budget and a set of identify-then-solve PDE cases. The harness must turn feedback into a solver program. That solver is then scored, after every round, on a hidden test split that no run ever sees.
Each case gives 256 spatial-temporal samples of an unknown equation's solution, without information about its class and coefficients. A recursive self-improving loop helps agents identify the equation, evolve the correct coefficients and write accurate code to solve the PDE.
Affiliations
- 1Purdue University
- 2University of Illinois Urbana-Champaign
- 3Utah State University
- 4Independent researcher
- 5Brown University
- 6Massachusetts Institute of Technology
- 7Arizona State University
What it measures
Every case is an identify-then-solve problem. A harness is judged by the solver it keeps, on hidden cases it never sees, after every round of self-improvement.
Held-out pass rate
The share of hidden-test cases passed by the solver the harness retained after N rounds. A round is one gated validation submission.
Gain over a single run
The held-out pass rate minus that of a single-run entrant with the same model: one model call, frozen as returned. This is what the self-improvement loop adds.
Dev–test gap
The validation pass rate minus the held-out pass rate of the same solver. A large gap means the loop overfit to the cases it could see.
The RSI-PDE harness
A frozen foundation model improves its solver program P, instruction policy π and memory M. The supervisor, controller and acceptance rules stay fixed.
The RSI-PDE loop
Identify the equation with residual loss, fit its coefficients, solve it, and use public relative L² errors and the validation gate to guide the next refinement.
full0), Noisy uses 50% RMS noise,
and Hard-OOD uses 1% RMS noise. Download PDFWhat's in RSI-PDEbench
There are 42 problem templates. They range from scalar diffusion, transport and fractional problems, through chaotic, incompressible, phase-field and electromagnetic systems, to data-driven problems whose inputs are real fields. Every template appears once in training, once in validation and five times in the hidden test. One seeded generator makes all of them, so the splits share one distribution.
Equation families
Calibration
Before any model was run, fixed solvers from earlier 2-hour runs solved all 1001 cases of the expanded release: 2,856 sandboxed solves, with no model calls. The probes separate clearly. Templates that every strong probe solved were hardened for the benchmark.
The pass rate of each probe solver over all variants of the calibrated templates. The 10 templates of Problems 12–16 had no earlier solver, so the baseline runs calibrate them.
Where the headroom is
Baseline results
Results per PDE
Residual check
How well each hidden-test solution satisfies the true equation, beside its relative L2 error. The residual needs no reference solution. It is an extra measure, not the pass test.
PDE gallery
Every equation family in RSI-PDEbench except the Elliptic equation, with one image per problem template drawn from the reference
solution of its public training case (Kolmogorov flow shows a supplied image): space–time maps for 1-D problems in time (x across, t upward), profiles
for steady 1-D problems and the fractional ODE, and fields for 2-D problems. Where pde_showcase.py
simulates the same equation, its 2-D run joins the family (thumbnails marked with a dot); Gray–Scott and complex
Ginzburg–Landau come from the showcase alone. Select a thumbnail to change the image.
Showcase simulations






Get started
Clone the repository and build the environment. Run your harness, or one of the built-in entrants, on a GPU node with bubblewrap. The maintainers score submissions on the hidden test.
git clone https://github.com/jeremylu916/RSI-PDE.git && cd RSI-PDE pip install -e . export PDE_RSI_PROJECT_ROOT=$PWD # the code defaults to the original cluster path # run a built-in entrant on all 42 templates (RSI-PDE, Claude Code or Codex; six cores, a GPU) python scripts/bench.py run --entrant pde-rsi-v3.4 --model opus55 --cores 0,1,2,3,4,5 --output runs/bench-v1-my-run # maintainers: score every round on the hidden test, then build the table python scripts/bench.py score --runs runs/bench-v1-my-run --cores 6,7,8,9 python scripts/bench.py report
Specification
docs/rsi-pde-bench.md: the rules, budget, splits, scoring and calibration.
Datasheet
Composition, provenance and licence. The DiffusionPDE inputs are CC BY-NC-SA 4.0.
Equations
pde_benchmarks.tex (Problems 1–16) and the data-driven problems.