leaderboard
Decompilers have advanced significantly over
the last 30 years, quickly approaching the
point where they can recover the exact source code from various binaries. This
benchmark ranks decompilers by their ability to recover exact source code,
measured across three metrics. All metrics are shown as the percentage of
functions on which a decompiler achieves a perfect score. Decompilers are
initially ranked by Union — their ability to score perfectly on
at least one of those metrics. Click a column to sort.
AI can also compete on these metrics, as seen on the
sample-set leaderboard
where Codex and Claude Code take on the traditional decompilers. You can find
more information about these metrics, datasets, and methodology on the
about page. You can also view some sample results
on the view page.
data
Benchmark-run data beyond the leaderboard’s perfect rates:
distance from perfection per metric,
how often decompiled output compiles again,
pipeline health (what our own tooling loses), and the
cost of producing each decompiler’s output.
distance
When a decompiler can’t yet achieve a perfect score on a function, it can be
helpful to understand the distance it is from perfection. For each
metric, we measure distance as the number of edits required to convert that
form of data into its source-code equivalent. For GED, that
is the number of edits to control-flow structures. For
types, that is the number of type-flips needed to reach
ground truth. For recompilation, that is the number of
assembly lines that must change to convert the recompiled assembly into the
ground-truth assembly.
Each cell shows the mean, the median, and how many functions
are already at distance 0 (perfect), averaged over the functions each
decompiler was scored on.
Over the selected dataset
(mean · median · #at-0 / #measured).
rows below the
break are sample-set-only backends (LLM coding agents): they are scored
only on the ~250-function sample-set slice, so on this dataset their numbers
cover just its overlap with that slice — not directly comparable to the
full-coverage rows above.
compiles
The share of each decompiler’s byte_match-measured functions whose output
actually recompiled after the uniform compilability-fixup pass — a
fairness control, not a metric (type recovery is scored separately). The
denominator is per-decompiler: functions where byte_match was measurable, so
ARM / PE targets with no host recompiler never count against it. This rate moves
with the selected dataset, like the columns on the leaderboard.
rows below the
break: sample-set-only backends — their rate covers only this dataset’s
overlap with the sample-set slice.
pipeline health (our own tooling)
GED depends on Joern parsing both the source and the decompiler output.
When Joern fails on the source, that’s our tooling — those
functions are excluded from GED for every decompiler (never counted against
them). When Joern fails on a single decompiler’s output,
that’s reported here (per decompiler), not folded into the headline score.
cost
What each decompiler’s output costs to produce. The two halves of the table
are not directly comparable: traditional decompilers are timed from
whole-binary batch decompilation (a binary’s wall time divided by its function
count), while the LLM coding agents are timed per function — one agentic call
each, including all their tool use (objdump runs, reasoning, retries). The
dollar figures are estimates: recorded token usage from the sample-set run,
priced at public list prices at render time — not billed amounts. - means
not applicable or no data (traditional decompilers have no per-token cost;
an unpriced model shows n/a rather than $0.00).
view
Original source next to a decompiler’s output. Difficulty is
derived from structural (GED) agreement across decompilers:
easy — most decompilers recover the control flow
perfectly; hard — the functions farthest from perfect
for everyone (the old hall of shame); medium — in
between. Pick a difficulty, a decompiler, and a metric to highlight.
decbench
Over the last 30 years, binary decompilers have made the steady march towards perfect decompilation: where decompilers recover the exact source code.
However, that perfect has yet to be measured meaningfully, and is often defined across multiple axes.
DecBench is an experimental benchmark for comparing decompilers and modern LLMs on the task of recovering exact source code. This benchmark uses new and previously known metrics (perfect match percentage) and datasets that represent the various directions of exactness for decompilers: control flow structure, types, and precise recompilability.
This benchmark is also living: as new decompiler/LLMs are released, their scores will be added to the leaderboard! Community feedback is welcome!
It is created by the Noelo Lab at the University of Georgia, led by Dr. Zion Leonahenahe Basque.
The project’s code and data are open source.
Email [email protected] to get your decompiler added to the public site.
why
There have been two other academic benchmarks in the past for end-to-end decompilers: Decompile-Bench and DecompileBench (yes, I know the names are confusing).
These works establish their own take on the problem, and are worth a read.
Both have limitations: one relies on readability metrics (which we’ve shown in prior work to be flawed) the other on metrics that sacrifice correctness for other achievements (passing a subset of testcases on recompilation).
DecBench uses metrics that place correctness as a first-class citizen and attempt to measure perfection on a function-level rather than a macro level.
Ideally, perfect decompilation is that which is correct.
Each metric you will find here attempts to measure the correctness across three widely explored areas.
Once a decompiler approaches 100% on all three metrics, you can consider that it nearly always recovers perfect decompilation (at least for this dataset!).
DecBench is also a response to static-benchmarks that often get outdated or fail to change when flaws are discovered.
It is very likely there is bugs in the code that runs DecBench, or there is projects which pull too much weight than others.
DecBench aims to change as the community changes: adding new decompilers, updating versions, and improving supported projects.
As such, this is a living project, widely different from a paper and more similar to the popular SWEBench.
the three metrics
We define three metrics that explore the three areas we believe are representative of “perfect” decompilation.
- Code Structure
- Types (args and vars)
- Byte-match Recompilability
There are cases where these three metrics can conflict with each other.
As such, decompilers are scored by Union: the overlap where at least one of these metrics is perfect.
None of these metrics, themselves are perfect and all come with limitations, ironically.
That speaks to the difficulty of measuring this field.
Find the extended metrics limitations below.
[1]Control-flow structure correctness
metric: Graph Edit Distance
Does the decompiled code branch and loop the same way the source does? We compare the control-flow graphs of the source and the decompilation with a Graph Edit Distance (GED) — the number of node/edge insertions, deletions, and substitutions needed to turn one CFG into the other.
how GED works: source → CFG → graph diff
A · lift the source to a control-flow graph
source .c
— joern →
control-flow graph
(same lift is applied to every decompiler’s C output)
source.c
// sum of |x[i]|
int sum_abs(int *x, int n) {
int i, s = 0;
for (i = 0; i < n; i++) {
if (x[i] < 0)
s -= x[i];
else
s += x[i];
}
return s;
}control-flow graph
green = the reference shape (dashed edge = loop back-edge)
B · the edit distance: reference vs a decompiler’s CFG
source CFG
5 nodes · 6 edges
decompiled CFG
matched nodes grey · inserted node + 2 edges red
GED = 3 — 1 node insertion + 2 edge insertions to align the two CFGs
GED = 0 → the two CFGs are graph-isomorphic — a perfect structural match.
Only control-flow shape is scored; node labels are ignored, so the signal is fair across decompilers.
Structural Correctness (GED): Joern lifts both the original source and each decompiler’s C output to control-flow graphs, then counts the fewest node/edge edits needed to make them isomorphic — 0 means an identical shape. Only control structure is scored, so the signal is fair across decompilers.
perfect = GED of 0 (graph-isomorphic control flow).
[2]Type correctness
metric: Type Correctness
Did the decompiler recover the right variable and argument types? We match the decompiled variables against DWARF ground truth (arguments by ABI position, stack variables by calibrated offset, the rest by name) and score the fraction recovered correctly.
how type matching works: DWARF ground truth recovered variables
[1] arguments by ABI position (name-independent)
[2] stack vars by calibrated frame offset
[3] remainder by exact name
✓ correct type · ≠ type mismatch · ✗ missed
score = matched-correct / recoverable = 2 / 4 = 0.50
1.0 → every recoverable variable typed correctly = perfect
Only variables carrying a DWARF location count as recoverable — fully optimized-out vars are dropped for everyone, so the denominator is identical across decompilers. Arguments match by ABI position (name-independent, so angr’s a1/a2 get fair credit), stack locals by an auto-calibrated frame-offset shift (here +0x10, so ground-truth -0x18 aligns to the decompiler’s -0x28), and the remainder by exact name.
perfect = 1.0 (every recoverable variable typed correctly).
[3]Recompilation correctness
metric: Recompilation Bytematch
Does the decompiled code recompile to the same machine code? We run a uniform compilability fixup (define decompiler pseudo-types, strip illegal symbol-version tokens, declare missing symbols) so every decompiler gets a fair shot at building, recompile each function with the original toolchain, and compare the resulting assembly — normalizing link-time-dependent operands (call/jump targets, PC-relative offsets) so only real differences count.
The leaderboard’s Compiles column reports the first half of this on its own — the share of a decompiler’s output that the fixup got to build at all (before any assembly comparison). It is measured only where a matching recompiler exists (x86); ARM/PE firmware and malware abstain rather than count as failures.
how bytematch works: fixup → recompile → normalized asm diff
A · rebuild the decompiler’s own C, the same way the original was built
decompiled .c
— compilability fixup →
buildable .c
— recompile →
assembly
(same toolchain & -O flags as the source: x86→gcc, ARM→arm-eabi, PE→MinGW)
decompiled .c — pseudo-types injected by the fixup
undefined4 scale(int a) {
uint x = a * 3;
log_val(x);
return x + limit;
}The fixup adds only typedefs for undefined4/uint — never rewrites logic
recompiled assembly (-O2, x86-64)
scale:
push rbx
imul ebx, edi, 3
mov edi, ebx
call log_val
mov eax, ebx
add eax, [rip+limit]
pop rbx
retB · diff the recompiled bytes against the original .text
original .textrecompiled
✓push rbxpush rbx
✗lea ebx, [rdi+rdi*2]imul ebx, edi, 3
✓mov edi, ebxmov edi, ebx
≈call ____call ____
✓mov eax, ebxmov eax, ebx
≈add eax, [rip+____]add eax, [rip+____]
✓pop rbxpop rbx
✓retret
✓ identical line
≈ matches after normalizing a link-time operand (____ = call target / [rip+disp])
✗ real difference (edit distance = changed asm lines)
byte_match = matching / total = 7 / 8 = 0.88
1.0 → recompiled assembly matches the original = perfect
Recompilation Bytematch rebuilds the decompiler’s own C the SAME way the original was built — toolchain and -O*/-m* flags read from the DWARF producer — then compares assembly line by line. A compilability fixup injects only what gcc reports missing (typedefs for pseudo-types like undefined4/uint, decls for implicit functions) and never rewrites logic. Link-time-dependent operands — call/branch targets and [rip±disp] displacements — are normalized away, so an unlinked address difference is not a penalty. Type recovery is scored separately (type_match), so fixing types just to compile is fair.
perfect = 1.0 (recompiled assembly matches the original).
[ = ] When a function is perfect on at least one metric,
the decompiler has exactly recovered that aspect of the original source:
the control flow, the types, or code that recompiles to the same bytes.
That is the Union column on the leaderboard.
metric limitations
Each metric comes with limitations, some due to the way they are measured, others due to their fundamental algorithm.
he first, and most wide-reaching limitation, is that we attempt to collect all metrics from only the decompilation text.
This is to assure that all decompilers can compete on the benchmark even when they do not expose deep APIs.
This also allows LLMs to compete, which may have no way to return to you something like an address mapping for a line reliably.
This can inject error because things like variables may be hard to align across samples, which has been explored in prior work.
Graph Edit Distance
Every CFG pair receives an exact isomorphism test first, with no size limit, so a perfect graph is always scored correctly.
Non-isomorphic graphs up to 200 nodes use the Vujosevic Janicic algorithm (VJ-GED).
Like most GED algorithms, VJ-GED is an approximation and can report more distance than actually exists.
Larger non-isomorphic graphs use a nonzero lower bound from their node and edge count differences.
There are other ways to inject error here.
We largely use Joern to parse the decompilation of each project.
If Joern fails, we fail.
There is also ways the .i files, which we parse, can have false information left behind by the compiler.
When we sampled this process, we found it was small.
Type Edit Distance
The fundamental flaw here is being unable to match a variable across a decompiler sample if the offset is not reported in the text.
We attempt to get around this by using heuristics, but, it is a known problem.
Recompilation Byte Edit Distance
The flaw here is that each function is evaluated alone.
There are cases where cross-function compiler optimizations can make it impossible for a decompiler to achieve a perfect score.
This is more common on the optimization dataset, and is a known limitation.
the dataset
summary
projects
Changelog
Significant changes to DecBench that introduce or update results, which can be viewable on the website.
2026-08-28
- Added the Ventris decompiler.
2026-08-27
- Added dated scoreboard snapshots. Any page accepts
?snapshot=DD-MM-YYYYto render
the numbers as they stood on that day, and/snapshots/lists them.
2026-08-08
- Update
Kunato versionv1.121, which has changed its rank. - Minor fix to GED correctness in PR #57. Changes the scores of all decompilers on structure, but has largely maintained the same order.
2026-07-27
- Added a warning about LLM based results having bias, based on Issue #43 discussion and analysis.
2026-07-25
- Fixed a caching bug in
sample-setthat prevented Codex/CC from having 3 samples graded/shown in the UI. Their scores have changed slightly. - External submission to DecBench are now open and can be done for closed source or private decompilers. See the README note for how.
2026-07-24
- Updated
aboutto include other related works and some limitations of the benchmark metrics.
2026-07-23
- DecBench v1.1
- Update
Kunato versionv1.0, which have shifted optimized results. - What was previously the
distancepage is now thedatapage and contains new info on LLM costs. - Removed
mirai-wintarget since it is not actually Windows, but just Linux binaries (which the benchmark already has).
2026-07-22
- DecBench goes live with support for 7 traditional decompilers, 2 LLMs (partial), and 3 defining metrics.
- An expanded evaluation of AI agents is planned after credits are secured for running those evaluations.
snapshots
A snapshot freezes the scoreboard on a given day, so a score you cite keeps a
stable link after the benchmark moves on. Open one by adding
?snapshot=DD-MM-YYYY to any page, or follow a date below.
Snapshots are recorded deliberately, not on a schedule — one is taken whenever a
change moves published scores or breaks comparability, which is the same moment
the changelog earns an entry. Use the filters to find the
snapshots where a decompiler was on a particular version.
Each snapshot carries the leaderboard, metrics, data and about pages exactly as
they stood. The view page is the one exception: its side-by-side source
is ~31 MB per build, far too heavy to freeze per date, so it always shows live
code.
