Resilience benchmarks

The runtime suite proves correctness. The resilience bench (utils/deobf_bench/) proves resistance — it runs real deobfuscation attacks against obfuscated functions and scores how much the attacker recovers.

Run this before claiming a hardening win

It is easy to tune a pass so it looks stronger against today's attack while being no harder to reverse in general — a fake win on a fragile metric. Measure with the bench; don't infer resistance from IR size or a decompiler screenshot.

What it attacks

The bench ships a set of attack modules (utils/deobf_bench/attacks/), driven mostly by Z3 and angr:

Attack module	What it tries
mba_smt	Simplify an MBA expression back to its original with an SMT solver.
opaque_predicate	Prove an opaque predicate's branch is dead.
dse_coverage	Cover and recover guarded logic with dynamic symbolic execution (angr).
cfg_recovery	Reconstruct the original control-flow graph.
handler_sig	Fingerprint VM handlers by signature across functions.
vm_bytecode	Recover structure/semantics from the virtualised bytecode.
call_target_recovery	Recover the real callee behind vcall indirection.
opt_survival	Check whether obfuscation survives a plain -O2/-O3 (or folds away).
string_extract	Recover encrypted string literals.
decompile_quality	Score decompiler-output quality/readability.

The scorer aggregates per-case and overall resilience figures — higher means the attacks recovered less. See VM devirtualization resistance for how the lifting/DSE attacks map onto the vm hardening knobs.

Running the bench

python -m utils.deobf_bench.cli --list                       # enumerate cases, no build needed
python -m utils.deobf_bench.cli \
    --build-dir <path-to-llvm-build> --config Release \
    --seeds 1,2,3 --filter <substr>

Flag	Meaning
--build-dir	LLVM build directory (required unless --list).
--config	MSVC multi-config (Debug/Release).
--seeds	Comma-separated seeds.
--attacks	Restrict to specific attacks.
--filter	Run only cases matching a substring.
--work	Work directory for temp artifacts.
--list	List all bench cases and exit.

The angr environment

The symbolic-execution attacks depend on angr, which needs its own Python (angr targets a specific CPython range and won't install into every interpreter). Create a dedicated virtual environment on a supported Python version and install angr + z3-solver there, then point the bench at that interpreter rather than the system one:

python3.12 -m venv .dse-venv
.dse-venv/Scripts/python -m pip install angr z3-solver   # .dse-venv/bin/python on Unix

The SMT/Z3 attacks work without angr; angr is only required for the symbolic-execution cases.

Interpreting results

• A resilience score near maximum means the attacks failed to reverse the function.
• A low score means the configuration is fragile against that attacker — even if it looks heavily obfuscated.
• Directional, not absolute. The bench ranks configurations against these attacks. A config that beats the bench can still fall to a bespoke manual attack. Obfuscation raises cost; it is not a boundary. See Legal & disclaimer.

When to run it

• Before merging any change that claims to harden a pass.
• When choosing between preset tiers for a high-value function.
• After changing MBA zero-injection knobs (sle / inputZero) — verify against both the SMT and linear-MBA attack paths, not just one.
• Before shipping vm as your anti-devirtualization defence — confirm the hardened tier holds on your actual functions.