Skip to content

Devirtualization resistance

Virtualisation is only as strong as its resistance to devirtualization — an attacker recovering the original semantics from the bytecode + interpreter.

Obfuscation is not a security boundary

A virtualised function raises the cost of reverse engineering; it is not unbreakable. Measure your own configuration against your own functions with the resilience bench — do not assume a tier is safe.

How automated devirtualization works

The dominant automated approach is semantics-based, and in its modern LLVM form it runs roughly:

  1. Dynamic taint analysis on the obfuscated function to find where the protected input flows.
  2. Execution-trace collection, split on tainted conditionals, to capture the path(s) actually taken.
  3. Lift to LLVM IR — e.g. Trail of Bits' remill — and reconstruct a CFG from the trace.
  4. Simplify with compiler optimizations (LLVM passes / -O3, plus program synthesis or SMT on individual handlers) to fold the interpreter away and recover the original logic.

This is what the repository's own lifting attack does (a remill lift followed by opt), and it mirrors published work such as Coogan–Lu–Debray's semantics-based devirtualization and the Thalium LLVM-powered devirtualization writeup.

Where the hardening knobs bite

The attack has documented weak points; the knobs target them:

Attacker step Friction
Collect an execution trace (dynamic) antiDebug / bindAntiDebug frustrate running under a tracer/debugger — the trace is where the whole pipeline starts.
Read the bytecode to recover handler semantics encBytecode, lazyDecrypt, constInStream keep the "virtual instructions" encrypted at rest — the exact factor the literature names as complicating analysis.
Simplify each handler back to a primitive op hardened, nestedVM, superOps make handler semantics heavier to synthesize/optimize.
Match handlers across functions by signature per-function opcode permutation, metamorphicEngines, perFnEngine, handlerVariants/handlerDecoys, randISA deny a stable cross-function handler fingerprint.
Follow one path only trace-based recovery captures a single execution path and struggles with loops — more so with layered, input-dependent structure.

preset=max enables this set together. A bare vm can look strong to a human reader while still handing an automated pipeline plaintext bytecode, a single un-mutated engine, and an untraced, un-hardened dispatch loop.

Layer the input, too

Virtualising already-obfuscated IR is strictly stronger than virtualising clean IR — the virtual ISA then encodes MBA / opaque-predicate logic, so even a successful lift recovers obfuscated code:

// pre-obfuscate, then virtualise the result
__attribute__((annotate("obf: mba(prob=70), bcf(prob=30), vm(preset=max)")))
int secret(int key, int data) { return key ^ (data + 0xDEAD); }

Cost of the strong tier

preset=max is the heaviest configuration. Budget it for your most sensitive functions only.

Configuration Overhead (rough, target-dependent)
Base vm (no hardening) 5–20× slowdown vs native
regEncrypt=1 +20–50% over base
hardened=1 +15–30% over base
lazyDecrypt=1 +30–100% over base
nestedVM=1 +2–10× over base
enginePoolSize/perFnEngine/metamorphicEngines negligible runtime; ~30 KB .text per engine
antiDebug=1 +1–5% over base

Mitigations: apply vm only to key derivation / license checks / protocol parsing; use minBlocks so trivial functions aren't virtualised for no benefit; and verify survival under a later -O2 with the runtime suite's --o2-gate.

Verify it yourself

Don't take the tiers on faith — the repository ships the attacks and the correctness gate: