Ransomware Reverse Engineering: A Practical Guide for Incident Responders

Before loading a sample into a disassembler, spend five minutes with Detect-It-Easy (DIE). DIE identifies the compiler, packer, and file format of a binary without executing it, which tells you how much effort the threat actor invested in evasion and what analysis approach to take. A sample compiled with MSVC and no packer is ready for immediate analysis in Ghidra. A sample showing UPX packing should be unpacked first using upx -d sample.exe before static analysis, or allowed to unpack itself in a debugger. A sample flagged as a custom packer or with high entropy across the entire PE section map suggests a custom obfuscation layer that will require dynamic analysis to resolve. DIE also extracts embedded strings, import tables, and version resource data. Import tables are particularly valuable at this stage: ransomware that imports CryptAcquireContext, CryptEncrypt, and CryptGenRandom from the Windows CryptoAPI is almost certainly using built-in Windows cryptographic primitives, which simplifies your search for key material. Ransomware that imports no cryptographic functions but shows calls to unusual networking libraries may be staging its encryption payload from a remote source. Record everything DIE reveals before moving to deeper analysis, because those initial observations shape every subsequent step.

Static analysis means examining the disassembled or decompiled code without running the binary. Ghidra (free, NSA-developed) and IDA Free (limited freeware version of IDA Pro) are the two most accessible tools for this stage. Load the sample into Ghidra, let auto-analysis complete, and start your triage from the entry point and the string list. Ransomware samples almost always contain hard-coded strings that reveal operational intent: ransom note templates, C2 hostnames or IP addresses, mutex names, registry key paths for persistence, and file extension lists used to target specific file types. Search for these strings early. From the entry point, trace execution flow to the main encryption loop. In most ransomware families, this loop iterates over files on local drives and network shares, opens each file, reads chunks into a buffer, encrypts the buffer, and writes it back. The encryption call is usually a wrapper around a symmetric cipher: AES-256 in CBC or CTR mode for most modern strains, with the AES key itself protected by an asymmetric scheme (RSA or elliptic curve) so that decryption requires the threat actor's private key. Identify the function that generates or receives the per-session AES key. This is the most critical artifact for potential decryption key recovery. Note the function address and any constants used in the key derivation routine. Ghidra's decompiler view makes this substantially faster than reading raw assembly: look for calls to BCryptGenerateSymmetricKey, CryptImportKey, or custom key expansion routines that expand a short seed into a full 256-bit key.

Different ransomware families have distinctive engineering choices that persist across samples and make attribution possible from code alone. LockBit 3.0 is known for its speed-optimized partial encryption approach: rather than encrypting entire files, it encrypts only the first 4 KB (or a configurable initial chunk), which allows it to process thousands of files per minute on modern hardware while making files unrecoverable without the key. If you see a read-encrypt-write loop that consistently targets a fixed small offset at the start of each file, LockBit 3.0 or a derivative is likely. Conti uses a thread-pool model with 32 to 64 concurrent encryption threads, maximizing CPU utilization across all cores. In Ghidra, a Conti sample's encryption dispatcher is recognizable by the creation of a large number of threads early in execution and a work queue that distributes file handles across those threads. BlackCat (ALPHV) is compiled in Rust, which produces distinctive calling conventions and data structures compared to C or C++ ransomware. The absence of familiar MSVC runtime library patterns and the presence of Rust's standard library mangled symbols are a reliable indicator. Identifying the family narrows your search for published decryptors: law enforcement operations against HIVE, REvil, and BlackMatter have released master private keys that allow full decryption of affected files, but only if you can confirm the family match.

Dynamic analysis means running the sample in a controlled environment and observing its behavior in real time. Never execute ransomware on a production machine or a VM connected to any network share you value. Use Flare-VM (a Windows-based malware analysis distribution built on top of Windows 10, deployable via a PowerShell script from the Mandiant GitHub repository) or REMnux (a Linux distribution with a curated set of analysis tools) as your isolated detonation environment. Flare-VM is the better choice for Windows ransomware because it provides a native execution environment with all analysis tools pre-installed. Before detonating, snapshot the VM so you can restore to a clean state after analysis. In x64dbg, set breakpoints on the cryptographic API calls you identified during static analysis: BCryptEncrypt, CryptEncrypt, and BCryptGenerateSymmetricKey are the highest-value targets. When execution hits those breakpoints, examine the register values and stack frames to locate the encryption key material in memory. For AES-CBC implementations, the key and initialization vector (IV) are passed as arguments to the encrypt function and will be present in registers or on the stack at the moment of the call. Capture those values. If the per-file AES key is derived from a master session key, set an earlier breakpoint at the key derivation function to capture the session key before it is used to generate per-file keys. Also monitor file system and registry activity using Process Monitor (included in Flare-VM) during detonation to capture the full set of persistence mechanisms, dropped files, and ransom note paths.

If ransomware is caught actively encrypting files, you may have a narrow window to recover the encryption key from process memory before it is overwritten or the process exits. This is one of the highest-value interventions available during a live incident. The approach is to capture a full process memory dump of the ransomware process using Task Manager (right-click the process, select Create dump file) or a tool like ProcDump (procdump -ma <pid> memdump.dmp). Once you have the dump, load it into a memory analysis framework such as Volatility or simply search the raw binary for the key material pattern. For AES-256, you are looking for 32 consecutive bytes that match the entropy and structure of an AES key, often preceded or followed by the IV. Tools like findaes (a standalone AES key finder) scan a memory dump for byte patterns consistent with AES key schedule expansion, making this search tractable even without knowing the exact memory address. The success rate of this approach depends on how much of the encryption has already completed: if the ransomware generates a new key per file and immediately discards the previous key, memory recovery may only cover files being actively encrypted at the moment of capture. If the ransomware uses a single session key for all files in a run, a single successful recovery decrypts all affected files.

Modern ransomware variants rarely operate as standalone encryptors. Most include a C2 beaconing component that registers the victim with the threat actor's infrastructure, sends a unique victim ID, and either receives the encryption key from the server or sends the locally generated public key to the server for safekeeping. Understanding this flow is critical for two reasons: it may reveal the key exchange mechanism and identify whether the encryption key was ever transmitted in a form that can be recovered from network captures, and it provides the C2 indicators of compromise (IOCs) needed to block the threat actor's infrastructure and detect lateral movement. In Ghidra, look for network API imports: WinHttpOpen, InternetOpenUrl, WSAConnect, or direct socket calls. Trace the code path from those imports back to the data that is serialized and transmitted. Ransomware C2 beacons typically send a JSON or custom binary payload containing the victim ID (often a hash of the machine hostname or MAC address), the victim's generated public key, and system metadata. The response from the C2 server typically contains either the attacker's public key (for hybrid encryption schemes) or a confirmation token. If you capture this exchange in a network dump during dynamic analysis, the public key material may help identify the encryption scheme even if the private key is not recoverable.

YARA rules translate the unique code patterns and strings you have extracted from reverse engineering into a format that detection engines and threat hunting platforms can use. A well-constructed YARA rule based on a reverse engineered sample will detect not just the exact binary you analyzed but future variants from the same threat actor, because it targets code patterns that are more stable than file hashes. Start with strings: the ransom note template, the C2 hostname, the mutex name, and any hard-coded encryption constants are all candidates. Strings that are unique to the threat actor and unlikely to appear in legitimate software are the highest-value targets. For code patterns, use Ghidra to identify byte sequences that correspond to the unique logic in the encryption loop or key derivation routine, then extract those as hex byte patterns for the YARA rule condition. Apply wildcards (??) to bytes that correspond to addresses or offsets that change between compiled versions of the same codebase, while keeping the surrounding opcode bytes fixed. A practical rule structure includes three to five string conditions combined with a condition that requires at least two or three of them to match, reducing false positive risk while maintaining sensitivity. Test your rules against a clean corpus using yara rule.yar /path/to/corpus before deploying them to your detection stack. Submit finalized rules to public repositories such as YARA-Rules on GitHub and share with your threat intel sharing communities to accelerate detection across the broader defender ecosystem.