Malware Obfuscation: Packing, Encoding & Evasion

The malware sample sitting on a disk is often a disguise. Its strings are scrambled, its real code is compressed behind a wrapper, and the moment you load it into a sandbox it checks whether it is being watched and goes quiet. This is obfuscation, the collection of tricks malware uses to hide what it is and what it does long enough to run. For a defender it is the reason a file can pass every static check and still be hostile.

Obfuscation is an arms race with one hard constraint working in the defender's favour. However well the code is hidden, it has to run correctly on the victim eventually, which means it has to reveal itself at some point. Everything defenders do to see through obfuscation exploits that single fact.

What obfuscation is trying to defeat

Obfuscation targets three audiences, and understanding which one is why each technique looks the way it does:

• Static signatures. Antivirus and rule-based tools match patterns of bytes and strings. Obfuscation changes those patterns so nothing matches.
• Human analysts. Reverse engineers read code, strings, and structure. Obfuscation makes that reading slow, tedious, and misleading.
• Automated sandboxes. Detonation environments run the sample and watch it. Anti-analysis logic tries to detect the sandbox and behave innocently while inside it.

A serious sample layers techniques against all three at once. The result is a file that is unrecognisable on disk, exhausting to analyse by hand, and boring to a sandbox.

The main families

Technique	What it does	What it defeats	ATT&CK
Packing	Compresses or encrypts the payload, unpacks in memory at runtime	Static signatures on the real code	T1027.002
Encoding	Represents data as base64, hex, or custom schemes	String and pattern matching	T1027
Encryption	Keeps strings, config, or the payload encrypted until needed	Both signatures and analyst reading	T1027
String obfuscation	Builds or decrypts strings only at runtime	Analysts hunting for URLs and commands	T1027
Anti-debugging	Detects and resists debuggers	Manual reverse engineering	T1622
Sandbox evasion	Detects analysis environments, then stalls or hides	Automated detonation	T1497

These overlap constantly. A packed sample often encrypts its payload, encodes its configuration, and adds sandbox checks in the unpacking stub. Peeling one layer usually reveals another.

Packing, the core technique

Packing is the technique most people mean by obfuscation. A packer takes the real malware, compresses or encrypts it into a data blob, and wraps it in a small stub. On disk, the file is mostly that opaque blob plus the stub. When it runs, the stub unpacks the real payload into memory and jumps to it.

The consequence for detection is direct. A static scan sees the stub and the blob, neither of which matches a signature for the actual malware, because the actual malware only exists once it is unpacked in memory. Legitimate software uses packers too, for compression and licensing, which means the mere presence of packing is suspicious but not conclusive.

The saving grace is that unpacking has to happen. The genuine code appears in memory at runtime, so a defender who can look at process memory after the stub has run sees the payload the disk file was hiding.

Encoding, encryption, and hidden strings

Below packing sits a layer aimed at the artifacts that give malware away. The strings inside a program, its command-and-control addresses, the commands it runs, the names it looks for, are gold to both signatures and analysts. Obfuscation hides them.

Encoding rewrites these as base64, hexadecimal, or a custom alphabet so a plain-text search finds nothing. Encryption goes further and keeps them unreadable until a key, sometimes fetched at runtime, decrypts them. String obfuscation constructs sensitive strings piece by piece at execution time so they never sit whole in the file. A domain name might be assembled character by character, a command might be built from scattered fragments, and a key might be derived from values computed at runtime, so that a plain search of the binary turns up nothing usable and an analyst has to trace the construction to recover the original. The payload file that eventually lands is frequently one of the abused formats catalogued in our File-Format Abuse Atlas, and knowing which format you are dealing with tells you where the hidden strings and encoded stages tend to live.

All of this maps to Obfuscated Files or Information (T1027). The common thread is delay: keep the meaningful data unreadable until the last possible moment.

Anti-analysis and sandbox evasion

The most frustrating layer for defenders is the code that changes behaviour when it senses scrutiny. Before doing anything malicious, the sample checks its surroundings:

• Sandbox and virtual-machine checks look for the artifacts of analysis environments, such as specific drivers, tiny disks, or the absence of normal user activity.
• Debugger detection notices when a debugger is attached and alters execution to mislead the analyst.
• Timing and stalling waits out the limited window a sandbox will run a sample, or requires user interaction such as a scroll or a click before proceeding.
• Environment gating only decrypts and runs the payload if it confirms a real target, sometimes by phoning home first.

If any check says it is being watched, the sample stays benign, so the sandbox records nothing malicious and clears the file. This maps to Virtualization/Sandbox Evasion (T1497).

How obfuscation evolved

The techniques did not all appear at once. They accumulated as each side answered the other.

The earliest hiding was simple encoding and self-modifying code, meant to dodge the first string-and-byte scanners. Once scanners learned the common encodings, authors moved to packing, wrapping the real payload in a compressed or encrypted blob so the meaningful bytes never sat on disk in a matchable form. Packing became so common that the mere presence of it stopped being a reliable verdict, since legitimate software packs for compression and licensing too.

When defenders started detonating samples in sandboxes, authors added anti-analysis logic: checks for the artifacts of virtual machines, debuggers, and automated environments, with the payload staying dormant if any check tripped. That pushed defenders toward sandboxes that resist detection and toward memory analysis, which sidesteps disk obfuscation entirely by waiting for the payload to unpack.

The arms race continues, and the shape of it is stable. Authors add layers to delay understanding. Defenders shift from reading the file to watching it run, because the one constraint the author cannot remove is that the real code must eventually execute. Every durable defensive technique is built on that single fact.

A worked unpacking walkthrough

Following a packed sample through analysis makes the constraint concrete. The steps below describe the general pattern for a straightforward packed binary, without any specific tool or sample named.

1. First look on disk. A static scan finds little. The file has high entropy across most of its length, a small import table, and few readable strings. High entropy means the bytes look random, which is what compression and encryption produce. This profile says packed, and it says nothing about what the payload does.
2. Identify the stub. At the entry point sits a small routine, the unpacking stub, distinct from the opaque blob that makes up the rest of the file. The stub's job is to reconstruct the real code in memory and transfer control to it.
3. Let it unpack under control. Rather than reverse the stub by hand, the analyst runs the sample in an instrumented, contained environment and lets the stub do its work. At the moment control transfers to the freshly written region, the real payload exists in memory in the clear.
4. Capture memory. Dumping the process memory at that point recovers the unpacked payload. Now the strings, the configuration, and the command-and-control addresses that were hidden on disk are readable, and rules can match the real code.
5. Peel the next layer. Frequently the recovered payload has its own encoding: encrypted configuration, strings built at runtime, or a second stage fetched from the network. Each layer comes off the same way, by letting the code reach the point where it needs the data readable and observing it there.

The lesson repeats at every layer. Obfuscation buys delay, and running the code in a place you control spends that delay cheaply, because the author had to make the data usable to the malware, which means usable to anyone watching the malware.

Obfuscation compared to related techniques

Obfuscation is often confused with the techniques next to it. Separating them clarifies what each actually changes.

Technique	What changes	What stays the same	Aimed at
Obfuscation	The appearance of the code and data	The underlying logic, which runs unchanged	Scanners, analysts, sandboxes
Polymorphism	The packer or encryption wrapper on each copy	The decrypted payload underneath	Signature matching across samples
Metamorphism	The actual instructions, rewritten each generation	The net behaviour	Both signatures and structural analysis
Crypters	An encryption layer applied to an existing binary	The wrapped malware itself	Static antivirus at delivery time

Polymorphism and metamorphism are ways of making obfuscation change between copies, so that two samples of the same malware share no static signature. Our polymorphic malware guide covers that moving-target property in detail. A crypter is a productised obfuscation layer sold to wrap someone else's binary. The common thread across all of them is delay and disguise, and the common weakness is that behaviour and unpacked memory give the game away.

Detection signals: recognising a hidden sample

Even before you run a file, its structure carries tells that it is trying to hide. None is conclusive alone, and together they build a strong case for deeper analysis.

• High, uniform entropy. Most of the file measures close to random, the signature of compression or encryption sitting where normal code and data should be.
• A tiny import table. Packed samples resolve the functions they need at runtime, so the on-disk import list is short and generic, missing the calls the malware's real behaviour would require.
• Few or no readable strings. URLs, file paths, registry keys, and commands are absent or scrambled, because they are built or decrypted only at execution time.
• A known packer stub. Recognisable entry-point routines and section names tie the file to a specific packer, common or custom, and a custom stub is more suspicious than a mainstream one.
• Sections that are writable and executable. Memory regions marked to be both written and run point to code that plans to unpack itself into place and then execute it.
• Mismatched section sizes. A section that is small on disk but declares a large size in memory is space reserved for an unpacked payload.
• Anti-analysis imports. Calls associated with checking for debuggers, measuring timing, or querying the environment suggest the sample intends to behave differently when watched.

On the runtime side, the behavioural tells are the ones obfuscation cannot erase: a process that allocates executable memory, writes to it, and jumps into it is unpacking; a sample that stalls for a long time before doing anything is likely waiting out a sandbox; a benign-looking file that spawns a script host and beacons out has revealed intent no disguise can cover.

The practical stance is to treat these signals as a funnel. Structural tells decide which files earn detonation, runtime behaviour decides which earn a memory capture, and the unpacked payload is where durable rules finally get written.

How to detect and defend

The winning move is to stop competing with obfuscation on its own terms. Static matching loses that race. Behaviour and memory win it, because the payload must eventually be real.

1. Detect behaviour, not appearance. Watch what a process does after it runs: the child processes it spawns, the files it touches, the connections it makes, the persistence it creates. Obfuscation hides the code, and it does not hide the actions, which happen in the clear on the endpoint.
2. Analyse memory after unpacking. Once the stub has run, the real payload is in memory. Memory scanning and process dumping recover it, and rules can then match the unpacked code that the packed file concealed.
3. Use evasion-resistant sandboxes. Prefer detonation environments that resist common checks, simulate user activity, and extend runtime to defeat stalling. Combine sandbox output with endpoint telemetry rather than trusting it alone.
4. Flag packing as a signal, not a verdict. High entropy, known packer stubs, and tiny import tables indicate packing. Weigh it alongside origin and behaviour instead of blocking on it outright, since legitimate software packs too.
5. Write rules for unpacked artifacts. Signatures on packed files are brittle. Rules that match strings, configuration, and structure in the unpacked, in-memory form are far more durable across repacked variants.
6. Correlate weak signals. No single indicator survives obfuscation, so combine them: a packed file from a suspicious source that spawns a script host and beacons out is convincing even when each fact alone is not.
7. Assume the payload is more capable than the sample looked. When obfuscation is heavy, treat the file as a serious threat by default and respond accordingly, rather than downgrading it because static tools found little.

Obfuscation is a delaying tactic dressed up as invisibility. It can beat any check that reads the file without running it, and it cannot beat the requirement to eventually run. Shift your detection from what a sample looks like to what it does, recover the payload from memory once it reveals itself, and the disguise stops mattering. The code has to be real in the end, and that is where you catch it.

Common misconceptions

"Packed means malicious." Packing is common in legitimate software for compression and licensing protection. It raises suspicion and warrants a closer look, and on its own it is not a verdict. Blocking every packed file breaks benign tools and trains people to ignore the alert.

"A clean sandbox result means the file is safe." Evasive samples detect analysis environments and stay dormant inside them. A benign detonation may mean the sample declined to run, so a clean result is inconclusive for anything sophisticated and should be corroborated with endpoint behaviour and memory analysis.

"Obfuscation makes malware more capable." It changes appearance and buys time. The payload underneath does exactly what it would have done unobfuscated. Heavy obfuscation signals an author trying to avoid detection, and it says nothing about added functionality.

"If antivirus is quiet, there is nothing there." Static antivirus is precisely what obfuscation is built to defeat. Silence from signature-based tools against a heavily obfuscated file is expected, and it is a reason to escalate to behavioural and memory analysis rather than a reason to relax.

"Unpacking requires reversing the stub by hand." Much of the time the cheapest route is to let the sample unpack itself in an instrumented, contained environment and capture memory at the moment the payload appears. The author's own code does the decryption for you.

Frequently asked questions

What is the difference between packing and encryption in malware?

Packing compresses or encrypts the whole payload into a blob that a stub reconstructs in memory at runtime, so the on-disk file is mostly an opaque wrapper. Encryption in the narrower sense keeps specific artifacts, such as strings, configuration, or a later stage, unreadable until a key decrypts them at the moment they are needed. Packing hides the entire payload; targeted encryption hides the parts most useful to signatures and analysts. Many samples use both.

Why does obfuscation defeat antivirus?

Signature-based antivirus matches known patterns of bytes and strings. Obfuscation changes those patterns by design, so nothing matches. Packing goes further and keeps the real code off the disk entirely until runtime, which means there is no stable pattern to sign in the first place. This is why static matching loses the race and why behavioural and memory-based detection are the durable answers.

Can obfuscation be defeated completely?

Not on disk, and it does not need to be. The payload has to run correctly on the victim eventually, which means the real code appears in memory and the real behaviour happens on the endpoint. Watching execution and recovering the unpacked payload from memory sidesteps the disguise, because the one thing the author cannot obfuscate away is the need to execute.

Is a high-entropy file always malware?

No. High entropy indicates compression or encryption, which packed malware shows, and so do legitimate compressed installers and protected commercial software. Entropy is a useful hint that a file is packed or encrypted, and it should be weighed with the file's origin, its behaviour, and the presence of a known packer stub before any action.

How do sandboxes handle evasive malware?

Better sandboxes resist common detection checks, simulate user activity such as scrolling and clicking, and extend runtime to defeat stalling. Even then, a sandbox verdict is combined with endpoint telemetry rather than trusted alone, because sophisticated samples are built specifically to look harmless inside analysis environments.

Which ATT&CK techniques cover obfuscation?

The core mapping is Obfuscated Files or Information (T1027), which covers packing, encoding, encryption, and string obfuscation. Anti-analysis behaviour maps to Virtualization/Sandbox Evasion (T1497), and debugger resistance is tracked separately. These identifiers give a stable vocabulary for describing what a sample is doing to hide.

Does obfuscation help attribution or hurt it?

It cuts both ways. Obfuscation is meant to frustrate analysis, and the specific packer, encoding scheme, or evasion checks a sample uses can themselves become a fingerprint. Reused custom obfuscation is sometimes as identifying as the payload it hides, which is why analysts record the disguise as well as the code underneath.

Why write detection rules against unpacked memory instead of the file on disk?

Rules that match the packed file are brittle, because repacking the same malware with a different wrapper produces an entirely different on-disk pattern while the underlying code is unchanged. Rules written against the unpacked, in-memory form match the real payload, so they survive across repacked variants and across the polymorphic wrappers designed specifically to break disk signatures. Memory is where the malware finally stops lying about itself.

Related guides

• What Is Polymorphic Malware
• Intro to Malware Analysis
• Malware Sandbox Evasion
• YARA Rules Explained