Windows rootkits of 2005, part three
James Butler, Sherri Sparks,

The third and final article in this series explores five different rootkit
detection techniques used to discover Windows rootkit deployments.
Additionally, nine different tools designed for administrators are
1. Introduction
Rootkits have become very sophisticated over the past few years, and in 2005
we have seen a surge in rootkit deployments in spyware, worms, botnets, and
even music CDs. Although once a computer system has been subverted by a
rootkit it is extremely difficult to detect or eradicate the rootkit, there
are still some different methodologies that detect the rootkit that have
worked to varying degrees. Part one looked at what Windows rootkits are and
what makes them so dangerous. Part two examined the latest cutting-edge
rootkit technologies and how they achieve stealth.

Now in part three, we explore five such detection techniques and, where
possible, provide information about different rootkit detection tools.
2. Signature based detection
Signature based detection methods have been in use by antiviral products for
years. The concept is simple. System files are scanned for a sequence of
bytes that comprise a "fingerprint" that is unique to a particular rootkit.
If the signature is found in a file on the user's system, it signals an
infection. As signature scanning has traditionally been applied to the
filesystem, its usefulness for rootkit detection is limited unless it is
combined with some more advanced detection techniques. This is due to the
rootkit's natural propensity to hide files using execution path hooking

Despite their antiquity, signature based detections are worth mentioning
because they may be applied with success to scanning system memory in
addition to filesystem scanning. Ironically, most public kernel rootkits are
susceptible to signature scans of kernel memory. As kernel drivers, they
typically reside in non-paged memory and few, if any, make an effort towards
any kind of polymorphic code obfuscation. Thus, a scan of kernel memory
should trivially identify most public kernel rootkits regardless of their
underlying "bag of tricks" (DKOM, SSDT, IDT hooking and the like). The key
words in that last sentence, however, are "public rootkits" because
signature based detection is, by definition, useless against malware for
which a known signature does not exist. Finally, signature based detection
methods are useless against Virtual Memory Manager (VMM) hooking rootkits
like Shadow Walker which are capable of controlling the memory reads of a
scanner application. [ref 1]
3. Heuristic / Behavioral detection
Where signature based detections fall short, heuristic detections take over.
Their primary advantage lies in their ability to identify new, previously
unidentified rootkits. They work by recognizing deviations in "normal"
system patterns or behaviors. Various heuristics have been proposed for
identifying rootkits based upon execution path hooking. In this section we
examine two such tools: VICE and Patchfinder.
3.1 VICE
VICE is a freeware tool written to detect hooks [ref 2]. It is a standalone
program that installs a device driver to analyze both user mode applications
and the operating system kernel. In the kernel, VICE checks the SSDT for
function pointers that do not resolve to ntoskrnl.exe. Also, you can add
devices to the file "driver.ini," and VICE will check the IRP major function
table of the corresponding driver. If a function pointer in the IRP major
function table of a driver does not consist of an address within the driver,
then the IRP has been hooked by an outside driver or piece of kernel code.
In user mode, VICE checks the address space of every application looking for
IAT hooks in every DLL that the application uses. Inline function hooks are
detected in DLL functions imported by applications and in the SSDT functions
themselves. VICE will resolve what function is being hooked and the address
of the hooking function. When possible, VICE will also display the full path
on the filesystem of the DLL or device driver doing the hooking so that a
System Administrator can remove the malicious software. Today, VICE will
detect most publicly known Windows rootkits and any stealth related
technology that uses hooking technologies. To run VICE, the host machine
must have the Microsoft .NET Framework installed, which is free for

The current version of VICE has been targeted and subverted by at least one
public rootkit. [ref 3] Rootkits have leveraged the fact that VICE always
executes with a specific process name. When the rootkit detects the VICE
process, it does not hook so VICE has nothing to detect. Another attack has
targeted VICE's communication channel between the user mode portion and the
device driver. However, VICE's biggest weakness may be the large number of
false positives it returns. VICE was designed to detect hooks, but there are
legitimate uses of hooks in the operating system. Microsoft themselves offer
"hooks" in the form of hot patching and DLL forwarding. It is difficult to
determine what is a malicious hook verses a benign, legitimate hook.
3.2 Patchfinder
Where VICE identifies rootkits based on static analysis of code and data
structures, Joanna Rutkowska proposes a detection method based upon runtime
execution path profiling [ref 4]. It is implemented in the proof of concept
tool, Patchfinder, and is available on her website,
Patchfinder is built upon the observation that a rootkit must add code to a
given execution path (i.e. to filter the results returned by a hooked
service, for example). Thus, if one is able to count the number of
instructions executed by some system services commonly hooked by rootkits,
the instruction count should be greater in an infected system than the count
observed by executing the same services on a clean system. Rutkowska uses
the "single step" feature of the x86 processor to perform this instruction
counting. When code is run in "single step" mode, the processor halts
execution and calls a special Interrupt Service Routine (ISR) after each
instruction is executed. The instruction count is updated in this routine.
Unfortunately, in a system as complex as Windows, there may be multiple
execution paths which result in non-deterministic behavior. In other words,
multiple executions of the tested service may return different counts under
varying system conditions. Rutkowska deals with this problem statistically
in Pathfinder by constructing a histogram and empirically demonstrating that
the primary peak on the histogram remains constant regardless of system
load, while the insertion of additional instructions by a rootkit causes a
shift in the peak. Nevertheless, there remain cases where false positives
can occur and the technique is vulnerable to rootkits which realize that
they are being traced.
4. Cross view based detection
Cross view based detection techniques are relatively new and show a lot of
promise. These techniques assume that the operating system has been
subverted, but this method leverages the fact that there is usually more
than one way to ask the same question. In a cross view based detection
method, the detection software calls the common APIs to enumerate key
elements within the computer system such as the list of files, processes, or
Registry keys. However, to be successful, the detection software must also
have an algorithm to generate a similar data set that does not rely upon the
common APIs. Any difference in these two data sets reveals something hidden
because it did not exist in the data set generated using the common APIs.
Cross view based detection relies upon the fact that API hooking or
manipulation of kernel data structures will taint the data returned by the
operating system APIs, but the low level methods used to glean the same
information will be designed in such a manner as not to be subverted by
hooks or DKOM tricks.
4.1 Rootkit Revealer
Rootkit Revealer is available from Sysinternals. [ref 5] Sysinternals
provides many free utilities for System Administrators and developers alike.
Rootkit Revealer targets a subset of rootkits called persistent rootkits.
Persistent rootkits are those that exist between reboots. In other words, a
memory resident rootkit would not be persistent. In order to be persistent
between reboots, Rootkit Revealer assumes that the rootkit will have to
exist on the filesystem and in the Windows Registry. Rootkit Revealer uses a
cross view based approach to detect these persistent rootkits. Rootkit
Revealer calls the highest level APIs in order to enumerate the files on
disk and the Registry keys. To compare against this, Rootkit Revealer parses
the raw filesystem structure on disk and the bare files that comprise the
Registry hives.

Like most rootkit detection tools, Rootkit Revealer can be subverted by a
rootkit that blocks or diverts access to disk volumes or the Registry files.
Rootkit Revealer also suffers from false positives if files or Registry keys
are created, deleted, or otherwise altered between the high level and the
low level enumeration. This is true for all cross view based techniques
because of the nature of the algorithm.
4.2 Klister
Klister is another proof of concept rootkit detection tool developed by
Joanna Rutkowska. [ref 4] Klister demonstrates a method useful for detecting
rootkits that use DKOM techniques to hide processes, like FU. FU hides
processes by unlinking EPROCESS objects from the PsActiveProcessList (note:
See part one for specific details). [ref 6] Intuitively, it seems that if
one removes a process from the OS's process list it would cease to run (in
other words cease to be scheduled for time on the CPU by the OS scheduler).
Interestingly, this is not actually the case. The OS dispatcher maintains
separate queues from which it schedules processes. Klister exploits this
redundancy in OS data structures to detect processes hidden by unlinking the
entries in the PsActiveProcessList. By comparing the active process list
with the dispatch queues, it is possible to identify discrepancies. A
process which appears in the dispatch queue but not in the active process
list is assumed to be a hidden rootkit process. Although Rutkowska only
applies this technique to process detection, comparisons between redundant
kernel data structures can be a valid form of detection for other DKOM
attacks as well.
4.3 Blacklight
Blacklight by F-Secure is one of the first commercial products aimed
specifically at rootkit detection. [ref 7] Although Blacklight technology is
being rolled into other products by F-Secure, Blacklight is available as a
free download for now. Blacklight targets processes and files hidden by
rootkits. It uses a cross view based method, enumerating all the processes
with higher level APIs and comparing that list to a list generated using a
much lower level method. Blacklight uses a similar method for detecting
hidden files.

Over the last year, the algorithm Blacklight uses for the lower level
enumeration has changed several times. Each time, Blacklight attempts to
identify a critical piece of data that would cause the system to fail if a
rootkit altered or subverted it. Blacklight and other cross view based
detection methods have had varying degrees of success identifying those key
elements that cannot be altered. Once the rootkit authors discover the
algorithm being used, a carefully designed rootkit can usually alter the
data without any negative consequences.
4.4 Strider GhostBuster
Similarly to Rootkit Revealer, Microsoft's Strider GhostBuster takes a
cross-view-diff based approach to detect hidden files, registry entries,
processes, and loaded modules. [ref 8] That is, it compares high level API
queries with low level queries obtained by manually parsing the underlying
operating / filesystem data structures (such as the NTFS Master File Table
and the Windows Registry hive). Differences between the queries are taken to
indicate the presence of "ghostware," a term coined by the MS research team
to include rootkits, trojans, keyloggers, and other commercial adware /
spyware that has a vested interest in concealing their presence on a system.

GhostBuster provides both inside-the-box and outside-the-box system scans.
By inside-the-box, we mean that the scan can be performed on a running
system. This is basically a scan very similar to Rootkit Revealer scan with
added detection for hidden processes. To perform this scan, an
inside-the-box scan is first performed on a live suspect system. Then, the
system is booted to a clean operating system (WinPE is used) where a low
level scan is then performed. The offline nature of the scan reduces the
possibility of interference or attack of the scanner application by a
rootkit. The tradeoff, however, is in convenience since it is clearly
impractical to reboot a server system for a routine malware scan.
Nevertheless, it may useful for providing forensic evidence if the system is
strongly suspected or already known to be compromised.

Although the cross view approach seems to be state-of-the-art in current
rootkit detection methodologies, it is worth mentioning that the approach is
not invulnerable to existing rootkit attack methodologies. Indeed, its
success depends in large part upon implementation, specifically the method
which is used to obtain the "low level" view of the system. For example, in
order to parse the NTFS disk layout, a detector must first obtain a handle
to the volume and then subsequently read its disk sectors. If the detector
uses an operating system API to perform these tasks, it may as well go back
to the drawing board. In other words, a detector must assume that the API
interface is compromised (API hooking is a tried and true rootkit technique
for cheating detection). Thus, the strongest implementation of a cross view
approach should only rely upon direct communication with the disk
controller. [ref 9]
5. Integrity based detection
Integrity based detection provides an alternative to both signatures and
heuristics. It relies upon comparing a current snapshot of the filesystem or
memory with a known, trusted baseline. Differences between the current and
baseline snapshots are taken as evidence of malicious activity.
Unfortunately, however, the integrity checker is usually not capable of
pinpointing the origin of the activity that has caused the changes.
5.1 Tripwire
Tripwire is a disk based integrity checker [ref 10]. It creates a trusted
database of unique CRC hash values for each of the system files on a user's
hard disk. When the user initiates a scan for malicious activity, it
recalculates the CRC's for all files and compares them to the original CRC's
in the database. It is based upon observation that system files should not
change (except perhaps in the rare system update or service pack), thus a
mismatch indicates compromise. Tripwire was very effective against early
rootkits that simply replaced system files on the disk with trojanized
versions. Unfortunately, rootkits adapted by moving their modifications from
disk to memory. This renders Tripwire virtually useless for modern rootkit
detection. Nevertheless, where disk based integrity checking fails, memory
based integrity checking may succeed.
5.2 System Virginity Verifier
System Virginity Verifier by Joanna Rutkowska is yet another prototype
rootkit detection tool. [ref 11] It combines heuristics similar to VICE with
memory based integrity checking technology. Like VICE, it checks the
integrity of operating system data structures (IAT / EAT / SSDT / IRP
tables). It also incorporates some more advanced heuristics to help deal
with the false positive problem resulting from benign hooking by legitimate
applications such as antivirus scanners and personal firewalls. Memory
integrity checking is provided by comparing the code sections for important
system libraries and drivers on disk with their corresponding loaded images
in memory.
6. Hardware detection - Copilot
Copilot is unique in that it is a hardware based detection tool. It began as
a project at the University of Maryland and has spawned an independent
company. Currently, Copilot [ref 12] is in the form of a PCI card that is
installed on the host being monitored for rootkit activity. The goal of the
PCI card is to remain as independent of the potentially subverted operating
system as possible. To do this, the PCI card has its own CPU and uses Direct
Memory Access (DMA) to scan the physical memory of the computer looking for
rootkit behavior such as hooks in the SSDT, alterations to kernel functions
(using kernel integrity checks), and modifications to key data structures
like those performed by a DKOM attack. The Copilot PCI board also has its
own network interface to communicate in a secure fashion to an
administrative component.

Because it is a hardware based solution, Copilot provides a high degree of
assurance; however, this is not without a price. Any hardware based solution
is going to be more costly to purchase and to maintain, but the added
assurance is worth it for some. If coupled with a software monitoring
component, Copilot would make an even more formidable adversary against the
rootkit authors. Copilot could even verify the integrity of its own software
running on the host.
7. Concluding part three
As we survey the existing rootkit detection techniques, it becomes apparent
that all of the approaches have strengths and weaknesses. Every one of the
described tools brings a piece of the puzzle to the table, but none of them
yet provide a truly comprehensive solution to the problem. Many of the tools
are still proof of concept and, ironically, some of them were written by the
rootkit authors themselves. It is, however, encouraging to see anti-rootkit
activity beginning in the commercial AV / Anti-malware arena (as we saw with
Blacklight and Strider GhostBuster) as well as in the high assurance market
(Copilot). In attempting to consider the "big picture," we can finally make
a few observations based upon our firsthand observations of some of the
battles that occur between rootkit authors and detectors. First, a number of
the aforementioned tools show promise. Hybrid approaches which combine
several of these techniques, however, will be stronger because they will
maximize their strengths while minimizing the effects of weaknesses. Second,
"lower is better" should be the mantra of the rootkit detector. Many
detectors get themselves into trouble by relying upon operating systems
API's which they should assume to be compromised. Finally, one must keep in
mind that the rootkit still has the upper hand. Unlike a rootkit which
rarely needs to concern itself with the stability of the victim's system,
this concern is of paramount importance to the detector, especially
commercial detectors with a paying client base. This often constrains the
rootkit detection software by forcing it to avoid some useful but dangerous
methods. Despite this, detectors are clearly "upping the ante." Now in early
2006, it will be interesting to see how the ongoing battle between rootkit
detector and developer plays out over the coming year.
8. References
[ref 1] Butler, James and Sparks, Sherri. "Shadow Walker: Raising The Bar
For Windows Rootkit Detection", Phrack 63, July 2005.
[ref 2] Butler, James, "VICE - Catch the hookers!" Black Hat, Las Vegas,
July, 2004.
[ref 3] Holy Father, Hacker Defender.
[ref 4] Rutkowska, Joanna. "Detecting Windows Server Compromises with
Patchfinder 2", Jan 2004.
[ref 5] Rootkit Revealer.
[ref 6] Fuzen, FU Rootkit.
[ref 7] BlackLight.
[ref 8] Y. Wang, D. Beck, R. Roussev, and C. Verbowski. "Detecting Stealth
Software with Strider GhostBuster". Technical Report, Feb 2005.
[ref 9] Rutkowska, Joanna. "Thoughts about Cross-View based Rootkit
Detection", June 2005 and
[ref 10] Tripwire, Inc.
[ref 11] Rutkowska, Joanna. "System Virginity Verifier: Defining the Roadmap
for Malware Detection on Windows Systems", Sept 2005.
[ref 12] N. Petroni, T. Fraser, J. Molina, and W. Arbaugh, "Copilot - a
Coprocessor-based Kernel Runtime Integrity Monitor," in Proc. Usenix
Security Symposium, Aug. 2004. On the Web at
About the authors
James Butler is the CTO of Komoku, which specializes in high assurance, host
integrity monitoring and management. Before that, Mr. Butler was the
Director of Engineering at HBGary, Inc. focusing on rootkits and other
subversive technologies. He is the co-author and a teacher of "Aspects of
Offensive Rootkit Technologies" and co-author of the newly released
bestseller "Rootkits: Subverting the Windows Kernel."

Sherri Sparks is a PhD student at the University of Central Florida.
Currently, her research interests include offensive / defensive malicious
code technologies and related issues in digital forensic applications.

You are a subscribed member of the infowarrior list. Visit for list information or to unsubscribe. This message 
may be redistributed freely in its entirety. Any and all copyrights 
appearing in list messages are maintained by their respective owners.

Reply via email to