Fingerprinting
Web server/application fingerprinting is similar to its predecessor, TCP/IP
Fingerprinting (with today's favorite scanner - Nmap) except that it is focused
on
the Application Layer of the OSI model instead of the Transport Layer. The
theory behind web server/application fingerprinting is to create an accurate
profile of the target's software, configurations and possibly even their
network
architecture/topology by analyzing the following:
- Implementation differences of the HTTP Protocol
- HTTP Response Headers
- File Extensions (.asp vs. jsp)
- Cookies (ASPSESSION)
- Error Pages (Default?)
- Directory Structures and Naming Conventions (Windows/Unix)
- Web Developer Interfaces (Frontpage/WebPublisher)
- Web Administrator Interfaces (iPlanet/Comanche)
- OS Fingerprinting Mismatches (IIS on Linux?)
The normal SOP for attackers is to footprint the target's web presence and
enumerate as much information as possible. With this information, the attacker
may develop an accurate attack scenario, which will effectively exploit a
vulnerability in the software type/version being utilized by the target host.
Accurately identifying this information for possible attack vectors is vitally
importantly since many security vulnerabilities (such as buffer overflows,
etc...)
are extremely dependent on a specific software vendor and version numbers.
Additionally, correctly identifying the software versions and choosing an
appropriate exploit reduces the overall "noise" of the attack while increasing
its
effectiveness. It is for this reason that a web server/application, which
obviously
identifies itself, is inviting trouble.
In fact, the HTTP RFC 2068 discusses this exact issue and urges web
administrators to take steps to hide the version of software being displayed by
the "Server" response header:
"Note: Revealing the specific software version of the server may allow the
server machine to become more vulnerable to attacks against software that
is known to contain security holes. Server implementers are encouraged to
make this field a configurable option."
Due to the fact that it is possible to infer the type and version of web
server/application that is being used by a target by correlating information
gathering by other Information Disclosure categories, we will focus only on the
HTTP Protocol implementation analyzation that today's web fingerprinting tools
utilize.
Examples:
All of the examples below demonstrate analysis techniques of the composition
and interpretation of HTTP requests by the target web servers.
Implementation differences of the HTTP Protocol
Lexical - The lexical characteristics category covers variations in the actual
words/phrases used, capitalization and punctuation displayed by the HTTP
Response Headers.
- Response Code Message - The error code 404, Apache reports "Not
Found" whereas Microsoft IIS/5.0 reports "Object Not Found".
Apache 1.3.29 - 404
Microsoft-IIS/4.0 - 404
# telnet target1.com 80
Trying target1.com...
Connected to target1.com.
Escape character is '^]'.
HEAD /non-existent-file.txt HTTP/1.0
HTTP/1.1 404 Not Found
Date: Mon, 07 Jun 2004 14:31:03 GMT
Server: Apache/1.3.29 (Unix)
mod_perl/1.29
Connection: close
Content-Type: text/html; charset=iso-
8859-1
Connection closed by foreign host.
# telnet target2.com 80
Trying target2.com...
Connected to target2.com.
Escape character is '^]'.
HEAD /non-existent-file.txt HTTP/1.0
HTTP/1.1 404 Object Not Found
Server: Microsoft-IIS/4.0
Date: Mon, 07 Jun 2004 14:41:22 GMT
Content-Length: 461
Content-Type: text/html
Connection closed by foreign host.
- Header Wording - The header "Content-Length" is returned vs. "Content-
length".
Netscape-Enterprise/6.0 - HEAD
Microsoft-IIS/4.0 - HEAD
# telnet target1.com 80
Trying target1.com...
Connected to target1.com.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Server: Netscape-Enterprise/6.0
Date: Mon, 07 Jun 2004 14:55:25 GMT
Content-length: 26248
Content-type: text/html
Accept-ranges: bytes
Connection closed by foreign host.
# telnet target2.com 80
Trying target2.com...
Connected to target2.com.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 404 Object Not Found
Server: Microsoft-IIS/4.0
Date: Mon, 07 Jun 2004 15:22:54 GMT
Content-Length: 461
Content-Type: text/html
Connection closed by foreign host.
Syntactic - Per the HTTP RFC, all web communications are required to have a
predefined structure and composition so that both parties can understand each
other. Variations in the HTTP Response header ordering and format still exist.
- Header Ordering - Apache servers consistently place the "Date" header
before the "Server" header while Microsoft-IIS has these headers in the
reverse order.
Apache 1.3.29- HEAD
Microsoft-IIS/4.0 - HEAD
# telnet target1.com 80
Trying target1.com...
Connected to target1.com.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Mon, 07 Jun 2004 15:21:24 GMT
Server: Apache/1.3.29 (Unix)
mod_perl/1.29
Content-Location: index.html.en
Vary: negotiate,accept-language,accept-
charset
TCN: choice
Last-Modified: Fri, 04 May 2001 00:00:38
GMT
ETag: "4de14-5b0-3af1f126;40a4ed5d"
Accept-Ranges: bytes
Content-Length: 1456
Connection: close
Content-Type: text/html
Content-Language: en
Expires: Mon, 07 Jun 2004 15:21:24 GMT
Connection closed by foreign host.
# telnet target2.com 80
Trying target2.com...
Connected to target2.com.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 404 Object Not Found
Server: Microsoft-IIS/4.0
Date: Mon, 07 Jun 2004 15:22:54 GMT
Content-Length: 461
Content-Type: text/html
Connection closed by foreign host.
- List Ordering - When an OPTIONS method is sent in an HTTP Request, a
list of allowed methods for the given URI are returned in an "Allow" header.
Apache only returns the "Allow: header, while IIS also includes a "Public"
header.
Apache 1.3.29- OPTIONS
Microsoft-IIS/5.0 - OPTIONS
# telnet target1.com 80
Trying target1.com...
Connected to target1.com.
Escape character is '^]'.
OPTIONS * HTTP/1.0
HTTP/1.1 200 OK
Date: Mon, 07 Jun 2004 16:21:58 GMT
Server: Apache/1.3.29 (Unix)
mod_perl/1.29
Content-Length: 0
Allow: GET, HEAD, OPTIONS, TRACE
Connection: close
Connection closed by foreign host.
# telnet target2.com 80
Trying target2.com...
Connected to target2.com.
Escape character is '^]'.
OPTIONS * HTTP/1.0
HTTP/1.1 200 OK
Server: Microsoft-IIS/5.0
Date: Mon, 7 Jun 2004 12:21:38 GMT
Content-Length: 0
Accept-Ranges: bytes
DASL: <DAV:sql>
DAV: 1, 2
Public: OPTIONS, TRACE, GET, HEAD,
DELETE, PUT, POST, COPY, MOVE, MKCOL,
PROPFIND, PROPPATCH, LOCK, UNLOCK,
SEARCH
Allow: OPTIONS, TRACE, GET, HEAD,
DELETE, PUT, POST, COPY, MOVE, MKCOL,
PROPFIND, PROPPATCH, LOCK, UNLOCK,
SEARCH
Cache-Control: private
Connection closed by foreign host.
Semantic - Besides the words and phrases that are returned in the HTTP
Response, there are obvious differences in how web servers interpret both well-
formed and abnormal/noncompliant requests.
- Presence of Specific Headers - A server has a choice of headers to include
in a Response. While some headers are required by the specification, most
headers (e.g. ETag) are optional. In the examples below, the Apache
servers response headers include additional entries such as: ETag, Vary and
Expires while the IIS server does not.
Apache 1.3.29- HEAD
Microsoft-IIS/4.0 - HEAD
# telnet target1.com 80
Trying target1.com...
Connected to target1.com.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Date: Mon, 07 Jun 2004 15:21:24 GMT
Server: Apache/1.3.29 (Unix)
mod_perl/1.29
Content-Location: index.html.en
Vary: negotiate,accept-language,accept-
charset
TCN: choice
Last-Modified: Fri, 04 May 2001 00:00:38
GMT
ETag: "4de14-5b0-3af1f126;40a4ed5d"
Accept-Ranges: bytes
Content-Length: 1456
Connection: close
Content-Type: text/html
Content-Language: en
Expires: Mon, 07 Jun 2004 15:21:24 GMT
Connection closed by foreign host.
# telnet target2.com 80
Trying target2.com...
Connected to target2.com.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 404 Object Not Found
Server: Microsoft-IIS/4.0
Date: Mon, 07 Jun 2004 15:22:54 GMT
Content-Length: 461
Content-Type: text/html
Connection closed by foreign host.
Response Codes for Abnormal Requests - Even though the same requests
are made to the target web servers, it is possible to interpretation of the
request
to be different and therefore different response codes generated. A perfect
example of this semantic difference in interpretation is the "Light
Fingerprinting"
check which the Whisker scanner utilizes. The section of Perl code below,
taken
from Whisker 2.1's main.test file, runs two tests to determine if the target
web
server is in fact an Apache server, regardless of what the Banner might report.
The first request is a "GET //" and if the HTTP Status Code is a 200, then the
next request is sent. The second request is "GET/%2f", which is URI Encoded -
and translates to "GET //". This time Apache returns a 404 - Not Found error
code. Other web servers - IIS - do not return the same status codes for these
requests.
# now do some light fingerprinting...
-- CUT --
my $Aflag=0;
$req{whisker}->{uri}='//';
if(!_do_request(\%req,\%G_RESP)){
_d_response(\%G_RESP);
if($G_RESP{whisker}->{code}==200){
$req{whisker}->{uri}='/%2f';
if(!_do_request(\%req,\%G_RESP)){
_d_response(\%G_RESP);
$Aflag++ if($G_RESP{whisker}-
>{code}==404);
} } }
m_re_banner('Apache',$Aflag);
After running Whisker against a target website, it reports, based on the
pre-tests
that the web server may in fact be an Apache server. Below is the example
Whisker report section:
-----------------------------------------------------------------------
Title: Server banner
Id: 100
Severity: Informational
The server returned the following banner:
Microsoft-IIS/4.0
-----------------------------------------------------------------------
Title: Alternate server type
Id: 103
Severity: Informational
Testing has identified the server might be an 'Apache' server. This
Change could be due to the server not correctly identifying itself (the
Admins changed the banner). Tests will now check for this server type
as well as the previously identified server types.
-----------------------------------------------------------------------
Not only does this alert the attacker that the web server administrators are
savvy
enough to alter the Server banner info, but Whisker will also add in all of the
Apache tests to its scan which would increase its accuracy.
Solutions:
It is not possible to remove every single identifying piece of information
provided
by your web server. The fact is that a determined attack will be able to
identify
your web server software. Your goal should be to raise the bar of
reconnaissance to a height that will cause the attacker to probe hard enough
that
they will most likely trigger a security alert. The steps below will aid in
this task.
The solutions are listed in order from easiest to implement to the most complex.
Alter the Server Banner Information
It is possible to edit out and/or alter (for deception purposes) the "Server"
field
information displayed by a web server's response headers. There has been
much debate in web security circles as to the amount of protection that can be
gained by changing the HTTP Server: token information. While altering the
banner info alone, and not taking any other steps to hide the software version,
probably doesn't provide much protection from REAL people who are actively
conducting reconnaissance, it does help with regards to blocking automated
WORM programs. Due to the increase in popularity of using worms to mass
infect systems; this method of protecting your web servers becomes vital. This
step could certainly buy organizations some time during the patching phase
when new worms are released into the wild and they are configured to attack
systems based on the server token response.
Apache Servers - ModSecurity has the SecServerSignature setting, which
allows the web admin to set the Server banner info from within the httpd.conf
file
instead of editing the Apache source code prior to compilation.
IIS Servers - By installing the IISLockDown and URLScan tools, you can update
the banner info returned to clients.
Minimize the Verboseness of Information in Headers
Restrict the amount of information returned in the response headers. For
instance, Apache allows the administrator to control the verboseness of the
Server banner token by editing the ServerTokens directive:
ServerTokens Prod[uctOnly]
Server sends (e.g.): Server: Apache
ServerTokens Min[imal]
Server sends (e.g.): Server: Apache/1.3.0
ServerTokens OS
Server sends (e.g.): Server: Apache/1.3.0 (Unix)
ServerTokens Full (or not specified)
Server sends (e.g.): Server: Apache/1.3.0 (Unix) PHP/3.0 MyMod/1.2
By minimizing the headers, you may hide information such as additional apache
modules that are installed.
Implement Fake Headers
An alternate technique for defeating/confusing web server fingerprinting is to
present a fake web topology. Attackers usually include Banner Grabbing
sessions as part of the overall Footprinting Process. During Footprinting, the
attacker is trying to gauge the target's enterprise architecture. By adding in
additional fake headers, we can simulate a complex web environment (I.E.-
DMZ). By adding additional headers to simulate the existence of a reverse
proxy
server, we can create the "appearance" of a complex architecture.
For Apache servers, we can add the following httpd.conf entry to accomplish
this
task:
- Header set Via "1.1 squid.proxy.companyx.com (Squid/2.4.STABLE6)"
- ErrorHeader set Via "1.1 squid.proxy.companyx.com
(Squid/2.4.STABLE6)"
- Header set X-Cache "MISS from www.nonexistenthost.com"
- ErrorHeader set X-Cache "MISS from www.nonexistenthost.com"
These entries add the "Via" and X-Cache HTTP response headers to all
responses as shown below:
# telnet localhost 80
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
HEAD / HTTP/1.0
HTTP/1.1 200 OK
Server: Microsoft-IIS/4.0
Date: Sun, 30 Mar 2003 21:59:46 GMT
Content-Location: index.html.en
Vary: negotiate,accept-language,accept-charset
TCN: choice
Via: 1.1 squid.proxy.companyx.com (Squid/2.4.STABLE6)
X-Cache: MISS from www.nonexistenthost.com
Content-Length: 2673
Connection: close
This gives the illusion that we are using a Squid Proxy server and are
presenting
web data from a non-existent server. This might entice the attacker into
either
launching Squid Exploits against our Apache server, which would of course be
unsuccessful or to attack the host specified in the X-Cache header with does
not
actually exist.
Install Third Party Web Security Tools
By installing additional web security applications or tools, such as
ModSecurity or
ServerMask, it is possible to either disrupt or total defeat today's web server
fingerprinting applications such as HTTPrint. As described in the example
sections above, these tools will probe target web servers with many different
requests to try and initiate a specific response. Below are some of the
abnormal
requests which HTTPrint sends to the web server:
192.168.139.101 - - [08/Jun/2004:11:21:40 -0400] "JUNKMETHOD / HTTP/1.0" 501
344 "-" "-"
192.168.139.101 - - [08/Jun/2004:11:21:40 -0400] "GET / JUNK/1.0" 400 381 "-"
"-"
192.168.139.101 - - [08/Jun/2004:11:21:40 -0400] "get / HTTP/1.0" 501 330 "-"
"-"
192.168.139.101 - - [08/Jun/2004:11:21:40 -0400] "GET / HTTP/0.8" 200 1456 "-"
"-"
192.168.139.101 - - [08/Jun/2004:11:21:40 -0400] "GET / HTTP/1.2" 200 1456 "-"
"-"
192.168.139.101 - - [08/Jun/2004:11:21:40 -0400] "GET / HTTP/3.0" 200 1456 "-"
"-"
192.168.139.101 - - [08/Jun/2004:11:21:40 -0400] "GET /../../ HTTP/1.0" 400 344
"-" "-"
If we implement a tool such as ModSecurity for Apache servers, we can create
HTTP RFC compliant filters, which will trigger on these abnormal requests.
Here are some ModSecurity httpd.conf entries, which may be used:
# This will return a 403 - Forbidden Status Code for all Mod_Security actions
SecFilterDefaultAction "deny,log,status:403"
# This will deny directory traversals
SecFilter "\.\./"
# This entry forces compliance of the request method. Any requests that do NOT
# start with either GET|HEAD|POST will be denied. This will catch/trigger on
# junk methods.
SecFilterSelective THE_REQUEST "!^(GET|HEAD|POST)"
# This entry will force HTTP compliance to the end portion of the request. If
# the request does NOT end with a valid HTTP version, then it will be denied.
SecFilterSelective THE_REQUEST "!HTTP\/(0\.9|1\.0|1\.1)$"
Source Code Editing
This is the most complex task for fingerprinting countermeasures, however it is
the most effective. The risk vs. reward for this task could vary greatly
depending
on your skill level of programming or your web architecture. Generally
speaking,
this task includes editing the source code of the web server either prior to
compilation or with the actual binary using a binary editor. For open source
web
servers such as Apache, the task is much easier since you have access to the
code.
Header Ordering - Below is a source code patch for the Apache 1.3.29 server
that will correct the DATE/SERVER order and also mimic the IIS OPTIONS
output data. This patch updates the http_protocol.c file in the
/apache_1.3.29/src/main directory. The OPTIONS section will return headers
which are normally associated with IIS response tokens. These include the
Public, DASL, DAV and Cache-Control headers.
--- http_protocol.c.orig Mon Apr 26 02:11:58 2004
+++ http_protocol.c Mon Apr 26 02:43:31 2004
@@ -1597,9 +1597,6 @@
/* output the HTTP/1.x Status-Line */
ap_rvputs(r, protocol, " ", r->status_line, CRLF, NULL);
- /* output the date header */
- ap_send_header_field(r, "Date", ap_gm_timestr_822(r->pool, r-
>request_time));
-
/* keep the set-by-proxy server header, otherwise
* generate a new server header */
if (r->proxyreq) {
@@ -1612,6 +1609,9 @@
ap_send_header_field(r, "Server", ap_get_server_version());
}
+ /* output the date header */
+ ap_send_header_field(r, "Date", ap_gm_timestr_822(r->pool, r-
>request_time));
+
/* unset so we don't send them again */
ap_table_unset(r->headers_out, "Date"); /* Avoid bogosity */
ap_table_unset(r->headers_out, "Server");
@@ -1716,7 +1716,9 @@
ap_basic_http_header(r);
ap_table_setn(r->headers_out, "Content-Length", "0");
+ ap_table_setn(r->headers_out, "Public", "OPTIONS, TRACE, GET, HEAD,
DELETE, PUT, POST, COPY, MOVE, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK,
SEARCH");
ap_table_setn(r->headers_out, "Allow", make_allow(r));
+ ap_table_setn(r->headers_out, "Cache-Control", "private");
ap_set_keepalive(r);
ap_table_do((int (*) (void *, const char *, const char *))
ap_send_header_field,
Also see Reference:
"An Introduction to HTTP fingerprinting"
http://net-square.com/httprint/httprint_paper.html
"Hypertext Transfer Protocol -- HTTP/1.1"
http://www.cis.ohio-state.edu/cgi-bin/rfc/rfc2068.html#sec-14.39
"HMAP: A Technique and Tool for Remote Identification of HTTP Servers"
http://seclab.cs.ucdavis.edu/papers/hmap-thesis.pdf
"Identifying Web Servers: A first-look into Web Server Fingerprinting"
http://www.blackhat.com/presentations/bh-asia-02/bh-asia-02-grossman.pdf
"Mask Your Web Server for Enhanced Security"
http://www.port80software.com/support/articles/maskyourwebserver
"Web Intrusion Detection and Prevention"
http://www.modsecurity.org
"IIS LockDown Tool 2.1"
http://www.microsoft.com/downloads/details.aspx?FamilyID=DDE9EFC0-BB30-
47EB-9A61-FD755D23CDEC&displaylang=en
"URLScan Tool"
http://www.microsoft.com/downloads/details.aspx?FamilyID=f4c5a724-cafa-
4e88-8c37-c9d5abed1863&DisplayLang=en
"ServerMask Tool"
http://www.port80software.com/products/servermask/
Try the new Yahoo! India Homepage. Click here. http://in.yahoo.com/trynew
[Non-text portions of this message have been removed]
