I finally bit the bullet and finished this thing up today, looking for some
feedback before I go any further.
The point of this script is to locate valid directories in the web root of a
server. The list of discovered directories will be saved in the KB and be
used by any plugins which need this information. One good example would be a
plugin which looks for copies of WS_FTP.LOG in the web root, it could dimply
depend on this plugin, then iterate through all directories in the KB looking
for WS_FTP.LOG in each directory. Another example would be a plugin which
looks for directory indexes.
With some minor modifications to webmirror.nasl, it could use the list of
directories found by this plugin to perform a much more complete crawl. The
results of the crawl could be used to even further populate the directory
list in the KB. The way I would like to see this implemented:
[DDI_Directory_Scanner.nasl] -> [webmirror.nasl] -> [ws_ftp_log.nasl]
Where ws_ftp_log has a script_dependencie() for webmirror and webmirror
depends on DDI_Directory_Scanner. You could even flip that relationship and
allow the directory scanner to search the subdirectories of those found in
webmirror...
The current implementation uses 200, 403, and 401 responses to determine if a
directory exists. Only the 200 and 403 responses are considered when
populating the KB tree (not implemented yet). There is a built-in
"no404.nasl"-like check which looks for common error messages and works
around them, for servers which respond with a 200 OK for non-existent
directories. The 403 check will try to verify whether the directory itself is
protected or whether just directory indexes are disabled and there is no
default page.
The first chunk of the directory array was taken from nikto and whisker, I
browsed a few dozen vhosts on some colo machines I have access to and
populated it further based on what I found. PLEASE submit more directories if
you can and custom 404 messages that arent already handled.
I would like to take this one further and recursively search the
subdirectories of those already found, but I wanted to get something out
today for you all to play with.
Please test using the 'nasl' command line interpreter only please, there is no
script_id assigned so just dropping it into your plugin directory wont do
much do good. Here is an example run (apologies to Renaud):
hdm@masada:~> nasl DDI_Directory_Scanner.nasl -t www.nessus.org
** WARNING : packet forgery will not work
** as NASL is not running as root
DDI_Directory_Scanner.nasl : Warning : evaluating unknown variable -
description
Discovered: doc
Discovered: icons
The following directories were discovered: /doc, /icons
-HD
##
# This plugin was written by H D Moore <[EMAIL PROTECTED]>
##
if(description)
{
# script_id(????);
script_version ("$Revision: 1.0 $");
name["english"] = "Directory Scanner";
script_name(english:name["english"]);
desc["english"] = "None Yet";
script_description(english:desc["english"]);
summary["english"] = "Directory Scanner";
script_summary(english:summary["english"]);
script_category(ACT_GATHER_INFO);
script_copyright(english:"This script is Copyright (C) 2002 Digital Defense
Inc.");
family["english"] = "Misc.";
script_family(english:family["english"]);
script_dependencie("find_service.nes");
script_require_ports(80);
exit(0);
}
function check_req (port, url)
{
soc = open_sock_tcp(port);
if (!soc) return (0);
req = http_get(item:url, port:port);
send(socket:soc, data:req);
http_resp = recv(socket:soc, length:16384);
close (soc);
return(string(http_resp));
}
dirs[0] = "Admin_files";
dirs[1] = "Administration";
dirs[2] = "DMR";
dirs[3] = "Mail";
dirs[4] = "PDG_Cart";
dirs[5] = "Stats";
dirs[6] = "StoreDB";
dirs[7] = "WebShop";
dirs[8] = "WebTrend";
dirs[9] = "Web_store";
dirs[10] = "_backup";
dirs[11] = "_old";
dirs[12] = "_private";
dirs[13] = "access";
dirs[14] = "account";
dirs[15] = "accounting";
dirs[16] = "admin";
dirs[17] = "administration";
dirs[18] = "administrator";
dirs[19] = "analog";
dirs[20] = "app";
dirs[21] = "applets";
dirs[22] = "apps";
dirs[23] = "archive";
dirs[24] = "archives";
dirs[25] = "asp";
dirs[26] = "atc";
dirs[27] = "back";
dirs[28] = "backend";
dirs[29] = "backup";
dirs[30] = "bak";
dirs[31] = "banner";
dirs[32] = "beta";
dirs[33] = "bin";
dirs[34] = "buy";
dirs[35] = "buynow";
dirs[36] = "c";
dirs[37] = "cache-stats";
dirs[38] = "cart";
dirs[39] = "ccard";
dirs[40] = "core";
dirs[41] = "css";
dirs[42] = "dat";
dirs[43] = "data";
dirs[44] = "database";
dirs[45] = "db";
dirs[46] = "dbase";
dirs[47] = "design";
dirs[48] = "dev";
dirs[49] = "devel";
dirs[50] = "development";
dirs[51] = "doc";
dirs[52] = "doc-html";
dirs[53] = "docs";
dirs[54] = "down";
dirs[55] = "download";
dirs[56] = "downloads";
dirs[57] = "employees";
dirs[58] = "exe";
dirs[59] = "file";
dirs[60] = "files";
dirs[61] = "forum";
dirs[62] = "fpadmin";
dirs[63] = "ftp";
dirs[64] = "guestbook";
dirs[65] = "guests";
dirs[66] = "hidden";
dirs[67] = "hit_tracker";
dirs[68] = "hitmatic";
dirs[69] = "home";
dirs[70] = "htdocs";
dirs[71] = "html";
dirs[72] = "ibill";
dirs[73] = "icons";
dirs[74] = "idea";
dirs[75] = "ideas";
dirs[76] = "imp";
dirs[77] = "import";
dirs[78] = "includes";
dirs[79] = "incoming";
dirs[80] = "info";
dirs[81] = "install";
dirs[82] = "intranet";
dirs[83] = "java";
dirs[84] = "jdbc";
dirs[85] = "js";
dirs[86] = "lib";
dirs[87] = "library";
dirs[88] = "links";
dirs[89] = "loader";
dirs[90] = "logfile";
dirs[91] = "logfiles";
dirs[92] = "logger";
dirs[93] = "logging";
dirs[94] = "login";
dirs[95] = "mail";
dirs[96] = "manual";
dirs[97] = "mysql";
dirs[98] = "new";
dirs[99] = "old";
dirs[100] = "oracle";
dirs[101] = "order";
dirs[102] = "orders";
dirs[103] = "outgoing";
dirs[104] = "pages";
dirs[105] = "passport";
dirs[106] = "password";
dirs[107] = "passwords";
dirs[108] = "perl5";
dirs[109] = "php";
dirs[110] = "phpMyAdmin";
dirs[111] = "poll";
dirs[112] = "postgres";
dirs[113] = "private";
dirs[114] = "pub";
dirs[115] = "public";
dirs[116] = "purchase";
dirs[117] = "purchases";
dirs[118] = "pw";
dirs[119] = "register";
dirs[120] = "registered";
dirs[121] = "reports";
dirs[122] = "reseller";
dirs[123] = "restricted";
dirs[124] = "retail";
dirs[125] = "sales";
dirs[126] = "secret";
dirs[127] = "sell";
dirs[128] = "setup";
dirs[129] = "shop";
dirs[130] = "shopper";
dirs[131] = "site";
dirs[132] = "staff";
dirs[133] = "stat";
dirs[134] = "statistic";
dirs[135] = "statistics";
dirs[136] = "stats";
dirs[137] = "status";
dirs[138] = "store";
dirs[139] = "support";
dirs[140] = "sys";
dirs[141] = "system";
dirs[142] = "temp";
dirs[143] = "templates";
dirs[144] = "test";
dirs[145] = "testing";
dirs[146] = "tmp";
dirs[147] = "tools";
dirs[148] = "trafficlog";
dirs[149] = "tree";
dirs[150] = "updates";
dirs[151] = "user";
dirs[152] = "users";
dirs[153] = "ustats";
dirs[154] = "vfs";
dirs[155] = "web";
dirs[156] = "web800fo";
dirs[157] = "webadmin";
dirs[158] = "webboard";
dirs[159] = "webcart";
dirs[160] = "webcart-lite";
dirs[161] = "webdata";
dirs[162] = "webimages";
dirs[163] = "webimages2";
dirs[164] = "weblog";
dirs[165] = "weblogs";
dirs[166] = "website";
dirs[167] = "webstats";
dirs[168] = "wstats";
dirs[169] = "wusage";
dirs[170] = "www";
dirs[171] = "www-sql";
dirs[172] = "wwwjoin";
dirs[173] = "wwwlog";
dirs[174] = "wwwstats";
dirs[175] = "zipfiles";
errmsg[0] = "not found";
errmsg[1] = "404";
errmsg[2] = "error has occurred";
errmsg[3] = "firewall-1 message";
errmsg[4] = "Reload acp_userinfo database";
errmsg[5] = "IMail Server Web Messaging";
errmsg[6] = "HP Web JetAdmin";
errmsg[7] = "Error processing SSI file";
errmsg[8] = "ExtendNet DX Configuration";
debug = 1;
report = string("The following directories were discovered: ");
found = 0;
authreport = string("The following directories require authentication: ");
authfound = 0;
fake404 = string("");
Check200 = 1;
Check401 = 1;
Check403 = 1;
port = get_kb_item("Services/www");
if(!port)port = 80;
if(!get_port_state(port)) exit(0);
# first we test for servers which return 200/403/401 for everything
soc = open_sock_tcp(port);
if (!soc) exit(0);
req = http_get(item:"/NonExistent/", port:port);
send(socket:soc, data:req);
http_resp = recv(socket:soc, length:16384);
close (soc);
if(ereg(pattern:"HTTP/1.[01] 200", string: http_resp))
{
fake404 = 0;
if(debug) display("This server returns 200 for non-existent directories.\n");
for(i=0;errmsg[i];i=i+1)
{
if (ereg(pattern:errmsg[i], string:http_resp, icase:TRUE))
{
fake404 = errmsg[i];
if(debug) display("Using '", fake404, "' as an indication of a 404
error\n");
}
}
if (!fake404)
{
if(debug) display("Could not find an error string to match against for the
fake 404 response.\n");
if(debug) display("Checks which rely on 200 responses are being disabled\n");
Check200 = 0;
}
} else {
fake404 = string("BadString0987654321*DDI*");
}
if(ereg(pattern:"HTTP/1.[01] 401", string: http_resp))
{
if(debug) display("This server requires authentication for non-existent
directories, disabling 401 checks.\n");
Check401 = 0;
}
if(ereg(pattern:"HTTP/1.[01] 403", string: http_resp))
{
if(debug) display("This server returns a 403 for non-existent directories,
disabling 403 checks.\n");
Check403 = 0;
}
for(i=0;dirs[i];i=i+1)
{
res = check_req(port:port, url:string("/", dirs[i], "/"));
if( Check200 &&
ereg(pattern:"HTTP/1.[01] 200", string:res) &&
! (ereg(pattern:fake404, string:res, icase:TRUE))
)
{
if(debug) display("Discovered: " , dirs[i], "\n");
if(found)
{
report = report + ", /" + dirs[i];
found=found+1;
} else {
report = report + "/" + dirs[i];
}
found=found+1;
}
if(Check403 && ereg(pattern:"HTTP/1.[01] 403", string: res))
{
if (debug) display("Got a 403, investigating...\n");
res2 = check_req(port:port, url:string("/", dirs[i], "/NonExistent.html"));
if(ereg(pattern:"HTTP/1.[01] 403", string:res2))
{
# the whole directory appears to be protected
if (debug) display("The 403 applies to the entire directory \n");
} else {
if (debug) display("The 403 applies to just directory indexes \n");
# the directory just has indexes turned off
if(found)
{
report = report + ", /" + dirs[i];
} else {
report = report + "/" + dirs[i];
}
found=found+1;
}
}
if(Check401 && ereg(pattern:"HTTP/1.[01] 401", string: res))
{
if (debug) display("Got a 401..\n");
if(found)
{
authreport = authreport + ", /" + dirs[i];
} else {
authreport = authreport + "/" + dirs[i];
}
authfound=authfound+1;
}
}
result = string("");
if (found)
{
result = report;
}
if (authfound)
{
result = result + string("\n", authreport);
}
if (strlen(result))
{
security_note(port:port, data:result);
}
