> On Wed, 2004-12-15 at 15:17, Didier McGillis wrote:
> > Considering this is a pretty knowledgable group is there a good free
> > HTTP Sniffer application I can use, I need to see what the HTTP
> > headers are returning on my site.
On Wed, Dec 15, 2004 at 05:48:53PM -0800, Bob Feretich wrote:
> I use Ethereal on my Windows client. It works well, but only for
> communication across a real port. It does not work if your accessing a
> site that is running on the client computer.
>
> The "Follow TCP Stream" tool assembles the entire request/response
> communication for easy viewing or saving to disk.
I think monitoring the actual http traffic of your application is
an essential step in both debugging and in learning about HTTP and how
your browser and server are behaving. You basically have three
options here, which are:
- setting up a proxy server
- sniffing the network traffic
- plugging into your browser or server
When I first started doing this, I was using a freeware HTTP
proxy called HttpSpy at the time (1997 I think). Later I found it had
become shareware. I just googled for it now and found:
http://rwtemple.com:81/software/HttpSpy/
Which says it's free. I'm not sure if this is the same
developer, though it's clearly not the same exact package (different
GUI and it's written in .NET). The nice thing about the version of
HttpSpy that I used was that it was fairly smart about the http
protocol, compared to tcpdump or tcpflow or a simple port-forwarding
proxy server. For example, HttpSpy wouldn't dump the binary data for
a graphic to your screen, it'd just say something like "binary data
file foo.jpg."
One slight downside of HttpSpy, IMHO, is that the GUI is *too*
nice, it kind of hides the essential underlying simplicity of the HTTP
protocol from you. I think it's really a good thing to get a strong
sense that you could literally type all of these HTTP requests and
responses out yourself, fairly easily with a good reference work in
front of you. This demystifies the process, and you lose that with
HttpSpy. But once you've acquired that feel for what's going on,
something like HttpSpy will probably be more convenient than paging
through a raw text screen.
For a while I used tcpdump, which is a little bit arcane to use
and has some political issues.
On the political level, some organizations get upset if you fire
up a packet sniffer on their network - and almost all modern ethernet
cards are hardwired to broadcast an announcement when you put them
into "promiscuous mode", where they start paying attentiont o all of
the packets that come across the wire and report them up to the
software level. Also, security-conscious sysadmins will often not
install tcpdump, or remove the tcpdump installation that is normally
there. Finally, on unix systems, if tcpdump is there, you normally
nneed to have root access to use it.
On the technical level, you have to puzzle out the command line
arguments, which isn't that hard, and you also have to do some
post-processing on the data to make it intelligible. Unfortunately, I
no longer have my notes from whenI puzzled this out for myself. The
man pages for tcpdump have a fair amount of example recipes for the
arguments. Some googling should turn up examples and explanations
of how to do this.
There is a port of tcpdump to Windows, named WinDump. Here's
the main site and a mirror:
http://windump.polito.it/
http://windump.mirror.ethereal.com/docs/default.htm
I limped along with tcpdump for a while, but I wanted something
that made life a little bit easier, so tried out tcpflow. Tcpflow is
basically a specialized packet sniffer, geared towards troubleshooting
session-oriented network protocols, like IMAP. It's also effective
for http.
The one thing I didn't like about tcpflow is that it creates a
separate file for each request and for each response, naming the files
with system names and timestamps, instead of in a series like HttpSpy
does. I emailed the creator, asking him if there was a way to get it
to present the data in a more sequential format, but in his reply he
explained that the apparent request/response sequentiality of http is
really an illusion. At the network level, they're all just packets
flying back and forth.
He also highly recommended "TCP/IP Illustrated", by Richard Stevens,
more formally known as:
TCP/IP Illustrated, Volume 1: The Protocols
Addison-Wesley Publishing Company
ISBN 0201633469
You can find it at Amazon, but I'd prefer you to spend your money
at tatteredcover.com, and support business owners like Joyce Meskis,
who are defending our rights:
http://fact.trib.com/1st.lev.tatteredcoverrec.html
http://www.google.com/search?num=100&hl=en&lr=lang_en&as_qdr=all&q=%22tattered+cover%22+%22first+amendment%22&btnG=Search&lr=lang_en
Still, tcpflow was pretty useful. I got something approximating
sequentiality by redirecting all of tcpflow's output to a single file.
I had some thoughts about writing some sort of specialized display to
line up the individual tcpflow output files in sequential order, but I
never really got around to it.
In between tcpdump and tcpflow, I used a pretty simple little
logging portforwarder, coded in java, that a friend whipped up for me
when I mentioned the problem. He's got a copy on his web page at
darksleep.com/player, at the bottom of the page.
Zaval, the monitoring proxy server which another poster
recommended, looks kind of neat:
http://www.zaval.org/products/proxy/
It's not clear from the page but it looks free. They're also
talking about porting it to java.
And in closing, here's a webpage with an interesting list of "the
top 75 network security tools":
http://www.insecure.org/tools.html
--
Steven J. Owens
[EMAIL PROTECTED]
"I'm going to make broad, sweeping generalizations and strong,
declarative statements, because otherwise I'll be here all night and
this document will be four times longer and much less fun to read.
Take it all with a grain of salt." - http://darksleep.com/notablog
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
