Re: site spider using mech

[Jonathan Kamens]

Use $mech->head instead of $mech->get and look at the content type of
the resulting response to find out whether it's something you want to
download.