After having installed and compiled htdig successfully (woohoo!), I came
across an aspect of the program that's not covered in the docs.
How does/should htdig handle these two urls:
http://www.somedomain.com/
http://www.somedomain.com/products.html
My observation has it that it will spider the site if it sees just a domain,
otherwise it will index the page if one is specified. The reason i ask
this, is that i pointed it to a file which is basically a map of the html
documents of a site, and it did not follow any links. but if i point it to
the main domain name, it does indeed follow.
Thanks,
Mark
------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
[EMAIL PROTECTED] containing the single word "unsubscribe" in
the SUBJECT of the message.
