if i insert a bunch of URLs into the db via a text file.. and those
urls have no corresponding server command, by default, it should have a
Follow of path and NOT site.
here is one example. i pull URLs from a sub section of dmoz using the
Href directive. Some of those are site URLs and some are a path of a
site. now, when i pull all the urls from that sub section of dmoz, the
current setting will index thru all the "whole" sites/servers instead
of just the paths.
if default is to follow path, then the whole site would still get
indexed. i hope that made sense. i brought this topic up about a month
ago and i still see this behavior in -current.
__________________________________________________
Do You Yahoo!?
Get email at your own domain with Yahoo! Mail.
http://personal.mail.yahoo.com/
___________________________________________
If you want to unsubscribe send "unsubscribe general"
to [EMAIL PROTECTED]
