> I've an index file (index.html) containing lots of URLs (more than 2000) > that I use as start_url.
You would probably be happier with a text file of URLs and doing something like: start_url: `/path/to/urls.txt` But to answer your question, it's not a matter of "max number of URLs" but most likely an issue of "maximum size of document," which exists to prevent a nasty webserver from spewing out data w/o limit to crash htdig. <http://www.htdig.org/FAQ.html#q5.1> > The "server_max_docs" parameter is not set in htdig.conf, so I assume > it's > the default: Yes. See, for example: <http://www.htdig.org/FAQ.html#q4.18> -- -Geoff Hutchison Williams Students Online http://wso.williams.edu/ ------------------------------------------------------- This sf.net email is sponsored by: Dice - The leading online job board for high-tech professionals. Search and apply for tech jobs today! http://seeker.dice.com/seeker.epl?rel_code=31 _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
