At 3:28 PM -0500 3/23/01, Walker, Ken wrote:
>I'm currently running htdig on our state web site. I need to index about 25
>sites, some small, some large. I'm running the indexing on a Sun E250 with
>dual processors, 2 gig of ram, and lots of disk space. I currently list the
>url for each web site in one config file. It is taking in excess of 20
>hours to do the initial dig.
Since you don't seem to have an idea of the number of URLs, it's hard
to say anything intelligent. For one, if you're indexing a lot of
URLs over a slow link (or the servers on the other end are choking),
it will take some time.
>Would there be any performance gain to setting up multiple conf
>files for different sites and then using htmerge to merge them all
>together? I'm running version 3.1.4.
For one, upgrade to 3.1.5. <http://www.htdig.org/RELEASE.html>
As for your question of digging multiple bits, it depends mostly on
your network connection and the speed of the other servers. If you're
not saturating your server's link with one connection, then you'll
see improvement doing multiple digs. If you're using a good chunk of
bandwidth or you want indexing to leave room for serving a
high-traffic site, then it's probably not such a great idea.
--
--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/
_______________________________________________
htdig-general mailing list <[EMAIL PROTECTED]>
To unsubscribe, send a message to <[EMAIL PROTECTED]> with a
subject of unsubscribe
FAQ: http://htdig.sourceforge.net/FAQ.html
