According to Dan Langille: > There are a number of variables in that equation, but I thought I'd share > my situation. I have a dual Pentium box with 64MB of ram. > > freebsddiary.org > > website #pages time > > 1 diary 500-ish 0:02:30 > 2 archive 13900 1:50:00-ish > 3 archive 13900 1:40:00-ish (using local_urls) > > All digs are run on the webserver. If they were run remotely, I'd expect > a much bigger different between example 2 and example 3.
Don't you mean the other way around? If they were run remotely, the local_urls wouldn't have much effect. Even locally, though, it may be that it's falling back to HTTP an awful lot. Note that only a very small set of file suffixes is handled by local_urls (see the attrs.html documentation for local_urls). Files without suffixes (or extensions) don't get handled locally, because htdig can't be sure what content-type they are. > The diary site gets reindexed as new pages are added. The frequency > varies from twice a week to once a month. It's no big deal to reindex in > full each time. > > The other site contains a mailing list archive. From time to time, we > get 80 new pages a day. It's not practical to reindex that every day. > I'm going to look at incremental digs. Sometime soon... Incremental digs are a useful way of dealing with small updates to large archives like this. They can be done with htdig -m in the new snapshots, or by digging a separate database and merging it into the main one. -- Gilles R. Detillieux E-mail: <[EMAIL PROTECTED]> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930 _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
