At 2:57 AM +0100 1/14/02, Ace wrote: >I have htdig running on a multiprocessor machine, and whilst the >first CPU has 99.9% load, the second is idle most of the time. > >If there was a way to split the work into some pieces, this might >speed up the indexing process a lot, I suppose.
Indexing is certainly not multithreaded, so there's no "SMP-enabled" htdig. Besides, the speed of your indexing is largely dependent on the speed of the network connection and the speed of the disk serving the database in many cases. If you can come up with ways of splitting the URLs into "chunks," you can consider running two htdig indexing process together. Then run htmerge on each database and then use the htmerge -m to merge one into the other (or each one into another database). <http://www.htdig.org/FAQ.html#q4.5> -- -- -Geoff Hutchison Williams Students Online http://wso.williams.edu/ _______________________________________________ htdig-general mailing list <[EMAIL PROTECTED]> To unsubscribe, send a message to <[EMAIL PROTECTED]> with a subject of unsubscribe FAQ: http://htdig.sourceforge.net/FAQ.html
