I have been using htdig for years to index my local pc. htdig is part of the Suse linux distribution I use. On a 2.4GHz cpu indexing the entire pc took 2 hours with htdig 3.1.6, but with version 3.2.0b6 it takes much longer. The words database becomes htdig's bottleneck when it exceeds 200MB on my pc. So I divided the search domain according to local directory structure and got smaller words databases and faster digs. The disadvantage is a lot of config files.
Searching itself is faster: 3000 hits for FAQ are found in 30 secs by 3.2.0b6 while 3.1.6 needed 60 secs. And phrases are a quantum leap. Currently merging of databases is not any faster than indexing directly. So if database update cannot be accelerated due to limitations of db software, techniques for divide and conquer could be improved and should be made more common in documentation and configuration examples of htdig. Thanks for a great software Klaus ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://productguide.itmanagersjournal.com/ _______________________________________________ ht://Dig Developer mailing list: [EMAIL PROTECTED] List information (subscribe/unsubscribe, etc.) https://lists.sourceforge.net/lists/listinfo/htdig-dev
