At 11:09 AM -0500 8/14/00, Gilles Detillieux wrote: > > database that I have currently is 2,5 GB big. I am running it on >RH 6.2, x86 > > . I am using Mylex RAID controller with 8 MB cache and 3x 18G 10.000 rpm > > drives with 4 MB cache on each, RAID level 5 for database. system >drives are > > on level 1. Processor is 333 MHz with 192 MB ram (getting upgraded to dual > > P3 733 , 512 MB ram this week). Searching takes very long time, 43 seconds > > for word that gives 4700 hits. Word with 620 hits takes 6 seconds. And that > > is with only "exact" method used. > > How can I improve performance? What kind of indexing does htdig use > > currently? Is there any plan for indexing in more levels? > > Final database is going to be 4x bigger, and I cant affor to let >people wait > > 2+ minutes for hit :( > >I'm not really the person to ask, so if you don't get any other response, >I suggest you repost to [EMAIL PROTECTED] as a separate thread. We've heard >some reports about 3.2.0b2 being slow for searches (one said it was slow >for the first few then sped up), but we haven't really gotten to the bottom >of this. Other reports indicated 3.2 was faster than 3.1.x, which is more >what we expected, due to the new database structure. I'm taking this to the list myself because it merits response. Searches in 3.2.x will definitely be faster, though certainly words with few hits will almost always be faster. Benchmarking by Loic and myself suggest that while indexing may be slower for some people, truly large databases will see a performance improvement because the new compression reduces disk I/O. I'll also point out that performance is one of the areas that we're working on heavily. The final 3.2.0 code will have result caching and a few other methods to speed up searching and the final code will also have faster indexing (than the betas) with some new database code we're developing. When developing 3.2, we've been concerned with improving the scaling and we hope we're getting there. It remains to be seen how much will make 3.2.0b3 and how much is still under development. -- -Geoff Hutchison Williams Students Online http://wso.williams.edu/ ------------------------------------ To unsubscribe from the htdig mailing list, send a message to [EMAIL PROTECTED] You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>
