Figured it out. It's interesting, academically. I was flushing every time I added something to the index. I forget exactly why I thought that was a good idea, but that's what I was doing.
Apparently, that's a bad idea under 0.11. Worked fine under 0.10, not so fine under 0.11. Like, 75x less fine. Now I'm just flushing after every batch. It's back to reasonable, 5-seconds-for-10,000-small-records performance. Thanks for all your help, Dave. Schnitz On 3/5/07, David Balmain <[EMAIL PROTECTED]> wrote:
On 3/5/07, Matt Schnitz <[EMAIL PROTECTED]> wrote: > Hi - I upgraded to Ferret 0.11.3 from 0.10.13. > > I used to index 10,000 records in 10 secs. Now it takes 13 minutes. > (That's a factor of ~75x) > > Did something change in the flush semantics, or something? Hi Matt, The opening of an index takes a little longer now. I guess if you have the index set to :auto_flush then it could take a fair bit longer but I didn't expect it to take that much longer. Unfortunately this slowdown was a price I had to pay to prevent the segfault and FileNotFound errors that people where getting. Having said that, you shouldn't have :auto_flush set when you are batch indexing anyway. If you send me a benchmark which approximates what you are doing, I'd be happy to take a look at it for you and tell you how to make it faster or add a fix to Ferret if the problem does happen to be at this end. Cheers, Dave -- Dave Balmain http://www.davebalmain.com/ _______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
_______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
