We had to patch it because we were getting seemingly random errors while searching a 2GB+ index. This the trac ticket: http:// ferret.davebalmain.com/trac/ticket/215. The patch I included changes some ints to off_t's, which solved the problem. As far as I know this patch was never applied to the trunk.
We build our index using a modified version of RDig. We basically run up to 80 EC2 servers in parallel to create 80 separate indexes, which we later combine into a single index. You could follow a similar route and still have AAF mange the index after it is built. You'd need to make sure that the documents created by RDig/whatever have the same fields that AAF expects. Erik On Aug 8, 2007, at 4:53 PM, Craig Jolicoeur wrote: > Erik Morton wrote: >> We have a 1 million record index that is about 6GB in size. We build >> it in parallel w/out AAF so it's hard to comment on the speed of your >> index build. However I will say that I did need to manually patch >> Ferret to better handle large indexes. >> > > > Erik, > > What issues did you find that caused you to patch the ferret code? > > ALso, you say you build the index in parallel w/out AAF; how do you do > that? Not sure I'm following how to do that so if you can explain, > I'd > appreciate it. > -- > Posted via http://www.ruby-forum.com/. > _______________________________________________ > Ferret-talk mailing list > [email protected] > http://rubyforge.org/mailman/listinfo/ferret-talk _______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
