Hey Dave, thanks for the reply. At this stage, the file size is about 20GB's, so it's a lot. Each file in the index (not optimized) is about 2GB, so it does seem we have hit the limit you are talking about. The box has over 140 GB's of storage, so it's not out of space, but it's consistently stopping at the same point each time.
We've used gem install ferret to grab the latest (0.10.9) release. What's the way to compile it with this "large-file" support flag? Thanks. > From: "David Balmain" <[EMAIL PROTECTED]> > Reply-To: [email protected] > Date: Thu, 5 Oct 2006 09:57:22 +0900 > To: [email protected] > Subject: Re: [Ferret-talk] Error while optimizing my index > > On 10/5/06, peter <[EMAIL PROTECTED]> wrote: >> Hello. >> >> We are indexing a large amount of data (about 850,000 records). During the >> indexing, every 100,000 items I flush and optimize the index, so the disk >> space doesn't get too large during indexing. >> >> About 700,000 items in, I got this error: >> Error occured in fs_store.c:226 - fso_flush_i >> flushing src of length 1024 >> >> >> >> /usr/lib/ruby/gems/1.8/gems/ferret-0.10.9/lib/ferret/index.rb:517:in `close' >> >> /usr/lib/ruby/gems/1.8/gems/ferret-0.10.9/lib/ferret/index.rb:517:in `flush' >> /usr/lib/ruby/1.8/monitor.rb:229:in `synchronize' >> >> /usr/lib/ruby/gems/1.8/gems/ferret-0.10.9/lib/ferret/index.rb:514:in `flush' >> >> >> Is there a way to find out what happened? It wasn't a space error, as >> there's plenty of space on the machine. >> >> Also, if I optimize during an index run such as this, do I need to close and >> then re-open the index, or will that not affect things? >> >> Thanks for any info you have. > > Hi Peter, > > How much space is the indexing taking up by this stage? You may be > hitting the 2Gb limit in which case you will need to compile Ferret > with large-file support. Otherwise, I'm not sure why you'd be getting > this error. I'm going to have to improve that error message. > > Cheers, > Dave > _______________________________________________ > Ferret-talk mailing list > [email protected] > http://rubyforge.org/mailman/listinfo/ferret-talk > _______________________________________________ Ferret-talk mailing list [email protected] http://rubyforge.org/mailman/listinfo/ferret-talk
