2.0 is pre Mike's fabulous indexing updates - which just for one means one thread doing the merging rather than multiple. I'm sure overall its much slower.
But you can't take advantage of the newer faster code without updating Lucene in your app. Your best bet is to put it another machine and take a week and put it back - but your down for a brief period on swapping in the index. So it seems just as good to update Lucene - swap in the update real quick (after fixing your code offline), and then do the faster optimize. You can still serve search requests during the 2 hour optimize - performance will be affected though. Erick Erickson wrote: > Would it work to copy your entire index to a new directory, perhaps on > a different machine and optimize *there*? Then copy back to your app. > Of course updates would be lost... > > But taking a week to optimize a 20G index seems just plain wrong. Have > you tried playing with the various options to see if you can get > better performance? And/or allocating more memory to the JVM? > > Of course I'm not very familiar with 2.0 performance, so..... > > Best > Erick > > On Thu, Oct 1, 2009 at 11:40 AM, lowfreq <hughmorri...@hotmail.com > <mailto:hughmorri...@hotmail.com>> wrote: > > > I have a Lucene index that is very large in size. > It was created using a pre 2.1 version of Lucene.net 2.0.0.4. > > The index is currently almost 20 GB, and has almost 7000 segment > files. > The problem I am having is that I need to optimize it, and cant do > this > without the search functionality of my app being down for a week. > > I used the Luke tool from getopt.org <http://getopt.org> and it > worked flawlessly, optimizing > the index in just over 2 hours. Problem is that my search cannot > use it, and > the error states Unknown Format Version errors, or just plain > nothing found. > > I understand that versions of Lucene that are newer than what the > index was > built and is searched with can cause problems. > > What can I do to make this work? I have tried older versions of > Luke, 0.7 > was the oldest I could lay hands on, but even it uses a newer > version of > Lucene. > > My index version shows as 633103800023469045. The version the index is > written as after optimizing with Luke 7.0 is 633103800023469057. > > Any help here would be awesome! > > Thank you, > > Hugh > > -- > View this message in context: > > http://www.nabble.com/Optimization-and-Corruption-Issues-tp25697034p25697034.html > Sent from the Lucene - Java Developer mailing list archive at > Nabble.com. > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > <mailto:java-dev-unsubscr...@lucene.apache.org> > For additional commands, e-mail: java-dev-h...@lucene.apache.org > <mailto:java-dev-h...@lucene.apache.org> > > -- - Mark http://www.lucidimagination.com --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
