On Thu, Aug 22, 2013 at 1:48 AM, Sean Bridges <sean.brid...@gmail.com> wrote: > Is there a supported DocValuesFormat that doesn't load all the values into > ram?
Not with any current release, but in lucene 4.5 if all goes well, the official implementation will work that way (I spent essentially the last entire week on this and committed it yesterday). Integrating new ideas into the official format takes a good amount of effort: lots of documentation and testing and so on, because we have to live with supporting that format for a long time, all the way until 5.9. . > > We can't reindex every time we upgrade lucene since our indexes are too > large. Should we copy the code from DiskDocValuesFormat and call it > CustomDiskDocValuesFormat, and give CustomDiskDocValuesFormat a new name so > that when we upgrade lucene, we won't use an incompatible version of > DiskDocValuesFormat? > You can certainly maintain your own codec components: you can even name them the same thing as long as you put your .jar file first in the classpath (thats how SPI works: first one wins). But its not really a cure-all, its some work either way: codec APIs themselves change too, so you have to deal with that on upgrade (e.g. DocValuesProducer gets a new method in 4.5 as its now capable of representing missing values, and the iterators in DocValuesConsumer now provide null when a document is missing a value). Or, you can avoid reindexing by using addIndexes as I suggested (just buy a few big chips of RAM for the upgrade). --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
