Is there a supported DocValuesFormat that doesn't load all the values into ram?
Our use is case is that we have 16 byte ids for all our documents. We used to store the ids in stored fields, and look up the stored field for each search hit. We got much better performance when we switched to storing our ids in DiskDocValues rather than StoredFields, especially when we had a lot of search hits. We could use the Lucene42DocValuesFormat, but that loads all the values into ram. We can't reindex every time we upgrade lucene since our indexes are too large. Should we copy the code from DiskDocValuesFormat and call it CustomDiskDocValuesFormat, and give CustomDiskDocValuesFormat a new name so that when we upgrade lucene, we won't use an incompatible version of DiskDocValuesFormat? Thanks, Sean On Wed, Aug 21, 2013 at 8:44 AM, Robert Muir <rcm...@gmail.com> wrote: > On Wed, Aug 21, 2013 at 11:30 AM, Sean Bridges <sean.brid...@gmail.com> > wrote: > > What is the recommended way to use DiskDocValuesFormat in production if > we > > can't reindex when we upgrade? > > I'm not going to recommend using any experimental codecs in production, > but... > > 1. with 4.3 jar file: IWC.setCodec(Codec.getDefault()) + > IndexWriter.addIndexes(IndexReader) -> converts index to official 4.3 > format > 2. with 4.4 jar file: IWC.setCodec(MyExperimentalCodec) + > IndexWriter.addIndexes(IndexReader) -> converts index to customized > codec on 4.4 > > > > > Will the 4.4 version of DDVF be backwards compatible > > no. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > >
