Re: IntField to IntPoint

2019-06-07 Thread Erick Erickson
Omitting norms and the like only matters for text fields, primitives (numerics, boolean, string) don’t have any of that information. You really have no choice but to re-index to jump from 4->7. Or, I should say you’re completely unsupported and you will have to deal with any anomalies. I supp

Re: IntField to IntPoint

2019-06-07 Thread Riccardo Tasso
Thanks Erik for your answer. Unfortunately I should migrate the index for time reasons. Maybe in a second moment we will have the opportunity to reindex. Our use case is to classify documents in the index with lucene queries, hence we're not really interested in ranking or sorting (which could be

Re: IntField to IntPoint

2019-06-06 Thread Erick Erickson
> On Jun 5, 2019, at 2:07 PM, Riccardo Tasso wrote: > > > Considering that the IndexUpgrader will efficiently do the most of the work > I should investigate how to fill this gap, without reindexing from scratch. > > This is actually a problem. IndexUpgraderTool creates a single massive seg

Re: IntField to IntPoint

2019-06-05 Thread Trejkaz
How we would do it: - update the index format to v7 (this in itself is fiddly but there are ways) - open the index in-place migrated: - get all the leaf indices and wrap each in a new subclass of FilterCodecReader - override getPointsReader() on that subclass to return a corr

Re: IntField to IntPoint

2019-06-05 Thread Riccardo Tasso
Ok, I know this policy and you perfectly explained why it makes sense. Anyway my index is really big and contains mostly textual data which are expensive to reindex (because of custom analysis). Considering that the IndexUpgrader will efficiently do the most of the work I should investigate how

Re: IntField to IntPoint

2019-06-05 Thread Erick Erickson
You cannot upgrade more than one major version, you must re-index from scratch. There’s a long discussion of why, but basically it’s summed up by this quote from Robert Muir: “I think the key issue here is Lucene is an index not a database. Because it is a lossy index and does not retain all of

IntField to IntPoint

2019-06-05 Thread Riccardo Tasso
Hello everybody, I have a (very big) lucene 4 index with documents using IntField. On that field, which should be stored and sortable, I should search and execute range queries. I've tried to upgrade it from 4 to 7 with IndexUpgrader but I observed that IntFields aren't searchable anymore. Which