Re: Require consistency between different data-structures sharing the same field name as of 9.0?

2020-04-21 Thread Michael McCandless
+1 We already have a minimal enforcement of consistent schema in IndexWriter. It checks that you are not trying to change the doc values type for a given field, e.g. from NUMERIC to SORTED. Maybe we could add these new checks there? E.g., if this field was previously indexed with both doc

Re: Require consistency between different data-structures sharing the same field name as of 9.0?

2020-04-21 Thread Adrien Grand
Thanks all, the initial feedback sounds positive so I opened https://issues.apache.org/jira/browse/LUCENE-9334. As Mike pointed out, this could be a first step towards having something that looks more like a proper schema. I like the idea of exposing higher-level abstractions, like a field that

Re: Require consistency between different data-structures sharing the same field name as of 9.0?

2020-04-20 Thread David Smiley
What Alan says makes sense to me -- simple and sufficiently addresses the pain point Adrien raises. I don't think we need a long class name which is also some extra class in addition to the current one (confusing); I'd prefer one simple numeric class name (per int/long/float/double) with a

Re: Require consistency between different data-structures sharing the same field name as of 9.0?

2020-04-20 Thread Michael Sokolov
Could we use this as a stepping stone towards a schema? Just a very lightweight schema that only enforces what we can easily enforce today, but put some minimal abstraction in place where we can hang future consistency checks. Re: value consistency; could we do a best-effort enforcement in

Re: Require consistency between different data-structures sharing the same field name as of 9.0?

2020-04-20 Thread Alan Woodward
One way of doing this might be to add an additional field type that adds both point and docvalues, and then have factory methods for queries and sorts on the field type. So for example a LongPointAndValue field would automatically index its value into both BKD and NumericDocValues, and then