Re: Solr Float/Double multivalues fields
Thanks for sharing the post, finally I had the time to read it :) It is really illuminating On Fri, Jul 3, 2020 at 1:28 PM Toke Eskildsen wrote: > On Fri, 2020-07-03 at 10:00 +0200, Vincenzo D'Amore wrote: > > Hi Erick, not sure I got. > > Does this mean that the order of values within a multivalued field: > > - docValues=true the result will be both re-ordered and deduplicated. > > - docValues=false the result order is guaranteed to be maintained for > > values in the insertion-order. > > > > Is this correct? > > Sorta, but it is not the complete picture. Things gets complicated when > you mix it with stored, so that you have "stored=true docValues=true". > There's an article about that at > > > https://sease.io/2020/03/docvalues-vs-stored-fields-apache-solr-features-and-performance-smackdown.html > > BTW: The documentation should definitely mention that stored preserves > order & duplicates. It is not obvious. > > - Toke Eskildsen, Royal Danish Library > > > -- Vincenzo D'Amore
Re: Solr Float/Double multivalues fields
Op vr 3 jul. 2020 om 14:11 schreef Bram Van Dam : > On 03/07/2020 09:50, Thomas Corthals wrote: > > I think this should go in the ref guide. If your product depends on this > > behaviour, you want reassurance that it isn't going to change in the next > > release. Not everyone will go looking through the javadoc to see if this > is > > implied. > > This is in the ref guide. Section DocValues. Here's the quote: > > DocValues are only available for specific field types. The types chosen > determine the underlying Lucene > docValue type that will be used. The available Solr field types are: > • StrField, and UUIDField: > ◦ If the field is single-valued (i.e., multi-valued is false), Lucene > will use the SORTED type. > ◦ If the field is multi-valued, Lucene will use the SORTED_SET type. > Entries are kept in sorted order and > duplicates are removed. > • BoolField: > ◦ If the field is single-valued (i.e., multi-valued is false), Lucene > will use the SORTED type. > © 2019, Apache Software Foundation > Guide Version 7.7 - Published: 2019-03-04 > Page 212 of 1426 > Apache Solr Reference Guide 7.7 > ◦ If the field is multi-valued, Lucene will use the SORTED_SET type. > Entries are kept in sorted order and > duplicates are removed. > • Any *PointField Numeric or Date fields, EnumFieldType, and > CurrencyFieldType: > ◦ If the field is single-valued (i.e., multi-valued is false), Lucene > will use the NUMERIC type. > ◦ If the field is multi-valued, Lucene will use the SORTED_NUMERIC type. > Entries are kept in sorted order > and duplicates are kept. > • Any of the deprecated Trie* Numeric or Date fields, EnumField and > CurrencyField: > ◦ If the field is single-valued (i.e., multi-valued is false), Lucene > will use the NUMERIC type. > ◦ If the field is multi-valued, Lucene will use the SORTED_SET type. > Entries are kept in sorted order and > duplicates are removed. > These Lucene types are related to how the values are sorted and stored. > Great for docValues. But I couldn't find anything similar for multiValued in the field type pages of the ref guide (unless I totally missed it of course). It doesn't have to be as elaborate, as long as it's clear and doesn't leave users wondering or assuming.
Re: Solr Float/Double multivalues fields
On 03/07/2020 09:50, Thomas Corthals wrote: > I think this should go in the ref guide. If your product depends on this > behaviour, you want reassurance that it isn't going to change in the next > release. Not everyone will go looking through the javadoc to see if this is > implied. This is in the ref guide. Section DocValues. Here's the quote: DocValues are only available for specific field types. The types chosen determine the underlying Lucene docValue type that will be used. The available Solr field types are: • StrField, and UUIDField: ◦ If the field is single-valued (i.e., multi-valued is false), Lucene will use the SORTED type. ◦ If the field is multi-valued, Lucene will use the SORTED_SET type. Entries are kept in sorted order and duplicates are removed. • BoolField: ◦ If the field is single-valued (i.e., multi-valued is false), Lucene will use the SORTED type. © 2019, Apache Software Foundation Guide Version 7.7 - Published: 2019-03-04 Page 212 of 1426 Apache Solr Reference Guide 7.7 ◦ If the field is multi-valued, Lucene will use the SORTED_SET type. Entries are kept in sorted order and duplicates are removed. • Any *PointField Numeric or Date fields, EnumFieldType, and CurrencyFieldType: ◦ If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type. ◦ If the field is multi-valued, Lucene will use the SORTED_NUMERIC type. Entries are kept in sorted order and duplicates are kept. • Any of the deprecated Trie* Numeric or Date fields, EnumField and CurrencyField: ◦ If the field is single-valued (i.e., multi-valued is false), Lucene will use the NUMERIC type. ◦ If the field is multi-valued, Lucene will use the SORTED_SET type. Entries are kept in sorted order and duplicates are removed. These Lucene types are related to how the values are sorted and stored.
Re: Solr Float/Double multivalues fields
On Fri, 2020-07-03 at 10:00 +0200, Vincenzo D'Amore wrote: > Hi Erick, not sure I got. > Does this mean that the order of values within a multivalued field: > - docValues=true the result will be both re-ordered and deduplicated. > - docValues=false the result order is guaranteed to be maintained for > values in the insertion-order. > > Is this correct? Sorta, but it is not the complete picture. Things gets complicated when you mix it with stored, so that you have "stored=true docValues=true". There's an article about that at https://sease.io/2020/03/docvalues-vs-stored-fields-apache-solr-features-and-performance-smackdown.html BTW: The documentation should definitely mention that stored preserves order & duplicates. It is not obvious. - Toke Eskildsen, Royal Danish Library
Re: Solr Float/Double multivalues fields
Hi Erick, not sure I got. Does this mean that the order of values within a multivalued field: - docValues=true the result will be both re-ordered and deduplicated. - docValues=false the result order is guaranteed to be maintained for values in the insertion-order. Is this correct? On Thu, Jul 2, 2020 at 8:37 PM Erick Erickson wrote: > This is true _unless_ you fetch from docValues. docValues are SORTED_SETs, > so the results will be both ordered and deduplicated if you return them > as part of the field list. > > Don’t really think it needs to go into the ref guide, it’s just inherent > in storing > any kind of value. You wouldn’t expect multiple text entries in a > multiValued > field to be rearranged when returning the stored values either. > > Best, > Erick > > > On Jul 2, 2020, at 2:21 PM, Vincenzo D'Amore wrote: > > > > Thanks, and genuinely asking: is there written somewhere in the > > documentation too? If no, could anyone suggest to me which doc page > should > > I try to update? > > > > On Thu, Jul 2, 2020 at 8:08 PM Colvin Cowie > > wrote: > > > >> The order of values within a multivalued field should match the > insertion > >> order. -- we certainly rely on that in our product. > >> > >> Order is guaranteed to be maintained for values in a multi-valued field. > >>> > >> > >> > https://lucene.472066.n3.nabble.com/order-question-on-solr-multi-value-field-tp4027695p4028057.html > >> > >> On Thu, 2 Jul 2020 at 18:52, Vincenzo D'Amore > wrote: > >> > >>> Hi all, > >>> > >>> simple question: Solr float/double multivalue fields preserve the order > >> of > >>> inserted values? > >>> > >>> Best regards, > >>> Vincenzo > >>> > >>> -- > >>> Vincenzo D'Amore > >>> > >> > > > > > > -- > > Vincenzo D'Amore > > -- Vincenzo D'Amore
Re: Solr Float/Double multivalues fields
I think this should go in the ref guide. If your product depends on this behaviour, you want reassurance that it isn't going to change in the next release. Not everyone will go looking through the javadoc to see if this is implied. Typically it'll either be something like "are always returned in insertion order" or "are currently returned in insertion order, but your code shouldn't rely on this behaviour because it can change in future releases". That's usually sufficient to make an informed decision on how to handle returned values. If it's different for docValues, that's even more reason to state it clearly in the ref guide to avoid confusion. Best, Thomas Op do 2 jul. 2020 om 20:37 schreef Erick Erickson : > This is true _unless_ you fetch from docValues. docValues are SORTED_SETs, > so the results will be both ordered and deduplicated if you return them > as part of the field list. > > Don’t really think it needs to go into the ref guide, it’s just inherent > in storing > any kind of value. You wouldn’t expect multiple text entries in a > multiValued > field to be rearranged when returning the stored values either. > > Best, > Erick > > > On Jul 2, 2020, at 2:21 PM, Vincenzo D'Amore wrote: > > > > Thanks, and genuinely asking: is there written somewhere in the > > documentation too? If no, could anyone suggest to me which doc page > should > > I try to update? > > > > On Thu, Jul 2, 2020 at 8:08 PM Colvin Cowie > > wrote: > > > >> The order of values within a multivalued field should match the > insertion > >> order. -- we certainly rely on that in our product. > >> > >> Order is guaranteed to be maintained for values in a multi-valued field. > >>> > >> > >> > https://lucene.472066.n3.nabble.com/order-question-on-solr-multi-value-field-tp4027695p4028057.html > >> > >> On Thu, 2 Jul 2020 at 18:52, Vincenzo D'Amore > wrote: > >> > >>> Hi all, > >>> > >>> simple question: Solr float/double multivalue fields preserve the order > >> of > >>> inserted values? > >>> > >>> Best regards, > >>> Vincenzo > >>> > >>> -- > >>> Vincenzo D'Amore > >>> > >> > > > > > > -- > > Vincenzo D'Amore > >
Re: Solr Float/Double multivalues fields
This is true _unless_ you fetch from docValues. docValues are SORTED_SETs, so the results will be both ordered and deduplicated if you return them as part of the field list. Don’t really think it needs to go into the ref guide, it’s just inherent in storing any kind of value. You wouldn’t expect multiple text entries in a multiValued field to be rearranged when returning the stored values either. Best, Erick > On Jul 2, 2020, at 2:21 PM, Vincenzo D'Amore wrote: > > Thanks, and genuinely asking: is there written somewhere in the > documentation too? If no, could anyone suggest to me which doc page should > I try to update? > > On Thu, Jul 2, 2020 at 8:08 PM Colvin Cowie > wrote: > >> The order of values within a multivalued field should match the insertion >> order. -- we certainly rely on that in our product. >> >> Order is guaranteed to be maintained for values in a multi-valued field. >>> >> >> https://lucene.472066.n3.nabble.com/order-question-on-solr-multi-value-field-tp4027695p4028057.html >> >> On Thu, 2 Jul 2020 at 18:52, Vincenzo D'Amore wrote: >> >>> Hi all, >>> >>> simple question: Solr float/double multivalue fields preserve the order >> of >>> inserted values? >>> >>> Best regards, >>> Vincenzo >>> >>> -- >>> Vincenzo D'Amore >>> >> > > > -- > Vincenzo D'Amore
Re: Solr Float/Double multivalues fields
Thanks, and genuinely asking: is there written somewhere in the documentation too? If no, could anyone suggest to me which doc page should I try to update? On Thu, Jul 2, 2020 at 8:08 PM Colvin Cowie wrote: > The order of values within a multivalued field should match the insertion > order. -- we certainly rely on that in our product. > > Order is guaranteed to be maintained for values in a multi-valued field. > > > > https://lucene.472066.n3.nabble.com/order-question-on-solr-multi-value-field-tp4027695p4028057.html > > On Thu, 2 Jul 2020 at 18:52, Vincenzo D'Amore wrote: > > > Hi all, > > > > simple question: Solr float/double multivalue fields preserve the order > of > > inserted values? > > > > Best regards, > > Vincenzo > > > > -- > > Vincenzo D'Amore > > > -- Vincenzo D'Amore
Re: Solr Float/Double multivalues fields
The order of values within a multivalued field should match the insertion order. -- we certainly rely on that in our product. Order is guaranteed to be maintained for values in a multi-valued field. > https://lucene.472066.n3.nabble.com/order-question-on-solr-multi-value-field-tp4027695p4028057.html On Thu, 2 Jul 2020 at 18:52, Vincenzo D'Amore wrote: > Hi all, > > simple question: Solr float/double multivalue fields preserve the order of > inserted values? > > Best regards, > Vincenzo > > -- > Vincenzo D'Amore >