RE: Migration Lucene 4.7.0 -->6.0.1 - NumericUtils

2016-12-05 Thread Ludovic Bertin
Hi Mike,

Sorry for the delay.
For some use cases, Fields are implemented by numeric values, but distinct 
values are not so big (<50 terms).
We have a Search GUI, where user can add criteria on predefined fields. For 
those fields, where the number of terms is limited, then we proposed the 
available values in a dropdown.

What are your recommendations ? Using String instead of Numeric Values ?

Thanks.
Ludovic



-Original Message-
From: Michael McCandless [mailto:luc...@mikemccandless.com]
Sent: jeudi, 22. septembre 2016 22:34
To: Lucene Users; Ludovic Bertin
Subject: Re: Migration Lucene 4.7.0 -->6.0.1 - NumericUtils

LegacyNumericUtils is the right solution for your index for now, but
longer term you should migrate to dimensional points instead, which
are a more efficient way to index and range search numerics.

But: why do you need all distinct values of a field?  In general this
is a very dangerous method to offer.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Sep 20, 2016 at 9:02 AM, Ludovic Bertin
 wrote:
> Hi there,
>
> I'm migrating an application from Lucene 4.7.0 to Lucene 6.0.1.
> I'm facing a problem with this piece of code :
>
> public List getDistinctValues(IndexReader reader, EventField field) 
> throws IOException {
>
> List values = new ArrayList();
> Fields fields = MultiFields.getFields(reader);
> if (fields == null) return values;
>
> Terms terms = fields.terms(field.name());
> if (terms == null) return values;
>
> TermsEnum iterator = terms.iterator();
>
> Class type = field.getJavaType();
> BytesRef value = iterator.next();
>
> while (value != null) {
> if (type == Long.class) {
> values.add(LegacyNumericUtils.prefixCodedToLong(value));
> } else if (type == Integer.class) {
> values.add(LegacyNumericUtils.prefixCodedToInt(value));
> } else if (type == Boolean.class) {
> values.add(LegacyNumericUtils.prefixCodedToInt(value) == 1 ? TRUE 
> : FALSE);
> } else if (type == Date.class) {
> values.add(new Date(LegacyNumericUtils.prefixCodedToLong(value)));
> } else if (type == String.class) {
> values.add(value.utf8ToString());
> } else {
> log.warn("getDistinctValues: ignoring field " + field + " of type 
> " + type);
> }
>
> value = iterator.next();
> }
>
> return values;
> }
>
> The aim of this method is to get all terms present in a field directly in the 
> correct java type. I'm seeing just now that NumericUtils is an internal 
> class, so I should not have used it.
> For now, I can use LegacyNumericUtils, but that's a very temporary solution.
>
> What is the best approach to achieve my goal ?
>
> Thanks in advance for any help.
> Ludovic
>
>  DISCLAIMER 
> This message is intended only for use by the person to
> whom it is addressed. It may contain information that is
> privileged and confidential. Its content does not constitute
> a formal commitment by Bank Lombard Odier & Co Ltd or any
> of its branches or affiliates. If you are not the intended recipient
> of this message, kindly notify the sender immediately and
> destroy this message. Thank You.
> *
[[ rethink everything. ]]<http://www.lombardodier.com>

DISCLAIMER **
This message is intended only for use by the person to
whom it is addressed. It may contain information that is
privileged and confidential. Its content does not constitute
a formal commitment by Bank Lombard Odier & Co Ltd
or any of its branches or affiliates. If you are not the
intended recipient of this message, kindly notify the sender
immediately and destroy this message. Thank You.
***



Re: Migration Lucene 4.7.0 -->6.0.1 - NumericUtils

2016-09-22 Thread Michael McCandless
LegacyNumericUtils is the right solution for your index for now, but
longer term you should migrate to dimensional points instead, which
are a more efficient way to index and range search numerics.

But: why do you need all distinct values of a field?  In general this
is a very dangerous method to offer.

Mike McCandless

http://blog.mikemccandless.com


On Tue, Sep 20, 2016 at 9:02 AM, Ludovic Bertin
 wrote:
> Hi there,
>
> I'm migrating an application from Lucene 4.7.0 to Lucene 6.0.1.
> I'm facing a problem with this piece of code :
>
> public List getDistinctValues(IndexReader reader, EventField field) 
> throws IOException {
>
> List values = new ArrayList();
> Fields fields = MultiFields.getFields(reader);
> if (fields == null) return values;
>
> Terms terms = fields.terms(field.name());
> if (terms == null) return values;
>
> TermsEnum iterator = terms.iterator();
>
> Class type = field.getJavaType();
> BytesRef value = iterator.next();
>
> while (value != null) {
> if (type == Long.class) {
> values.add(LegacyNumericUtils.prefixCodedToLong(value));
> } else if (type == Integer.class) {
> values.add(LegacyNumericUtils.prefixCodedToInt(value));
> } else if (type == Boolean.class) {
> values.add(LegacyNumericUtils.prefixCodedToInt(value) == 1 ? TRUE 
> : FALSE);
> } else if (type == Date.class) {
> values.add(new Date(LegacyNumericUtils.prefixCodedToLong(value)));
> } else if (type == String.class) {
> values.add(value.utf8ToString());
> } else {
> log.warn("getDistinctValues: ignoring field " + field + " of type 
> " + type);
> }
>
> value = iterator.next();
> }
>
> return values;
> }
>
> The aim of this method is to get all terms present in a field directly in the 
> correct java type. I'm seeing just now that NumericUtils is an internal 
> class, so I should not have used it.
> For now, I can use LegacyNumericUtils, but that's a very temporary solution.
>
> What is the best approach to achieve my goal ?
>
> Thanks in advance for any help.
> Ludovic
>
>  DISCLAIMER 
> This message is intended only for use by the person to
> whom it is addressed. It may contain information that is
> privileged and confidential. Its content does not constitute
> a formal commitment by Bank Lombard Odier & Co Ltd or any
> of its branches or affiliates. If you are not the intended recipient
> of this message, kindly notify the sender immediately and
> destroy this message. Thank You.
> *

-
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org