Re: JSON facet SUM precision and accuracy is incorrect

[pchankh]

Dear Yonik,

Thanks for the response.  As many companies using solr are using function to
compute results, the loss of precision is a critical problem to resolve and
is important to us.  How can we escalate this given that there is already a
JIRA filed in 2016. Appreciate if the Solr team can help to make the tools
even more reliable given the great stride of new features available in 7.0.  

Looking forward.

Many thanks.
Dr. Patrick 



--
View this message in context: 
http://lucene.472066.n3.nabble.com/JSON-facet-SUM-precision-and-accuracy-is-incorrect-tp4349571p4351902.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: JSON facet SUM precision and accuracy is incorrect

[Zheng Lin Edwin Yeo]

Hi Yonik,

Thanks for your reply.

Will this be solved in the Solr 7.0 version?

For now, is there other method which we can do to overcome this issue?

Regards,
Edwin


On 8 August 2017 at 19:56, Yonik Seeley  wrote:

> This is due to function queries currently lacking type information
> (this problem will occur anywhere function queries are used and is not
> unique to JSON Facet).
> Function queries were originally only used in lucene scoring (which
> only uses float).
> The inner sum(amount1_d,amount2_d) uses SumFloatFunction, hence the
> loss of precision.
>
> One should see the same loss of precision using pseudo-fields with
> function queries for example:
> q=my_document=id, amount1_d, amount2_d ,sum(amount1_d,amount2_d)
>
> The JIRA for this issue: https://issues.apache.org/jira/browse/SOLR-6575
>
> -Yonik
>
>
> On Tue, Aug 8, 2017 at 4:48 AM, Patrick Chan  wrote:
> > Appreciate if anyone can help raise an issue for the JSON facet sum error
> > my staff Edwin raised earlier
> >
> > but have not gotten any response from the Solr community and developers.
> >
> > Our production operation is urgently needing this accuracy to proceed as
> it
> > impacts audit issues.
> >
> >
> > Best regards,
> >
> > Dr.Patrick
> >
> >
> > On Tue, Jul 25, 2017 at 6:27 PM, Zheng Lin Edwin Yeo <
> edwinye...@gmail.com>
> >
> > wrote:
> >
> >> This is the way which I put my JSON facet.
> >>
> >> totalAmount:"sum(sum(amount1_d,amount2_d))"
> >>
> >> amount1_d: 69446961 <6944%206961>.2
> >> amount2_d: 0
> >>
> >> Result I get: 69446959 <6944%206959>.27
> >>
> >>
> >> Regards,
> >> Edwin
> >>
> >>
> >> On 25 July 2017 at 20:44, Zheng Lin Edwin Yeo 
> >> wrote:
> >>
> >> > Hi,
> >> >
> >> > I'm trying to do a sum of two double fields in JSON Facet. One of the
> >> > field has a value of 69446961 <6944%206961>.2, while the other is 0.
> > However, when I
> >> get
> >> > the result, I'm getting a value of 69446959 <6944%206959>.27. This is
> > 1.93 lesser than
> >> > the original value.
> >> >
> >> > What could be the reason?
> >> >
> >> > I'm using Solr 6.5.1.
> >> >
> >> > Regards,
> >> > Edwin
> >> >
>

Re: JSON facet SUM precision and accuracy is incorrect

[Yonik Seeley]

This is due to function queries currently lacking type information
(this problem will occur anywhere function queries are used and is not
unique to JSON Facet).
Function queries were originally only used in lucene scoring (which
only uses float).
The inner sum(amount1_d,amount2_d) uses SumFloatFunction, hence the
loss of precision.

One should see the same loss of precision using pseudo-fields with
function queries for example:
q=my_document=id, amount1_d, amount2_d ,sum(amount1_d,amount2_d)

The JIRA for this issue: https://issues.apache.org/jira/browse/SOLR-6575

-Yonik


On Tue, Aug 8, 2017 at 4:48 AM, Patrick Chan  wrote:
> Appreciate if anyone can help raise an issue for the JSON facet sum error
> my staff Edwin raised earlier
>
> but have not gotten any response from the Solr community and developers.
>
> Our production operation is urgently needing this accuracy to proceed as it
> impacts audit issues.
>
>
> Best regards,
>
> Dr.Patrick
>
>
> On Tue, Jul 25, 2017 at 6:27 PM, Zheng Lin Edwin Yeo 
>
> wrote:
>
>> This is the way which I put my JSON facet.
>>
>> totalAmount:"sum(sum(amount1_d,amount2_d))"
>>
>> amount1_d: 69446961 <6944%206961>.2
>> amount2_d: 0
>>
>> Result I get: 69446959 <6944%206959>.27
>>
>>
>> Regards,
>> Edwin
>>
>>
>> On 25 July 2017 at 20:44, Zheng Lin Edwin Yeo 
>> wrote:
>>
>> > Hi,
>> >
>> > I'm trying to do a sum of two double fields in JSON Facet. One of the
>> > field has a value of 69446961 <6944%206961>.2, while the other is 0.
> However, when I
>> get
>> > the result, I'm getting a value of 69446959 <6944%206959>.27. This is
> 1.93 lesser than
>> > the original value.
>> >
>> > What could be the reason?
>> >
>> > I'm using Solr 6.5.1.
>> >
>> > Regards,
>> > Edwin
>> >

JSON facet SUM precision and accuracy is incorrect

[Patrick Chan]

Appreciate if anyone can help raise an issue for the JSON facet sum error
my staff Edwin raised earlier

but have not gotten any response from the Solr community and developers.

Our production operation is urgently needing this accuracy to proceed as it
impacts audit issues.


Best regards,

Dr.Patrick


On Tue, Jul 25, 2017 at 6:27 PM, Zheng Lin Edwin Yeo 

wrote:

> This is the way which I put my JSON facet.
>
> totalAmount:"sum(sum(amount1_d,amount2_d))"
>
> amount1_d: 69446961 <6944%206961>.2
> amount2_d: 0
>
> Result I get: 69446959 <6944%206959>.27
>
>
> Regards,
> Edwin
>
>
> On 25 July 2017 at 20:44, Zheng Lin Edwin Yeo 
> wrote:
>
> > Hi,
> >
> > I'm trying to do a sum of two double fields in JSON Facet. One of the
> > field has a value of 69446961 <6944%206961>.2, while the other is 0.
However, when I
> get
> > the result, I'm getting a value of 69446959 <6944%206959>.27. This is
1.93 lesser than
> > the original value.
> >
> > What could be the reason?
> >
> > I'm using Solr 6.5.1.
> >
> > Regards,
> > Edwin
> >