[ https://issues.apache.org/jira/browse/ARROW-10263?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Krisztian Szucs updated ARROW-10263: ------------------------------------ Fix Version/s: (was: 2.0.0) 3.0.0 > [C++][Compute] Improve numerical stability of variances merging > --------------------------------------------------------------- > > Key: ARROW-10263 > URL: https://issues.apache.org/jira/browse/ARROW-10263 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ > Reporter: Yibo Cai > Assignee: Yibo Cai > Priority: Major > Labels: pull-request-available > Fix For: 3.0.0 > > Time Spent: 1h 40m > Remaining Estimate: 0h > > For chunked array, variance kernel needs to merge variances. > Tested with two single value chunk, [40000008000000490], [40000008000000400]. > The merged variance is 3872. If treated as single array with two values, the > variance is 3904, same as numpy outputs. > So current merging method is not stable in extreme cases when chunks are very > short and with approximate mean values. -- This message was sent by Atlassian Jira (v8.3.4#803005)