tisonkun commented on issue #702:
URL:
https://github.com/apache/datasketches-java/issues/702#issuecomment-3669064445
Even we filter all inf/-inf/nan, it's possible that f64::MAX + 1 produces
inf and later result in NaN.
I noticed the cause is from our calculation in the new `mean`:
```java
centroidMeans_[numCentroids_ - 1] += ((values[current] -
centroidMeans_[numCentroids_ - 1])
* weights[current]) / centroidWeights_[numCentroids_ - 1];
```
And in
https://github.com/apache/datasketches-rust/pull/23/changes/b35cdb270916e5dd70fa7d5ebd5e50b5b96d4c90,
I change the calculation from:
```
m1 + ((m2 - m1) * w2 / (w1 + w2))
```
to
```
m1 * (w1/(w1 + w2)) + m2 * (w2/(w1 + w2))
```
... with calling mul_add to avoid double precision as much as possible.
Hopefully this would eliminate all the NaN possible sources.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
