[
https://issues.apache.org/jira/browse/STATISTICS-82?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17800889#comment-17800889
]
Alex Herbert commented on STATISTICS-82:
----------------------------------------
Statistics that use the count at the end to compute the statistic:
* IntMean/LongMean
* IntVariance/LongVariance
* IntStandardDeviation/LongStandardDeviation
* GeometricMean
These statistics could:
# Raise an error if the count is negative.
# Use the count as an unsigned 64-bit integer thus allowing support via
incrementing to be up to 2^64 observations. If a count is created by combine to
just under 2^63, this allows incrementing beyond 2^63 and the statistics is
still correct.
# Ignore this unlikely use case.
Statistics that use the count within the accept method for each additional
value:
* Mean
* Higher moments: Variance/Skewness/Kurtosis
These statistics are limited to < 2^63 observations. Options are:
# Guard against incrementing above this level for each new value. This would
affect performance.
# Ignore this unlikely use case.
Not handling overflow in counts is how this has been implemented in the JDK in
e.g. {{{}java.util.DoubleSummaryStatistics{}}}. A note in the class header
javadoc states:
{noformat}
"This implementation does not check for overflow of the count."{noformat}
This includes in the method to combine two instances of the summary statistics.
Given the unlikely case of creating over 2^63 actual observations a minimal
solution is to document the behaviour:
{noformat}
"Supports up to 2^63 (exclusive) observations. This implementation does not
check for overflow of the count."{noformat}
> Raise exception for integer overflow during statistic computation.
> ------------------------------------------------------------------
>
> Key: STATISTICS-82
> URL: https://issues.apache.org/jira/browse/STATISTICS-82
> Project: Commons Statistics
> Issue Type: Improvement
> Components: descriptive
> Reporter: Alex Herbert
> Priority: Minor
>
> Many statistics count the number of observations using a long. This limited
> computation to ~2^63 values.
> This is effectively impossible to achieve using single values. It is very
> easy to achieve using a combine:
> {code:java}
> final IntMean m = IntMean.of(2, 3);
> System.out.println(m.getAsDouble());
> for (int i = 0; i < 62; i++) {
> m.combine(m);
> }
> // n = 2^63 = -9223372036854775808
> System.out.println(m.getAsDouble());
> {code}
> Prints:
> {noformat}
> 2.5
> -0.5
> {noformat}
> Overflow can be detected in combine, e.g.
> {code:java}
> // throws ArithmeticException when n+m overflows
> n = Math.addExact(n, other.n)
> {code}
> However it is possible to create a statistic with a count very close to 2^63
> via combine and then increment it past 2^63 with single values. To avoid an
> expensive check at each addition, this case will have to detect overflow of n
> to negative during the final computation of the statistic.
> Raising an exception will guard against misuse. It is unlikely to affect any
> real use cases. Without an exception the user will have an invalid result
> which may cause downstream problems in the dependent code.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
