[
https://issues.apache.org/jira/browse/ORC-1147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yiqun Zhang updated ORC-1147:
-----------------------------
Description:
{code:java}
if (!Double.isFinite(dstas.getSum())) {
LOG.debug("Not using predication pushdown on {} because stats contain
NaN values",
predicate.getColumnName());
return dstas.hasNull() ? TruthValue.YES_NO_NULL : TruthValue.YES_NO;
}
}
{code}
The isFinite function does not determine whether there is a NaN write or not,
and there is a possibility of a sum overflow; we should use `isNaN` instead to
determine this
was:
{code:java}
if (!Double.isFinite(dstas.getSum())) {
LOG.debug("Not using predication pushdown on {} because stats contain
NaN values",
predicate.getColumnName());
return dstas.hasNull() ? TruthValue.YES_NO_NULL : TruthValue.YES_NO;
}
}
{code}
The isFinite function does not determine whether there is a NaN write or not,
and there is a possibility of a sum overflow; we should use
`Objects.equals(dstas.getSum(), Double.NaN)` instead to determine this
> Use `isNaN` instead of `isFinite` to determine the contain NaN values
> ---------------------------------------------------------------------
>
> Key: ORC-1147
> URL: https://issues.apache.org/jira/browse/ORC-1147
> Project: ORC
> Issue Type: Improvement
> Components: Java
> Affects Versions: 1.7.3, 1.6.13
> Reporter: Yiqun Zhang
> Priority: Minor
>
> {code:java}
> if (!Double.isFinite(dstas.getSum())) {
> LOG.debug("Not using predication pushdown on {} because stats contain
> NaN values",
> predicate.getColumnName());
> return dstas.hasNull() ? TruthValue.YES_NO_NULL : TruthValue.YES_NO;
> }
> }
> {code}
> The isFinite function does not determine whether there is a NaN write or not,
> and there is a possibility of a sum overflow; we should use `isNaN` instead
> to determine this
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
