ArnavBalyan opened a new pull request, #7879:
URL: https://github.com/apache/paimon/pull/7879
### Purpose
- Spark, Flink and Iceberg all support IsNaN as a first-class predicate and
use NaN counts in file/partition pruning.
- Paimon's SimpleColStats only tracks min, max and null count today, there
is no signal at the manifest layer to skip files.
- Add a nanCount field to SimpleColStats and update the collectors to count
the nans, and further be used for engine predicate pushdown.
### Tests
- UT
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
