yadavay-amzn commented on code in PR #56941:
URL: https://github.com/apache/spark/pull/56941#discussion_r3580638824
##########
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/statsEstimation/FilterEstimationSuite.scala:
##########
@@ -1020,4 +1032,87 @@ class FilterEstimationSuite extends
StatsEstimationTestBase {
}
}
+ // Tests for nanosecond-precision timestamp types (SPARK-57839)
Review Comment:
Added the two gaps you flagged (the high-magnitude 2024-era coverage is
already present via the "NTZ/LTZ nanos filter estimation at high magnitude
(2024 timestamps)" cases at epochMicros ~1.7e15, which exercise the lossy
toDouble path):
- FilterEstimationSuite: a within-one-ULP guard - min/max are chosen so
toDouble(min) == toDouble(max) at high magnitude (asserted as the precondition,
with min != max), then a partial-overlap filter estimates without an
AssertionError and returns a sane estimate, locking the invariant against
future edits.
- A pre-1970/negative case for fromDouble's floorDiv/floorMod: a
FilterEstimationSuite estimation over negative epoch-nanos bounds, plus a
negative (pre-1970) value added to the nanos catalog round-trip in
StatisticsCollectionSuite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
