voonhous commented on PR #20524: URL: https://github.com/apache/hudi/pull/20524#issuecomment-6647119091
Pushed 0935fd57ec3b: - `Decreasing` ordering: enforced in `TermRange.derive` and pinned by a test (inline). - Golden pairs: `TestDataSkippingGoldenFilters` checks 1,136 `filter -> index filter` pairs in `data-skipping/golden-index-filters.txt`. They were generated from the PR base's translator (8e5608ec), and the new code matches them on Spark 4.0 (harness) and Spark 3.5 (the test). To keep it stable across Spark versions, inputs carry an explicit time zone (otherwise `cast(s as date)` resolves on Spark 4 but not on 3.5), and the file uses its own rendering because `Expression.sql` differs across versions and formats timestamps in the session time zone. On a mismatch, the test writes the actual lines under `target/` so an intended change can be copied over. - Stage unit tests: `TestDataSkippingStages` covers `Leaf` orientation, one `Verdict` per `Op`, `Step`, and `derive`. - `CastRule`: guarded on `dataType` rather than `resolved` (inline). - Renamed `Range` to `TermRange` and `Compiler` to `StatsFilterCompiler`. - CI on 3e483501: GH Spark 4.x suites green (https://github.com/apache/hudi/actions/runs/41872766010). Azure, which runs the Spark 3.5 datasource and expression index functional suites, had not finished. Both rerun on 0935fd57ec3b. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
