diqiu50 commented on PR #14643: URL: https://github.com/apache/gravitino/pull/14643#issuecomment-6658385986
On Spark 3.5, the NTZ change breaks pushed-down timestamp filters. Spark 3.5.3 `compileValue` doesn't handle `LocalDateTime`, so a filter on a zone-free timestamp compiles to `"ts" > 2024-01-02T03:04:05`, which is invalid SQL. Spark 4.0.3 quotes it correctly. Tables without string-like columns keep pushdown, so they are affected. The current tests don't filter on `ts` for such a table. Two smaller points: - `!jdbc-mysql` also moves Doris, StarRocks, and OceanBase to NTZ, though they can use MySQL URLs and weren't tested. - Disabling pushdown per table also stops numeric filters such as `id = 1`. Could this be limited to the affected columns or operations? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
