AliaksandrAleksiukCollibra opened a new pull request, #55727: URL: https://github.com/apache/spark/pull/55727
### What changes were proposed in this pull request? `toBoundValueInWhereClause` in `JDBCRelation` now accepts the resolved `JdbcDialect` and calls `dialect.compileValue` for date and timestamp partition bounds instead of hardcoding bare quoted string literals. ### Why are the changes needed? When partitioning a JDBC table by a date or timestamp column, Spark generates WHERE clauses like `col < '2024-01-01'`. Strict-typing engines such as Athena and Phoenix reject this with a type mismatch error (Cannot apply operator: date < varchar) because the bare quoted string is treated as `VARCHAR`, not `DATE/TIMESTAMP`. `JdbcDialect.compileValue` already exists for dialect-specific value formatting and is used in filter pushdown, but was never wired into partition bound generation. Dialects for strict-typing engines can now override compileValue to emit typed literals such as `DATE '2024-01-01'` or `TIMESTAMP '2024-01-01 00:00:00'`. ### Does this PR introduce _any_ user-facing change? Yes. Users with a custom JdbcDialect that overrides compileValue will now see that override applied to partition WHERE clauses as well as filter pushdown. The base `JdbcDialect.compileValue` returns the same bare-quoted string as before, so all built-in dialects and users without a custom dialect are unaffected. ### How was this patch tested? Added `SPARK-28587: columnPartition should use dialect.compileValue for date/timestamp bounds` in `JDBCSuite`. The test registers a custom dialect that emits `DATE '...'` / `TIMESTAMP '...'` typed literals, calls `JDBCRelation.columnPartition` directly, and asserts the generated WHERE clauses contain typed literals rather than bare quoted strings. Existing tests SPARK-34843 and SPARK-22814 continue to pass, confirming backward compatibility. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Sonnet 4.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
