andygrove opened a new pull request, #5221: URL: https://github.com/apache/datafusion-comet/pull/5221
## Which issue does this PR close? Related to #5010. ## Rationale for this change `spark.comet.exceptionOnDatetimeRebase` is declared in `CometConf` but is never read anywhere in the codebase — JVM or native. Setting it to `true` had no effect: Comet did not raise on dates/timestamps written with the legacy hybrid (Julian + Gregorian) calendar, it silently read them as Proleptic Gregorian. The config therefore advertised a safety guard that does not exist, which is worse than not offering one at all. The real gap — no datetime rebasing in the native Parquet scan — is tracked by #5010 and remains documented in the scan compatibility guide. ## What changes are included in this PR? - Remove `COMET_EXCEPTION_ON_LEGACY_DATE_TIMESTAMP` / `spark.comet.exceptionOnDatetimeRebase` from `CometConf.scala`. - Drop the sentence in `docs/source/user-guide/latest/compatibility/scans.md` that called out the config as dead code. The surrounding paragraph already describes the missing rebasing support and links to #5010. No upgrade guide entry is added: removing the key is not a behavior change, since no code path ever consulted it, and Spark ignores unrecognized `spark.comet.*` keys that users may still have set. ## How are these changes tested? Nothing referenced the config, so there is no behavior to test. Verified with a repo-wide grep for both the key string and the Scala symbol (only the two edited sites matched) and confirmed `common` + `spark` compile cleanly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
