andygrove commented on PR #4799: URL: https://github.com/apache/datafusion-comet/pull/4799#issuecomment-5063487999
Comparing the curated lists here against the inventory in #4180, these `spark.sql.legacy.*` configs from that issue are not currently handled by this PR (no session-wide fallback, no per-expression dispatch, no support-level check): - `spark.sql.legacy.sizeOfNull` (`size(null)` returns `-1` vs `null`) - `spark.sql.legacy.allowCastNumericToTimestamp` (`CAST(bigint AS timestamp)` permitted or not) - `spark.sql.legacy.followThreeValuedLogicInArrayExists` (`exists(array, pred)` null handling) - `spark.sql.legacy.integerGroupingIdEnabled` (`grouping_id()` return type) - `spark.sql.legacy.json.allowEmptyString.enabled` (JSON parsing behavior) - `spark.sql.legacy.createEmptyCollectionUsingStringType` (empty `array()` / `map()` element type) Should any of these be added to `legacyConfDefaults` or gated per-expression in this PR, or is the intent to leave them for follow-up work under #4180? Some may already be safe to ignore. A few are analyzer-side and get resolved before Comet ever sees the plan (`integerGroupingIdEnabled`, `createEmptyCollectionUsingStringType`, `allowCastNumericToTimestamp`), so if that reasoning holds it would be worth capturing in the compatibility doc alongside the existing note for `allowNegativeScaleOfDecimal`, so a reader can tell "considered and safe" from "not yet looked at". The ones that seem most likely to need real handling are `sizeOfNull` and `followThreeValuedLogicInArrayExists`, since both change runtime results for expressions Comet executes natively. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
