peterxcli opened a new pull request, #5752: URL: https://github.com/apache/datafusion-comet/pull/5752
## Which issue does this PR close? Closes #5749. ## Rationale for this change `CAST(string AS TIMESTAMP_NTZ)` can silently return NULL under ANSI mode when the input matches a supported timestamp pattern but the timestamp decoder rejects it. For example, casting a Parquet-backed string column containing `294249-01-01` raises `CAST_INVALID_INPUT` in Spark 4.1.3, while Comet previously returned NULL. The equivalent Comet cast to `TIMESTAMP` already raises an error. The discrepancy comes from `timestamp_ntz_parser_inner`: a `None` returned by `parse_to_timestamp_info` was mapped directly to `Ok(None)`, bypassing the evaluation-mode check. Six-digit years outside the decoder's allowed range reach this branch because they pass the timestamp pattern check. This is an existing bug, discovered while reviewing #5682, rather than a regression introduced by that PR. ## What changes are included in this PR? The NTZ parser now returns immediately only when decoding and conversion to microseconds both succeed. If either rejects a recognized input, parsing stops and falls through to the existing evaluation-mode handling, which produces `InvalidInputInCastToDatetime` for ANSI mode and NULL for legacy and try modes. Unrecognized input continues through the same handling. This removes the duplicated ANSI error construction and ensures decoder rejection and conversion rejection follow the same policy without changing the accepted timestamp formats or year limits. Regression coverage includes `294249-01-01`, `294249-01-01 00:00:00`, and `-290310-01-01`. Native tests check the ANSI error's value and source/target types, plus NULL results in legacy and try modes. Spark comparison tests exercise both `TIMESTAMP` and `TIMESTAMP_NTZ` using Parquet-backed columns to prevent constant folding. Each malformed value is checked separately so an exception from one value cannot hide another case. ## How are these changes tested? - Focused NTZ Rust unit tests: 5 passed. - Spark 4.1.3 out-of-range-year regressions: 2 passed, covering ANSI, legacy, and try casts. - Native build, Rust formatting, Spotless, and `git diff --check` passed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
