peterxcli commented on code in PR #5014:
URL: https://github.com/apache/datafusion-comet/pull/5014#discussion_r3642907444
##########
native/spark-expr/src/conversion_funcs/string.rs:
##########
@@ -1930,6 +1929,31 @@ fn date_parser(date_str: &str, eval_mode: EvalMode) ->
SparkResult<Option<i32>>
Ok(None)
}
}
+
+ /// Turns parsed year/month/day segments into an epoch day. Shared by both
parsing paths so
+ /// that the decision of what a segment triple *means* lives in exactly
one place.
+ fn finish(
+ year: i64,
+ month: i64,
+ day: i64,
+ date_str: &str,
+ eval_mode: EvalMode,
+ ) -> SparkResult<Option<i32>> {
+ // Spark builds a `LocalDate` and narrows its epoch day to an `Int`,
so an invalid
+ // calendar date or an epoch day that overflows `i32` both yield
`None` there, which
+ // `stringToDateAnsi` turns into CAST_INVALID_INPUT.
+ let Some(days) = ymd_to_epoch_day(year, month, day).and_then(|d|
i32::try_from(d).ok())
+ else {
+ return return_result(date_str, eval_mode);
+ };
+ // Spark accepts years beyond what chrono can represent, and
downstream date kernels
+ // cannot handle those values, so Comet keeps returning null for them
in every eval mode
+ // rather than raising. This is a Comet limitation, not a malformed
input.
Review Comment:
It would be good to add the coresponding chrono issue or its code snippet
here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
