andygrove commented on code in PR #5039:
URL: https://github.com/apache/datafusion-comet/pull/5039#discussion_r3723948919
##########
spark/src/main/scala/org/apache/comet/serde/datetime.scala:
##########
@@ -963,6 +963,39 @@ object CometMakeYMInterval extends
CometCodegenDispatch[MakeYMInterval]
object CometMakeDTInterval extends CometCodegenDispatch[MakeDTInterval]
+object CometMakeInterval extends CometExpressionSerde[MakeInterval] with
CodegenDispatchFallback {
+ private val incompatReason =
+ "The native implementation converts seconds to `Float64`, which can lose
microsecond" +
+ " precision, and stores time in nanoseconds, which overflows for large
time components" +
+ " (hours, minutes, seconds) that Spark can represent."
+
+ override def getCompatibleNotes(): Seq[String] = Seq(
+ "Both the default JVM codegen-dispatch path and the native path encode
elapsed time as" +
+ " Arrow nanoseconds. Spark intervals whose microseconds cannot be
multiplied by 1,000" +
+ " in an `i64` are not supported" +
+ " ([#5279](https://github.com/apache/datafusion-comet/issues/5279)).")
Review Comment:
nit: I wonder if this could be a more user-friendly message stating that
this only affects extreme intervals spanning hundreds of years?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
