snmvaughan commented on code in PR #7522:
URL: https://github.com/apache/datafusion-comet/pull/7522#discussion_r4485863868
##########
spark/src/main/scala/org/apache/comet/expressions/CometCast.scala:
##########
@@ -200,6 +200,7 @@ object CometCast
castBuilder.setIsSpark4Plus(isSpark40Plus)
castBuilder.setDecimalZeroAnyExponent(
SparkFixes.castsZeroWithAnyExponentToDecimal(SPARK_VERSION))
+
castBuilder.setDecimalTinyToZero(SparkFixes.castsTinyStringToDecimalZero(SPARK_VERSION))
Review Comment:
Thanks. I considered falling back. A negative-scale `Unsupported` runs
Spark's cast through the JVM codegen dispatcher per batch, which is slower than
the native parse, and the whole projection falls back to Spark when the
dispatcher can't take it (codegen flag off, `maxFields`). Keeping the cast
native keeps it vectorized and follows the `SparkFixes` pattern from #7171. The
cost is the version table, and the ANSI `'1e2147483647'` case on 4.1.4 stays
open; I'll handle it separately if you prefer. I'm happy to switch to the
fallback if you'd rather have exact parity on every release and vendor build.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
