parthchandra commented on code in PR #3939:
URL: https://github.com/apache/datafusion-comet/pull/3939#discussion_r3102895720
##########
spark/src/main/scala/org/apache/comet/expressions/CometCast.scala:
##########
@@ -242,12 +242,18 @@ object CometCast extends CometExpressionSerde[Cast] with
CometExprShim {
"There can be differences in precision. " +
"For example, the input \"1.4E-45\" will produce 1.0E-45 " +
"instead of 1.4E-45"))
+ case d: DecimalType if d.scale < 0 =>
+ // Negative-scale decimals require
spark.sql.legacy.allowNegativeScaleOfDecimal=true.
+ // When that config is enabled, Spark formats them using Java
BigDecimal.toString()
+ // which produces scientific notation (e.g. "1.23E+4"). Comet matches
this behavior.
+ // When the config is disabled, negative-scale decimals cannot be
created in Spark,
+ // so we mark this as incompatible to avoid native execution on
unexpected inputs.
+ val allowNegativeScale = SQLConf.get
+ .getConfString("spark.sql.legacy.allowNegativeScaleOfDecimal",
"false")
+ .toBoolean
+ if (allowNegativeScale) Compatible() else Incompatible()
Review Comment:
yes. added a comment
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
