coderfender commented on code in PR #3364:
URL: https://github.com/apache/datafusion-comet/pull/3364#discussion_r2757518300
##########
spark/src/main/scala/org/apache/comet/serde/arrays.scala:
##########
@@ -198,7 +198,11 @@ object CometArrayMin extends
CometExpressionSerde[ArrayMin] {
object CometArraysOverlap extends CometExpressionSerde[ArraysOverlap] {
- override def getSupportLevel(expr: ArraysOverlap): SupportLevel =
Incompatible(None)
+ override def getSupportLevel(expr: ArraysOverlap): SupportLevel =
Incompatible(Some(
+ "Null handling differs from Spark: DataFusion's array_has_any returns
false when no " +
+ "common elements are found, even if null elements exist. Spark returns
null in such " +
+ "cases following three-valued logic (SQL standard). Example: " +
+ "arrays_overlap(array(1, null, 3), array(4, 5)) returns null in Spark
but false in Comet."))
Review Comment:
Thank you for writing this. I would probably keep the message short and
route to the documentation to help declutter the logs :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
