Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22029#discussion_r208835392
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1476,6 +1476,16 @@ object SQLConf {
"are performed before any UNION, EXCEPT and MINUS operations.")
.booleanConf
.createWithDefault(false)
+
+ val LEGACY_IN_FALSE_FOR_NULL_FIELD =
+ buildConf("spark.sql.legacy.inOperator.falseForNullField")
+ .internal()
+ .doc("When set to true, the IN operator returns false when comparing
literal structs " +
+ "containing a null field. When set to false (default), it returns
null, instead. This is " +
+ "important especially when using NOT IN as in the second case, it
filters out the rows " +
+ "when a null is present in a filed; while in the first one, those
rows are returned.")
+ .booleanConf
+ .createWithDefault(false)
--- End diff --
it can be done, WDYT @cloud-fan @gatorsmile ?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
