Github user mgaido91 commented on a diff in the pull request:
https://github.com/apache/spark/pull/22029#discussion_r229706800
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ---
@@ -1561,6 +1561,16 @@ object SQLConf {
.booleanConf
.createWithDefault(false)
+ val LEGACY_IN_FALSE_FOR_NULL_FIELD =
+ buildConf("spark.sql.legacy.inOperator.falseForNullField")
+ .internal()
+ .doc("When set to true (default), the IN operator returns false when
comparing multiple " +
+ "values containing a null. When set to false, it returns null,
instead. This is " +
+ "important especially when using NOT IN as in the second case, it
filters out the rows " +
+ "when a null is present in a field; while in the first one, those
rows are returned.")
+ .booleanConf
+ .createWithDefault(true)
--- End diff --
yes, I agree, let me switch it, thanks.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
