GitHub user rednaxelafx opened a pull request:
https://github.com/apache/spark/pull/23079
[SPARK-26107][SQL] Extend ReplaceNullWithFalseInPredicate to support
higher-order functions: ArrayExists, ArrayFilter, MapFilter
## What changes were proposed in this pull request?
Extend the `ReplaceNullWithFalse` optimizer rule introduced in SPARK-25860
(https://github.com/apache/spark/pull/22857) to also support optimizing
predicates in higher-order functions of `ArrayExists`, `ArrayFilter`,
`MapFilter`.
Also rename the rule to `ReplaceNullWithFalseInPredicate` to better reflect
its intent.
## How was this patch tested?
Added new unit test cases to the `ReplaceNullWithFalseInPredicateSuite`
(renamed from `ReplaceNullWithFalseSuite`).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rednaxelafx/apache-spark catalyst-master
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/23079.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #23079
----
commit 710c8862b3138f6146fe2309d6379707f8d4ac14
Author: Kris Mok <kris.mok@...>
Date: 2018-11-18T09:09:53Z
Extend ReplaceNullWithFalseInPredicate to support higher-order functions:
ArrayExists, ArrayFilter, MapFilter
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
