GitHub user dilipbiswal opened a pull request:
https://github.com/apache/spark/pull/22542
[SPARK-25519][SQL] ArrayRemove function may return incorrect result when
right expression is implicitly downcasted.
## What changes were proposed in this pull request?
In ArrayPosition, we currently cast the right hand side expression to match
the element type of the left hand side Array. This may result in down casting
and may return wrong result or questionable result.
Example :
```SQL
spark-sql> select array_remove(array(1,2,3), 1.23D);
[2,3]
```
```SQL
spark-sql> select array_remove(array(1,2,3), 'foo');
NULL
```
We should safely coerce both left and right hand side expressions.
## How was this patch tested?
Added tests in DataFrameFunctionsSuite
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/dilipbiswal/spark SPARK-25519
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/22542.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #22542
----
commit ac9cb1a2d2e5b73f6c6e89c510f84a1f10efb60e
Author: Dilip Biswal <dbiswal@...>
Date: 2018-09-16T02:36:18Z
[SPARK-25519] ArrayRemove function may return incorrect result when right
expression is implicitly downcasted.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
