GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/21407
[SPARK-24313][SQL][BACKPORT-2.3] Fix collection operations' interpreted
evaluation for complex types
## What changes were proposed in this pull request?
The interpreted evaluation of several collection operations works only for
simple datatypes. For complex data types, for instance, `array_contains` it
returns always `false`. The list of the affected functions is `array_contains`
and `GetMapValue`.
The PR fixes the behavior for all the datatypes.
## How was this patch tested?
added UT
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/spark SPARK-24313_2.3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21407.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21407
----
commit 9c554d4f0068fd7f3165ea307d1d838ba3ccd771
Author: Marco Gaido <marcogaido91@...>
Date: 2018-05-22T13:08:49Z
[SPARK-24313][SQL][BACKPORT-2.3] Fix collection operations' interpreted
evaluation for complex types
The interpreted evaluation of several collection operations works only for
simple datatypes. For complex data types, for instance, `array_contains` it
returns always `false`. The list of the affected functions is `array_contains`,
`array_position`, `element_at` and `GetMapValue`.
The PR fixes the behavior for all the datatypes.
added UT
Author: Marco Gaido <[email protected]>
Closes #21361 from mgaido91/SPARK-24313.
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
