GitHub user mgaido91 opened a pull request:
https://github.com/apache/spark/pull/20976
[SPARK-23835][SQL] Add not-null check to Tuples' arguments deserialization
## What changes were proposed in this pull request?
There was no check on nullability for arguments of `Tuple`s. This could
lead to have weird behavior when a null value had to be deserialized into a
non-nullable Scala object: in those cases, the `null` got silently transformed
in a valid value (like `-1` for `Int`), corresponding to the default value we
are using in the SQL codebase. This situation was very likely to happen when
deserializing to a Tuple of primitive Scala types (like Double, Int, ...).
The PR adds the `AssertNotNull` to arguments of tuples which have been
asked to be converted to non-nullable types.
## How was this patch tested?
added UT
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/mgaido91/spark SPARK-23835
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20976.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20976
----
commit b3e70e95069a704abce88313559e4de686768e78
Author: Marco Gaido <marcogaido91@...>
Date: 2018-04-04T14:58:15Z
[SPARK-23835][SQL] Add not-null check to Tuples' arguments deserialization
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
