GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/10296
[SPARK-12323][SQL] Makes BoundReference respect nullability
This PR partially fixes SPARK-12323 by making `BoundReference` respect
nullability. Now if a "top-level" column is defined as non-nullable, nulls
appear in input data will cause a runtime exception with nice error message.
What this PR doesn't fix is cases of non-nullable nested fields, which are
covered in the 3 newly added ignored test cases. These test cases fail because
`NewInstance`, `MakeObjects`, and potentially some other expressions along the
way are always nullable. To fix these test cases, these expressions must also
respect nullability.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liancheng/spark
spark-12323.non-nullable-ds-fields
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10296.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10296
----
commit de8b44292300e86f5f152dd7138555b94160d3c6
Author: Cheng Lian <[email protected]>
Date: 2015-12-14T17:43:08Z
Makes BoundReference respect nullability
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
