[
https://issues.apache.org/jira/browse/SPARK-13903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Liang-Chi Hsieh updated SPARK-13903:
------------------------------------
Description:
With constraints and optimization, we can make sure some outputs of a Join (or
Filter) operator are not nulls. We should modify output nullability
accordingly. We can use this information in later execution to avoid null
checking.
Another reason to modify plan output is that we will use the output to
determine schema. We should keep correct nullability in the schema.
was:With constraints and optimization, we can make sure some outputs of a
Join operator are not nulls. We should modify output nullability accordingly.
We can use this information in later execution to avoid null checking.
> Modify output nullability with constraints for Join and Filter operators
> ------------------------------------------------------------------------
>
> Key: SPARK-13903
> URL: https://issues.apache.org/jira/browse/SPARK-13903
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Reporter: Liang-Chi Hsieh
>
> With constraints and optimization, we can make sure some outputs of a Join
> (or Filter) operator are not nulls. We should modify output nullability
> accordingly. We can use this information in later execution to avoid null
> checking.
> Another reason to modify plan output is that we will use the output to
> determine schema. We should keep correct nullability in the schema.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
