GitHub user cloud-fan reopened a pull request:
https://github.com/apache/spark/pull/11673
[SPARK-13853][SQL] QueryPlan sub-classes should override producedAttributes
to fix missingInput
## What changes were proposed in this pull request?
Some operators will produce new attributes as output(e.g. most leaf nodes
and Generate), and we should exclude them when calculating missing input.
Nowadays subclasses can override `producedAttributes` to fix this issue, which
is the recommended way. However, they can also override `reference` or
`missingInput`, we should stop doing this and mark these 2 methods as `final`
to force subclasses to override `producedAttributes` only.
Except `producedAttributes`, sub-classes can also produce new attributes by
`Alias`. This approach is better but can't fit in all situations, e.g.
`Generate`, `Expand`, `ScriptTransform`, etc. These operators have nothing to
alias and behave like a leaf node. But `EvaluatePython` operator can produce
new attribute by alias, so this PR also changes `EveluatePython` to use the
alias approach.
## How was this patch tested?
existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark minor
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/11673.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #11673
----
commit ce0b9fee5006e0dc89869e25056ccaa72db972e5
Author: Wenchen Fan <[email protected]>
Date: 2016-08-01T06:07:32Z
remove MaxOf and MinOf
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
