GitHub user scwf opened a pull request:
https://github.com/apache/spark/pull/5776
[SPARK-7232] [SQL] Add a Substitution batch for spark sql analyzer
Added a new batch named `Substitution` before Resolution batch. The
motivation for this is there are kind of cases we want to do some substitution
on the parsed logical plan before resolve it.
Consider this two cases:
1 CTE, for cte we first build a row logical plan
'With Map(q1 -> 'Subquery q1
'Project ['key]
'UnresolvedRelation [src], None)
'Project [*]
'Filter ('key = 5)
'UnresolvedRelation [q1], None
In `With` logicalplan here is a map stored the (q1-> subquery), we want
first take off the with command and substitute the q1 of UnresolvedRelation by
the subquery
2 Another example is Window function, in window function user may define
some windows, we also need substitute the window name of child by the concrete
window. this should also done in the Substitution batch.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/scwf/spark addbatch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5776.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5776
----
commit 1c9a0920c34fc4e64fa9ef91aec4cfeb02917b2f
Author: wangfei <[email protected]>
Date: 2015-04-29T09:17:57Z
added Substitution bastch
commit 06257caf8ee43c38ca40f9aeb2de8464601273be
Author: wangfei <[email protected]>
Date: 2015-04-29T09:34:56Z
fix compile
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
