GitHub user dilipbiswal opened a pull request:

    https://github.com/apache/spark/pull/16954

    [SPARK-18874][SQL] First phase: Deferring the correlated predicate pull up 
to Optimizer phase

    ## What changes were proposed in this pull request?
    Currently Analyzer as part of ResolveSubquery, pulls up the correlated 
predicates to its
    originating SubqueryExpression. The subquery plan is then transformed to 
remove the correlated
    predicates after they are moved up to the outer plan. In this PR, the task 
of pulling up
    correlated predicates is deferred to Optimizer. This is the initial work 
that will allow us to
    support the form of correlated subqueries that we don't support today. The 
design document
    from @nsyca can be found in the following link :
    
[DesignDoc](https://docs.google.com/document/d/1QDZ8JwU63RwGFS6KVF54Rjj9ZJyK33d49ZWbjFBaIgU/edit#)
    
    The brief description of code changes (hopefully to aid with code review) 
can be be found in the
    following link:
    
[CodeChanges](https://docs.google.com/document/d/18mqjhL9V1An-tNta7aVE13HkALRZ5GZ24AATA-Vqqf0/edit#)
    
    
    ## How was this patch tested?
    The test case PRs were submitted earlier using.
    [16337](https://github.com/apache/spark/pull/16337) 
[16759](https://github.com/apache/spark/pull/16759) 
[16841](https://github.com/apache/spark/pull/16841) 
[16915](https://github.com/apache/spark/pull/16915) 
[16798](https://github.com/apache/spark/pull/16798) 
[16712](https://github.com/apache/spark/pull/16712) 
[16710](https://github.com/apache/spark/pull/16710) 
[16760](https://github.com/apache/spark/pull/16760) 
[16802](https://github.com/apache/spark/pull/16802)


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dilipbiswal/spark SPARK-18874

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/spark/pull/16954.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #16954
    
----
commit 33844bb8c6e72c35e12a7ea9f124ad3c6b8bd43c
Author: Dilip Biswal <dbis...@us.ibm.com>
Date:   2017-02-13T23:25:39Z

    [SPARK-18874] First phase: Deferring the correlated predicate pull up to 
Optimizer phase

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

Reply via email to