GitHub user skambha opened a pull request:
https://github.com/apache/spark/pull/17185
[SPARK-19602][SQL] Support column resolution of fully qualified column name
( 3 part name)
## What changes were proposed in this pull request?
The design details is attached to the JIRA issue
[here](https://issues.apache.org/jira/secure/attachment/12854681/Design_ColResolution_JIRA19602.pdf)
High level overview of the changes are:
- Enhance the qualifier to be more than one string
- Add support to store the qualifier. Enhance the lookupRelation to keep
the qualifier appropriately.
- Enhance the table matching column resolution algorithm to account for
qualifier being more than a string.
- Enhance the table matching algorithm in UnresolvedStar.expand
- Ensure that we continue to support select t1.i1 from db1.t1
## How was this patch tested?
- New tests are added.
- Several test scenarios were added in a separate [test pr
17067](https://github.com/apache/spark/pull/17067). The tests that were not
supported earlier are marked with TODO markers and those are now supported with
the code changes here.
- Existing unit tests ( hive, catalyst and sql) were run successfully.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/skambha/spark colResolution
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/17185.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #17185
----
commit 9b27c0bb832444775f2e6f5488d66b8b4415350e
Author: Sunitha Kambhampati <[email protected]>
Date: 2016-11-01T23:33:45Z
Changes to resolve fully qualified column attribute references from a
logical plan node children
commit 66c004d46e6fa837b940c88384ff0bc11e34482c
Author: Sunitha Kambhampati <[email protected]>
Date: 2017-03-03T23:07:16Z
Add AliasIdentifier, cleanups, merge and rebases
commit cd2e18bac541dc26b77af219e5120b29fbbcb87a
Author: Sunitha Kambhampati <[email protected]>
Date: 2017-03-04T00:21:56Z
Tests now support qualified column name (3 part) and rebase changes
commit e23407fbb2e472d0dfc470afdb12c09cd4ca4d9f
Author: Sunitha Kambhampati <[email protected]>
Date: 2017-03-05T01:14:10Z
merge changes
commit 0b3e6f8eb8f019404d292984a16a63a6c75ffe88
Author: Sunitha Kambhampati <[email protected]>
Date: 2017-03-06T23:14:10Z
Cleanup
commit 8d6bd3f74a509614f1de81803ef207b1440eaa17
Author: Sunitha Kambhampati <[email protected]>
Date: 2017-03-06T23:52:10Z
test cleanup
commit fb72118ce1b780deb0613578cf69320e16f12f5e
Author: Sunitha Kambhampati <[email protected]>
Date: 2017-03-07T00:12:35Z
formatting
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
