GitHub user marmbrus opened a pull request:
https://github.com/apache/spark/pull/202
SPARK-1294 Fix resolution of uppercase field names using a HiveContext.
Fixing this bug required the following:
- Creation of a new logical node that converts a schema to lowercase.
- Generalization of the subquery eliding rule to also elide this new node
- Fixing of several places where too tight assumptions were made on the
types of `InsertIntoTable` children.
- I also removed an API that was left in by accident that exposed catalyst
data structures, and fix the logic that pushes down filters into hive tables
scans to correctly compare attribute references.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/marmbrus/spark upperCaseFieldNames
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/202.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #202
----
commit 7018b1d80e129fc1a70fe026ab378b5915ba09b2
Author: Michael Armbrust <[email protected]>
Date: 2014-03-22T01:13:43Z
Attributes should be compared using exprId, not TreeNode.id.
commit cbeb2eb2994e1b0150c9cbfafa308649bf57624e
Author: Michael Armbrust <[email protected]>
Date: 2014-03-22T01:14:45Z
Remove API that exposes internal catalyst data structures.
commit 3955c8cfd42e20f04bfdbb49f7941583b6e3f5b7
Author: Michael Armbrust <[email protected]>
Date: 2014-03-22T01:42:59Z
Support for resolving mixed case fields from a reflected schema using
HiveQL.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
