GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/15566
[SPARK-18026][SQL] should not always lowercase partition columns of
partition spec in parser
## What changes were proposed in this pull request?
Currently we always lowercase the partition columns of partition spec in
parser, with the assumption that table partition columns are always lowercased.
However, this is not true for data source tables, which are case
preserving. It's safe for now because data source tables don't store partition
spec in metastore and don't support `ADD PARTITION`, `DROP PARTITION`, `RENAME
PARTITION`, but we should make our code future-proof.
This PR makes partition spec case preserving, and improve the
`PreprocessTableInsertion` analyzer rule to normalize the partition columns in
partition spec, w.r.t. the table partition columns.
## How was this patch tested?
existing tests.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark partition-spec
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/15566.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #15566
----
commit 82c9f8661260ba2acb34311aaa9e4fc6842bb65e
Author: Wenchen Fan <[email protected]>
Date: 2016-10-20T09:02:32Z
should not always lowercase partition columns of partition spec in parser
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
