GitHub user saucam opened a pull request:
https://github.com/apache/spark/pull/5141
[SPARK-6471][SQL]: Metastore schema should only be a subset of parquet
schema to support dropping of columns using replace columns
Currently in the parquet relation 2 implementation, error is thrown in case
merged schema is not exactly the same as metastore schema.
But to support cases like deletion of column using replace column command,
we can relax the restriction so that even if metastore schema is a subset of
merged parquet schema, the query will work.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/saucam/spark replace_col
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/5141.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5141
----
commit 5f2f4674084b4f6202c0eb884b798f0980659b4b
Author: Yash Datta <[email protected]>
Date: 2015-03-23T17:35:45Z
SPARK-6471: Metastore schema should only be a subset of parquet schema to
support dropping of columns using replace columns
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
