GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/12204
[SPARK-14427][SQL] Support persisting partitioned data source relations in
Hive compatible format
## What changes were proposed in this pull request?
JIRA: https://issues.apache.org/jira/browse/SPARK-14427
Currently we can't persist partitioned data source relations in Hive
compatible format. If you want to persist partitioned data source relations
into Hive, you will get a SparkSQL specific metastore table that Hive can't
read.
We should add the support to improve compatibility with Hive.
## How was this patch tested?
Add tests into `SQLQuerySuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 write-partitioned-hive-table
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12204.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12204
----
commit f910115ca23615868119015e51600e3ac18efaa1
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-03-24T09:43:03Z
init import.
commit 727623243426561254bf7be23ac1775920d73fa8
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-03-30T02:49:11Z
Merge remote-tracking branch 'upstream/master' into
write-partitioned-hive-table
commit 39474ba981cbc31babd4648ae1874196bc91321e
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-04-06T02:33:12Z
Merge remote-tracking branch 'upstream/master' into
write-partitioned-hive-table
commit 6e53aa85755e0f9ef8c932550afde406da30c1e5
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-04-06T10:19:14Z
Make sure we create new partitions in Hive for SaveMode.Append.
commit 57078cf027ce7b395adfb0df886a4295778554cd
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-04-06T10:38:32Z
Add test cases.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
