GitHub user liancheng opened a pull request:
https://github.com/apache/spark/pull/7967
[SPARK-6923] [SPARK-7550] [SQL] Persists data source relations in Hive
compatible format when possible
This PR is a fork of PR #5733 authored by @chenghao-intel.
When a data source relation meets the following requirements, we persist it
in Hive compatible format, so that other systems like Hive can access it:
1. It's a `HadoopFsRelation`
2. It has only one input path
3. It's non-partitioned
4. It's data source provider can be naturally mapped to a Hive builtin
SerDe (e.g. ORC and Parquet)
This PR should be attributed to Hao from Intel. For committers who's going
to merge this PR, please set the author to "Cheng Hao <[email protected]>".
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/liancheng/spark spark-6923/refactoring-pr-5733
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7967.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7967
----
commit 6f576699c5440d03e56916b5845ecff4797bbfed
Author: Cheng Hao <[email protected]>
Date: 2015-07-29T04:57:48Z
write schema info to hivemetastore for data source
commit 3490cdc77a33a0eac7be29a53e26887a81b001c0
Author: Cheng Hao <[email protected]>
Date: 2015-07-29T05:23:28Z
update the scaladoc
commit 864aceec0479f1b8faa4eb707c7d1dc57a354c15
Author: Cheng Lian <[email protected]>
Date: 2015-08-05T14:00:45Z
Refactors PR #5733
commit 38701669121ce9e6a32d1878ece5efe41cd7e612
Author: Cheng Lian <[email protected]>
Date: 2015-08-05T18:02:33Z
Fixes build error and comments
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
