GitHub user cloud-fan opened a pull request:
https://github.com/apache/spark/pull/14809
[SPARK-17238][SQL] simplify the logic for converting data source table into
hive compatible format
## What changes were proposed in this pull request?
Previously we have 2 conditions to decide whether a data source table is
hive-compatible:
1. the data source is file-based and has a corresponding Hive serde
2. have a `path` entry in data source options/storage properties
However, if condition 1 is true, condition 2 must be true too, as we will
put the default table path into data source options/storage properties for
managed data source tables.
There is also a potential issue: we will set the `locationUri` even for
managed table.
This PR removes the condition 2 and only set the `locationUri` for external
data source tables.
Note: this is also a first step to unify the `path` of data source tables
and `locationUri` of hive serde tables. For hive serde tables, `locationUri` is
only set for external table. For data source tables, `path` is always set. We
can make them consistent after this PR.
## How was this patch tested?
existing tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/cloud-fan/spark minor2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/14809.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #14809
----
commit 915d2b5a1dd8c26a37d0b99ba0503a0d95b6f3f3
Author: Wenchen Fan <[email protected]>
Date: 2016-08-25T15:11:23Z
simplify the logic for converting data source table into hive compatible
format
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
