GitHub user xwu0226 opened a pull request:
https://github.com/apache/spark/pull/12406
[SPARK-14346] [SQL][WIP] Show create table
## What changes were proposed in this pull request?
Allow users to issue "`SHOW CREATE TABLE`" command natively in SparkSQL.
1. For tables that are created by Hive, this command will display the DDL
in hive syntax. If the syntax includes `CLUSTERED BY, SKEWED BY or STORED BY`
clause, there will be a warning message saying that this DDL is not supported
in SparkSQL native DDL yet.
2. For tables that are created by datasource DDL, such as "`CREATE TABLE...
USING ... OPTIONS (...)`", it will show the DDL in this syntax.
3. For tables that are created by dataframe API, such as
"`df.write.partitionBy(...).saveAsTable(...)`", currently the command will
display DDL with the syntax "CREATE TABLE.. USING...OPTIONS(...)". However,
this syntax lose the partitioning information. It is proposed to display create
table in the dataframe API format (in progress)
## How was this patch tested?
Unit tests are created.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xwu0226/spark show_create_table_2
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12406.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12406
----
commit 0ebb0142e13db3ce8fb474ee5682528b0f87d2d2
Author: xin Wu <[email protected]>
Date: 2016-04-02T01:46:16Z
show create table DDL -- hive metastore table
commit 6d060be797d4127f0b86fa59c1bc848d75215533
Author: xin Wu <[email protected]>
Date: 2016-04-02T06:01:46Z
update upon review
commit 2799672162d715b209cad9a5c103d6f09692d8dc
Author: xin Wu <[email protected]>
Date: 2016-04-02T18:19:26Z
ignoring sqlContext temp table and considering datasource table ddl
commit 98c020aa9a5374861d1470fa0c305148e8314ada
Author: xin Wu <[email protected]>
Date: 2016-04-04T21:54:32Z
fix scala style issue
commit efd889821bf84e328ef6dd8d0b6a645729248251
Author: xin Wu <[email protected]>
Date: 2016-04-04T22:40:26Z
fix scala style issue in testcase
commit b370630f5827071bc5076e9b3fa9c92720b27eb2
Author: xin Wu <[email protected]>
Date: 2016-04-05T01:31:46Z
fix testcase for test failure
commit 8cb7a7299df84f2608b91b092a7df6795b85d41e
Author: xin Wu <[email protected]>
Date: 2016-04-06T18:12:07Z
continue the database ddl generation
commit 8b67d22c5ed8fd6b309df772e4a372e741acf630
Author: xin Wu <[email protected]>
Date: 2016-04-08T20:57:12Z
support datasource ddl
commit 9ab863fb7f8127d1acd083b1ba857f5c1fd2769c
Author: xin Wu <[email protected]>
Date: 2016-04-08T22:04:05Z
scala style fix
commit a40273c7989bebdf62b93ce6e604bb14cacce100
Author: xin Wu <[email protected]>
Date: 2016-04-13T22:54:16Z
merge the code committed by CREATE TABLE native support
commit d214a3b0c54641a6234ba39eef82b2b8ac4c87dd
Author: xin Wu <[email protected]>
Date: 2016-04-14T23:49:03Z
rework show create ddl based on new native supported create table DDL work
commit 1680ea0403f0d29185d9a3f8f81d15599be81aac
Author: xin Wu <[email protected]>
Date: 2016-04-14T23:51:03Z
Merge branch 'show_create_table_1' into show_create_table_2
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
