GitHub user xwu0226 opened a pull request:
https://github.com/apache/spark/pull/12579
[SPARK-14346][SQL] Show Create Table (Native)
This is a rebased version of
[#12132](https://github.com/apache/spark/pull/12132) and
[#12406](https://github.com/apache/spark/pull/12406)
## What changes were proposed in this pull request?
Allow users to issue "`SHOW CREATE TABLE`" command natively in SparkSQL.
-- For tables that are created by Hive, this command will display the DDL
in hive syntax. If the syntax includes `CLUSTERED BY, SKEWED BY or STORED BY`
clause, there will be a warning message saying that this DDL is not supported
in SparkSQL native DDL yet.
-- For tables that are created by datasource DDL, such as "`CREATE TABLE...
USING ... OPTIONS (...)`", it will show the DDL in this syntax.
-- For tables that are created by dataframe API, such as
"`df.write.partitionBy(...).saveAsTable(...)`", currently the command will
display DDL with the syntax "CREATE TABLE.. USING...OPTIONS(...)". However,
this syntax lose the partitioning information. It is proposed to display create
table in the dataframe API format.
## How was this patch tested?
Unit tests are created.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/xwu0226/spark show_create_table_3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12579.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12579
----
commit 0ebb0142e13db3ce8fb474ee5682528b0f87d2d2
Author: xin Wu <[email protected]>
Date: 2016-04-02T01:46:16Z
show create table DDL -- hive metastore table
commit 6d060be797d4127f0b86fa59c1bc848d75215533
Author: xin Wu <[email protected]>
Date: 2016-04-02T06:01:46Z
update upon review
commit 2799672162d715b209cad9a5c103d6f09692d8dc
Author: xin Wu <[email protected]>
Date: 2016-04-02T18:19:26Z
ignoring sqlContext temp table and considering datasource table ddl
commit 98c020aa9a5374861d1470fa0c305148e8314ada
Author: xin Wu <[email protected]>
Date: 2016-04-04T21:54:32Z
fix scala style issue
commit efd889821bf84e328ef6dd8d0b6a645729248251
Author: xin Wu <[email protected]>
Date: 2016-04-04T22:40:26Z
fix scala style issue in testcase
commit b370630f5827071bc5076e9b3fa9c92720b27eb2
Author: xin Wu <[email protected]>
Date: 2016-04-05T01:31:46Z
fix testcase for test failure
commit 8cb7a7299df84f2608b91b092a7df6795b85d41e
Author: xin Wu <[email protected]>
Date: 2016-04-06T18:12:07Z
continue the database ddl generation
commit 8b67d22c5ed8fd6b309df772e4a372e741acf630
Author: xin Wu <[email protected]>
Date: 2016-04-08T20:57:12Z
support datasource ddl
commit 9ab863fb7f8127d1acd083b1ba857f5c1fd2769c
Author: xin Wu <[email protected]>
Date: 2016-04-08T22:04:05Z
scala style fix
commit a40273c7989bebdf62b93ce6e604bb14cacce100
Author: xin Wu <[email protected]>
Date: 2016-04-13T22:54:16Z
merge the code committed by CREATE TABLE native support
commit d214a3b0c54641a6234ba39eef82b2b8ac4c87dd
Author: xin Wu <[email protected]>
Date: 2016-04-14T23:49:03Z
rework show create ddl based on new native supported create table DDL work
commit 1680ea0403f0d29185d9a3f8f81d15599be81aac
Author: xin Wu <[email protected]>
Date: 2016-04-14T23:51:03Z
Merge branch 'show_create_table_1' into show_create_table_2
commit fa8373c3fd2d27cf2b3356ee0214c8e04dfc0f36
Author: xin Wu <[email protected]>
Date: 2016-04-15T02:03:41Z
remove spaces
commit 5095b6c871de55e871c5ea606ade6ab0b2166627
Author: xin Wu <[email protected]>
Date: 2016-04-15T16:24:53Z
update upon review - use visitTableIdentifier
commit 15f226c7d4f195947cbb1acc341eaaae4072d4a6
Author: xin Wu <[email protected]>
Date: 2016-04-20T18:28:29Z
generate dataframe API create table for some datasource tables
commit 601867ae71cc370770deddd56cc8883b04dcf8ee
Author: xin Wu <[email protected]>
Date: 2016-04-20T18:31:27Z
synch up with master branch
commit 687f7aca56cf5c032ceac09c341b2dfd00129b8e
Author: xin Wu <[email protected]>
Date: 2016-04-20T21:54:51Z
update upon review
commit bf3512ba01e773a514350030cfa91087de10fc03
Author: xin Wu <[email protected]>
Date: 2016-04-20T22:07:55Z
synch up with latest change
commit ca44d67584f358bd588743d33de2b7d689df584d
Author: xin Wu <[email protected]>
Date: 2016-04-21T05:35:04Z
synch up again
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
