GitHub user chenghao-intel opened a pull request:
https://github.com/apache/spark/pull/3013
[SPARK-4152] [SQL] Avoid data change in CTAS while table already existed
CREATE TABLE t1 (a String);
CREATE TABLE t1 AS SELECT key FROM src; â throw exception
CREATE TABLE if not exists t1 AS SELECT key FROM src; â expect do
nothing, actually will overwrite the t1.
This PR actually contains the following change:
1) Fix bug describe above;
2) Whitelist the test case.
3) Disable the file split for small single table file(make input splits
based on HDFS block)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/chenghao-intel/spark ctas_unittest
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3013.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3013
----
commit 0e8df89854c55f490d89cf8518cd56951c577a5c
Author: Cheng Hao <[email protected]>
Date: 2014-10-10T05:26:09Z
Keep 1 split for small file in table scanning
commit 1d3bc4dfe81f5c56f825487877ab248da0446918
Author: Cheng Hao <[email protected]>
Date: 2014-10-30T07:55:59Z
Fix bug for overwrite the existed table while CTAS & add whitelist the unit
test
commit 6d18aa736bd039d1a8194c5014ad671b08e9e928
Author: Cheng Hao <[email protected]>
Date: 2014-10-30T08:19:12Z
Add missing golden file
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
