GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/13068
[SPARK-15279][SQL] Catch conflicting SerDe when creating table
## What changes were proposed in this pull request?
The user may do something like:
```
CREATE TABLE my_tab ROW FORMAT SERDE 'anything' STORED AS PARQUET
CREATE TABLE my_tab ROW FORMAT SERDE 'anything' STORED AS ... SERDE
'myserde'
CREATE TABLE my_tab ROW FORMAT DELIMITED ... STORED AS ORC
CREATE TABLE my_tab ROW FORMAT DELIMITED ... STORED AS ... SERDE 'myserde'
```
None of these should be allowed because the SerDe's conflict. As of this
patch:
- `ROW FORMAT DELIMITED` is only compatible with `TEXTFILE`
- `ROW FORMAT SERDE` is only compatible with `TEXTFILE`, `RCFILE` and
`SEQUENCEFILE`
## How was this patch tested?
New tests in `DDLCommandSuite`.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark row-format-conflict
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13068.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13068
----
commit 796adc3407094b1a1e7adfc10690d3164e9453be
Author: Andrew Or <[email protected]>
Date: 2016-05-11T23:53:39Z
Throw exception on conflicting SerDes
commit 62c27f2f9a0dc516f3c4d39cad78c37532666764
Author: Andrew Or <[email protected]>
Date: 2016-05-12T00:21:20Z
Add more tests
commit bf6b23d6d840d97ecc9376b7937cd3f17e3a68e8
Author: Andrew Or <[email protected]>
Date: 2016-05-12T00:28:56Z
Clean up duplicate code a little
commit d4b5a32c6ee1ae1581b8f50ade984751ea4a3ca1
Author: Andrew Or <[email protected]>
Date: 2016-05-12T00:30:24Z
Merge branch 'master' of github.com:apache/spark into row-format-conflict
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
