Github user zero323 commented on the issue:
https://github.com/apache/spark/pull/17938
@gatorsmile Huh... in that case it looks like parser (?) needs a little
bit of work, unless of course following are features.
- Omitting `USING` doesn't work
```sql
CREATE TABLE user_info_bucketed(user_id BIGINT, firstname STRING,
lastname STRING)
CLUSTERED BY(user_id) INTO 256 BUCKETS
```
with:
```
Error in query:
Operation not allowed: CREATE TABLE ... CLUSTERED BY(line 1, pos 0)
== SQL ==
CREATE TABLE user_info_bucketed(user_id BIGINT, firstname STRING,
lastname STRING)
^^^
CLUSTERED BY(user_id) INTO 256 BUCKETS
```
- Omitting `USING` adding `PARTITION BY` with column not present in the
main clause (valid Hive DDL) doesn't work:
```sql
CREATE TABLE user_info_bucketed(user_id BIGINT, firstname STRING,
lastname STRING)
PARTITIONED BY (department STRING)
CLUSTERED BY(user_id) INTO 256 BUCKETS
```
with
```
Error in query:
Operation not allowed: CREATE TABLE ... CLUSTERED BY(line 1, pos 2)
== SQL ==
CREATE TABLE user_info_bucketed(user_id BIGINT, firstname STRING,
lastname STRING)
--^^^
PARTITIONED BY (department STRING)
CLUSTERED BY(user_id) INTO 256 BUCKETS
```
- `PARTITION BY` alone works:
```sql
CREATE TABLE user_info_bucketed(user_id BIGINT, firstname STRING,
lastname STRING)
PARTITIONED BY (department STRING)
```
- `PARTITION BY` with `USING` when partition column is in the main spec
works:
```sql
CREATE TABLE user_info_bucketed(
user_id BIGINT, firstname STRING, lastname STRING, department STRING)
USING parquet
PARTITIONED BY (department)
```
- `CLUSTERED BY` + `PARTITION BY` with `USING` when partition column is
in the main spec works:
```sql
CREATE TABLE user_info_bucketed(
user_id BIGINT, firstname STRING, lastname STRING, department STRING)
USING parquet
PARTITIONED BY (department)
CLUSTERED BY(user_id) INTO 256 BUCKETS
```
- `PARTITION BY` when parition column is in the main spec, `USING` omitted:
```sql
CREATE TABLE user_info_bucketed(
user_id BIGINT, firstname STRING, lastname STRING, department STRING)
PARTITIONED BY (department)
```
with:
```
Error in query:
mismatched input ')' expecting {'SELECT', 'FROM', 'ADD', 'AS', 'ALL',
'DISTINCT', 'WHERE', 'GROUP', 'BY', 'GROUPING', 'SETS', 'CUBE', 'ROLLUP',
'ORDER', 'HAVING', 'LIMIT', 'AT', 'OR', 'AND', 'IN', NOT, 'NO', 'EXISTS',
'BETWEEN', 'LIKE', RLIKE, 'IS', 'NULL', 'TRUE', 'FALSE', 'NULLS', 'ASC',
'DESC', 'FOR', 'INTERVAL', 'CASE', 'WHEN', 'THEN', 'ELSE', 'END', 'JOIN',
'CROSS', 'OUTER', 'INNER', 'LEFT', 'SEMI', 'RIGHT', 'FULL', 'NATURAL', 'ON',
'LATERAL', 'WINDOW', 'OVER', 'PARTITION', 'RANGE', 'ROWS', 'UNBOUNDED',
'PRECEDING', 'FOLLOWING', 'CURRENT', 'FIRST', 'AFTER', 'LAST', 'ROW', 'WITH',
'VALUES', 'CREATE', 'TABLE', 'VIEW', 'REPLACE', 'INSERT', 'DELETE', 'INTO',
'DESCRIBE', 'EXPLAIN', 'FORMAT', 'LOGICAL', 'CODEGEN', 'COST', 'CAST', 'SHOW',
'TABLES', 'COLUMNS', 'COLUMN', 'USE', 'PARTITIONS', 'FUNCTIONS', 'DROP',
'UNION', 'EXCEPT', 'MINUS', 'INTERSECT', 'TO', 'TABLESAMPLE', 'STRATIFY',
'ALTER', 'RENAME', 'ARRAY', 'MAP', 'STRUCT', 'COMMENT', 'SET', 'RESET', 'DATA',
'START', 'TRANSA
CTION', 'COMMIT', 'ROLLBACK', 'MACRO', 'IGNORE', 'IF', 'DIV', 'PERCENT',
'BUCKET', 'OUT', 'OF', 'SORT', 'CLUSTER', 'DISTRIBUTE', 'OVERWRITE',
'TRANSFORM', 'REDUCE', 'USING', 'SERDE', 'SERDEPROPERTIES', 'RECORDREADER',
'RECORDWRITER', 'DELIMITED', 'FIELDS', 'TERMINATED', 'COLLECTION', 'ITEMS',
'KEYS', 'ESCAPED', 'LINES', 'SEPARATED', 'FUNCTION', 'EXTENDED', 'REFRESH',
'CLEAR', 'CACHE', 'UNCACHE', 'LAZY', 'FORMATTED', 'GLOBAL', TEMPORARY,
'OPTIONS', 'UNSET', 'TBLPROPERTIES', 'DBPROPERTIES', 'BUCKETS', 'SKEWED',
'STORED', 'DIRECTORIES', 'LOCATION', 'EXCHANGE', 'ARCHIVE', 'UNARCHIVE',
'FILEFORMAT', 'TOUCH', 'COMPACT', 'CONCATENATE', 'CHANGE', 'CASCADE',
'RESTRICT', 'CLUSTERED', 'SORTED', 'PURGE', 'INPUTFORMAT', 'OUTPUTFORMAT',
DATABASE, DATABASES, 'DFS', 'TRUNCATE', 'ANALYZE', 'COMPUTE', 'LIST',
'STATISTICS', 'PARTITIONED', 'EXTERNAL', 'DEFINED', 'REVOKE', 'GRANT', 'LOCK',
'UNLOCK', 'MSCK', 'REPAIR', 'RECOVER', 'EXPORT', 'IMPORT', 'LOAD', 'ROLE',
'ROLES', 'COMPACTIONS', 'PRINCIPALS', 'T
RANSACTIONS', 'INDEX', 'INDEXES', 'LOCKS', 'OPTION', 'ANTI', 'LOCAL',
'INPATH', 'CURRENT_DATE', 'CURRENT_TIMESTAMP', IDENTIFIER,
BACKQUOTED_IDENTIFIER}(line 3, pos 30)
== SQL ==
CREATE TABLE user_info_bucketed(
user_id BIGINT, firstname STRING, lastname STRING, department
STRING)
PARTITIONED BY (department)
------------------------------^^^
```
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
