Chetan Bhat created CARBONDATA-4297: ---------------------------------------
Summary: Create table(Carbon and Parquet) with combination of partitioned by, Clustered by, Sorted by and with options parameter fails with parser errors in Carbon session in Spark 2.4.5 Key: CARBONDATA-4297 URL: https://issues.apache.org/jira/browse/CARBONDATA-4297 Project: CarbonData Issue Type: Bug Components: sql Affects Versions: 2.3.0 Environment: Spark 2.4.5 Reporter: Chetan Bhat *Issue 1 : Create table* *(Carbon and Parquet) with combination of partitioned by, Clustered by, Sorted by fails -* *Queries-* CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as carbondata OPTIONS (a '1', b '2') PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS COMMENT 'table_comment' TBLPROPERTIES (t 'test'); CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as parquet OPTIONS (a '1', b '2') PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS COMMENT 'table_comment' TBLPROPERTIES (t 'test'); 0: jdbc:hive2://7.187.185.158:23040/default> CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as carbondata 0: jdbc:hive2://7.187.185.158:23040/default> OPTIONS (a '1', b '2') 0: jdbc:hive2://7.187.185.158:23040/default> PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS 0: jdbc:hive2://7.187.185.158:23040/default> COMMENT 'table_comment' 0: jdbc:hive2://7.187.185.158:23040/default> TBLPROPERTIES (t 'test'); Error: org.apache.spark.sql.AnalysisException: == Spark Parser: org.apache.spark.sql.execution.SparkSqlParser == mismatched input 'OPTIONS' expecting <EOF>(line 2, pos 0) == SQL == CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as carbondata OPTIONS (a '1', b '2') ^^^ PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS COMMENT 'table_comment' TBLPROPERTIES (t 'test') == Carbon Parser: org.apache.spark.sql.parser.CarbonExtensionSpark2SqlParser == [1.8] failure: identifier matching regex (?i)MATERIALIZED expected CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as carbondata ^; == Antlr Parser: org.apache.spark.sql.parser.CarbonAntlrParser == Antlr SQL Parser will only deal with Merge Into SQL Command; (state=,code=0) 0: jdbc:hive2://7.187.185.158:23040/default> CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as parquet 0: jdbc:hive2://7.187.185.158:23040/default> OPTIONS (a '1', b '2') 0: jdbc:hive2://7.187.185.158:23040/default> PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS 0: jdbc:hive2://7.187.185.158:23040/default> COMMENT 'table_comment' 0: jdbc:hive2://7.187.185.158:23040/default> TBLPROPERTIES (t 'test'); Error: org.apache.spark.sql.AnalysisException: == Spark Parser: org.apache.spark.sql.execution.SparkSqlParser == mismatched input 'OPTIONS' expecting <EOF>(line 2, pos 0) == SQL == CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as parquet OPTIONS (a '1', b '2') ^^^ PARTITIONED BY (c, d) CLUSTERED BY (a) SORTED BY (b ASC) INTO 2 BUCKETS COMMENT 'table_comment' TBLPROPERTIES (t 'test') == Carbon Parser: org.apache.spark.sql.parser.CarbonExtensionSpark2SqlParser == [1.8] failure: identifier matching regex (?i)MATERIALIZED expected CREATE TABLE t (a STRING, b INT, c STRING, d STRING) stored as parquet ^; == Antlr Parser: org.apache.spark.sql.parser.CarbonAntlrParser == Antlr SQL Parser will only deal with Merge Into SQL Command; (state=,code=0) 0: jdbc:hive2://7.187.185.158:23040/default> *Issue 2 : Create table with options parameter fails-* *Queries-* CREATE TABLE tbl (a INT, b STRING, c INT) stored as carbondata OPTIONS ('a' 1); CREATE TABLE tbl1 (a INT, b STRING, c INT) stored as parquet OPTIONS ('a' 1); 0: jdbc:hive2://7.187.185.158:23040/default> CREATE TABLE tbl (a INT, b STRING, c INT) stored as carbondata OPTIONS ('a' 1); Error: org.apache.spark.sql.AnalysisException: == Spark Parser: org.apache.spark.sql.execution.SparkSqlParser == mismatched input 'OPTIONS' expecting <EOF>(line 1, pos 63) == SQL == CREATE TABLE tbl (a INT, b STRING, c INT) stored as carbondata OPTIONS ('a' 1) ---------------------------------------------------------------^^^ == Carbon Parser: org.apache.spark.sql.parser.CarbonExtensionSpark2SqlParser == [1.8] failure: identifier matching regex (?i)MATERIALIZED expected CREATE TABLE tbl (a INT, b STRING, c INT) stored as carbondata OPTIONS ('a' 1) ^; == Antlr Parser: org.apache.spark.sql.parser.CarbonAntlrParser == Antlr SQL Parser will only deal with Merge Into SQL Command; (state=,code=0) 0: jdbc:hive2://7.187.185.158:23040/default> CREATE TABLE tbl1 (a INT, b STRING, c INT) stored as parquet OPTIONS ('a' 1); Error: org.apache.spark.sql.AnalysisException: == Spark Parser: org.apache.spark.sql.execution.SparkSqlParser == mismatched input 'OPTIONS' expecting <EOF>(line 1, pos 61) == SQL == CREATE TABLE tbl1 (a INT, b STRING, c INT) stored as parquet OPTIONS ('a' 1) -------------------------------------------------------------^^^ == Carbon Parser: org.apache.spark.sql.parser.CarbonExtensionSpark2SqlParser == [1.8] failure: identifier matching regex (?i)MATERIALIZED expected CREATE TABLE tbl1 (a INT, b STRING, c INT) stored as parquet OPTIONS ('a' 1) ^; == Antlr Parser: org.apache.spark.sql.parser.CarbonAntlrParser == Antlr SQL Parser will only deal with Merge Into SQL Command; (state=,code=0) -- This message was sent by Atlassian Jira (v8.3.4#803005)