[
https://issues.apache.org/jira/browse/CALCITE-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17629426#comment-17629426
]
Jiajun Xie edited comment on CALCITE-4455 at 11/6/22 7:34 AM:
--------------------------------------------------------------
"SqlOverwriteTable" maybe better than "SqlInsertOverwriteTable".
I think that expanding SqlInsert is not a viable solution. Core module codes
will be broken because OVERWRITE DIRECTORY.
I'm looking for a way to improve the extensibility of SqlValidatorImpl: I hope
to call custom interfaces in SqlValidatorImpl, and then implement those
interfaces in the babel module.
was (Author: jiajunbernoulli):
"SqlOverwriteTable" maybe better than "SqlInsertOverwriteTable".
I think expanding SqlInsert is not a viable solution. Core module codes will
be broken because OVERWRITE DIRECTORY.
I'm looking for a way to improve the extensibility of SqlValidatorImpl: I hope
to call custom interfaces in SqlValidatorImpl, and then implement those
interfaces in the babel module.
> Babel parser support Spark INSERT OVERWRITE TABLE/DIRECTORY statement
> ---------------------------------------------------------------------
>
> Key: CALCITE-4455
> URL: https://issues.apache.org/jira/browse/CALCITE-4455
> Project: Calcite
> Issue Type: Sub-task
> Components: spark
> Reporter: shradha
> Assignee: Jiajun Xie
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Spark has [INSERT OVERWRITE TABLE
> statement|https://spark.apache.org/docs/3.1.2/sql-ref-syntax-dml-insert-overwrite-table.html]
> and [INSERT OVERWRITE DIRECTORY
> statement|https://spark.apache.org/docs/3.1.2/sql-ref-syntax-dml-insert-overwrite-directory.html].
>
> We can support part of their grammar, here is syntax that babel parser used:
> - INSERT OVERWRITE TABLE
> -- Syntax
> {code:java}
> INSERT OVERWRITE TABLE table_identifier [ partition_spec [ IF NOT EXISTS ] ]
> [ ( column_list ) ]
> { VALUES ( { value | NULL } [ , ... ] ) [ , ( ... ) ] | query } {code}
> -- Example
> {code:java}
> -- Insert Using a SELECT Statement
> INSERT OVERWRITE TABLE students PARTITION (student_id = 222222)
> SELECT name, address FROM persons WHERE name = "Dora Williams";
> -- Insert Using a SELECT Statement with column list and omit partition
> INSERT OVERWRITE TABLE target (c1) SELECT * FROM source
> -- Insert Using a VALUES Clause
> INSERT OVERWRITE TABLE students
> VALUES ('Ashua Hill', '456 Erica Ct, Cupertino', 111111),
> ('Brian Reed', '723 Kern Ave, Palo Alto', 222222); {code}
> - INSERT OVERWRITE DIRECTORY
> -- Syntax
> {code:java}
> INSERT OVERWRITE [ LOCAL ] DIRECTORY directory_path
> USING file_format [ OPTIONS ( key = val [ , ... ] ) ]
> { VALUES ( { value | NULL } [ , ... ] ) [ , ( ... ) ] | query }{code}
> -- Example
> {code:java}
> INSERT OVERWRITE DIRECTORY '/tmp/destination'
> USING parquet
> OPTIONS (col1=1, col2=2, col3='test')
> SELECT * FROM test_table;
> -- omit options
> INSERT OVERWRITE DIRECTORY '/tmp/destination'
> USING parquet
> SELECT * FROM test_table;{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
