alamb opened a new issue, #10414: URL: https://github.com/apache/datafusion/issues/10414
### Is your feature request related to a problem or challenge? `CREATE EXTERNAL TABLE` (see [docs](https://datafusion.apache.org/user-guide/sql/ddl.html#create-external-table)) has several syntax items that only make sense for certain formats, specifically ```text CREATE [UNBOUNDED] EXTERNAL TABLE [ IF NOT EXISTS ] <TABLE_NAME>[ (<column_definition>) ] STORED AS <file_type> [ WITH HEADER ROW ] <-- only makes sense for CSV [ DELIMITER <char> ] <-- only makes sense for CSV, JSON [ COMPRESSION TYPE <GZIP | BZIP2 | XZ | ZSTD> ] <-- only makes sense for CSV, JSON [ PARTITIONED BY (<column list>) ] [ WITH ORDER (<ordered column list>) ] [ OPTIONS (<key_value_list>) ] LOCATION <literal> ``` Now, thanks to @metesynnada @devinjdangelo and others, we have a way to support arbitrary format specific options such as `has_header`: ```sql CREATE EXTERNAL TABLE aggregate_simple ( c1 FLOAT NOT NULL, c2 DOUBLE NOT NULL, c3 BOOLEAN NOT NULL ) STORED AS CSV LOCATION '../core/tests/data/aggregate_simple.csv' OPTIONS('format.has_header' 'true') ``` However, the old inconsistent syntax is still supported, which causes additional code complexity as noted by @ozankabak in https://github.com/apache/datafusion/pull/10404#issuecomment-2099134059 and potential confusion with users (who don't know how to look for the options) ### Describe the solution you'd like 1. I think we should at least update the documentation so it sends people to the new preferred syntax (`OPTIONS(..)`). 2. We should consider deprecating / removing the old `CREATE TABLE` syntax `DELIMITER`, `WITH HEADER ROW` and `COMPRESSION` ### Describe alternatives you've considered _No response_ ### Additional context @metesynnada did some great work to get `COPY` to align to external table: https://github.com/apache/datafusion/pull/9604 @berkaysynnada has noticed issues related to inconsistency in the `EXTERNAL TABLE` syntax and config options https://github.com/apache/datafusion/issues/9945 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
