alamb commented on code in PR #9382:
URL: https://github.com/apache/arrow-datafusion/pull/9382#discussion_r1508146478
##########
datafusion/sqllogictest/test_files/copy.slt:
##########
@@ -390,11 +400,11 @@ query IT
COPY source_table
to 'test_files/scratch/copy/table_csv_with_options'
(format csv,
-header false,
-compression 'uncompressed',
-datetime_format '%FT%H:%M:%S.%9f',
-delimiter ';',
-null_value 'NULLVAL');
+'csv.has_header' false,
+'csv.compression' uncompressed,
+'csv.datetime_format' '%FT%H:%M:%S.%9f',
+'csv.delimiter' ';',
+'csv.null_value' 'NULLVAL');
Review Comment:
From a UX perspective, I think the original was easier to write
Could we possible make both
```sql
COPY source_table
to 'test_files/scratch/copy/table_csv_with_options'
(format csv,
header false,
compression 'uncompressed',
datetime_format '%FT%H:%M:%S.%9f',
delimiter ';',
null_value 'NULLVAL');
```
AND
```sql
'csv.has_header' false,
'csv.compression' uncompressed,
'csv.datetime_format' '%FT%H:%M:%S.%9f',
'csv.delimiter' ';',
'csv.null_value' 'NULLVAL');
```
Accepted? Perhaps we could do so by first checking directly with the option
name, and if that didn't find a matching config option, try
`<format>.<config_name>` if the `format` option has been specified
##########
datafusion/sqllogictest/test_files/copy.slt:
##########
@@ -152,11 +152,21 @@ FileSinkExec: sink=ParquetSink(file_groups=[])
query error DataFusion error: Invalid or Unsupported Configuration: Format not
explicitly set and unable to get file extension!
EXPLAIN COPY source_table to 'test_files/scratch/copy/table/'
-query error DataFusion error: SQL error: ParserError\("Expected end of
statement, found: query"\)
-EXPLAIN COPY source_table to 'test_files/scratch/copy/table/' (format parquet)
query TT
+EXPLAIN COPY source_table to 'test_files/scratch/copy/table/' (format parquet)
Review Comment:
nice!
##########
datafusion/sqllogictest/test_files/copy.slt:
##########
@@ -307,7 +317,7 @@ select * from validate_parquet_single;
# copy from table to folder of compressed json files
query IT
-COPY source_table to 'test_files/scratch/copy/table_json_gz' (format json,
compression 'gzip');
+COPY source_table to 'test_files/scratch/copy/table_json_gz' (format json,
'json.compression' gzip);
Review Comment:
Does this PR now support syntax like `COPY source_table to
'test_files/scratch/copy/table_json_gz' STORED AS PARQUET`?
I am not saying it needs to / should but I thought that was one of the goals
of https://github.com/apache/arrow-datafusion/issues/9369
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
