devinjdangelo commented on PR #7466:
URL:
https://github.com/apache/arrow-datafusion/pull/7466#issuecomment-1709201991
> This looks great to me -- thank you @devinjdangelo 🙏
>
> The only other thing I think is needed for this PR is to an end to end
test showing this syntax being used (perhaps in copy.slt)
>
> Also, we should add documentation for this syntax to dml.md, but I think
that can be done in a follow on PR (I will file a ticket to track it).
>
> Also again sorry for the delay in reviews
Np, hope you had a good vacation! 🌴
I just pushed up an expanded test in copy.slt that uses all available
options.
```sql
COPY source_table
TO 'test_files/scratch/copy/table_with_options'
(format parquet,
single_file_output false,
compression snappy,
'compression::col1' 'zstd(5)',
'compression::col2' snappy,
max_row_group_size 12345,
data_pagesize_limit 1234,
write_batch_size 1234,
writer_version 2.0,
dictionary_page_size_limit 123,
created_by 'DF copy.slt',
column_index_truncate_length 123,
data_page_row_count_limit 1234,
bloom_filter_enabled true,
'bloom_filter_enabled::col1' false,
'bloom_filter_fpp::col2' 0.456,
'bloom_filter_ndv::col2' 456,
encoding plain,
'encoding::col1' DELTA_BINARY_PACKED,
'dictionary_enabled::col2' true,
dictionary_enabled false,
statistics_enabled page,
'statistics_enabled::col2' none,
max_statistics_size 123,
bloom_filter_fpp 0.001,
bloom_filter_ndv 100
)
```
The only thing that bothers me a bit with the syntax is that you have to put
column level options inside single quotes (i.e. 'compression::col1') otherwise
the :: generates a syntax error I think coming from sqlparser-rs.
```rust
External error: query failed: DataFusion error: SQL error:
ParserError("Expected string or numeric value, found: ::")
```
Would there be an easy way to relax the syntax requirements to allow ::
without a string literal quote?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
