devinjdangelo commented on issue #9269: URL: https://github.com/apache/arrow-datafusion/issues/9269#issuecomment-1952539543
I tried searching documentation of various engines to see if ' is allowed in partition columns. I didn't find anything concrete. However, I tried the equivalent example in DuckDB and it does work (see below). It would probably be best to tighten up our parsing of non standard column names. It may be easier to make this robust by extending sqlparser-rs upstream. The recent discussion on the mailing list is relevant https://lists.apache.org/[email protected]. ```bash devinjd@devinjd$ ./duckdb v0.10.0 20b1486d11 Enter ".help" for usage hints. Connected to a transient in-memory database. Use ".open FILENAME" to reopen on a persistent database. D create table test ("'test'" varchar, "'test2'" varchar, "'test3'" varchar); D insert into test VALUES ('a', 'x', 'aa'), ('b','y', 'bb'), ('c', 'z', 'cc'); D select * from test; ┌─────────┬─────────┬─────────┐ │ 'test' │ 'test2' │ 'test3' │ │ varchar │ varchar │ varchar │ ├─────────┼─────────┼─────────┤ │ a │ x │ aa │ │ b │ y │ bb │ │ c │ z │ cc │ └─────────┴─────────┴─────────┘ D copy test to '/tmp/escape_quote' (format csv, partition_by ('''test2''','''test3''')); D select * from read_csv('/tmp/escape_quote/*/*/*.csv', hive_partitioning=1, header=true); ┌─────────┬─────────┬─────────┐ │ 'test' │ 'test2' │ 'test3' │ │ varchar │ varchar │ varchar │ ├─────────┼─────────┼─────────┤ │ a │ x │ aa │ │ b │ y │ bb │ │ c │ z │ cc │ └─────────┴─────────┴─────────┘ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
