tokoko commented on PR #49796: URL: https://github.com/apache/arrow/pull/49796#issuecomment-6107050406
@ianmcook As you suggested I expanded the test suite quite of bit both in terms of drivers and use cases (different data types mainly) over in [spark-adbc](https://github.com/tokoko/spark-adbc). I included all "proper" sql adbc drivers from dbc (plus gizmodata's oracle driver). There were additional gaps as @CurtHagenlocher predicted, mainly around literal building. To render literals correctly I would have needed all of these and maybe more: | Flag | Why | |---|---| | Boolean literal | `TRUE`/`FALSE` vs `1`/`0` (already in the PR) | | Date/time literal | `DATE '...'` vs bare string (already in the PR), and it has to hold for every such type and for fractional seconds | | Timestamp with time zone literal | Three spellings, none accepted everywhere. Some engines parse a spelling and silently drop the offset, and two have no working form at all | | String literal escape | Backslash is an escape character on MySQL, ClickHouse, SingleStore and Spark, so a literal backslash has to be doubled there and nowhere else | | Non-ASCII string literal | SQL Server needs `N'...'` or the characters are lost | | Binary literal | Seven spellings across the engines: `X'..'`, `0x..`, two different `'\x..'` string forms, `HEXTORAW`, `UNHEX`, `FROM_HEX` | So I've changed direction. Instead of describing literals, the PR now reports how to write a parameter marker (`SQL_SUPPORTED_PARAMETER_MARKER`: `?`, `$1`, `:1`, `@p1`), so values are bound as Arrow data and the driver or server owns the conversion. That replaces all six flags above with one, and it doesn't grow when a new type or quirk shows up. What remains is syntax that binding can't replace: limit/offset grammar, `NULLS FIRST`/`LAST`, and LIKE escaping. Binding is not uniform in today's drivers: - In my tests it worked for every type on 9 of the 14 engines; the rest were driver bugs with specific types, which I think is a better kind of problem to have than missing flags. I'll go hunt those down in respective repos. - Spark driver doesn't support bind parameters at all, but that should be easily fixable. - ClickHouse only supports typed markers (`{name:Type}`), which a generic client can't write without knowing ClickHouse's type names. I opened https://github.com/ClickHouse/adbc_clickhouse/issues/81 to ask about that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
