tokoko opened a new issue, #53872: URL: https://github.com/apache/arrow/issues/53872
### Describe the enhancement requested Flight SQL and ADBC can already say whether a backend accepts Substrait and which versions it accepts. They can’t say which part of Substrait it accepts. In practice no consumer implements the whole spec, so producers find out a plan is unsupported only when execution fails. Substrait now has a standard format for this: [dialect files](https://github.com/substrait-io/substrait/blob/main/text/dialect_schema.yaml). A dialect is a YAML document listing the types, relations, expressions and function signatures a system supports, plus optional mapping metadata (e.g. the engine’s own function names). For example, substrait-java already publishes one for its Spark integration ([spark_dialect.yaml](https://github.com/substrait-io/substrait-java/blob/main/spark/spark_dialect.yaml)). Flight SQL and ADBC should be able to advertise the exact dialect supported on a connection. The biggest issue here is the size, though. A dialect listing every function impl in the standard Substrait extensions (207 functions, 507 impls) plus all types, relations and expressions comes to about 34 KiB / ~1,900 lines of YAML, or about 3.6 KiB gzipped. Real dialects are subsets of this. That’s small enough to return inline once per connection, but can probably cause headaches for a particularly chatty client. So the proposal is to add two separate info codes Substrait Dialect (**SQL_SUBSTRAIT_DIALECT**) and it's hash (**SQL_SUBSTRAIT_DIALECT_HASH**). The hash is simply there to allow clients to implement some sort of a client-side cache if necessary. I'm thinking of putting these in [500, 1000) syntax codes range because they are describing substrait syntax similar to othere sql syntax codes in the same range. If you think I should move them up with vendor info codes I can do that, though that would mean the change would apply only to Flight SQL here and we will need to add them to adbc separately. ### Component(s) Format -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
