[
https://issues.apache.org/jira/browse/ARROW-15535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17486429#comment-17486429
]
Jeroen van Straten commented on ARROW-15535:
--------------------------------------------
As a followup to this (or maybe a predecessor, not sure which makes more
sense?), it may also be useful to expose an API to generate additional YAML for
custom compute functions that the user registers at runtime.
> [C++] Investigate automatic generation of YAML for Substrait extension
> functions & types
> ----------------------------------------------------------------------------------------
>
> Key: ARROW-15535
> URL: https://issues.apache.org/jira/browse/ARROW-15535
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Priority: Major
> Labels: substrait
>
> Arrow supports data types that are not in the Substrait spec (e.g. unsigned
> integers) and supports functions that are not in the Substrait spec.
> As a convenience for consumers we can generate a YAML document that describes
> all the things Arrow supports that are not part of the Substrait spec.
> ARROW-15238 has an incomplete example of what this YAML might look like.
> This YAML can allow for things like better ahead-of-time validation by
> milddleman validators and optimizers. It is not strictly required to use
> these features.
> We should investigate our ability to generate this automatically. However,
> this can get fairly complicated. For example, we don't want to blindly
> generate YAML for all functions and types because many of the functions and
> types map to existing functions and types in the core Substrait spec.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
