[
https://issues.apache.org/jira/browse/ARROW-15582?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17527869#comment-17527869
]
Weston Pace commented on ARROW-15582:
-------------------------------------
It's unclear how much testing should be done here. Conformance testing for
Substrait is a large question and could/should be shared by multiple
implementations. If there were a sophisticated conformance tester then testing
of the mapping code could be fairly minimal.
Also, if there were conformance testing then I think one could prioritize /
only implement the Substrait -> Arrow path as the Arrow -> Substrait path is
currently only used for testing (although one can imagine non-testing
applications).
> [C++] Add support for registering tricky functions with the Substrait
> consumer (or add a bunch of substrait meta functions)
> ---------------------------------------------------------------------------------------------------------------------------
>
> Key: ARROW-15582
> URL: https://issues.apache.org/jira/browse/ARROW-15582
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Weston Pace
> Priority: Major
> Labels: substrait
>
> Sometimes one Substrait function will map to multiple Arrow functions. For
> example, the Substrait {{add}} function might be referring to Arrow's {{add}}
> or {{add_checked}}. We need to figure out how to register this correctly
> (e.g. one possible approach would be a {{substrait_add}} meta function).
> Other times a substrait function will encode something Arrow considers an
> "option" as a function argument. For example, the is_in Arrow function is
> unary with an option for the lookup set. The substrait function is binary
> but the second argument must be constant and be the lookup set. Neither of
> which is to be confused with a truly binary is_in function which takes in a
> different set at every row.
> It's possible there is no work to do here other than adding a bunch of
> substrait_ meta functions in Arrow. In that case all the work will be done
> in other JIRAs. Or, it is possible that there is some kind of extension we
> can make to the function registry that bypasses the need for the meta
> functions. I'm leaving this JIRA open so future contributors can consider
> this second option.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
