Will Jones created ARROW-16844:
----------------------------------
Summary: [C++][Python] Implement to/from substrait for Expression
Key: ARROW-16844
URL: https://issues.apache.org/jira/browse/ARROW-16844
Project: Apache Arrow
Issue Type: Improvement
Components: C++, Python
Reporter: Will Jones
DataFusion has the ability to convert between Substrait expressions and it's
own internal expressions. (See:
[https://github.com/datafusion-contrib/datafusion-substrait] .) It would be
cool if we had a similar conversion for Acero's Expression class.
This might unlock allowing datafusion-python to easily use PyArrow datasets, by
using Substrait as intermediate format to pass down filter and projections from
Datafusion into the scanner. (See early draft here:
[https://github.com/datafusion-contrib/datafusion-python/pull/21].)
One problem is that it's unclear what should be the type of the object in
Python representing the Substrait expression. IIUC Python doesn't have direct
bindings to the Substrait protobuf.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
