westonpace opened a new issue, #40695: URL: https://github.com/apache/arrow/issues/40695
### Describe the enhancement requested The Arrow<->Substrait conversion currently only works with the types that are supported by both Arrow and Substrait. I would like to use Substrait expression conversion for filter pushdown (polars can convert to a pyarrow.compute expression, and datafusion can consume a substrait expression, and I would like to bridge the two). This is currently blocked by the fact that polars uses large_string by default and pyarrow.compute expressions fail to serialize if they contain a large_string type. Since I know that both the source and destination are arrow I should be able to use the arrow-specific types (substrait will consider them user defined types). To simplify things, this request only asks for support for non-parameterized types. Arrow-specific parameterized types (e.g. decimal256, large_string, etc.) can come in a future request. ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
