jkosh44 commented on issue #16285: URL: https://github.com/apache/datafusion/issues/16285#issuecomment-2952715421
The arrow C++ substrait library also doesn't support Durations, but they have a comment about using UDTs to support them: https://github.com/apache/arrow/blob/1d169cc90f65be6ee0bb1fcc03b4f22cf99be897/cpp/src/arrow/engine/substrait/type_internal.cc#L444-L445. If you follow a couple of GH links it brings you here: https://github.com/apache/arrow/issues/40695. The specific use case is where arrow is both the source and destination, so it's OK to roundtrip the type through an opaque UDT. However, one of Datafusion's stated goals is "Use DataFusion to create plans to run on other systems" (https://docs.rs/datafusion-substrait/latest/datafusion_substrait/) which wouldn't work that well with UDT unless the other system understood the UDT we created. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
