alamb commented on code in PR #9260: URL: https://github.com/apache/arrow-datafusion/pull/9260#discussion_r1495343688
########## datafusion/substrait/src/lib.rs: ########## @@ -15,6 +15,61 @@ // specific language governing permissions and limitations // under the License. +//! Serialize / Deserialize DataFusion Plans to [Substrait.io] +//! +//! This crate provides support for serializing and deserializing DataFusion plans +//! to and from the generated types in [substrait::proto] from the [substrait] crate. +//! +//! [Substrait.io] provides a cross-language serialization format for relational +//! algebra (e.g. query plans and expressions), based on protocol buffers. +//! +//! [Substrait.io]: https://substrait.io/ +//! +//! [`LogicalPlan`]: datafusion::logical_expr::LogicalPlan +//! [`ExecutionPlan`]: datafusion::physical_plan::ExecutionPlan +//! +//! Potential uses of this crate: +//! * Use DataFusion run Substrait plans created by other systems (e.g. Apache Calcite) +//! * Use DataFusion to create plans to run on other systems +//! * Pass query plans over FFI boundaries, such as from Python to Rust Review Comment: > I'm considering combining other consumer/producer together so the "node" need not be the same at that time. That makes a lot of sense > It turns out the work is tremendous and the related ecosystem is not ready yet 🤣 Yeah, that is my feeling too. One reason I felt this PR would be good is that it might help find / attract people who want to play with the bleeding edge and make DataFusion's substrait support better. We'll see 🎣 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
