rtpsw commented on PR #35005: URL: https://github.com/apache/arrow/pull/35005#issuecomment-1512695209
> But what changed to make this possible? I'm not qualified to answer for Substrait, but here are some point to consider: 1. The [Substrait spec for column selection](https://substrait.io/tutorial/sql_to_substrait/#column-selection-and-emit) speaks about emit output indices (see also [this post](https://github.com/apache/arrow/issues/35004#issue-1660861440)). 2. The [Substrait spec for joining](https://substrait.io/relations/logical_relations/#join-types) does not describe anything about as-of join. 3. The [Substrait algebra proto](https://github.com/substrait-io/substrait/blob/b5d7ed26a17c0a0bd6d0779d312942e5216ea9fa/proto/substrait/algebra.proto) (also [here](https://github.com/substrait-io/substrait/blob/b5d7ed26a17c0a0bd6d0779d312942e5216ea9fa/proto/substrait/algebra.proto)) does not define an as-of-join relation. 4. The [as-of-join relation is defined as an extension](https://github.com/apache/arrow/blob/a2548a79a191c280966d869c7778643fd64fdcd6/cpp/proto/substrait/extension_rels.proto#L29-L46) in Arrow. This suggests to me that we are allowed to change the output schema for as-of-join within Arrow, and we need to adhere to the emit output indices defined by Substrait. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
