thinkharderdev commented on pull request #1677:
URL:
https://github.com/apache/arrow-datafusion/pull/1677#issuecomment-1032496217
@houqp Thanks! Removed the commented code.
I like the `ExtensionCodec` approach as well but I think it would work with
the `AsLogicalPlan`/`AsExecutionPlan` since it gives us a hook to "inject" the
extension codec. So we would end up with something that looks like:
```rust
pub trait AsLogicalPlan: Debug + Send + Sync + Clone {
fn try_decode(buf: &[u8]) -> Result<Self, BallistaError>
where
Self: Sized;
fn try_encode<B>(&self, buf: &mut B) -> Result<(), BallistaError>
where
B: BufMut,
Self: Sized;
fn try_into_logical_plan(
&self,
ctx: &ExecutionContext,
extension_codec: &dyn LogicalExtensionCodec,
) -> Result<LogicalPlan, BallistaError>;
fn try_from_logical_plan(
plan: &LogicalPlan,
extension_codec: &dyn LogicalExtensionCodec,
) -> Result<Self, BallistaError>
where
Self: Sized;
}
pub trait LogicalExtensionCodec: Debug + Send + Sync {
fn try_decode(
&self,
buf: &[u8],
inputs: &Vec<LogicalPlan>,
) -> Result<Extension, BallistaError>;
fn try_encode(&self, node: Extension, buf: &mut Vec<u8>)
-> Result<(), BallistaError>;
}
```
where we extend the protobuf encoding of `LogicalPlanNode` with a new enum
```
message ExtensionPlanNode {
bytes plan = 1;
repeated LogicalPlanNode inputs = 2;
}
```
I think this will be useful as well if we ultimately want to migrate to
Substrait for a serializable representation of the plans. We can experiment
without having to do a "hard cutover".
I can also work on integrating the `ExtensionCodec`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
