alamb commented on PR #8667: URL: https://github.com/apache/arrow-datafusion/pull/8667#issuecomment-1881672553
> The above is based on my understanding having studied a bit how we handle external `TableProviders`. @alamb, does the above sound reasonable to you? Yes. > My main uncertainty to the above is how we would handle serialization. Perhaps the `FileType` trait could have a method which returns a `Vec<u8>` based serialization of itself. I'm not sure how that would work on the receiving end exactly... I think the way serialization is handled today is via "ExtensionCodec" * https://docs.rs/datafusion-proto/latest/datafusion_proto/physical_plan/trait.PhysicalExtensionCodec.html * https://docs.rs/datafusion-proto/latest/datafusion_proto/logical_plan/trait.LogicalExtensionCodec.html Which do exactly as you suggest and serialize table providers to/from bytes maybe @thinkharderdev has some more thoughts to share -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
