yjshen edited a comment on pull request #811: URL: https://github.com/apache/arrow-datafusion/pull/811#issuecomment-904677639
@alamb I might be wrong on this: is it possible to not provide a `RemoteParquetTable`, but provide a `RemoteParquetTableBuilder` that uses the `ObjectStore` API on the async listing but build a `ParquetTable` asynchronously? By doing this, we may pass async table building logic from planning API to users' hands, during they construct `ParquetTable` TableProvider. Then they could register ParquetTable using `context::register_table(self, table_ref, provider)`. Does this volatiles the idiomatic async in Rust? I think of this from the perspective of ballista, even though I'm not quite familiar with the code there, it seems ballista could only [serialize/deserialize](https://github.com/apache/arrow-datafusion/blob/master/ballista/rust/core/src/serde/logical_plan/to_proto.rs#L708-L742) known typed TableProviders, therefore `RemoteParquetTable` outside DataFusion might not be preferable? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
