alexanderbianchi commented on PR #27210:
URL: https://github.com/apache/datafusion/pull/27210#issuecomment-6968942353
I think you've identified all the core design decisions to make here in that
last message. IMO I agree that versioning seems like an abstract enough concept
that it can be natively supported in DataFusion, so long as the same mechanism
can be used.
If you think of Datafusion's UDF's - they all implement the same API's that
user defined functions do. The code I think is close Andrew's suggestion of
using an extension. There needs to be an extension point that simplifies
TableProvider equality, passing information to SchemaProvider, and allows async
metadata resolution like. Could versioning just be an implementation of
`TableReadSelector`
```rust
pub enum TableReadSelector {
AsOfVersion(ScalarValue),
AsOfTimestamp(i64),
Extension(Arc<dyn TableReadSelectorExtension>),
}
```
Effectively that means removing the Enum and making `AsOfVersion` /
`AsOfTimestamp` an implementation of `TableReadSelectorExtension`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]