tustvold commented on issue #6339:
URL:
https://github.com/apache/arrow-datafusion/issues/6339#issuecomment-1545947024
> the atomicity of a write transaction can only be guaranteed on a partition
basis
I think you could do something where the last write_stream to complete
flushes, but it is a bit odd for sure
This does lead to one question though, what actually is the meaning of
partitioning in this context? The formulation of `Distribution` I don't believe
is sufficient to express the partitioning that something like an object_store
would require.
What does exposing the partitioning yield over something simple like
```
impl TableProvider {
async fn insert(&self, plan: Arc<dyn ExecutionPlan>) -> Result<()>;
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
