tustvold commented on issue #6339: URL: https://github.com/apache/arrow-datafusion/issues/6339#issuecomment-1545962773
> I think it would be a common pattern I'd wager almost 100% of workloads would want atomicity at the `ExecutionPlan` level > Are you thinking of "partition by value in a column (like the date)" Partitioning or bucketing by value would be the most common use-case, which is distinct from the sort of partitioning currently implemented by DataFusion. > It makes it easier to implement DataSink as you don't have to worry about the details of ExecutionPlans and connecting things up Unless I'm missing something, its the difference between calling [ExecutionPlan::execute](https://docs.rs/datafusion/latest/datafusion/physical_plan/trait.ExecutionPlan.html#tymethod.execute) and being given the result? Is that really a meaningful complexity? I guess I'm just trying to play devils advocate for Keep-It-Simple :smile: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
