metesynnada opened a new pull request, #5520:
URL: https://github.com/apache/arrow-datafusion/pull/5520
# Which issue does this PR close?
Improves #5130.
# Rationale for this change
`"INSERT INTO abc SELECT * FROM xyz"` support for `MemTable`.
# What changes are included in this PR?
- `insert_into` API for the `TableProvider` trait.
- `insert_into` implementation for the `MemTable`
If we have different partition counts with created plan, we handle it as
```rust
// Get the number of partitions in the plan and the table.
let plan_partition_count = plan.output_partitioning().partition_count();
let table_partition_count = self.batches.read().await.len();
// Adjust the plan as necessary to match the number of partitions in the
table.
let plan: Arc<dyn ExecutionPlan> =
if plan_partition_count == table_partition_count {
plan
} else if table_partition_count == 1 {
// If the table has only one partition, coalesce the partitions in
the plan.
Arc::new(CoalescePartitionsExec::new(plan))
} else {
// Otherwise, repartition the plan using a round-robin partitioning
scheme.
Arc::new(RepartitionExec::try_new(
plan,
Partitioning::RoundRobinBatch(table_partition_count),
)?)
};
```
# Are these changes tested?
Yes
# Are there any user-facing changes?
Query is supported.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
