mingmwang commented on PR #5837:
URL:
https://github.com/apache/arrow-datafusion/pull/5837#issuecomment-1494370086
> As far as I can see, this only works for single partitions as input and
not repartitioning in between (e.g. no concurrency), could you confirm?
No always. We will see the adjacent Partial + Final Aggregator for normal
join and aggregation on the same key.
I will add more UTs and intg tests tomorrow to show the cases:
```rust
select distinct(t1.t1_id) from t1 inner join t2 on t1.t1_id = t2.t2_id;
```
```sql
AggregateExec: mode=Single, gby=[t1_id@0 as t1_id], aggr=[]",
ProjectionExec: expr=[t1_id@0 as t1_id]",
CoalesceBatchesExec: target_batch_size=4096",
HashJoinExec: mode=Partitioned, join_type=Inner, on=[(Column {
name: \"t1_id\", index: 0 }, Column { name: \"t2_id\", index: 0 })]",
CoalesceBatchesExec: target_batch_size=4096",
RepartitionExec: partitioning=Hash([Column { name: \"t1_id\",
index: 0 }], 2), input_partitions=2",
RepartitionExec: partitioning=RoundRobinBatch(2),
input_partitions=1",
MemoryExec: partitions=1, partition_sizes=[1]",
CoalesceBatchesExec: target_batch_size=4096",
RepartitionExec: partitioning=Hash([Column { name: \"t2_id\",
index: 0 }], 2), input_partitions=2",
RepartitionExec: partitioning=RoundRobinBatch(2),
input_partitions=1",
MemoryExec: partitions=1, partition_sizes=[1]",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
