Copilot commented on code in PR #18747:
URL: https://github.com/apache/iotdb/pull/18747#discussion_r4129094706


##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/exchange/source/SourceHandle.java:
##########
@@ -305,7 +309,10 @@ private synchronized void trySubmitGetDataBlocksTask() {
     long reservedBytes = 0L;
     Pair<ListenableFuture<Void>, Boolean> pair = null;
     long blockedSize = 0L;
-    while (sequenceIdToDataBlockSize.containsKey(endSequenceId)) {
+    // Keep one in-flight block per source handle when it participates in a 
shared Collect quota.
+    // Standalone exchanges retain their existing batching behavior.
+    while (sequenceIdToDataBlockSize.containsKey(endSequenceId)
+        && (!usesSharedMemoryReservationGroup || endSequenceId == 
startSequenceId)) {

Review Comment:
   The one-block guard resets `startSequenceId` to `nextSequenceId` on every 
invocation, so every new `TNewDataBlockEvent` can reserve and fetch another 
block before the previous one is consumed. `SinkChannel.send()` emits one such 
event per block, allowing a single remote child to accumulate many in-flight 
blocks and monopolize the shared Collect quota. Also require `nextSequenceId == 
currSequenceId` before admitting the grouped handle's one block.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/planner/LocalExecutionPlanContext.java:
##########
@@ -261,16 +270,50 @@ public void addExchangeOperator(ExchangeOperator 
exchangeOperator) {
     this.exchangeOperatorList.add(exchangeOperator);
   }
 
+  public CollectQuotaGroup enterCollectQuotaGroup(String planNodeId) {
+    CollectQuotaGroup group = new CollectQuotaGroup("collect-fan-in-" + 
planNodeId);
+    collectQuotaGroups.add(group);
+    activeCollectQuotaGroups.push(group);
+    return group;
+  }
+
+  public void exitCollectQuotaGroup(CollectQuotaGroup group) {
+    if (activeCollectQuotaGroups.peek() != group) {
+      throw new IllegalStateException();
+    }
+    activeCollectQuotaGroups.pop();
+  }
+
+  public void registerExchangeSourceHandle(ISourceHandle sourceHandle) {
+    if (!activeCollectQuotaGroups.isEmpty()) {
+      activeCollectQuotaGroups.peek().addSourceHandle(sourceHandle);
+    }
+  }
+
   public void setMaxBytesOneHandleCanReserve() {
     long maxBytesOneHandleCanReserve = getMaxBytesOneHandleCanReserve();
     LOGGER.debug(
         DataNodeQueryMessages
             
.MAXBYTESONEHANDLECANRESERVE_FOR_EXCHANGEOPERATOR_IS_ARG_EXCHANGESUMNUM_IS_ARG,
         maxBytesOneHandleCanReserve,
         exchangeSumNum);
+    java.util.Set<ISourceHandle> groupedSourceHandles =
+        Collections.newSetFromMap(new IdentityHashMap<>());
+    long maxBytesPerFI =
+        
IoTDBDescriptor.getInstance().getMemoryConfig().getMaxBytesPerFragmentInstance();
+    for (CollectQuotaGroup group : collectQuotaGroups) {
+      for (ISourceHandle sourceHandle : group.getSourceHandles()) {
+        groupedSourceHandles.add(sourceHandle);
+        
sourceHandle.setMemoryReservationGroupId(group.getReservationPlanNodeId());
+        sourceHandle.setMaxBytesCanReserve(maxBytesPerFI);

Review Comment:
   Each Collect group gets a distinct reservation key and the full 
`maxBytesPerFragmentInstance`. `MemoryPool` enforces the supplied limit per 
plan-node key, not across all keys, so a fragment with two Collect groups (or 
one group plus ungrouped exchanges) can reserve more than its fragment-instance 
data-exchange budget. The full budget must be partitioned across reservation 
scopes, or all scopes must additionally share an FI-wide cap.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/plan/relational/planner/distribute/TableDistributedPlanGenerator.java:
##########
@@ -2956,6 +2956,12 @@ private static OrderingScheme 
constructOrderingSchema(List<Symbol> symbols) {
     return new OrderingScheme(symbols, orderings);
   }
 
+  private static boolean isStreamingAggregationEnabled(AggregationNode node) {
+    // Keep an explicit ordering boundary for plans whose logical grouping 
property is streamable.
+    // The local operator generator still chooses blocking HashAggregation for 
these inputs.
+    return node.isStreamable();

Review Comment:
   This helper has no callers, so it does not implement the stated 
blocking-aggregation boundary. Physical final aggregations produced by 
`split()` retain `preGroupedSymbols`, and `TableOperatorGenerator` therefore 
still creates a streaming aggregation above a Collect. Since Collect now 
interleaves ready children, equal groups from one child can be separated by 
another child and emitted as duplicate groups. Clear the physical aggregation's 
pre-grouped symbols when its input may be Collect, while preserving ordering 
metadata separately where needed.



##########
iotdb-core/datanode/src/main/java/org/apache/iotdb/db/queryengine/execution/exchange/source/SourceHandle.java:
##########
@@ -677,6 +701,7 @@ public void run() {
                     indexOfUpstreamSinkHandle);
               }
               if (tsBlockNum == 0) {
+                fail(new IllegalStateException());

Review Comment:
   An empty response now fails the fragment with an `IllegalStateException` 
whose message is null, hiding the actual exchange failure from diagnostics. Use 
the existing localized unexpected-response-size message, as the non-empty 
mismatch path already does.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to