Michael Ho has uploaded this change for review. ( http://gerrit.cloudera.org:8080/9202
Change subject: IMPALA-6369: Exchange node's memory usage should include its receiver's ...................................................................... IMPALA-6369: Exchange node's memory usage should include its receiver's A DataStreamRecvr is co-owned by the DataStreamMgr and an Exchange node. However, the life time of the memory allocations (e.g. row batches) of a DataStreamRecvr never exceeds that of its owning Exchange node. Previously, we used the fragment instance's MemTracker as the parent of the DataStreamRecvr's MemTracker. This change switches to using the MemTracker of the owning Exchange node as the parent tracker of the DataStreamRecvr. This makes it easier to identify the peak memory usage of the receivers of different exchange nodes in the runtime profile and query summary. Sample output from TPCH-Q21: EXCHANGE_NODE (id=18):(Total: 1s448ms, non-child: 265.818ms, % non-child: 18.35%) - ConvertRowBatchTime: 223.895ms - PeakMemoryUsage: 10.04 MB (10524943) - RowsReturned: 1.27M (1267464) - RowsReturnedRate: 875.19 K/sec RecvrSide: BytesReceived(500.000ms): 0, 1.64 MB, 9.98 MB, 9.98 MB, 10.01 MB, 10.01 MB, 10.01 MB, 31.79 MB, 60.19 MB, 87.84 MB - FirstBatchArrivalWaitTime: 0.000ns - TotalBytesReceived: 93.07 MB (97594728) - TotalGetBatchTime: 1s194ms - DataArrivalTimer: 1s183ms SenderSide: - DeserializeRowBatchTime: 344.343ms - NumBatchesAccepted: 3.80K (3796) - NumBatchesDeferred: 5 (5) - NumEarlySenders: 0 (0) Change-Id: I8ca3c47d87bfcd221d34565eda1878f3c15d5c45 --- M be/src/exec/exchange-node.cc M be/src/runtime/data-stream-mgr-base.h M be/src/runtime/data-stream-mgr.cc M be/src/runtime/data-stream-mgr.h M be/src/runtime/data-stream-test.cc M be/src/runtime/krpc-data-stream-mgr.cc M be/src/runtime/krpc-data-stream-mgr.h M tests/query_test/test_observability.py 8 files changed, 35 insertions(+), 36 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/02/9202/1 -- To view, visit http://gerrit.cloudera.org:8080/9202 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I8ca3c47d87bfcd221d34565eda1878f3c15d5c45 Gerrit-Change-Number: 9202 Gerrit-PatchSet: 1 Gerrit-Owner: Michael Ho <k...@cloudera.com>