Lars Volker has uploaded this change for review. ( http://gerrit.cloudera.org:8080/8914
Change subject: IMPALA-6193: Track memory of incoming data streams ...................................................................... IMPALA-6193: Track memory of incoming data streams This changes adds memory tracking to incoming transmit data RPCs when using KRPC. We track memory inside the service pools until it is handed over to the stream manager. There we track it in a global tracker until a receiver registers and takes over the early sender RPCs. Inside the receiver memory for deferred RPCs is tracked against the instance tracker until we unpack the batches and add them to the row batch queue. I tested this change manually by running TPCH and TPCDS on my local dev machine and on a private cluster. TODO: more testing Change-Id: I2df1204d2483313a8a18e5e3be6cec9e402614c4 --- M be/src/rpc/impala-service-pool.cc M be/src/rpc/impala-service-pool.h M be/src/rpc/rpc-mgr.cc M be/src/rpc/rpc-mgr.h M be/src/runtime/exec-env.cc M be/src/runtime/krpc-data-stream-mgr.cc M be/src/runtime/krpc-data-stream-mgr.h A be/src/runtime/krpc-data-stream-mgr.inline.h M be/src/runtime/krpc-data-stream-recvr.cc M be/src/runtime/krpc-data-stream-recvr.h M be/src/util/memory-metrics.h M common/protobuf/data_stream_service.proto 12 files changed, 289 insertions(+), 156 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/8914/1 -- To view, visit http://gerrit.cloudera.org:8080/8914 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I2df1204d2483313a8a18e5e3be6cec9e402614c4 Gerrit-Change-Number: 8914 Gerrit-PatchSet: 1 Gerrit-Owner: Lars Volker <[email protected]>
