Michael Ho has posted comments on this change. ( http://gerrit.cloudera.org:8080/9282 )
Change subject: IMPALA-6116: Bound memory usage of DataStreamSevice's service queue ...................................................................... Patch Set 2: (3 comments) http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/exec-env.cc File be/src/runtime/exec-env.cc: http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/exec-env.cc@304 PS2, Line 304: RETURN_IF_ERROR(data_svc_->Init()); > now that we don't pass parameters, if there are still dependencies in the o Done http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/krpc-data-stream-mgr.cc File be/src/runtime/krpc-data-stream-mgr.cc: http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/krpc-data-stream-mgr.cc@176 PS2, Line 176: service_mem_tracker_->Release(transfer_size); > what is the impact of the transient double counting now? There may be a temporary window in which the process mem limit is exceeded, leading to other callers of MemTracker::TryConsume() to fail. Worst case, we may be double counting by # service threads * avg row batch size. On the other hand, leaving the window open may lead to a higher chance of going over the process mem limit. http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/service/data-stream-service.h File be/src/service/data-stream-service.h: http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/service/data-stream-service.h@45 PS2, Line 45: /// Initializes the service by registering it with the singleton RPC manager. > note here that the RpcMgr::Init() must have already been called (or whateve Done -- To view, visit http://gerrit.cloudera.org:8080/9282 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Idea4262dfb0e0aa8d58ff6ea6a8aaaa248e880b9 Gerrit-Change-Number: 9282 Gerrit-PatchSet: 2 Gerrit-Owner: Michael Ho <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Lars Volker <[email protected]> Gerrit-Reviewer: Michael Ho <[email protected]> Gerrit-Reviewer: Tim Armstrong <[email protected]> Gerrit-Comment-Date: Wed, 14 Feb 2018 23:57:28 +0000 Gerrit-HasComments: Yes
