Michael Ho has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/9282 )

Change subject: IMPALA-6116: Bound memory usage of DataStreamSevice's service 
queue
......................................................................


Patch Set 2:

(3 comments)

http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/exec-env.cc
File be/src/runtime/exec-env.cc:

http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/exec-env.cc@304
PS2, Line 304:     RETURN_IF_ERROR(data_svc_->Init());
> now that we don't pass parameters, if there are still dependencies in the o
Done


http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/krpc-data-stream-mgr.cc
File be/src/runtime/krpc-data-stream-mgr.cc:

http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/runtime/krpc-data-stream-mgr.cc@176
PS2, Line 176:   service_mem_tracker_->Release(transfer_size);
> what is the impact of the transient double counting now?
There may be a temporary window in which the process mem limit is exceeded, 
leading to other callers of MemTracker::TryConsume() to fail. Worst case, we 
may be double counting by # service threads * avg row batch size.

On the other hand, leaving the window open may lead to a higher chance of going 
over the process mem limit.


http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/service/data-stream-service.h
File be/src/service/data-stream-service.h:

http://gerrit.cloudera.org:8080/#/c/9282/2/be/src/service/data-stream-service.h@45
PS2, Line 45:   /// Initializes the service by registering it with the 
singleton RPC manager.
> note here that the RpcMgr::Init() must have already been called (or whateve
Done



--
To view, visit http://gerrit.cloudera.org:8080/9282
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Idea4262dfb0e0aa8d58ff6ea6a8aaaa248e880b9
Gerrit-Change-Number: 9282
Gerrit-PatchSet: 2
Gerrit-Owner: Michael Ho <k...@cloudera.com>
Gerrit-Reviewer: Dan Hecht <dhe...@cloudera.com>
Gerrit-Reviewer: Lars Volker <l...@cloudera.com>
Gerrit-Reviewer: Michael Ho <k...@cloudera.com>
Gerrit-Reviewer: Tim Armstrong <tarmstr...@cloudera.com>
Gerrit-Comment-Date: Wed, 14 Feb 2018 23:57:28 +0000
Gerrit-HasComments: Yes

Reply via email to