Internal Jenkins has submitted this change and it was merged. Change subject: IMPALA-2076: Correct execution time tracking for DataStreamSender. ......................................................................
IMPALA-2076: Correct execution time tracking for DataStreamSender. DataStreamSender uses multiple channels (each channel uses one thread) to send data. We want to know the active wall-clock network time used by DataStreamSender no matter how many threads are used so we only count concurrent sending time and row batch serialization time. We don't want to count recv_TransmitData() time because it is mainly to wait for receiver to deserialize data and process data, not using much network, plus those part are already tracked by Exchange node. Added a ConcurrentStopWatch class to get concurrent transmit time. If a thread is already running, the following thread won't reset the stop watch. And the stop watch is stopped only after all threads finish their work. Change-Id: I9c530b2056d852c8bcac6263e9e6b1a6bede1047 Reviewed-on: http://gerrit.cloudera.org:8080/2578 Reviewed-by: Juan Yu <[email protected]> Reviewed-by: Dan Hecht <[email protected]> Tested-by: Internal Jenkins --- A be/src/runtime/backend-client.h M be/src/runtime/client-cache.h M be/src/runtime/coordinator.cc M be/src/runtime/coordinator.h M be/src/runtime/data-stream-sender.cc M be/src/runtime/data-stream-sender.h M be/src/runtime/data-stream-test.cc M be/src/runtime/exec-env.cc M be/src/runtime/exec-env.h M be/src/runtime/runtime-filter.cc M be/src/runtime/runtime-state.h M be/src/service/fragment-exec-state.cc M be/src/service/fragment-exec-state.h M be/src/util/runtime-profile-test.cc M be/src/util/runtime-profile.cc M be/src/util/runtime-profile.h M be/src/util/stopwatch.h 17 files changed, 371 insertions(+), 44 deletions(-) Approvals: Juan Yu: Looks good to me, but someone else must approve Internal Jenkins: Verified Dan Hecht: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/2578 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-MessageType: merged Gerrit-Change-Id: I9c530b2056d852c8bcac6263e9e6b1a6bede1047 Gerrit-PatchSet: 9 Gerrit-Project: Impala Gerrit-Branch: cdh5-trunk Gerrit-Owner: Juan Yu <[email protected]> Gerrit-Reviewer: Dan Hecht <[email protected]> Gerrit-Reviewer: Henry Robinson <[email protected]> Gerrit-Reviewer: Internal Jenkins Gerrit-Reviewer: Juan Yu <[email protected]> Gerrit-Reviewer: Sailesh Mukil <[email protected]>
