Impala Public Jenkins has submitted this change and it was merged. (
http://gerrit.cloudera.org:8080/12000 )
Change subject: IMPALA-6741: Add timestamp of fragment instance's status updates
......................................................................
IMPALA-6741: Add timestamp of fragment instance's status updates
Currently, the profile of a running query doesn't contain any
timestamps for the last updates from the fragment instances.
This makes it hard to differentiate between when a fragment
instance failed to send status reports to the coordinator
for various reasons (e.g. IMPALA-2990) or a truly stuck
fragment instance.
This change adds a timestamp to a fragment instance's profile
to record the time when the coordinator last received a status
update from it. Note that it's possible that there is delay
between when the status was created on the executor and when
it arrived at the coordinator. Given that the clocks are not
necessarily synchronized across all executors, the receiving
time of the update at the coordinator seems easier to make sense of.
Sample output:
Fragment F01:
Instance 494d948d3235441a:23eae17900000001 (host=???):(Total: 15.099ms,
non-child: 263.951us, % non-child: 1.75%)
Last report received time: 2018-11-27 16:57:30.014
Hdfs split stats (<volume id>:<# splits>/<split lengths>): 0:1/1.58 KB
Fragment Instance Lifecycle Event Timeline: 15.622ms
- Prepare Finished: 1.026ms (1.026ms)
- Open Finished: 1.137ms (110.297us)
- First Batch Produced: 15.010ms (13.873ms)
- First Batch Sent: 15.080ms (70.715us)
- ExecInternal Finished: 15.622ms (541.181us)
Change-Id: Iae3dcddc292d694d7003d10ed0caccfceed7d8fa
Reviewed-on: http://gerrit.cloudera.org:8080/12000
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
---
M be/src/runtime/coordinator-backend-state.cc
M be/src/runtime/coordinator-backend-state.h
M be/src/util/runtime-profile.cc
M be/src/util/runtime-profile.h
M tests/query_test/test_observability.py
5 files changed, 66 insertions(+), 11 deletions(-)
Approvals:
Impala Public Jenkins: Looks good to me, approved; Verified
--
To view, visit http://gerrit.cloudera.org:8080/12000
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: Iae3dcddc292d694d7003d10ed0caccfceed7d8fa
Gerrit-Change-Number: 12000
Gerrit-PatchSet: 8
Gerrit-Owner: Michael Ho <[email protected]>
Gerrit-Reviewer: Balazs Jeszenszky <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Lars Volker <[email protected]>
Gerrit-Reviewer: Michael Ho <[email protected]>
Gerrit-Reviewer: Zoram Thanga <[email protected]>
