[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 8: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 8 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 07 Apr 2023 04:31:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. IMPALA-11970: Add query timeline display to Impala WebUI This change adds a chart to the Impala WebUI for displaying query fragments and plan node timings. The chart list execution timings in a compact form with the plan/fragment tree shown on the left side and a Gantt chart with bars for each execution phase on the right side. Plan node labels are color-coded by fragment. The node list will scroll for large plans so that the scale and legend always stay visible. Exchange nodes have dotted boxes that show when rows are sent from the corresponding senders below. Additionally, 'X' and 'O' symbols are used to node where join builds arrive. By default, fragments are printed with nodes grouped within each fragment for easier timing analysis. A checkbox is provided that changes the ordering so that nodes are printed in plan order similar to the Summary tab. The length of each colored phase interval depicts the longest instance time while instances that complete earlier are shown as grey lines within the phase interval. On the server side, the query profile handler has been modified to return the profile JSON as a document element instead of document content as it is for JSON profile download. This allows the profile to be accessed in full using scripts in templates. Dense profiles will not render correctly as they currently do not contain the necessary timestamps. This limitation will be addressed at a later time with corresponding changes to the dense profile. Example output is attached to the IMPALA-11970 ticket Testing: Manual validation of rendering using TPC-H and TPC-DS queries Will look into adding tests to test_web_pages.py and test_observability.py in a subsequent patch Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Reviewed-on: http://gerrit.cloudera.org:8080/19583 Tested-by: Impala Public Jenkins Reviewed-by: Wenzhe Zhou --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.cc M tests/webserver/test_web_pages.py M www/query_detail_tabs.tmpl A www/query_timeline.tmpl 7 files changed, 526 insertions(+), 9 deletions(-) Approvals: Impala Public Jenkins: Verified Wenzhe Zhou: Looks good to me, approved -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 9 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 8: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 8 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 07 Apr 2023 04:10:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 8: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 8 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 06 Apr 2023 22:51:46 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 8: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9210/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 8 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 06 Apr 2023 22:53:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12761/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 8 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 06 Apr 2023 19:52:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Kurt Deschler has posted comments on this change. (
http://gerrit.cloudera.org:8080/19583 )
Change subject: IMPALA-11970: Add query timeline display to Impala WebUI
..
Patch Set 7:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/service/impala-http-handler.cc@348
PS7, Line 348: if (!status.ok()) {
: Value error(ss.str().c_str(), document->GetAllocator());
: document->AddMember("error", error,
document->GetAllocator());
: return;
: }
> This changes the behavior not to return profile if WebRequest does not have
Done
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/util/runtime-profile.cc
File be/src/util/runtime-profile.cc:
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/util/runtime-profile.cc@1230
PS7, Line 1230: content
> contents
Done
--
To view, visit http://gerrit.cloudera.org:8080/19583
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db
Gerrit-Change-Number: 19583
Gerrit-PatchSet: 7
Gerrit-Owner: Kurt Deschler
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: David Rorke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Riza Suminto
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Thu, 06 Apr 2023 19:33:44 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Hello Quanlong Huang, Riza Suminto, Abhishek Rawat, David Rorke, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19583 to look at the new patch set (#8). Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. IMPALA-11970: Add query timeline display to Impala WebUI This change adds a chart to the Impala WebUI for displaying query fragments and plan node timings. The chart list execution timings in a compact form with the plan/fragment tree shown on the left side and a Gantt chart with bars for each execution phase on the right side. Plan node labels are color-coded by fragment. The node list will scroll for large plans so that the scale and legend always stay visible. Exchange nodes have dotted boxes that show when rows are sent from the corresponding senders below. Additionally, 'X' and 'O' symbols are used to node where join builds arrive. By default, fragments are printed with nodes grouped within each fragment for easier timing analysis. A checkbox is provided that changes the ordering so that nodes are printed in plan order similar to the Summary tab. The length of each colored phase interval depicts the longest instance time while instances that complete earlier are shown as grey lines within the phase interval. On the server side, the query profile handler has been modified to return the profile JSON as a document element instead of document content as it is for JSON profile download. This allows the profile to be accessed in full using scripts in templates. Dense profiles will not render correctly as they currently do not contain the necessary timestamps. This limitation will be addressed at a later time with corresponding changes to the dense profile. Example output is attached to the IMPALA-11970 ticket Testing: Manual validation of rendering using TPC-H and TPC-DS queries Will look into adding tests to test_web_pages.py and test_observability.py in a subsequent patch Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.cc M tests/webserver/test_web_pages.py M www/query_detail_tabs.tmpl A www/query_timeline.tmpl 7 files changed, 526 insertions(+), 9 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19583/8 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 8 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. (
http://gerrit.cloudera.org:8080/19583 )
Change subject: IMPALA-11970: Add query timeline display to Impala WebUI
..
Patch Set 7:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/service/impala-http-handler.cc@348
PS7, Line 348: if (!status.ok()) {
: Value error(ss.str().c_str(), document->GetAllocator());
: document->AddMember("error", error,
document->GetAllocator());
: return;
: }
This changes the behavior not to return profile if WebRequest does not have
query_id. It breaks test case
test_web_pages.py::TestWebPage::test_query_profile_encoded_unknown_query_id
since the test try to get profile without query_id.
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/util/runtime-profile.cc
File be/src/util/runtime-profile.cc:
http://gerrit.cloudera.org:8080/#/c/19583/7/be/src/util/runtime-profile.cc@1230
PS7, Line 1230: content
contents
--
To view, visit http://gerrit.cloudera.org:8080/19583
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db
Gerrit-Change-Number: 19583
Gerrit-PatchSet: 7
Gerrit-Owner: Kurt Deschler
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: David Rorke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Riza Suminto
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Sat, 01 Apr 2023 07:44:05 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 7: -Code-Review There are some profile related test failures: hs2.test_hs2.TestHS2.test_get_profile observability.test_profile_tool.TestProfileTool.test_json_output webserver.test_web_pages.TestWebPage.test_query_profile_encoded_unknown_query_id webserver.test_web_pages.TestWebPage.test_download_profile -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 7 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Sat, 01 Apr 2023 01:29:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 7: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/9190/ -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 7 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Sat, 01 Apr 2023 01:16:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12729/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 7 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 20:07:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 7: Code-Review+1 I can give +2 if no more comments from other reviewers. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 7 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 19:59:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 7: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/9190/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 7 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 19:59:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Kurt Deschler has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 6: (10 comments) http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl File www/query_timeline.tmpl: http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@183 PS6, Line 183: en++ > nit: ++en to keep consistent with other for loop Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@214 PS6, Line 214: + > nit: space around two '+' in this line Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@230 PS6, Line 230: - > nit: space around '-' Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@232 PS6, Line 232: - > nit: space around '-' Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@241 PS6, Line 241: en++ > nit: ++en to keep consistent with other for loop Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@300 PS6, Line 300: + > nit: space around two '+' in this line Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@306 PS6, Line 306: en++ > nit: ++en to keep consistent with other for loop Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@324 PS6, Line 324: + > nit: space around two '+' in this line Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@341 PS6, Line 341: / > nit: space around '/' Done http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@418 PS6, Line 418: height/2+3 > nit: space around '/' and '+' Done -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 6 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 19:46:54 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Hello Quanlong Huang, Riza Suminto, Abhishek Rawat, David Rorke, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19583 to look at the new patch set (#7). Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. IMPALA-11970: Add query timeline display to Impala WebUI This change adds a chart to the Impala WebUI for displaying query fragments and plan node timings. The chart list execution timings in a compact form with the plan/fragment tree shown on the left side and a Gantt chart with bars for each execution phase on the right side. Plan node labels are color-coded by fragment. The node list will scroll for large plans so that the scale and legend always stay visible. Exchange nodes have dotted boxes that show when rows are sent from the corresponding senders below. Additionally, 'X' and 'O' symbols are used to node where join builds arrive. By default, fragments are printed with nodes grouped within each fragment for easier timing analysis. A checkbox is provided that changes the ordering so that nodes are printed in plan order similar to the Summary tab. The length of each colored phase interval depicts the longest instance time while instances that complete earlier are shown as grey lines within the phase interval. On the server side, the query profile handler has been modified to return the profile JSON as a document element instead of document content as it is for JSON profile download. This allows the profile to be accessed in full using scripts in templates. Dense profiles will not render correctly as they currently do not contain the necessary timestamps. This limitation will be addressed at a later time with corresponding changes to the dense profile. Example output is attached to the IMPALA-11970 ticket Testing: Manual validation of rendering using TPC-H and TPC-DS queries Will look into adding tests to test_web_pages.py and test_observability.py in a subsequent patch Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.cc M www/query_detail_tabs.tmpl A www/query_timeline.tmpl 6 files changed, 525 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19583/7 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 7 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 6: (11 comments) looks good to me, just a few more nits http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl File www/query_timeline.tmpl: http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@438 PS5, Line 438: pending_fragments.push > Just looked and unshift is O(n) per call. push is O(1) and reverse is O(n). You are right http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl File www/query_timeline.tmpl: http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@183 PS6, Line 183: en++ nit: ++en to keep consistent with other for loop http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@214 PS6, Line 214: + nit: space around two '+' in this line http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@230 PS6, Line 230: - nit: space around '-' http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@232 PS6, Line 232: - nit: space around '-' http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@241 PS6, Line 241: en++ nit: ++en to keep consistent with other for loop http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@300 PS6, Line 300: + nit: space around two '+' in this line http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@306 PS6, Line 306: en++ nit: ++en to keep consistent with other for loop http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@324 PS6, Line 324: + nit: space around two '+' in this line http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@341 PS6, Line 341: / nit: space around '/' http://gerrit.cloudera.org:8080/#/c/19583/6/www/query_timeline.tmpl@418 PS6, Line 418: height/2+3 nit: space around '/' and '+' -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 6 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 16:36:31 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12726/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 6 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Fri, 31 Mar 2023 13:46:57 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Hello Quanlong Huang, Riza Suminto, Abhishek Rawat, David Rorke, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19583 to look at the new patch set (#6). Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. IMPALA-11970: Add query timeline display to Impala WebUI This change adds a chart to the Impala WebUI for displaying query fragments and plan node timings. The chart list execution timings in a compact form with the plan/fragment tree shown on the left side and a Gantt chart with bars for each execution phase on the right side. Plan node labels are color-coded by fragment. The node list will scroll for large plans so that the scale and legend always stay visible. Exchange nodes have dotted boxes that show when rows are sent from the corresponding senders below. Additionally, 'X' and 'O' symbols are used to node where join builds arrive. By default, fragments are printed with nodes grouped within each fragment for easier timing analysis. A checkbox is provided that changes the ordering so that nodes are printed in plan order similar to the Summary tab. The length of each colored phase interval depicts the longest instance time while instances that complete earlier are shown as grey lines within the phase interval. On the server side, the query profile handler has been modified to return the profile JSON as a document element instead of document content as it is for JSON profile download. This allows the profile to be accessed in full using scripts in templates. Dense profiles will not render correctly as they currently do not contain the necessary timestamps. This limitation will be addressed at a later time with corresponding changes to the dense profile. Example output is attached to the IMPALA-11970 ticket Testing: Manual validation of rendering using TPC-H and TPC-DS queries Will look into adding tests to test_web_pages.py and test_observability.py in a subsequent patch Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.cc M www/query_detail_tabs.tmpl A www/query_timeline.tmpl 6 files changed, 524 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19583/6 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 6 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: Abhishek Rawat Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Kurt Deschler has posted comments on this change. (
http://gerrit.cloudera.org:8080/19583 )
Change subject: IMPALA-11970: Add query timeline display to Impala WebUI
..
Patch Set 5:
(13 comments)
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl
File www/query_timeline.tmpl:
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@185
PS5, Line 185: for(var instance = 1; instance <
fp.child_profiles.length; ++instance) {
> nit: space after 'for'
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@199
PS5, Line 199: %
> nit: add space around '%'
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@214
PS5, Line 214: }
> Do we need to set node_name for other kinds of scan node?
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@230
PS5, Line 230: } else if(all_nodes.length) {
> nit: space after 'if'
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@237
PS5, Line 237:
> nit: remove empty line
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@244
PS5, Line 244:
> nit: extra space
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@322
PS5, Line 322: *
> nit: space around '*'
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@341
PS5, Line 341: /
> nit: space around '/' in this line
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@392
PS5, Line 392: if(node.parent_node != undefined) {
> nit: space after 'if'
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@415
PS5, Line 415: /
> nit: space around '/' and '+' i in this line and next three lines
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@429
PS5, Line 429: if(node.num_children) // Scan (leaf) node
> nit: space after 'if'
Done
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@438
PS5, Line 438: pending_fragments.push
> Could we use unshift() to add element to the beginning so that we don't nee
Just looked and unshift is O(n) per call. push is O(1) and reverse is O(n).
Going to leave this as-is.
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@451
PS5, Line 451: +
> nit: space around '+'
Done
--
To view, visit http://gerrit.cloudera.org:8080/19583
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db
Gerrit-Change-Number: 19583
Gerrit-PatchSet: 5
Gerrit-Owner: Kurt Deschler
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: David Rorke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Riza Suminto
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Fri, 31 Mar 2023 13:25:20 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. (
http://gerrit.cloudera.org:8080/19583 )
Change subject: IMPALA-11970: Add query timeline display to Impala WebUI
..
Patch Set 5:
(13 comments)
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl
File www/query_timeline.tmpl:
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@185
PS5, Line 185: for(var instance = 1; instance <
fp.child_profiles.length; ++instance) {
nit: space after 'for'
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@199
PS5, Line 199: %
nit: add space around '%'
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@214
PS5, Line 214: }
Do we need to set node_name for other kinds of scan node?
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@230
PS5, Line 230: } else if(all_nodes.length) {
nit: space after 'if'
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@237
PS5, Line 237:
nit: remove empty line
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@244
PS5, Line 244:
nit: extra space
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@322
PS5, Line 322: *
nit: space around '*'
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@341
PS5, Line 341: /
nit: space around '/' in this line
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@392
PS5, Line 392: if(node.parent_node != undefined) {
nit: space after 'if'
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@415
PS5, Line 415: /
nit: space around '/' and '+' i in this line and next three lines
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@429
PS5, Line 429: if(node.num_children) // Scan (leaf) node
nit: space after 'if'
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@438
PS5, Line 438: pending_fragments.push
Could we use unshift() to add element to the beginning so that we don't need to
call reverse()?
http://gerrit.cloudera.org:8080/#/c/19583/5/www/query_timeline.tmpl@451
PS5, Line 451: +
nit: space around '+'
--
To view, visit http://gerrit.cloudera.org:8080/19583
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db
Gerrit-Change-Number: 19583
Gerrit-PatchSet: 5
Gerrit-Owner: Kurt Deschler
Gerrit-Reviewer: Abhishek Rawat
Gerrit-Reviewer: David Rorke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Riza Suminto
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Fri, 31 Mar 2023 08:12:39 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12681/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 5 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Thu, 23 Mar 2023 14:44:53 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Hello Quanlong Huang, Riza Suminto, David Rorke, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19583 to look at the new patch set (#5). Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. IMPALA-11970: Add query timeline display to Impala WebUI This change adds a chart to the Impala WebUI for displaying query fragments and plan node timings. The chart list execution timings in a compact form with the plan/fragment tree shown on the left side and a Gantt chart with bars for each execution phase on the right side. Plan node labels are color-coded by fragment. The node list will scroll for large plans so that the scale and legend always stay visible. Exchange nodes have dotted boxes that show when rows are sent from the corresponding senders below. Additionally, 'X' and 'O' symbols are used to node where join builds arrive. By default, fragments are printed with nodes grouped within each fragment for easier timing analysis. A checkbox is provided that changes the ordering so that nodes are printed in plan order similar to the Summary tab. The length of each colored phase interval depicts the longest instance time while instances that complete earlier are shown as grey lines within the phase interval. On the server side, the query profile handler has been modified to return the profile JSON as a document element instead of document content as it is for JSON profile download. This allows the profile to be accessed in full using scripts in templates. Dense profiles will not render correctly as they currently do not contain the necessary timestamps. This limitation will be addressed at a later time with corresponding changes to the dense profile. Example output is attached to the IMPALA-11970 ticket Testing: Manual validation of rendering using TPC-H and TPC-DS queries Will look into adding tests to test_web_pages.py and test_observability.py in a subsequent patch Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.cc M www/query_detail_tabs.tmpl A www/query_timeline.tmpl 6 files changed, 524 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19583/5 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 5 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 4: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/12673/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 4 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 22 Mar 2023 23:08:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Hello Quanlong Huang, Riza Suminto, David Rorke, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19583 to look at the new patch set (#4). Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. IMPALA-11970: Add query timeline display to Impala WebUI This change adds a chart to the Impala WebUI for displaying query fragments and plan node timings. The chart list execution timings in a compact form with the plan/fragment tree shown on the left side and a Gantt chart with bars for each execution phase on the right side. Plan node labels are color-coded by fragment. The node list will scroll for large plans so that the scale and legend always stay visible. Exchange nodes have dotted boxes that show when rows are sent from the corresponding senders below. Additionally, 'X' and 'O' symbols are used to node where join builds arrive. By default, fragments are printed with nodes grouped within each fragment for easier timing analysis. A checkbox is provided that changes the ordering so that nodes are printed in plan order similar to the Summary tab. The length of each colored phase interval depicts the longest instance time while instances that complete earlier are shown as grey lines within the phase interval. On the server side, the query profile handler has been modified to return the profile JSON as a document element instead of document content as it is for JSON profile download. This allows the profile to be accessed in full using scripts in templates. Dense profiles will not render correctly as they currently do not contain the necessary timestamps. This limitation will be addressed at a later time with corresponding changes to the dense profile. Example output is attached to the IMPALA-11970 ticket Testing: Manual validation of rendering using TPC-H and TPC-DS queries Will look into adding tests to test_web_pages.py and test_observability.py in a subsequent revision Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.cc M www/query_detail_tabs.tmpl A www/query_timeline.tmpl 6 files changed, 524 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19583/4 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 4 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Riza Suminto has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 3: Code-Review+2 Carry +1 from Wenzhe. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 3 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 08 Mar 2023 05:25:41 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Wenzhe Zhou has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 3: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 3 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 08 Mar 2023 05:18:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/19583 ) Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/12579/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 3 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou Gerrit-Comment-Date: Wed, 08 Mar 2023 03:58:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Kurt Deschler has posted comments on this change. (
http://gerrit.cloudera.org:8080/19583 )
Change subject: IMPALA-11970: Add query timeline display to Impala WebUI
..
Patch Set 2:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/19583/2/be/src/service/impala-http-handler.cc
File be/src/service/impala-http-handler.cc:
http://gerrit.cloudera.org:8080/#/c/19583/2/be/src/service/impala-http-handler.cc@190
PS2, Line 190: query_timing
> nit: I think "query_timeline" sounds better, but I'm fine either way.
Updated all timing references to timeline
http://gerrit.cloudera.org:8080/#/c/19583/2/be/src/service/impala-http-handler.cc@343
PS2, Line 343: ss.str(Substitute("Could not obtain runtime profile: $0",
status.GetDetail()));
> Should we set ss with just status.GetDetail() if status is not ok when call
Done
--
To view, visit http://gerrit.cloudera.org:8080/19583
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db
Gerrit-Change-Number: 19583
Gerrit-PatchSet: 2
Gerrit-Owner: Kurt Deschler
Gerrit-Reviewer: David Rorke
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Kurt Deschler
Gerrit-Reviewer: Quanlong Huang
Gerrit-Reviewer: Riza Suminto
Gerrit-Reviewer: Wenzhe Zhou
Gerrit-Comment-Date: Wed, 08 Mar 2023 03:41:10 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-11970: Add query timeline display to Impala WebUI
Hello Quanlong Huang, Riza Suminto, David Rorke, Wenzhe Zhou, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/19583 to look at the new patch set (#3). Change subject: IMPALA-11970: Add query timeline display to Impala WebUI .. IMPALA-11970: Add query timeline display to Impala WebUI This change adds a chart to the Impala WebUI for displaying query fragments and plan node timings. The chart list execution timings in a compact form with the plan/fragment tree shown on the left side and a Gantt chart with bars for each execution phase on the right side. Fragments are printed in right-left order to simplify the graph display and group dependent fragments. Plan node labels are color-coded by fragment. The node list will scroll for large plans so that the scale and legend always stay visible. Exchange nodes have dotted boxes that show when rows are sent from the corresponding senders below. Additionally, 'X' and 'O' symbols are used to node where join builds arrive. The length of each colored phase interval depicts the longest instance time while instances that complete earlier are shown as grey lines within the phase interval. On the server side, the query profile handler has been modified to return the profile JSON as a document element instead of document content as it is for JSON profile download. This allows the profile to be accessed in full using scripts in templates. Dense profiles will not render correctly as they currently do not contain the necessary timestamps. This limitation will be addressed at a later time with corresponding changes to the dense profile. Example output is attached to the IMPALA-11970 ticket Testing: Manual validation of rendering using TPC-H and TPC-DS queries Will look into adding tests to test_web_pages.py and test_observability.py in a subsequent revision Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db --- M be/src/service/impala-http-handler.cc M be/src/service/impala-http-handler.h M be/src/service/impala-server.cc M be/src/util/runtime-profile.cc M www/query_detail_tabs.tmpl A www/query_timeline.tmpl 6 files changed, 458 insertions(+), 8 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/83/19583/3 -- To view, visit http://gerrit.cloudera.org:8080/19583 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8b5826107af0f5a7fe306cb986a875cff261d9db Gerrit-Change-Number: 19583 Gerrit-PatchSet: 3 Gerrit-Owner: Kurt Deschler Gerrit-Reviewer: David Rorke Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Kurt Deschler Gerrit-Reviewer: Quanlong Huang Gerrit-Reviewer: Riza Suminto Gerrit-Reviewer: Wenzhe Zhou
