Surya Hebbar has uploaded a new patch set (#14). ( http://gerrit.cloudera.org:8080/20355 )
Change subject: IMPALA-12364: Display memory, disk and network metrics in webUI's query timeline ...................................................................... IMPALA-12364: Display memory, disk and network metrics in webUI's query timeline The fragment's plan nodes are enlarged with an animated transition on hovering over the query timeline's fragment diagram. On clicking the plan nodes, total thread and memory usage of the parent fragment are displayed, after accumulating memory and thread usage of all child nodes. Thread usage is being shown on the additional Y-axis. In this way, memory and thread usage of multiple fragments can be compared alongside. A fragment's usage can be hidden by clicking on any of the child plan nodes again. These counters are available within the profile with following names. - MemoryUsage - ThreadUsage Once a fragment's metrics are displayed, they are updated as they are collected from the profile. A grid-line is displayed along with a tooltip on hovering over the fragment diagram, containing the instantaneous time at that position. This grid-line also triggers tooltips and gridlines in other charts. A warning is displayed on clicking a fragment with less number of samples available. RESOURCE_TRACE_RATIO query option provides the utilization values to be traced within the RuntimeProfile. It contains samples of disk and network usage on each host. These time series counters are available within the profile having the following names. - HostDiskWriteThroughput - HostDiskReadThroughput - HostNetworkRx - HostNetworkTx The additional Y-axis within the utilization chart is used to represent the average of these metrics. The memory units in tooltips and ticks on co-ordinate axes are being displayed in human readable form such as KB, MB, GB and PB for convenience. Both of the charts contain controls to close the chart. These charts can also be resized until a maximum and minmum limit through the resize bar. Timeticks are being autoscaled during fragment diagram's horizontal zoom. In addition to the scrollbar, hovering on edges of the window allows horizontal scrolling. To help with manually testing the query timeline web page, the tests have been integrated using JEST testing framework based on nodejs. 'jest-environment-jsdom' supports rendering DOM elements and testing them without the requirement of a browser. 'run_js_tests.sh' runs JEST test suits, after it fetches nodejs binaries and related nodejs packages, if they are not present locally. For implementing unit and integration tests, the script has been divided into multiple properly functioning modules, that are imported as ES6 modules. Unit tests have been written for functions that accumulate values by parsing the profile. As the order and name of counters may change within the profile, to ensure appropriate mappings between plotted readings, initially the profile is traversed to find the correct mapping. Change-Id: Ifd25e6f0bc9fbd664ec98936daff3f27182dfc7f --- M .gitignore M bin/run-all-tests.sh A tests/run-js-tests.sh M www/query_timeline.tmpl A www/scripts/.gitignore A www/scripts/package.json A www/scripts/query_timeline/chart_commons.js A www/scripts/query_timeline/chart_commons.test.js A www/scripts/query_timeline/fragment_diagram.js A www/scripts/query_timeline/fragment_diagram.test.js A www/scripts/query_timeline/fragment_metrics_diagram.js A www/scripts/query_timeline/fragment_metrics_diagram.test.js A www/scripts/query_timeline/global_dom.js A www/scripts/query_timeline/global_members.js A www/scripts/query_timeline/host_utilization_diagram.js A www/scripts/query_timeline/host_utilization_diagram.test.js M www/scripts/util.js 17 files changed, 2,254 insertions(+), 730 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/20355/14 -- To view, visit http://gerrit.cloudera.org:8080/20355 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifd25e6f0bc9fbd664ec98936daff3f27182dfc7f Gerrit-Change-Number: 20355 Gerrit-PatchSet: 14 Gerrit-Owner: Surya Hebbar <sheb...@cloudera.com> Gerrit-Reviewer: Impala Public Jenkins <impala-public-jenk...@cloudera.com> Gerrit-Reviewer: Kurt Deschler <kdesc...@cloudera.com> Gerrit-Reviewer: Riza Suminto <riza.sumi...@cloudera.com> Gerrit-Reviewer: Surya Hebbar <sheb...@cloudera.com> Gerrit-Reviewer: Wenzhe Zhou <wz...@cloudera.com>
