subject:"Review Request 61663\: WebUI query plan graphs"

Re: Review Request 61663: WebUI query plan graphs

2018-09-14 Thread Peter Vary via Review Board


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/#review208622
---



Fix it then ship it!

Thanks for the revitalization of this change!


ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java
Line 508 (original), 509 (patched)


Might want to use LOG.error(String, Throwable)



common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Lines 3099 (patched)


After committing the change, please do not forget to update the 
configuration Wiki. :)


- Peter Vary


On szept. 7, 2018, 3:24 du, Karen Coppage wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61663/
> ---
> 
> (Updated szept. 7, 2018, 3:24 du)
> 
> 
> Review request for hive, Peter Vary and Xuefu Zhang.
> 
> 
> Bugs: HIVE-17300
> https://issues.apache.org/jira/browse/HIVE-17300
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
> the option to display the query plan as a nice graph (scroll down for 
> screenshots). If you click on one of the graph’s stages, the plan for that 
> stage appears as text below.
> Stages are color-coded if they have a status (Success, Error, Running), and 
> the rest are grayed out. Coloring is based on status already available in the 
> WebUI, under the Stages tab.
> There is an additional option to display stats for MapReduce tasks. This 
> includes the job’s ID, tracking URL (where the logs are found), and mapper 
> and reducer numbers/progress, among other info.
> The library I’m using for the graph is called vis.js (http://visjs.org/). It 
> has an Apache license, and the only necessary file to be included from this 
> library is about 700 KB.
> I tried to keep server-side changes minimal, and graph generation is taken 
> care of by the client. Plans with more than a given number of stages 
> (default: 25) won't be displayed in order to preserve resources.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/LogUtils.java 5068eb5be7 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 40ea3ac0c5 
>   
> itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
>  95b46a8149 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java dad2035362 
>   ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java ac45ec46de 
>   ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java 9a77c2969e 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
> eb6cbf71e2 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java a71faf8576 
>   service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon f04d655440 
>   service/src/resources/hive-webapps/static/css/query-plan-graph.css 
> PRE-CREATION 
>   service/src/resources/hive-webapps/static/js/query-plan-graph.js 
> PRE-CREATION 
>   service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61663/diff/5/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> HIVE-17300.7.patch
>   
> https://reviews.apache.org/media/uploaded/files/2018/09/07/e8ada965-ea33-48a2-a0b1-e56e8185c4fa__HIVE-17300.7.patch
> 
> 
> Thanks,
> 
> Karen Coppage
> 
>

Re: Review Request 61663: WebUI query plan graphs

2018-09-07 Thread Karen Coppage


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/
---

(Updated Sept. 7, 2018, 3:24 p.m.)


Review request for hive, Peter Vary and Xuefu Zhang.


Changes
---

Added tests and checks, disabled zoom


Bugs: HIVE-17300
https://issues.apache.org/jira/browse/HIVE-17300


Repository: hive-git


Description
---

I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
the option to display the query plan as a nice graph (scroll down for 
screenshots). If you click on one of the graph’s stages, the plan for that 
stage appears as text below.
Stages are color-coded if they have a status (Success, Error, Running), and the 
rest are grayed out. Coloring is based on status already available in the 
WebUI, under the Stages tab.
There is an additional option to display stats for MapReduce tasks. This 
includes the job’s ID, tracking URL (where the logs are found), and mapper and 
reducer numbers/progress, among other info.
The library I’m using for the graph is called vis.js (http://visjs.org/). It 
has an Apache license, and the only necessary file to be included from this 
library is about 700 KB.
I tried to keep server-side changes minimal, and graph generation is taken care 
of by the client. Plans with more than a given number of stages (default: 25) 
won't be displayed in order to preserve resources.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/LogUtils.java 5068eb5be7 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 40ea3ac0c5 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
 95b46a8149 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java dad2035362 
  ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java ac45ec46de 
  ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java 9a77c2969e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
eb6cbf71e2 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java a71faf8576 
  service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon f04d655440 
  service/src/resources/hive-webapps/static/css/query-plan-graph.css 
PRE-CREATION 
  service/src/resources/hive-webapps/static/js/query-plan-graph.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 


Diff: https://reviews.apache.org/r/61663/diff/5/

Changes: https://reviews.apache.org/r/61663/diff/4-5/


Testing
---


File Attachments (updated)


HIVE-17300.7.patch
  
https://reviews.apache.org/media/uploaded/files/2018/09/07/e8ada965-ea33-48a2-a0b1-e56e8185c4fa__HIVE-17300.7.patch


Thanks,

Karen Coppage

Re: Review Request 61663: WebUI query plan graphs

2018-08-30 Thread Karen Coppage


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/
---

(Updated Aug. 30, 2018, 2:42 p.m.)


Review request for hive, Peter Vary and Xuefu Zhang.


Bugs: HIVE-17300
https://issues.apache.org/jira/browse/HIVE-17300


Repository: hive-git


Description
---

I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
the option to display the query plan as a nice graph (scroll down for 
screenshots). If you click on one of the graph’s stages, the plan for that 
stage appears as text below.
Stages are color-coded if they have a status (Success, Error, Running), and the 
rest are grayed out. Coloring is based on status already available in the 
WebUI, under the Stages tab.
There is an additional option to display stats for MapReduce tasks. This 
includes the job’s ID, tracking URL (where the logs are found), and mapper and 
reducer numbers/progress, among other info.
The library I’m using for the graph is called vis.js (http://visjs.org/). It 
has an Apache license, and the only necessary file to be included from this 
library is about 700 KB.
I tried to keep server-side changes minimal, and graph generation is taken care 
of by the client. Plans with more than a given number of stages (default: 25) 
won't be displayed in order to preserve resources.


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/LogUtils.java 5068eb5be7 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8c39de3e77 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 737debd2ad 
  ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java ac45ec46de 
  ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java 9a77c2969e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
eb6cbf71e2 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java a71faf8576 
  service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon f04d655440 
  service/src/resources/hive-webapps/static/css/query-plan-graph.css 
PRE-CREATION 
  service/src/resources/hive-webapps/static/js/query-plan-graph.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 


Diff: https://reviews.apache.org/r/61663/diff/4/


Testing
---


Thanks,

Karen Coppage

Re: Review Request 61663: WebUI query plan graphs

2018-08-30 Thread Karen Coppage


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/
---

(Updated Aug. 30, 2018, 1:09 p.m.)


Review request for hive, Peter Vary and Xuefu Zhang.


Changes
---

Rebased patch (includes changes made by PVary in HIVE-17300.5.patch)


Repository: hive-git


Description
---

I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
the option to display the query plan as a nice graph (scroll down for 
screenshots). If you click on one of the graph’s stages, the plan for that 
stage appears as text below.
Stages are color-coded if they have a status (Success, Error, Running), and the 
rest are grayed out. Coloring is based on status already available in the 
WebUI, under the Stages tab.
There is an additional option to display stats for MapReduce tasks. This 
includes the job’s ID, tracking URL (where the logs are found), and mapper and 
reducer numbers/progress, among other info.
The library I’m using for the graph is called vis.js (http://visjs.org/). It 
has an Apache license, and the only necessary file to be included from this 
library is about 700 KB.
I tried to keep server-side changes minimal, and graph generation is taken care 
of by the client. Plans with more than a given number of stages (default: 25) 
won't be displayed in order to preserve resources.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/LogUtils.java 5068eb5be7 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 8c39de3e77 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 737debd2ad 
  ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java ac45ec46de 
  ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java 9a77c2969e 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
eb6cbf71e2 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java a71faf8576 
  service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon f04d655440 
  service/src/resources/hive-webapps/static/css/query-plan-graph.css 
PRE-CREATION 
  service/src/resources/hive-webapps/static/js/query-plan-graph.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 


Diff: https://reviews.apache.org/r/61663/diff/4/

Changes: https://reviews.apache.org/r/61663/diff/3-4/


Testing
---


Thanks,

Karen Coppage

Re: Review Request 61663: WebUI query plan graphs

2017-10-06 Thread Peter Vary



> On Aug. 30, 2017, 4:47 a.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java
> > Lines 127 (patched)
> > 
> >
> > What happens if this is a map-only task?

In the next patch I split the if statement


> On Aug. 30, 2017, 4:47 a.m., Xuefu Zhang wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java
> > Lines 132 (patched)
> > 
> >
> > It might be better if the null check is put in getCountersJson() method.

I think we should keep this, as it is - it is better to omit the "counter" 
element, than adding an empty one? What do you think?


> On Aug. 30, 2017, 4:47 a.m., Xuefu Zhang wrote:
> > service/src/resources/hive-webapps/static/css/query-plan-graph.css
> > Lines 1 (patched)
> > 
> >
> > Apache license header if possible.

Done


> On Aug. 30, 2017, 4:47 a.m., Xuefu Zhang wrote:
> > service/src/resources/hive-webapps/static/js/query-plan-graph.js
> > Lines 1 (patched)
> > 
> >
> > I think we need apache license header.

Done


> On Aug. 30, 2017, 4:47 a.m., Xuefu Zhang wrote:
> > service/src/resources/hive-webapps/static/js/vis.min.js
> > Lines 1 (patched)
> > 
> >
> > Apache license header.

Done


- Peter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/#review184121
---


On Aug. 16, 2017, 1:55 p.m., Karen Coppage wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61663/
> ---
> 
> (Updated Aug. 16, 2017, 1:55 p.m.)
> 
> 
> Review request for hive, Peter Vary and Xuefu Zhang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
> the option to display the query plan as a nice graph (scroll down for 
> screenshots). If you click on one of the graph’s stages, the plan for that 
> stage appears as text below.
> Stages are color-coded if they have a status (Success, Error, Running), and 
> the rest are grayed out. Coloring is based on status already available in the 
> WebUI, under the Stages tab.
> There is an additional option to display stats for MapReduce tasks. This 
> includes the job’s ID, tracking URL (where the logs are found), and mapper 
> and reducer numbers/progress, among other info.
> The library I’m using for the graph is called vis.js (http://visjs.org/). It 
> has an Apache license, and the only necessary file to be included from this 
> library is about 700 KB.
> I tried to keep server-side changes minimal, and graph generation is taken 
> care of by the client. Plans with more than a given number of stages 
> (default: 25) won't be displayed in order to preserve resources.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/LogUtils.java 0a3e0c7201 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3c158a6692 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 4e7c80f184 
>   ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 4b6051485e 
>   ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java bf6cb91745 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
> 3c0719717c 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 1bd4db7805 
>   service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon ff7476ee02 
>   service/src/resources/hive-webapps/static/css/query-plan-graph.css 
> PRE-CREATION 
>   service/src/resources/hive-webapps/static/js/query-plan-graph.js 
> PRE-CREATION 
>   service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61663/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Karen Coppage
> 
>

Re: Review Request 61663: WebUI query plan graphs

2017-08-29 Thread Xuefu Zhang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/#review184121
---



Patch looks good. I didn't read the js files, but hope it's okay. a few minor 
comments.


ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java
Lines 127 (patched)


What happens if this is a map-only task?



ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java
Lines 132 (patched)


It might be better if the null check is put in getCountersJson() method.



service/src/resources/hive-webapps/static/css/query-plan-graph.css
Lines 1 (patched)


Apache license header if possible.



service/src/resources/hive-webapps/static/js/query-plan-graph.js
Lines 1 (patched)


I think we need apache license header.



service/src/resources/hive-webapps/static/js/vis.min.js
Lines 1 (patched)


Apache license header.


- Xuefu Zhang


On Aug. 16, 2017, 1:55 p.m., Karen Coppage wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/61663/
> ---
> 
> (Updated Aug. 16, 2017, 1:55 p.m.)
> 
> 
> Review request for hive, Peter Vary and Xuefu Zhang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
> the option to display the query plan as a nice graph (scroll down for 
> screenshots). If you click on one of the graph’s stages, the plan for that 
> stage appears as text below.
> Stages are color-coded if they have a status (Success, Error, Running), and 
> the rest are grayed out. Coloring is based on status already available in the 
> WebUI, under the Stages tab.
> There is an additional option to display stats for MapReduce tasks. This 
> includes the job’s ID, tracking URL (where the logs are found), and mapper 
> and reducer numbers/progress, among other info.
> The library I’m using for the graph is called vis.js (http://visjs.org/). It 
> has an Apache license, and the only necessary file to be included from this 
> library is about 700 KB.
> I tried to keep server-side changes minimal, and graph generation is taken 
> care of by the client. Plans with more than a given number of stages 
> (default: 25) won't be displayed in order to preserve resources.
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/common/LogUtils.java 0a3e0c7201 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3c158a6692 
>   ql/src/java/org/apache/hadoop/hive/ql/Driver.java 4e7c80f184 
>   ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 4b6051485e 
>   ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java bf6cb91745 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
> 3c0719717c 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 1bd4db7805 
>   service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon ff7476ee02 
>   service/src/resources/hive-webapps/static/css/query-plan-graph.css 
> PRE-CREATION 
>   service/src/resources/hive-webapps/static/js/query-plan-graph.js 
> PRE-CREATION 
>   service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/61663/diff/3/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Karen Coppage
> 
>

Re: Review Request 61663: WebUI query plan graphs

2017-08-16 Thread Karen Coppage


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/
---

(Updated Aug. 16, 2017, 1:55 p.m.)


Review request for hive, Peter Vary and Xuefu Zhang.


Repository: hive-git


Description
---

I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
the option to display the query plan as a nice graph (scroll down for 
screenshots). If you click on one of the graph’s stages, the plan for that 
stage appears as text below.
Stages are color-coded if they have a status (Success, Error, Running), and the 
rest are grayed out. Coloring is based on status already available in the 
WebUI, under the Stages tab.
There is an additional option to display stats for MapReduce tasks. This 
includes the job’s ID, tracking URL (where the logs are found), and mapper and 
reducer numbers/progress, among other info.
The library I’m using for the graph is called vis.js (http://visjs.org/). It 
has an Apache license, and the only necessary file to be included from this 
library is about 700 KB.
I tried to keep server-side changes minimal, and graph generation is taken care 
of by the client. Plans with more than a given number of stages (default: 25) 
won't be displayed in order to preserve resources.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/LogUtils.java 0a3e0c7201 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3c158a6692 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 4e7c80f184 
  ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 4b6051485e 
  ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java bf6cb91745 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
3c0719717c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 1bd4db7805 
  service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon ff7476ee02 
  service/src/resources/hive-webapps/static/css/query-plan-graph.css 
PRE-CREATION 
  service/src/resources/hive-webapps/static/js/query-plan-graph.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 


Diff: https://reviews.apache.org/r/61663/diff/3/

Changes: https://reviews.apache.org/r/61663/diff/2-3/


Testing
---


Thanks,

Karen Coppage

Re: Review Request 61663: WebUI query plan graphs

2017-08-16 Thread Karen Coppage


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/
---

(Updated Aug. 16, 2017, 1:47 p.m.)


Review request for hive, Peter Vary and Xuefu Zhang.


Repository: hive-git


Description
---

I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
the option to display the query plan as a nice graph (scroll down for 
screenshots). If you click on one of the graph’s stages, the plan for that 
stage appears as text below.
Stages are color-coded if they have a status (Success, Error, Running), and the 
rest are grayed out. Coloring is based on status already available in the 
WebUI, under the Stages tab.
There is an additional option to display stats for MapReduce tasks. This 
includes the job’s ID, tracking URL (where the logs are found), and mapper and 
reducer numbers/progress, among other info.
The library I’m using for the graph is called vis.js (http://visjs.org/). It 
has an Apache license, and the only necessary file to be included from this 
library is about 700 KB.
I tried to keep server-side changes minimal, and graph generation is taken care 
of by the client. Plans with more than a given number of stages (default: 25) 
won't be displayed in order to preserve resources.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/common/LogUtils.java 0a3e0c7201 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 3c158a6692 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 4e7c80f184 
  ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 4b6051485e 
  ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java bf6cb91745 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
3c0719717c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 1bd4db7805 
  service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon ff7476ee02 
  service/src/resources/hive-webapps/static/css/query-plan-graph.css 
PRE-CREATION 
  service/src/resources/hive-webapps/static/js/query-plan-graph.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 


Diff: https://reviews.apache.org/r/61663/diff/2/

Changes: https://reviews.apache.org/r/61663/diff/1-2/


Testing
---


Thanks,

Karen Coppage

Review Request 61663: WebUI query plan graphs

2017-08-15 Thread Karen Coppage


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/61663/
---

Review request for hive, Peter Vary and Xuefu Zhang.


Repository: hive-git


Description
---

I’m working on a feature of the Hive WebUI Query Plan tab that would provide 
the option to display the query plan as a nice graph (scroll down for 
screenshots). If you click on one of the graph’s stages, the plan for that 
stage appears as text below.
Stages are color-coded if they have a status (Success, Error, Running), and the 
rest are grayed out. Coloring is based on status already available in the 
WebUI, under the Stages tab.
There is an additional option to display stats for MapReduce tasks. This 
includes the job’s ID, tracking URL (where the logs are found), and mapper and 
reducer numbers/progress, among other info.
The library I’m using for the graph is called vis.js (http://visjs.org/). It 
has an Apache license, and the only necessary file to be included from this 
library is about 700 KB.
I tried to keep server-side changes minimal, and graph generation is taken care 
of by the client. Plans with more than a given number of stages (default: 25) 
won't be displayed in order to preserve resources.


Diffs
-

  common/src/java/org/apache/hadoop/hive/common/LogUtils.java 0a3e0c7201 
  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 056f2d7834 
  
itests/hive-unit/src/test/java/org/apache/hive/service/cli/session/TestQueryDisplay.java
 155c65dd26 
  ql/src/java/org/apache/hadoop/hive/ql/Driver.java 4e7c80f184 
  ql/src/java/org/apache/hadoop/hive/ql/MapRedStats.java 4b6051485e 
  ql/src/java/org/apache/hadoop/hive/ql/QueryDisplay.java 5bf22107dc 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java 
3c0719717c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapRedTask.java 1bd4db7805 
  service/src/jamon/org/apache/hive/tmpl/QueryProfileTmpl.jamon 5e2d68c4a4 
  service/src/java/org/apache/hive/service/servlet/QueryProfileServlet.java 
27a3b1423a 
  service/src/resources/hive-webapps/static/css/query-plan-graph.css 
PRE-CREATION 
  service/src/resources/hive-webapps/static/js/query-plan-graph.js PRE-CREATION 
  service/src/resources/hive-webapps/static/js/vis.min.js PRE-CREATION 


Diff: https://reviews.apache.org/r/61663/diff/1/


Testing
---


Thanks,

Karen Coppage

Re: Review Request 61663: WebUI query plan graphs

Re: Review Request 61663: WebUI query plan graphs

Re: Review Request 61663: WebUI query plan graphs

Re: Review Request 61663: WebUI query plan graphs

Re: Review Request 61663: WebUI query plan graphs

Re: Review Request 61663: WebUI query plan graphs

Re: Review Request 61663: WebUI query plan graphs

Re: Review Request 61663: WebUI query plan graphs

Review Request 61663: WebUI query plan graphs

9 matches

Site Navigation

Mail list logo

Footer information