[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez

2020-12-29 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=529337&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529337
 ]

ASF GitHub Bot logged work on HIVE-24286:
-

Author: ASF GitHub Bot
Created on: 30/Dec/20 01:05
Start Date: 30/Dec/20 01:05
Worklog Time Spent: 10m 
  Work Description: github-actions[bot] closed pull request #1588:
URL: https://github.com/apache/hive/pull/1588


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 529337)
Time Spent: 1h 10m  (was: 1h)

> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>  
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>  
> This ticket would add the current time to the head of each line.
>  
> {code:java}
> 2020-10-19 13:32:41,162   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:44,231   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:46,813   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:49,878   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,416   Map 1: 1/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,936   Map 1: 1/1  Reducer 2: 0(+1)/1  
> 2020-10-19 13:32:52,877   Map 1: 1/1  Reducer 2: 1/1  
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez

2020-12-20 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=526635&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-526635
 ]

ASF GitHub Bot logged work on HIVE-24286:
-

Author: ASF GitHub Bot
Created on: 21/Dec/20 00:57
Start Date: 21/Dec/20 00:57
Worklog Time Spent: 10m 
  Work Description: github-actions[bot] commented on pull request #1588:
URL: https://github.com/apache/hive/pull/1588#issuecomment-748703242


   This pull request has been automatically marked as stale because it has not 
had recent activity. It will be closed if no further activity occurs.
   Feel free to reach out on the d...@hive.apache.org list if the patch is in 
need of reviews.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 526635)
Time Spent: 1h  (was: 50m)

> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>  
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>  
> This ticket would add the current time to the head of each line.
>  
> {code:java}
> 2020-10-19 13:32:41,162   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:44,231   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:46,813   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:49,878   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,416   Map 1: 1/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,936   Map 1: 1/1  Reducer 2: 0(+1)/1  
> 2020-10-19 13:32:52,877   Map 1: 1/1  Reducer 2: 1/1  
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez

2020-10-20 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502953&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502953
 ]

ASF GitHub Bot logged work on HIVE-24286:
-

Author: ASF GitHub Bot
Created on: 21/Oct/20 02:43
Start Date: 21/Oct/20 02:43
Worklog Time Spent: 10m 
  Work Description: okumin commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508954919



##
File path: 
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map 
vertexProgressMap) {
   renderProgress(monitor.progressMonitor(status, vertexProgressMap));
   String report = getReport(vertexProgressMap);
   if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());

Review comment:
   @dengzhhu653 Thanks for clarifying that! I removed the final.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 502953)
Time Spent: 50m  (was: 40m)

> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>  
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>  
> This ticket would add the current time to the head of each line.
>  
> {code:java}
> 2020-10-19 13:32:41,162   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:44,231   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:46,813   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:49,878   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,416   Map 1: 1/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,936   Map 1: 1/1  Reducer 2: 0(+1)/1  
> 2020-10-19 13:32:52,877   Map 1: 1/1  Reducer 2: 1/1  
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez

2020-10-20 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502948&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502948
 ]

ASF GitHub Bot logged work on HIVE-24286:
-

Author: ASF GitHub Bot
Created on: 21/Oct/20 02:31
Start Date: 21/Oct/20 02:31
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508951394



##
File path: 
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map 
vertexProgressMap) {
   renderProgress(monitor.progressMonitor(status, vertexProgressMap));
   String report = getReport(vertexProgressMap);
   if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());

Review comment:
   yes, align with other variables and there seems no neccesarry to declare 
it as final, as no one would change it after the if branch.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 502948)
Time Spent: 40m  (was: 0.5h)

> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>  
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>  
> This ticket would add the current time to the head of each line.
>  
> {code:java}
> 2020-10-19 13:32:41,162   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:44,231   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:46,813   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:49,878   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,416   Map 1: 1/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,936   Map 1: 1/1  Reducer 2: 0(+1)/1  
> 2020-10-19 13:32:52,877   Map 1: 1/1  Reducer 2: 1/1  
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez

2020-10-20 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502943&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502943
 ]

ASF GitHub Bot logged work on HIVE-24286:
-

Author: ASF GitHub Bot
Created on: 21/Oct/20 02:12
Start Date: 21/Oct/20 02:12
Worklog Time Spent: 10m 
  Work Description: okumin commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508946873



##
File path: 
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map 
vertexProgressMap) {
   renderProgress(monitor.progressMonitor(status, vertexProgressMap));
   String report = getReport(vertexProgressMap);
   if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());

Review comment:
   @dengzhhu653 I don't disagree, but what's your thought? To align with 
other variables?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 502943)
Time Spent: 0.5h  (was: 20m)

> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>  
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>  
> This ticket would add the current time to the head of each line.
>  
> {code:java}
> 2020-10-19 13:32:41,162   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:44,231   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:46,813   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:49,878   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,416   Map 1: 1/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,936   Map 1: 1/1  Reducer 2: 0(+1)/1  
> 2020-10-19 13:32:52,877   Map 1: 1/1  Reducer 2: 1/1  
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez

2020-10-20 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502940&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502940
 ]

ASF GitHub Bot logged work on HIVE-24286:
-

Author: ASF GitHub Bot
Created on: 21/Oct/20 01:59
Start Date: 21/Oct/20 01:59
Worklog Time Spent: 10m 
  Work Description: dengzhhu653 commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508943308



##
File path: 
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map 
vertexProgressMap) {
   renderProgress(monitor.progressMonitor(status, vertexProgressMap));
   String report = getReport(vertexProgressMap);
   if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());

Review comment:
   Can we remove the final here?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 502940)
Time Spent: 20m  (was: 10m)

> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>  
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>  
> This ticket would add the current time to the head of each line.
>  
> {code:java}
> 2020-10-19 13:32:41,162   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:44,231   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:46,813   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:49,878   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,416   Map 1: 1/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,936   Map 1: 1/1  Reducer 2: 0(+1)/1  
> 2020-10-19 13:32:52,877   Map 1: 1/1  Reducer 2: 1/1  
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez

2020-10-19 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502214&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502214
 ]

ASF GitHub Bot logged work on HIVE-24286:
-

Author: ASF GitHub Bot
Created on: 19/Oct/20 14:10
Start Date: 19/Oct/20 14:10
Worklog Time Spent: 10m 
  Work Description: okumin opened a new pull request #1588:
URL: https://github.com/apache/hive/pull/1588


   
   
   ### What changes were proposed in this pull request?
   
   Add date and time to progress logs for Hive on Tez like MapReduce and Spark.
   
   ### Why are the changes needed?
   
   For convenience. The date and time will give us more insights when looking 
through logs.
   
   ### Does this PR introduce _any_ user-facing change?
   
   This PR has the only changes for logging. We may add a configuration if this 
can be a breaking change and we can't accept it.
   
   
   ### How was this patch tested?
   
   Run `beeline --hiveconf hive.server2.in.place.progress=false` and check the 
progress logs. I have checked the following logs show up.
   
   ```
   INFO  : 2020-10-19 13:32:41,162  Map 1: 0/1  Reducer 2: 0/1  
   INFO  : 2020-10-19 13:32:44,231  Map 1: 0/1  Reducer 2: 0/1  
   INFO  : 2020-10-19 13:32:46,813  Map 1: 0(+1)/1  Reducer 2: 0/1  
   INFO  : 2020-10-19 13:32:49,878  Map 1: 0(+1)/1  Reducer 2: 0/1  
   INFO  : 2020-10-19 13:32:51,416  Map 1: 1/1  Reducer 2: 0/1  
   INFO  : 2020-10-19 13:32:51,936  Map 1: 1/1  Reducer 2: 0(+1)/1  
   INFO  : 2020-10-19 13:32:52,877  Map 1: 1/1  Reducer 2: 1/1  
   ```
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 502214)
Remaining Estimate: 0h
Time Spent: 10m

> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>  
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
>  * 
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>  
> This ticket would add the current time to the head of each line.
>  
> {code:java}
> 2020-10-19 13:32:41,162   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:44,231   Map 1: 0/1  Reducer 2: 0/1  
> 2020-10-19 13:32:46,813   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:49,878   Map 1: 0(+1)/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,416   Map 1: 1/1  Reducer 2: 0/1  
> 2020-10-19 13:32:51,936   Map 1: 1/1  Reducer 2: 0(+1)/1  
> 2020-10-19 13:32:52,877   Map 1: 1/1  Reducer 2: 1/1  
> {code}
>  
>  



--
This message was sent by Atlassian Jira
(v8.3.4#803005)