[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=529337&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-529337
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 30/Dec/20 01:05
Start Date: 30/Dec/20 01:05
Worklog Time Spent: 10m
Work Description: github-actions[bot] closed pull request #1588:
URL: https://github.com/apache/hive/pull/1588
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 529337)
Time Spent: 1h 10m (was: 1h)
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=526635&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-526635
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 21/Dec/20 00:57
Start Date: 21/Dec/20 00:57
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #1588:
URL: https://github.com/apache/hive/pull/1588#issuecomment-748703242
This pull request has been automatically marked as stale because it has not
had recent activity. It will be closed if no further activity occurs.
Feel free to reach out on the d...@hive.apache.org list if the patch is in
need of reviews.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 526635)
Time Spent: 1h (was: 50m)
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502953&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502953
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 21/Oct/20 02:43
Start Date: 21/Oct/20 02:43
Worklog Time Spent: 10m
Work Description: okumin commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508954919
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map
vertexProgressMap) {
renderProgress(monitor.progressMonitor(status, vertexProgressMap));
String report = getReport(vertexProgressMap);
if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());
Review comment:
@dengzhhu653 Thanks for clarifying that! I removed the final.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502953)
Time Spent: 50m (was: 40m)
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502948&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502948
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 21/Oct/20 02:31
Start Date: 21/Oct/20 02:31
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508951394
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map
vertexProgressMap) {
renderProgress(monitor.progressMonitor(status, vertexProgressMap));
String report = getReport(vertexProgressMap);
if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());
Review comment:
yes, align with other variables and there seems no neccesarry to declare
it as final, as no one would change it after the if branch.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502948)
Time Spent: 40m (was: 0.5h)
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502943&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502943
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 21/Oct/20 02:12
Start Date: 21/Oct/20 02:12
Worklog Time Spent: 10m
Work Description: okumin commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508946873
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map
vertexProgressMap) {
renderProgress(monitor.progressMonitor(status, vertexProgressMap));
String report = getReport(vertexProgressMap);
if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());
Review comment:
@dengzhhu653 I don't disagree, but what's your thought? To align with
other variables?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502943)
Time Spent: 0.5h (was: 20m)
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502940&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502940
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 21/Oct/20 01:59
Start Date: 21/Oct/20 01:59
Worklog Time Spent: 10m
Work Description: dengzhhu653 commented on a change in pull request #1588:
URL: https://github.com/apache/hive/pull/1588#discussion_r508943308
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/monitoring/RenderStrategy.java
##
@@ -57,7 +60,8 @@ public void update(DAGStatus status, Map
vertexProgressMap) {
renderProgress(monitor.progressMonitor(status, vertexProgressMap));
String report = getReport(vertexProgressMap);
if (showReport(report)) {
-renderReport(report);
+final String time = FORMATTER.format(LocalDateTime.now());
Review comment:
Can we remove the final here?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502940)
Time Spent: 20m (was: 10m)
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (HIVE-24286) Render date and time with progress of Hive on Tez
[
https://issues.apache.org/jira/browse/HIVE-24286?focusedWorklogId=502214&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502214
]
ASF GitHub Bot logged work on HIVE-24286:
-
Author: ASF GitHub Bot
Created on: 19/Oct/20 14:10
Start Date: 19/Oct/20 14:10
Worklog Time Spent: 10m
Work Description: okumin opened a new pull request #1588:
URL: https://github.com/apache/hive/pull/1588
### What changes were proposed in this pull request?
Add date and time to progress logs for Hive on Tez like MapReduce and Spark.
### Why are the changes needed?
For convenience. The date and time will give us more insights when looking
through logs.
### Does this PR introduce _any_ user-facing change?
This PR has the only changes for logging. We may add a configuration if this
can be a breaking change and we can't accept it.
### How was this patch tested?
Run `beeline --hiveconf hive.server2.in.place.progress=false` and check the
progress logs. I have checked the following logs show up.
```
INFO : 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
INFO : 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
INFO : 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
INFO : 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
INFO : 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
INFO : 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502214)
Remaining Estimate: 0h
Time Spent: 10m
> Render date and time with progress of Hive on Tez
> -
>
> Key: HIVE-24286
> URL: https://issues.apache.org/jira/browse/HIVE-24286
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 4.0.0
>Reporter: okumin
>Assignee: okumin
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Add date/time to each line written by RenderStrategy like MapReduce and Spark.
>
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/mr/HadoopJobExecHelper.java#L350]
> *
> [https://github.com/apache/hive/blob/31c1658d9884eb4f31b06eaa718dfef8b1d92d22/ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/RenderStrategy.java#L64-L67]
>
> This ticket would add the current time to the head of each line.
>
> {code:java}
> 2020-10-19 13:32:41,162 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:44,231 Map 1: 0/1 Reducer 2: 0/1
> 2020-10-19 13:32:46,813 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:49,878 Map 1: 0(+1)/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,416 Map 1: 1/1 Reducer 2: 0/1
> 2020-10-19 13:32:51,936 Map 1: 1/1 Reducer 2: 0(+1)/1
> 2020-10-19 13:32:52,877 Map 1: 1/1 Reducer 2: 1/1
> {code}
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
