[jira] [Updated] (HIVE-14215) Displaying inconsistent CPU usage data with MR execution engine
[
https://issues.apache.org/jira/browse/HIVE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14215:
---
Fix Version/s: 2.1.1
> Displaying inconsistent CPU usage data with MR execution engine
> ---
>
> Key: HIVE-14215
> URL: https://issues.apache.org/jira/browse/HIVE-14215
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Fix For: 2.2.0, 2.1.1
>
> Attachments: HIVE-14215.patch
>
>
> If the MR task is finished after printing the cumulative CPU time then there
> is the possibility to print inconsistent CPU usage information.
> Correct one:
> {noformat}
> 2016-07-12 11:31:42,961 Stage-3 map = 0%, reduce = 0%
> 2016-07-12 11:31:48,237 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 2.5
> sec
> MapReduce Total cumulative CPU time: 2 seconds 500 msec
> Ended Job = job_1468321038188_0003
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.5 sec HDFS Read: 5864 HDFS Write:
> 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 500 msec
> {noformat}
> One type of inconsistent data (easily reproducible one):
> {noformat}
> 2016-07-12 11:39:00,540 Stage-3 map = 0%, reduce = 0%
> Ended Job = job_1468321038188_0004
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.51 sec HDFS Read: 5864 HDFS
> Write: 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 510 msec
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14215) Displaying inconsistent CPU usage data with MR execution engine
[
https://issues.apache.org/jira/browse/HIVE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14215:
---
Target Version/s: 2.1.1
> Displaying inconsistent CPU usage data with MR execution engine
> ---
>
> Key: HIVE-14215
> URL: https://issues.apache.org/jira/browse/HIVE-14215
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Fix For: 2.2.0
>
> Attachments: HIVE-14215.patch
>
>
> If the MR task is finished after printing the cumulative CPU time then there
> is the possibility to print inconsistent CPU usage information.
> Correct one:
> {noformat}
> 2016-07-12 11:31:42,961 Stage-3 map = 0%, reduce = 0%
> 2016-07-12 11:31:48,237 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 2.5
> sec
> MapReduce Total cumulative CPU time: 2 seconds 500 msec
> Ended Job = job_1468321038188_0003
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.5 sec HDFS Read: 5864 HDFS Write:
> 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 500 msec
> {noformat}
> One type of inconsistent data (easily reproducible one):
> {noformat}
> 2016-07-12 11:39:00,540 Stage-3 map = 0%, reduce = 0%
> Ended Job = job_1468321038188_0004
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.51 sec HDFS Read: 5864 HDFS
> Write: 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 510 msec
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14215) Displaying inconsistent CPU usage data with MR execution engine
[
https://issues.apache.org/jira/browse/HIVE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergio Peña updated HIVE-14215:
---
Resolution: Fixed
Fix Version/s: 2.2.0
Status: Resolved (was: Patch Available)
Thanks [~pvary]. I committed this patch to 2.2
> Displaying inconsistent CPU usage data with MR execution engine
> ---
>
> Key: HIVE-14215
> URL: https://issues.apache.org/jira/browse/HIVE-14215
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Fix For: 2.2.0
>
> Attachments: HIVE-14215.patch
>
>
> If the MR task is finished after printing the cumulative CPU time then there
> is the possibility to print inconsistent CPU usage information.
> Correct one:
> {noformat}
> 2016-07-12 11:31:42,961 Stage-3 map = 0%, reduce = 0%
> 2016-07-12 11:31:48,237 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 2.5
> sec
> MapReduce Total cumulative CPU time: 2 seconds 500 msec
> Ended Job = job_1468321038188_0003
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.5 sec HDFS Read: 5864 HDFS Write:
> 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 500 msec
> {noformat}
> One type of inconsistent data (easily reproducible one):
> {noformat}
> 2016-07-12 11:39:00,540 Stage-3 map = 0%, reduce = 0%
> Ended Job = job_1468321038188_0004
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.51 sec HDFS Read: 5864 HDFS
> Write: 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 510 msec
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14215) Displaying inconsistent CPU usage data with MR execution engine
[
https://issues.apache.org/jira/browse/HIVE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-14215:
--
Status: Patch Available (was: Open)
Tested the patch with the same sleep changes, and is is working
> Displaying inconsistent CPU usage data with MR execution engine
> ---
>
> Key: HIVE-14215
> URL: https://issues.apache.org/jira/browse/HIVE-14215
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Attachments: HIVE-14215.patch
>
>
> If the MR task is finished after printing the cumulative CPU time then there
> is the possibility to print inconsistent CPU usage information.
> Correct one:
> {noformat}
> 2016-07-12 11:31:42,961 Stage-3 map = 0%, reduce = 0%
> 2016-07-12 11:31:48,237 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 2.5
> sec
> MapReduce Total cumulative CPU time: 2 seconds 500 msec
> Ended Job = job_1468321038188_0003
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.5 sec HDFS Read: 5864 HDFS Write:
> 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 500 msec
> {noformat}
> One type of inconsistent data (easily reproducible one):
> {noformat}
> 2016-07-12 11:39:00,540 Stage-3 map = 0%, reduce = 0%
> Ended Job = job_1468321038188_0004
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.51 sec HDFS Read: 5864 HDFS
> Write: 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 510 msec
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (HIVE-14215) Displaying inconsistent CPU usage data with MR execution engine
[
https://issues.apache.org/jira/browse/HIVE-14215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-14215:
--
Attachment: HIVE-14215.patch
Not sure how to create a unit test for this situation, but created a simple
patch which contains reordering the cpu time calculation.
> Displaying inconsistent CPU usage data with MR execution engine
> ---
>
> Key: HIVE-14215
> URL: https://issues.apache.org/jira/browse/HIVE-14215
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.2.0
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Minor
> Attachments: HIVE-14215.patch
>
>
> If the MR task is finished after printing the cumulative CPU time then there
> is the possibility to print inconsistent CPU usage information.
> Correct one:
> {noformat}
> 2016-07-12 11:31:42,961 Stage-3 map = 0%, reduce = 0%
> 2016-07-12 11:31:48,237 Stage-3 map = 100%, reduce = 0%, Cumulative CPU 2.5
> sec
> MapReduce Total cumulative CPU time: 2 seconds 500 msec
> Ended Job = job_1468321038188_0003
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.5 sec HDFS Read: 5864 HDFS Write:
> 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 500 msec
> {noformat}
> One type of inconsistent data (easily reproducible one):
> {noformat}
> 2016-07-12 11:39:00,540 Stage-3 map = 0%, reduce = 0%
> Ended Job = job_1468321038188_0004
> MapReduce Jobs Launched:
> Stage-Stage-3: Map: 1 Cumulative CPU: 2.51 sec HDFS Read: 5864 HDFS
> Write: 103 SUCCESS
> Total MapReduce CPU Time Spent: 2 seconds 510 msec
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
