[jira] [Commented] (HIVE-3707) Round map/reduce progress down when it is in the range [99.5, 100)
[ https://issues.apache.org/jira/browse/HIVE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13547882#comment-13547882 ] Hudson commented on HIVE-3707: -- Integrated in Hive-trunk-hadoop2 #54 (See [https://builds.apache.org/job/Hive-trunk-hadoop2/54/]) HIVE-3707 Round map/reduce progress down when it is in the range [99.5, 100) (Kevin Wilfong via namit) (Revision 1409680) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1409680 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java Round map/reduce progress down when it is in the range [99.5, 100) -- Key: HIVE-3707 URL: https://issues.apache.org/jira/browse/HIVE-3707 Project: Hive Issue Type: Improvement Components: Logging, Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Priority: Minor Fix For: 0.10.0 Attachments: HIVE-3707.1.patch.txt In HadoopJobExecHelper the mapProgress and reduceProgress are the value of these counters taken from the running job rounded to an integer percentage. This means that e.g. if the mappers are 99.5% done this is stored as 100%. One of the most common questions I see from new users is, the map and reduce both report being 100% done, why is the query still running? By rounding down the value in this interval so it's only 100% when it's really 100% we could avoid that confusion. Also, the way it appears the QueryPlan and MapRedTask determine if the map/reduce phases are done is by checking if this value == 100. I couldn't find anywhere where they're used for anything significant, but they're reporting early completion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3707) Round map/reduce progress down when it is in the range [99.5, 100)
[ https://issues.apache.org/jira/browse/HIVE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13498209#comment-13498209 ] Hudson commented on HIVE-3707: -- Integrated in Hive-trunk-h0.21 #1798 (See [https://builds.apache.org/job/Hive-trunk-h0.21/1798/]) HIVE-3707 Round map/reduce progress down when it is in the range [99.5, 100) (Kevin Wilfong via namit) (Revision 1409680) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1409680 Files : * /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java Round map/reduce progress down when it is in the range [99.5, 100) -- Key: HIVE-3707 URL: https://issues.apache.org/jira/browse/HIVE-3707 Project: Hive Issue Type: Improvement Components: Logging, Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Priority: Minor Fix For: 0.10.0 Attachments: HIVE-3707.1.patch.txt In HadoopJobExecHelper the mapProgress and reduceProgress are the value of these counters taken from the running job rounded to an integer percentage. This means that e.g. if the mappers are 99.5% done this is stored as 100%. One of the most common questions I see from new users is, the map and reduce both report being 100% done, why is the query still running? By rounding down the value in this interval so it's only 100% when it's really 100% we could avoid that confusion. Also, the way it appears the QueryPlan and MapRedTask determine if the map/reduce phases are done is by checking if this value == 100. I couldn't find anywhere where they're used for anything significant, but they're reporting early completion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3707) Round map/reduce progress down when it is in the range [99.5, 100)
[ https://issues.apache.org/jira/browse/HIVE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496946#comment-13496946 ] Mark Grover commented on HIVE-3707: --- +1 on the idea! Round map/reduce progress down when it is in the range [99.5, 100) -- Key: HIVE-3707 URL: https://issues.apache.org/jira/browse/HIVE-3707 Project: Hive Issue Type: Improvement Components: Logging, Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Priority: Minor Attachments: HIVE-3707.1.patch.txt In HadoopJobExecHelper the mapProgress and reduceProgress are the value of these counters taken from the running job rounded to an integer percentage. This means that e.g. if the mappers are 99.5% done this is stored as 100%. One of the most common questions I see from new users is, the map and reduce both report being 100% done, why is the query still running? By rounding down the value in this interval so it's only 100% when it's really 100% we could avoid that confusion. Also, the way it appears the QueryPlan and MapRedTask determine if the map/reduce phases are done is by checking if this value == 100. I couldn't find anywhere where they're used for anything significant, but they're reporting early completion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3707) Round map/reduce progress down when it is in the range [99.5, 100)
[ https://issues.apache.org/jira/browse/HIVE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13497776#comment-13497776 ] Namit Jain commented on HIVE-3707: -- +1 on the implementation too. Round map/reduce progress down when it is in the range [99.5, 100) -- Key: HIVE-3707 URL: https://issues.apache.org/jira/browse/HIVE-3707 Project: Hive Issue Type: Improvement Components: Logging, Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Priority: Minor Attachments: HIVE-3707.1.patch.txt In HadoopJobExecHelper the mapProgress and reduceProgress are the value of these counters taken from the running job rounded to an integer percentage. This means that e.g. if the mappers are 99.5% done this is stored as 100%. One of the most common questions I see from new users is, the map and reduce both report being 100% done, why is the query still running? By rounding down the value in this interval so it's only 100% when it's really 100% we could avoid that confusion. Also, the way it appears the QueryPlan and MapRedTask determine if the map/reduce phases are done is by checking if this value == 100. I couldn't find anywhere where they're used for anything significant, but they're reporting early completion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HIVE-3707) Round map/reduce progress down when it is in the range [99.5, 100)
[ https://issues.apache.org/jira/browse/HIVE-3707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13496653#comment-13496653 ] Kevin Wilfong commented on HIVE-3707: - https://reviews.facebook.net/D6705 Round map/reduce progress down when it is in the range [99.5, 100) -- Key: HIVE-3707 URL: https://issues.apache.org/jira/browse/HIVE-3707 Project: Hive Issue Type: Improvement Components: Logging, Query Processor Affects Versions: 0.10.0 Reporter: Kevin Wilfong Assignee: Kevin Wilfong Priority: Minor Attachments: HIVE-3707.1.patch.txt In HadoopJobExecHelper the mapProgress and reduceProgress are the value of these counters taken from the running job rounded to an integer percentage. This means that e.g. if the mappers are 99.5% done this is stored as 100%. One of the most common questions I see from new users is, the map and reduce both report being 100% done, why is the query still running? By rounding down the value in this interval so it's only 100% when it's really 100% we could avoid that confusion. Also, the way it appears the QueryPlan and MapRedTask determine if the map/reduce phases are done is by checking if this value == 100. I couldn't find anywhere where they're used for anything significant, but they're reporting early completion. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
