[jira] [Updated] (MAPREDUCE-7339) Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
[
https://issues.apache.org/jira/browse/MAPREDUCE-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mudit Sharma updated MAPREDUCE-7339:
Description:
DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
speculationValue if the tasks is not a good candidate for speculation. It
returns these values:
{noformat}
returns ALREADY_SPECULATING if that is true. This has priority.
returns TOO_NEW if our companion task hasn't gotten any information
returns PROGRESS_IS_GOOD if the task is sailing through
returns NOT_RUNNING if the task is not running{noformat}
{color:#172b4d}But there is one bug, due to which even if task is speculating,
but if its progress is good or its too late to speculate, code returns
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not ALREADY_SPECULATING{color}
{color:#172b4d}Raised PR for the fix:
https://github.com/apache/hadoop/pull/2951{color}
was:
DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
speculationValue if the tasks is not a good candidate for speculation. It
returns these values:
{noformat}
returns ALREADY_SPECULATING if that is true. This has priority.
returns TOO_NEW if our companion task hasn't gotten any information
returns PROGRESS_IS_GOOD if the task is sailing through
returns NOT_RUNNING if the task is not running{noformat}
{color:#172b4d}But there is one bug, due to which even if task is speculating,
but if its progress is good or its too late to speculate, code returns
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not ALREADY_SPECULATING{color}
> Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or
> PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
>
>
> Key: MAPREDUCE-7339
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7339
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Mudit Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
> speculationValue if the tasks is not a good candidate for speculation. It
> returns these values:
>
> {noformat}
> returns ALREADY_SPECULATING if that is true. This has priority.
> returns TOO_NEW if our companion task hasn't gotten any information
> returns PROGRESS_IS_GOOD if the task is sailing through
> returns NOT_RUNNING if the task is not running{noformat}
>
>
> {color:#172b4d}But there is one bug, due to which even if task is
> speculating, but if its progress is good or its too late to speculate, code
> returns TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not
> ALREADY_SPECULATING{color}
>
> {color:#172b4d}Raised PR for the fix:
> https://github.com/apache/hadoop/pull/2951{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7339) Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
[
https://issues.apache.org/jira/browse/MAPREDUCE-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mudit Sharma updated MAPREDUCE-7339:
Description:
DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
speculationValue if the task is not a good candidate for speculation. It
returns these values:
{noformat}
returns ALREADY_SPECULATING if that is true. This has priority.
returns TOO_NEW if our companion task hasn't gotten any information
returns PROGRESS_IS_GOOD if the task is sailing through
returns NOT_RUNNING if the task is not running{noformat}
{color:#172b4d}But there is one bug, due to which even if task is speculating,
but if its progress is good or its too late to speculate, code returns
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not ALREADY_SPECULATING{color}
{color:#172b4d}Raised PR for the fix:
[https://github.com/apache/hadoop/pull/2951]{color}
was:
DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
speculationValue if the tasks is not a good candidate for speculation. It
returns these values:
{noformat}
returns ALREADY_SPECULATING if that is true. This has priority.
returns TOO_NEW if our companion task hasn't gotten any information
returns PROGRESS_IS_GOOD if the task is sailing through
returns NOT_RUNNING if the task is not running{noformat}
{color:#172b4d}But there is one bug, due to which even if task is speculating,
but if its progress is good or its too late to speculate, code returns
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not ALREADY_SPECULATING{color}
{color:#172b4d}Raised PR for the fix:
https://github.com/apache/hadoop/pull/2951{color}
> Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or
> PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
>
>
> Key: MAPREDUCE-7339
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7339
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Mudit Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
> speculationValue if the task is not a good candidate for speculation. It
> returns these values:
>
> {noformat}
> returns ALREADY_SPECULATING if that is true. This has priority.
> returns TOO_NEW if our companion task hasn't gotten any information
> returns PROGRESS_IS_GOOD if the task is sailing through
> returns NOT_RUNNING if the task is not running{noformat}
>
>
> {color:#172b4d}But there is one bug, due to which even if task is
> speculating, but if its progress is good or its too late to speculate, code
> returns TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not
> ALREADY_SPECULATING{color}
>
> {color:#172b4d}Raised PR for the fix:
> [https://github.com/apache/hadoop/pull/2951]{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7339) Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
[
https://issues.apache.org/jira/browse/MAPREDUCE-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated MAPREDUCE-7339:
--
Labels: pull-request-available (was: )
> Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or
> PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
>
>
> Key: MAPREDUCE-7339
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7339
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Mudit Sharma
>Priority: Major
> Labels: pull-request-available
> Time Spent: 10m
> Remaining Estimate: 0h
>
> DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
> speculationValue if the tasks is not a good candidate for speculation. It
> returns these values:
>
> {noformat}
> returns ALREADY_SPECULATING if that is true. This has priority.
> returns TOO_NEW if our companion task hasn't gotten any information
> returns PROGRESS_IS_GOOD if the task is sailing through
> returns NOT_RUNNING if the task is not running{noformat}
>
>
> {color:#172b4d}But there is one bug, due to which even if task is
> speculating, but if its progress is good or its too late to speculate, code
> returns TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not
> ALREADY_SPECULATING{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Work logged] (MAPREDUCE-7339) Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
[
https://issues.apache.org/jira/browse/MAPREDUCE-7339?focusedWorklogId=588253&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-588253
]
ASF GitHub Bot logged work on MAPREDUCE-7339:
-
Author: ASF GitHub Bot
Created on: 24/Apr/21 05:33
Start Date: 24/Apr/21 05:33
Worklog Time Spent: 10m
Work Description: mudit-97 opened a new pull request #2951:
URL: https://github.com/apache/hadoop/pull/2951
ASF Jira: https://issues.apache.org/jira/browse/MAPREDUCE-7339
Change: DefaultSpeculator returns TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD
speculation value even if task is ALREADY SPECULATING. This PR targets to fix
that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 588253)
Remaining Estimate: 0h
Time Spent: 10m
> Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or
> PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
>
>
> Key: MAPREDUCE-7339
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7339
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Mudit Sharma
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
> speculationValue if the tasks is not a good candidate for speculation. It
> returns these values:
>
> {noformat}
> returns ALREADY_SPECULATING if that is true. This has priority.
> returns TOO_NEW if our companion task hasn't gotten any information
> returns PROGRESS_IS_GOOD if the task is sailing through
> returns NOT_RUNNING if the task is not running{noformat}
>
>
> {color:#172b4d}But there is one bug, due to which even if task is
> speculating, but if its progress is good or its too late to speculate, code
> returns TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not
> ALREADY_SPECULATING{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7339) Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
[
https://issues.apache.org/jira/browse/MAPREDUCE-7339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mudit Sharma updated MAPREDUCE-7339:
Summary: Default Speculator returns speculationValue as
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD even if the task is
ALREADY_SPECULATING (was: Default Speculatoor returns speculationValue as
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD eve if the task is
ALREADY_SPECULATING)
> Default Speculator returns speculationValue as TOO_LATE_TO_SPECULATE or
> PROGRESS_IS_GOOD even if the task is ALREADY_SPECULATING
>
>
> Key: MAPREDUCE-7339
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7339
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Mudit Sharma
>Priority: Major
>
> DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
> speculationValue if the tasks is not a good candidate for speculation. It
> returns these values:
>
> {noformat}
> returns ALREADY_SPECULATING if that is true. This has priority.
> returns TOO_NEW if our companion task hasn't gotten any information
> returns PROGRESS_IS_GOOD if the task is sailing through
> returns NOT_RUNNING if the task is not running{noformat}
>
>
> {color:#172b4d}But there is one bug, due to which even if task is
> speculating, but if its progress is good or its too late to speculate, code
> returns TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not
> ALREADY_SPECULATING{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7339) Default Speculatoor returns speculationValue as TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD eve if the task is ALREADY_SPECULATING
Mudit Sharma created MAPREDUCE-7339:
---
Summary: Default Speculatoor returns speculationValue as
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD eve if the task is ALREADY_SPECULATING
Key: MAPREDUCE-7339
URL: https://issues.apache.org/jira/browse/MAPREDUCE-7339
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Mudit Sharma
DefaultSpeculator implementation in Mapreduce returns some MAGIC numbers in
speculationValue if the tasks is not a good candidate for speculation. It
returns these values:
{noformat}
returns ALREADY_SPECULATING if that is true. This has priority.
returns TOO_NEW if our companion task hasn't gotten any information
returns PROGRESS_IS_GOOD if the task is sailing through
returns NOT_RUNNING if the task is not running{noformat}
{color:#172b4d}But there is one bug, due to which even if task is speculating,
but if its progress is good or its too late to speculate, code returns
TOO_LATE_TO_SPECULATE or PROGRESS_IS_GOOD and not ALREADY_SPECULATING{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7302) Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to fail
[
https://issues.apache.org/jira/browse/MAPREDUCE-7302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Badger updated MAPREDUCE-7302:
---
Fix Version/s: 3.2.3
2.10.2
3.3.1
3.1.5
I've cherry-picked this back all the way to branch-2.10. There were small
import conflicts that I fixed because of us shading some jars in the newer
branches, but not the older ones.
This has now been committed to trunk (3.4), branch-3.3, branch-3.2, branch-3.1,
and branch-2.10
> Upgrading to JUnit 4.13 causes testcase TestFetcher.testCorruptedIFile() to
> fail
>
>
> Key: MAPREDUCE-7302
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7302
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Fix For: 3.4.0, 3.1.5, 3.3.1, 2.10.2, 3.2.3
>
> Attachments: MAPREDUCE-7302-001.patch, MAPREDUCE-7302-002.patch,
> MAPREDUCE-7302-003.patch
>
>
> See related ticket YARN-10460. JUnit 4.13 causes the same failure:
> {noformat}
> [ERROR] Tests run: 16, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
> 1.851 s <<< FAILURE! - in org.apache.hadoop.mapreduce.task.reduce.TestFetcher
> [ERROR]
> testCorruptedIFile(org.apache.hadoop.mapreduce.task.reduce.TestFetcher) Time
> elapsed: 0.15 s <<< ERROR!
> java.lang.IllegalThreadStateException
> at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867)
> at java.lang.Thread.init(Thread.java:405)
> at java.lang.Thread.init(Thread.java:349)
> at java.lang.Thread.(Thread.java:678)
> at
> java.util.concurrent.Executors$DefaultThreadFactory.newThread(Executors.java:613)
> at
> org.apache.hadoop.thirdparty.com.google.common.util.concurrent.ThreadFactoryBuilder$1.newThread(ThreadFactoryBuilder.java:163)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619)
> at
> java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932)
> at
> java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367)
> at
> org.apache.hadoop.io.ReadaheadPool.submitReadahead(ReadaheadPool.java:159)
> at
> org.apache.hadoop.io.ReadaheadPool.readaheadStream(ReadaheadPool.java:141)
> at
> org.apache.hadoop.mapred.IFileInputStream.doReadahead(IFileInputStream.java:159)
> at
> org.apache.hadoop.mapred.IFileInputStream.(IFileInputStream.java:88)
> at
> org.apache.hadoop.mapreduce.task.reduce.TestFetcher.testCorruptedIFile(TestFetcher.java:587)
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-7330) Add access check for getJobAttempts
[ https://issues.apache.org/jira/browse/MAPREDUCE-7330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17313166#comment-17313166 ] lujie edited comment on MAPREDUCE-7330 at 4/23/21, 7:42 PM: again ping-> was (Author: xiaoheipangzi): ping-> > Add access check for getJobAttempts > > > Key: MAPREDUCE-7330 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7330 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: lujie >Assignee: lujie >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Hi, i have found an inconsist, all the methods getting job infomation own > access check, except getJobAttempts. I will fix it. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
