[jira] [Updated] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6741: -- Description: JHS today displays all Job conf properties in Web UI directly. Users may have some credentials or any sensitive information they added to the job conf but do not want to be shown in Web UI. It'd be nice if we can allow users to specify a set of properties which JHS will filter out when Job conf is displayed. (was: JHS today display all Job conf properties in Web UI directly. Users may have some credentials or any sensitive information they added to the job conf but do not want to be shown in Web UI. It'd be nice if we can allow users to specify a set of properties which JHS will filter out when Job conf is displayed.) > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today displays all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6741: -- Description: JHS today display all Job conf properties in Web UI directly. Users may have some credentials or any sensitive information they added to the job conf but do not want to be shown in Web UI. It'd be nice if we can allow users to specify a set of properties which JHS will filter out when Job conf is displayed. (was: JHS today display all Job conf properties in Web UI directly. Anyone who has access to JHS web UI essentially has access to Job Conf of any other users' job. Users may have some credentials or any sensitive information they added to the job conf. It'd be nice if we can allow users to specify a set of properties which JHS will filter out when Job conf is displayed.) > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today display all Job conf properties in Web UI directly. Users may have > some credentials or any sensitive information they added to the job conf but > do not want to be shown in Web UI. It'd be nice if we can allow users to > specify a set of properties which JHS will filter out when Job conf is > displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388543#comment-15388543 ] Haibo Chen commented on MAPREDUCE-6741: --- Thanks [~jlowe] for pointing out my false statement above. Will update the jira description. > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today display all Job conf properties in Web UI directly. Anyone who has > access to JHS web UI essentially has access to Job Conf of any other users' > job. Users may have some credentials or any sensitive information they added > to the job conf. It'd be nice if we can allow users to specify a set of > properties which JHS will filter out when Job conf is displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
[ https://issues.apache.org/jira/browse/MAPREDUCE-6741?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388293#comment-15388293 ] Jason Lowe commented on MAPREDUCE-6741: --- bq. Anyone who has access to JHS web UI essentially has access to Job Conf of any other users' job. This is not true. The job's setting for mapreduce.job.acl-view-job is respected by the history server, and users who do not have view access will not be able to see the job's configuration even though they have access to the JHS web UI. Having the ability to filter individual conf keys is one thing, but they should not be completely wide-open today unless the user didn't bother to restrict the visibility of their job. > add JHS support to hide job conf properties from Web UI > --- > > Key: MAPREDUCE-6741 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.2 >Reporter: Haibo Chen >Assignee: Haibo Chen > > JHS today display all Job conf properties in Web UI directly. Anyone who has > access to JHS web UI essentially has access to Job Conf of any other users' > job. Users may have some credentials or any sensitive information they added > to the job conf. It'd be nice if we can allow users to specify a set of > properties which JHS will filter out when Job conf is displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6741) add JHS support to hide job conf properties from Web UI
Haibo Chen created MAPREDUCE-6741: - Summary: add JHS support to hide job conf properties from Web UI Key: MAPREDUCE-6741 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6741 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 2.7.2 Reporter: Haibo Chen Assignee: Haibo Chen JHS today display all Job conf properties in Web UI directly. Anyone who has access to JHS web UI essentially has access to Job Conf of any other users' job. Users may have some credentials or any sensitive information they added to the job conf. It'd be nice if we can allow users to specify a set of properties which JHS will filter out when Job conf is displayed. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6740) Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval
[
https://issues.apache.org/jira/browse/MAPREDUCE-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388238#comment-15388238
]
Hadoop QA commented on MAPREDUCE-6740:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 27s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
15s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app:
The patch generated 0 new + 3 unchanged - 1 fixed = 3 total (was 4) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
40s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 8m 49s
{color} | {color:green} hadoop-mapreduce-client-app in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 59s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12819429/mapreduce6740.002.patch
|
| JIRA Issue | MAPREDUCE-6740 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 2638bcf1426f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / ecff7d0 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6637/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6637/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> Enforce mapreduce.task.timeout to be at least
> mapreduce.task.progress-report.interval
> -
>
> Key: MAPREDUCE-6740
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6740
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>
[jira] [Commented] (MAPREDUCE-6738) TestJobListCache.testAddExisting failed intermittently in slow VM testbed
[ https://issues.apache.org/jira/browse/MAPREDUCE-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388234#comment-15388234 ] Hudson commented on MAPREDUCE-6738: --- SUCCESS: Integrated in Hadoop-trunk-Commit #10130 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/10130/]) MAPREDUCE-6738. TestJobListCache.testAddExisting failed intermittently (aajisaka: rev ecff7d03595b45d7a5d6aed73b9931fef85a56ee) * hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/test/java/org/apache/hadoop/mapreduce/v2/hs/TestJobListCache.java > TestJobListCache.testAddExisting failed intermittently in slow VM testbed > - > > Key: MAPREDUCE-6738 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6738 > Project: Hadoop Map/Reduce > Issue Type: Test >Affects Versions: 2.7.3 >Reporter: Junping Du >Assignee: Junping Du >Priority: Minor > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: MAPREDUCE-6738.patch > > > Kick off Jenkins test which occasionally failed for this test with stack > trace below: > org.apache.hadoop.mapreduce.v2.hs.TestJobListCache.testAddExisting > java.lang.Exception: test timed out after 1000 milliseconds > at org.mockito.cglib.proxy.Enhancer.generateClass(Enhancer.java:483) > at > org.mockito.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25) > at > org.mockito.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:217) > at org.mockito.cglib.proxy.Enhancer.createHelper(Enhancer.java:378) > at org.mockito.cglib.proxy.Enhancer.createClass(Enhancer.java:318) > at > org.mockito.internal.creation.jmock.ClassImposterizer.createProxyClass(ClassImposterizer.java:93) > at > org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:50) > at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:54) > at org.mockito.internal.MockitoCore.mock(MockitoCore.java:45) > at org.mockito.Mockito.mock(Mockito.java:921) > at org.mockito.Mockito.mock(Mockito.java:816) > at > org.apache.hadoop.mapreduce.v2.hs.TestJobListCache.testAddExisting(TestJobListCache.java:42) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6738) TestJobListCache.testAddExisting failed intermittently in slow VM testbed
[ https://issues.apache.org/jira/browse/MAPREDUCE-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated MAPREDUCE-6738: - Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 3.0.0-alpha2 2.8.0 Status: Resolved (was: Patch Available) Committed this to trunk, branch-2, and branch-2.8. Thanks [~djp] for the contribution! > TestJobListCache.testAddExisting failed intermittently in slow VM testbed > - > > Key: MAPREDUCE-6738 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6738 > Project: Hadoop Map/Reduce > Issue Type: Test >Affects Versions: 2.7.3 >Reporter: Junping Du >Assignee: Junping Du >Priority: Minor > Fix For: 2.8.0, 3.0.0-alpha2 > > Attachments: MAPREDUCE-6738.patch > > > Kick off Jenkins test which occasionally failed for this test with stack > trace below: > org.apache.hadoop.mapreduce.v2.hs.TestJobListCache.testAddExisting > java.lang.Exception: test timed out after 1000 milliseconds > at org.mockito.cglib.proxy.Enhancer.generateClass(Enhancer.java:483) > at > org.mockito.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25) > at > org.mockito.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:217) > at org.mockito.cglib.proxy.Enhancer.createHelper(Enhancer.java:378) > at org.mockito.cglib.proxy.Enhancer.createClass(Enhancer.java:318) > at > org.mockito.internal.creation.jmock.ClassImposterizer.createProxyClass(ClassImposterizer.java:93) > at > org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:50) > at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:54) > at org.mockito.internal.MockitoCore.mock(MockitoCore.java:45) > at org.mockito.Mockito.mock(Mockito.java:921) > at org.mockito.Mockito.mock(Mockito.java:816) > at > org.apache.hadoop.mapreduce.v2.hs.TestJobListCache.testAddExisting(TestJobListCache.java:42) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6738) TestJobListCache.testAddExisting failed intermittently in slow VM testbed
[ https://issues.apache.org/jira/browse/MAPREDUCE-6738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated MAPREDUCE-6738: - Summary: TestJobListCache.testAddExisting failed intermittently in slow VM testbed (was: TestJobListCache.testAddExisting failed intermiddently in slow VM testbed) > TestJobListCache.testAddExisting failed intermittently in slow VM testbed > - > > Key: MAPREDUCE-6738 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6738 > Project: Hadoop Map/Reduce > Issue Type: Test >Affects Versions: 2.7.3 >Reporter: Junping Du >Assignee: Junping Du >Priority: Minor > Attachments: MAPREDUCE-6738.patch > > > Kick off Jenkins test which occasionally failed for this test with stack > trace below: > org.apache.hadoop.mapreduce.v2.hs.TestJobListCache.testAddExisting > java.lang.Exception: test timed out after 1000 milliseconds > at org.mockito.cglib.proxy.Enhancer.generateClass(Enhancer.java:483) > at > org.mockito.cglib.core.DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25) > at > org.mockito.cglib.core.AbstractClassGenerator.create(AbstractClassGenerator.java:217) > at org.mockito.cglib.proxy.Enhancer.createHelper(Enhancer.java:378) > at org.mockito.cglib.proxy.Enhancer.createClass(Enhancer.java:318) > at > org.mockito.internal.creation.jmock.ClassImposterizer.createProxyClass(ClassImposterizer.java:93) > at > org.mockito.internal.creation.jmock.ClassImposterizer.imposterise(ClassImposterizer.java:50) > at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:54) > at org.mockito.internal.MockitoCore.mock(MockitoCore.java:45) > at org.mockito.Mockito.mock(Mockito.java:921) > at org.mockito.Mockito.mock(Mockito.java:816) > at > org.apache.hadoop.mapreduce.v2.hs.TestJobListCache.testAddExisting(TestJobListCache.java:42) -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6724) Unsafe conversion from long to int in MergeManagerImpl.unconditionalReserve()
[ https://issues.apache.org/jira/browse/MAPREDUCE-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15388146#comment-15388146 ] Gera Shegalov commented on MAPREDUCE-6724: -- [~haibochen], do you want to keep your test, or do you think my more minor modification (verifyReserveLargeMapOutput) of the existing test covers the regression as well and we can use it instead with the benefit of explicitly testing MAX_VALUE? > Unsafe conversion from long to int in MergeManagerImpl.unconditionalReserve() > - > > Key: MAPREDUCE-6724 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6724 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Haibo Chen >Assignee: Haibo Chen > Attachments: mapreduce6724.001.patch, mapreduce6724.002.patch, > mapreduce6724.003.patch, mapreduce6724.004.patch, mapreduce6724.005.patch, > mapreduce6724.006.patch > > > When shuffle is done in memory, MergeManagerImpl converts the requested size > to an int to allocate an instance of InMemoryMapOutput. This results in an > overflow if the requested size is bigger than Integer.MAX_VALUE and > eventually causes the reducer to fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6669) Jobs with encrypted spills don't tolerate AM failures
[ https://issues.apache.org/jira/browse/MAPREDUCE-6669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen resolved MAPREDUCE-6669. --- Resolution: Duplicate > Jobs with encrypted spills don't tolerate AM failures > - > > Key: MAPREDUCE-6669 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6669 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.8.0 >Reporter: Karthik Kambatla >Assignee: Haibo Chen >Priority: Critical > > The key used for encrypting intermediate data is not persisted anywhere, and > hence can't be recovered the same way other MR jobs can be. We should support > recovering these jobs as well, hopefully without having to re-run completed > tasks. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6740) Enforce mapreduce.task.timeout to be at least mapreduce.task.progress-report.interval
[ https://issues.apache.org/jira/browse/MAPREDUCE-6740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6740: -- Attachment: mapreduce6740.002.patch Uploading new patch to fix the checkstyle issue reported. > Enforce mapreduce.task.timeout to be at least > mapreduce.task.progress-report.interval > - > > Key: MAPREDUCE-6740 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6740 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mr-am >Affects Versions: 2.8.0 >Reporter: Haibo Chen >Assignee: Haibo Chen >Priority: Minor > Attachments: mapreduce6740.001.patch, mapreduce6740.002.patch > > > Mapreduce-6242 makes task status update interval configurable to ease the > pressure on MR AM to process status updates, but it did not ensure that > mapreduce.task.timeout is no smaller than the configured value of task report > interval. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387755#comment-15387755
]
Tatyana But commented on MAPREDUCE-6737:
[~vinodkv], could you please check the patch and commit if everything is ok ?
I'm asking you because you was the last person who patched JobIndexInfo class.
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387734#comment-15387734
]
Hadoop QA commented on MAPREDUCE-6737:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 27s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
50s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 23s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
36s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
18s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 20s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
42s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 41s
{color} | {color:green} hadoop-mapreduce-client-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
18s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 10s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12819317/MAPREDUCE-6737.patch |
| JIRA Issue | MAPREDUCE-6737 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 7891073789e1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 557a245 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6636/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6636/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
>
[jira] [Commented] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387678#comment-15387678
]
Hadoop QA commented on MAPREDUCE-6737:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
3s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
10s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
45s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 42s
{color} | {color:green} hadoop-mapreduce-client-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 27s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12819317/MAPREDUCE-6737.patch |
| JIRA Issue | MAPREDUCE-6737 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux bc6887e72c5f 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 557a245 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6635/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6635/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
>
[jira] [Commented] (MAPREDUCE-6380) AggregatedLogDeletionService will throw exception when there are some other directories in remote-log-dir
[
https://issues.apache.org/jira/browse/MAPREDUCE-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387643#comment-15387643
]
Kai Sasaki commented on MAPREDUCE-6380:
---
[~varun_saxena]
Sorry I overlooked the inside of {{deleteOldLogDirsFrom}}. As you said, it is
not necessary to call {{listStatus}} or {{listStatusIterator}} since it is
called inside the method. Moreover {{deleteOldLogDirsFrom}} handles
{{IOException}} including {{FileNotFoundException}}, it seems no chance to skip
succeeding user log dirs.
Sorry about that and thank you so much for pointing out!
> AggregatedLogDeletionService will throw exception when there are some other
> directories in remote-log-dir
> -
>
> Key: MAPREDUCE-6380
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6380
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: Zhang Wei
>Assignee: Kai Sasaki
>Priority: Trivial
> Attachments: MAPREDUCE-6380.01.patch, MAPREDUCE-6380.02.patch,
> MAPREDUCE-6380.03.patch, MAPREDUCE-6380.04.patch, MAPREDUCE-6380.05.patch,
> MAPREDUCE-6380.06.patch, MAPREDUCE-6380.07.patch
>
>
> AggregatedLogDeletionService will throw FileNotFoundException when there are
> some extraneous directories put in remote-log-dir. The deletion function will
> try to listStatus against the "extraneous-dir + suffix" dir. I think it
> would be better if the function can ignore these directories.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Status: Patch Available (was: Open)
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.5.1, 2.7.0
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Attachment: MAPREDUCE-6737.patch
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Attachment: (was: MAPREDUCE-6737.patch)
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Status: Open (was: Patch Available)
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.5.1, 2.7.0
>Reporter: Roman Gavryliuk
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387520#comment-15387520
]
Hadoop QA commented on MAPREDUCE-6737:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
52s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 20s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
19s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 16s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 10s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common:
The patch generated 3 new + 9 unchanged - 0 fixed = 12 total (was 9) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 21s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
9s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
43s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 41s
{color} | {color:green} hadoop-mapreduce-client-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s {color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 17s {color}
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:9560f25 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12819301/MAPREDUCE-6737.patch |
| JIRA Issue | MAPREDUCE-6737 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux dd6147281d89 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 557a245 |
| Default Java | 1.8.0_91 |
| findbugs | v3.0.0 |
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6634/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-common.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6634/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6634/console |
| Powered by | Apache Yetus 0.3.0 http://yetus.apache.org |
This message was automatically generated.
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
>
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Status: Patch Available (was: Open)
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.5.1, 2.7.0
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Attachment: MAPREDUCE-6737.patch
Test added.
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Attachment: (was: MAPREDUCE-6737.patch)
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.7.0, 2.5.1
>Reporter: Roman Gavryliuk
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6737) HS: job history recovery fails with NumericFormatException if the job wasn't initted properly
[
https://issues.apache.org/jira/browse/MAPREDUCE-6737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roman Gavryliuk updated MAPREDUCE-6737:
---
Status: Open (was: Patch Available)
> HS: job history recovery fails with NumericFormatException if the job wasn't
> initted properly
> -
>
> Key: MAPREDUCE-6737
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6737
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Affects Versions: 2.5.1, 2.7.0
>Reporter: Roman Gavryliuk
> Attachments: MAPREDUCE-6737.patch
>
>
> The problem shows itself while recovering old apps information:
> 2016-07-18 16:08:35,031 WARN
> org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils: Unable to parse
> start time from job history file
> job_1468716177837_21790-1468845880296-username-applicationname-1468845889100-0-0-FAILED-root.queuename--1.jhist
> : java.lang.NumberFormatException: For input string:
> ""
> The problem is in JobHistoryEventHandler.java class in the
> following part of code:
> //initialize the launchTime in the JobIndexInfo of MetaInfo
> if(event.getHistoryEvent().getEventType() == EventType.JOB_INITED ){
> JobInitedEvent jie = (JobInitedEvent) event.getHistoryEvent();
> mi.getJobIndexInfo().setJobStartTime(jie.getLaunchTime());
> Because of job was not initialized properly, the 'if' statement takes value
> 'false' and .setJobStartTime() is not called.
> In JobIndexInfo constructor, we have a default value for jobStartTime:
> this.jobStartTime = -1;
> When history server recovers any application's info, it passes all parameters
> to array of strings jobDetails:
> String[] jobDetails = fileName.split(DELIMITER);
> Please note, DELIMETER is initialized in the following way:
> static final String DELIMITER = "-";
> So, jobDetails array has 10 elements - job ID, submit time, username, job
> name,
> finish time, number of maps, number of reducers, job status, queue, and start
> time).
> If jobStartTime = -1, the minus symbol is considered as delimeter and the code
> will assign an empty string "" as a value for 9-th element in jobDetails
> array.
> In org.apache.hadoop.mapreduce.v2.jobhistory.FileNameIndexUtils class a
> NumberFormatException will appear while trying to parse empty string to long
> type.
> Long.parseLong(decodeJobHistoryFileName(jobDetails[JOB_START_TIME_INDEX])));
> The most simple fix is to change the value this.jobStartTime to 0 in
> JobIndexInfo constructor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6679) on node failure, only restart mappers whose output is not copied
[ https://issues.apache.org/jira/browse/MAPREDUCE-6679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15387330#comment-15387330 ] Haibo Chen commented on MAPREDUCE-6679: --- [~alvin.ch...@gmail.com] We could avoid rescheduling a succeeded map task on a bad node only if all reducers have copied its output. Shuffle requests are served by ShuffleHandlers on NM that do not communicate with MR AM, so I believe MR AM has no way to tell if all reducers have fetched from a map task. On the other hand, even if all reducers have copied output from a succeeded mapper, there is still a possibility that we need to reschedule that map task. Like you said, any of the reducers can fail after they have copied the output, the second attempt of the failed reducer task will likely fail as well because map output is on a bad node. > on node failure, only restart mappers whose output is not copied > > > Key: MAPREDUCE-6679 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6679 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Affects Versions: 2.7.0 >Reporter: Alvin Chyan >Priority: Minor > > When we detect a bad node, we reschedule all succeeded map tasks on that node > in JobImpl.actOnUnusableNode. Wouldn't we be able to get away with only > rescheduling the map tasks that have not had their outputs copied to a > reducer already? > One consideration could be that the reducer that fetched the mapper output is > then killed itself. However, in testing, it seems that once a reducer has > moved past the shuffle phase and is reducing, even if the mapper node fails, > the mappers don't get rescheduled. The same mechanism that occurs then if a > reducer dies can then be applied in this scenario. > This is helpful in general, but is especially beneficial in cloud > environments that offer spot/preemptible instances. As long as reducers are > running to continually fetch mapper outputs, the job can make progress as > long as the preemptible instances stay up long enough for a map task to > complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-5739) DirectoryCollection#createNonExistentDirs() may use an invalid iterator
[
https://issues.apache.org/jira/browse/MAPREDUCE-5739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen resolved MAPREDUCE-5739.
---
Resolution: Fixed
> DirectoryCollection#createNonExistentDirs() may use an invalid iterator
> ---
>
> Key: MAPREDUCE-5739
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5739
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Ted Yu
>
> Here is related code:
> {code}
> for (final String dir : localDirs) {
> try {
> createDir(localFs, new Path(dir), perm);
> } catch (IOException e) {
> LOG.warn("Unable to create directory " + dir + " error " +
> e.getMessage() + ", removing from the list of valid
> directories.");
> localDirs.remove(dir);
> {code}
> Call to localDirs.remove() modifies Iterable "localDirs" which invalidates
> the iterator.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-5752) Potential invalid iterator in NMClientImpl#cleanupRunningContainers()
[
https://issues.apache.org/jira/browse/MAPREDUCE-5752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Haibo Chen resolved MAPREDUCE-5752.
---
Resolution: Invalid
> Potential invalid iterator in NMClientImpl#cleanupRunningContainers()
> -
>
> Key: MAPREDUCE-5752
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5752
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> In cleanupRunningContainers() :
> {code}
> for (StartedContainer startedContainer : startedContainers.values()) {
> try {
> stopContainer(startedContainer.getContainerId(),
> startedContainer.getNodeId());
> {code}
> Removal of container is done in removeStartedContainer():
> {code}
> startedContainers.remove(container.containerId);
> {code}
> This may result in invalid iterator for the loop on startedContainers.values()
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
