[jira] [Commented] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106960#comment-16106960
]
Hadoop QA commented on MAPREDUCE-6921:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}103m
33s{color} | {color:green} hadoop-mapreduce-client-jobclient in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}123m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | MAPREDUCE-6921 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879567/MAPREDUCE-6921.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 2547d093149c 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 481385e |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7031/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7031/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build
[jira] [Commented] (MAPREDUCE-6892) Issues with the count of failed/killed tasks in the jhist file
[
https://issues.apache.org/jira/browse/MAPREDUCE-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107906#comment-16107906
]
Haibo Chen commented on MAPREDUCE-6892:
---
Thanks for bringing out the compatibility issue. Let's keep the names and just
add new fields.
> Issues with the count of failed/killed tasks in the jhist file
> --
>
> Key: MAPREDUCE-6892
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6892
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client, jobhistoryserver
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
> Attachments: MAPREDUCE-6892-001.patch, MAPREDUCE-6892-002.PATCH
>
>
> Recently we encountered some issues with the value of failed tasks. After
> parsing the jhist file, {{JobInfo.getFailedMaps()}} returned 0, but actually
> there were failures.
> Another minor thing is that you cannot get the number of killed tasks
> (although this can be calculated).
> The root cause is that {{JobUnsuccessfulCompletionEvent}} contains only the
> successful map/reduce task counts. Number of failed (or killed) tasks are not
> stored.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Attachment: MAPREDUCE-6921.patch
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Attachment: (was: MAPREDUCE-6921.patch)
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Status: Patch Available (was: Open)
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Attachment: MAPREDUCE-6921-2.patch
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Attachment: (was: MAPREDUCE-6921-2.patch)
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Status: Open (was: Patch Available)
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106852#comment-16106852
]
Sonia Garudi commented on MAPREDUCE-6921:
-
Thanks [~ajisakaa] and [~vincent he] for the review. I have updated the patch.
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Szegedi updated MAPREDUCE-6922:
--
Attachment: YARN-6922.branch-2.000.patch
Attached a revert patch.
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107986#comment-16107986
]
Junping Du commented on MAPREDUCE-5875:
---
As discussed in MAPREDUCE-6288, I have revert this patch from trunk and
branch-2. We need to reopen this jira for an improved solution. But it looks
like I cannot reopen it - can someone able to do this?
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6286:
--
Fix Version/s: (was: 3.0.0-alpha1)
(was: 2.9.0)
> A typo in HistoryViewer makes some code useless, which causes counter limits
> are not reset correctly.
> -
>
> Key: MAPREDUCE-6286
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Attachments: MAPREDUCE-6286.000.patch
>
>
> A typo in HistoryViewer makes some code useless and it causes counter limits
> are not reset correctly.
> The typo is
> Limits.reset(conf);
> We should use jobConf instead of conf.
> With the typo, the following code becomes useless:
> {code}
> final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
> + "_" + jobDetails[1] + "_" + jobDetails[2] + "_conf.xml");
> final Configuration jobConf = new Configuration(conf);
> jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
> {code}
> The code wants to load the configuration from the Job configuration file and
> reset the Limits based on the new configuration loaded from the Job
> configuration file. But with the typo, the Limits are reset with the old
> configuration.
> So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6199) AbstractCounters are not reset completely on deserialization
[ https://issues.apache.org/jira/browse/MAPREDUCE-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du resolved MAPREDUCE-6199. --- Resolution: Won't Fix > AbstractCounters are not reset completely on deserialization > > > Key: MAPREDUCE-6199 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6199 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Anubhav Dhoot >Assignee: Anubhav Dhoot > Attachments: mr-6199.001.patch, mr-6199.001.patch, mr-6199.002.patch > > > AbstractCounters are partially reset on deserialization. This patch > completely resets them. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6199) AbstractCounters are not reset completely on deserialization
[ https://issues.apache.org/jira/browse/MAPREDUCE-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107992#comment-16107992 ] Junping Du commented on MAPREDUCE-6199: --- MAPREDUCE-5875 is revert, so patch here is not needed. reopen and resolve it as won't fix. > AbstractCounters are not reset completely on deserialization > > > Key: MAPREDUCE-6199 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6199 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Anubhav Dhoot >Assignee: Anubhav Dhoot > Fix For: 2.8.0, 3.0.0-alpha1 > > Attachments: mr-6199.001.patch, mr-6199.001.patch, mr-6199.002.patch > > > AbstractCounters are partially reset on deserialization. This patch > completely resets them. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated MAPREDUCE-6922:
--
Target Version/s: 2.9.0
Looks like FrameworkCounterGroup needs to be updated to protect itself from
out-of-bounds counter indices coming from the task report. Until that has been
rolled out as a baseline we can't ever safely add new counters without breaking
this code (or using a distributed cache deploy for MapReduce, see below).
Ultimately moving to protocol buffers would be even better, but that's not
something we could accomplish until a major release like Hadoop 3.0.
I should point out that jobs would not be susceptible to this failure if they
deployed MapReduce via HDFS rather than picking it up from the nodes. See
http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/DistributedCacheDeploy.html.
That deployment method has the benefit that a job always runs with a
consistent version of MapReduce, and therefore changes like this _would_ be OK
during rolling upgrade. Jobs would either run entirely with the old MapReduce
version or the new one instead of a hodgepodge of both which could lead to
errors like this.
If distributed cache deploy is not sufficient and this still needs to be
reverted then normally we would simply revert MAPREDUCE-6829 from branch-2,
remove 2.9.0 from the fix version of that JIRA, and mark this as a duplicate of
that JIRA. 2.9.0 never shipped, so we don't need a tracking JIRA to note
something was removed since we never shipped a 2.x release saying it was added.
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108080#comment-16108080
]
Junping Du commented on MAPREDUCE-5875:
---
bq. At a minimum, -beta1 will need to include a note that says it was removed
or, preferably, removed but replaced with something better.
I believe MAPREDUCE-6924 already address this.
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du reassigned MAPREDUCE-5875:
-
Assignee: Junping Du (was: Gera Shegalov)
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Junping Du
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6288) mapred job -status fails with AccessControlException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du resolved MAPREDUCE-6288.
---
Resolution: Done
I have revert MAPREDUCE-6286, MAPREDUCE-6199 and MAPREDUCE-5875 from trunk and
branch-2. Resolve this jira as Done as we will reopen MAPREDUCE-5875 for a
better solution.
> mapred job -status fails with AccessControlException
> -
>
> Key: MAPREDUCE-6288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Priority: Blocker
> Attachments: MAPREDUCE-6288.002.patch, MAPREDUCE-6288-gera-001.patch,
> MAPREDUCE-6288.patch
>
>
> After MAPREDUCE-5875, we're seeing this Exception when trying to do {{mapred
> job -status job_1427080398288_0001}}
> {noformat}
> Exception in thread "main" org.apache.hadoop.security.AccessControlException:
> Permission denied: user=jenkins, access=EXECUTE,
> inode="/user/history/done":mapred:hadoop:drwxrwx---
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:257)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:238)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:180)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:137)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:138)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6553)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6535)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPathAccess(FSNamesystem.java:6460)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1919)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1870)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1850)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1822)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:545)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:87)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1213)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1201)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1191)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:299)
> at
> org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:265)
> at org.apache.hadoop.hdfs.DFSInputStream.(DFSInputStream.java:257)
> at
[jira] [Updated] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Miklos Szegedi updated MAPREDUCE-6922:
--
Status: Patch Available (was: Open)
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du resolved MAPREDUCE-6286.
---
Resolution: Won't Fix
MAPREDUCE-5875 is revert, so patch here is not needed. reopen and resolve it as
won't fix.
> A typo in HistoryViewer makes some code useless, which causes counter limits
> are not reset correctly.
> -
>
> Key: MAPREDUCE-6286
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-6286.000.patch
>
>
> A typo in HistoryViewer makes some code useless and it causes counter limits
> are not reset correctly.
> The typo is
> Limits.reset(conf);
> We should use jobConf instead of conf.
> With the typo, the following code becomes useless:
> {code}
> final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
> + "_" + jobDetails[1] + "_" + jobDetails[2] + "_conf.xml");
> final Configuration jobConf = new Configuration(conf);
> jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
> {code}
> The code wants to load the configuration from the Job configuration file and
> reset the Limits based on the new configuration loaded from the Job
> configuration file. But with the typo, the Limits are reset with the old
> configuration.
> So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Reopened] (MAPREDUCE-6286) A typo in HistoryViewer makes some code useless, which causes counter limits are not reset correctly.
[
https://issues.apache.org/jira/browse/MAPREDUCE-6286?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du reopened MAPREDUCE-6286:
---
> A typo in HistoryViewer makes some code useless, which causes counter limits
> are not reset correctly.
> -
>
> Key: MAPREDUCE-6286
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6286
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.6.0
>Reporter: zhihai xu
>Assignee: zhihai xu
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-6286.000.patch
>
>
> A typo in HistoryViewer makes some code useless and it causes counter limits
> are not reset correctly.
> The typo is
> Limits.reset(conf);
> We should use jobConf instead of conf.
> With the typo, the following code becomes useless:
> {code}
> final Path jobConfPath = new Path(jobFile.getParent(), jobDetails[0]
> + "_" + jobDetails[1] + "_" + jobDetails[2] + "_conf.xml");
> final Configuration jobConf = new Configuration(conf);
> jobConf.addResource(fs.open(jobConfPath), jobConfPath.toString());
> {code}
> The code wants to load the configuration from the Job configuration file and
> reset the Limits based on the new configuration loaded from the Job
> configuration file. But with the typo, the Limits are reset with the old
> configuration.
> So this typo is apparent.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6923) YARN Shuffle I/O for small partitions
Robert Schmidtke created MAPREDUCE-6923:
---
Summary: YARN Shuffle I/O for small partitions
Key: MAPREDUCE-6923
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6923
Project: Hadoop Map/Reduce
Issue Type: Improvement
Environment: Observed in Hadoop 2.7.3 and above (judging from the
source code of future versions), and Ubuntu 16.04.
Reporter: Robert Schmidtke
When a job configuration results in small partitions read by each reducer from
each mapper (e.g. 65 kilobytes as in my setup: a
[TeraSort|https://github.com/apache/hadoop/blob/branch-2.7.3/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraSort.java]
of 256 gigabytes using 2048 mappers and reducers each), and setting
{code:xml}
mapreduce.shuffle.transferTo.allowed
false
{code}
then the default setting of
{code:xml}
mapreduce.shuffle.transfer.buffer.size
131072
{code}
results in almost 100% overhead in reads during shuffle in YARN, because for
each 65K needed, 128K are read.
I propose a fix in
[FadvisedFileRegion.java|https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/FadvisedFileRegion.java#L114]
as follows:
{code:java}
ByteBuffer byteBuffer = ByteBuffer.allocate(Math.min(this.shuffleBufferSize,
trans > Integer.MAX_VALUE ? Integer.MAX_VALUE : (int) trans));
{code}
e.g.
[here|https://github.com/apache/hadoop/compare/branch-2.7.3...robert-schmidtke:adaptive-shuffle-buffer].
This sets the shuffle buffer size to the minimum value of the shuffle buffer
size specified in the configuration (128K by default), and the actual partition
size (65K on average in my setup). In my benchmarks this reduced the read
overhead in YARN from about 100% (255 additional gigabytes as described above)
down to about 18% (an additional 45 gigabytes). The runtime of the job remained
the same in my setup.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Reopened] (MAPREDUCE-6199) AbstractCounters are not reset completely on deserialization
[ https://issues.apache.org/jira/browse/MAPREDUCE-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du reopened MAPREDUCE-6199: --- > AbstractCounters are not reset completely on deserialization > > > Key: MAPREDUCE-6199 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6199 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Anubhav Dhoot >Assignee: Anubhav Dhoot > Attachments: mr-6199.001.patch, mr-6199.001.patch, mr-6199.002.patch > > > AbstractCounters are partially reset on deserialization. This patch > completely resets them. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6199) AbstractCounters are not reset completely on deserialization
[ https://issues.apache.org/jira/browse/MAPREDUCE-6199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Junping Du updated MAPREDUCE-6199: -- Fix Version/s: (was: 3.0.0-alpha1) (was: 2.8.0) > AbstractCounters are not reset completely on deserialization > > > Key: MAPREDUCE-6199 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6199 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Anubhav Dhoot >Assignee: Anubhav Dhoot > Attachments: mr-6199.001.patch, mr-6199.001.patch, mr-6199.002.patch > > > AbstractCounters are partially reset on deserialization. This patch > completely resets them. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated MAPREDUCE-6922:
--
Resolution: Duplicate
Status: Resolved (was: Patch Available)
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108026#comment-16108026
]
Robert Kanter commented on MAPREDUCE-5875:
--
[~djp], it's "Closed" so I don't think we can reopen it. We should probably
just open up a new JIRA to address this issue.
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6829) Add peak memory usage counter for each task
[ https://issues.apache.org/jira/browse/MAPREDUCE-6829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-6829: -- Fix Version/s: (was: 2.9.0) I reverted this from branch-2 due to the backward incompatibility that can manifest during rolling upgrades while a job is running. See the discussion in MAPREDUCE-6922. > Add peak memory usage counter for each task > --- > > Key: MAPREDUCE-6829 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6829 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Reporter: Yufei Gu >Assignee: Miklos Szegedi > Fix For: 3.0.0-alpha4 > > Attachments: MAPREDUCE-6829.000.patch, MAPREDUCE-6829.001.patch, > MAPREDUCE-6829.002.patch, MAPREDUCE-6829.003.patch, MAPREDUCE-6829.004.patch, > MAPREDUCE-6829.005.patch > > > Each task has counters PHYSICAL_MEMORY_BYTES and VIRTUAL_MEMORY_BYTES, which > are snapshots of memory usage of that task. They are not sufficient for users > to understand peak memory usage by that task, e.g. in order to diagnose task > failures, tune job parameters or change application design. This new feature > will add two more counters for each task: PHYSICAL_MEMORY_BYTES_MAX and > VIRTUAL_MEMORY_BYTES_MAX. > This JIRA has the same feature from MAPREDUCE-4710. I file this new YARN > JIRA since MAPREDUCE-4710 is pretty old one from MR 1.x era, it more or less > assumes a branch-1 architecture, should be close at this point. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108033#comment-16108033
]
Miklos Szegedi commented on MAPREDUCE-6922:
---
Thank you, [~jlowe]!
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108045#comment-16108045
]
Allen Wittenauer commented on MAPREDUCE-5875:
-
It's closed because it was already in a release. You absolutely need to open a
new JIRA for it.
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6925) CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
Junping Du created MAPREDUCE-6925:
-
Summary: CLONE - Make Counter limits consistent across JobClient,
MRAppMaster, and YarnChild
Key: MAPREDUCE-6925
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6925
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: applicationmaster, client, task
Affects Versions: 2.4.0
Reporter: Gera Shegalov
Assignee: Gera Shegalov
Fix For: 2.9.0, 3.0.0-alpha1
Currently, counter limits "mapreduce.job.counters.*" handled by
{{org.apache.hadoop.mapreduce.counters.Limits}} are initialized asymmetrically:
on the client side, and on the AM, job.xml is ignored whereas it's taken into
account in YarnChild.
It would be good to make the Limits job-configurable, such that max
counters/groups is only increased when needed. With the current Limits
implementation relying on static constants, it's going to be challenging for
tools that submit jobs concurrently without resorting to class loading
isolation.
The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
Miklos Szegedi created MAPREDUCE-6922:
-
Summary: MapReduce jobs may fail during rolling upgrade due to
MAPREDUCE-6829
Key: MAPREDUCE-6922
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
Project: Hadoop Map/Reduce
Issue Type: Bug
Reporter: Miklos Szegedi
Assignee: Miklos Szegedi
Priority: Blocker
MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
{code}
2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
org.apache.hadoop.ipc.Server: Unable to read call parameters for client
10.17.242.22on connection protocol
org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
java.lang.ArrayIndexOutOfBoundsException: 23
at
org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
at
org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
at
org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
at
org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
at
org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
at
org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
at
org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
at
org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
at
org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6288) mapred job -status fails with AccessControlException
[
https://issues.apache.org/jira/browse/MAPREDUCE-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108021#comment-16108021
]
Andrew Wang commented on MAPREDUCE-6288:
Thanks for handling the reverts Junping. I filed and linked MAPREDUCE-6924 so
the reverts show up in the beta1 changelog, since I believe they were included
in the alpha releases.
> mapred job -status fails with AccessControlException
> -
>
> Key: MAPREDUCE-6288
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6288
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 2.8.0
>Reporter: Robert Kanter
>Priority: Blocker
> Attachments: MAPREDUCE-6288.002.patch, MAPREDUCE-6288-gera-001.patch,
> MAPREDUCE-6288.patch
>
>
> After MAPREDUCE-5875, we're seeing this Exception when trying to do {{mapred
> job -status job_1427080398288_0001}}
> {noformat}
> Exception in thread "main" org.apache.hadoop.security.AccessControlException:
> Permission denied: user=jenkins, access=EXECUTE,
> inode="/user/history/done":mapred:hadoop:drwxrwx---
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkFsPermission(DefaultAuthorizationProvider.java:257)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.check(DefaultAuthorizationProvider.java:238)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkTraverse(DefaultAuthorizationProvider.java:180)
> at
> org.apache.hadoop.hdfs.server.namenode.DefaultAuthorizationProvider.checkPermission(DefaultAuthorizationProvider.java:137)
> at
> org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkPermission(FSPermissionChecker.java:138)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6553)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPermission(FSNamesystem.java:6535)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkPathAccess(FSNamesystem.java:6460)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1919)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1870)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1850)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1822)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:545)
> at
> org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:87)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:73)
> at
> org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:1213)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1201)
> at
> org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:1191)
> at
> org.apache.hadoop.hdfs.DFSInputStream.fetchLocatedBlocksAndGetLastBlockLength(DFSInputStream.java:299)
> at
> org.apache.hadoop.hdfs.DFSInputStream.openInfo(DFSInputStream.java:265)
> at
[jira] [Commented] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108052#comment-16108052
]
Junping Du commented on MAPREDUCE-5875:
---
Drop 2.9.0 as the patch get reverted from branch-2. Just clone MAPREDUCE-6924
for improved fix.
To be clear, this fix never land in any non-alpha release, so we shouldn't
claim the fix is released or any regression due to revert of this fix.
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108068#comment-16108068
]
Allen Wittenauer commented on MAPREDUCE-5875:
-
bq. To be clear, this fix never land in any non-alpha release, so we shouldn't
claim the fix is released or any regression due to revert of this fix.
Doesn't matter. The release notes for alpha1 said that this patch is there. At
a minimum, -beta1 will need to include a note that says it was removed or,
preferably, removed but replaced with something better.
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du reassigned MAPREDUCE-5875:
-
Assignee: Gera Shegalov (was: Junping Du)
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108008#comment-16108008
]
Hudson commented on MAPREDUCE-5875:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12080 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/12080/])
Revert "MAPREDUCE-5875. Make Counter limits consistent across JobClient,
(junping_du: rev fbb7d6bcbb887ce52ab1e9d5a1fed67a7f8a4be8)
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/JobSubmitter.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/CompletedJob.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Cluster.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/counters/Limits.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HistoryViewer.java
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 2.9.0, 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108011#comment-16108011
]
Miklos Szegedi commented on MAPREDUCE-6922:
---
Thank you, Jason. I agree the distributed cache deployment solves this the
issue, however I am not sure how many users use the other option. I think we
should revert from branch-2. My understanding is that there is no rolling
upgrade requirement from 2.x to 3.0.
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6922) MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
[
https://issues.apache.org/jira/browse/MAPREDUCE-6922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108014#comment-16108014
]
Jason Lowe commented on MAPREDUCE-6922:
---
OK I'll revert MAPREDUCE-6829 from branch-2 and update that JIRA accordingly.
> MapReduce jobs may fail during rolling upgrade due to MAPREDUCE-6829
>
>
> Key: MAPREDUCE-6922
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6922
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Miklos Szegedi
>Assignee: Miklos Szegedi
>Priority: Blocker
> Attachments: YARN-6922.branch-2.000.patch
>
>
> MAPREDUCE-6829 should be reverted from branch-2 because rolling upgrade fails.
> {code}
> 2017-06-08 17:43:37,173 WARN [Socket Reader #1 for port 41187]
> org.apache.hadoop.ipc.Server: Unable to read call parameters for client
> 10.17.242.22on connection protocol
> org.apache.hadoop.mapred.TaskUmbilicalProtocol for rpcKind RPC_WRITABLE
> java.lang.ArrayIndexOutOfBoundsException: 23
> at
> org.apache.hadoop.mapreduce.counters.FrameworkCounterGroup.readFields(FrameworkCounterGroup.java:261)
> at org.apache.hadoop.mapred.Counters$Group.readFields(Counters.java:324)
> at
> org.apache.hadoop.mapreduce.counters.AbstractCounters.readFields(AbstractCounters.java:306)
> at org.apache.hadoop.mapred.TaskStatus.readFields(TaskStatus.java:489)
> at
> org.apache.hadoop.mapred.MapTaskStatus.readFields(MapTaskStatus.java:88)
> at
> org.apache.hadoop.io.ObjectWritable.readObject(ObjectWritable.java:285)
> at
> org.apache.hadoop.ipc.WritableRpcEngine$Invocation.readFields(WritableRpcEngine.java:160)
> at
> org.apache.hadoop.ipc.Server$Connection.processRpcRequest(Server.java:1909)
> at
> org.apache.hadoop.ipc.Server$Connection.processOneRpc(Server.java:1841)
> at
> org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:1600)
> at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:820)
> at
> org.apache.hadoop.ipc.Server$Listener$Reader.doRunLoop(Server.java:693)
> at org.apache.hadoop.ipc.Server$Listener$Reader.run(Server.java:664)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-6924) Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875
[ https://issues.apache.org/jira/browse/MAPREDUCE-6924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Wang resolved MAPREDUCE-6924. Resolution: Fixed Fix Version/s: 3.0.0-beta1 Resolving this changelog tracking JIRA. Thanks to [~djp] for doing the reverts! > Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875 > --- > > Key: MAPREDUCE-6924 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6924 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 3.0.0-alpha1 >Reporter: Andrew Wang >Assignee: Junping Du > Fix For: 3.0.0-beta1 > > > Filing this JIRA so the reverts show up in the changelog. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-6924) Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875
Andrew Wang created MAPREDUCE-6924: -- Summary: Revert MAPREDUCE-6199 MAPREDUCE-6286 and MAPREDUCE-5875 Key: MAPREDUCE-6924 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6924 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 3.0.0-alpha1 Reporter: Andrew Wang Assignee: Junping Du Filing this JIRA so the reverts show up in the changelog. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-5875:
--
Fix Version/s: (was: 2.9.0)
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (MAPREDUCE-5875) Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-5875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108052#comment-16108052
]
Junping Du edited comment on MAPREDUCE-5875 at 7/31/17 10:05 PM:
-
Drop 2.9.0 as the patch get reverted from branch-2. Just clone MAPREDUCE-6925
for improved fix.
To be clear, this fix never land in any non-alpha release, so we shouldn't
claim the fix is released or any regression due to revert of this fix.
was (Author: djp):
Drop 2.9.0 as the patch get reverted from branch-2. Just clone MAPREDUCE-6924
for improved fix.
To be clear, this fix never land in any non-alpha release, so we shouldn't
claim the fix is released or any regression due to revert of this fix.
> Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
> ---
>
> Key: MAPREDUCE-5875
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5875
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
> Fix For: 3.0.0-alpha1
>
> Attachments: MAPREDUCE-5875.v01.patch, MAPREDUCE-5875.v02.patch,
> MAPREDUCE-5875.v03.patch, MAPREDUCE-5875.v04.patch, MAPREDUCE-5875.v05.patch,
> MAPREDUCE-5875.v06.patch, MAPREDUCE-5875.v07.patch, MAPREDUCE-5875.v08.patch,
> MAPREDUCE-5875.v09.patch
>
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6925) CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6925:
--
Hadoop Flags: (was: Reviewed)
> CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and
> YarnChild
> ---
>
> Key: MAPREDUCE-6925
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6925
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6925) CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6925:
--
Target Version/s: 2.9.0, 3.0.0-beta1 (was: 2.7.0)
> CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and
> YarnChild
> ---
>
> Key: MAPREDUCE-6925
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6925
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6925) CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and YarnChild
[
https://issues.apache.org/jira/browse/MAPREDUCE-6925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Junping Du updated MAPREDUCE-6925:
--
Fix Version/s: (was: 3.0.0-alpha1)
(was: 2.9.0)
> CLONE - Make Counter limits consistent across JobClient, MRAppMaster, and
> YarnChild
> ---
>
> Key: MAPREDUCE-6925
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6925
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: applicationmaster, client, task
>Affects Versions: 2.4.0
>Reporter: Gera Shegalov
>Assignee: Gera Shegalov
>
> Currently, counter limits "mapreduce.job.counters.*" handled by
> {{org.apache.hadoop.mapreduce.counters.Limits}} are initialized
> asymmetrically: on the client side, and on the AM, job.xml is ignored whereas
> it's taken into account in YarnChild.
> It would be good to make the Limits job-configurable, such that max
> counters/groups is only increased when needed. With the current Limits
> implementation relying on static constants, it's going to be challenging for
> tools that submit jobs concurrently without resorting to class loading
> isolation.
> The patch that I am uploading is not perfect but demonstrates the issue.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108313#comment-16108313
]
Wenxin He commented on MAPREDUCE-6921:
--
Thanks for your updating, [~Sonia].
One more nit: would you please indent two spaces per level like this, according
to
[HowToContribute|https://wiki.apache.org/hadoop/HowToContribute#Making_Changes]'s
"Making Changes" section?
{noformat}
static {
-((Log4JLogger) Client.LOG).getLogger().setLevel(Level.ALL);
-((Log4JLogger) Server.LOG).getLogger().setLevel(Level.ALL);
-((Log4JLogger) SaslRpcClient.LOG).getLogger().setLevel(Level.ALL);
-((Log4JLogger) SaslRpcServer.LOG).getLogger().setLevel(Level.ALL);
-((Log4JLogger) SaslInputStream.LOG).getLogger().setLevel(Level.ALL);
+GenericTestUtils.setLogLevel(Client.LOG, Level.TRACE);
+GenericTestUtils.setLogLevel(Server.LOG, Level.TRACE);
+GenericTestUtils.setLogLevel(SaslRpcClient.LOG, Level.TRACE);
+GenericTestUtils.setLogLevel(SaslRpcServer.LOG, Level.TRACE);
+GenericTestUtils.setLogLevel(SaslInputStream.LOG, Level.TRACE);
}
{noformat}
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108317#comment-16108317
]
Wenxin He commented on MAPREDUCE-6921:
--
Strange, [~ajisakaa], MAPREDUCE-precommit job seems do not check this no indent
situation.There's no checkstyle warning or some else warning.
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Status: Open (was: Patch Available)
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Attachment: MAPREDUCE-6921.patch
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Attachment: (was: MAPREDUCE-6921.patch)
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sonia Garudi updated MAPREDUCE-6921:
Status: Patch Available (was: Open)
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108331#comment-16108331
]
Sonia Garudi commented on MAPREDUCE-6921:
-
[~vincent he] I have updated the patch again. Could you please review? Thanks.
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108407#comment-16108407
]
Hadoop QA commented on MAPREDUCE-6921:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green}106m
12s{color} | {color:green} hadoop-mapreduce-client-jobclient in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}126m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:14b5c93 |
| JIRA Issue | MAPREDUCE-6921 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12879766/MAPREDUCE-6921.patch |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 1f9110b706a7 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / a4aa1cb |
| Default Java | 1.8.0_131 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7033/testReport/ |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7033/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build
[jira] [Commented] (MAPREDUCE-6921) TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
[
https://issues.apache.org/jira/browse/MAPREDUCE-6921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16108410#comment-16108410
]
Akira Ajisaka commented on MAPREDUCE-6921:
--
bq. MAPREDUCE-precommit job seems do not check this no indent situation.There's
no checkstyle warning or some else warning.
Umm. It seems Apache Yetus should be upgraded to 0.5.0. (YETUS-489).
LGTM, +1 pending Jenkins.
> TestUmbilicalProtocolWithJobToken#testJobTokenRpc test fails in trunk
> -
>
> Key: MAPREDUCE-6921
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6921
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 3.0.0-beta1
> Environment: Ubuntu 14.04
> x86, ppc64le
> $ java -version
> openjdk version "1.8.0_111"
> OpenJDK Runtime Environment (build 1.8.0_111-8u111-b14-3~14.04.1-b14)
> OpenJDK 64-Bit Server VM (build 25.111-b14, mixed mode)
>Reporter: Sonia Garudi
>Assignee: Sonia Garudi
> Attachments: MAPREDUCE-6921.patch
>
>
> The test fails consistently without the error below :
> {code}
> java.lang.ExceptionInInitializerError: null
> at
> org.apache.hadoop.mapreduce.security.TestUmbilicalProtocolWithJobToken.(TestUmbilicalProtocolWithJobToken.java:76)
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
