[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property mapred.child.java.opts
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267431#comment-14267431 ] Masatake Iwasaki commented on MAPREDUCE-6205: - bq. Uncomment both mapreduce.map|reduce.java.opts and set their value as '-Xmx200m'. [~sam liu], [the comment in MAPREDUCE-5130|https://issues.apache.org/jira/browse/MAPREDUCE-5130?focusedCommentId=13623376page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13623376] says it breaks compatibility. Update the value of the new version properties of the deprecated property mapred.child.java.opts -- Key: MAPREDUCE-6205 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: trunk Reporter: sam liu Assignee: sam liu Priority: Minor Attachments: MAPREDUCE-6205.patch, MAPREDUCE-6205.patch In current hadoop code, the old property mapred.child.java.opts is deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the deprecated property mapred.child.java.opts, hadoop won't automatically update its new versions properties MRJobConfig.MAP_JAVA_OPTS(mapreduce.map.java.opts) and MRJobConfig.REDUCE_JAVA_OPTS(mapreduce.reduce.java.opts). As hadoop will update the new version properties for many other deprecated properties, we also should support such feature on the old property mapred.child.java.opts, otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6206) TestAggregatedTransferRate fails on non-US systems
[
https://issues.apache.org/jira/browse/MAPREDUCE-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267547#comment-14267547
]
Hudson commented on MAPREDUCE-6206:
---
FAILURE: Integrated in Hadoop-Yarn-trunk #800 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk/800/])
MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed
by Jens Rabe (jlowe: rev a0aeed100b839d5c122974bd0ea12f9168ee36a5)
* hadoop-mapreduce-project/CHANGES.txt
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
TestAggregatedTransferRate fails on non-US systems
--
Key: MAPREDUCE-6206
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6206
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: trunk
Reporter: Jens Rabe
Assignee: Jens Rabe
Priority: Critical
Fix For: 2.7.0
Attachments: MAPREDUCE-6206.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
When running the test TestAggregatedTransferRate from
org.apache.hadoop.mapreduce.task.reduce.TestShuffleScheduler on a non-US
system (or, to be more precise, a system which does not use the point as a
decimal separator), the test fails.
E.g. on a German system, the progress message is:
{{copy task(attempt_test__m_00_0 succeeded at 1,00 MB/s) Aggregated
copy rate(1 of 10 at 1,00 MB/s)}}
But the test checks for:
{{copy task(attempt_test__m_00_0 succeeded at 1.00 MB/s) Aggregated
copy rate(1 of 10 at 1.00 MB/s)}}
which causes it to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6206) TestAggregatedTransferRate fails on non-US systems
[
https://issues.apache.org/jira/browse/MAPREDUCE-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267665#comment-14267665
]
Hudson commented on MAPREDUCE-6206:
---
FAILURE: Integrated in Hadoop-Hdfs-trunk #1998 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1998/])
MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed
by Jens Rabe (jlowe: rev a0aeed100b839d5c122974bd0ea12f9168ee36a5)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
* hadoop-mapreduce-project/CHANGES.txt
TestAggregatedTransferRate fails on non-US systems
--
Key: MAPREDUCE-6206
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6206
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: trunk
Reporter: Jens Rabe
Assignee: Jens Rabe
Priority: Critical
Fix For: 2.7.0
Attachments: MAPREDUCE-6206.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
When running the test TestAggregatedTransferRate from
org.apache.hadoop.mapreduce.task.reduce.TestShuffleScheduler on a non-US
system (or, to be more precise, a system which does not use the point as a
decimal separator), the test fails.
E.g. on a German system, the progress message is:
{{copy task(attempt_test__m_00_0 succeeded at 1,00 MB/s) Aggregated
copy rate(1 of 10 at 1,00 MB/s)}}
But the test checks for:
{{copy task(attempt_test__m_00_0 succeeded at 1.00 MB/s) Aggregated
copy rate(1 of 10 at 1.00 MB/s)}}
which causes it to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6206) TestAggregatedTransferRate fails on non-US systems
[
https://issues.apache.org/jira/browse/MAPREDUCE-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267541#comment-14267541
]
Hudson commented on MAPREDUCE-6206:
---
FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #66 (See
[https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/66/])
MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed
by Jens Rabe (jlowe: rev a0aeed100b839d5c122974bd0ea12f9168ee36a5)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
* hadoop-mapreduce-project/CHANGES.txt
TestAggregatedTransferRate fails on non-US systems
--
Key: MAPREDUCE-6206
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6206
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: trunk
Reporter: Jens Rabe
Assignee: Jens Rabe
Priority: Critical
Fix For: 2.7.0
Attachments: MAPREDUCE-6206.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
When running the test TestAggregatedTransferRate from
org.apache.hadoop.mapreduce.task.reduce.TestShuffleScheduler on a non-US
system (or, to be more precise, a system which does not use the point as a
decimal separator), the test fails.
E.g. on a German system, the progress message is:
{{copy task(attempt_test__m_00_0 succeeded at 1,00 MB/s) Aggregated
copy rate(1 of 10 at 1,00 MB/s)}}
But the test checks for:
{{copy task(attempt_test__m_00_0 succeeded at 1.00 MB/s) Aggregated
copy rate(1 of 10 at 1.00 MB/s)}}
which causes it to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property mapred.child.java.opts
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated MAPREDUCE-6205: - Attachment: MAPREDUCE-6205.003.patch I think [~sam liu]'s approach works well now that deprecation deltas are added. [~sam liu], please upload patches for easier review of issues you are discussing. To catch up on this, I took the liberty to capture the current state in 003 patch. I think we should do the same with env properties as well. Update the value of the new version properties of the deprecated property mapred.child.java.opts -- Key: MAPREDUCE-6205 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: trunk Reporter: sam liu Assignee: sam liu Priority: Minor Attachments: MAPREDUCE-6205.003.patch, MAPREDUCE-6205.patch, MAPREDUCE-6205.patch In current hadoop code, the old property mapred.child.java.opts is deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the deprecated property mapred.child.java.opts, hadoop won't automatically update its new versions properties MRJobConfig.MAP_JAVA_OPTS(mapreduce.map.java.opts) and MRJobConfig.REDUCE_JAVA_OPTS(mapreduce.reduce.java.opts). As hadoop will update the new version properties for many other deprecated properties, we also should support such feature on the old property mapred.child.java.opts, otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property mapred.child.java.opts
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268495#comment-14268495 ] Jason Lowe commented on MAPREDUCE-6205: --- I think we need to be _very_ careful with a patch like this, or it could easily create more problems than it solves. Something like this can readily break backwards compatibility. For example, if a user does -Dmapreduce.map.java.opts=X -Dmapred.child.java.opts=Y, then maps should get X and reduces should get Y. We need to make sure that this code doesn't end up setting maps and reduces to Y because of the order we processed those settings and how they're aliased. mapred.child.java.opts is indeed deprecated, but it is not strictly an alias for the map/reduce-specific settings. Update the value of the new version properties of the deprecated property mapred.child.java.opts -- Key: MAPREDUCE-6205 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: trunk Reporter: sam liu Assignee: sam liu Priority: Minor Attachments: MAPREDUCE-6205.003.patch, MAPREDUCE-6205.patch, MAPREDUCE-6205.patch In current hadoop code, the old property mapred.child.java.opts is deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the deprecated property mapred.child.java.opts, hadoop won't automatically update its new versions properties MRJobConfig.MAP_JAVA_OPTS(mapreduce.map.java.opts) and MRJobConfig.REDUCE_JAVA_OPTS(mapreduce.reduce.java.opts). As hadoop will update the new version properties for many other deprecated properties, we also should support such feature on the old property mapred.child.java.opts, otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property mapred.child.java.opts
[ https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268556#comment-14268556 ] Gera Shegalov commented on MAPREDUCE-6205: -- I agree. I think the proposed deprecation chain handles this correctly but we should definitely capture these requirements in unit tests. And I hope you can review it [~jlowe]. Update the value of the new version properties of the deprecated property mapred.child.java.opts -- Key: MAPREDUCE-6205 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: trunk Reporter: sam liu Assignee: sam liu Priority: Minor Attachments: MAPREDUCE-6205.003.patch, MAPREDUCE-6205.patch, MAPREDUCE-6205.patch In current hadoop code, the old property mapred.child.java.opts is deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the deprecated property mapred.child.java.opts, hadoop won't automatically update its new versions properties MRJobConfig.MAP_JAVA_OPTS(mapreduce.map.java.opts) and MRJobConfig.REDUCE_JAVA_OPTS(mapreduce.reduce.java.opts). As hadoop will update the new version properties for many other deprecated properties, we also should support such feature on the old property mapred.child.java.opts, otherwise it might bring some imcompatible issues. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6205) Update the value of the new version properties of the deprecated property mapred.child.java.opts
[
https://issues.apache.org/jira/browse/MAPREDUCE-6205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268589#comment-14268589
]
Hadoop QA commented on MAPREDUCE-6205:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12690653/MAPREDUCE-6205.003.patch
against trunk revision e13a484.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color}. The applied patch generated 1219 javac
compiler warnings (more than the trunk's current 1218 warnings).
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 13 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The test build failed in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//artifact/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
Javac warnings:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//artifact/patchprocess/diffJavacWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/5099//console
This message is automatically generated.
Update the value of the new version properties of the deprecated property
mapred.child.java.opts
--
Key: MAPREDUCE-6205
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6205
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: trunk
Reporter: sam liu
Assignee: sam liu
Priority: Minor
Attachments: MAPREDUCE-6205.003.patch, MAPREDUCE-6205.patch,
MAPREDUCE-6205.patch
In current hadoop code, the old property mapred.child.java.opts is
deprecated and its new versions are MRJobConfig.MAP_JAVA_OPTS and
MRJobConfig.REDUCE_JAVA_OPTS. However, when user set a value to the
deprecated property mapred.child.java.opts, hadoop won't automatically
update its new versions properties
MRJobConfig.MAP_JAVA_OPTS(mapreduce.map.java.opts) and
MRJobConfig.REDUCE_JAVA_OPTS(mapreduce.reduce.java.opts). As hadoop will
update the new version properties for many other deprecated properties, we
also should support such feature on the old property
mapred.child.java.opts, otherwise it might bring some imcompatible issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-3745) mapred/yarn scripts should use lib/'*' instead looping the dir for jar to create the classpath
[ https://issues.apache.org/jira/browse/MAPREDUCE-3745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Nauroth resolved MAPREDUCE-3745. -- Resolution: Not a Problem This is no longer a problem after the shell rewrite done by [~aw] in trunk. I'm resolving this issue. mapred/yarn scripts should use lib/'*' instead looping the dir for jar to create the classpath -- Key: MAPREDUCE-3745 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3745 Project: Hadoop Map/Reduce Issue Type: Improvement Affects Versions: 0.23.1, 0.24.0 Reporter: Alejandro Abdelnur Fix For: 0.24.0 The scripts do a for loop on the JAR contents of the directory, they should just use '*' in the classpath, ie lib/'*' This will reduce the length of the generated classpath significantly -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3283) mapred classpath CLI does not display the complete classpath
[
https://issues.apache.org/jira/browse/MAPREDUCE-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated MAPREDUCE-3283:
-
Summary: mapred classpath CLI does not display the complete classpath
(was: yarn classpath CLI does not display the complete classpath)
mapred classpath CLI does not display the complete classpath
Key: MAPREDUCE-3283
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3283
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Labels: newbie
bin/yarn classpath does not display the complete classpath. Below is how the
classpath looks like:
{noformat}
$HADOOP_CONF_DIR:$HADOOP_CONF_DIR::$TOOLS_JAR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOME/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:
$HADOOP_MAPRED_HOME/bin/../modules/*:$HADOOP_MAPRED_HOME/bin/../lib/*
{noformat}
* has to be substituted with the actual jars. Also, $HADOOP_CONF_DIR
appears twice in the classpath
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-3283) mapred classpath CLI does not display the complete classpath
[
https://issues.apache.org/jira/browse/MAPREDUCE-3283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated MAPREDUCE-3283:
-
Component/s: (was: mrv2)
scripts
Priority: Minor (was: Major)
Target Version/s: 2.7.0
HADOOP-10903 added support for this in the {{hadoop classpath}} command, but we
didn't do the same for the other scripts' classpath commands. We can repurpose
this issue to cover that change for the {{mapred}} script. Thanks to [~aw] for
pointing it out to me.
mapred classpath CLI does not display the complete classpath
Key: MAPREDUCE-3283
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3283
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: scripts
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Priority: Minor
Labels: newbie
bin/yarn classpath does not display the complete classpath. Below is how the
classpath looks like:
{noformat}
$HADOOP_CONF_DIR:$HADOOP_CONF_DIR::$TOOLS_JAR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOME/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:
$HADOOP_MAPRED_HOME/bin/../modules/*:$HADOOP_MAPRED_HOME/bin/../lib/*
{noformat}
* has to be substituted with the actual jars. Also, $HADOOP_CONF_DIR
appears twice in the classpath
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6212) UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative() happened when starting MRAppMaster
[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli resolved MAPREDUCE-6212. Resolution: Invalid Fix Version/s: (was: 2.6.0) This seems like an environmental issue. JIRA is a place for tracking potential bugs in the project, not usage questions. Can you please use the user mailing lists for help with this (http://hadoop.apache.org/mailing_lists.html#User)? Thanks. UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative() happened when starting MRAppMaster Key: MAPREDUCE-6212 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6212 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 2.6.0 Environment: Ubuntu 64bit Reporter: Dinh Hoang Mai I have just started to work with Hadoop 2. After installing with basic configs, I always failed to run any examples. Has anyone seen this problem and please help me? This is the log 2015-01-08 01:52:01,599 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1420648881673_0004_01 2015-01-08 01:52:01,764 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) at org.apache.hadoop.security.Groups.init(Groups.java:70) at org.apache.hadoop.security.Groups.init(Groups.java:66) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:299) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1473) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129) ... 7 more Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.clinit(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.init(JniBasedUnixGroupsMappingWithFallback.java:39) ... 12 more 2015-01-08 01:52:01,767 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (MAPREDUCE-6212) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing
[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith moved YARN-3012 to MAPREDUCE-6212: - Component/s: (was: security) security Fix Version/s: (was: 2.6.0) 2.6.0 Target Version/s: (was: 2.6.0) Affects Version/s: (was: 2.6.0) 2.6.0 Key: MAPREDUCE-6212 (was: YARN-3012) Project: Hadoop Map/Reduce (was: Hadoop YARN) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing - Key: MAPREDUCE-6212 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6212 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 2.6.0 Environment: Ubuntu 64bit Reporter: Dinh Hoang Mai Priority: Critical Fix For: 2.6.0 I have just started to work with Hadoop 2. After installing with basic configs, I always failed to run any examples. Has anyone seen this problem and please help me? This is the log 2015-01-08 01:52:01,599 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1420648881673_0004_01 2015-01-08 01:52:01,764 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) at org.apache.hadoop.security.Groups.init(Groups.java:70) at org.apache.hadoop.security.Groups.init(Groups.java:66) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:299) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1473) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129) ... 7 more Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.clinit(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.init(JniBasedUnixGroupsMappingWithFallback.java:39) ... 12 more 2015-01-08 01:52:01,767 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6212) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing
[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268142#comment-14268142 ] Rohith commented on MAPREDUCE-6212: --- Since issue is in MRAppMaster, moving to MapReduce project. Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing - Key: MAPREDUCE-6212 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6212 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 2.6.0 Environment: Ubuntu 64bit Reporter: Dinh Hoang Mai Priority: Critical Fix For: 2.6.0 I have just started to work with Hadoop 2. After installing with basic configs, I always failed to run any examples. Has anyone seen this problem and please help me? This is the log 2015-01-08 01:52:01,599 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1420648881673_0004_01 2015-01-08 01:52:01,764 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) at org.apache.hadoop.security.Groups.init(Groups.java:70) at org.apache.hadoop.security.Groups.init(Groups.java:66) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:299) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1473) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129) ... 7 more Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.clinit(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.init(JniBasedUnixGroupsMappingWithFallback.java:39) ... 12 more 2015-01-08 01:52:01,767 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6206) TestAggregatedTransferRate fails on non-US systems
[
https://issues.apache.org/jira/browse/MAPREDUCE-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267713#comment-14267713
]
Hudson commented on MAPREDUCE-6206:
---
FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #67 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/67/])
MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed
by Jens Rabe (jlowe: rev a0aeed100b839d5c122974bd0ea12f9168ee36a5)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
* hadoop-mapreduce-project/CHANGES.txt
TestAggregatedTransferRate fails on non-US systems
--
Key: MAPREDUCE-6206
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6206
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: trunk
Reporter: Jens Rabe
Assignee: Jens Rabe
Priority: Critical
Fix For: 2.7.0
Attachments: MAPREDUCE-6206.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
When running the test TestAggregatedTransferRate from
org.apache.hadoop.mapreduce.task.reduce.TestShuffleScheduler on a non-US
system (or, to be more precise, a system which does not use the point as a
decimal separator), the test fails.
E.g. on a German system, the progress message is:
{{copy task(attempt_test__m_00_0 succeeded at 1,00 MB/s) Aggregated
copy rate(1 of 10 at 1,00 MB/s)}}
But the test checks for:
{{copy task(attempt_test__m_00_0 succeeded at 1.00 MB/s) Aggregated
copy rate(1 of 10 at 1.00 MB/s)}}
which causes it to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6212) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing
[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6212: -- Priority: Major (was: Critical) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing - Key: MAPREDUCE-6212 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6212 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 2.6.0 Environment: Ubuntu 64bit Reporter: Dinh Hoang Mai Fix For: 2.6.0 I have just started to work with Hadoop 2. After installing with basic configs, I always failed to run any examples. Has anyone seen this problem and please help me? This is the log 2015-01-08 01:52:01,599 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1420648881673_0004_01 2015-01-08 01:52:01,764 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) at org.apache.hadoop.security.Groups.init(Groups.java:70) at org.apache.hadoop.security.Groups.init(Groups.java:66) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:299) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1473) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129) ... 7 more Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.clinit(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.init(JniBasedUnixGroupsMappingWithFallback.java:39) ... 12 more 2015-01-08 01:52:01,767 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6212) UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative() happened when starting MRAppMaster
[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated MAPREDUCE-6212: -- Summary: UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative() happened when starting MRAppMaster (was: Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing) UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative() happened when starting MRAppMaster Key: MAPREDUCE-6212 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6212 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 2.6.0 Environment: Ubuntu 64bit Reporter: Dinh Hoang Mai Fix For: 2.6.0 I have just started to work with Hadoop 2. After installing with basic configs, I always failed to run any examples. Has anyone seen this problem and please help me? This is the log 2015-01-08 01:52:01,599 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1420648881673_0004_01 2015-01-08 01:52:01,764 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) at org.apache.hadoop.security.Groups.init(Groups.java:70) at org.apache.hadoop.security.Groups.init(Groups.java:66) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:299) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1473) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129) ... 7 more Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.clinit(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.init(JniBasedUnixGroupsMappingWithFallback.java:39) ... 12 more 2015-01-08 01:52:01,767 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6212) Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing
[ https://issues.apache.org/jira/browse/MAPREDUCE-6212?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14268354#comment-14268354 ] Wangda Tan commented on MAPREDUCE-6212: --- It's more like a configuration issue instead of bug, [~maidh91], could you provide more information about what's the basic configuration?. Hadoop 2.6.0: Basic error “starting MRAppMaster” after installing - Key: MAPREDUCE-6212 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6212 Project: Hadoop Map/Reduce Issue Type: Bug Components: security Affects Versions: 2.6.0 Environment: Ubuntu 64bit Reporter: Dinh Hoang Mai Priority: Critical Fix For: 2.6.0 I have just started to work with Hadoop 2. After installing with basic configs, I always failed to run any examples. Has anyone seen this problem and please help me? This is the log 2015-01-08 01:52:01,599 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1420648881673_0004_01 2015-01-08 01:52:01,764 FATAL [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster java.lang.RuntimeException: java.lang.reflect.InvocationTargetException at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:131) at org.apache.hadoop.security.Groups.init(Groups.java:70) at org.apache.hadoop.security.Groups.init(Groups.java:66) at org.apache.hadoop.security.Groups.getUserToGroupsMappingService(Groups.java:280) at org.apache.hadoop.security.UserGroupInformation.initialize(UserGroupInformation.java:271) at org.apache.hadoop.security.UserGroupInformation.setConfiguration(UserGroupInformation.java:299) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1473) at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1429) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:408) at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129) ... 7 more Caused by: java.lang.UnsatisfiedLinkError: org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative()V at org.apache.hadoop.security.JniBasedUnixGroupsMapping.anchorNative(Native Method) at org.apache.hadoop.security.JniBasedUnixGroupsMapping.clinit(JniBasedUnixGroupsMapping.java:49) at org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback.init(JniBasedUnixGroupsMappingWithFallback.java:39) ... 12 more 2015-01-08 01:52:01,767 INFO [main] org.apache.hadoop.util.ExitUtil: Exiting with status 1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6206) TestAggregatedTransferRate fails on non-US systems
[
https://issues.apache.org/jira/browse/MAPREDUCE-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267749#comment-14267749
]
Hudson commented on MAPREDUCE-6206:
---
SUCCESS: Integrated in Hadoop-Mapreduce-trunk #2017 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2017/])
MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed
by Jens Rabe (jlowe: rev a0aeed100b839d5c122974bd0ea12f9168ee36a5)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
* hadoop-mapreduce-project/CHANGES.txt
TestAggregatedTransferRate fails on non-US systems
--
Key: MAPREDUCE-6206
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6206
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: trunk
Reporter: Jens Rabe
Assignee: Jens Rabe
Priority: Critical
Fix For: 2.7.0
Attachments: MAPREDUCE-6206.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
When running the test TestAggregatedTransferRate from
org.apache.hadoop.mapreduce.task.reduce.TestShuffleScheduler on a non-US
system (or, to be more precise, a system which does not use the point as a
decimal separator), the test fails.
E.g. on a German system, the progress message is:
{{copy task(attempt_test__m_00_0 succeeded at 1,00 MB/s) Aggregated
copy rate(1 of 10 at 1,00 MB/s)}}
But the test checks for:
{{copy task(attempt_test__m_00_0 succeeded at 1.00 MB/s) Aggregated
copy rate(1 of 10 at 1.00 MB/s)}}
which causes it to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6206) TestAggregatedTransferRate fails on non-US systems
[
https://issues.apache.org/jira/browse/MAPREDUCE-6206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14267685#comment-14267685
]
Hudson commented on MAPREDUCE-6206:
---
SUCCESS: Integrated in Hadoop-Hdfs-trunk-Java8 #63 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/63/])
MAPREDUCE-6206. TestAggregatedTransferRate fails on non-US systems. Contributed
by Jens Rabe (jlowe: rev a0aeed100b839d5c122974bd0ea12f9168ee36a5)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java
* hadoop-mapreduce-project/CHANGES.txt
TestAggregatedTransferRate fails on non-US systems
--
Key: MAPREDUCE-6206
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6206
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: trunk
Reporter: Jens Rabe
Assignee: Jens Rabe
Priority: Critical
Fix For: 2.7.0
Attachments: MAPREDUCE-6206.patch
Original Estimate: 0.5h
Remaining Estimate: 0.5h
When running the test TestAggregatedTransferRate from
org.apache.hadoop.mapreduce.task.reduce.TestShuffleScheduler on a non-US
system (or, to be more precise, a system which does not use the point as a
decimal separator), the test fails.
E.g. on a German system, the progress message is:
{{copy task(attempt_test__m_00_0 succeeded at 1,00 MB/s) Aggregated
copy rate(1 of 10 at 1,00 MB/s)}}
But the test checks for:
{{copy task(attempt_test__m_00_0 succeeded at 1.00 MB/s) Aggregated
copy rate(1 of 10 at 1.00 MB/s)}}
which causes it to fail.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
