[
https://issues.apache.org/jira/browse/MAPREDUCE-6590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15072055#comment-15072055
]
Hadoop QA commented on MAPREDUCE-6590:
--------------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s
{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 33s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 37s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 4s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 16s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
in trunk has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 42s
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
55s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 6s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 6s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 15s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 15s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 21s
{color} | {color:red} Patch generated 1 new checkstyle issues in
hadoop-mapreduce-project/hadoop-mapreduce-client (total was 548, now 548).
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 3s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
59s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 40s
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 2s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 101m 29s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 29s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 19s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_91. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 32s
{color} | {color:red} Patch generated 18 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 251m 11s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.mapred.TestMiniMRChildTask |
| | hadoop.mapred.TestNetworkedJob |
| | hadoop.mapreduce.v2.TestMRJobsWithProfiler |
| JDK v1.7.0_91 Failed junit tests | hadoop.mapred.TestMiniMRChildTask |
| | hadoop.mapred.TestNetworkedJob |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12779559/MAPREDUCE-6590.patch |
| JIRA Issue | MAPREDUCE-6590 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux 67fe2de2a337 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh
|
| git revision | trunk / 8cfd672 |
| Default Java | 1.7.0_91 |
| Multi-JDK versions | /usr/lib/jvm/java-8-oracle:1.8.0_66
/usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
|
| unit |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt
|
| unit |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_91.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.7.0_91.txt
|
| JDK v1.7.0_91 Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient
U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Max memory used | 76MB |
| Powered by | Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6239/console |
This message was automatically generated.
> Update MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV value for loading correct
> navite lib when using ubertask feature
> -----------------------------------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-6590
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6590
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mr-am
> Affects Versions: 2.7.1
> Reporter: NING DING
> Assignee: NING DING
> Attachments: MAPREDUCE-6590.patch
>
>
> By default, the MR AM unable to load native library without
> MR_AM_ADMIN_USER_ENV set.
> If yarn.app.mapreduce.am.admin.user.env (or yarn.app.mapreduce.am.env) is not
> configured to set LD_LIBRARY_PATH, MR AM will fail to load the native
> library, then you can find the error message as below.
> {panel}
> org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library
> for your platform... using builtin-java classes where applicable.
> {panel}
> I found the patch in https://issues.apache.org/jira/browse/MAPREDUCE-5799
> update YARNRunner code and use MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV as
> the default value. The code is as below.
> {code:title=YARNRunner.java|borderStyle=solid}
> MRApps.setEnvFromInputString(environment,
> conf.get(MRJobConfig.MR_AM_ADMIN_USER_ENV,
> MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV), conf);
> {code}
> In fact the parameters yarn.app.mapreduce.am.env and
> yarn.app.mapreduce.am.admin.user.env are resolved in YARNRunner running on
> hadoop client host.
> But their values, e.g. LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native, will
> be used on hadoop cluster hosts.
> If the hadoop client host installs hadoop on a different path from that on
> hadoop cluster hosts,
> the $HADOOP_COMMON_HOME/lib/native will be resolved to a path which is not
> existed on hadoop cluster hosts.
> Then MRAppMaster running on hadoop cluster host cannot load native lib.
> If don't use MAPREDUCE-5799 patch, I set my hadoop client's mapred-site.xml
> with the following content.
> {code:xml}
> <property>
> <name>yarn.app.mapreduce.am.admin.user.env</name>
> <value>LD_LIBRARY_PATH={{HADOOP_COMMON_HOME}}/lib/native</value>
>
> </property>
> {code}
> In this way, the YARNRunner will put
> LD_LIBRARY_PATH={{HADOOP_COMMON_HOME}}/lib/native into environment.
> The method ContainerLaunch.expandEnvironment running in NodeManager can
> translate LD_LIBRARY_PATH={{HADOOP_COMMON_HOME}}/lib/native
> to LD_LIBRARY_PATH=$HADOOP_COMMON_HOME/lib/native.
> The host running NodeManager can find its $HADOOP_COMMON_HOME/lib/native path.
> I suggest MRJobConfig.DEFAULT_MAPRED_ADMIN_USER_ENV should be defined as
> below.
> {code:title=MRJobConfig.java|borderStyle=solid}
> public final String DEFAULT_MAPRED_ADMIN_USER_ENV =
> Shell.WINDOWS ?
> "PATH={{PATH}};{{HADOOP_COMMON_HOME}}\\bin":
> "LD_LIBRARY_PATH={{HADOOP_COMMON_HOME}}/lib/native";
> {code}
> Please see my patch.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
