[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178004#comment-14178004 ] Allen Wittenauer commented on YARN-2161: I've re-opened YARN-2701. > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Fix For: 2.6.0 > > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177969#comment-14177969 ] Binglin Chang commented on YARN-2161: - Hi [~xgong], sorry for break the code. I see in YARN-2701 you already have fix code, but decide to revert the code in the end to be more safe, but this breaks the mac build, how about use #ifdef to use old code when compiling on glibc > 2.10(http://linux.die.net/man/2/openat) and use your fixing code otherwise? > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Fix For: 2.6.0 > > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177756#comment-14177756 ] Xuan Gong commented on YARN-2161: - The changes for mkdirs in container-executor.c bring the race condition when two containers are trying to check and create directory at the same time. > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Fix For: 2.6.0 > > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14177753#comment-14177753 ] Xuan Gong commented on YARN-2161: - [~decster] [~aw] For fixing YARN-2701, i need to revert the native code changes for mkdirs in container-executor.c > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Fix For: 2.6.0 > > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147977#comment-14147977 ] Hudson commented on YARN-2161: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #1907 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1907/]) YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) (aw: rev 034df0e2eb2824fb46a1e75b52d43d9914a04e56) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Fix For: 2.6.0 > > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147772#comment-14147772 ] Hudson commented on YARN-2161: -- SUCCESS: Integrated in Hadoop-Hdfs-trunk #1882 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1882/]) YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) (aw: rev 034df0e2eb2824fb46a1e75b52d43d9914a04e56) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c * hadoop-yarn-project/CHANGES.txt > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Fix For: 2.6.0 > > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14147648#comment-14147648 ] Hudson commented on YARN-2161: -- FAILURE: Integrated in Hadoop-Yarn-trunk #691 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/691/]) YARN-2161. Fix build on macosx: YARN parts (Binglin Chang via aw) (aw: rev 034df0e2eb2824fb46a1e75b52d43d9914a04e56) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/configuration.c * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/impl/container-executor.c * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/config.h.cmake * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/CMakeLists.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/container-executor/test/test-container-executor.c > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Fix For: 2.6.0 > > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146037#comment-14146037 ] Hadoop QA commented on YARN-2161: - {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12670922/YARN-2161.v2.patch against trunk revision ef784a2. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5094//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5094//console This message is automatically generated. > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Attachments: YARN-2161.v1.patch, YARN-2161.v2.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14145626#comment-14145626 ] Hadoop QA commented on YARN-2161: - {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12650514/YARN-2161.v1.patch against trunk revision b93d960. {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 2.0.3) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5089//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5089//console This message is automatically generated. > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Attachments: YARN-2161.v1.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14145585#comment-14145585 ] Allen Wittenauer commented on YARN-2161: Looks good for the most part! A few minor things: {code} +#if defined(__MACH__) + // only those fds are opened assuming no bug + fclose(LOGFILE); + fclose(ERRORFILE); + fclose(stdin); + fclose(stdout); + fclose(stderr); +#else fcloseall(); +#endif {code} It would probably be better to detect if fcloseall is a valid function. If not, then use the manual fclose. This would allow the code to run on not just OS X but other systems as well. {code} +#if defined(__FreeBSD__) || defined(__MACH__) + fprintf(LOGFILE, "Failed to mount cgroup controller, not supported\n"); + return -1; +#else {code} I sort of prefer the fix from YARN-1327 here, where it does +#ifndef __linux instead. {code} +#if !defined(__MACH__) // macosx do not have user bin if (check_user("bin") == NULL) { printf("FAIL: failed check for whitelisted system user bin\n"); exit(1); } +#endif {code} Rather than ignore this test completely, can we chose a different user instead? Also, can you incorporate the other changes from YARN-1327 so that we can do one commit? Thanks! > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Attachments: YARN-2161.v1.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032175#comment-14032175 ] Hadoop QA commented on YARN-2161: - {color:green}+1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12650514/YARN-2161.v1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 1 new or modified test files. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 javadoc{color}. There were no new javadoc warning messages. {color:green}+1 eclipse:eclipse{color}. The patch built with eclipse:eclipse. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 core tests{color}. The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager. {color:green}+1 contrib tests{color}. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-YARN-Build/3991//testReport/ Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3991//console This message is automatically generated. > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Attachments: YARN-2161.v1.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-2161) Fix build on macosx: YARN parts
[ https://issues.apache.org/jira/browse/YARN-2161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14032164#comment-14032164 ] Binglin Chang commented on YARN-2161: - Changes: container-executor.c: 1. make mkdirs more compatible, remove usage of mkdirat/openat 2. use sysconf() to get LOGIN_NAME_MAX 3. macosx doesn't have fcloseall, so close all opened fds on macosx 4. disable cgroup on macosx test-container-executor.c: 1. macosx do not have user bin, skip a check 2. change /etc/passwd(not exists on mac) to /bin/ls > Fix build on macosx: YARN parts > --- > > Key: YARN-2161 > URL: https://issues.apache.org/jira/browse/YARN-2161 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Binglin Chang >Assignee: Binglin Chang > Attachments: YARN-2161.v1.patch > > > When compiling on macosx with -Pnative, there are several warning and errors, > fix this would help hadoop developers with macosx env. -- This message was sent by Atlassian JIRA (v6.2#6252)