[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[
https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403002#comment-13403002
]
Colin Patrick McCabe commented on HADOOP-7147:
--
bq. I haven't figured out how to do OS or compiler detection to do the correct
thing on non-gcc/non-Linux. (e.g., on Darwin, we should really be passing
-framework JavaVM amongst other flags when building libhadoop.dylib so we get
linked properly, can build a fat binary, etc.).
You want something like this:
{code}
IF(${CMAKE_SYSTEM_NAME} MATCHES Darwin)
set(CMAKE_C_FLAGS ${CMAKE_C_FLAGS} -framework JavaVM)
ENDIF()
{code}
There are a bunch of CMake variables that are set based on the platform and
environment. See http://www.cmake.org/Wiki/CMake_Useful_Variables. Of course,
it's usually better to check for the specific feature you need if that is at
all possible-- CHECK_FUNCTION_EXISTS is a good example of that.
You can also match against CMAKE_SYSTEM_NAME Apple, but I think that covers
Mac OS classic too. I *hope* that nobody is trying to compile Hadoop on that,
although of course you can never be sure :)
setnetgrent in native code is not portable
--
Key: HADOOP-7147
URL: https://issues.apache.org/jira/browse/HADOOP-7147
Project: Hadoop Common
Issue Type: Bug
Components: native
Affects Versions: 0.22.0, 0.23.0
Reporter: Todd Lipcon
Assignee: Allen Wittenauer
Attachments: HADOOP-7147.patch, hadoop-7147.patch
HADOOP-6864 uses the setnetgrent function in a way which is not compatible
with BSD APIs, where the call returns void rather than int. This prevents the
native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403005#comment-13403005 ] Colin Patrick McCabe commented on HADOOP-7147: -- Oops, meant to say you can check to see if APPLE is set, but that covers Mac OS classic too. setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: HADOOP-7147.patch, hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403147#comment-13403147 ] Allen Wittenauer commented on HADOOP-7147: -- I'm thinking about compiler flags, so CHECK_FUNCTION_EXISTS won't work here. (i.e., -xCC, -xc99, -xstrconst, -xparallel, etc). Also need to pop all the GNU specific bits into protected areas so as not to infect everything else. setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: HADOOP-7147.patch, hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13403347#comment-13403347 ] Colin Patrick McCabe commented on HADOOP-7147: -- Yeah, using CMAKE_SYSTEM_NAME to determine what platform flags to use seems totally reasonable to me. setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: HADOOP-7147.patch, hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[
https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13402848#comment-13402848
]
Colin Patrick McCabe commented on HADOOP-7147:
--
{code}
int maxposix=sysconf(_SC_NGROUPS_MAX);
gid_t *gps = (gid_t *) malloc(maxposix * sizeof (gid_t));
if (!gps) {
*ngroups = 0;
free(pwbuf);
return ENOMEM;
}
...
{code}
Allocating an array of size _SC_NGROUPS_MAX seems a little excessive. I think
it would be better to use realloc() to expand an array of gid_t as needed.
{code}
// set the name of the group for subsequent calls to getnetgrent
// note that we want to end group lokup regardless whether setnetgrent
// was successful or not (as long as it was called we need to call
// endnetgrent)
setnetgrentCalledFlag = 1;
setnetgrent(cgroup); // some platforms return void here
UserList *current = NULL;
// three pointers are for host, user, domain, we only care
// about user now
char *p[3];
while(getnetgrent(p, p + 1, p + 2)) {
if(p[1]) {
current = (UserList *)malloc(sizeof(UserList));
current-string = malloc(strlen(p[1]) + 1);
strcpy(current-string, p[1]);
current-next = userListHead;
userListHead = current;
userListSize++;
}
}
// shortcut null, nonexistent groups
if (userListSize == 0)
goto END;
{code}
It seems like this should be extracted into a function which would just return
a linked list of UserList. There's no reason why the JNI function itself needs
to reference setgrent(), endnetgrent(), etc.
I'm also disturbed by the non-re-entrancy of the getnetgrent() function. It
seems like if we're going to use this function, we need to put a mutex around
it. Alternately, getnetgrent_r() is re-entrant, and available on Linux (but
not on some other platforms). You could do exactly the same thing you did here
and add this to the CMakeLists.txt:
{code}
CHECK_FUNCTION_EXISTS(getnetgrent_r HAVE_NETGRENT_R)
{code}
The uses of getnetgrent() need to be surrounded by a mutex, probably similar to
this:
{code}
static pthread_mutex_t g_getnetgrent_lock = PTHREAD_MUTEX_INITIALIZER;
UserList *get_net_groups(const char *name) {
pthread_mutex_lock(lock);
... do non-thread-safe stuff ...
pthread_mutex_unlock(lock);
}
{code}
setnetgrent in native code is not portable
--
Key: HADOOP-7147
URL: https://issues.apache.org/jira/browse/HADOOP-7147
Project: Hadoop Common
Issue Type: Bug
Components: native
Affects Versions: 0.22.0, 0.23.0
Reporter: Todd Lipcon
Assignee: Allen Wittenauer
Attachments: HADOOP-7147.patch, hadoop-7147.patch
HADOOP-6864 uses the setnetgrent function in a way which is not compatible
with BSD APIs, where the call returns void rather than int. This prevents the
native libs from building on OSX, for example.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13402856#comment-13402856 ] Allen Wittenauer commented on HADOOP-7147: -- It just occurred to me that this latest patch is incomplete anyway. getgrouplist() has weird behavior on OS X. I just forgot to forward port it from what I'm running in 1.x. In any case, this patch is here for those that need it. I'm not really planning on working on it to get it commit-ready. (note this bug is marked as won't fix). setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: HADOOP-7147.patch, hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13402872#comment-13402872 ] Allen Wittenauer commented on HADOOP-7147: -- BTW, Colin, see my note in the other bug about using get*_r routines on Solaris. Now that we're in cmake-ville, I haven't figured out how to do OS or compiler detection to do the correct thing on non-gcc/non-Linux. (e.g., on Darwin, we should *really* be passing -framework JavaVM amongst other flags when building libhadoop.dylib so we get linked properly, can build a fat binary, etc.). setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: HADOOP-7147.patch, hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198934#comment-13198934 ] Alejandro Abdelnur commented on HADOOP-7147: If it still an issue, but not a big priority, shouldn't we leave it open, with low priority? If the patch gets rebased to trunk I'd be happy to tested it review it in Linux and OSX (don't have Solaris box around) setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198939#comment-13198939 ] Allen Wittenauer commented on HADOOP-7147: -- There is so much broken in the native code that ships with Hadoop with regards to portability that it isn't worth spending the time to fix. A major attitude shift really needs to happen first. So while I had a working libhadoop.so for OS X, Solaris, and other OSes for hadoop 0.20.20x (and I've been too busy fixing bugs 1.x to bring them up to date to 1.x), I don't feel particularly compelled to contribute the patches because they are just going to rot like this one did. setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13198538#comment-13198538 ] Allen Wittenauer commented on HADOOP-7147: -- It's still an issue in the code sense, but not an issue in the priority sense. Probably better to just close as won't fix since portability isn't a concern for the project. setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HADOOP-7147) setnetgrent in native code is not portable
[ https://issues.apache.org/jira/browse/HADOOP-7147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13045665#comment-13045665 ] Hadoop QA commented on HADOOP-7147: --- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12481747/hadoop-7147.patch against trunk revision 1133125. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/589//console This message is automatically generated. setnetgrent in native code is not portable -- Key: HADOOP-7147 URL: https://issues.apache.org/jira/browse/HADOOP-7147 Project: Hadoop Common Issue Type: Bug Components: native Affects Versions: 0.22.0, 0.23.0 Reporter: Todd Lipcon Assignee: Allen Wittenauer Attachments: hadoop-7147.patch HADOOP-6864 uses the setnetgrent function in a way which is not compatible with BSD APIs, where the call returns void rather than int. This prevents the native libs from building on OSX, for example. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
