Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/1031#issuecomment-57470785
OK so I was thinking about this some more and I'm not sure how much the
current change will help. @pwendell have you seen specific requests from other
users? If so what was the problem they were seeing?
So I'm not 100% on all of these as I haven't specifically tested these but
from what I've read on the MR jira my understanding is:
- From my understanding one of the issues with java.library.path is that
it searches differently from the system ld one. I thought that once it finds a
library that matches the name/symbol it will try to load it (even if its the
wrong one - 32 bit vs 64 bit) and once it fails it stops. With LD_LIBRARY_PATH
it will continue looking to see if the right one is later in the path.
- I also thought that once java.library.path is set it doesn't necessarily
fall back and use the system library paths (/usr/lib:/usr/lib64, etc), and
anything set in LD_LIBRARY_PATH. So if we just pull anything set in
LD_LIBRARY_PATH, I don't think it would fall back to search the normal
/usr/lib, etc.
Perhaps we should write a quick test program to make sure the above are
true. I'm not sure if this would be different between java version but it
probably wouldn't hurt to check.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
