Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/1031#issuecomment-57317439
I haven't looked at the code to fully understand what all changes would be
required and why its so much more complex. So if it makes the implementation
much simpler then I guess its a step in the right direction.
I don't see any huge issues with it, other then you are grabbing the
LD_LIBRARY_PATH of machines that you aren't necessarily going to run on. We
grab it on the client for the AM and on the AM for the executors. If you have a
heterogeneous cluster those could potentially be different, especially between
the client and the AM as in many cases we have gateways that aren't managed by
grid Se.
Are you thinking we will have a followup to move to using LD_LIBRARY_PATH
or thinking its not necessary?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
