Github user ziky90 commented on the pull request:
https://github.com/apache/spark/pull/2836#issuecomment-59570517
Ok, you are probably right.
What about at least something like user defined bootstrap script execution
automatically in the EMR like style + maybe automatic pip installation?
Do you something like this would be acceptable for spark-ec2 as a new
feature?
I'd be interested in implementing this since it'd help me (I hope also to
others) in setting up spark on EC2. Currently I just don't know if there is
some easy way how to perform scp of the script, but I can at least try it.
You're right that when one of the commands fails, then one of these Python
packages will not be installed or all the packages that depends on the package
that failed.
And I think that you can pass as many options as you'd like to. Could you
please give me an example if you see some where possibly might be the problem.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
