GitHub user nkronenfeld opened a pull request:
https://github.com/apache/spark/pull/3699
[SPARK-4848] Stand-alone cluster: Allow differences between workers with
multiple instances
I've changed the stand-alone cluster run scripts to allow different workers
to have different numbers of instances, and base webui ports.
I did this by moving the loop over instances from start-slaves to
start-slave.
In order to stop things properly, I had to make similar changes in
stop-slaves (and introduce stop-slave).
While I was at it, I changed SPARK_WORKER_PORT to work the same way as
SPARK_WORKER_UI_PORT, since the new methods works fine for both.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/nkronenfeld/spark-1 startup-scripts
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/3699.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3699
----
commit 479c31c9d3e580879d76146e2a687b5235c87b33
Author: Nathan Kronenfeld <[email protected]>
Date: 2014-12-15T06:58:08Z
Move looping through instances from the master to the workers, so that each
worker respects its own number of instances and web-ui port.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
