[
https://issues.apache.org/jira/browse/SPARK-13305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen resolved SPARK-13305.
-------------------------------
Resolution: Not A Problem
Hm, no I take that back. The issue here is not internal handling of the env
variable and flags. {{start-slave.sh}} does not indicate it wants
{{--webui-port}} as an argument. It says to use {{SPARK_WORKER_WEBUI_PORT}}.
The redundancy is specifying that flag in the initial invocation.
{{start-slave.sh}} overrides {{SPARK_WORKER_WEBUI_PORT}} since it may need to
create many workers on many ports on one machine. The flag wins, as intended.
{{SPARK_WORKER_WEBUI_PORT}} is really the 'base' port for multiple workers.
It happens to then be passed twice in two ways, and that happens to be
redundant when there is 1 worker per machine, but this is invisible to a caller
and works correctly.
> SPARK_WORKER_WEBUI_PORT handled twice in start-slave.sh script and
> WorkerArguments
> ----------------------------------------------------------------------------------
>
> Key: SPARK-13305
> URL: https://issues.apache.org/jira/browse/SPARK-13305
> Project: Spark
> Issue Type: Improvement
> Components: Spark Core
> Affects Versions: 2.0.0
> Reporter: Jacek Laskowski
> Priority: Minor
>
> Executing the following command to start a worker:
> {code}
> SPARK_WORKER_WEBUI_PORT=11111 ./sbin/start-slave.sh spark://localhost:7077
> --webui-port 22222
> {code}
> ends up with the following Spark command (in the log file) -- some characters
> cut off to make it relevant:
> {code}
> Spark Command: [cut] org.apache.spark.deploy.worker.Worker --webui-port 11111
> spark://localhost:7077 --webui-port 22222
> {code}
> Note {{--webui-port}} set twice.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
