[
https://issues.apache.org/jira/browse/SPARK-11267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15012083#comment-15012083
]
Shixiong Zhu commented on SPARK-11267:
--------------------------------------
Mac OS X (may be unix?) and Linux has different behaviors of SO_REUSE_SOCKET.
E.g.,
Mac OS X:
{code}
$ ./bin/spark-shell --conf spark.driver.port=4039 2>&1 | grep port
15/11/18 13:32:50 INFO Utils: Successfully started service 'HTTP class server'
on port 50179.
15/11/18 13:32:52 INFO Utils: Successfully started service 'sparkDriver' on
port 4039.
15/11/18 13:32:53 INFO Utils: Successfully started service
'sparkDriverActorSystem' on port 4040.
15/11/18 13:32:53 INFO Utils: Successfully started service 'HTTP file server'
on port 50181.
15/11/18 13:32:53 INFO Utils: Successfully started service 'SparkUI' on port
4040.
15/11/18 13:32:53 INFO Utils: Successfully started service
'org.apache.spark.network.netty.NettyBlockTransferService' on port 50182.
{code}
Linux:
{code}
$ ./bin/spark-shell --conf spark.driver.port=4039 2>&1 | grep port
15/11/18 21:33:37 INFO Utils: Successfully started service 'HTTP class server'
on port 44756.
15/11/18 21:33:41 INFO Utils: Successfully started service 'sparkDriver' on
port 4039.
15/11/18 21:33:41 INFO Utils: Successfully started service
'sparkDriverActorSystem' on port 4040.
15/11/18 21:33:42 INFO Utils: Successfully started service 'HTTP file server'
on port 48386.
15/11/18 21:33:52 WARN Utils: Service 'SparkUI' could not bind on port 4040.
Attempting port 4041.
15/11/18 21:34:02 INFO Utils: Successfully started service 'SparkUI' on port
4041.
15/11/18 21:34:02 INFO Utils: Successfully started service
'org.apache.spark.network.netty.NettyBlockTransferService' on port 37827.
{code}
> NettyRpcEnv and sparkDriver services report the same port in the logs
> ---------------------------------------------------------------------
>
> Key: SPARK-11267
> URL: https://issues.apache.org/jira/browse/SPARK-11267
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Environment: the version built from today's sources - Spark version
> 1.6.0-SNAPSHOT
> Reporter: Jacek Laskowski
> Priority: Minor
>
> When starting {{./bin/spark-shell --conf spark.driver.port=2222}} Spark
> reports two services - NettyRpcEnv and sparkDriver - using the same {{2222}}
> port:
> {code}
> 15/10/22 23:09:32 INFO SparkContext: Running Spark version 1.6.0-SNAPSHOT
> 15/10/22 23:09:32 INFO SparkContext: Spark configuration:
> spark.app.name=Spark shell
> spark.driver.port=2222
> spark.home=/Users/jacek/dev/oss/spark
> spark.jars=
> spark.logConf=true
> spark.master=local[*]
> spark.repl.class.uri=http://192.168.1.4:52645
> spark.submit.deployMode=client
> ...
> 15/10/22 23:09:33 INFO Utils: Successfully started service 'NettyRpcEnv' on
> port 2222.
> ...
> 15/10/22 23:09:33 INFO Utils: Successfully started service 'sparkDriver' on
> port 2222.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
