[
https://issues.apache.org/jira/browse/SPARK-15974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15333619#comment-15333619
]
Steve Loughran commented on SPARK-15974:
----------------------------------------
# if someone hard codes a port in, that's the one that should be used.
# otherwise, I'd recommend taking a range of ports and searching for a free
one. That allows people whose clusters use IP tables to lock down all but a
small subset of ports to be able to bring up a service in that range, without
having to explicitly pre-allocate ports. Not that once you do that you have a
new problem: all the ports have been scanned and there is still no capacity.
> Create a socket on YARN AM start-up
> -----------------------------------
>
> Key: SPARK-15974
> URL: https://issues.apache.org/jira/browse/SPARK-15974
> Project: Spark
> Issue Type: New Feature
> Components: YARN
> Reporter: Mingyu Kim
>
> YARN provides a way for AppilcationMaster to register a RPC port so that a
> client outside the YARN cluster can reach the application for any RPCs, but
> Spark’s YARN AMs simply register a dummy port number of 0. For the Spark
> programs that starts up a server, this makes it hard for the submitter to
> discover the server port securely. Spark's ApplicationMaster should
> optionally create a ServerSocket and pass it to the Spark user program. This
> socket initialization should be disabled by default.
> Some discussion on dev@spark thread:
> http://apache-spark-developers-list.1001551.n3.nabble.com/Utilizing-YARN-AM-RPC-port-field-td17892.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
