[
https://issues.apache.org/jira/browse/SPARK-6343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-6343:
-----------------------------------
Assignee: (was: Apache Spark)
> Make doc more explicit regarding network connectivity requirements
> ------------------------------------------------------------------
>
> Key: SPARK-6343
> URL: https://issues.apache.org/jira/browse/SPARK-6343
> Project: Spark
> Issue Type: Improvement
> Components: Documentation
> Reporter: Peter Parente
> Priority: Minor
>
> As a new user of Spark, I read through the official documentation before
> attempting to stand-up my own cluster and write my own driver application.
> But only after attempting to run my app remotely against my cluster did I
> realize that full network connectivity (layer 3) is necessary between my
> driver program and worker nodes (i.e., my driver was *listening* for
> connections from my workers).
> I returned to the documentation to see how I had missed this requirement. On
> a second read-through, I saw that the doc hints at it in a few places (e.g.,
> [driver
> config|http://spark.apache.org/docs/1.2.0/configuration.html#networking],
> [submitting applications
> suggestion|http://spark.apache.org/docs/1.2.0/submitting-applications.html],
> [cluster overview|http://spark.apache.org/docs/1.2.0/cluster-overview.html])
> but never outright says it.
> I think it would help would-be users better understand how Spark works to
> state the network connectivity requirements right up-front in the overview
> section of the doc. I suggest revising the diagram and accompanying text
> found on the [overview
> page|http://spark.apache.org/docs/1.2.0/cluster-overview.html]:
> !http://spark.apache.org/docs/1.2.0/img/cluster-overview.png!
> so that it depicts at least the directionality of the network connections
> initiated (perhaps like so):
> !http://i.imgur.com/2dqGbCr.png!
> and states that the driver must listen for and accept connections from other
> Spark components on a variety of ports.
> Please treat my diagram and text as strawmen: I expect more experienced Spark
> users and developers will have better ideas on how to convey these
> requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
