Alan Braithwaite created SPARK-13005:
----------------------------------------
Summary: Don't require spark.shuffle.service.port to be set on job
conf
Key: SPARK-13005
URL: https://issues.apache.org/jira/browse/SPARK-13005
Project: Spark
Issue Type: Improvement
Components: Block Manager, Shuffle
Affects Versions: 1.6.0
Reporter: Alan Braithwaite
It appears that the shuffle client gets the external service port from the
spark properties instead of discovering it from the driver or the master.
Not sure how this can be made cleaner, but when you're dynamically allocating
ports for things like the shuffle service, it becomes a pain.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
