GitHub user pianista215 opened a pull request:
https://github.com/apache/spark/pull/20083
Supporting different host for Spark-shuffle
## What changes were proposed in this pull request?
Adding an option as spark.shuffle.service.host let you to specify different
ip for the external shuffle service. For example if you have this service in a
Docker, or behind a CNI, you won't have the same ip in the service vs the spark
executors.
## How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration
tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise,
remove this)
Please review http://spark.apache.org/contributing.html before opening a
pull request.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/pianista215/spark spark_shuffle_host_config
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/20083.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #20083
----
commit a3d302dcffcf71fb9d6a94a0860a762fbd80efb1
Author: Unai <usarasola@...>
Date: 2017-12-26T16:47:47Z
Supporting different host for Spark-shuffle
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
