GitHub user wangyum opened a pull request:
https://github.com/apache/spark/pull/21734
[SPARK-24149][YARN][FOLLOW-UP] Add a config to control automatic namespaces
discovery
## What changes were proposed in this pull request?
Our HDFS cluster configured 5 nameservices: `nameservices1`,
`nameservices2`, `nameservices3`, `nameservices-dev1` and `nameservices4`, but
`nameservices-dev1` unstable. So sometimes an error occurred and causing the
entire job failed since
[SPARK-24149](https://issues.apache.org/jira/browse/SPARK-24149):
I think it's best to add a switch here.
## How was this patch tested?
manual tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/wangyum/spark SPARK-24149
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/21734.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #21734
----
commit 8885ffff03efe1bbc2afa86b24f15c0413d2c05d
Author: Yuming Wang <yumwang@...>
Date: 2018-07-09T06:24:50Z
Add spark.yarn.access.all.hadoopFileSystems
----
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
