Github user vanzin commented on the issue:
https://github.com/apache/spark/pull/21216
Is this really necessary? I was under the impression the HDFS libraries
would do that for you (and that's what we observed in tests, assuming our tests
were correct). If you look at the code that's what seems to happen:
https://github.com/apache/hadoop/blob/branch-3.0/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/viewfs/ViewFs.java#L615
Unless Spark is somehow resolving the viewfs URI to hdfs? Or maybe you're
providing the hdfs URI instead of the viewfs one?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
