Github user vanzin commented on the pull request:
https://github.com/apache/spark/pull/7042#issuecomment-115749763
Are you sure this doesn't happen on master? As Sean points out, the code
there is the same.
I was trying to figure out what exactly is going on here; my hunch is that
this is because `copyFileToRemote` calls `makeQualified` when returning the
destination, and that changes the filesystem if the default fs is `viewfs` (so
that it becomes a regular `hdfs` URL pointing directly at the right NN).
Logging the paths before and after the call to `copyFileToRemote` would let us
know if that's really what's going on.
That being said, the change looks correct, and probably ok for the 1.4
branch. On master it might be worth it to follow Sean's suggestion of
simplifying `ClientDistributedCacheManager.addResource`.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
