Github user mridulm commented on the pull request:
https://github.com/apache/spark/pull/546#issuecomment-42110226
It is not about a few uses here or there - either spark codebase as a whole
moves to a) canonical path always; or always sticks to b) paths relative to cwd
and/or what is returned by File.createTempFile - there is no middle ground IMO.
(b) is where we are at currently with some bugs as @srowen mentioned.
Either we fix those to conform to (b) or move to (a) entirely.
Btw, regarding cost - we do quite a lot of path manipulations elsewhere
(block management, shuffle, etc) - which adds to the cost.
Trying to carefully cordon off different sections of the code to different
idioms is just asking for bugs as the codebase evolves. As we are apparently
already hitting !
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
