Github user tgravescs commented on the pull request:
https://github.com/apache/spark/pull/204#issuecomment-39453542
Also to clarify that the history files are actually first written to the
users staging directory, which is only user readable/writable, and then moved
over to done_intermediate when the job finishes/fails. MR does generally make
sure the permissions on directories/files it creating are set appropriately or
according to configs the admin specifies.
Note that I'm definitely not saying we have to do it like MR, just giving
an example of how it does it. As long as things are protected its fine. And
ideally have at least some basic management system to remove old ones. We can
add the management stuff later, although would be nice for 1.0.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
