[
https://issues.apache.org/jira/browse/HADOOP-7110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12983805#action_12983805
]
Todd Lipcon commented on HADOOP-7110:
-------------------------------------
bq. So if it fails right after setting them to 000 it will be hard to remove
such file or directory
It's not due to failure of the process doing the chmod. It's due to other
processes or threads working inside the directory failing because there is a
moment in which they can't traverse the path inside the directory. As we've
seen in MR-2238 this race really does happen - we've got build failures on
three separate environments (Apache Hudson, my Hudson, and Greg's build box) to
prove it.
bq. So why don't we just set permissions to the values specified (in the right
order, which is probably important) rather than introducing more JNI code
The API is not flexible enough to do this. If you have a file 755 and want it
to be 700, the only way to get rid of the group/other bits is to first clear
those bits for all, then add them back for user only.
bq. Nobody said we should support posix permissions
Since we now support security, we have to be able to make some files mode 700
and others 755, etc. There's no way around it. The question here is
implementation method, not whether we should support an API since we've had
since 2008. Removing the ability to chmod isn't an option, considering
compatibility.
bq. Your patch is an alternative implementation of HADOOP-6304
And HADOOP-6304 is a sort-of-copy-paste of code originally introduced in
MAPREDUCE-842. I filed this JIRA as part of the fix for MAPREDUCE-2238 (which
is caused by the MR code that has the same problems)
Do you have a particular issue with the JNI code?
> Implement chmod with JNI
> ------------------------
>
> Key: HADOOP-7110
> URL: https://issues.apache.org/jira/browse/HADOOP-7110
> Project: Hadoop Common
> Issue Type: Improvement
> Components: io, native
> Affects Versions: 0.22.0
> Reporter: Todd Lipcon
> Assignee: Todd Lipcon
> Fix For: 0.22.0
>
> Attachments: hadoop-7110.txt
>
>
> MapReduce is currently using a race-prone workaround to approximate chmod()
> because forking chmod is too expensive. This race is causing build failures
> (and probably task failures too). We should implement chmod in the NativeIO
> library so we can have good performance (ie not fork) and still not be prone
> to races.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
