[
https://issues.apache.org/jira/browse/MAPREDUCE-7443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ConfX updated MAPREDUCE-7443:
-----------------------------
Summary: State polluter for system file permissions (was: state polluter
for system file permissions)
> State polluter for system file permissions
> -------------------------------------------
>
> Key: MAPREDUCE-7443
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7443
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Reporter: ConfX
> Priority: Critical
> Attachments: reproduce.sh
>
>
> h2. What happened:
> After setting {{fs.permissions.umask-mode}} to disable write permission of a
> file, the file's write permission is also disabled in host permission.
> h2. Buggy code:
> When creating {{target/test-dir/output}} the RawLocalFileSystem directly
> manipulate the system permission (line 978 of
> {{{}RawLocalFileSystem.java{}}}):
> {noformat}
> String perm = String.format("%04o", permission.toShort());
> Shell.execCommand(Shell.getSetPermissionCommand(perm, false,
> FileUtil.makeShellPath(pathToFile(p), true)));{noformat}
> If the permission turns off the write permission to the folder, the test
> would fail due to permission denied. However, the test does not clean the
> folder properly (by chmod and clean in an [@after|https://github.com/after]
> method), causing all the subsequent runs to be polluted.
> h2. StackTrace:
> {noformat}
> java.io.IOException: Mkdirs failed to create
> file:/home/ctestfuzz/fuzz-hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/target/test-dir/output/_temporary/1/_temporary/attempt_200707121733_0001_m_000000_0
> (exists=false,
> cwd=file:/home/ctestfuzz/fuzz-hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core),
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:515),
> at
> org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:500),
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1195),
> at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1081),
> at
> org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:125),
> at
> org.apache.hadoop.mapred.TestFileOutputCommitter.testRecoveryInternal(TestFileOutputCommitter.java:109),
> at
> org.apache.hadoop.mapred.TestFileOutputCommitter.testRecoveryUpgradeV1V2(TestFileOutputCommitter.java:171){noformat}
> h2. How to reproduce:
> There are two ways to reproduce:
> # (1) Set {{fs.permissions.umask-mode}} to {{243}}
> (2) Run test:
> {{org.apache.hadoop.mapred.TestFileOutputCommitter#testRecoveryUpgradeV1V2}}
> and observe an IOException
> (3) Check to see that current user has lost writing access to
> {{target/test-dir/output}}
> # (1) Add an {{assertTrue(False);}} to line 112 of
> {{TestFileOutputCommitter.java}} to simulate the test failing in the middle
> (2) Run test:
> {{org.apache.hadoop.mapred.TestFileOutputCommitter#testRecoveryUpgradeV1V2}}
> and observe an AssertionError
> (3) Set {{fs.permissions.umask-mode}} to {{243}}
> (4) Run test:
> {{org.apache.hadoop.mapred.TestFileOutputCommitter#testRecoveryUpgradeV1V2}}
> and observe an IOException.
> (5) Check to see that the current user has lost writing access to
> {{test-dir/output/_temporary/1/_temporary/attempt_200707121733_0001_m_000000_0}}
>
> For an easy reproduction, run the reproduce.sh in the attachment.
> We are happy to provide a patch if this issue is confirmed.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
