[
https://issues.apache.org/jira/browse/HADOOP-11321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Nauroth updated HADOOP-11321:
-----------------------------------
Attachment: HADOOP-11321.003.patch
Patch v3 implements a native code solution for creating local file system
directories and files with an initial security descriptor equivalent to the
mode passed by the caller. I have tested this locally. It fixes the original
reported problem for a Windows client accessing an SMB share. For a Linux
client accessing an SMB share, there is no change in behavior in the Hadoop
code, and it still works fine via the error handling in Linux CIFS mounts
(silently ignoring the chmod failure).
Here is a description of the changes.
* {{RawLocalFileSystem}}: On Windows, make a JNI call to open a file descriptor
for write instead of using {{java.io.FileOutputStream}}. There are 2 new
methods that can be overridden in subclasses: {{mkOneDirWithMode}} and
{{createOutputStreamWithMode}}. These are intended to support
{{ElevatedRawLocalFileSystem}}, discussed in more detail below. We cannot
remove the older protected methods {{mkOneDir}} and {{createOutputStream}},
because the class is annotated {{Public}} and {{Stable}}. These are now
implemented as simple forwarding methods.
* {{ChecksumFileSystem}}: This needed to change some of the file creation calls
it uses on the wrapped file system to pass down the mode. There is one awkward
point here that required an {{instanceof}} check, but I think it's preferable
over adding more abstract methods to {{FileSystem}} just to cover one special
case. Several calls are rerouted to equivalent overloaded methods that can
accept mode as an argument.
* {{NativeIO}}: Added Windows-specific JNI wrappers for a native implementation
of directory and file creation. For file creation, I needed to be careful to
match the semantics that were implemented by {{java.io.FileOutputStream}}.
This involved reading a bit of the JDK code, and I've described the details in
comments.
* {{NativeIO.c}}: Added JNI implementations for the 2 new functions. These are
pretty boring and just pass through to the real action in...
* {{libwinutils.c}}: This contains the implementation of calling the relevant
Windows APIs. I've reused our implementation of translating a mode to a
discretionary access control list ({{GetWindowsDACLs}}), so this will be
consistent with, for example, {{winutils.exe chmod}}.
* {{WindowsSecureContainerExecutor}}: This needed some adjustment to be able to
handle the changes required in {{RawLocalFileSystem}}. [~rusanu], this
implementation still does creation and setting permissions in separate
syscalls. I suppose we could consolidate to a single syscall if we update the
IDL for the RPC protocol to accept a mode argument, but I didn't want to tackle
that in this patch. I also don't know the compatibility constraints around
versioning this RPC protocol. This change was focused on a bug fix related to
SMB shares. I don't believe this is a major limitation for
{{ElevatedRawLocalFileSystem}}. If NodeManager is running backed by an SMB
share for the localization directory, then there are probably lots of other
problems anyway. :-) If you think it's important to come back to this, please
file a separate jira for that.
> copyToLocal cannot save a file to an SMB share unless the user has Full
> Control permissions.
> --------------------------------------------------------------------------------------------
>
> Key: HADOOP-11321
> URL: https://issues.apache.org/jira/browse/HADOOP-11321
> Project: Hadoop Common
> Issue Type: Bug
> Components: fs
> Affects Versions: 2.6.0
> Reporter: Chris Nauroth
> Assignee: Chris Nauroth
> Attachments: HADOOP-11321.003.patch, HADOOP-11321.1.patch,
> HADOOP-11321.2.patch, winutils.tmp.patch
>
>
> In Hadoop 2, it is impossible to use {{copyToLocal}} to copy a file from HDFS
> to a destination on an SMB share. This is because in Hadoop 2, the
> {{copyToLocal}} maps to 2 underlying {{RawLocalFileSystem}} operations:
> {{create}} and {{setPermission}}. On an SMB share, the user may be
> authorized for the {{create}} but denied for the {{setPermission}}. Windows
> denies the {{WRITE_DAC}} right required by {{setPermission}} unless the user
> has Full Control permissions. Granting Full Control isn't feasible for most
> deployments, because it's insecure. This is a regression from Hadoop 1,
> where {{copyToLocal}} only did a {{create}} and didn't do a separate
> {{setPermission}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
