[ https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519205 ]
kryzthov edited comment on HADOOP-1298 at 8/11/07 11:26 AM: -------------------------------------------------------------------- Here is a quite complete description of the current state of this work so that you know what is going here. The patch relies on a subset of the authentication framework that should be provided in the future by this issue [HADOOP-1701|https://issues.apache.org/jira/browse/HADOOP-1701]. It then includes a subset of this framework for now and until HADOOP-1701 will be committed. It provides: * authorizations for most DFS File operations by extending the Java authorization framework * a subset of POSIX permissions (no groups for now and no "execution" right) To preserve compatibility, the default permission for all files is: owner=root and POSIX permissions are rwx---rwx. All entities (web apps, fsck, clients...) for now use a fake Root user account. The authorization framework should allow an easy transition toward a permission checked environment. The patch is about 4000 lines (including HADOOP-1701 code). However it is still *incomplete* (see the Todo list at the end of this message). All JUnit tests pass. h3.Changes * dfs.ClientProtocol: *- added a Ticket parameter for most operations *- added a setPermissions(Path, PermissionCollection) operation * dfs.DFSClient: *- added a Ticket parameter for most operations (symmetric of dfs.ClientProtocol) *- dfs.DFSClient.DFSInputStream and DFSOutputStream: added a Ticket field (these are stateful objects that keep the Ticket used to create them for further operations) * dfs.DFSPolicy: *- new file which implements a policy provider that relies on INode's permissions data to generate policies; the current DFSPolicy concretely implements the POSIX permission checking scheme for now (without groups) * dfs.DFSSecurityManager: *- new file which implements the SecurityManager for the JVM that runs the DFS *- it currently ignores OS permissions checking as we trust Hadoop DFS code, and only processes fs.permissions.HFilePermission *- it also provides one helper method for each type of HFileAction we can check for * dfs.DistributedFileSystem: *- added a Ticket parameter implicitely used for all the operations provided by this client interface *- the Ticket is defaulted to a fake ROOT user for now so as to preserve compatibility, and we will restrict this progressively after * dfs.DSConstants: *- updated the layout version * dfs.FSDirectory: *- added a WritablePermissionCollection field to INode; this field is defaulted to owner=root and mode=rwx---rwx so as to preserve compatibility as much as possible and to allow easy and progressive restrictions to be integrated in the future *- added a getter and a setter for INode.permissions *- added a INode getDeepestExistingNode(String path) method that copies the INode getNode(String path) method but always returns an existing INode (the deepest one in the INode tree that leads to the full given path); this to allow easy permission checking against the already existing directory along the given path. * dfs.FSImage: *- updated the saveImage() method to write the permissions *- updated the loadFSImage() method to read the permissions or provide a default permissions (owner=root and mode=rwx---rwx) when upgrading from a previous layout * dfs.FSNamesystem: *- added a setPermissions() operation * dfs.NameNode: *- updated the init() method to setup the SecurityManager and the Policy provider for the JVM *- scattered DFSSecurityManager.checkDFS{Read,Write,Create,Delete,SetPermissions}() in all operations *- added a setPermissions() (to implement dfs.ClientProtocol) *- added a few method to setup the appropriate Subject in the AccessControlContext (getSubjectOfMessage() and doAsSubject()) *- wrapped all operation that have a Ticket parameter so as to extract the appropriate Subject from the Ticket and run the real operation under this Subject's principal. * dfs.NamenodeFsck: *- updated to used a default Root Ticket * dfs.StreamFile: *- updated to use a default Root Ticket * fs.permission: new package *- fs.permission.HFileAction: represents the 5 actions related to DFS Files (create, delete, read, write, set permissions) *- fs.permission.HFilePermission: unmutable Permission object to represent a user requested action on a file *- fs.permission.POSIXFileAction: represents the 3 POSIX file actions (read, write, execute) *- fs.permission.POSIXFilePermission: unmutable Permission object to represent a user requested POSIX action on a file *- fs.permission.POSIXFilePermissions: mutable WritablePermissionCollection that stores the current permissions for an INode in terms of POSIX actions (contains an owner ID and a mode) *- fs.permission.WritablePermissionCollection: abstract PermissionCollection that must implement Writable also * ipc.Client: *- updated Writable call(Writable param) so that it can propagates AccessControlException to the user process * ipc.RPC: *- updated Writable RPC.Server.call(Writable param) so that it correctly serializes AccessControlException * security.simple.SimpleTicket: *- updated to provide 3 fake users (for compatibility and for testing purpose) * Tests *- dfs.ClusterTestDFS, dfs.ClusterTestDFSNamespaceLogging, dfs.TestReplication: use the default Root Ticket *- dfs.TestPermission: new test case to test the underlying permission checking infrastructure a bit * Web apps *- browseBlock.jsp, browseDirectory.jsp, tail.jsp: updated to use the default Root Ticket (use DistributedFileSystem) h3.To do * High priority *- Update the fsEditLog for the setPermissions() operation *- Figure out how to provide a getPermissions() and to extend fs.FileStatus for clients *- FsShell command to get/set permissions *- Deep tests *- real use of authentication * Low priority *- Sticky bit (really helpful to have a shared /scratch or /tmp directory where you don't want others to remove your temporary files!) *- Groups? That's it for now! I hope this will help you to get a better idea of how this has been designed for now. Any comment will be highly appreciated. Thanks, Christophe T. was (Author: kryzthov): Here is a quite complete description of the current state of this work so that you know what is going here. The patch relies on a subset of the authentication framework that should be provided in the future by this issue [HADOOP-1701|https://issues.apache.org/jira/browse/HADOOP-1701]. It then includes a subset of this framework for now and until HADOOP-1701 will be committed. It provides: * authorizations for most DFS File operations by extending the Java authorization framework * a subset of POSIX permissions (no groups for now and no "execution" right) To preserve compatibility, the default permission for all files is: owner=root and POSIX permissions are rwx---rwx. All entities (web apps, fsck, clients...) for now use a fake Root user account. The authorization framework should allow an easy transition toward a permission checked environment. The patch is about 4000 lines (including HADOOP-1701 code). However it is still *incomplete* (see the Todo list at the end of this message). All JUnit tests pass. h3.Changes * dfs.ClientProtocol: *- added a Ticket parameter for most operations *- added a setPermissions(Path, PermissionCollection) operation * dfs.DFSClient: *- added a Ticket parameter for most operations (symmetric of dfs.ClientProtocol) *- dfs.DFSClient.DFSInputStream and DFSOutputStream: added a Ticket field (these are stateful objects that keep the Ticket used to create them for further operations) * dfs.DFSPolicy: *- new file which implements a policy provider that relies on INode's permissions data to generate policies; the current DFSPolicy concretely implements the POSIX permission checking scheme for now (without groups) * dfs.DFSSecurityManager: *- new file which implements the SecurityManager for the JVM that runs the DFS *- it currently ignores OS permissions checking as we trust Hadoop DFS code, and only processes fs.permissions.HFilePermission *- it also provides one helper method for each type of HFileAction we can check for * dfs.DistributedFileSystem: *- added a Ticket parameter implicitely used for all the operations provided by this client interface *- the Ticket is defaulted to a fake ROOT user for now so as to preserve compatibility, and we will restrict this progressively after * dfs.DSConstants: *- updated the layout version * dfs.FSDirectory: *- added a WritablePermissionCollection field to INode; this field is defaulted to owner=root and mode=rwx---rwx so as to preserve compatibility as much as possible and to allow easy and progressive restrictions to be integrated in the future *- added a getter and a setter for INode.permissions *- added a INode getDeepestExistingNode(String path) method that copies the INode getNode(String path) method but always returns an existing INode (the deepest one in the INode tree that leads to the full given path); this to allow easy permission checking against the already existing directory along the given path. * dfs.FSImage: *- updated the saveImage() method to write the permissions *- updated the loadFSImage() method to read the permissions or provide a default permissions (owner=root and mode=rwx---rwx) when upgrading from a previous layout * dfs.FSNamesystem: *- added a setPermissions() operation * dfs.NameNode: *- updated the init() method to setup the SecurityManager and the Policy provider for the JVM *- scattered DFSSecurityManager.checkDFS{Read,Write,Create,Delete,SetPermissions}() in all operations *- added a setPermissions() (to implement dfs.ClientProtocol) *- added a few method to setup the appropriate Subject in the AccessControlContext (getSubjectOfMessage() and doAsSubject()) *- wrapped all operation that have a Ticket parameter so as to extract the appropriate Subject from the Ticket and run the real operation under this Subject's principal. * dfs.NamenodeFsck: *- updated to used a default Root Ticket * dfs.StreamFile: *- updated to use a default Root Ticket *fs.permission: new package *- fs.permission.HFileAction: represents the 5 actions related to DFS Files (create, delete, read, write, set permissions) *- fs.permission.HFilePermission: unmutable Permission object to represent a user requested action on a file *- fs.permission.POSIXFileAction: represents the 3 POSIX file actions (read, write, execute) *- fs.permission.POSIXFilePermission: unmutable Permission object to represent a user requested POSIX action on a file *- fs.permission.POSIXFilePermissions: mutable WritablePermissionCollection that stores the current permissions for an INode in terms of POSIX actions (contains an owner ID and a mode) *- fs.permission.WritablePermissionCollection: abstract PermissionCollection that must implement Writable also * ipc.Client: *- updated Writable call(Writable param) so that it can propagates AccessControlException to the user process * ipc.RPC: *- updated Writable RPC.Server.call(Writable param) so that it correctly serializes AccessControlException * security.simple.SimpleTicket: *- updated to provide 3 fake users (for compatibility and for testing purpose) * Tests *- dfs.ClusterTestDFS, dfs.ClusterTestDFSNamespaceLogging, dfs.TestReplication: use the default Root Ticket *- dfs.TestPermission: new test case to test the underlying permission checking infrastructure a bit * Web apps *- browseBlock.jsp, browseDirectory.jsp, tail.jsp: updated to use the default Root Ticket (use DistributedFileSystem) h3.To do * High priority *- Update the fsEditLog for the setPermissions() operation *- Figure out how to provide a getPermissions() and to extend fs.FileStatus for clients *- FsShell command to get/set permissions *- Deep tests *- real use of authentication * Low priority *- Sticky bit (really helpful to have a shared /scratch or /tmp directory where you don't want others to remove your temporary files!) *- Groups? That's it for now! I hope this will help you to get a better idea of how this has been designed for now. Any comment will be highly appreciated. Thanks, Christophe T. > adding user info to file > ------------------------ > > Key: HADOOP-1298 > URL: https://issues.apache.org/jira/browse/HADOOP-1298 > Project: Hadoop > Issue Type: New Feature > Components: dfs, fs > Reporter: Kurtis Heimerl > Fix For: 0.15.0 > > Attachments: hadoop-authorization-20070810b.patch, > hadoop-user-munncha.patch, hadoop-user-munncha.patch, > hadoop-user-munncha.patch, hadoop-user-munncha.patch10, > hadoop-user-munncha.patch11, hadoop-user-munncha.patch12, > hadoop-user-munncha.patch13, hadoop-user-munncha.patch14, > hadoop-user-munncha.patch15, hadoop-user-munncha.patch16, > hadoop-user-munncha.patch17, hadoop-user-munncha.patch4, > hadoop-user-munncha.patch5, hadoop-user-munncha.patch6, > hadoop-user-munncha.patch7, hadoop-user-munncha.patch8, > hadoop-user-munncha.patch9, layout20070731.patch > > > I'm working on adding a permissions model to hadoop's DFS. The first step is > this change, which associates user info with files. Following this I'll > assoicate permissions info, then block methods based on that user info, then > authorization of the user info. > So, right now i've implemented adding user info to files. I'm looking for > feedback before I clean this up and make it offical. > I wasn't sure what release, i'm working off trunk. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.