[ 
https://issues.apache.org/jira/browse/HADOOP-1298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12519205
 ] 

kryzthov edited comment on HADOOP-1298 at 8/11/07 11:26 AM:
--------------------------------------------------------------------

Here is a quite complete description of the current state of this work so that 
you know what is going here.

The patch relies on a subset of the authentication framework that should be 
provided in the future by this issue 
[HADOOP-1701|https://issues.apache.org/jira/browse/HADOOP-1701]. It then 
includes a subset of this framework for now and until HADOOP-1701 will be 
committed.
It provides:
* authorizations for most DFS File operations by extending the Java 
authorization framework
* a subset of POSIX permissions (no groups for now and no "execution" right)

To preserve compatibility, the default permission for all files is: owner=root 
and POSIX permissions are rwx---rwx. All entities (web apps, fsck, clients...) 
for now use a fake Root user account. The authorization framework should allow 
an easy transition toward a permission checked environment.

The patch is about 4000 lines (including HADOOP-1701 code). However it is still 
*incomplete* (see the Todo list at the end of this message). All JUnit tests 
pass.

h3.Changes

* dfs.ClientProtocol:
*- added a Ticket parameter for most operations
*- added a setPermissions(Path, PermissionCollection) operation

* dfs.DFSClient:
*- added a Ticket parameter for most operations (symmetric of 
dfs.ClientProtocol)
*- dfs.DFSClient.DFSInputStream and DFSOutputStream: added a Ticket field 
(these are stateful objects that keep the Ticket used to create them for 
further operations)

* dfs.DFSPolicy:
*- new file which implements a policy provider that relies on INode's 
permissions data to generate policies; the current DFSPolicy concretely 
implements the POSIX permission checking scheme for now (without groups)

* dfs.DFSSecurityManager:
*- new file which implements the SecurityManager for the JVM that runs the DFS
*- it currently ignores OS permissions checking as we trust Hadoop DFS code, 
and only processes fs.permissions.HFilePermission
*- it also provides one helper method for each type of HFileAction we can check 
for

* dfs.DistributedFileSystem:
*- added a Ticket parameter implicitely used for all the operations provided by 
this client interface
*- the Ticket is defaulted to a fake ROOT user for now so as to preserve 
compatibility, and we will restrict this progressively after

* dfs.DSConstants:
*- updated the layout version

* dfs.FSDirectory:
*- added a WritablePermissionCollection field to INode; this field is defaulted 
to owner=root and mode=rwx---rwx so as to preserve compatibility as much as 
possible and to allow easy and progressive restrictions to be integrated in the 
future
*- added a getter and a setter for INode.permissions
*- added a INode getDeepestExistingNode(String path) method that copies the 
INode getNode(String path) method but always returns an existing INode (the 
deepest one in the INode tree that leads to the full given path); this to allow 
easy permission checking against the already existing directory along the given 
path.

* dfs.FSImage:
*- updated the saveImage() method to write the permissions
*- updated the loadFSImage() method to read the permissions or provide a 
default permissions (owner=root and mode=rwx---rwx) when upgrading from a 
previous layout

* dfs.FSNamesystem:
*- added a setPermissions() operation

* dfs.NameNode:
*- updated the init() method to setup the SecurityManager and the Policy 
provider for the JVM
*- scattered 
DFSSecurityManager.checkDFS{Read,Write,Create,Delete,SetPermissions}() in all 
operations
*- added a setPermissions() (to implement dfs.ClientProtocol)
*- added a few method to setup the appropriate Subject in the 
AccessControlContext (getSubjectOfMessage() and doAsSubject())
*- wrapped all operation that have a Ticket parameter so as to extract the 
appropriate Subject from the Ticket and run the real operation under this 
Subject's principal.

* dfs.NamenodeFsck:
*- updated to used a default Root Ticket

* dfs.StreamFile:
*- updated to use a default Root Ticket

* fs.permission: new package
*- fs.permission.HFileAction: represents the 5 actions related to DFS Files 
(create, delete, read, write, set permissions)
*- fs.permission.HFilePermission: unmutable Permission object to represent a 
user requested action on a file
*- fs.permission.POSIXFileAction: represents the 3 POSIX file actions (read, 
write, execute)
*- fs.permission.POSIXFilePermission: unmutable Permission object to represent 
a user requested POSIX action on a file
*- fs.permission.POSIXFilePermissions: mutable WritablePermissionCollection 
that stores the current permissions for an INode in terms of POSIX actions 
(contains an owner ID and a mode)
*- fs.permission.WritablePermissionCollection: abstract PermissionCollection 
that must implement Writable also

* ipc.Client:
*- updated Writable call(Writable param) so that it can propagates 
AccessControlException to the user process

* ipc.RPC:
*- updated Writable RPC.Server.call(Writable param) so that it correctly 
serializes AccessControlException

* security.simple.SimpleTicket:
*- updated to provide 3 fake users (for compatibility and for testing purpose)

* Tests
*- dfs.ClusterTestDFS, dfs.ClusterTestDFSNamespaceLogging, dfs.TestReplication: 
use the default Root Ticket
*- dfs.TestPermission: new test case to test the underlying permission checking 
infrastructure a bit

* Web apps
*- browseBlock.jsp, browseDirectory.jsp, tail.jsp: updated to use the default 
Root Ticket (use DistributedFileSystem)


h3.To do

* High priority
*- Update the fsEditLog for the setPermissions() operation
*- Figure out how to provide a getPermissions() and to extend fs.FileStatus for 
clients
*- FsShell command to get/set permissions
*- Deep tests
*- real use of authentication

* Low priority
*- Sticky bit (really helpful to have a shared /scratch or /tmp directory where 
you don't want others to remove your temporary files!)
*- Groups?


That's it for now!

I hope this will help you to get a better idea of how this has been designed 
for now.
Any comment will be highly appreciated.

Thanks,
Christophe T.

      was (Author: kryzthov):
    Here is a quite complete description of the current state of this work so 
that you know what is going here.

The patch relies on a subset of the authentication framework that should be 
provided in the future by this issue 
[HADOOP-1701|https://issues.apache.org/jira/browse/HADOOP-1701]. It then 
includes a subset of this framework for now and until HADOOP-1701 will be 
committed.
It provides:
* authorizations for most DFS File operations by extending the Java 
authorization framework
* a subset of POSIX permissions (no groups for now and no "execution" right)

To preserve compatibility, the default permission for all files is: owner=root 
and POSIX permissions are rwx---rwx. All entities (web apps, fsck, clients...) 
for now use a fake Root user account. The authorization framework should allow 
an easy transition toward a permission checked environment.

The patch is about 4000 lines (including HADOOP-1701 code). However it is still 
*incomplete* (see the Todo list at the end of this message). All JUnit tests 
pass.

h3.Changes

* dfs.ClientProtocol:
*- added a Ticket parameter for most operations
*- added a setPermissions(Path, PermissionCollection) operation

* dfs.DFSClient:
*- added a Ticket parameter for most operations (symmetric of 
dfs.ClientProtocol)
*- dfs.DFSClient.DFSInputStream and DFSOutputStream: added a Ticket field 
(these are stateful objects that keep the Ticket used to create them for 
further operations)

* dfs.DFSPolicy:
*- new file which implements a policy provider that relies on INode's 
permissions data to generate policies; the current DFSPolicy concretely 
implements the POSIX permission checking scheme for now (without groups)

* dfs.DFSSecurityManager:
*- new file which implements the SecurityManager for the JVM that runs the DFS
*- it currently ignores OS permissions checking as we trust Hadoop DFS code, 
and only processes fs.permissions.HFilePermission
*- it also provides one helper method for each type of HFileAction we can check 
for

* dfs.DistributedFileSystem:
*- added a Ticket parameter implicitely used for all the operations provided by 
this client interface
*- the Ticket is defaulted to a fake ROOT user for now so as to preserve 
compatibility, and we will restrict this progressively after

* dfs.DSConstants:
*- updated the layout version

* dfs.FSDirectory:
*- added a WritablePermissionCollection field to INode; this field is defaulted 
to owner=root and mode=rwx---rwx so as to preserve compatibility as much as 
possible and to allow easy and progressive restrictions to be integrated in the 
future
*- added a getter and a setter for INode.permissions
*- added a INode getDeepestExistingNode(String path) method that copies the 
INode getNode(String path) method but always returns an existing INode (the 
deepest one in the INode tree that leads to the full given path); this to allow 
easy permission checking against the already existing directory along the given 
path.

* dfs.FSImage:
*- updated the saveImage() method to write the permissions
*- updated the loadFSImage() method to read the permissions or provide a 
default permissions (owner=root and mode=rwx---rwx) when upgrading from a 
previous layout

* dfs.FSNamesystem:
*- added a setPermissions() operation

* dfs.NameNode:
*- updated the init() method to setup the SecurityManager and the Policy 
provider for the JVM
*- scattered 
DFSSecurityManager.checkDFS{Read,Write,Create,Delete,SetPermissions}() in all 
operations
*- added a setPermissions() (to implement dfs.ClientProtocol)
*- added a few method to setup the appropriate Subject in the 
AccessControlContext (getSubjectOfMessage() and doAsSubject())
*- wrapped all operation that have a Ticket parameter so as to extract the 
appropriate Subject from the Ticket and run the real operation under this 
Subject's principal.

* dfs.NamenodeFsck:
*- updated to used a default Root Ticket

* dfs.StreamFile:
*- updated to use a default Root Ticket

*fs.permission: new package
*- fs.permission.HFileAction: represents the 5 actions related to DFS Files 
(create, delete, read, write, set permissions)
*- fs.permission.HFilePermission: unmutable Permission object to represent a 
user requested action on a file
*- fs.permission.POSIXFileAction: represents the 3 POSIX file actions (read, 
write, execute)
*- fs.permission.POSIXFilePermission: unmutable Permission object to represent 
a user requested POSIX action on a file
*- fs.permission.POSIXFilePermissions: mutable WritablePermissionCollection 
that stores the current permissions for an INode in terms of POSIX actions 
(contains an owner ID and a mode)
*- fs.permission.WritablePermissionCollection: abstract PermissionCollection 
that must implement Writable also

* ipc.Client:
*- updated Writable call(Writable param) so that it can propagates 
AccessControlException to the user process

* ipc.RPC:
*- updated Writable RPC.Server.call(Writable param) so that it correctly 
serializes AccessControlException

* security.simple.SimpleTicket:
*- updated to provide 3 fake users (for compatibility and for testing purpose)

* Tests
*- dfs.ClusterTestDFS, dfs.ClusterTestDFSNamespaceLogging, dfs.TestReplication: 
use the default Root Ticket
*- dfs.TestPermission: new test case to test the underlying permission checking 
infrastructure a bit

* Web apps
*- browseBlock.jsp, browseDirectory.jsp, tail.jsp: updated to use the default 
Root Ticket (use DistributedFileSystem)


h3.To do

* High priority
*- Update the fsEditLog for the setPermissions() operation
*- Figure out how to provide a getPermissions() and to extend fs.FileStatus for 
clients
*- FsShell command to get/set permissions
*- Deep tests
*- real use of authentication

* Low priority
*- Sticky bit (really helpful to have a shared /scratch or /tmp directory where 
you don't want others to remove your temporary files!)
*- Groups?


That's it for now!

I hope this will help you to get a better idea of how this has been designed 
for now.
Any comment will be highly appreciated.

Thanks,
Christophe T.
  
> adding user info to file
> ------------------------
>
>                 Key: HADOOP-1298
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1298
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs, fs
>            Reporter: Kurtis Heimerl
>             Fix For: 0.15.0
>
>         Attachments: hadoop-authorization-20070810b.patch, 
> hadoop-user-munncha.patch, hadoop-user-munncha.patch, 
> hadoop-user-munncha.patch, hadoop-user-munncha.patch10, 
> hadoop-user-munncha.patch11, hadoop-user-munncha.patch12, 
> hadoop-user-munncha.patch13, hadoop-user-munncha.patch14, 
> hadoop-user-munncha.patch15, hadoop-user-munncha.patch16, 
> hadoop-user-munncha.patch17, hadoop-user-munncha.patch4, 
> hadoop-user-munncha.patch5, hadoop-user-munncha.patch6, 
> hadoop-user-munncha.patch7, hadoop-user-munncha.patch8, 
> hadoop-user-munncha.patch9, layout20070731.patch
>
>
> I'm working on adding a permissions model to hadoop's DFS. The first step is 
> this change, which associates user info with files. Following this I'll 
> assoicate permissions info, then block methods based on that user info, then 
> authorization of the user info. 
> So, right now i've implemented adding user info to files. I'm looking for 
> feedback before I clean this up and make it offical. 
> I wasn't sure what release, i'm working off trunk. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.

Reply via email to