Hi Ashish, The changes I was mentioning in the mail were changes in hive code. For howl, we will have howl specific semantic analyzers which will enforce authorization for ddl (like create table/drop table) against hdfs permissions. This is our initial thought on authorization for DDL through howl CLI - note, this does NOT change anything for hive CLI. I did notice there was jira in hive for authorization which seems similar to SQL authorization. The big issue there is reconciling SQL permissions with hdfs permissions (https://issues.apache.org/jira/browse/HIVE-78?focusedCommentId=12682719 &page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel #action_12682719)
So for now, we are going the hdfs route initially in howl - this may not be the final story for authorization in howl but is a beginning. I was initially thinking of using the hive conf variables and not table properties but I guess either could be used. Initially I thought we won't need to persist the group and permissions in the metastore and just use that info while creating the table dir. Later when creating partition dirs, we would just consult the table dir. If we use table properties, we can persist the group/perms info in the metastore and partition creation can get it from the metastore. Pradeep -----Original Message----- From: Ashish Thusoo [mailto:athu...@facebook.com] Sent: Thursday, July 29, 2010 3:01 PM To: hive-dev@hadoop.apache.org Cc: howl...@yahoogroups.com Subject: RE: [howldev] Initial thoughts on authorization in howl Hi Pradeep, I get from this note that the authorization that you are talking about here are basically the management of the permissions on the hdfs directories corresponding to the tables and the partitions. So from that angle this sounds good to me. There is a whole set of permissions/authorizations with regard to the metadata operations themselves eg. Who should be able to run an alter table add column or describe table etc. I presume that would be beyond the scope of this change and would come in later? I am thinking more in terms of the permissions model that is supported in SQL using GRANT statements etc. I also presume that by conf variables you mean the key value properties that Hive can store in the metadata and not the hive conf variables, right? Ashish -----Original Message----- From: John Sichi [mailto:jsi...@facebook.com] Sent: Wednesday, July 28, 2010 2:22 PM To: hive-dev@hadoop.apache.org Subject: Fwd: [howldev] Initial thoughts on authorization in howl Begin forwarded message: From: Pradeep Kamath <prade...@yahoo-inc.com<mailto:prade...@yahoo-inc.com>> Date: July 27, 2010 4:38:42 PM PDT To: <howl...@yahoogroups.com<mailto:howl...@yahoogroups.com>> Subject: [howldev] Initial thoughts on authorization in howl Reply-To: <howl...@yahoogroups.com<mailto:howl...@yahoogroups.com>> The initial thoughts on authorization in howl are to model authorization (for DDL ops like create table/drop table/add partition etc) after hdfs permissions. To be able to do this, we would like to extend createTable() to add the ability to record a different group from the user's primary group and to record the complete unix permissions on the table directory. Also, we would like to have a way for partition directories to inherit permissions and group information based on the table directory. To keep the metastore backward compatible for use with hive, I propose having conf variables to achieve these objectives: - table.group.name<http://table.group.name> - value will indicate the name of the unix group for the table directory. This will be used by createTable() to perform a chgrp to the value provided. This property will provide the user the ability to choose from one of the many unix groups he is part of to associate with the table. - table.permissions - value will be of the form rwxrwxrwx to indicate read-write-execute permissions on the table directory. This will be used by createTable() to perform a chmod to the value provided. This will let the user decide what permissions he wants on the table. - partitions.inherit.permissions - a value of true will indicate that partitions inherit the group name and permissions of the table level directory. This will be used by addPartition() to perform a chgrp and chmod to the values as on the table directory. I favor conf properties over API changes since the complete authorization design for hive is not finalized yet. These properties can be deprecated/removed when that is in place. These properties would also be useful to some installation of vanilla hive since at least DFS level authorization can now be achieved by hive without the user having to manually perform chgrp and chmod operations on DFS. I would like to hear from hive developers/committers whether this would be acceptable for hive and also thoughts from others. Pradeep __._,_.___ Your email settings: Individual Email|Traditional Change settings via the Web<http://groups.yahoo.com/group/howldev/join;_ylc=X3oDMTJnZXE5ZHNwBF9T Azk3NDc2NTkwBGdycElkAzYzNDIwNTA4BGdycHNwSWQDMTcwNzI4MTk0MgRzZWMDZnRyBHNs awNzdG5ncwRzdGltZQMxMjgwMjczOTQ2> (Yahoo! ID required) Change settings via email: Switch delivery to Daily Digest<mailto:howldev-dig...@yahoogroups.com?subject=email%20delivery:%2 0Digest> | Switch to Fully Featured<mailto:howldev-fullfeatu...@yahoogroups.com?subject=change%20de livery%20Format:%20Fully%20Featured> Visit Your Group <http://groups.yahoo.com/group/howldev;_ylc=X3oDMTJlOWw0Y3F0BF9TAzk3NDc2 NTkwBGdycElkAzYzNDIwNTA4BGdycHNwSWQDMTcwNzI4MTk0MgRzZWMDZnRyBHNsawNocGYE c3RpbWUDMTI4MDI3Mzk0Ng--> | Yahoo! Groups Terms of Use <http://docs.yahoo.com/info/terms/> | Unsubscribe <mailto:howldev-unsubscr...@yahoogroups.com?subject=unsubscribe> __,_._,___
