> On 2010-08-04 22:40:35, stack wrote: > > conf/hadoop-policy.xml, line 1 > > <http://review.cloudera.org/r/406/diff/2/?file=3872#file3872line1> > > > > Should it be named hbase-policy.xml or is it a case of the name of the > > file being hard-coded (as it is for metrics config -- see > > hadoop-metrics.xml in the hbase/conf dir).
The default is down in org.apache.hadoop.security.authorize.ServiceAuthorizationManager, but looks like we can override it with "hadoop.policy.file" in config. I'll do that and update. > On 2010-08-04 22:40:35, stack wrote: > > pom.xml, line 480 > > <http://review.cloudera.org/r/406/diff/2/?file=3873#file3873line480> > > > > How much of this patch do you want to commit to TRUNK Gary? Would hbase > > 0.90.0 have a dependency on secure hadoop and if so are there implications > > (performance?) or are you thinking it secure hbase a separate offering, > > somehow? At the moment, none of this is really eligible for trunk, due to dependency on secure hadoop. I'm looking at enabling pluggable RPC in HBASE-2321 as a way to introduce secure RPC without the hard dependency. Don't know how messy that will get, but I should have a good idea in the next few days. The idea would be to make secure RPC a conditional build target and enable optional configuration of it as the RPC implementation, so that it doesn't block the ability of HBase to run on Hadoop 0.20 versions without the security features. As long as that proves out, I would use HBASE-2321 as the framework for pluggable RPC, with a new JIRA for the secure RPC plugin. > On 2010-08-04 22:40:35, stack wrote: > > src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java, line 85 > > <http://review.cloudera.org/r/406/diff/2/?file=3888#file3888line85> > > > > What changed in here? Looks like just a whitespace change, I'll remove. In one of the intermediate changes I had wrapped this in UserGroupInformation.doAs() to enable execution of RS's as different users in LocalHBaseCluster (since some tests require this). But I would up moving out to MiniHBaseCluster to keep it isolated to test cases. > On 2010-08-04 22:40:35, stack wrote: > > src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java, line 116 > > <http://review.cloudera.org/r/406/diff/2/?file=3875#file3875line116> > > > > Man, I hate this final stuff (I know its not you Gary.... you're just > > bringing it over... thats fine... but I hate it anyways). Understood and I don't disagree. :) Most of this RPC code is not very extensible and sometimes serves a common case, but sometimes frustrates it. Maybe HBASE-2321 can help towards trying alternatives. > On 2010-08-04 22:40:35, stack wrote: > > src/main/java/org/apache/hadoop/hbase/ipc/ConnectionHeader.java, line 1 > > <http://review.cloudera.org/r/406/diff/2/?file=3874#file3874line1> > > > > If we are running in insecure mode, does this thing send the only > > single-byte or whatever it was identifying the rpc? Or, to ask in another > > way, does insecure rpc have same character with this patch as it does w/o > > it? This does change the over the wire format for both secure and insecure cases in an incompatible way. Existing format is: "hrpc" version (3) UGI ticket "org.apache.hadoop.security.UserGroupInformation" "org.apache.hadoop.security.UnixUserGroupInformation" "STRING_UGI" username group names length each group name However the UGI ticket always seems to be null so none of that is passed. New format is: "hrpc" version (4) auth method (single byte code) short class name (HMasterInterface, HRegionInterface, HMasterRegionInterface) username present? (true|false) + if true username real username present? (true|false) + if true real username - Gary ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: http://review.cloudera.org/r/406/#review680 ----------------------------------------------------------- On 2010-07-29 12:40:06, Gary Helmling wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > http://review.cloudera.org/r/406/ > ----------------------------------------------------------- > > (Updated 2010-07-29 12:40:06) > > > Review request for hbase. > > > Summary > ------- > > This patch ports over the secure Hadoop RPC changes from the latest Yahoo > 0.20 based branch (yahoo-hadoop-0.20.104). This patch is produced against > HBase trunk, but is targeted as the first step in a "security" feature branch > for a full role-based access control implementation (HBASE-1697). > > RPC Changes > -------------------- > The primary changes are updates from the classes: > org.apache.hadoop.ipc.Client -> org.apache.hadoop.hbase.ipc.HBaseClient > org.apache.hadoop.ipc.RPC -> org.apache.hadoop.hbase.ipc.HBaseRPC > org.apache.hadoop.ipc.Server -> org.apache.hadoop.hbase.ipc.HBaseServer > > The new classes were also ported: > org.apache.hadoop.hbase.security.HBaseSaslRpcClient > org.apache.hadoop.hbase.security.HBaseSaslRpcServer > > Due to type dependencies on the Hadoop RPC classes, the original Hadoop > SaslRpc* classes could not be used. > > The RPC port provides client authentication via Kerberos, and SASL > negotiation of client server connections for mutual authentication and > optionally encryption, so it also provides the authentication functionality > for HBASE-2016. The ported RPC code contains dependencies on other classes > in secure Hadoop/Hadoop trunk, preventing it from currently running on 0.20 > branches missing the security changes. > > Process Authentication > --------------------------- > The HMaster and HRegionServer processes have been updated to allow > configuration of the Kerberos principals used to run the processes. The new > configuration parameters are: > > * hbase.master.keytab.file - Path to the keytab file containing the master > principal's credentials > * hbase.master.kerberos.principal - Kerberos principal name used to login the > HMaster process > * hbase.master.kerberos.https.principal - Kerberos principal name used to > login the HMaster info server > * hbase.regionserver.keytab.file - Path to the keytab file containing the > region server's credentials > * hbase.regionserver.kerberos.principal - Kerberos principal name used to > login the HRegionServer process > * hbase.regionserver.kerberos.https.principal - Kerberos principal name used > to login the HRegionServer info server > > The new class org.apache.hadoop.hbase.security.HBasePolicyProvider and new > file conf/hadoop-policy.xml allow restriction of the users and groups > permitting to utilize each of the RPC protocol interfaces (HMasterInterface, > HMasterRegionInterface, HRegionInterface). > > Testing Updates > -------------------- > Parts of the test code (org.apache.hadoop.hbase.HBaseTestingUtility and > org.apache.hadoop.hbase.MiniHBaseCluster) were directly using the internal > Hadoop UnixUserGroupInformation class to manipulate process ownership for > testing. These have been updated to use UserGroupInformation.doAs() instead. > > > This addresses bugs HBASE-2016 and HBASE-2742. > http://issues.apache.org/jira/browse/HBASE-2016 > http://issues.apache.org/jira/browse/HBASE-2742 > > > Diffs > ----- > > conf/hadoop-policy.xml PRE-CREATION > pom.xml 2d3d75a > src/main/java/org/apache/hadoop/hbase/ipc/ConnectionHeader.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/ipc/HBaseClient.java 2b5eeb6 > src/main/java/org/apache/hadoop/hbase/ipc/HBaseRPC.java 9873172 > src/main/java/org/apache/hadoop/hbase/ipc/HBaseRpcMetrics.java d88c12d > src/main/java/org/apache/hadoop/hbase/ipc/HBaseServer.java d3c6c21 > src/main/java/org/apache/hadoop/hbase/ipc/HMasterInterface.java bd48a4b > src/main/java/org/apache/hadoop/hbase/ipc/HMasterRegionInterface.java > 71a0447 > src/main/java/org/apache/hadoop/hbase/ipc/HRegionInterface.java 1157fe1 > src/main/java/org/apache/hadoop/hbase/ipc/Status.java PRE-CREATION > src/main/java/org/apache/hadoop/hbase/master/HMaster.java e4bd30d > src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java > 6a54736 > src/main/java/org/apache/hadoop/hbase/security/HBasePolicyProvider.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcClient.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/security/HBaseSaslRpcServer.java > PRE-CREATION > src/main/java/org/apache/hadoop/hbase/util/JVMClusterUtil.java 280b91d > src/main/resources/hbase-default.xml e3a9669 > src/test/java/org/apache/hadoop/hbase/HBaseTestingUtility.java 4d09fe9 > src/test/java/org/apache/hadoop/hbase/MiniHBaseCluster.java 9c49e36 > src/test/java/org/apache/hadoop/hbase/regionserver/TestStore.java 0b47975 > src/test/java/org/apache/hadoop/hbase/regionserver/wal/TestWALReplay.java > c982662 > > Diff: http://review.cloudera.org/r/406/diff > > > Testing > ------- > > > Thanks, > > Gary > >
