This is a bit of an X-Y discussion. The error here is not ZooKeeper related in any way:
> 12/06/09 16:40:47 WARN ipc.HBaseServer: IPC Server listener on 60000: > readAndProcess threw exception > org.apache.hadoop.security.AccessControlException: Authentication is > required. Count of bytes read: 0 > org.apache.hadoop.security.AccessControlException: Authentication is required This says the Hadoop RPC client is configured for AuthMethod.SIMPLE but what the HBase master wants is AuthMethod.KERBEROS. See http://hbase.apache.org/book/security.html and insure the server and client side configurations conform before we should proceed further. Separately, > BTW: I knew that the hbase master authenticated to zookeeper via quorum > ensemble in order to find where hadoop dfs lives, but I didn't realize the > region servers and hbase clients also needed to authenticate to zookeeper. > Explain? The master and regionservers authenticate to ZooKeeper to access protected znodes that serve a variety of functions. If clients could access them, such clients could subvert system functions including security. So we set up typically a "hbase" service principal and only that principal can access those protected znodes. Thus all HBase daemons must authenticate with ZooKeeper using that principal. Clients do not need to authenticate with ZooKeeper. The znodes which clients must access are not sensitive and do not have restrictive ACLs. (Although clients wanting to take administrative action currently must, see https://issues.apache.org/jira/browse/HBASE-6068.) On Sat, Jun 9, 2012 at 1:51 PM, Tony Dean <[email protected]> wrote: > Hi Harsh, > > Thanks for re-routing to HBase user-group. ;-) > > I followed the same steps as you, at least I tried to. > > My cluster appears to be working and I outlined my client configuration below. > > BTW: I knew that the hbase master authenticated to zookeeper via quorum > ensemble in order to find where hadoop dfs lives, but I didn't realize the > region servers and hbase clients also needed to authenticate to zookeeper. > Explain? > > Anyway, here are the traces that I collected. > > HBase master: > > 12/06/09 16:40:36 DEBUG security.HBaseSaslRpcClient: Will send token of size > 50 from initSASLContext. > 12/06/09 16:40:36 DEBUG security.HBaseSaslRpcClient: SASL client context > established. Negotiated QoP: auth > 12/06/09 16:40:47 WARN ipc.HBaseServer: IPC Server listener on 60000: > readAndProcess threw exception > org.apache.hadoop.security.AccessControlException: Authentication is > required. Count of bytes read: 0 > org.apache.hadoop.security.AccessControlException: Authentication is required > at > org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:414) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) > 12/06/09 16:40:48 WARN ipc.HBaseServer: IPC Server listener on 60000: > readAndProcess threw exception > org.apache.hadoop.security.AccessControlException: Authentication is > required. Count of bytes read: 0 > org.apache.hadoop.security.AccessControlException: Authentication is required > at > org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:414) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) > 12/06/09 16:40:49 WARN ipc.HBaseServer: IPC Server listener on 60000: > readAndProcess threw exception > org.apache.hadoop.security.AccessControlException: Authentication is > required. Count of bytes read: 0 > org.apache.hadoop.security.AccessControlException: Authentication is required > at > org.apache.hadoop.hbase.ipc.SecureServer$SecureConnection.readAndProcess(SecureServer.java:414) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:703) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.doRunLoop(HBaseServer.java:495) > at > org.apache.hadoop.hbase.ipc.HBaseServer$Listener$Reader.run(HBaseServer.java:470) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) > at java.lang.Thread.run(Thread.java:619) > > > zookeeper: > > 12/06/09 16:40:47 DEBUG server.ZooKeeperServer: Responding to client SASL > token. > 12/06/09 16:40:47 DEBUG server.ZooKeeperServer: Size of client SASL token: 67 > Krb5Context.unwrap: token=[60 41 06 09 2a 86 48 86 f7 12 01 02 02 02 01 11 00 > ff ff ff ff 65 66 6b 58 fd f1 6b ec 27 53 22 23 5d 7b 03 33 0b e3 2d 7f d3 a9 > 13 62 01 01 00 00 73 61 73 70 61 64 40 4e 41 2e 53 41 53 2e 43 4f 4d 01 ] > Krb5Context.unwrap: data=[01 01 00 00 73 61 73 70 61 64 40 4e 41 2e 53 41 53 > 2e 43 4f 4d ] > 12/06/09 16:40:47 INFO auth.SaslServerCallbackHandler: Successfully > authenticated client: [email protected]; > [email protected]. > 12/06/09 16:40:47 INFO auth.SaslServerCallbackHandler: Setting authorizedID: > saspad > 12/06/09 16:40:47 INFO server.ZooKeeperServer: adding SASL authorization for > authorizationID: saspad > 12/06/09 16:40:50 DEBUG server.FinalRequestProcessor: Processing request:: > sessionid:0x137d2f4f3350005 type:ping cxid:0xfffffffffffffffe > zxid:0xffffffffffff > > It looks like my client identity, "saspad", flowed across the wire > successfully. > > > Thanks again for taking a look at this. > > > > > > > -----Original Message----- > From: Harsh J [mailto:[email protected]] > Sent: Saturday, June 09, 2012 11:26 AM > To: [email protected] > Cc: Tony Dean > Subject: Re: hbase client security (cluster is secure) > > Hi again Tony, > > Moving this to [email protected] (bcc'd [email protected]). > Please use the right user group lists for best responses. I've added you to > CC in case you aren't subscribed to the HBase user lists. > > Can you share the whole error/stacktrace-if-any/logs you get at the HMaster > that says AccessControlException? Would be helpful to see what particular > class/operation logged it to help you specifically. > > I have an instance of 0.92-based cluster running after having followed > http://hbase.apache.org/book.html#zookeeper and > https://ccp.cloudera.com/display/CDH4DOC/HBase+Security+Configuration > and it seems to work well enough with auth enabled. > > On Sat, Jun 9, 2012 at 3:41 AM, Tony Dean <[email protected]> wrote: >> Hi all, >> >> I have created a hadoop/hbase/zookeeper cluster that is secured and >> verified. Now a simple test is to connect an hbase client (e.g, shell) to >> see its behavior. >> >> Well, I get the following message on the hbase master: >> AccessControlException: authentication is required. >> >> Looking at the code it appears that the client passed "simple" >> authentication byte in the rpc header. Why, I don't know? >> >> My client configuration is as follows: >> >> hbase-site.xml: >> <property> >> <name>hbase.security.authentication</name> >> <value>kerberos</value> >> </property> >> >> <property> >> <name>hbase.rpc.engine</name> >> <value>org.apache.hadoop.hbase.ipc.SecureRpcEngine</value> >> </property> >> >> hbase-env.sh: >> export HBASE_OPTS="$HBASE_OPTS >> -Djava.security.auth.login.config=/usr/local/hadoop/hbase/conf/hbase.jaas" >> >> hbase.jaas: >> Client { >> com.sun.security.auth.module.Krb5LoginModule required >> useKeyTab=false >> useTicketCache=true >> }; >> >> I issue kinit for the client I want to use. Then invoke hbase shell. I >> simply issue list and see the error on the server. >> >> Any ideas what I am doing wrong? >> >> Thanks so much! >> >> >> _____________________________________________ >> From: Tony Dean >> Sent: Tuesday, June 05, 2012 5:41 PM >> To: [email protected] >> Subject: hadoop file permission 1.0.3 (security) >> >> >> Can someone detail the options that are available to set file permissions at >> the hadoop and os level? Here's what I have discovered thus far: >> >> dfs.permissions = true|false (works as advertised) dfs.supergroup = >> supergroup (works as advertised) dfs.umaskmode = umask (I believe this >> should be used in lieu of dfs.umask) - it appears to set the permissions for >> files created in hadoop fs (minus execute permission). >> why was dffs.umask deprecated? what's difference between the 2. >> dfs.datanode.data.dir.perm = perm (not sure this is working at all?) I >> thought it was supposed to set permission on blks at the os level. >> >> Are there any other file permission configuration properties? >> >> What I would really like to do is set data blk file permissions at the os >> level so that the blocks can be locked down from all users except super and >> supergroup, but allow it to be used accessed by hadoop API as specified by >> hdfs permissions. Is this possible? >> >> Thanks. >> >> >> Tony Dean >> SAS Institute Inc. >> Senior Software Developer >> 919-531-6704 >> >> << OLE Object: Picture (Device Independent Bitmap) >> >> >> >> > > > > -- > Harsh J > > -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
