[
https://issues.apache.org/jira/browse/HBASE-14581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-14581:
---------------------------
Attachment: 14581-v1.txt
Patch which has been tested on secure hbase cluster.
{code}
2015-10-08 15:46:56,419 INFO [main-SendThread(39358-h-3.novalocal:2181)]
zookeeper.Login: successfully logged in.
2015-10-08 15:46:56,438 INFO [Thread-3] zookeeper.Login: TGT refresh thread
started.
2015-10-08 15:46:56,446 INFO [main-SendThread(39358-h-3.novalocal:2181)]
client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism.
2015-10-08 15:46:56,462 INFO [main-SendThread(39358-h-3.novalocal:2181)]
zookeeper.ClientCnxn: Opening socket connection to server
39358-h-3.novalocal/172.22.111.105:2181. Will attempt to SASL-authenticate
using Login Context section 'Client'
2015-10-08 15:46:56,469 INFO [Thread-3] zookeeper.Login: TGT valid starting
at: Thu Oct 08 15:46:56 UTC 2015
2015-10-08 15:46:56,469 INFO [Thread-3] zookeeper.Login: TGT expires:
Fri Oct 09 15:46:56 UTC 2015
2015-10-08 15:46:56,470 INFO [Thread-3] zookeeper.Login: TGT refresh sleeping
until: Fri Oct 09 11:22:22 UTC 2015
2015-10-08 15:46:56,470 INFO [main-SendThread(39358-h-3.novalocal:2181)]
zookeeper.ClientCnxn: Socket connection established to
39358-h-3.novalocal/172.22.111.105:2181, initiating session
2015-10-08 15:46:56,487 INFO [main-SendThread(39358-h-3.novalocal:2181)]
zookeeper.ClientCnxn: Session establishment complete on server
39358-h-3.novalocal/172.22.111.105:2181, sessionid = 0x35043be3beb0016,
negotiated timeout = 30000
{code}
> Znode cleanup throws auth exception in secure mode
> --------------------------------------------------
>
> Key: HBASE-14581
> URL: https://issues.apache.org/jira/browse/HBASE-14581
> Project: HBase
> Issue Type: Bug
> Reporter: Ted Yu
> Assignee: Ted Yu
> Attachments: 14581-v1.txt
>
>
> When the master process or region server process dies on Linux, we invoke:
> {code}
> if [ -f ${HBASE_ZNODE_FILE} ]; then
> if [ "$command" = "master" ]; then
> $bin/hbase master clear > /dev/null 2>&1
> else
> #call ZK to delete the node
> ZNODE=`cat ${HBASE_ZNODE_FILE}`
> $bin/hbase zkcli delete ${ZNODE} > /dev/null 2>&1
> fi
> rm ${HBASE_ZNODE_FILE}
> fi
> {code}
> We delete its znode from the process which started the server JVM for faster
> crash recovery.
> In secure deployment however, the second process does not authenticate to
> zookeeper properly and fails to delete the znode:
> {code}
> 2015-06-11 11:05:06,238 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)]
> client.ZooKeeperSaslClient: Could not login: the client is being asked for a
> password, but the Zookeeper client code does not currently support obtaining
> a password from the user. Make sure that the client is configured to use a
> ticket cache (using the JAAS config
> 2015-06-11 11:05:06,248 WARN
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> SASL configuration failed: javax.security.auth.login.LoginException: No
> password provided Will continue connection to Zookeeper server without SASL
> authentication, if Zookeeper server allows it.
> 2015-06-11 11:05:06,251 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Opening socket connection to server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181
> 2015-06-11 11:05:06,263 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Socket connection established to
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, initiating session
> 2015-06-11 11:05:06,294 INFO
> [main-SendThread(ip-172-31-32-230.ec2.internal:2181)] zookeeper.ClientCnxn:
> Session establishment complete on server
> ip-172-31-32-230.ec2.internal/172.31.32.230:2181, sessionid =
> 0x14de1dd0f3200cf, negotiated timeout = 40000
> 2015-06-11 11:05:06,664 WARN [main] util.HeapMemorySizeUtil:
> hbase.regionserver.global.memstore.upperLimit is deprecated by
> hbase.regionserver.global.memstore.size
> 2015-06-11 11:05:09,070 WARN [main] zookeeper.ZooKeeperNodeTracker: Can't
> get or delete the master znode
> org.apache.zookeeper.KeeperException$NoAuthException: KeeperErrorCode =
> NoAuth for /hbase-secure/master
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:113)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
> at
> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(RecoverableZooKeeper.java:179)
> at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1345)
> at
> org.apache.hadoop.hbase.zookeeper.MasterAddressTracker.deleteIfEquals(MasterAddressTracker.java:270)
> at org.apache.hadoop.hbase.ZNodeClearer.clear(ZNodeClearer.java:149)
> {code}
> This is due to REGIONSERVER_OPTS / HBASE_MASTER_OPTS not being passed for
> invoking the zkcli command.
> Thanks to Enis who spotted the issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
