Re: Authenticate from SQL
Hi In one old project where usernames and passwords are in RDB, we need authenticate users from RDB before they can go via REST to HBase. So the first thing was Knox. Best regards, Margus Roo skype: margusja phone: +372 51 48 780 web: http://margus.roo.ee On 04/11/14 04:02, Bharath Vissapragada wrote: Hi, What do you mean by auth in SQL? It supports SPNEGO incase you are interested. On Mon, Nov 3, 2014 at 12:16 PM, Margusja mar...@roo.ee wrote: Hi I am looking solutions where users before using HBase rest will be authenticate from SQL (in example from Oracle). Is there any best practices or ready solutions for HBase? -- Best regards, Margus Roo skype: margusja phone: +372 51 48 780 web: http://margus.roo.ee
Authenticate from SQL
Hi I am looking solutions where users before using HBase rest will be authenticate from SQL (in example from Oracle). Is there any best practices or ready solutions for HBase? -- Best regards, Margus Roo skype: margusja phone: +372 51 48 780 web: http://margus.roo.ee
Re: ZooKeeper available but no active master location found
Yes there is: groupIdorg.apache.hbase/groupId artifactIdhbase/artifactId version0.92.1/version Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314) On 10/04/14 00:57, Ted Yu wrote: Have you modified pom.xml of twitbase ? If not, this is the dependency you get: dependency groupIdorg.apache.hbase/groupId artifactIdhbase/artifactId version0.92.1/version 0.92.1 and 0.96.0 are not compatible. Cheers On Wed, Apr 9, 2014 at 10:58 AM, Margusja mar...@roo.ee wrote: Hi I downloaded and installed hortonworks sandbox 2.0 for virtualbox. HBase version is: 0.96.0.2.0.6.0-76-hadoop2, re6d7a56f72914d01e55c0478d74e5 cfd3778f231 [hbase@sandbox twitbase-master]$ cat /etc/hosts # Do not remove the following line, or various programs # that require network functionality will fail. 127.0.0.1 localhost.localdomain localhost 10.0.2.15 sandbox.hortonworks.com sandbox [hbase@sandbox twitbase-master]$ hostname sandbox.hortonworks.com [root@sandbox ~]# netstat -lnp | grep 2181 tcp0 0 0.0.0.0:2181 0.0.0.0:* LISTEN 19359/java [root@sandbox ~]# netstat -lnp | grep 6 tcp0 0 10.0.2.15:6 0.0.0.0:* LISTEN 28549/java [hbase@sandbox twitbase-master]$ hbase shell 14/04/05 05:56:44 INFO Configuration.deprecation: hadoop.native.lib is deprecated. Instead, use io.native.lib.available HBase Shell; enter 'helpRETURN' for list of supported commands. Type exitRETURN to leave the HBase Shell Version 0.96.0.2.0.6.0-76-hadoop2, re6d7a56f72914d01e55c0478d74e5cfd3778f231, Thu Oct 17 18:15:20 PDT 2013 hbase(main):001:0 list TABLE SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/usr/lib/zookeeper/ lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/usr/lib/hadoop/lib/ slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. ambarismoketest mytable simple_hcat_load_table users weblogs 5 row(s) in 4.6040 seconds = [ambarismoketest, mytable, simple_hcat_load_table, users, weblogs] hbase(main):002:0 So far is good. I'd like to play with a code: https://github.com/hbaseinaction/twitbase downloaded and made package: mvn package and got twitbase-1.0.0.jar. When I try to exec code I will get: 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:host.name= sandbox.hortonworks.com 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.version=1.6.0_30 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun Microsystems Inc. 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.class.path=target/twitbase-1.0.0.jar 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.library.path=/usr/lib/jvm/java-1.6.0- openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:/usr/lib/jvm/ java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:/usr/lib/ jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64:/ usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/tmp 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:java.compiler=NA 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:os.name =Linux 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:os.arch=amd64 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:os.version=2.6.32-431.11.2.el6.x86_64 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:user.name =hbase 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:user.home=/home/hbase 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Client environment:user.dir=/home/hbase/twitbase-master 14/04/05 05:59:50 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=10.0.2.15:2181 sessionTimeout=18 watcher=hconnection 14/04/05 05:59:50 INFO zookeeper.ClientCnxn: Opening socket connection to server /10.0.2.15:2181 14/04/05 05:59:50 INFO zookeeper.RecoverableZooKeeper: The identifier of this process is 30...@sandbox.hortonworks.com 14/04/05 05:59:50 INFO client.ZooKeeperSaslClient: Client will not SASL-authenticate because the default JAAS configuration section 'Client' could not be found. If you are not using SASL, you may ignore this. On the other hand, if you expected SASL to work, please fix your JAAS configuration. 14/04/05 05:59:51 INFO zookeeper.ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session 14/04/05
This server is in the failed servers list
:2181 14/04/05 11:03:03 INFO zookeeper.ClientCnxn: Opening socket connection to server sandbox.hortonworks.com/10.0.2.15:2181. Will not attempt to authenticate using SASL (unknown error) 14/04/05 11:03:03 INFO zookeeper.ClientCnxn: Socket connection established to sandbox.hortonworks.com/10.0.2.15:2181, initiating session 14/04/05 11:03:03 INFO zookeeper.ClientCnxn: Session establishment complete on server sandbox.hortonworks.com/10.0.2.15:2181, sessionid = 0x1453145e9500056, negotiated timeout = 4 14/04/05 11:03:04 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 1 of 35 failed; retrying after sleep of 100, exception=com.google.protobuf.ServiceException: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: org.apache.hadoop.net.NetUtils.getInputStream(Ljava/net/Socket;)Lorg/apache/hadoop/net/SocketInputWrapper; 14/04/05 11:03:04 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 2 of 35 failed; retrying after sleep of 201, exception=com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.RpcClient$FailedServerException: This server is in the failed servers list: sandbox.hortonworks.com/10.0.2.15:6 14/04/05 11:03:04 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 3 of 35 failed; retrying after sleep of 300, exception=com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.RpcClient$FailedServerException: This server is in the failed servers list: sandbox.hortonworks.com/10.0.2.15:6 14/04/05 11:03:05 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 4 of 35 failed; retrying after sleep of 500, exception=com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.RpcClient$FailedServerException: This server is in the failed servers list: sandbox.hortonworks.com/10.0.2.15:6 14/04/05 11:03:05 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 5 of 35 failed; retrying after sleep of 1001, exception=com.google.protobuf.ServiceException: org.apache.hadoop.hbase.ipc.RpcClient$FailedServerException: This server is in the failed servers list: sandbox.hortonworks.com/10.0.2.15:6 14/04/05 11:03:06 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 6 of 35 failed; retrying after sleep of 2014, exception=com.google.protobuf.ServiceException: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: org.apache.hadoop.net.NetUtils.getInputStream(Ljava/net/Socket;)Lorg/apache/hadoop/net/SocketInputWrapper; 14/04/05 11:03:08 INFO client.HConnectionManager$HConnectionImplementation: getMaster attempt 7 of 35 failed; retrying after sleep of 4027, exception=com.google.protobuf.ServiceException: org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.NoSuchMethodError: org.apache.hadoop.net.NetUtils.getInputStream(Ljava/net/Socket;)Lorg/apache/hadoop/net/SocketInputWrapper; [hbase@sandbox hbase_connect]$ jps 4355 HMaster 5335 Jps 4711 HRegionServer 4715 ThriftServer 4717 RESTServer tcp 0 0 0.0.0.0:2181 0.0.0.0:* LISTEN tcp 0 0 10.0.2.15:6 0.0.0.0:* LISTEN 4355/java [root@sandbox ~]# cat /etc/hosts 127.0.0.1 localhost.localdomain localhost 10.0.2.15 sandbox.hortonworks.com sandbox Any hints? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)
Re: This server is in the failed servers list
/10.0.2.15:2181. Will not attempt to
authenticate using SASL (unknown error)
2014-04-05 12:09:34,663 INFO
[main-SendThread(sandbox.hortonworks.com:2181)] zookeeper.ClientCnxn
(ClientCnxn.java:primeConnection(849)) - Socket connection established
to sandbox.hortonworks.com/10.0.2.15:2181, initiating session
2014-04-05 12:09:34,741 INFO
[main-SendThread(sandbox.hortonworks.com:2181)] zookeeper.ClientCnxn
(ClientCnxn.java:onConnected(1207)) - Session establishment complete on
server sandbox.hortonworks.com/10.0.2.15:2181, sessionid =
0x1453145e9500058, negotiated timeout = 4
2014-04-05 12:09:36,539 INFO [main] Configuration.deprecation
(Configuration.java:warnOnceIfDeprecated(840)) - hadoop.native.lib is
deprecated. Instead, use io.native.lib.available
Table = ambarismoketest
Table = mytable
Table = simple_hcat_load_table
Table = users
Table = weblogs
Maybe my mistake helps somebody :)
Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)
On 10/04/14 11:12, Margusja wrote:
Hi
I have java code:
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hbase.HBaseConfiguration;
import org.apache.hadoop.hbase.HColumnDescriptor;
import org.apache.hadoop.hbase.HTableDescriptor;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.util.Bytes;
public class Hbase_connect {
public static void main(String[] args) throws Exception {
Configuration conf = HBaseConfiguration.create();
conf.set(hbase.zookeeper.quorum,
sandbox.hortonworks.com);
conf.set(hbase.zookeeper.property.clientPort, 2181);
conf.set(hbase.rootdir,
hdfs://sandbox.hortonworks.com:8020/apps/hbase/data);
conf.set(zookeeper.znode.parent, /hbase-unsecure);
HBaseAdmin admin = new HBaseAdmin(conf);
HTableDescriptor[] tabdesc = admin.listTables();
for(int i=0; itabdesc.length; i++) {
System.out.println(Table = + new
String(tabdesc [i].getName()));
}
}
}
^C[hbase@sandbox hbase_connect]$ ls -lah libs/
total 80M
drwxr-xr-x 3 hbase hadoop 4.0K Apr 5 10:42 .
drwxr-xr-x 3 hbase hadoop 4.0K Apr 5 11:02 ..
-rw-r--r-- 1 hbase hadoop 2.5K Oct 6 23:39 hadoop-client-2.2.0.jar
-rw-r--r-- 1 hbase hadoop 4.1M Jul 24 2013 hadoop-core-1.2.1.jar
drwxr-xr-x 4 hbase hadoop 4.0K Apr 5 09:40 hbase-0.96.2-hadoop2
-rw-r--r-- 1 hbase hadoop 76M Apr 3 16:18
hbase-0.96.2-hadoop2-bin.tar.gz
[hbase@sandbox hbase_connect]$ java -cp
./:./libs/*:./libs/hbase-0.96.2-hadoop2/lib/* Hbase_connect
14/04/05 11:03:03 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
GMT
14/04/05 11:03:03 INFO zookeeper.ZooKeeper: Client
environment:host.name=sandbox.hortonworks.com
14/04/05 11:03:03 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_30
14/04/05 11:03:03 INFO zookeeper.ZooKeeper: Client
environment:java.vendor=Sun Microsystems Inc.
14/04/05 11:03:03 INFO zookeeper.ZooKeeper: Client
environment:java.home=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre
14/04/05 11:03:03 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=./:./libs/hadoop-client-2.2.0.jar:./libs/hadoop-core-1.2.1.jar:./libs/hbase-0.96.2-hadoop2/lib/management-api-3.0.0-b012.jar:./libs/hbase-0.96.2-hadoop2/lib/jackson-core-asl-1.8.8.jar:./libs/hbase-0.96.2-hadoop2/lib/slf4j-log4j12-1.6.4.jar:./libs/hbase-0.96.2-hadoop2/lib/hbase-server-0.96.2-hadoop2.jar:./libs/hbase-0.96.2-hadoop2/lib/jsp-2.1-6.1.14.jar:./libs/hbase-0.96.2-hadoop2/lib/log4j-1.2.17.jar:./libs/hbase-0.96.2-hadoop2/lib/hadoop-mapreduce-client-core-2.2.0.jar:./libs/hbase-0.96.2-hadoop2/lib/commons-codec-1.7.jar:./libs/hbase-0.96.2-hadoop2/lib/hadoop-mapreduce-client-jobclient-2.2.0.jar:./libs/hbase-0.96.2-hadoop2/lib/hbase-common-0.96.2-hadoop2.jar:./libs/hbase-0.96.2-hadoop2/lib/jersey-server-1.8.jar:./libs/hbase-0.96.2-hadoop2/lib/hbase-it-0.96.2-hadoop2.jar:./libs/hbase-0.96.2-hadoop2/lib/commons-el-1.0.jar:./libs/hbase-0.96.2-hadoop2/lib/commons-collections-3.2.1.jar:./libs/hbase-0.96.2-hadoop2/lib/hadoop-mapreduce-client-common-2.2.0.jar:./libs/hbase-0.96.2-hadoop2/lib/jersey-grizzly2-1.9.jar:./libs/hbase-0.96.2-hadoop2/lib/protobuf-java-2.5.0.jar:./libs/hbase-0.96.2-hadoop2/lib/hadoop-client-2.2.0.jar:./libs/hbase-0.96.2-hadoop2/lib/hadoop-common-2.2.0.jar:./libs/hbase-0.96.2-hadoop2/lib/hadoop-yarn-api-2.2.0.jar:./libs/hbase-0.96.2-hadoop2/lib/hadoop-mapreduce-client-app-2.2.0.jar:./libs/hbase-0.96.2-hadoop2/lib/hamcrest-core-1.3.jar:./libs/hbase-0.96.2-hadoop2/lib/commons-beanutils-core-1.8.0.jar:./libs/hbase-0.96.2-hadoop2/lib/hbase-client-0.96.2-hadoop2.jar:./libs/hbase-0.96.2-hadoop2/lib/slf4j-api-1.6.4.jar:./libs/hbase-0.96.2-hadoop2/lib/commons-compress-1.4.1.jar:./libs/hbase-0.96.2-hadoop2/lib
ZooKeeper available but no active master location found
(HConnectionManager.java:634) at org.apache.hadoop.hbase.client.HBaseAdmin.init(HBaseAdmin.java:106) at HBaseIA.TwitBase.InitTables.main(InitTables.java:20) InitTables.java https://github.com/hbaseinaction/twitbase/blob/master/src/main/java/HBaseIA/TwitBase/InitTables.java The line that drops error is: HBaseAdminadmin=newHBaseAdmin(conf); Log line: ZooKeeper available but no active master location found is from org/apache/hadoop/hbase/client/HConnectionManager.java [hbase@sandbox twitbase-master]$ jps 30473 Jps 28549 HMaster 28896 RESTServer 28916 HRegionServer 28902 ThriftServer [hbase@sandbox twitbase-master]$ As much I read ther might be somthing wrong with binding and DNS resolving. Any hints? -- Best regards, Margus (Margusja) Roo +372 51 48 780 http://margus.roo.ee http://ee.linkedin.com/in/margusroo skype: margusja ldapsearch -x -h ldap.sk.ee -b c=EE (serialNumber=37303140314)
