JDBC Connection to Apache Phoenix failing
Can someone help with this? TIA
I am trying to connect to Apache Phoenix from the spark-shell, and the
following stack trace is getting thown.
val con =
DriverManager.getConnection("jdbc:phoenix:ec2-xx.xx.xx.xx.ap-southeast-1.compute.amazonaws.com:2181
:/hbase-unsecure")
2018-12-04 17:32:16 WARN ConnectionImplementation:529 - Retrieve cluster
id failed
java.util.concurrent.ExecutionException:
org.apache.phoenix.shaded.org.apache.zookeeper.KeeperException$NoNodeException:
KeeperErrorCode = NoNode for /hbase-unsecure/hbaseid
at
java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1895)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.retrieveClusterId(ConnectionImplementation.java:527)
at
org.apache.hadoop.hbase.client.ConnectionImplementation.(ConnectionImplementation.java:287)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:219)
at
org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:114)
at
org.apache.phoenix.query.HConnectionFactory$HConnectionFactoryImpl.createConnection(HConnectionFactory.java:47)
at
org.apache.phoenix.query.ConnectionQueryServicesImpl.openConnection(ConnectionQueryServicesImpl.java:430)
at
org.apache.phoenix.query.ConnectionQueryServicesImpl.access$400(ConnectionQueryServicesImpl.java:272)
at
org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2556)
at
org.apache.phoenix.query.ConnectionQueryServicesImpl$12.call(ConnectionQueryServicesImpl.java:2532)
at
org.apache.phoenix.util.PhoenixContextExecutor.call(PhoenixContextExecutor.java:76)
at
org.apache.phoenix.query.ConnectionQueryServicesImpl.init(ConnectionQueryServicesImpl.java:2532)
at
org.apache.phoenix.jdbc.PhoenixDriver.getConnectionQueryServices(PhoenixDriver.java:255)
at
org.apache.phoenix.jdbc.PhoenixEmbeddedDriver.createConnection(PhoenixEmbeddedDriver.java:150)
at org.apache.phoenix.jdbc.PhoenixDriver.connect(PhoenixDriver.java:221)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:270)
at $line17.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:24)
at $line17.$read$$iw$$iw$$iw$$iw$$iw$$iw$$iw.(:29)
at $line17.$read$$iw$$iw$$iw$$iw$$iw$$iw.(:31)
at $line17.$read$$iw$$iw$$iw$$iw$$iw.(:33)
at $line17.$read$$iw$$iw$$iw$$iw.(:35)
at $line17.$read$$iw$$iw$$iw.(:37)
at $line17.$read$$iw$$iw.(:39)
at $line17.$read$$iw.(:41)
at $line17.$read.(:43)
at $line17.$read$.(:47)
at $line17.$read$.()
at $line17.$eval$.$print$lzycompute(:7)
at $line17.$eval$.$print(:6)
at $line17.$eval.$print()
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:786)
at
scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1047)
at
scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:638)
at
scala.tools.nsc.interpreter.IMain$WrappedRequest$$anonfun$loadAndRunReq$1.apply(IMain.scala:637)
at
scala.reflect.internal.util.ScalaClassLoader$class.asContext(ScalaClassLoader.scala:31)
at
scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
at
scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:637)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:569)
at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:565)
at
scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:807)
at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:681)
at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:395)
at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:415)
at
scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply$mcZ$sp(ILoop.scala:923)
at
scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
at
scala.tools.nsc.interpreter.ILoop$$anonfun$process$1.apply(ILoop.scala:909)
at
scala.reflect.internal.util.ScalaClassLoader$.savingContextLoader(ScalaClassLoader.scala:97)
at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:909)
at
Re: JDBC Connection URL to Phoenix on Azure HDInsight
Thanks Josh for the quick response.
Did some digging, the Zookeeper quorum needs to be used for the thick
client like Josh suggested. JDBL URL was fine and found there were two
issues with Azure HDI setup:
1. Zookeeper Server was not started
2. Zookeeper port(2181) is not open
*1.* Zookeeper Server is not automatically started if you are using Azure
HDI, it has to be started by running the below command
`./usr/hdp/2.xxcx/zookeeper/bin/zkServer.sh start`
The above command starts the Zookeeper server in standalone-mode. If you
need to start the Zookeeper in cluster-mode, you need to add the Zookeeper
Server FQDNs(server.{1..n}) to the ./usr/hdp/2.xxcx/zookeeper/conf/zoo.cfg
file if there are not present already.
maxClientCnxns=50
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/var/lib/zookeeper
clientPort=2181
server.1=zk0-apache.x.tx.internal.cloudapp.net:2888:3888
server.2=zk1-apache.x.tx.internal.cloudapp.net:2888:3888
server.3=zk2-apache.x.tx.internal.cloudapp.net:2888:3888
You need to stop and start the zkServer after the configuration changes. More
info on Zookeeper documentation
<https://zookeeper.apache.org/doc/r3.3.3/zookeeperAdmin.html#sc_zkMulitServerSetup>
.
*2.* Azure HDI Cluster, by default, is created inside a network which
doesn't expose your cluster ports(eg. 2181, 8765) to the public. You need
to have a Virtual Network created as explained in this artcile
<https://docs.microsoft.com/en-us/azure/hdinsight/hdinsight-extend-hadoop-virtual-network>
and create the HDI cluster inside this network. Through the network, you
will be able to open the desired ports.
On Wed, Nov 28, 2018, 2:54 AM Josh Elser Are you trying to use the thick driver (direct HBase connection) or the
> thin driver (via Phoenix Query Server)? You're providing examples of both.
>
> If the thick driver: by default, I believe that HDI configures a root
> znode of "/hbase" not "/hbase-unsecure" which would be a problem. You
> need to pass your ZK quorum which makes your 2nd HDI "example"
> nonsensical. You can read the Phoenix home page for information about
> how to construct a Phoenix JDBC URL for the thick driver at
> https://phoenix.apache.org/#connStr
>
> If you are still having troubles, I'd suggest that you clarify exactly
> what you want to do (rather than everything you tried) and include the
> error you ran into.
>
> On 11/27/18 7:54 AM, Raghavendra Channarayappa wrote:
> > Dear all,
> >
> > My current problem is fairly trivial, i guess, but I am currently stuck
> > on this.
> >
> > The following was the JDBC connection string for Phoenix on EMR
> >
> >
> jdbc:phoenix:thin:url=http://ec2-12-345-678-90.ap-southeast-1.compute.amazonaws.com:8765;serialization
> =PROTOBUF;autocommit=true
> >
> > which worked perfectly fine.
> >
> > But, for Phoenix on Azure HDInsight(which uses HDP2.6.0), I am unable to
> > figure out the equivalent JDBC url
> >
> > The following options which *DID NOT* work:
> > *Azure HDInsight Ambari Dashboard URL as the server_url:*
> > jdbc:phoenix:thin:url=http://randomcluterdashboard.
> azurehdinsight.net:8765;serialization=PROTOBUF;autocommit=true
> > *Phoenix Query Servers(comma-separated) as the server_url:*
> > jdbc:phoenix:
> wn0-apache.x.tx.internal.cloudapp.net
> > <http://wn0-apache.x.tx.internal.cloudapp.net>,
> wn1-apache.x.tx.internal.cloudapp.net
> > <http://wn1-apache.x.tx.internal.cloudapp.net>,
> wn2-apache.x.tx.internal.cloudapp.net
> > <http://wn2-apache.x.tx.internal.cloudapp.net
> >,wn3-apache.x.tx.internal.cloudapp.net:2181
> :/hbase-unsecure
> > *Zookeeper Quorum(comma-separated) as the server_url:*
> > jdbc:phoenix:
> zk0-apache.x.tx.internal.cloudapp.net
> > <http://zk0-apache.x.tx.internal.cloudapp.net>,
> zk1-apache.x.tx.internal.cloudapp.net
> > <http://zk1-apache.x.tx.internal.cloudapp.net
> >,zk2-apache.x.tx.internal.cloudapp.net:2181
> :/hbase-unsecure
> >
> > Where can I find the JDBC url in Ambari dashboard?
> >
> > Thanks in advance,
> >
> > Raghavendra
> >
> >
> >
> >
>
Re: JDBC Connection URL to Phoenix on Azure HDInsight
+Samya Ghosh On Tue, Nov 27, 2018 at 6:24 PM Raghavendra Channarayappa < raghavendra.channaraya...@myntra.com> wrote: > Dear all, > > My current problem is fairly trivial, i guess, but I am currently stuck on > this. > > The following was the JDBC connection string for Phoenix on EMR > > jdbc:phoenix:thin:url=http://ec2-12-345-678-90.ap-southeast-1.compute.amazonaws.com:8765;serialization=PROTOBUF;autocommit=true > > which worked perfectly fine. > > But, for Phoenix on Azure HDInsight(which uses HDP2.6.0), I am unable to > figure out the equivalent JDBC url > > The following options which *DID NOT* work: > *Azure HDInsight Ambari Dashboard URL as the server_url:* > jdbc:phoenix:thin:url=http://randomcluterdashboard.azurehdinsight.net:8765;serialization=PROTOBUF;autocommit=true > *Phoenix Query Servers(comma-separated) as the server_url:* > jdbc:phoenix:wn0-apache.x.tx.internal.cloudapp.net,wn1-apache.x.tx.internal.cloudapp.net,wn2-apache.x.tx.internal.cloudapp.net,wn3-apache.x.tx.internal.cloudapp.net:2181:/hbase-unsecure > *Zookeeper Quorum(comma-separated) as the server_url:* > jdbc:phoenix:zk0-apache.x.tx.internal.cloudapp.net,zk1-apache.x.tx.internal.cloudapp.net,zk2-apache.x.tx.internal.cloudapp.net:2181:/hbase-unsecure > > Where can I find the JDBC url in Ambari dashboard? > > Thanks in advance, > > Raghavendra > > > > >
