'hbase classpath' is a good command to know, thank you.
Mark On Mon, Jan 16, 2012 at 11:44 AM, Peter Wolf <[email protected]> wrote: > Many, many thanks J-D > > I grepped 'hbase classpath' and found this... > > /usr/lib/hbase/bin/../conf > > /usr/lib/hbase/conf contains a hbase-site.xml which is identical to the > one in /etc/hbase/conf. In fact /usr/lib/hbase/conf is just a link to > /etc/hbase/conf. > > So, I think it should have worked. Furthermore, it seems to work on my > Mac OSX (Lion). > > Here is a new pastbin, with a log of 'hbase shell -d' > > http://pastebin.com/78zJkdti > > Looking at commands.rb, it seems to be just allocating an HBaseAdmin. Is > there some additional logging I can switch on to help isolate this? > > P > > > > > > On 1/13/12 9:12 PM, Jean-Daniel Cryans wrote: > >> Apache HBase doesn't require that, as you can tell by the >> documentation: >> http://hbase.apache.org/book.**html<http://hbase.apache.org/book.html> >> >> You are running CDH3u2, which has its own quirks/features, so as to >> why it's not picking it up while it should is a question for the >> cloudera mailing lists. >> >> J-D >> >> On Fri, Jan 13, 2012 at 5:24 PM, Peter Wolf<[email protected]> wrote: >> >>> My apologies, >>> >>> Here is what "hbase classpath" returns >>> >>> http://pastebin.com/2MP9c6Yq >>> >>> As you can see, /etc/hbase/conf is not on there, so that explains my >>> problem. The documentation indicates that hbase-site.xml is normally >>> located in /etc/hbase/conf. Is this a problem with 'hbase', or did I >>> mis-understand? >>> >>> Thanks >>> P >>> >>> >>> >>> >>> On 1/13/12 12:43 PM, Jean-Daniel Cryans wrote: >>> >>>> Sorry what I meant by "pastebin all the debug" was to use a service >>>> like pastebin.com to keep the emails short. >>>> >>>> So in there I see: >>>> >>>> 12/01/13 02:21:22 INFO zookeeper.ClientCnxn: Opening socket connection >>>>> to >>>>> server localhost/127.0.0.1:2181 >>>>> >>>> Which means that it's connecting to the default value of >>>> hbase.zookeeper.quorum which is localhost. I don't know how you set >>>> HBase on your remote machine but it's not picking up your >>>> hbase-site.xml. It should normally be in the shell's classpath if you >>>> start it with "bin/hbase shell", but you can easily verify that by >>>> doing "bin/hbase classpath" and you should see which conf directory is >>>> being picked up. >>>> >>>> J-D >>>> >>>> On Thu, Jan 12, 2012 at 6:24 PM, Peter Wolf<[email protected]> >>>> wrote: >>>> >>>>> Here you go, and thanks! >>>>> >>>>> P >>>>> >>>>> >>>>> at sun.nio.ch.SocketChannelImpl.**checkConnect(Native Method) >>>>> at >>>>> sun.nio.ch.SocketChannelImpl.**finishConnect(** >>>>> SocketChannelImpl.java:592) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.run(** >>>>> ClientCnxn.java:1119) >>>>> 12/01/13 02:21:20 DEBUG zookeeper.ClientCnxn: Ignoring exception during >>>>> shutdown input >>>>> java.nio.channels.**ClosedChannelException >>>>> at >>>>> sun.nio.ch.SocketChannelImpl.**shutdownInput(** >>>>> SocketChannelImpl.java:656) >>>>> at sun.nio.ch.SocketAdaptor.**shutdownInput(SocketAdaptor.** >>>>> java:378) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.cleanup(** >>>>> ClientCnxn.java:1205) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.run(** >>>>> ClientCnxn.java:1170) >>>>> 12/01/13 02:21:20 DEBUG zookeeper.ClientCnxn: Ignoring exception during >>>>> shutdown output >>>>> java.nio.channels.**ClosedChannelException >>>>> at >>>>> sun.nio.ch.SocketChannelImpl.**shutdownOutput(** >>>>> SocketChannelImpl.java:667) >>>>> at sun.nio.ch.SocketAdaptor.**shutdownOutput(SocketAdaptor.** >>>>> java:386) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.cleanup(** >>>>> ClientCnxn.java:1212) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.run(** >>>>> ClientCnxn.java:1170) >>>>> 12/01/13 02:21:20 DEBUG zookeeper.ZooKeeperWatcher: Retrying zk create >>>>> for >>>>> another 8565ms; set 'hbase.zookeeper.recoverable.**waittime' to >>>>> change wait >>>>> time); KeeperErrorCode = ConnectionLoss for /hbase >>>>> 12/01/13 02:21:22 INFO zookeeper.ClientCnxn: Opening socket connection >>>>> to >>>>> server localhost/127.0.0.1:2181 >>>>> 12/01/13 02:21:22 WARN zookeeper.ClientCnxn: Session 0x0 for server >>>>> null, >>>>> unexpected error, closing socket connection and attempting reconnect >>>>> java.net.ConnectException: Connection refused >>>>> at sun.nio.ch.SocketChannelImpl.**checkConnect(Native Method) >>>>> at >>>>> sun.nio.ch.SocketChannelImpl.**finishConnect(** >>>>> SocketChannelImpl.java:592) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.run(** >>>>> ClientCnxn.java:1119) >>>>> 12/01/13 02:21:22 DEBUG zookeeper.ClientCnxn: Ignoring exception during >>>>> shutdown input >>>>> java.nio.channels.**ClosedChannelException >>>>> at >>>>> sun.nio.ch.SocketChannelImpl.**shutdownInput(** >>>>> SocketChannelImpl.java:656) >>>>> at sun.nio.ch.SocketAdaptor.**shutdownInput(SocketAdaptor.** >>>>> java:378) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.cleanup(** >>>>> ClientCnxn.java:1205) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.run(** >>>>> ClientCnxn.java:1170) >>>>> 12/01/13 02:21:22 DEBUG zookeeper.ClientCnxn: Ignoring exception during >>>>> shutdown output >>>>> java.nio.channels.**ClosedChannelException >>>>> at >>>>> sun.nio.ch.SocketChannelImpl.**shutdownOutput(** >>>>> SocketChannelImpl.java:667) >>>>> at sun.nio.ch.SocketAdaptor.**shutdownOutput(SocketAdaptor.** >>>>> java:386) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.cleanup(** >>>>> ClientCnxn.java:1212) >>>>> at >>>>> org.apache.zookeeper.**ClientCnxn$SendThread.run(** >>>>> ClientCnxn.java:1170) >>>>> 12/01/13 02:21:22 DEBUG zookeeper.ZooKeeperWatcher: Retrying zk create >>>>> for >>>>> another 6603ms; set 'hbase.zookeeper.recoverable.**waittime' to >>>>> change wait >>>>> time); KeeperErrorCode = ConnectionLoss for /hbase >>>>> >>>>> >>>>> >>>>> >>>>> On 1/12/12 5:36 PM, Jean-Daniel Cryans wrote: >>>>> >>>>>> Interesting, could you start the shell with "-d" and pastebin all the >>>>>> debug that comes out after the first command? >>>>>> >>>>>> BTW the shell does work on remote clusters, so it's some other issue. >>>>>> >>>>>> J-D >>>>>> >>>>>> On Thu, Jan 12, 2012 at 1:56 PM, Peter Wolf<[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Sorry, that's a typo in my email. Here is my config file again (that >>>>>>> doesn't work) >>>>>>> >>>>>>> >>>>>>> <configuration> >>>>>>> <property> >>>>>>> <name>hbase.zookeeper.quorum</**name> >>>>>>> <value>ip-AA-BBB-C-DDD.ec2.**internal</value> >>>>>>> <description>Standalone Server</description> >>>>>>> </property> >>>>>>> </configuration> >>>>>>> >>>>>>> I double checked, and I am using ip-AA-BBB-C-DDD.ec2.internal >>>>>>> consistently >>>>>>> in config files and code. >>>>>>> >>>>>>> P >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> On 1/12/12 4:24 PM, Jean-Daniel Cryans wrote: >>>>>>> >>>>>>>> Yes, it's the same thing, which is why I think the additional >>>>>>>> ec2.internal in your hbase-site is suspicious. Let me reiterate: >>>>>>>> >>>>>>>> This works: >>>>>>>> >>>>>>>> echo stat|nc ip-XX-YYY-Z-QQQ.ec2.internal 2181 >>>>>>>> >>>>>>>> But this config doesn't: >>>>>>>> >>>>>>>> <value>ip-XX-YYY-Z-QQQ.ec2.**internal.ec2.internal</value> >>>>>>>> >>>>>>>> Now what happens if you just use the same one, the one that works? >>>>>>>> >>>>>>>> J-D >>>>>>>> >>>>>>>> On Thu, Jan 12, 2012 at 12:45 PM, Peter Wolf<[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I'm a N00B, so I'm not sure of anything... but it is working now >>>>>>>>> using >>>>>>>>> the >>>>>>>>> Java Client API, and XXX.ec2.internal address on both server and >>>>>>>>> client. >>>>>>>>> >>>>>>>>> The problem seems to be 'hbase shell', which is odd as I would have >>>>>>>>> thought >>>>>>>>> it sat on top of the Java API. >>>>>>>>> >>>>>>>>> P >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On 1/12/12 1:22 PM, Jean-Daniel Cryans wrote: >>>>>>>>> >>>>>>>>>> Your config file on the remote machine has: >>>>>>>>>> >>>>>>>>>> ip-XX-YYY-Z-QQQ.ec2.internal.**ec2.internal >>>>>>>>>> >>>>>>>>>> You sure about the extra ec2.internal? >>>>>>>>>> >>>>>>>>>> J-D >>>>>>>>>> >>>>>>>>>> On Thu, Jan 12, 2012 at 9:26 AM, Peter Wolf<[email protected]> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Oh yeah! The code did it :-D >>>>>>>>>>> >>>>>>>>>>> For those that come after, I guess 'hbase shell' is broken for >>>>>>>>>>> remote >>>>>>>>>>> access. Use the raw Java API >>>>>>>>>>> >>>>>>>>>>> Many thanks again Mark! >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On 1/12/12 11:40 AM, Mark Kerzner wrote: >>>>>>>>>>> >>>>>>>>>>>> 1. Look in the logs; >>>>>>>>>>>> 2. I think hbase shell works only locally; >>>>>>>>>>>> 3. The code below worked for me, and I don't use a config file, >>>>>>>>>>>> but >>>>>>>>>>>> give >>>>>>>>>>>> the params directly: >>>>>>>>>>>> >>>>>>>>>>>> public void connect() throws IOException { >>>>>>>>>>>> Configuration hConf = HBaseConfiguration.create(); >>>>>>>>>>>> >>>>>>>>>>>> hConf.set(MyConstants.HBASE_**CONFIGURATION_ZOOKEEPER_**QUORUM, >>>>>>>>>>>> MyConstants.zookeeperUrl); >>>>>>>>>>>> >>>>>>>>>>>> hConf.set(MyConstants..HBASE_**CONFIGURATION_ZOOKEEPER_** >>>>>>>>>>>> CLIENTPORT, >>>>>>>>>>>> MyConstants.zookeeperPort); >>>>>>>>>>>> String tableName = "MyTable"; >>>>>>>>>>>> HTable hTable = new HTable(hConf, tableName); >>>>>>>>>>>> hTable.close(); >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> Mark >>>>>>>>>>>> >>>>>>>>>>>> On Thu, Jan 12, 2012 at 10:20 AM, Peter Wolf<[email protected]> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>> Still no love... Any suggestions? >>>>>>>>>>>>> >>>>>>>>>>>>> I'm on EC2, and I am trying to set up a Pseudo-Distributed >>>>>>>>>>>>> HBaser >>>>>>>>>>>>> Server >>>>>>>>>>>>> on one machine, and access it from another. Both machines are >>>>>>>>>>>>> EC2. >>>>>>>>>>>>> >>>>>>>>>>>>> I have already found the doc below, and I followed the >>>>>>>>>>>>> instructions >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> http://hbase.apache.org/book.****html#client_dependencies<http://hbase.apache.org/book.**html#client_dependencies> >>>>>>>>>>>>> <http**://hbase.apache.org/book.html#**client_dependencies<http://hbase.apache.org/book.html#client_dependencies> >>>>>>>>>>>>> > >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> My server is up and running, and I can access it from an 'hbase >>>>>>>>>>>>> shell' >>>>>>>>>>>>> on >>>>>>>>>>>>> the machine. The hbase-site.xml is ... >>>>>>>>>>>>> >>>>>>>>>>>>> <configuration> >>>>>>>>>>>>> <property> >>>>>>>>>>>>> <name>hbase.zookeeper.quorum</****name> >>>>>>>>>>>>> <value>ip-XX-YYY-Z-QQQ.ec2.****internal</value> >>>>>>>>>>>>> </property> >>>>>>>>>>>>> <property> >>>>>>>>>>>>> <name>hbase.cluster.****distributed</name> >>>>>>>>>>>>> <value>true</value> >>>>>>>>>>>>> </property> >>>>>>>>>>>>> <property> >>>>>>>>>>>>> <name>hbase.rootdir</name> >>>>>>>>>>>>> <value>hdfs://localhost/hbase<****/value> >>>>>>>>>>>>> </property> >>>>>>>>>>>>> </configuration> >>>>>>>>>>>>> >>>>>>>>>>>>> The client machine hbase-site.xml is... >>>>>>>>>>>>> >>>>>>>>>>>>> <configuration> >>>>>>>>>>>>> <property> >>>>>>>>>>>>> <name>hbase.zookeeper.quorum</****name> >>>>>>>>>>>>> <value>ip-XX-YYY-Z-QQQ.ec2.****internal.ec2.internal</value> >>>>>>>>>>>>> <description>Pseudo Distributed Server</description> >>>>>>>>>>>>> </property> >>>>>>>>>>>>> </configuration> >>>>>>>>>>>>> >>>>>>>>>>>>> And when I try 'hbase shell' remotely, I get this... >>>>>>>>>>>>> >>>>>>>>>>>>> hbase(main):001:0> status 'detailed' >>>>>>>>>>>>> >>>>>>>>>>>>> ERROR: org.apache.hadoop.hbase.**** >>>>>>>>>>>>> ZooKeeperConnectionException: >>>>>>>>>>>>> HBase >>>>>>>>>>>>> is able to connect to ZooKeeper but the connection closes >>>>>>>>>>>>> immediately. This could be a sign that the server has too >>>>>>>>>>>>> many >>>>>>>>>>>>> connections (30 is the default). Consider inspecting your ZK >>>>>>>>>>>>> server >>>>>>>>>>>>> logs for that error and then make sure you are reusing >>>>>>>>>>>>> HBaseConfiguration as often as you can. See HTable's javadoc >>>>>>>>>>>>> for >>>>>>>>>>>>> more information. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >
