One other thing which is different in my setup is I am using filesystem for hbase rather hbase-indexer needs hdfs setup to be fully functional. So I need to change that.
Also, do you have local hbase setup or hbase cluster mode ? Pankil On Fri, Oct 23, 2015 at 11:23 AM, Pankil Doshi <[email protected]> wrote: > Hi Beeshma, > > Thanks for your response. > > I am running zookeeper locally. but I am not managing it with hbase i.e I > have this set: > export HBASE_MANAGES_ZK=false > > and also: > <property> > <name>hbase.cluster.distributed</name> > <value>true</value> > </property> > > even though I have everything running locally in standalone mode. > > If I dont set "hbase.cluster.distributed" I am seeing zookeeper being > started with start of my hbase. I am not sure if there is any other good > way not to start or stop zookeeper with hbase as by only setting > (HBASE_MANAGES_ZK=false) it doesnt work. > > Were you able to setup hbase-indexer at all ? > > Pankil > > > On Fri, Oct 23, 2015 at 10:51 AM, beeshma r <[email protected]> wrote: > >> Hi Pankil, >> >> Are you sure your hbase is running with external zookeeper ensemble ? >> >> As per documentation on Hbase Replication >> >> >> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html >> >> zookeeper must not be managed by HBase,.But i havent tried this >> >> On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi < >> [email protected]> wrote: >> >> > Hi Pankil. >> > >> > A similar issue was reported few days back ( >> > >> > >> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication >> > ). >> > >> > May be this is due to hbase-indexer code ? >> > One more Q, did you upgrade hbase from 0.94 and you see this issue ? >> > >> > Regards, >> > Ashish Singhi >> > >> > On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <[email protected]> >> wrote: >> > >> > > Hi, >> > > >> > > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily ( >> > > http://ngdata.github.io/hbase-indexer/). >> > > >> > > I am seeing below error when I add my indexer: >> > > >> > > >> > > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread] >> > > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode >> > > expired, triggering peerListChanged event >> > > >> > > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread] >> > > regionserver.ReplicationSourceManager: Error while adding a new peer >> > > >> > > org.apache.hadoop.hbase.replication.ReplicationException: Error adding >> > peer >> > > with id=Indexer_newtest2 >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468) >> > > >> > > at >> > > >> > > >> > >> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522) >> > > >> > > at >> org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498) >> > > >> > > Caused by: org.apache.hadoop.hbase.replication.ReplicationException: >> > Error >> > > starting the peer state tracker for peerId=Indexer_newtest2 >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384) >> > > >> > > ... 6 more >> > > >> > > Caused by: >> > org.apache.zookeeper.KeeperException$DataInconsistencyException: >> > > KeeperErrorCode = DataInconsistency >> > > >> > > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452) >> > > >> > > ... 7 more >> > > >> > > Caused by: >> org.apache.hadoop.hbase.exceptions.DeserializationException: >> > > Missing pb magic PBUF prefix >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90) >> > > >> > > at >> > > >> > > >> > >> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83) >> > > >> > > ... 8 more >> > > >> > > >> > > >> > > My Hbase-site.xml: >> > > >> > > >> > > <?xml version="1.0"?> >> > > >> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> >> > > >> > > <!-- >> > > >> > > /** >> > > >> > > * Copyright 2010 The Apache Software Foundation >> > > >> > > * >> > > >> > > * Licensed to the Apache Software Foundation (ASF) under one >> > > >> > > * or more contributor license agreements. See the NOTICE file >> > > >> > > * distributed with this work for additional information >> > > >> > > * regarding copyright ownership. The ASF licenses this file >> > > >> > > * to you under the Apache License, Version 2.0 (the >> > > >> > > * "License"); you may not use this file except in compliance >> > > >> > > * with the License. You may obtain a copy of the License at >> > > >> > > * >> > > >> > > * http://www.apache.org/licenses/LICENSE-2.0 >> > > >> > > * >> > > >> > > * Unless required by applicable law or agreed to in writing, software >> > > >> > > * distributed under the License is distributed on an "AS IS" BASIS, >> > > >> > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or >> > implied. >> > > >> > > * See the License for the specific language governing permissions and >> > > >> > > * limitations under the License. >> > > >> > > */ >> > > >> > > --> >> > > >> > > <configuration> >> > > >> > > <!--<property> >> > > >> > > <name>zookeeper.znode.parent</name> >> > > >> > > <value>/hbase-unsecure</value> >> > > >> > > </property>--> >> > > >> > > >> > > <property> >> > > >> > > <name>hbase.cluster.distributed</name> >> > > >> > > <value>true</value> >> > > >> > > </property> >> > > >> > > //Here you have to set the path where you want HBase to store its >> files. >> > > >> > > <property> >> > > >> > > <name>hbase.rootdir</name> >> > > >> > > <value>file:/tmp/HBase/HFiles</value> >> > > >> > > </property> >> > > >> > > <property> >> > > >> > > <name>hbase.zookeeper.property.clientPort</name> >> > > >> > > <value>2181</value> >> > > >> > > <description>Property from ZooKeeper's config zoo.cfg. >> > > >> > > The port at which the clients will connect. >> > > >> > > </description> >> > > >> > > </property> >> > > >> > > <property> >> > > >> > > <name>hbase.zookeeper.quorum</name> >> > > >> > > <value>localhost</value> >> > > >> > > <description>Comma separated list of servers in the ZooKeeper >> > Quorum. >> > > >> > > For example, "host1.mydomain.com,host2.mydomain.com, >> > > host3.mydomain.com >> > > ". >> > > >> > > By default this is set to localhost for local and >> > pseudo-distributed >> > > modes >> > > >> > > of operation. For a fully-distributed setup, this should be set >> to >> > a >> > > full >> > > >> > > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in >> > > hbase-env.sh >> > > >> > > this is the list of servers which we will start/stop ZooKeeper >> on. >> > > >> > > </description> >> > > >> > > </property> >> > > >> > > <property> >> > > >> > > <name>hbase.zookeeper.property.dataDir</name> >> > > >> > > <value>/tmp/zookeeper</value> >> > > >> > > <description>Property from ZooKeeper config zoo.cfg. >> > > >> > > The directory where the snapshot is stored. >> > > >> > > </description> >> > > >> > > </property> >> > > >> > > // FOR HBASE INDEXER >> > > >> > > >> > > <!-- SEP is basically replication, so enable it --> >> > > >> > > <property> >> > > >> > > <name>hbase.replication</name> >> > > >> > > <value>true</value> >> > > >> > > </property> >> > > >> > > <!-- Source ratio of 100% makes sure that each SEP consumer is >> actually >> > > >> > > used (otherwise, some can sit idle, especially with small >> > clusters) >> > > --> >> > > >> > > <property> >> > > >> > > <name>replication.source.ratio</name> >> > > >> > > <value>1.0</value> >> > > >> > > </property> >> > > >> > > <!-- Maximum number of hlog entries to replicate in one go. If this >> is >> > > >> > > large, and a consumer takes a while to process the events, the >> > > >> > > HBase rpc call will time out. --> >> > > >> > > <property> >> > > >> > > <name>replication.source.nb.capacity</name> >> > > >> > > <value>1000</value> >> > > >> > > </property> >> > > >> > > <!-- A custom replication source that fixes a few things and adds >> > > >> > > some functionality (doesn't interfere with normal replication >> > > >> > > usage). --> >> > > >> > > <property> >> > > >> > > <name>replication.replicationsource.implementation</name> >> > > >> > > <value>com.ngdata.sep.impl.SepReplicationSource</value> >> > > >> > > </property> >> > > >> > > </configuration> >> > > >> > > >> > > All research so far shows me one can face problem while upgrading >> hbase. >> > > But in my case, I am not upgrading. >> > > >> > > Any ideas/suggestions will be very helpful. >> > > >> > > Thanks, >> > > Pankil >> > > >> > >> >> >> >> -- >> > >
