Hi Beeshma,
Thanks for your response.
I am running zookeeper locally. but I am not managing it with hbase i.e I
have this set:
export HBASE_MANAGES_ZK=false
and also:
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
even though I have everything running locally in standalone mode.
If I dont set "hbase.cluster.distributed" I am seeing zookeeper being
started with start of my hbase. I am not sure if there is any other good
way not to start or stop zookeeper with hbase as by only setting
(HBASE_MANAGES_ZK=false) it doesnt work.
Were you able to setup hbase-indexer at all ?
Pankil
On Fri, Oct 23, 2015 at 10:51 AM, beeshma r <[email protected]> wrote:
> Hi Pankil,
>
> Are you sure your hbase is running with external zookeeper ensemble ?
>
> As per documentation on Hbase Replication
>
>
> http://www.cloudera.com/content/www/en-us/documentation/archive/cdh/4-x/4-2-0/CDH4-Installation-Guide/cdh4ig_topic_20_11.html
>
> zookeeper must not be managed by HBase,.But i havent tried this
>
> On Fri, Oct 23, 2015 at 9:55 AM, Ashish Singhi <
> [email protected]> wrote:
>
> > Hi Pankil.
> >
> > A similar issue was reported few days back (
> >
> >
> http://search-hadoop.com/m/YGbbknQt52rKBDS1&subj=HRegionServer+failed+due+to+replication
> > ).
> >
> > May be this is due to hbase-indexer code ?
> > One more Q, did you upgrade hbase from 0.94 and you see this issue ?
> >
> > Regards,
> > Ashish Singhi
> >
> > On Fri, Oct 23, 2015 at 2:47 AM, Pankil Doshi <[email protected]>
> wrote:
> >
> > > Hi,
> > >
> > > I am using hbase-0.98.15-hadoop2 and hbase-indexer from lily (
> > > http://ngdata.github.io/hbase-indexer/).
> > >
> > > I am seeing below error when I add my indexer:
> > >
> > >
> > > 2015-10-22 14:08:27,468 INFO [regionserver60020-EventThread]
> > > replication.ReplicationTrackerZKImpl: /hbase/replication/peers znode
> > > expired, triggering peerListChanged event
> > >
> > > 2015-10-22 14:08:27,473 ERROR [regionserver60020-EventThread]
> > > regionserver.ReplicationSourceManager: Error while adding a new peer
> > >
> > > org.apache.hadoop.hbase.replication.ReplicationException: Error adding
> > peer
> > > with id=Indexer_newtest2
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:386)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.peerAdded(ReplicationPeersZKImpl.java:358)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager.peerListChanged(ReplicationSourceManager.java:514)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationTrackerZKImpl$PeersWatcher.nodeChildrenChanged(ReplicationTrackerZKImpl.java:189)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.process(ZooKeeperWatcher.java:468)
> > >
> > > at
> > >
> > >
> >
> org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:522)
> > >
> > > at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:498)
> > >
> > > Caused by: org.apache.hadoop.hbase.replication.ReplicationException:
> > Error
> > > starting the peer state tracker for peerId=Indexer_newtest2
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:454)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createAndAddPeer(ReplicationPeersZKImpl.java:384)
> > >
> > > ... 6 more
> > >
> > > Caused by:
> > org.apache.zookeeper.KeeperException$DataInconsistencyException:
> > > KeeperErrorCode = DataInconsistency
> > >
> > > at org.apache.hadoop.hbase.zookeeper.ZKUtil.convert(ZKUtil.java:2063)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:85)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeersZKImpl.createPeer(ReplicationPeersZKImpl.java:452)
> > >
> > > ... 7 more
> > >
> > > Caused by: org.apache.hadoop.hbase.exceptions.DeserializationException:
> > > Missing pb magic PBUF prefix
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.expectPBMagicPrefix(ProtobufUtil.java:270)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.parseStateFrom(ReplicationPeerZKImpl.java:243)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.isStateEnabled(ReplicationPeerZKImpl.java:232)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.readPeerStateZnode(ReplicationPeerZKImpl.java:90)
> > >
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.replication.ReplicationPeerZKImpl.startStateTracker(ReplicationPeerZKImpl.java:83)
> > >
> > > ... 8 more
> > >
> > >
> > >
> > > My Hbase-site.xml:
> > >
> > >
> > > <?xml version="1.0"?>
> > >
> > > <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> > >
> > > <!--
> > >
> > > /**
> > >
> > > * Copyright 2010 The Apache Software Foundation
> > >
> > > *
> > >
> > > * Licensed to the Apache Software Foundation (ASF) under one
> > >
> > > * or more contributor license agreements. See the NOTICE file
> > >
> > > * distributed with this work for additional information
> > >
> > > * regarding copyright ownership. The ASF licenses this file
> > >
> > > * to you under the Apache License, Version 2.0 (the
> > >
> > > * "License"); you may not use this file except in compliance
> > >
> > > * with the License. You may obtain a copy of the License at
> > >
> > > *
> > >
> > > * http://www.apache.org/licenses/LICENSE-2.0
> > >
> > > *
> > >
> > > * Unless required by applicable law or agreed to in writing, software
> > >
> > > * distributed under the License is distributed on an "AS IS" BASIS,
> > >
> > > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
> > implied.
> > >
> > > * See the License for the specific language governing permissions and
> > >
> > > * limitations under the License.
> > >
> > > */
> > >
> > > -->
> > >
> > > <configuration>
> > >
> > > <!--<property>
> > >
> > > <name>zookeeper.znode.parent</name>
> > >
> > > <value>/hbase-unsecure</value>
> > >
> > > </property>-->
> > >
> > >
> > > <property>
> > >
> > > <name>hbase.cluster.distributed</name>
> > >
> > > <value>true</value>
> > >
> > > </property>
> > >
> > > //Here you have to set the path where you want HBase to store its
> files.
> > >
> > > <property>
> > >
> > > <name>hbase.rootdir</name>
> > >
> > > <value>file:/tmp/HBase/HFiles</value>
> > >
> > > </property>
> > >
> > > <property>
> > >
> > > <name>hbase.zookeeper.property.clientPort</name>
> > >
> > > <value>2181</value>
> > >
> > > <description>Property from ZooKeeper's config zoo.cfg.
> > >
> > > The port at which the clients will connect.
> > >
> > > </description>
> > >
> > > </property>
> > >
> > > <property>
> > >
> > > <name>hbase.zookeeper.quorum</name>
> > >
> > > <value>localhost</value>
> > >
> > > <description>Comma separated list of servers in the ZooKeeper
> > Quorum.
> > >
> > > For example, "host1.mydomain.com,host2.mydomain.com,
> > > host3.mydomain.com
> > > ".
> > >
> > > By default this is set to localhost for local and
> > pseudo-distributed
> > > modes
> > >
> > > of operation. For a fully-distributed setup, this should be set
> to
> > a
> > > full
> > >
> > > list of ZooKeeper quorum servers. If HBASE_MANAGES_ZK is set in
> > > hbase-env.sh
> > >
> > > this is the list of servers which we will start/stop ZooKeeper
> on.
> > >
> > > </description>
> > >
> > > </property>
> > >
> > > <property>
> > >
> > > <name>hbase.zookeeper.property.dataDir</name>
> > >
> > > <value>/tmp/zookeeper</value>
> > >
> > > <description>Property from ZooKeeper config zoo.cfg.
> > >
> > > The directory where the snapshot is stored.
> > >
> > > </description>
> > >
> > > </property>
> > >
> > > // FOR HBASE INDEXER
> > >
> > >
> > > <!-- SEP is basically replication, so enable it -->
> > >
> > > <property>
> > >
> > > <name>hbase.replication</name>
> > >
> > > <value>true</value>
> > >
> > > </property>
> > >
> > > <!-- Source ratio of 100% makes sure that each SEP consumer is
> actually
> > >
> > > used (otherwise, some can sit idle, especially with small
> > clusters)
> > > -->
> > >
> > > <property>
> > >
> > > <name>replication.source.ratio</name>
> > >
> > > <value>1.0</value>
> > >
> > > </property>
> > >
> > > <!-- Maximum number of hlog entries to replicate in one go. If this
> is
> > >
> > > large, and a consumer takes a while to process the events, the
> > >
> > > HBase rpc call will time out. -->
> > >
> > > <property>
> > >
> > > <name>replication.source.nb.capacity</name>
> > >
> > > <value>1000</value>
> > >
> > > </property>
> > >
> > > <!-- A custom replication source that fixes a few things and adds
> > >
> > > some functionality (doesn't interfere with normal replication
> > >
> > > usage). -->
> > >
> > > <property>
> > >
> > > <name>replication.replicationsource.implementation</name>
> > >
> > > <value>com.ngdata.sep.impl.SepReplicationSource</value>
> > >
> > > </property>
> > >
> > > </configuration>
> > >
> > >
> > > All research so far shows me one can face problem while upgrading
> hbase.
> > > But in my case, I am not upgrading.
> > >
> > > Any ideas/suggestions will be very helpful.
> > >
> > > Thanks,
> > > Pankil
> > >
> >
>
>
>
> --
>
