Re: Please welcome new HBase committer Jing Chen (Jerry) He

Jerry, congratulations! well deserved On Wed, Apr 1, 2015 at 11:23 AM, Esteban Gutierrez wrote: > Congrats, Jerry! > > -- > Cloudera, Inc. > > > On Wed, Apr 1, 2015 at 10:55 AM, Ted Yu wrote: > > > Congratulations, Jerry. > > > > On Wed, Apr 1, 2015 at 10:53 AM, Andrew Purtell > > wrote: > > >

Re: read HBase regions in MPP way?

ce programs. Probably you can use code from that > package to turn a Scan into input splits, which contain region name > and RegionServer location, and consume those from your custom coordinator. > > -n > > On Tuesday, February 3, 2015, Demai Ni wrote: > > > hi, Guys, > &

read HBase regions in MPP way?

hi, Guys, I am looking for a way to Read HBase table through MPP(Postgres-XC). And hoping to get some suggestions to either validate or invalidate the approach. Kind of like Apache Drill, but through PostgresSQL. Long story about why Postgres, and how c/c++ will give me headache for months to com

Re: Replication terminology "master-master"

Agree with you guys. I also experienced difficulty to explain about Master-Master and cyclical to my users before, until I began to talk about M-S only as building-block, which can be applied as one please. Just like to chime in another point, maybe the document should emphasize that the Master o

[jira] [Created] (HBASE-11617) AgeOfLastAppliedOp in MetricsSink got increased when no new replication sink OP

Demai Ni created HBASE-11617: Summary: AgeOfLastAppliedOp in MetricsSink got increased when no new replication sink OP Key: HBASE-11617 URL: https://issues.apache.org/jira/browse/HBASE-11617 Project

Re: meaning for AgeOfLastAppliedOp in Replication MetricsSink

n 0.94. > Looks like 0.94 is doing the right thing with ageOfLastAppliedEdit, but > 0.98+ is not. > > -- Lars > > ---------- > *From:* lars hofhansl > *To:* Demai Ni ; "dev@hbase.apache.org" < > dev@hbase.apache.org> >

Re: meaning for AgeOfLastAppliedOp in Replication MetricsSink

- > the time an edit entered the system (it's WAL time) > > > -- Lars > > > > > From: Demai Ni > To: "dev@hbase.apache.org" > Sent: Tuesday, July 29, 2014 3:48 PM > Subject: meaning for AgeOfLastAppliedOp in Replication Metrics

meaning for AgeOfLastAppliedOp in Replication MetricsSink

hi, A quick question to clarify this AgeOfLastAppliedOp in MetricsSink.java. I assume it is used as an indicator about how long for a Sink OP to be applied; but instead, it is more like to show how long since the last Sink OP applied /** * Set the age of the last applied operation * *

[jira] [Created] (HBASE-11566) make ExportSnapshot extendable by removing 'final'

Demai Ni created HBASE-11566: Summary: make ExportSnapshot extendable by removing 'final' Key: HBASE-11566 URL: https://issues.apache.org/jira/browse/HBASE-11566 Project: HBase

Re: hadoopQA links to findbugs warning broken

Stack, I don't know how to fix it. :-) Can you please give me a pointer? thanks Demai On Wed, Jul 2, 2014 at 5:51 PM, Stack wrote: > On Wed, Jul 2, 2014 at 1:23 PM, Ted Yu wrote: > > > You can execute the following command locally: > > mvn clean package findbugs:findbugs > > > > Why not fix

Re: hadoopQA links to findbugs warning broken

2014-07-02 13:23 GMT-07:00 Ted Yu : > > > > > You can execute the following command locally: > > > mvn clean package findbugs:findbugs > > > > > > then you would be able to check target/findbugsXml.xml under respective > > > module. > > &

hadoopQA links to findbugs warning broken

hi, folks, I encountered this problem a couple times already. Basically, hadoopQA cames back with complain about "-1 findbugs. ..", then I tried to following the links in the hadoopQA report to diag, but none of the findbugs links works. The latest example is here: for hbase-9531 https://issues.ap

[jira] [Created] (HBASE-11452) add userPermission feature in AccessControlClient as client API

Demai Ni created HBASE-11452: Summary: add userPermission feature in AccessControlClient as client API Key: HBASE-11452 URL: https://issues.apache.org/jira/browse/HBASE-11452 Project: HBase

Re: how to use different Sun/IBM classes according to JDK? (HBase-10336)

ed to add handling of the x509 references. > > St.Ack > > > On Thu, Jun 26, 2014 at 11:57 AM, Demai Ni wrote: > >> hi, folks, >> >> a few days ago, jira HBase-10336 added a UT KeyStoreTestUtil.java, which >> leverages the following sun classes: >

how to use different Sun/IBM classes according to JDK? (HBase-10336)

hi, folks, a few days ago, jira HBase-10336 added a UT KeyStoreTestUtil.java, which leverages the following sun classes: import sun.security.x509.AlgorithmId; import sun.security.x509.CertificateAlgorithmId; well, it happens that I am amongst the minority who are using IBM JDK, and w

Re: java implementation like security.rb (ACL grant, revoke, etc.)

can change arbitrarily at any time at > our option. > > You want to look at AccessControlClient in hbase-client. If there are > things you'd like to see there which are currently missing, please file a > JIRA and consider providing a patch. > > > On Fri, Jun 20, 2014 at 9:40

Re: java implementation like security.rb (ACL grant, revoke, etc.)

n Fri, Jun 20, 2014 at 3:12 PM, Demai Ni wrote: > > > hi, folks, > > > > I am looking for java API that can do 'grant','revoke' etc, which is > > exactly the functions implemented in security.rb. So that the function > can > > be called through j

java implementation like security.rb (ACL grant, revoke, etc.)

hi, folks, I am looking for java API that can do 'grant','revoke' etc, which is exactly the functions implemented in security.rb. So that the function can be called through java client. It is kind of straigtforward to do a syntax translation from jruby to java. Well, just like to check whether we

[jira] [Created] (HBASE-11327) ExportSnapshot hit stackoverflow error when target snapshotDir doesn't contain uri

Demai Ni created HBASE-11327: Summary: ExportSnapshot hit stackoverflow error when target snapshotDir doesn't contain uri Key: HBASE-11327 URL: https://issues.apache.org/jira/browse/HBASE-11327 Pr

Re: coprocessor piggy-back on compaction logic

Andrew, I will definitely take a look, would be a good one to play with ... Demai On Thu, Jun 5, 2014 at 12:20 PM, Andrew Purtell wrote: > Would be awesome if you pick that up Demai! > > > On Thu, Jun 5, 2014 at 11:47 AM, Demai Ni wrote: > > > Jesse, you are fast. :-)

Re: coprocessor piggy-back on compaction logic

Jesse, you are fast. :-) thanks a lot... Demai On Thu, Jun 5, 2014 at 11:45 AM, Jesse Yates wrote: > HBASE-7958 > > --- > Jesse Yates > @jesse_yates > jyates.github.com > > > On Thu, Jun 5, 2014 at 11:43 AM, Demai Ni wrote: > > > hi, fo

coprocessor piggy-back on compaction logic

hi, folks, I am thinking about the possibility to implement a table statistic collector feature by plugging in exiting compaction logic through coprocessor. Wondering someone can give me a couple pointer/jira that already piggy-back on compaction? so I don't need to reinvent the wheel? Many than

Re: Gathering opinions and recommendations

2014 at 2:59 PM, Enis Söztutar wrote: > > > On Tue, May 27, 2014 at 2:47 PM, Demai Ni wrote: > > > > > Enis, > > > > > > for customer already on 0.96, can they rolling restart to 0.98, and > then > > > from 0.98 rolling to 1.0? > > >

Re: Gathering opinions and recommendations

Enis, for customer already on 0.96, can they rolling restart to 0.98, and then from 0.98 rolling to 1.0? Demai On Tue, May 27, 2014 at 2:40 PM, Enis Söztutar wrote: > Inlined. > > > > > > 2014-05-27 16:10 GMT-04:00 Michael Segel : > > > > > Isn’t HDP 2.1 shipping 0.98.x ? > > > > Yes. > > > >

Re: [ANNOUNCE] Apache Phoenix has graduated as a top level project

Congratulations! Phoenix has been attracting a lot of userst. Good job, guys! Demai On Thu, May 22, 2014 at 2:47 PM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > Oh, nice!!! > > Congratulations to all the Phoenix team!!! > > JM > Le 2014-05-22 17:46, "James Taylor" a écrit : > > >

Re: HBase Backup/Restore solution(HBase-7912)

d > be able to understand the code better. > > > Cheers > > > > On Mon, May 19, 2014 at 1:38 PM, Demai Ni wrote: > > > hi, folks, > > > > We are working on jira HBase-7912( > > https://issues.apache.org/jira/browse/HBASE-7912) to open source a

HBase Backup/Restore solution(HBase-7912)

hi, folks, We are working on jira HBase-7912( https://issues.apache.org/jira/browse/HBASE-7912) to open source a backup solution based on Snapshot and WAL This JiRA provides a general solution to common users who need to backup hbase table(s) with a Fullbackup(HBase-10900) as baseline and periode

Re: jetty 9 on hbase? (HADOOP-10075)

x27;t be actionable. We can't upgrade our jetty > dependency if it's going to conflict with what Hadoop uses and brings in > transitively. > > On Sunday, May 18, 2014, Demai Ni wrote: > > > Sounds good. I will open one if I can make it works on 96.x. > > > > Curr

Re: jetty 9 on hbase? (HADOOP-10075)

ngs. > > Cheers > > > On Fri, May 16, 2014 at 7:49 PM, Demai Ni wrote: > >> Ted, >> >> Thanks. I am changing the poms (both top and the one under Hbase- server). >> Due to the incompatible btw jetty 6 and 9, infoserver and RESTserver need >> code cha

Re: jetty 9 on hbase? (HADOOP-10075)

075 is not integrated yet. > It doesn't have Fix Version(s) either. > > For HBase, you can change the version strings in pom.xml to match the ones > from your hadoop build : >6.1.26 >6.1.14 > > Cheers > > > On Fri, May 16, 2014 at 2:49 PM, Demai Ni

jetty 9 on hbase? (HADOOP-10075)

hi, folks, wondering anyone already began to use Jetty 9 with HBase. We are trying to use Jetty9 in hadoop, and got the following exceptions when start HBase: ... java.lang.ClassCastException: org.eclipse.jetty.servlet.ServletContextHandler incompatible with org.mortbay.jetty.servlet.Context

[jira] [Created] (HBASE-11174) show backup/restore progress

Demai Ni created HBASE-11174: Summary: show backup/restore progress Key: HBASE-11174 URL: https://issues.apache.org/jira/browse/HBASE-11174 Project: HBase Issue Type: New Feature Affects

[jira] [Created] (HBASE-11173) Show Backup History

Demai Ni created HBASE-11173: Summary: Show Backup History Key: HBASE-11173 URL: https://issues.apache.org/jira/browse/HBASE-11173 Project: HBase Issue Type: New Feature Affects Versions

[jira] [Created] (HBASE-11172) Cancal a backup process

Demai Ni created HBASE-11172: Summary: Cancal a backup process Key: HBASE-11172 URL: https://issues.apache.org/jira/browse/HBASE-11172 Project: HBase Issue Type: New Feature Affects

[jira] [Created] (HBASE-11175) improve Backup/Restore framework by abstracting out zookeeper

Demai Ni created HBASE-11175: Summary: improve Backup/Restore framework by abstracting out zookeeper Key: HBASE-11175 URL: https://issues.apache.org/jira/browse/HBASE-11175 Project: HBase Issue

Re: HBase Hackathon @ Salesforce 05/06/2014 notes

Stack, thanks for sharing. I can't access the google doc mentioned after "...Ryan then presented this doc..." due to permission. Is there a way to get it? thanks Demai On Mon, May 12, 2014 at 12:12 PM, Jean-Marc Spaggiari < jean-m...@spaggiari.org> wrote: > Nice! Thanks for sharing St.Ack. >

Re: Root and Meta removed in 0.96?

JD, thanks a lot. You are fast. :-).. Demai On Mon, Apr 28, 2014 at 10:02 AM, Jean-Daniel Cryans wrote: > Only ROOT https://issues.apache.org/jira/browse/HBASE-3171 > > > On Mon, Apr 28, 2014 at 10:00 AM, Demai Ni wrote: > > > hi, > > > > a quick question

Root and Meta removed in 0.96?

hi, a quick question. I remembered that these two are removed starting 0.96? not sure my memory serves me well on this. I googled but can't find the jira or document for this. Can someone give me a pointer? many thanks Demai

[jira] [Created] (HBASE-11085) Incremental Backup Restore support

Demai Ni created HBASE-11085: Summary: Incremental Backup Restore support Key: HBASE-11085 URL: https://issues.apache.org/jira/browse/HBASE-11085 Project: HBase Issue Type: New Feature

Re: a way to record # of rows 'deleted' due to TTL

ions). > > > -- Lars > > > > ________ > From: Demai Ni > To: "dev@hbase.apache.org" > Sent: Friday, April 18, 2014 2:27 PM > Subject: a way to record # of rows 'deleted' due to TTL > > > hi, folks, > > Understand that TT

a way to record # of rows 'deleted' due to TTL

hi, folks, Understand that TTL plays its role in the background, I am wondering whether there is a way to have some information of the retired rows? for example, 1) 100 rows were put initially 2) during one day period, 2 rows was deleted by user, and 10 rows 'retired' due to TTL is there a way t

[jira] [Created] (HBASE-10950) Add a configuration point for MaxVersion of Column Family

Demai Ni created HBASE-10950: Summary: Add a configuration point for MaxVersion of Column Family Key: HBASE-10950 URL: https://issues.apache.org/jira/browse/HBASE-10950 Project: HBase Issue

[jira] [Created] (HBASE-10941) default for max version isn't updated in doc after change on 0.96

Demai Ni created HBASE-10941: Summary: default for max version isn't updated in doc after change on 0.96 Key: HBASE-10941 URL: https://issues.apache.org/jira/browse/HBASE-10941 Project:

[jira] [Created] (HBASE-10900) FULL table backup and restore

Demai Ni created HBASE-10900: Summary: FULL table backup and restore Key: HBASE-10900 URL: https://issues.apache.org/jira/browse/HBASE-10900 Project: HBase Issue Type: Task Reporter

[jira] [Created] (HBASE-10812) Backport HBASE-9426(Make custom distributed barrier procedure pluggable) to 0.98.x

Demai Ni created HBASE-10812: Summary: Backport HBASE-9426(Make custom distributed barrier procedure pluggable) to 0.98.x Key: HBASE-10812 URL: https://issues.apache.org/jira/browse/HBASE-10812 Project

[jira] [Created] (HBASE-10793) AuthFailed as a valid zookeeper state

Demai Ni created HBASE-10793: Summary: AuthFailed as a valid zookeeper state Key: HBASE-10793 URL: https://issues.apache.org/jira/browse/HBASE-10793 Project: HBase Issue Type: Bug

[jira] [Created] (HBASE-10732) should be 'drop_namespace', instead of 'delete_namespace'

Demai Ni created HBASE-10732: Summary: should be 'drop_namespace', instead of 'delete_namespace' Key: HBASE-10732 URL: https://issues.apache.org/jira/browse/HBASE-10732 Project: H

[jira] [Created] (HBASE-10476) HBase Master log grows very fast after stopped hadoop (due to connection exception)

Demai Ni created HBASE-10476: Summary: HBase Master log grows very fast after stopped hadoop (due to connection exception) Key: HBASE-10476 URL: https://issues.apache.org/jira/browse/HBASE-10476 Project

Re: HBase log grows very fast after stopped hadoop (due to connection exception)

gt; > > On Tue, Feb 4, 2014 at 2:09 PM, Demai Ni wrote: > >> hi, >> >> we are using hbase 96.0(also saw the same issue on 94.x) on single node >> cluster. At some point, we stopped Hadoop, but keep hbase rerunning. As >> expected, hbase began to throw follo

HBase log grows very fast after stopped hadoop (due to connection exception)

hi, we are using hbase 96.0(also saw the same issue on 94.x) on single node cluster. At some point, we stopped Hadoop, but keep hbase rerunning. As expected, hbase began to throw following errors: .. 2014-02-04 11:05:12,820 ERROR org.apache.hadoop.hbase.executor.EventHandler: Caught throwable whil

where is TestMXBean.java on 96.0?

hi, folks, a quick question. On hbase94.x, we have a testcase called /hbase/trunk/src/test/java/org/apache/hadoop/hbase/master/TestMXBean.java for some reason, I can't find it on 96.0 anymore. (maybe the version of hbase I got doesn't contain everything?). any ideas? thanks Demai

Re: question about how to add a new field in matching in ClusterStatusProtos:ServerLoad

after made and installed protoc 2.5, everything works fine now. Many thanks for the help Demai On Mon, Dec 16, 2013 at 7:13 PM, Demai Ni wrote: > Sergey, > > thanks a lot. This is the first time I am playing with proto buffer. You > saved me a lot of time. > > I used yum t

Re: question about how to add a new field in matching in ClusterStatusProtos:ServerLoad

, Sergey Shelukhin wrote: > This is generated protobuf code. > You just need to edit .proto file and generate code; see > ./hbase-protocol/README.txt > > > On Mon, Dec 16, 2013 at 6:13 PM, Demai Ni wrote: > > > hi, folks, > > > > I am reading the code inside

question about how to add a new field in matching in ClusterStatusProtos:ServerLoad

hi, folks, I am reading the code inside ClusterStatusProtos:ServerLoad, and couldn't figure out some simple logic when trying to add a new field. Can someone enlighten me please? --- I use the trunk version for @line @4517~4580, is there a relationship

[HBase 96.x] getter in MetricsSource, MetricsReplicationSource, BaseSourceImpl and BaseSource

Dear dev, I am wondering how to get some metrics values set inside MetricsSource.java. Particularly, I am interest in this "MetricsReplicationSource rms". Inside MetricsSource.java, there are setters like : rms.setGauge( ). Is there a way to get the value back using a getter? like "long get

[jira] [Created] (HBASE-10099) javadoc warning instroduced by LabelExpander 188: warning - @return tag has no arguments

Demai Ni created HBASE-10099: Summary: javadoc warning instroduced by LabelExpander 188: warning - @return tag has no arguments Key: HBASE-10099 URL: https://issues.apache.org/jira/browse/HBASE-10099

how to add a field in HServerLoad (readFields() and write()) without break compatibility within same major release?

hi, folks, I encountered a similar problem that fixed by HBASE-5795. But in this case, the cluster is hbase 0.94.9, and the downlevel client is 0.94.3. Does this mean that we cannot add additional field within the same major release? thanks. The fo

Re: Setting up NxN replication

Vlad, nice one. :-) Very good point. Unless your company have data center in 10 different locations, I don't see a good use case for such complex setup Also, please keep in mind, currently the only replication type is 'global'. Before we find a good way to specify peer at table:family level. It wi

Re: Setting up NxN replication

hi, Ishan, can you please elaborate a bit about your use case? for your example of 3 cluster, I think it should be set up as two M-S for the same table:family first one: A = Master, B = Slave 2nd one: B = slave, C = Master. There is no need to setup the direct relationship between A and C Demai

Re: how long the Hadoop-QA result is kept available?

ocWarnings.txt > > > > > https://builds.apache.org/job/PreCommit-HBASE-Build/7779/artifact/trunk/patchprocess/patchJavadocWarnings.txt > > > > By navigating up the directory tree, you should be able to figure out how > > to get there. > > > > Cheers > > >

Re: how long the Hadoop-QA result is kept available?

warning message(is there a link to file patchJavadocWarnings.txt?) I also ran the 'test-patch.sh' locally, which didn't encounter additional javadoc warning. Thanks for your help in advance. Demai On Thu, Nov 7, 2013 at 10:52 AM, Demai Ni wrote: > Ted, yeah. I think the -

Re: how long the Hadoop-QA result is kept available?

tests: > > You can always re-submit the latest patch to see if the test failure was > reproducible (I assume you have run replication-related tests locally). > > > On Thu, Nov 7, 2013 at 9:15 AM, Demai Ni wrote: > > > Ted, thank you so much for the tips. It helps me to save

Re: how long the Hadoop-QA result is kept available?

s were kept. > There are many patches submitted everyday. Each QA result would be kept > shorter than 24 hours. > > If you're investigating javadoc / findbugs warnings, you can find the > respective command from dev-support/test-patch.sh > > Cheers > > > On Thu

how long the Hadoop-QA result is kept available?

hi, folks, sorry for asking the native question. I ran a Hadoop-QA yesterday afternoon, and thought the result will be on the server for 24 hours. However, it isn't available anymore (~12 hours) It is for hbase-9047, and this is link of the result. https://builds.apache.org/job/PreCommit-HBASE-Bu

[Question] replication got impact by zoo.cfg (if put under ./hbase/conf)

hi, folks, I encountered a problem last week that my cluster was replicating to itself. The problem was discussed and described here: http://mail-archives.apache.org/mod_mbox/hbase-dev/201311.mbox/%3CCAOEq2C5%2ByqsewgzpkPO%3D8FRvQVv6tqOoo5wrdxDVN%2BzRzV1iWw%40mail.gmail.com%3E Finally, we found t

Re: replicating to oneself?

the problem is figured out by one of my co-worker. Someone put a zoo.cfg under ./hbase/conf, which messed up the quorum look up. many thanks for your help On Fri, Nov 1, 2013 at 12:48 PM, Demai Ni wrote: > I injected more debug code into ReplicationPeer. > > public Replic

Re: replicating to oneself?

vm134.svl.ibm.com:2181* 2013-11-01 12:40:33,351 INFO org.apache.hadoop.hbase.replication.ReplicationPeer: Demai @ReplicationPeer : this.zkw=connection to cluster: hdtest014.svl.ibm.com:2181:/hbase On Fri, Nov 1, 2013 at 11:12 AM, Demai Ni wrote: > Himanshu and Nick, > > many thanks for y

Re: replicating to oneself?

you clearing ZK as part of deploying a fresh cluster or > are you re-using existing znodes? How did you configure replication in the > shell? Can you provide those commands? I'd request debug logs from > o.a.h.h.regionserver.Replication but i don't see much logging in there > any

Re: replicating to oneself?

k wrote: > Can you post your replication settings from hbase-site.xml? > > On Thursday, October 31, 2013, Demai Ni wrote: > > > hi, folks, > > > > I got a strange thing happening on my cluster(hbase 0.94.9) recently. I > am > > setting up a new cluster for re

replicating to oneself?

hi, folks, I got a strange thing happening on my cluster(hbase 0.94.9) recently. I am setting up a new cluster for replication, and didn't see the data being replicated over the peer. Then, I found the following in the log of the regionserver of the Master: 2013-10-31 13:33:03,293 INFO org.apache

Re: [Question: Replication] how to detect whether there is a replication pending?

, unless there's some problem preventing > replication). > > J-D > > > On Wed, Sep 18, 2013 at 9:33 AM, Demai Ni wrote: > > > hi, folks, > > > > I am looking for a way to flag that NO replication is pending. For > example > > of a simple Master

Re: [Replication] how ReplicationLogCleaner is triggered during hbase start up?

n Wed, Oct 2, 2013 at 12:39 PM, Demai Ni wrote: > >> hi, folks, >> >> I am trying to figure out this logic and probably leverage it. >> >> When HBase started, it will exam the replication log through >> RecoverableZooKeeper, and call ReplicationLogClea

[Replication] how ReplicationLogCleaner is triggered during hbase start up?

hi, folks, I am trying to figure out this logic and probably leverage it. When HBase started, it will exam the replication log through RecoverableZooKeeper, and call ReplicationLogCleaner. Here is the portion of the log below. I am wondering where is the code trigger it? Thanks 2013-10-02 11:51:

Re: Random reads are 5x faster in 0.96.0! Or not

hi, J-D, It is a very exciting improvement. If we'd like to make 0.94 user benefit from this, will backport HBASE-7008 be good enough, code wise? Certainly, more testing is needed to guard it. Thanks. Demai On Mon, Sep 23, 2013 at 5:10 PM, Jean-Daniel Cryans wrote: > Hey devs, > > I was testin

Re: [Question: Replication] how to detect whether there is a replication pending?

e to look at the shipped ops/batch rates (if it > sits at 0, you're not replicating, unless there's some problem preventing > replication). > > J-D > > > On Wed, Sep 18, 2013 at 9:33 AM, Demai Ni wrote: > > > hi, folks, > > > > I am looking for

Re: [Question: Replication] how to detect whether there is a replication pending?

on). > > J-D > > > On Wed, Sep 18, 2013 at 9:33 AM, Demai Ni wrote: > > > hi, folks, > > > > I am looking for a way to flag that NO replication is pending. For > example > > of a simple Master-Slave setup for table t1. At a given point of time, > can

[Question: Replication] how to detect whether there is a replication pending?

hi, folks, I am looking for a way to flag that NO replication is pending. For example of a simple Master-Slave setup for table t1. At a given point of time, can I tell whether there are still Edits pending for replication? I am thinking about using replicationsource.metrics.sizeOfLogQueue. The onl

[jira] [Created] (HBASE-9531) a command line (hbase shell) interface to retreive the replication metrics and show replication lag

Demai Ni created HBASE-9531: --- Summary: a command line (hbase shell) interface to retreive the replication metrics and show replication lag Key: HBASE-9531 URL: https://issues.apache.org/jira/browse/HBASE-9531

Re: [question] command line to get replication metrics info

Many thanks Demai On Mon, Aug 19, 2013 at 3:27 PM, Demai Ni wrote: > JD, thanks a lot. I will look into it Demai > > > On Mon, Aug 19, 2013 at 2:44 PM, Jean-Daniel Cryans > wrote: > >> See what the "status" command calls in the shell, I would su

Re: [question] command line to get replication metrics info

on'" option, > instead of having a different command. > > J-D > > > On Mon, Aug 19, 2013 at 10:23 AM, Demai Ni wrote: > > > hi, folks > > > > Looking for your kind suggestions again. > > > > I am working on a command line feature to re

Re: how to get ZooKeeperWatcher info ?

19, 2013 at 11:09 AM, Stack wrote: > On Sun, Aug 18, 2013 at 11:37 AM, Demai Ni wrote: > > > St.Ack and Ted, > > > > many thanks for your help and suggestions on this question. I am working > > HBASE-9047 while encountering this question. To provide a whole picture

[question] command line to get replication metrics info

hi, folks Looking for your kind suggestions again. I am working on a command line feature to retrieve the replication metrics info such as : replicationSource: ageOfLastShippedOp, timeStampsOfLastShippedOp and sizeOfLogQueue replicationSink: ageOfLastAppliedOp, and timeStampsOfLastAppliedOp The

Re: how to get ZooKeeperWatcher info ?

restarting. will RecoverableZooKeeper instance have the access to the replication info from original zookeeper? Demai On Sun, Aug 18, 2013 at 9:35 AM, Stack wrote: > On Fri, Aug 16, 2013 at 9:58 AM, Demai Ni wrote: > > > hi, folks, > > > > I am writing some code to get the

Re: how to get ZooKeeperWatcher info ?

gt; > On Fri, Aug 16, 2013 at 10:52 AM, Demai Ni wrote: > >> Ted, thanks. I will need to find away around it. :-) ... demai >> >> >> On Fri, Aug 16, 2013 at 10:38 AM, Ted Yu wrote: >> >>> Actually getKeepAliveZooKeeperWatcher() is currently package

Re: how to get ZooKeeperWatcher info ?

Ted, thanks. I will need to find away around it. :-) ... demai On Fri, Aug 16, 2013 at 10:38 AM, Ted Yu wrote: > Actually getKeepAliveZooKeeperWatcher() is currently package private. > > FYI > > > On Fri, Aug 16, 2013 at 10:32 AM, Demai Ni wrote: > > > hi, Ted, &

Re: how to get ZooKeeperWatcher info ?

e you've > have finished with it. > * @return The shared instance. Never returns null. > */ > ZooKeeperKeepAliveConnection getKeepAliveZooKeeperWatcher() > > > On Fri, Aug 16, 2013 at 9:58 AM, Demai Ni wrote: > > > hi, folks, > > > > I am

how to get ZooKeeperWatcher info ?

hi, folks, I am writing some code to get the zookeeper while hbase is offline. conf = HBaseConfiguration.create(); connection = HConnectionManager.getConnection(conf); zkw = connection.*getZooKeeperWatcher();* replicationZK = new ReplicationZookeeper(connection, conf, zkw

[jira] [Created] (HBASE-9220) An API(and shell command) to list tables replicated TO the current cluster

Demai Ni created HBASE-9220: --- Summary: An API(and shell command) to list tables replicated TO the current cluster Key: HBASE-9220 URL: https://issues.apache.org/jira/browse/HBASE-9220 Project: HBase

Re: [question about replication] how to apply delta from Master to Slave after crash ?

thing here is that you could start that tool on all your > > machines and the load will be spread out, but that might not be a big > > concern if replication wasn't lagging since it would take a few > > seconds to finish replicating the missing data for each regio

Re: [question about replication] how to apply delta from Master to Slave after crash ?

JD, yeah. that sounds what I will need to do. a tool like this [slave_cluster]$tool_to_syncup master_ZKquorum table_name start_timestamp so two tasks for me: 1) identify the start_timestamp 2) write the tool_to_syncup which will reach to master_ZK, copy the HLOGs from makster, replay the HLOGs on

Re: [question about replication] how to apply delta from Master to Slave after crash ?

wrote: > As soon as you restart your master, it will continue replicating from > where it was, so it will send that delta. > > J-D > > On Mon, Jul 22, 2013 at 4:42 PM, Demai Ni wrote: > > hi, folks, > > > > I am trying to address a recovery scenario after Mast

[question about replication] how to apply delta from Master to Slave after crash ?

hi, folks, I am trying to address a recovery scenario after Master crash. In a simple Master - Slave replication setup for t1, and suddenly Master crashes. there will be a small delta of t1 already changed on Master, but haven't been replicated to Slave yet. Assuming Master's filesystem and netw

[jira] [Created] (HBASE-8780) a column Family can have VERSIONS less than zero

Demai Ni created HBASE-8780: --- Summary: a column Family can have VERSIONS less than zero Key: HBASE-8780 URL: https://issues.apache.org/jira/browse/HBASE-8780 Project: HBase Issue Type: Bug

[jira] [Created] (HBASE-8771) ensure replication_scope's value is either local(0) or global(1)

Demai Ni created HBASE-8771: --- Summary: ensure replication_scope's value is either local(0) or global(1) Key: HBASE-8771 URL: https://issues.apache.org/jira/browse/HBASE-8771 Project: HBase

Re: how to use HBase Shell formatter?

Ted, Thanks for pointing me the direction. I will looking into the enhancement Demai on the run On Jun 2, 2013, at 1:43 PM, Ted Yu wrote: > Looking at the following from formatter.rb: >col1width = (not widths or widths.length == 0) ? @max_width / 4 > : @max_width * widths[0] / 100

[jira] [Created] (HBASE-8663) a HBase Shell command to list the tables replicated (from or to) current cluster

Demai Ni created HBASE-8663: --- Summary: a HBase Shell command to list the tables replicated (from or to) current cluster Key: HBASE-8663 URL: https://issues.apache.org/jira/browse/HBASE-8663 Project: HBase

Re: [jira] [Resolved] (HBASE-8591) Doc Improvement: Replication blog

Himanshu, Thanks a lot. Your suggestion saved me a lot of time if have to figure it out myself Demai on the run On May 22, 2013, at 11:00 AM, Himanshu Vashishtha wrote: > Since the source metrics would not be available in case the master cluster > is down, > your best bet would be "sink.ageO

[jira] [Created] (HBASE-8566) Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsisten

Demai Ni created HBASE-8566: --- Summary: Master-Slave replication: truncate action isn't sent over to slave cluster and cause data inconsisten Key: HBASE-8566 URL: https://issues.apache.org/jira/browse/HBASE