CFP for Dataworks Summit, San Jose, 2018

[Devaraj Das]

All, Dataworks Summit San Jose 2018 is June 17-21. The call for  abstracts is 
open through February 9th. Tracks like Datawarehousing and Operational Data 
Store might be a good fit for HBase & Phoenix talks. You can submit an abstract 
at https://dataworkssummit.com/san-jose-2018/​​
Thanks,
Devaraj.

Re: [ANNOUNCE] Chunhui Shen joins the Apache HBase PMC

[Devaraj Das]

Congratulations, Chunhui!

From: Yu Li 
Sent: Monday, July 03, 2017 10:24 PM
To: dev@hbase.apache.org; Hbase-User
Subject: [ANNOUNCE] Chunhui Shen joins the Apache HBase PMC

On behalf of the Apache HBase PMC I am pleased to announce that Chunhui Shen
has accepted our invitation to become a PMC member on the Apache
HBase project. He has been an active contributor to HBase for past many
years. Looking forward for many more contributions from him.

Please join me in welcoming Chunhui to the HBase PMC!

Best Regards,
Yu

Re: [ANNOUNCE] Devaraj Das joins the Apache HBase PMC

[Devaraj Das]

Thanks, everyone!! 

From: Phil Yang <ud1...@gmail.com>
Sent: Thursday, July 06, 2017 1:45 AM
To: HBase Dev List
Subject: Re: [ANNOUNCE] Devaraj Das joins the Apache HBase PMC

Congratulations!

Thanks,
Phil


2017-07-06 15:39 GMT+08:00 Chia-Ping Tsai <chia7...@apache.org>:

> Congratulations!!!
>
> On 2017-07-06 00:27 (+0800), Josh Elser <els...@apache.org> wrote:
> > I'm pleased to announce yet another PMC addition in the form of Devaraj
> > Das. One of the "old guard" in the broader Hadoop umbrella, he's also a
> > long-standing member in our community. We all look forward to the
> > continued contributions and project leadership.
> >
> > Please join me in welcoming Devaraj!
> >
> > - Josh (on behalf of the PMC)
> >
>

Re: Problem with IntegrationTestRegionReplicaReplication

[Devaraj Das]

If it is failing consistently I'd suspect we have introduced a bug in the 1.2 
line or something. We do run the same test with a version based on 1.1.2 
(HDP-2.3 and beyond) and it works fine




On Sun, Jun 18, 2017 at 8:26 AM -0700, "Peter Somogyi" 
<psomo...@cloudera.com<mailto:psomo...@cloudera.com>> wrote:


I'm using hbase based on 1.2 version.

On Sat, Jun 17, 2017 at 4:00 PM, Devaraj Das  wrote:

> Peter which version of HBase are tou testing with?
>
>
>
>
> On Thu, Jun 15, 2017 at 11:57 PM -0700, "Peter Somogyi" <
> psomo...@cloudera.com> wrote:
>
>
> I tried with those parameters but the test still failed.
> I noticed that some of the rows were not replicated to the replicas just
> after I called flush manually. I think memstore replication is not working
> on my system even though it is enabled in the configuration.
> I will look into it today.
>
> On Fri, Jun 16, 2017 at 7:09 AM, Devaraj Das  wrote:
>
> > Peter, do have a look at IntegrationTestRegionReplicaReplication.java ..
> > At the top of the file, the ways to specify the options are documented ..
> > You need to add something like -DIntegrationTestRegionReplicaR
> eplication.read_delay_ms
> > ..
> > 
> > From: Josh Elser
> > Sent: Thursday, June 15, 2017 10:40 AM
> > To: dev@hbase.apache.org
> > Subject: Re: Problem with IntegrationTestRegionReplicaReplication
> >
> > I'd start trying a read_delay_ms=6, region_replication=2,
> > num_keys_per_server=5000, num_regions_per_server=5 with a maybe 10's of
> > reader and writer threads.
> >
> > Again, this can be quite dependent on the kind of hardware you have.
> > You'll definitely have to tweak ;)
> >
> > On 6/15/17 4:44 AM, Peter Somogyi wrote:
> > > Thanks Josh and Devaraj!
> > >
> > > I will try to increase the timeouts. Devaraj, could you share the
> > > parameters you used for this test which worked?
> > >
> > > On Thu, Jun 15, 2017 at 6:44 AM, Devaraj Das
> > wrote:
> > >
> > >> That sounds about right, Josh. Peter, in our internal testing we have
> > seen
> > >> this test failing and increasing timeouts (look at the test code
> > options to
> > >> do with increasing timeout) helped quite some.
> > >> 
> > >> From: Josh Elser
> > >> Sent: Wednesday, June 14, 2017 3:17 PM
> > >> To: dev@hbase.apache.org
> > >> Subject: Re: Problem with IntegrationTestRegionReplicaReplication
> > >>
> > >> On 6/14/17 3:53 AM, Peter Somogyi wrote:
> > >>> Hi,
> > >>>
> > >>> As one of my first task with HBase I started to look into
> > >>> why IntegrationTestRegionReplicaReplication fails. I would like to
> get
> > >> some
> > >>> suggestions from you.
> > >>>
> > >>> I noticed when I run the test using normal cluster or minicluster I
> get
> > >> the
> > >>> same error messages: "Error checking data for key [null], no data
> > >>> returned". I looked into the code and here are my conclusions.
> > >>>
> > >>> There are multiple threads writing data parallel which are read by
> > >> multiple
> > >>> reader threads simultaneously. Each writer gets a portion of the keys
> > to
> > >>> write (e.g. 0-2000) and these keys are added to a ConstantDelayQueue.
> > >>> The reader threads get the elements (e.g. key=1000) from the queue
> and
> > >>> these reader threads assume that all the keys up to this are already
> in
> > >> the
> > >>> database. Since we're using multiple writers it can happen that
> another
> > >>> thread has not yet written key=500 and verifying these keys will
> cause
> > >> the
> > >>> test failure.
> > >>>
> > >>> Do you think my assumption is correct?
> > >>
> > >> Hi Peter,
> > >>
> > >> No, as my memory serves, this is not correct. Readers are not made
> aware
> > >> of keys to verify until the write occur plus some delay. The delay is
> > >> used to provide enough time for the internal region replication to
> take
> > >> effect.
> > >>
> > >> So: primary-write, pause, [region replication happens in background],
> > >> add updated key to read queue, reader gets key from queue verifies the
> > >> value on a replica.
> > >>
> > >> The primary should always have seen the new value for a key. If the
> test
> > >> is showing that a replica does not see the result, it's either a
> timing
> > >> issue (you need to give a larger delay for HBase to perform the region
> > >> replication) or a bug in the region replication framework itself. That
> > >> said, if you can show that you are seeing what you describe, that
> sounds
> > >> like the test framework itself is broken :)
> > >>
> > >>
> > >>
> > >>
> > >
> >
> >
> >
>
>

Re: Problem with IntegrationTestRegionReplicaReplication

[Devaraj Das]

Peter which version of HBase are tou testing with?




On Thu, Jun 15, 2017 at 11:57 PM -0700, "Peter Somogyi" 
<psomo...@cloudera.com<mailto:psomo...@cloudera.com>> wrote:


I tried with those parameters but the test still failed.
I noticed that some of the rows were not replicated to the replicas just
after I called flush manually. I think memstore replication is not working
on my system even though it is enabled in the configuration.
I will look into it today.

On Fri, Jun 16, 2017 at 7:09 AM, Devaraj Das  wrote:

> Peter, do have a look at IntegrationTestRegionReplicaReplication.java ..
> At the top of the file, the ways to specify the options are documented ..
> You need to add something like 
> -DIntegrationTestRegionReplicaReplication.read_delay_ms
> ..
> 
> From: Josh Elser
> Sent: Thursday, June 15, 2017 10:40 AM
> To: dev@hbase.apache.org
> Subject: Re: Problem with IntegrationTestRegionReplicaReplication
>
> I'd start trying a read_delay_ms=6, region_replication=2,
> num_keys_per_server=5000, num_regions_per_server=5 with a maybe 10's of
> reader and writer threads.
>
> Again, this can be quite dependent on the kind of hardware you have.
> You'll definitely have to tweak ;)
>
> On 6/15/17 4:44 AM, Peter Somogyi wrote:
> > Thanks Josh and Devaraj!
> >
> > I will try to increase the timeouts. Devaraj, could you share the
> > parameters you used for this test which worked?
> >
> > On Thu, Jun 15, 2017 at 6:44 AM, Devaraj Das
> wrote:
> >
> >> That sounds about right, Josh. Peter, in our internal testing we have
> seen
> >> this test failing and increasing timeouts (look at the test code
> options to
> >> do with increasing timeout) helped quite some.
> >> 
> >> From: Josh Elser
> >> Sent: Wednesday, June 14, 2017 3:17 PM
> >> To: dev@hbase.apache.org
> >> Subject: Re: Problem with IntegrationTestRegionReplicaReplication
> >>
> >> On 6/14/17 3:53 AM, Peter Somogyi wrote:
> >>> Hi,
> >>>
> >>> As one of my first task with HBase I started to look into
> >>> why IntegrationTestRegionReplicaReplication fails. I would like to get
> >> some
> >>> suggestions from you.
> >>>
> >>> I noticed when I run the test using normal cluster or minicluster I get
> >> the
> >>> same error messages: "Error checking data for key [null], no data
> >>> returned". I looked into the code and here are my conclusions.
> >>>
> >>> There are multiple threads writing data parallel which are read by
> >> multiple
> >>> reader threads simultaneously. Each writer gets a portion of the keys
> to
> >>> write (e.g. 0-2000) and these keys are added to a ConstantDelayQueue.
> >>> The reader threads get the elements (e.g. key=1000) from the queue and
> >>> these reader threads assume that all the keys up to this are already in
> >> the
> >>> database. Since we're using multiple writers it can happen that another
> >>> thread has not yet written key=500 and verifying these keys will cause
> >> the
> >>> test failure.
> >>>
> >>> Do you think my assumption is correct?
> >>
> >> Hi Peter,
> >>
> >> No, as my memory serves, this is not correct. Readers are not made aware
> >> of keys to verify until the write occur plus some delay. The delay is
> >> used to provide enough time for the internal region replication to take
> >> effect.
> >>
> >> So: primary-write, pause, [region replication happens in background],
> >> add updated key to read queue, reader gets key from queue verifies the
> >> value on a replica.
> >>
> >> The primary should always have seen the new value for a key. If the test
> >> is showing that a replica does not see the result, it's either a timing
> >> issue (you need to give a larger delay for HBase to perform the region
> >> replication) or a bug in the region replication framework itself. That
> >> said, if you can show that you are seeing what you describe, that sounds
> >> like the test framework itself is broken :)
> >>
> >>
> >>
> >>
> >
>
>
>

Re: Problem with IntegrationTestRegionReplicaReplication

[Devaraj Das]

That sounds about right, Josh. Peter, in our internal testing we have seen this 
test failing and increasing timeouts (look at the test code options to do with 
increasing timeout) helped quite some.

From: Josh Elser 
Sent: Wednesday, June 14, 2017 3:17 PM
To: dev@hbase.apache.org
Subject: Re: Problem with IntegrationTestRegionReplicaReplication

On 6/14/17 3:53 AM, Peter Somogyi wrote:
> Hi,
>
> As one of my first task with HBase I started to look into
> why IntegrationTestRegionReplicaReplication fails. I would like to get some
> suggestions from you.
>
> I noticed when I run the test using normal cluster or minicluster I get the
> same error messages: "Error checking data for key [null], no data
> returned". I looked into the code and here are my conclusions.
>
> There are multiple threads writing data parallel which are read by multiple
> reader threads simultaneously. Each writer gets a portion of the keys to
> write (e.g. 0-2000) and these keys are added to a ConstantDelayQueue.
> The reader threads get the elements (e.g. key=1000) from the queue and
> these reader threads assume that all the keys up to this are already in the
> database. Since we're using multiple writers it can happen that another
> thread has not yet written key=500 and verifying these keys will cause the
> test failure.
>
> Do you think my assumption is correct?

Hi Peter,

No, as my memory serves, this is not correct. Readers are not made aware
of keys to verify until the write occur plus some delay. The delay is
used to provide enough time for the internal region replication to take
effect.

So: primary-write, pause, [region replication happens in background],
add updated key to read queue, reader gets key from queue verifies the
value on a replica.

The primary should always have seen the new value for a key. If the test
is showing that a replica does not see the result, it's either a timing
issue (you need to give a larger delay for HBase to perform the region
replication) or a bug in the region replication framework itself. That
said, if you can show that you are seeing what you describe, that sounds
like the test framework itself is broken :)

[jira] [Created] (HBASE-18214) Replace the folly::AtomicHashMap usage in the RPC layer

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-18214:
---

 Summary: Replace the folly::AtomicHashMap usage in the RPC layer
 Key: HBASE-18214
 URL: https://issues.apache.org/jira/browse/HBASE-18214
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das


In my tests, I saw that folly::AtomicHashMap usage is not appropriate for one, 
rather common use case. It'd become sort of unusable (inserts would hang) after 
a bunch of inserts and erases. This hashmap is used to keep track of call-Id 
after a connection is set up in the RPC layer (insert a call-id/msg pair when 
an RPC is sent, and erase the pair when the corresponding response is 
received). Here is a simple program that will demonstrate the issue:
{code}
folly::AtomicHashMap<int, int> f(100);
int i = 0;
while (i < 1) {
try {
  f.insert(i,100);
  LOG(INFO) << "Inserted " << i << "  " << f.size();
  f.erase(i);
  LOG(INFO) << "Deleted " << i << "  " << f.size();
  i++;
} catch (const std::exception ) {
  LOG(INFO) << "Exception " << e.what();
  break;
}
}
{code}
After poking around a little bit, it is indeed called out as a limitation here 
https://github.com/facebook/folly/blob/master/folly/docs/AtomicHashMap.md (grep 
for 'erase'). Proposal is to replace this with something that will fit in in 
the above usecase (thinking of using std::unordered_map).



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: CFP for Dataworks Summit Sydney

[Devaraj Das]

I meant HBase. Oops :)

From: Devaraj Das
Sent: Thursday, May 04, 2017 12:51 PM
To: u...@hbase.apache.org; dev@hbase.apache.org
Subject: CFP for Dataworks Summit Sydney

The Australia/Pacific version of Dataworks Summit is in Sydney this year, 
September 20-21.   This is a great place to talk about work you are doing in 
Apache Knox or how you are using Knox.  Information on submitting an abstract 
is at https://dataworkssummit.com/sydney-2017/abstracts/submit-abstract/

Tracks:
Apache Hadoop
Apache Spark and Data Science
Cloud and Applications
Data Processing and Warehousing
Enterprise Adoption
IoT and Streaming
Operations, Governance and Security

Deadline: Friday, May 26th, 2017.

Devaraj

CFP for Dataworks Summit Sydney

[Devaraj Das]

The Australia/Pacific version of Dataworks Summit is in Sydney this year, 
September 20-21.   This is a great place to talk about work you are doing in 
Apache Knox or how you are using Knox.  Information on submitting an abstract 
is at https://dataworkssummit.com/sydney-2017/abstracts/submit-abstract/

Tracks:
Apache Hadoop
Apache Spark and Data Science
Cloud and Applications
Data Processing and Warehousing
Enterprise Adoption
IoT and Streaming
Operations, Governance and Security

Deadline: Friday, May 26th, 2017.

Devaraj

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Devaraj Das]

Vlad, on the first point, I think what Stack is saying is that creating the new 
branch (as Ted did) ignores the feedback incorporated thus far in the 
iterations of the mega-patch. That's a wrong way to go.
On the separation into a backup module, again, that was reverted to ease 
reviews of the mega-patch, and was noted as work to be done later. I think 
Stack just wants to make the list of remaining work more complete by citing 
that as pending work.

From: Vladimir Rodionov 
Sent: Monday, March 13, 2017 3:09 PM
To: dev@hbase.apache.org
Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

>>  It ignores the feedback

If I "ignore" feedback, I put my comment - why? I am always  open for
further discussions. If reviewer does not insist on a particular request -
it will be dropped. I think it is fair.

>> he list is incomplete because a bunch of
>> follow-ons came of the review cycle (including moving backup/restore out
of
>> core to live in its own module).

For those who were not following our lengthy conversation on a review
board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack  wrote:

> On Fri, Mar 10, 2017 at 9:09 PM, Stack  wrote:
>
> > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu  wrote:
> >
> >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> >>
> >
> > The patch put up for VOTE here was done on a branch. The call to merge
> > seems to have been premature given the many cycles of review and test
> that
> > happened subsequent (The cycles burned a bunch of dev resource).
> >
> > The patch as is is now in a state where it is too big for our infra; rb
> > and JIRA are creaking under the size and # of iterations.
> >
> > Adding finish of new JIRAs to this merge implies a new round of review
> and
> > test of an already massive patch. Who is going to do this work?
> >
> > Going back to a new branch seems wrong route to take.
> >
> > St.Ack
> >
> >
> >
> To be more explicit, this patch was developed on a branch and then a bunch
> of dev resources were burned getting it into a state where it could be
> merged to master. Going back to a branch to bulk up the merge so it
> includes more JIRAs than the many it already incorporates is the wrong
> direction for us to be headed in. It ignores the feedback given and the
> work done by Vladimir slimming down an already over-broad scope. It is also
> predicated on abundant review and testing resource being on tap to cycle on
> a feature that is useful, but non-core.
>
> The patch is ready for merge IMO. Geoffrey makes a nice list of what is
> still to do though IIRC, the list is incomplete because a bunch of
> follow-ons came of the review cycle (including moving backup/restore out of
> core to live in its own module).
>
> The patch needs three votes to merge. I am not against merge but I am not
> voting for the patch because I do have any more time to spend on this
> non-core feature and feel that a vote will have me assume a responsibility
> I will not shirk.
>
> S
>
>
>
> >
> >
> >
> >> FYI
> >>
> >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu  wrote:
> >>
> >> > Thanks for the feedback, Andrew.
> >> >
> >> > How about the following plan:
> >> >
> >> > create branch HBASE-14123 off of master with mega patch v61 as the
> first
> >> > commit (reviewed by Stack and Enis)
> >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123
> >> >  branch
> >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we
> >> > propose to community for merging into branch-2 (master branch, if
> >> branch-2
> >> > doesn't materialize for whatever reason) again
> >> >
> >> > Cheers
> >> >
> >> >
> >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell 
> >> > wrote:
> >> >
> >> >> Thanks for the offer but I like that you were honest about compiling
> a
> >> >> list
> >> >> of issues that you thought were blockers for release. Since this
> >> proposal
> >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on
> this
> >> >> merge until those blockers are addressed.
> >> >>
> >> >> I had a look at the list.
> >> >>
> >> >> I think the documentation issue is important but not actually a
> >> blocker.
> >> >> That may be a controversial opinion, but documentation can be
> >> back-filled
> >> >> worst case. So take HBASE-17133 off the list.
> >> >>
> >> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227.
> >> They
> >> >> all have patches attached to the respective JIRAs so completing this
> >> work
> >> >> won't be onerous. Get these committed and I will lift my -1. The
> others
> >> >> who
> >> >> voted +1 on this thread surely can help with that.
> >> >>
> >> >> Thanks.
> >> >>
> >> >>
> 

DataWorks Summit/Hadoop Summit - Call for abstracts

[Devaraj Das]

?Just a quick note on the DataWorks Summit / Hadoop Summit Should have sent 
this across earlier but anyway..

The DataWorks Summit/Hadoop Summit Organizing Committee invites you to submit 
an abstract to be considered for the summit in 
San Jose on June 13-15.

We are looking for abstracts for the following 
tracks:
Business Focus
· Enterprise Adoption
· Applications

Technical Focus
· Data Processing & Warehousing
· Apache Hadoop
· Governance & Security
· IoT and Streaming
· Cloud & Operations
· Apache Spark & Data Science
·
To learn more about the process and submit your abstract, please click 
here.

The submission deadline is Friday, Feb 10, 2017.
Don't miss this opportunity - submit your abstract now!
SUBMIT YOUR ABSTRACT


?

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das]

Hi Stack, hats off to you for spending so much time on this! Thanks! From my 
understanding, Vlad has raised follow-up jiras for the issues you raised, 
and/or he answered most of the review feedback. So, do you think we could do a 
merge vote now?
Devaraj.

From: Vladimir Rodionov <vladrodio...@gmail.com>
Sent: Monday, November 21, 2016 8:34 PM
To: dev@hbase.apache.org
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

>> I have spent a good bit of time reviewing and testing this feature. I
would
>> like my review and concerns addressed and I'd like it to be clear how;
>> either explicit follow-on issues, pointers to where in the patch or doc
my
>> remarks have been catered to, etc. Until then, I am against commit.

Stack, mega patch review comments will be addressed in the dedicated JIRA:
HBASE-16940
I have open several other JIRAs to address your other comments (not on
review board).

Details are here (end of the thread):
https://issues.apache.org/jira/browse/HBASE-14123

Let me know what else should we do to move merge forward.

-Vlad


On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> wrote:

> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > Thanks, Matteo.
> >
> > bq. restore is not clear if given an incremental id it will do the full
> > restore from full up to that point or if i need to apply manually
> > everything
> >
> > The restore takes into consideration of the dependent backup(s).
> > So there is no need to apply preceding backup(s) manually.
> >
> >
> I ask this question on the issue. It is not clear from the usage or doc how
> to run a restore from incremental. Can you fix in doc and usage how so I
> can be clear and try it. Currently I am stuck verifying a round trip backup
> restore made of incrementals.
>
> Thanks,
> S
>
>
>
> > On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi <
> theo.berto...@gmail.com>
> > wrote:
> >
> > > I did one last pass to the mega patch. I don't see anything major that
> > > should block the merge.
> > >
> > > - most of the code is isolated in the backup package
> > > - all the backup code is client side
> > > - there are few changes to the server side, mainly for cleaners, wal
> > > rolling and similar (which is ok)
> > > - there is a good number of tests, and an integration test
> > >
> > > the code seems to have still some left overs from the old
> implementation,
> > > and some stuff needs a cleanup. but I don't think this should be used
> as
> > an
> > > argument to block the merge. I think the guys will keep working on this
> > and
> > > they may also get help of others once the patch is in master.
> > >
> > > I still have my concerns about the current limitations, but these are
> > > things already planned for phase 3, so some of this stuff may even be
> in
> > > the final 2.0.
> > > but as long as we have a "current limitations" section in the user
> guide
> > > mentioning important stuff like the ones below, I'm ok with it.
> > >  - if you write to the table with Durability.SKIP_WALS your data will
> not
> > > be in the incremental-backup
> > >  - if you bulkload files that data will not be in the incremental
> backup
> > > (HBASE-14417)
> > >  - the incremental backup will not only contains the data of the table
> > you
> > > specified but also the regions from other tables that are on the same
> set
> > > of RSs (HBASE-14141) ...maybe a note about security around this topic
> > >  - the incremental backup will not contains just the "latest row"
> between
> > > backup A and B, but it will also contains all the updates occurred in
> > > between. but the restore does not allow you to restore up to a certain
> > > point in time, the restore will always be up to the "latest backup
> > point".
> > >  - you should limit the number of "incremental" up to N (or maybe
> SIZE),
> > to
> > > avoid replay time becoming the bottleneck. (HBASE-14135)
> > >
> > > I'll be ok even with the above not being in the final 2.0,
> > > but i'd like to see as blocker for the final 2.0 (not the merge)
> > >  - the backup code moved in an hbase-backup module
> > >  - and some more work around tools, especially to try to unify and make
> > > simple the backup experience (simple example: in some case there is a
> > > backup_id argument in others a backupId argument. or things like..
> > restore
> > > is not clear if given an incremental id it will do the full restore
> from
> > > full up to that point or if i need to apply manually everything).
> > >
> > > in conclusion, I think we can open a merge vote. I'll be +1 on it, and
> I
> > > think we should try to reject -1 with just a "code cleanup" motivation,
> > > since there will still be work going on on the code after the merge.
> > >
> > > Matteo
> > >
> > >
> > > On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das <d...@hortonworks.com>
> > wrote:
> > >
> > > > Stack and others, anything else on the patch? Merge to master now?
> > > >
> > >
> >
>

Re: [DISCUSS] hbase-spark module in branch-1 and branch-2

[Devaraj Das]

Hi Sean, I did a quick check with someone from the Spark team here and his 
opinion was that the hbase-spark module as it currently stands can be used by 
downstream users to do basic stuff and to try some simple things out, etc. The 
integration is improving.
I think we should get what we have in 2.0 (which is the default action anyways).
Thanks
Devaraj

From: Sean Busbey 
Sent: Wednesday, November 16, 2016 9:49 AM
To: dev
Subject: [DISCUSS] hbase-spark module in branch-1 and branch-2

Hi folks!

With 2.0 releases coming up, I'd like to revive our prior discussion
on the readiness of the hbase-spark module for downstream users.

We've had a ticket for tracking the milestones set up for inclusion in
branch-1 releases for about 1.5 years:

https://issues.apache.org/jira/browse/HBASE-14160

We still haven't gotten all of the blocker issues completed, AFAIK.

Is anyone interested in volunteering to knock the rest of these out?

If they aren't, shall we plan to leave hbase-spark in master and
revert it from branch-2 once it forks for the HBase 2.0 release line?

This feature isn't a blocker for 2.0; just as we've been planning to
add the hbase-spark module to some 1.y release we can also include it
in a 2.1+ release.

This does appear to be a feature our downstream users could benefit
from, so I'd hate to continue the current situation where no official
releases include it. This is especially true now that we're looking at
ways to handle changes between Spark 1.6 and Spark 2.0 in HBASE-16179.

-
busbey

Re: Status of HBASE-4755?

[Devaraj Das]

Lars, the work to complete this is happening over in 
https://issues.apache.org/jira/browse/HBASE-15531. Thiruvel and Francis are 
bringing to Apache what they have deployed in prod at Yahoo. I believe 
RegionPlacementMaintainer is not being run in their prod. Correct me if I am 
wrong, Thiruvel/Francis.
Thanks,
Devaraj.

From: Lars George 
Sent: Wednesday, November 09, 2016 11:19 PM
To: dev@hbase.apache.org
Subject: Status of HBASE-4755?

Hi,

I am wondering where https://issues.apache.org/jira/browse/HBASE-4755
is up to? Seeing this lasting many years now, why is it not closed? It
seems the balancer is in place.

What worries me is that the tool to recompute a missing region server
location after a failover is having a comment like this (see the TODO
below):

/**
 * A tool that is used for manipulating and viewing favored nodes information
 * for regions. Run with -h to get a list of the options
 */
@InterfaceAudience.Private
// TODO: Remove? Unused. Partially implemented only.
public class RegionPlacementMaintainer {

If everything is in place but the recomputation, which should be
automated as discussed in the JIRA years ago, what is the hold up? Why
do we have such a feature lingering away?

Or let me ask the other way around: who is actively using this
feature, and how are they doing it?

Cheers,
Lars

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das]

; > > > https://reviews.apache.org/r/52748
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > > > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov
> <
> > > >>>> >>> >> > > > vladrodio...@gmail.com
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > wrote:
> > > >>>> >>> >> > > > >
> > > >>>> >>> >> > > > > > >> How hard to put in an hbase-backup module?
> > > >>>> hbase-server
> > > >>>> >>> is
> > > >>>> >>> >> fat
> > > >>>> >>> >> > > > enough
> > > >>>> >>> >> > > > > > >> already. Could be done as a follow-up.
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > https://issues.apache.org/
> jira/browse/HBASE-16727?
> > > >>>> >>> >> > > > > > focusedCommentId=15531237&
> page=com.atlassian.jira.
> > > >>>> >>> >> > > > > > plugin.system.issuetabpanels:c
> > > >>>> >>> omment-tabpanel#comment-15531237
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > Can we do merge first? Then we can discuss
> separate
> > > >>>> module.
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > -Vlad
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <
> > > >>>> >>> yuzhih...@gmail.com>
> > > >>>> >>> >> > wrote:
> > > >>>> >>> >> > > > > >
> > > >>>> >>> >> > > > > >> Looks like the first quote was cut off.
> > > >>>> >>> >> > > > > >> The original sentence was:
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> bq. no mapreduce job launched from master or
> > region
> > > >>>> server.
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> mapreduce job is launched from the node where
> > > command
> > > >>>> line
> > > >>>> >>> >> tool is
> > > >>>> >>> >> > > > run.
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <
> > > >>>> st...@duboce.net>
> > > >>>> >>> >> wrote:
> > > >>>> >>> >> > > > > >>
> > > >>>> >>> >> > > > > >> > bq. launched from master or region server.
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > What does this mean please? Has to be run from
> > > >>>> Master or
> > > >>>> >>> >> > > > RegionServer?
> > > >>>> >>> >> > > > > >> Can
> > > >>>> >>> >> > > > > >> > it be run from another node altogether?
> > > >>>> >>> >> > > > > >> >
> > > >>>> >>> >> > > > > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir
> > > Rodionov <
> > > >>>> >>> >> > > > > >> vladrodio...@gmail.com
> > > >>>> >>> >> > > > > >> > >
> > > >>>> >>> >> > > > > >>

[jira] [Created] (HBASE-16849) Document for HBASE-14918 (in-memory flushes/compactions)

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-16849:
---

 Summary: Document for HBASE-14918 (in-memory flushes/compactions)
 Key: HBASE-16849
 URL: https://issues.apache.org/jira/browse/HBASE-16849
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Anastasia Braginsky
Priority: Blocker


As discussed here - 
https://issues.apache.org/jira/browse/HBASE-16608?focusedCommentId=15577213=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15577213
[~stack] [~anoop.hbase] [~ram_krish]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das]

 no
>>> >> > > > mapreduce
>>> >> > > > > >> job
>>> >> > > > > >> > >
>>> >> > > > > >> > > 1. We have no code in the client module anymore, due to
>>> >> > > dependency
>>> >> > > > > on
>>> >> > > > > >> > > internal server API (HFile and WAL access).
>>> >> > > > > >> > > 2. Backup/ restore are client - driven operations, but
>>> all
>>> >> the
>>> >> > > > code
>>> >> > > > > >> > resides
>>> >> > > > > >> > > in the server module
>>> >> > > > > >> > >
>>> >> > > > > >> >
>>> >> > > > > >> > How hard to put in an hbase-backup module? hbase-server
>>> is
>>> >> fat
>>> >> > > > enough
>>> >> > > > > >> > already. Could be done as a follow-up.
>>> >> > > > > >> >
>>> >> > > > > >> > Thanks,
>>> >> > > > > >> > St.Ack
>>> >> > > > > >> >
>>> >> > > > > >> >
>>> >> > > > > >> >
>>> >> > > > > >> > > 3. No MR in Master, no procedure - driven execution.
>>> >> > > > > >> > > 4. Old good MR from command-line.
>>> >> > > > > >> > > 5. Security was simplified and now only super-user is
>>> >> allowed
>>> >> > to
>>> >> > > > run
>>> >> > > > > >> > > backup/restores.
>>> >> > > > > >> > > 6. HBase Backup API was gone due to 1. Now only
>>> >> command-line
>>> >> > > > access
>>> >> > > > > to
>>> >> > > > > >> > > backup tools.
>>> >> > > > > >> > >
>>> >> > > > > >> > > These consequences of refactoring has been discussed in
>>> >> > > > HBASE-16727.
>>> >> > > > > >> > >
>>> >> > > > > >> > > -Vlad
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > >
>>> >> > > > > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <
>>> >> yuzhih...@gmail.com>
>>> >> > > > > wrote:
>>> >> > > > > >> > >
>>> >> > > > > >> > > > Reviving this thread.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > The following has taken place:
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > mapreduce dependency has been moved to client side -
>>> no
>>> >> > > > mapreduce
>>> >> > > > > >> job
>>> >> > > > > >> > > > launched from master or region server.
>>> >> > > > > >> > > > document patch (HBASE-16574) has been integrated.
>>> >> > > > > >> > > > Updated mega patch has been attached to HBASE-14123:
>>> this
>>> >> > > covers
>>> >> > > > > the
>>> >> > > > > >> > > > refactor in #1 above and the protobuf 3 merge.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > If community has more feedback on the merge
>>> proposal, I
>>> >> > would
>>> >> > > > love
>>> >> > > > > >> to
>>> >> > > > > >> > > hear
>>> >> > > > > >> > > > it.
>>> >> > > > > >> > > >
>>> >> > > > > >> > > > Thanks
>>> >> > > >

Re: Issues while running HBase native client build (Ref: HBASE-14850)

[Devaraj Das]

Enis, the last time around, the RPC wasn't based on Folly... Sudeep is trying 
to POC an implementation of an async RPC based on Folly, and is facing issues 
with the Folly library. Elliott, any chance you could help here please.

From: Enis Söztutar 
Sent: Friday, October 07, 2016 11:40 AM
To: dev@hbase.apache.org
Cc: Elliott Clark
Subject: Re: Issues while running HBase native client build (Ref: HBASE-14850)

Sudeep, do you mind creating a ticket for this?

I had tried the simple client before, and did not run into any problems. Do
you want me to test again?

Enis

On Thu, Oct 6, 2016 at 3:29 AM, Sudeep Sunthankar <
sudeep.sunthan...@gmail.com> wrote:

> Hi Team,
>
>
>
> I am facing segfaults while running the simple-client present in
> hbase-native-client.  As per the build steps, when I executed the binary
> from the docker environemnt, I get the foll error:-
>
> 2016-10-06 10:08:53,536:93(0x7f6cc256a900):ZOO_ERROR@getaddrs@613:
> getaddrinfo: No such file or directory
>
>
>
> So, I tried installing all the dependencies on my local environment and
> tried running the simple-client on it, instead of docker environment. This
> time  I got a different segfault as below.
>
> io/async/AsyncSocket.cpp:875: virtual void folly::AsyncSocket::closeNow():
> Assertion `eventBase_ == nullptr || eventBase_->isInEventBaseThread()'
> failed.
>
>
>
> Can you please help me with some pointers.
>
>
>
>
>
> --
>
> Warm Regards,
>
> Sudeep Sunthankar
>
>
>
>

Re: Backup Implementation (WAS => Re: [DISCUSSION] MR jobs started by Master or RS)

[Devaraj Das]

If tests pass with the patch (which I believe they are), let's commit the 
patch. Follow it up with an updated mega patch for review...


From: Ted Yu <yuzhih...@gmail.com>
Sent: Tuesday, October 04, 2016 6:28 PM
To: dev@hbase.apache.org
Subject: Re: Backup Implementation (WAS => Re: [DISCUSSION] MR jobs started by 
Master or RS)

Refactoring work over in HBASE-16727 is ready for review.

Kindly provide your feedback.

Thanks

On Mon, Oct 3, 2016 at 3:05 PM, Andrew Purtell <apurt...@apache.org> wrote:

> This sounds good to me.
> I'd be at least +0 as to merging the branch as long as we are not 'shelling
> out' to MR from master.
>
> > All or most of the Backup/Restore operations (especially the MR job
> spawns) should be moved to the client.
>
> We have a home grown backup solution at Salesforce that to a first order of
> approximation is this. I would like to see something like this merged.
>
> > In the future, if someone needs to support self-service operations (any
> user can take a backup/restore his/her tables), we can discuss the "backup
> service" or something else.
>
> I can't commit the time of the team here (smile), but we always strive to
> minimize the amount of local code we need to manage HBase. For example, we
> use VerifyReplication and other tools that ship with HBase, and we have
> contributed minor operational improvements as we've developed them (like
> the region mover and canary stuff). I suspect we will have some adoption of
> this tooling and further refinement insofar it fits into a backup workflow
> at 30kft view using snapshots, replication (or file shipping), and WAL
> replay.
>
>
> On Mon, Sep 26, 2016 at 9:57 PM, Devaraj Das <d...@hortonworks.com> wrote:
>
> > Vlad, thinking about it a little more, since the master is not
> > orchestrating the backup, let's make it dead simple as a first pass. I
> > think we should do the following: All or most of the Backup/Restore
> > operations (especially the MR job spawns) should be moved to the client.
> > Ignore security for the moment - let's live with what we have as the
> > current "limitation" for tools that need HDFS access - they need to run
> as
> > hbase (or whatever the hbase daemons runs as). Consistency/cleanup needs
> to
> > be handled as well as much as possible - if the client fails after
> > initiating the backup/restore, who restores consistency in the
> hbase:backup
> > table, or cleans up the half copied data in the hdfs dirs, etc.
> > In the future, if someone needs to support self-service operations (any
> > user can take a backup/restore his/her tables), we can discuss the
> "backup
> > service" or something else.
> > Folks - Stack / Andrew / Matteo / others, please speak up if you disagree
> > with the above. Would like to get over this merge-to-master hump
> obviously.
> >
> > 
> > From: Vladimir Rodionov <vladrodio...@gmail.com>
> > Sent: Monday, September 26, 2016 11:48 AM
> > To: dev@hbase.apache.org
> > Subject: Re: Backup Implementation (WAS => Re: [DISCUSSION] MR jobs
> > started by Master or RS)
> >
> > Ok, we had internal discussion and this is what we are suggesting now:
> >
> > 1. We will create separate module (hbase-backup) and move server-side
> code
> > there.
> > 2. Master and RS will be MR and backup free.
> > 3. The code from Master will be moved into standalone service
> > (BackupService) for procedure orchestration,
> >  operation resume/abort and SECURITY. It means - one additional
> > (process) similar to REST/Thrift server will be required
> > to operate backup.
> >
> > I would like to note that separate process running under hbase super user
> > is required to implement security properly in a multi-tenant environment,
> > otherwise, only hbase super user will be allowed to operate backups
> >
> > Please let us know, what do you think, HBase people :?
> >
> > -Vlad
> >
> >
> >
> > On Sat, Sep 24, 2016 at 2:49 PM, Stack <st...@duboce.net> wrote:
> >
> > > On Sat, Sep 24, 2016 at 9:58 AM, Andrew Purtell <
> > andrew.purt...@gmail.com>
> > > wrote:
> > >
> > > > At branch merge voting time now more eyes are getting on the design
> > > issues
> > > > with dissenting opinion emerging. This is the branch merge process
> > > working
> > > > as our community has designed it. Because this is the first full
> > project
> > > > review of the code and implementation I thi

Re: Backup Implementation (WAS => Re: [DISCUSSION] MR jobs started by Master or RS)

[Devaraj Das]

t;>>>>> On Fri, Sep 23, 2016 at 11:22 AM, Vladimir Rodionov <
> > >>>>>>>> vladrodio...@gmail.com>
> > >>>>>>>> wrote:
> > >>>>>>>>
> > >>>>>>>>>>> + MR is dead
> > >>>>>>>>>
> > >>>>>>>>> Does MR know that? :)
> > >>>>>>>>>
> > >>>>>>>>> Again. With all due respect, stack - still no suggestions
> > >> what
> > >>>>> should
> > >>>>>>> we
> > >>>>>>>>> use for "bulk data move and transformation" instead of MR?
> > >>>>>>>>>
> > >>>>>>>>
> > >>>>>>>> Use whatever distributed engine suits your fancy -- MR, Spark,
> > >>>>>>> distributed
> > >>>>>>>> shell -- just don't have HBase core depend on it, even
> > >>> optionally.
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> I suggest voting first on "do we need backup in HBase"? In my
> > >>>>>> opinion,
> > >>>>>>>> some
> > >>>>>>>>> group members still not sure about that and some will give -1
> > >>>>>>>>> in any case. Just because ...
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>> We could run a vote, sure. -1 on that backup be in core hbase
> > >> (+1
> > >>>> on
> > >>>>>>> adding
> > >>>>>>>> all the API any such external tool might need to run).
> > >>>>>>>>
> > >>>>>>>> St.Ack
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>
> > >>>>>>>>> -Vlad
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>>
> > >>>>>>>>> On Fri, Sep 23, 2016 at 10:57 AM, Stack <st...@duboce.net>
> > >>>> wrote:
> > >>>>>>>>>
> > >>>>>>>>>> On Fri, Sep 23, 2016 at 6:46 AM, Matteo Bertozzi <
> > >>>>>>>>> theo.berto...@gmail.com>
> > >>>>>>>>>> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>>> let me try to go back to my original topic.
> > >>>>>>>>>>> this question was meant to be generic, and provide some
> > >>> rule
> > >>>>> for
> > >>>>>>>> future
> > >>>>>>>>>>> code.
> > >>>>>>>>>>>
> > >>>>>>>>>>> from what I can gather, a rule that may satisfy everyone
> > >>> can
> > >>>>> be:
> > >>>>>>>>>>> - we don't want any core feature (e.g.
> > >>>>> compaction/log-split/log-
> > >>>>>>>>> reply)
> > >>>>>>>>>>> over MR, because some cluster may not want or may have an
> > >>>>>>>>>>> external/uncontrolled MR setup.
> > >>>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>> +1
> > >>>>>>>>>>
> > >>>>>>>>>>
> > >>>>>>>>>>> - we allow non-core features (e.g. features enabled by a
> > >>>> flag)
> > >>>>>> to
> > >>>>>>>> run
> > >>>>>>>>> MR
> > >>>>>>>>>>> jobs from hbase, because unless you use the feature, MR
> > >> is
> > >>>> not
> > >>>>>>>>> required.
> > >>>>>>>>>>>
> > >>>>>>>>>>>
> &g

Re: [DISCUSSION] MR jobs started by Master or RS

[Devaraj Das]

 > > > >
> > > > > And for security, I have a issue pending for a long time. Can
> someone
> > > > help
> > > > > taking a simple look at it? This is what I mean, ugly code...
> logout
> > > and
> > > > > destroy the credentials in a subject when it is still being used,
> and
> > > > > declared as LimitPrivacy so I can not change the behivor and the
> only
> > > way
> > > > > to fix it is to write another piece of ugly code...
> > > > >
> > > > > https://issues.apache.org/jira/browse/HADOOP-13433
> > > > >
> > > > > 2016-09-23 12:53 GMT+08:00 Vladimir Rodionov <
> vladrodio...@gmail.com
> > >:
> > > > >
> > > > > > >> If in the future, we find better ways of doing this without
> > using
> > > > MR,
> > > > > we
> > > > > > can certainly consider that
> > > > > >
> > > > > > Our framework for distributed operations is abstract and allows
> > > > > > different implementations. MR is just one implementation we
> > provide.
> > > > > >
> > > > > > -Vlad
> > > > > >
> > > > > > On Thu, Sep 22, 2016 at 9:38 PM, Devaraj Das <
> d...@hortonworks.com
> > >
> > > > > wrote:
> > > > > >
> > > > > > > Guys, first off apologies for bringing in the topic of MR-based
> > > > > > > compactions.. But I was thinking more about the SpliceMachine
> > > > approach
> > > > > of
> > > > > > > managing compactions in Spark where apparently they saw a lot
> of
> > > > > > benefits.
> > > > > > > Apologies for giving you that sore throat Andrew; I really
> didn't
> > > > mean
> > > > > to
> > > > > > > :-)
> > > > > > >
> > > > > > > So on this issue, we have these on the plate:
> > > > > > > 0. Somehow not use MR but something like that
> > > > > > > 1. Run a standalone service other than master
> > > > > > > 2. Shell out from the master
> > > > > > >
> > > > > > > I don't think we have a good answer to (0), and I don't think
> > it's
> > > > even
> > > > > > > worth the effort of trying to build something when MR is
> already
> > > > there,
> > > > > > and
> > > > > > > being used by HBase already for some operations.
> > > > > > >
> > > > > > > On (1), we have to deal with a myriad of issues - HA of the
> > server
> > > > not
> > > > > > > being the least of them all. Security (kerberos authentication,
> > > > another
> > > > > > > keytab to manage, etc. etc. etc.). IMO, that approach is DOA.
> > > Instead
> > > > > > let's
> > > > > > > substitute that (1) with the HBase Master. I haven't seen any
> > good
> > > > > reason
> > > > > > > why the HBase master shouldn't launch MR jobs if needed. It's
> not
> > > > > ideal;
> > > > > > > agreed.
> > > > > > >
> > > > > > > Now before going to (2), let's see what are the benefits of
> > running
> > > > the
> > > > > > > backup/restore jobs from the master. I think Ted has summarized
> > > some
> > > > of
> > > > > > the
> > > > > > > issues that we need to take care of - basically, the master can
> > > keep
> > > > > > track
> > > > > > > of running jobs, and should it fail, the backup master can
> > continue
> > > > > > keeping
> > > > > > > track of it (since the jobId would have been recorded in the
> proc
> > > > WAL).
> > > > > > The
> > > > > > > master can also do cleanup, etc. of failed backup/restore
> > > processes.
> > > > > > > Security is another issue - the job needs to run as 'hbase'
> since
> > > it
> > > > > owns
> > > > > > > the data. Having the master launch the job makes it get that
> > > > privilege.
> > > > > > In
> > > > > > > the (2) appr

Re: [DISCUSSION] MR jobs started by Master or RS

[Devaraj Das]

All the better, Vlad!





On Thu, Sep 22, 2016 at 9:53 PM -0700, "Vladimir Rodionov" 
<vladrodio...@gmail.com<mailto:vladrodio...@gmail.com>> wrote:

>> If in the future, we find better ways of doing this without using MR, we
can certainly consider that

Our framework for distributed operations is abstract and allows
different implementations. MR is just one implementation we provide.

-Vlad

On Thu, Sep 22, 2016 at 9:38 PM, Devaraj Das <d...@hortonworks.com> wrote:

> Guys, first off apologies for bringing in the topic of MR-based
> compactions.. But I was thinking more about the SpliceMachine approach of
> managing compactions in Spark where apparently they saw a lot of benefits.
> Apologies for giving you that sore throat Andrew; I really didn't mean to
> :-)
>
> So on this issue, we have these on the plate:
> 0. Somehow not use MR but something like that
> 1. Run a standalone service other than master
> 2. Shell out from the master
>
> I don't think we have a good answer to (0), and I don't think it's even
> worth the effort of trying to build something when MR is already there, and
> being used by HBase already for some operations.
>
> On (1), we have to deal with a myriad of issues - HA of the server not
> being the least of them all. Security (kerberos authentication, another
> keytab to manage, etc. etc. etc.). IMO, that approach is DOA. Instead let's
> substitute that (1) with the HBase Master. I haven't seen any good reason
> why the HBase master shouldn't launch MR jobs if needed. It's not ideal;
> agreed.
>
> Now before going to (2), let's see what are the benefits of running the
> backup/restore jobs from the master. I think Ted has summarized some of the
> issues that we need to take care of - basically, the master can keep track
> of running jobs, and should it fail, the backup master can continue keeping
> track of it (since the jobId would have been recorded in the proc WAL). The
> master can also do cleanup, etc. of failed backup/restore processes.
> Security is another issue - the job needs to run as 'hbase' since it owns
> the data. Having the master launch the job makes it get that privilege. In
> the (2) approach, it's hard to do some of the above management.
>
> Guys, just to reiterate, the patch as such is ready from the overall
> design/arch point of view (maybe code review is still pending from Matteo).
> If in the future, we find better ways of doing this without using MR, we
> can certainly consider that. But IMO don't think we should block this patch
> from getting merged.
>
> 
> From: 张铎 <palomino...@gmail.com>
> Sent: Thursday, September 22, 2016 8:32 PM
> To: dev@hbase.apache.org
> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
>
> So what about a standalone service other than master? You can use your own
> procedure store in that service?
>
> 2016-09-23 11:28 GMT+08:00 Ted Yu <yuzhih...@gmail.com>:
>
> > An earlier implementation was client driven.
> >
> > But with that approach, it is hard to resume if there is error midway.
> > Using Procedure V2 makes the backup / restore more robust.
> >
> > Another consideration is for security. It is hard to enforce security (to
> > be implemented) for client driven actions.
> >
> > Cheers
> >
> > > On Sep 22, 2016, at 8:15 PM, Andrew Purtell <andrew.purt...@gmail.com>
> > wrote:
> > >
> > > No, this misses Matteo's finer point, which is "shelling out" from the
> > master directly to run MR is a first. Why not drive this with a utility
> > derived from Tool?
> > >
> > > On Sep 22, 2016, at 7:57 PM, Vladimir Rodionov <vladrodio...@gmail.com
> >
> > wrote:
> > >
> > >>>> In our production cluster,  it is a common case we just have HDFS
> and
> > >>>> HBase deployed.
> > >>>> If our Master/RS depend on MR framework (especially some features we
> > >>>> have not used at all),  it introduced another cost for maintain.  I
> > >>>> don't think it is a good idea.
> > >>
> > >> So , you are not backup users in this case. Many our customers have
> full
> > >> stack deployed and
> > >> want see backup to be a standard feature. Besides this, nothing will
> > happen
> > >> in your cluster
> > >> if you won't be doing backups.
> > >>
> > >> This discussion (we do not want see M/R dependency) goes to nowhere.
> We
> > >> asked already, at least twice, to suggest another framework (other
> than
> > M/R)
> > >> for

Re: [DISCUSSION] MR jobs started by Master or RS

[Devaraj Das]

 framework (especially some features we
> >>>> have not used at all),  it introduced another cost for maintain.  I
> >>>> don't think it is a good idea.
> >>>>
> >>>> 2016-09-23 10:28 GMT+08:00 张铎 <palomino...@gmail.com>:
> >>>>> To be specific, for example, our nice Backup/Restore feature, if we
> >>> think
> >>>>> this is not a core feature of HBase, then we could make it depend on
> >>> MR,
> >>>>> and start a standalone BackupManager instance that submits MR jobs to
> >>> do
> >>>>> periodical maintenance job. And if we think this is a core feature
> that
> >>>>> everyone should use it, then we'd better implement it without MR
> >>>>> dependency, like DLS.
> >>>>>
> >>>>> Thanks.
> >>>>>
> >>>>> 2016-09-23 10:11 GMT+08:00 张铎 <palomino...@gmail.com>:
> >>>>>
> >>>>>> I‘m -1 on let master or rs launch MR jobs. It is OK that some of our
> >>>>>> features depend on MR but I think the bottom line is that we should
> >>>> launch
> >>>>>> the jobs from outside manually or by other services.
> >>>>>>
> >>>>>> 2016-09-23 9:47 GMT+08:00 Andrew Purtell <andrew.purt...@gmail.com
> >:
> >>>>>>
> >>>>>>> Ok, got it. Well "shelling out" is on the line I think, so a fair
> >>>>>>> question.
> >>>>>>>
> >>>>>>> Can this be driven by a utility derived from Tool like our other MR
> >>>> apps?
> >>>>>>> The issue is needing the AccessController to decide if allowed? But
> >>>> nothing
> >>>>>>> prevents the user from running the job manually/independently,
> right?
> >>>>>>>
> >>>>>>>> On Sep 22, 2016, at 3:44 PM, Matteo Bertozzi <
> >>>> theo.berto...@gmail.com>
> >>>>>>> wrote:
> >>>>>>>>
> >>>>>>>> just a remark. my query was not about tools using MR (everyone i
> >>>> think
> >>>>>>> is
> >>>>>>>> ok with those).
> >>>>>>>> the topic was about: "are we ok with running MR jobs from Master
> >>> and
> >>>> RSs
> >>>>>>>> code?" since this will be the first time we do this
> >>>>>>>>
> >>>>>>>> Matteo
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>> On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <
> >>> d...@hortonworks.com>
> >>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>> Very much agree; for tools like ExportSnapshot / Backup /
> Restore,
> >>>> it's
> >>>>>>>>> fine to be dependent on MR. MR is the right framework for such.
> We
> >>>>>>> should
> >>>>>>>>> also do compactions using MR (just saying :) )
> >>>>>>>>> 
> >>>>>>>>> From: Ted Yu <yuzhih...@gmail.com>
> >>>>>>>>> Sent: Thursday, September 22, 2016 2:00 PM
> >>>>>>>>> To: dev@hbase.apache.org
> >>>>>>>>> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
> >>>>>>>>>
> >>>>>>>>> I agree - backup / restore is in the same category as import /
> >>>> export.
> >>>>>>>>>
> >>>>>>>>> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <
> >>>>>>> andrew.purt...@gmail.com>
> >>>>>>>>> wrote:
> >>>>>>>>>
> >>>>>>>>>> Backup is extra tooling around core in my opinion. Like import
> or
> >>>>>>> export.
> >>>>>>>>>> Or the optional MOB tool. It's fine.
> >>>>>>>>>>
> >>>>>>>>>>> On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <
> >>>> mberto...@apache.org>
> >>>>>>>>>> wrote:
> >>>>>>>>>>>
> >>>>>>>>>>> What's the latest opinion around running MR jobs from hbase
> >>>> (Master
> >>>>>>> or
> >>>>>>>>>> RS)?
> >>>>>>>>>>>
> >>>>>>>>>>> I remember in the past that there was discussion about not
> >>> having
> >>>> MR
> >>>>>>>>> has
> >>>>>>>>>>> direct dependency of hbase.
> >>>>>>>>>>>
> >>>>>>>>>>> I think some of discussion where around MOB that had a MR job
> to
> >>>>>>>>> compact,
> >>>>>>>>>>> that later was transformed in a non-MR job to be merged, I
> think
> >>>> we
> >>>>>>>>> had a
> >>>>>>>>>>> similar discussion for log split/replay.
> >>>>>>>>>>>
> >>>>>>>>>>> the latest is the new Backup feature (HBASE-7912), that runs a
> >>> MR
> >>>> job
> >>>>>>>>>> from
> >>>>>>>>>>> the master to copy data or restore data.
> >>>>>>>>>>> (backup is also "not really core" as in.. if you don't use
> >>> backup
> >>>>>>>>> you'll
> >>>>>>>>>>> not end up running MR jobs, but this was probably true for MOB
> >>> as
> >>>> in
> >>>>>>>>> "if
> >>>>>>>>>>> you don't enable MOB you don't need MR")
> >>>>>>>>>>>
> >>>>>>>>>>> any thoughts? do we a rule that says "we don't want to have
> >>> hbase
> >>>> run
> >>>>>>>>> MR
> >>>>>>>>>>> jobs, only tool started manually by the user can do that". or
> >>> can
> >>>> we
> >>>>>>>>>> start
> >>>>>>>>>>> adding MR calls around without problems?
> >>>
>

Re: [DISCUSSION] MR jobs started by Master or RS

[Devaraj Das]

Not practical to do those tools without MR, JM. We should be using the right 
framework for the use cases in hand. MR fits this really well. 
JM, when you say "if we can do without MR, then, why not?", do you have a 
framework in mind that performs/scale as well as MR? Curious.

From: Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Sent: Thursday, September 22, 2016 4:29 PM
To: dev
Subject: Re: [DISCUSSION] MR jobs started by Master or RS

Well, I'm just not using those features ;) But was hopping for the MOBs ;)
My point is, if we can do it without MR, then, why not? )

2016-09-22 19:25 GMT-04:00 Vladimir Rodionov <vladrodio...@gmail.com>:

> Forgot WALPlayer :)
>
> -Vlad
>
> On Thu, Sep 22, 2016 at 4:21 PM, Vladimir Rodionov <vladrodio...@gmail.com
> >
> wrote:
>
> > >> and
> > >> backups too, but don't want to bother having to install and configure
> > YARN
> > >> just for that, as well as removing resources from HBase to give it to
> >
> > Any suggestions on how to do bulk data move with transformation from/to
> > HBase cluster w/o MapReduce?
> >
> > Opposition to M/R does not make sense imo, as since we have a lot of
> tools
> > in HBase which depend on MapReduce:
> >
> > CountRows
> > CountCells
> > Import
> > Export
> > ImportTsv
> > ExportTsv
> > CopyTable
> > VerifyReplication
> > ExportSnapshot
> >
> > and new backup create/restore of course.
> >
> >
> > -Vlad
> >
> >
> >
> >
> > On Thu, Sep 22, 2016 at 4:15 PM, Jean-Marc Spaggiari <
> > jean-m...@spaggiari.org> wrote:
> >
> >> My 2¢: I have a strong preference for NOT having a dependency on MR
> >> anywhere :( I run my HBase cluste without YARN. Just HBase and HDFS. I
> >> like
> >> all the features that we built. Would love to be able to use MOBs and
> >> backups too, but don't want to bother having to install and configure
> YARN
> >> just for that, as well as removing resources from HBase to give it to
> >> yarn
> >>
> >> JMS
> >>
> >> 2016-09-22 18:44 GMT-04:00 Matteo Bertozzi <theo.berto...@gmail.com>:
> >>
> >> > just a remark. my query was not about tools using MR (everyone i think
> >> is
> >> > ok with those).
> >> > the topic was about: "are we ok with running MR jobs from Master and
> RSs
> >> > code?" since this will be the first time we do this
> >> >
> >> > Matteo
> >> >
> >> >
> >> > On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <d...@hortonworks.com>
> >> wrote:
> >> >
> >> > > Very much agree; for tools like ExportSnapshot / Backup / Restore,
> >> it's
> >> > > fine to be dependent on MR. MR is the right framework for such. We
> >> should
> >> > > also do compactions using MR (just saying :) )
> >> > > 
> >> > > From: Ted Yu <yuzhih...@gmail.com>
> >> > > Sent: Thursday, September 22, 2016 2:00 PM
> >> > > To: dev@hbase.apache.org
> >> > > Subject: Re: [DISCUSSION] MR jobs started by Master or RS
> >> > >
> >> > > I agree - backup / restore is in the same category as import /
> export.
> >> > >
> >> > > On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <
> >> > andrew.purt...@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Backup is extra tooling around core in my opinion. Like import or
> >> > export.
> >> > > > Or the optional MOB tool. It's fine.
> >> > > >
> >> > > > > On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <
> >> mberto...@apache.org>
> >> > > > wrote:
> >> > > > >
> >> > > > > What's the latest opinion around running MR jobs from hbase
> >> (Master
> >> > or
> >> > > > RS)?
> >> > > > >
> >> > > > > I remember in the past that there was discussion about not
> having
> >> MR
> >> > > has
> >> > > > > direct dependency of hbase.
> >> > > > >
> >> > > > > I think some of discussion where around MOB that had a MR job to
> >> > > compact,
> >> > > > > that later was transformed in a non-MR job to be merged, I think
> >> we
> >> > > had a
> >> > > > > similar discussion for log split/replay.
> >> > > > >
> >> > > > > the latest is the new Backup feature (HBASE-7912), that runs a
> MR
> >> job
> >> > > > from
> >> > > > > the master to copy data or restore data.
> >> > > > > (backup is also "not really core" as in.. if you don't use
> backup
> >> > > you'll
> >> > > > > not end up running MR jobs, but this was probably true for MOB
> as
> >> in
> >> > > "if
> >> > > > > you don't enable MOB you don't need MR")
> >> > > > >
> >> > > > > any thoughts? do we a rule that says "we don't want to have
> hbase
> >> run
> >> > > MR
> >> > > > > jobs, only tool started manually by the user can do that". or
> can
> >> we
> >> > > > start
> >> > > > > adding MR calls around without problems?
> >> > > >
> >> > >
> >> >
> >>
> >
> >
>

Re: [DISCUSSION] MR jobs started by Master or RS

[Devaraj Das]

Matteo, the Master won't spawn the job unless someone actually wants to use the 
backup/restore. So I'd argue we still don't have a 'hard' dependency - it's 
still much like the other tools that you consider as being outside the core.

From: Matteo Bertozzi <theo.berto...@gmail.com>
Sent: Thursday, September 22, 2016 3:44 PM
To: dev@hbase.apache.org
Subject: Re: [DISCUSSION] MR jobs started by Master or RS

just a remark. my query was not about tools using MR (everyone i think is
ok with those).
the topic was about: "are we ok with running MR jobs from Master and RSs
code?" since this will be the first time we do this

Matteo


On Thu, Sep 22, 2016 at 2:49 PM, Devaraj Das <d...@hortonworks.com> wrote:

> Very much agree; for tools like ExportSnapshot / Backup / Restore, it's
> fine to be dependent on MR. MR is the right framework for such. We should
> also do compactions using MR (just saying :) )
> 
> From: Ted Yu <yuzhih...@gmail.com>
> Sent: Thursday, September 22, 2016 2:00 PM
> To: dev@hbase.apache.org
> Subject: Re: [DISCUSSION] MR jobs started by Master or RS
>
> I agree - backup / restore is in the same category as import / export.
>
> On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell <andrew.purt...@gmail.com>
> wrote:
>
> > Backup is extra tooling around core in my opinion. Like import or export.
> > Or the optional MOB tool. It's fine.
> >
> > > On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi <mberto...@apache.org>
> > wrote:
> > >
> > > What's the latest opinion around running MR jobs from hbase (Master or
> > RS)?
> > >
> > > I remember in the past that there was discussion about not having MR
> has
> > > direct dependency of hbase.
> > >
> > > I think some of discussion where around MOB that had a MR job to
> compact,
> > > that later was transformed in a non-MR job to be merged, I think we
> had a
> > > similar discussion for log split/replay.
> > >
> > > the latest is the new Backup feature (HBASE-7912), that runs a MR job
> > from
> > > the master to copy data or restore data.
> > > (backup is also "not really core" as in.. if you don't use backup
> you'll
> > > not end up running MR jobs, but this was probably true for MOB as in
> "if
> > > you don't enable MOB you don't need MR")
> > >
> > > any thoughts? do we a rule that says "we don't want to have hbase run
> MR
> > > jobs, only tool started manually by the user can do that". or can we
> > start
> > > adding MR calls around without problems?
> >
>

Re: [DISCUSSION] MR jobs started by Master or RS

[Devaraj Das]

Very much agree; for tools like ExportSnapshot / Backup / Restore, it's fine to 
be dependent on MR. MR is the right framework for such. We should also do 
compactions using MR (just saying :) )

From: Ted Yu 
Sent: Thursday, September 22, 2016 2:00 PM
To: dev@hbase.apache.org
Subject: Re: [DISCUSSION] MR jobs started by Master or RS

I agree - backup / restore is in the same category as import / export.

On Thu, Sep 22, 2016 at 1:58 PM, Andrew Purtell 
wrote:

> Backup is extra tooling around core in my opinion. Like import or export.
> Or the optional MOB tool. It's fine.
>
> > On Sep 22, 2016, at 1:50 PM, Matteo Bertozzi 
> wrote:
> >
> > What's the latest opinion around running MR jobs from hbase (Master or
> RS)?
> >
> > I remember in the past that there was discussion about not having MR has
> > direct dependency of hbase.
> >
> > I think some of discussion where around MOB that had a MR job to compact,
> > that later was transformed in a non-MR job to be merged, I think we had a
> > similar discussion for log split/replay.
> >
> > the latest is the new Backup feature (HBASE-7912), that runs a MR job
> from
> > the master to copy data or restore data.
> > (backup is also "not really core" as in.. if you don't use backup you'll
> > not end up running MR jobs, but this was probably true for MOB as in "if
> > you don't enable MOB you don't need MR")
> >
> > any thoughts? do we a rule that says "we don't want to have hbase run MR
> > jobs, only tool started manually by the user can do that". or can we
> start
> > adding MR calls around without problems?
>

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das]

Just reviving this thread. Thanks Sean, Stack, Dima, and others for the 
thorough reviews and testing. Thanks Ted and Vlad for taking care of the 
feedback. Are we all good to do the merge now? Rather do sooner than later.

From: saint@gmail.com  on behalf of Stack 

Sent: Monday, September 12, 2016 1:18 PM
To: HBase Dev List
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu  wrote:

> Mega patch (rev 18) is on HBASE-14123.
>
> Please comment on HBASE-14123 on how you want to review.
>


Yeah. That was my lost tab. Last rb was 6 months ago. Suggest updating it.
RB is pretty good for review. Patch is only 1.5M so should be fine.

St.Ack


>
> Thanks
>
> On Mon, Sep 12, 2016 at 12:15 PM, Stack  wrote:
>
> > On review of the 'patch', do I just compare the branch to master or is
> > there a megapatch posted somewhere (I think I saw one but it seemed stale
> > and then I 'lost' the tab). Sorry for dumb question.
> > St.Ack
> >
> > On Mon, Sep 12, 2016 at 12:01 PM, Stack  wrote:
> >
> > > Late to the game. A few comments after rereading this thread as a
> 'user'.
> > >
> > > + Before merge, a user-facing feature like this should work (If this is
> > "higher-bar
> > > for new features", bring it on -- smile).
> > > + As a user, I tried the branch with tools after reviewing the
> > just-posted
> > > doc. I had an 'interesting' experience (left comments up on issue). I
> > think
> > > the tooling/doc. important to get right. If it breaks easily or is
> > > inconsistent (or lacks 'polish'), operators will judge the whole
> > > backup/restore tooling chain as not trustworthy and abandon it. Lets
> not
> > > have this happen to this feature.
> > > + Matteo's suggestion (with a helpful starter list) that there needs to
> > be
> > > explicit qualification on what is actually being delivered --
> including a
> > > listing of limitations (some look serious such as data bleed from other
> > > regions in WALs, but maybe I don't care for my use case...) -- needs to
> > > accompany the merge. Lets fold them into the user doc. in the technical
> > > overview area as suggested so user expectations are properly managed
> > > (otherwise, they expect the world and will just give up when we fall
> > > short). Vladimir did a list of what is in each of the phases above
> which
> > > would serve as a good start.
> > > + Is this feature 'experimental' (Matteo asks above). I'd prefer it is
> > > not. If it is, it should be labelled all over that it is so. I see
> > current
> > > state called out as a '... technical preview feature'. Does this mean
> > > not-for-users?
> > >
> > > St.Ack
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu  wrote:
> > >
> > >> Sean:
> > >> Do you have more comments ?
> > >>
> > >> Cheers
> > >>
> > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir Rodionov <
> > vladrodio...@gmail.com
> > >> >
> > >> wrote:
> > >>
> > >> > Sean,
> > >> >
> > >> > Backup/Restore can fail due to various reasons: network outage
> > (cluster
> > >> > wide), various time-outs in HBase and HDFS layer, M/R failure due to
> > >> "HDFS
> > >> > exceeded quota", user error (manual deletion of data) and so on so
> on.
> > >> That
> > >> > is impossible to enumerate all possible types of failures in a
> > >> distributed
> > >> > system - that is not our goal/task.
> > >> >
> > >> > We focus completely on backup system table consistency in a presence
> > of
> > >> any
> > >> > type of failure. That is what I call "tolerance to failures".
> > >> >
> > >> > On a failure:
> > >> >
> > >> > BACKUP. All backup system information (prior to backup) will be
> > restored
> > >> > and all temporary data, related to a failed session, in HDFS will be
> > >> > deleted
> > >> > RESTORE. We do not care about system data, because restore does not
> > >> change
> > >> > it. Temporary data in HDFS will be cleaned up and table will be in a
> > >> state
> > >> > back to where it was before operation started.
> > >> >
> > >> > This is what user should expect in case of a failure.
> > >> >
> > >> > -Vlad
> > >> >
> > >> >
> > >> > -Vlad
> > >> >
> > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey 
> > wrote:
> > >> >
> > >> > > Failing in a consistent way, with docs that explain the various
> > >> > > expected failures would be sufficient.
> > >> > >
> > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir Rodionov
> > >> > >  wrote:
> > >> > > > Do not worry Sean, doc is coming today as a preview and our
> writer
> > >> > Frank
> > >> > > > will be working on a putting  it into Apache repo. Timeline
> > depends
> > >> on
> > >> > > > Franks schedule but I hope we will get it rather sooner than
> > later.
> > >> > > >
> > >> > > > As for 

Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

[Devaraj Das]

+1 for merge of HBASE-7912 to master. I think this feature can go in in its 
present form and bake for sometime in the master. Work is continuing to harden 
it.


From: Vladimir Rodionov 
Sent: Friday, September 02, 2016 8:51 AM
To: dev@hbase.apache.org
Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch HBASE-7912

Snapshot robustness is better now with introduction of region splits/merges
on/off feature. Region splits during snapshots was the major problem.

-Vlad

On Fri, Sep 2, 2016 at 8:12 AM, Vladimir Rodionov 
wrote:

> >>Are they independent enough that we can get backup/restore tolerant to
> >>failures prior to merge to master? Prior to backport to branch-1?
>
> As we stated already, snapshots are not part of the feature, snapshots has
> been merged into the master long time ago
> and as far as I understood - without requiring them to be 100% robust and
> fault tolerant and they are widely used in many production systems
> nevertheless. https://issues.apache.org/jira/browse/HBASE-14415 relies on
> Snapshots v2 but we can reconsider it, there are some thoughts how to make
> backups snapshotless.
>
> Backups are fault tolerant to some extent - in case of failure (and
> failures can happen) we clean everything up and do not leave system table
> in inconsistent state. Would it be enough, Sean Busbey?
>
> -Vlad
>
> On Fri, Sep 2, 2016 at 7:38 AM, Ted Yu  wrote:
>
>> We're continuing to make backup / restore more robust.
>> Work in progress (both are close to being integrated):
>>
>> HBASE-15565 Rewrite restore with Procedure V2
>> HBASE-15449 Support physical table layout change
>>
>> Since snapshot is dependency in the full backup, backup / restore wouldn't
>> be more robust than snapshot is.
>>
>> On Fri, Sep 2, 2016 at 7:03 AM, Sean Busbey  wrote:
>>
>> > right, they're separate features but when asked about "robust
>> > backup/restore" (which is what I care about for this feature getting
>> > merged) things were pawned off on snapshots.
>> >
>> > Are they independent enough that we can get backup/restore tolerant to
>> > failures prior to merge to master? Prior to backport to branch-1?
>> >
>> > On Thu, Sep 1, 2016 at 1:11 PM, Andrew Purtell 
>> > wrote:
>> > > I agree these are separate features FWIW
>> > >
>> > > On Thu, Sep 1, 2016 at 11:09 AM, Vladimir Rodionov <
>> > vladrodio...@gmail.com>
>> > > wrote:
>> > >
>> > >> >> Do we have JIRA issue(s) covering making snapshots robust in the
>> face
>> > >> >> of monkeys?
>> > >>
>> > >> I would like to mention that "robust snapshots" and "table
>> > backup/restore"
>> > >> are totally separate features, but we have separate JIRA for fault
>> > >> tolerance (HBASE-14413).
>> > >>
>> > >> -Vlad
>> > >>
>> > >> On Thu, Sep 1, 2016 at 9:28 AM, Ted Yu  wrote:
>> > >>
>> > >> > Sean:
>> > >> > Please see HBASE-14413 for the last question.
>> > >> >
>> > >> > FYI
>> > >> >
>> > >> > On Thu, Sep 1, 2016 at 9:24 AM, Sean Busbey 
>> > wrote:
>> > >> >
>> > >> > > On Sat, Aug 20, 2016 at 12:59 PM, Vladimir Rodionov
>> > >> > >  wrote:
>> > >> > > > Not sure what do you mean, Andrew by "trying out the branch via
>> > the
>> > >> > IT",
>> > >> > > > but we do not recommend running this with monkey enabled.
>> > >> > > > It has not been tested in a such scenario yet and frankly
>> > speaking it
>> > >> > is
>> > >> > > > not supposed to work (snapshots will fail anyway and we
>> depends on
>> > >> > > > snapshots)
>> > >> > > >
>> > >> > > >
>> > >> > >
>> > >> > > Also won't have time to test out the branch this week, but if
>> we're
>> > >> > > not going to handle failures do we have tools or guidance on
>> > >> > > recovering in the case of things falling over?
>> > >> > >
>> > >> > > Do we have JIRA issue(s) covering making snapshots robust in the
>> > face
>> > >> > > of monkeys?
>> > >> > >
>> > >> > > --
>> > >> > > busbey
>> > >> > >
>> > >> >
>> > >>
>> > >
>> > >
>> > >
>> > > --
>> > > Best regards,
>> > >
>> > >- Andy
>> > >
>> > > Problems worthy of attack prove their worth by hitting back. - Piet
>> Hein
>> > > (via Tom White)
>> >
>> >
>> >
>> > --
>> > busbey
>> >
>>
>
>

Re: Hangouts for C++ client discussions

[Devaraj Das]

The link is not working for me Enis

Sent from Outlook Mobile




On Tue, May 3, 2016 at 11:58 AM -0700, "Enis Söztutar" 
> wrote:

Devs,

If you have been following, we have two initiatives for writing a C++
client for HBase. We are planning on doing a hangouts tomorrow, Wed 10-11am
PDT to discuss various things related to the efforts. Please feel free to
join if you are interested.

The proposed agenda is here:
https://docs.google.com/document/d/10O626jxK6f7nmdBjG0h9f_PYkbWNMGKH0-1H1i6SmaU/edit


Hangouts link is here:
https://hangouts.google.com/call/lige2qdmyfakrbzacfcafhzrh4e

Enis

Re: What's going on? Two C++ clients being developed

[Devaraj Das]

(Meant to send this earlier but got delayed)
Thanks Clay for the inputs. Would like to give a quick update on where we are 
at this point and solicit thoughts on how to proceed from here:

1. The patch (from Vamsi) that has been uploaded on RB last has the copyright 
and other license related stuff taken care of.

2. We are taking a look at making the configuration pluggable, and provide an 
implementation that works with XML files. Maybe, something like this - if the 
configured conf directory has XML files, assume they are the default config 
files. If not, use the config loader meant for that particular type. The 
filename extension can be used to make this choice I guess..

3. On the sync/async RPC implementation, we are continuing to investigate this. 
This is something we could work together on with Elliott actively. On a related 
note, we have an implementation of the "batch" calls that's doing the get/put 
sequentially for each of the get/put. The HBase Java-client's AsyncProcess does 
it so that multiple regionservers are reached out to in parallel, etc. Looking 
at if implementing the RPC async from the get go would obviate the need for 
AsyncProcess in c++ client...

4. We can look at making smaller patches for the various client API and 
associated classes like GET, PUT, TableName, etc. (this is called out in the 
last mail from Enis). The way I see it - there is the front end work for 
providing classes for the APIs, and there is the back end work - Connection 
management, RPC, AsyncProcess-like-stuff.. There is a good amount of work done 
in Vamsi's patch for the former, and there is an Async RPC basis for the 
backend in Elliott's branch. We should see how we can leverage both and come up 
with one unified implementation if possible.

Thoughts?


From: Clay Baenziger (BLOOMBERG/ 731 LEX) 
Sent: Monday, April 25, 2016 11:01 AM
To: dev@hbase.apache.org
Subject: Re: What's going on? Two C++ clients being developed

>From an operator's view-point, I would add:

I am concerned that as an operator who often has to build Hadoop eco-system 
components and is chiefly interested in these C++ bindings that a non Apache, 
GNU or otherwise large-scale community supported open source utility in the 
build chain is liability to this codebase and its adoption.

As to the configuration process, I would really like to keep with XML. I am 
looking to use Maven repositories to host the configurations of our clusters 
(e.g. a POM-file per cluster hosting hbase-site.xml, hdfs-site.xml, etc.); it 
would be a pain to have to synchronize two configurations of the same 
information both on the publishing side and client side dependent on the use. 
It would be possible to duplicate all this information just because of 
different consumers but XML should not be terribly difficult for C/C++ code to 
parse -- e.g. OpenSolaris's use in SMF, Zones, etc. Further, for an example of 
an incubator project which uses XML configs already, see Apache HAWQ's use of 
hdfs-client.xml and similar for YARN with their pure non-Java implementation 
for HDFS and YARN client: 
https://github.com/apache/incubator-hawq/blob/9452055bc74e64f308a8b6cc2b7ab946e5584ba8/src/backend/utils/misc/etc/hdfs-client.xml.

I certainly would not be opposed to a pluggable configuration system. I'd 
imagine Apache Ambari could use that to not need to materialize XML configs 
from Postgres; I could see using Zookeeper akin to how Apache Solr Cloud uses 
Zookeeper for configuration information. But at this time, we have XML files 
for better or worse and a pluggable configuration system sounds like a great 
separate JIRA.

-Clay


From: dev@hbase.apache.org At: Apr 19 2016 15:31:25
To: dev@hbase.apache.org
Subject: Fwd:Re: What's going on? Two C++ clients being developed at the moment?

So there are a couple of technical topics that we can further discuss and
hopefully come to a conclusion for going forward.

1. Build system. I am in the auto-tools camp, unless there is a very good
reason to use a non-standard tool like Buck / Bazel, etc. Not sure whether
it makes sense to have two different build systems concurrently. Can we do
the main build with make, and create a wrapper with Buck?

2. XML based configuration versus something native. I strongly believe that
we should support standard hbase-site.xml. A lot of tooling in the Hadoop
ecosystem has already been developed for managing and deploying XML based
configurations over the years. Puppet / Chef scripts, Ambari, CM, etc all
understand and support hbase-site.xml. This is also true for hadoop
operators who should be familiar with modifying these files. So it would be
a real pain if we suddenly come up with yet another config format that
requires the operators and tools to learn how to deploy and manage. What if
there are both java clients and C++ clients in the same nodes. How do you
keep two config files in sync? Then there is the issue of

Re: What's going on? Two C++ clients being developed at the moment?

[Devaraj Das]

Elliott, on the point about sync versus async, as I said earlier, that's an 
area we are looking at, and we'd welcome contributions from you very much as 
well. But at the same time, I don't want the perfect to be the enemy of the 
good. If something is working well in the sync mode, why would we block it.

On your point about configuration, I don't see why we shouldn't be providing a 
Java-client compatible configuration story. You raise the point about pure C++ 
clients. I say that we have cases where we have deployment tools (like Ambari) 
deploy configurations and it'll be better to not introduce a C++ configuration 
for them to deploy. But again this is an area that we could look at as a 
pluggable thing..

On your point about Support, we have customers waiting to consume the library. 
Even Facebook is looking at using HBase C++ client I thought? If there are 
users for a feature, it shouldn't rot otherwise we are not doing a good job of 
taking care of our users, right? But agree that we have to careful about this 
one given this is an entire new implementation of a large piece of software - 
we should look at what we could do better here w.r.t code rot.

On your point about Copyright/Authors/GNU-license-stuff, Vamsi has responded to 
such comments from you on RB already, I believe. In any case, if there are 
issues around these topics, we will address them as a top priority. (Putting my 
vendor hat, as a vendor of open-source software, we will have to do this before 
we can ship this in any HDP release).


From: Elliott Clark <ecl...@apache.org>
Sent: Monday, April 18, 2016 8:41 PM
To: dev@hbase.apache.org
Subject: Re: What's going on? Two C++ clients being developed at the moment?

On Mon, Apr 18, 2016 at 4:57 PM, Devaraj Das <d...@hortonworks.com> wrote:

> 1. Be able to do async RPC. Vamsi's current implementation does sync. I
> know Vamsi is already looking at that aspect. Just to be clear, this in my
> mind this doesn't qualify as a blocker - since we don't have an async
> client API to support yet.
>

There's no deadline on creating a good client. We should do what's better
not what's fastest to implement. We know that the way HBase is used results
in way too many threads if everything is sync. There's no prize for getting
things done before they are the best possible. We should be learning from
our many many mistakes in the first client and creating something better.
We shouldn't be re-creating the same exact mess in a different language.

There's no backwards compat rules, so we should make what works the best
for HBase going forward. And as clusters get larger and larger one thread
per server or per request is not going to scale. We're already in a place
that the number of threads is a factor. Just imagine as clusters continue
to grow. Locking and context switches will abound.


> 2. Switch to C++ ways of configuring the hbase client configuration. This
> is something I am really not sure about. By going this route, we'd have to
> be able to manage two different ways of configuring things - one for Java
> and another for C++. This will lead to unnecessary duplication of configs
> and such (and the deployment tools would now have to be aware about a new
> way of configuring c++ clients). But we can take a look at making this
> configuration method pluggable if it makes sense.
>

 We should do what is best for each use case. You don't see xml going well
with python, ruby, javascript configs. If the native client is going to be
used by people outside of the java world, then we should give them the
experience that delights them. Not that makes them think about java. None
of the best designed libraries in cpp that I can think of use an xml file
for a config.

Elliott, regarding your concern as to who would support the large code drop
> .. we did talk about breaking the patch up into smaller ones if it makes
> sense for reviews and such.
>

Splitting it up is nice, and shows that software is being developed in the
open, and not code dropped. However my concern is less about the size of
the patch and more about the body of work being supported. HBase has had a
bad history of taking code, having no one support it, and then having bit
rot in our repository.


> I think Vamsi has already addressed the other concerns to do with
> Copyright headers, etc.
>

Sorry this hasn't been addressed at all.
There are copyright headers from other projects (install-sh, missing).
Importing some code from other projects seems dubious at best. And at worst
seems like copy paste. That could come back and bite us. Once trust in a
code's origin is lost it's really hard to integrate it. I don't want to
have to go through every line and see where it came from.

There's other people's names in the code. Everyone who wrote the code needs
to assign copyright to Apache. Saying "I worked with someone else, I can
r

Re: [DISCUSS] Criteria for including MOB feature backport in branch-1

[Devaraj Das]

Thanks for chiming in Jincheng. Really appreciated. But from this thread, it's 
apparent that people don't have enough enthusiasm to see this backported in 
branch-1. So, maybe what would be really nice to instead do is to make sure 
that MOB is performant / reliable / releasable in the HBASE-2.0 release. As 
suggested in this thread, let's run YCSB / ITBLL and other such tests to make 
sure we have enough confidence on this before HBASE-2.0 goes out. Of course, 
Unit Tests is another area in the master branch that MOB is still flaky in

From: Du, Jingcheng 
Sent: Thursday, March 03, 2016 9:55 PM
To: dev@hbase.apache.org
Subject: RE: [DISCUSS] Criteria for including MOB feature backport in branch-1

Sorry for the late chime in and thank you all for the discussion.

I agree with Jon's suggestion, to make a new branch for MOB in branch-1.x, and 
backport it when it is definitely ready.
I can start to work on that branch after the distributed mob compaction is 
finished.

I think the distributed mob compaction is not a blocker. The old mob compaction 
can be disabled by configuration, this would not impact the read/write 
performance of mob files. Users can count on the TTL cleaner to remove the 
expired mob files which is very light weighted.
And I will keep eyes on the UT results, and get ready to fix the coming up 
failures. Thanks.


Regards,
Jingcheng

-Original Message-
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Friday, March 4, 2016 9:53 AM
To: dev@hbase.apache.org
Subject: Re: [DISCUSS] Criteria for including MOB feature backport in branch-1

Good questions, Enis.

If my bandwidth permits, I am planning to collect performance statistics using 
ycsb against cluster with and without MOB feature enabled.

Cheers

On Thu, Mar 3, 2016 at 3:49 PM, Enis Söztutar  wrote:

> Regardless of the backport, did we do the same regression analysis for
> the master branch merge at the time of the merge? Like make sure that
> the latencies and stability of non-mob is affected or not. Sorry, I
> was not following the merge vote closely.
>
> The reason I am asking is that although we can allow some flexibility
> in master, we still want to keep master releasable and prevent a
> 1-year effort to re-stabilize master when we want to release 2.0. If
> we have design / stability / impact questions, when and how they will
> be addressed for a 2.0 release? Let's say that we would like to
> release 2.0 by summer time as Matteo was quoting, how do we make sure
> that the feature as is, is stable and supportable?
>
> Enis
>
> On Thu, Mar 3, 2016 at 3:36 PM, Andrew Purtell 
> wrote:
>
> > Just to be clear, I did not actually oppose a backport of MOB.
> > Although
> - I
> > have to say am sympathetic to Elliott's position that there's a
> > project management reason not to. My insistence here is "merely" for
> > this change
> in
> > particular a backport to the branch we are making production
> > releases
> from
> > demands a credible data driven assessment on stability,
> > functionality,
> and
> > performance - both avoidance of regression and affirmative
> > indication of the alleged benefit.
> >
> > On Thu, Mar 3, 2016 at 3:16 PM, Ted Yu  wrote:
> >
> > > In the middle of writing response to this thread, I saw subsequent
> > comments
> > > from Andrew, Jon and Elliott.
> > >
> > > In light of opposition from the community w.r.t. backporting MOB
> > feature, I
> > > think it suffices to say that this wouldn't be done now.
> > >
> > > Thanks everyone for chiming in.
> > >
> > > On Thu, Mar 3, 2016 at 2:42 PM, Elliott Clark 
> wrote:
> > >
> > > > I just don't see a why we would back port. We're going to
> > > > release a
> 2.0
> > > > when things are ready. It will be a major feature release. MOB
> > > > is a
> > major
> > > > feature. Without compelling reason backporting to branch-1 seems
> > > > like
> > an
> > > > end run around what sem ver is supposed to mean (not the api
> > guarantees,
> > > > the actual meaning).
> > > >
> > > > Backporting major features is a bad habit that the Hadoop
> > > > community
> > seems
> > > > to have. We shouldn't follow their lead.
> > > >
> > > > On Thu, Mar 3, 2016 at 1:01 PM, Stack  wrote:
> > > >
> > > > > On Thu, Mar 3, 2016 at 9:02 AM, Ted Yu 
> wrote:
> > > > >
> > > > > > Hi,
> > > > > > As requested by Sean Busbey, I am starting a new thread w.r.t.
> > > > > backporting
> > > > > > MOB feature to branch-1.
> > > > > >
> > > > > This is to solicit discussion on the criteria for including
> > > > > MOB
> > feature
> > > > > > backport in branch-1.
> > > > > >
> > > > > > I can think of the following:
> > > > > > 1. whether there is customer interest
> > > > > >
> > > > > > There is.
> > > > > > See Jonathan's response here:
> > > > > http://search-hadoop.com/m/YGbbDSqxD1PYXK62
> > > > > 

Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)

[Devaraj Das]

Sounds good to me, Jonathan. Although I suspect that at the end this will be 
close to what Ted Yu has in the backport patch but this is a more structured / 
reliable way of doing it. When you say this - "backport the patches that came 
after the merge to trunk", is there a reliable way to identify all those. 
In the backport jira, there is a mention of some additional jiras after the 
original merge - https://goo.gl/6stN7Y - we need to make sure if these are the 
only ones relevant.

From: Jonathan Hsieh <j...@cloudera.com>
Sent: Wednesday, March 02, 2016 2:03 PM
To: dev@hbase.apache.org
Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 
HBase MOB to trunk)

I'll offer what I think would be another reasonable and likely less
destabilizing approach:  essentially treat it as a branch.  We could start
from the existing hbase-11339 branch, call it hbase-11339-branch-1 and do
the work to merge it into branch-1 (instead of trunk).  To merge into
branch-1 it would require the 3 +1's just like any other branch merge.

Mob initially started off as a mega patch and was broken down and committed
in reviewable pieces to branch hbase-11339.  Most of this was actually when
1.0.0-SNAPSHOT was trunk so many of the older commits may actually jive
better with branch-1. Towards the end we did periodic merge commits with
trunk as we shook out issues integration testing found, and as branch merge
reviews uncovered other short comings.  These merge patches actually showed
how we modified the mob code to account for the later changes in trunk.  In
this particular case we coud take hbase-11339-branch-1,  backport the
patches that came after the merge to trunk with the normal single +1 review
process on the branch, and then propose a branch merge with branch-1. When
all pieces are ready and we had stable unit tests (admittedly that was a
one shortcoming when I sheparded hbase-11339) we do 3 +1s to merge to
branch-1 (and possibly branch-1.3).

This has the nice benefit compared to andrew's suggesting of not
potentially blocking branch-1 or having a half complete mob feature in
branch-1 if hbase-11339-branch-1 isn't ready.

Jon.




On Wed, Mar 2, 2016 at 1:31 PM, Andrew Purtell <andrew.purt...@gmail.com>
wrote:

> On the question of change provenance, if I can make a suggestion: How I
> would approach the backport of a feature developed in trunk over multiple
> commits and JIRAs is I would look through git history, find each related
> commit, and apply each commit in sequence to branch-1, fixing things up as
> needed each time. Then the resulting working branch can be pushed up to aid
> review of the aggregate result by the MOB developers and the larger
> community. Ted, if you approached the backport in this way, then pushing up
> your working branch will aid review.
>
> > On Mar 2, 2016, at 1:17 PM, Andrew Purtell <andrew.purt...@gmail.com>
> wrote:
> >
> > I concur with Stack's concerns with a developer not involved in building
> the MOB feature attempting a backport in general, and furthermore without
> detailed provenance of all of the incorporated changes. I'm also concerned
> about how well MOB works in real production given people use branch-1 in
> production and not trunk.
> >
> > As for Ted's issue, I'm not going to let him commit it (I will veto)
> without clear proof that it works and doesn't hurt perf of non MOB cases by
> way of reproducible benchmarks - benchmarks I can personally reproduce
> afterward (I'm not volunteering to do Ted's necessary legwork) - done with
> branch-1 with the proposed patch applied. We are not there yet though,
> barely, into review, but this will be coming up very soon.
> >
> >>> On Mar 2, 2016, at 1:03 PM, Stack <st...@duboce.net> wrote:
> >>>
> >>> On Wed, Mar 2, 2016 at 11:26 AM, Devaraj Das <d...@hortonworks.com>
> wrote:
> >>>
> >>> Stack, as things stand, Ted Yu has a patch that is a backport of MOB.
> He
> >>> told me offline he has taken a look at the jiras that went in in the
> master
> >>> that is to do with MOB, and got them in the backport.
> >>
> >>
> >>
> >> On the issue, a near 800k blob is dumped which is described as "...a
> >> backport of MOB... " but w/o attribution of provenance nor any other
> >> description of what it contains. Only when this is pointed-out in the
> issue
> >> do we get a short listing of supposed JIRAs included with no
> justification
> >> of what is covered, what is included/excluded, and what machinations
> were
> >> done to make it fit branch-1 (Yet you offline were given this info?).
> >>
> >> Such poor practice only makes me more in

Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)

[Devaraj Das]

Stack, as things stand, Ted Yu has a patch that is a backport of MOB. He told 
me offline he has taken a look at the jiras that went in in the master that is 
to do with MOB, and got them in the backport. Now, to your point, I agree that 
someone familiar with MOB code should do the backport but the question is, is 
that dev available to do it now? The next best thing is to get Ted Yu's patch 
reviewed by someone familiar with the feature. I really hope that we can get 
cycles from the original MOB devs on this.

Agree that we shouldn't be adding flaky tests. The question is if the failures 
on master to do with MOB are really MOB related or something else (for 
argument's sake, let's say, procv2)..

On the point about the DISCUSS thread, yeah, it's fine to do it if folks feel 
it's the right way to go.

From: saint@gmail.com  on behalf of Stack 

Sent: Wednesday, March 02, 2016 10:55 AM
To: HBase Dev List
Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 
HBase MOB to trunk)

(Doing another resend...)

I have objection to you, Ted Yu, doing it. MOB has spread all about master
branch. Backport should be done by someone who knows MOB.

Higher up in this thread, Sean asks: "Can we get a [DISCUSS] flagged thread
to see what, if anything, folks
would like to see gate inclusion in branch-1?"  Shouldn't we do this before
we 'create a backport...'.

For me, there should be no new flakies in branch-1. Branch-1 builds are a
hard-won stable(-ish). Looking at master build, MOB seems quiet lately but
going by HBASE-15012, our flakies umbrella issue, I see notes that
TestMobExportSnapshot has failed a few times (that is probably because the
test it derives from is flakey) and TestMobRestoreFlushSnapshotFromClient
gets a mention.

St.Ack



On Tue, Mar 1, 2016 at 5:27 PM, Stack  wrote:

> (Doing a resend of below... it doesn't seem to have gone through)
>
> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu  wrote:
>
>> If there is no objection, I will create a backport JIRA tomorrow and
>> attach patch.
>>
>>
> I have objection to you doing it. MOB has spread all about master branch.
> Backport should be done by someone who knows MOB.
>
> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] flagged
> thread to see what, if anything, folks
> would like to see gate inclusion in branch-1?"  Shouldn't we do this
> before we 'create a backport...'.
>
> For me, there should be no new flakies in branch-1. Branch-1 builds are a
> hard-won stable(-ish). Looking at master build, MOB seems quiet lately but
> going by HBASE-15012, our flakies umbrella issue, I see notes that
> TestMobExportSnapshot has failed a few times (that is probably because the
> test it derives from is flakey) and TestMobRestoreFlushSnapshotFromClient
> gets a mention.
>
> St.Ack
>
>
>
> On Tue, Mar 1, 2016 at 9:32 AM, Stack  wrote:
>
>> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu  wrote:
>>
>>> If there is no objection, I will create a backport JIRA tomorrow and
>>> attach patch.
>>>
>>>
>> I have objection to you doing it. MOB has spread all about master branch.
>> Backport should be done by someone who knows MOB.
>>
>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS] flagged
>> thread to see what, if anything, folks
>> would like to see gate inclusion in branch-1?"  Shouldn't we do this
>> before we 'create a backport...'.
>>
>> For me, there should be no new flakies in branch-1. Branch-1 builds are a
>> hard-won stable(-ish). Looking at master build, MOB seems quiet lately but
>> going by HBASE-15012, our flakies umbrella issue, I see notes that
>> TestMobExportSnapshot has failed a few times (that is probably because the
>> test it derives from is flakey) and TestMobRestoreFlushSnapshotFromClient
>> gets a mention.
>>
>> St.Ack
>>
>

[jira] [Reopened] (HBASE-8549) Integrate Favored Nodes into StochasticLoadBalancer

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-8549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das reopened HBASE-8549:


I was discussing this offline with [~toffer], and there is a patch that Yahoo 
has developed that includes this and other fixes for running HBase with 
FavoredNodes enabled. Reopening this. Speaking for Francis, but we should have 
something up on jira soon..

> Integrate Favored Nodes into StochasticLoadBalancer
> ---
>
> Key: HBASE-8549
> URL: https://issues.apache.org/jira/browse/HBASE-8549
> Project: HBase
>  Issue Type: Bug
>  Components: Balancer
>Reporter: Elliott Clark
> Attachments: HBASE-8549-0.patch
>
>
> Right now we have a FavoredNodeLoadBalancer.  It would be pretty easy to 
> integrate the favored node list into the strochastic balancer.  Then we would 
> have the best of both worlds.  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14963) Remove Guava dependency from HBase client code

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-14963:
---

 Summary: Remove Guava dependency from HBase client code
 Key: HBASE-14963
 URL: https://issues.apache.org/jira/browse/HBASE-14963
 Project: HBase
  Issue Type: Improvement
  Components: Client
Reporter: Devaraj Das
Assignee: Devaraj Das


We ran into an issue where an application bundled its own Guava (and that 
happened to be in the classpath first) and HBase's MetaTableLocator threw an 
exception due to the fact that Stopwatch's constructor wasn't compatible... 
Might be better to not depend on Stopwatch at all in MetaTableLocator since the 
functionality is easily doable without.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-14536) Balancer & SSH interfering with each other leading to unavailability

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-14536:
---

 Summary: Balancer & SSH interfering with each other leading to 
unavailability
 Key: HBASE-14536
 URL: https://issues.apache.org/jira/browse/HBASE-14536
 Project: HBase
  Issue Type: Bug
  Components: master, Region Assignment
Reporter: Devaraj Das


Came across this in our cluster:
1. The meta was assigned to a server 10.0.0.149,16020,1443507203340
{noformat}
2015-09-29 06:16:22,472 DEBUG [AM.ZK.Worker-pool2-t56] 
master.RegionStates: Onlined 1588230740 on 
10.0.0.149,16020,1443507203340 {ENCODED => 1588230740, NAME => 
'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}
{noformat}

2. The server dies at some point:
{noformat}
2015-09-29 06:18:25,952 INFO  [main-EventThread] 
zookeeper.RegionServerTracker: RegionServer ephemeral node deleted, 
processing expiration [10.0.0.149,16020,1443507203340]

2015-09-29 06:18:25,955 DEBUG [main-EventThread] master.AssignmentManager: 
based on AM, current 
region=hbase:meta,,1.1588230740 is on server=10.0.0.149,16020,1443507203340 
server being checked: 
10.0.0.149,16020,1443507203340
{noformat}

3. The balancer had computed a plan that contained a move for the meta:
{noformat}
2015-09-29 06:18:26,833 INFO  
[B.defaultRpcServer.handler=12,queue=0,port=16000] master.HMaster: 
balance hri=hbase:meta,,1.1588230740, 
src=10.0.0.149,16020,1443507203340, dest=10.0.0.205,16020,1443507257905
{noformat}

4. The following ensues after this, leading to the meta remaining unassigned:
{noformat}
2015-09-29 06:18:26,859 DEBUG 
[B.defaultRpcServer.handler=12,queue=0,port=16000] 
master.AssignmentManager: Offline hbase:meta,,1.1588230740, no need to 
unassign since it's on a dead server: 10.0.0.149,16020,1443507203340
..
2015-09-29 06:18:26,899 INFO  
[B.defaultRpcServer.handler=12,queue=0,port=16000] master.RegionStates: 
Offlined 1588230740 from 10.0.0.149,16020,1443507203340
.
2015-09-29 06:18:26,914 INFO  
[B.defaultRpcServer.handler=12,queue=0,port=16000] 
master.AssignmentManager: Skip assigning hbase:meta,,1.1588230740, it is 
on a dead but not processed yet server: 10.0.0.149,16020,1443507203340

2015-09-29 06:18:26,915 DEBUG [AM.ZK.Worker-pool2-t58] 
master.AssignmentManager: Znode hbase:meta,,1.1588230740 deleted, 
state: {1588230740 state=OFFLINE, ts=1443507506914, 
server=10.0.0.149,16020,1443507203340}

2015-09-29 06:18:29,447 DEBUG 
[MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] master.AssignmentManager: 
based on AM, current 
region=hbase:meta,,1.1588230740 is on server=null server being checked: 
10.0.0.149,16020,1443507203340

2015-09-29 06:18:29,451 INFO  [MASTER_META_SERVER_OPERATIONS-
10.0.0.148:16000-2] handler.MetaServerShutdownHandler: META has been 
assigned to otherwhere, skip assigning.

2015-09-29 06:18:29,452 DEBUG 
[MASTER_META_SERVER_OPERATIONS-10.0.0.148:16000-2] 
master.DeadServer: Finished processing 10.0.0.149,16020,1443507203340
{noformat}




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Please welcome new HBase committer Stephen Yuan Jiang

[Devaraj Das]

Congratulations Stephen!

 On Aug 20, 2015, at 7:10 PM, Andrew Purtell apurt...@apache.org wrote:
 
 On behalf of the Apache HBase PMC, I am pleased to announce that Stephen
 Jiang has accepted the PMC's invitation to become a committer on the
 project. We appreciate all of Stephen's hard work and generous
 contributions thus far, and look forward to his continued involvement.
 
 Congratulations and welcome, Stephen!
 
 -- 
 Best regards,
 
   - Andy
 
 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

[jira] [Created] (HBASE-13938) Deletes done during the region merge transaction may get eclipsed

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-13938:
---

 Summary: Deletes done during the region merge transaction may get 
eclipsed
 Key: HBASE-13938
 URL: https://issues.apache.org/jira/browse/HBASE-13938
 Project: HBase
  Issue Type: Bug
  Components: master, regionserver
Reporter: Devaraj Das
 Fix For: 1.1.1


Was looking at an issue from our internal testing. It seems the Deletes of the 
region rows from the meta done during the merge transaction could be eclipsed 
by the Put of a region row that might have happened moments before.
The master logs this for the merge:
{noformat}
2015-06-18 13:13:46,018 INFO  [AM.ZK.Worker-pool2-t12] 
master.AssignmentManager: Handled MERGED event; 
merged=IntegrationTestIngest,a65c,1434633226681.0927319db6bf5e128e3bec2a420819aa.,
 
region_a=IntegrationTestIngest,a65c,1434631353820.8b911862d7705ac808b8d132d0154c16.,
 
region_b=IntegrationTestIngest,acc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec.,
 on ddas-2-5.openstacklocal,16020,1434632778438
{noformat}

One of the regions that got merged got Opened a few seconds back:
{noformat}
2015-06-18 13:13:46,591 INFO  [RS_OPEN_REGION-ddas-2-5:16020-1] 
regionserver.HRegion: Onlined 1bdaf759862f45d133ef77fdbda21aec; next 
sequenceid=182988
{noformat}
The above would have done a Put in the meta.

Looking at the raw scan of the meta, for the new merged region, the creation 
timestamp is 1434633226101:
{noformat}
 IntegrationTestIngest,a65c,1434633226681.0927319db6bf5e128e3bec2a420819aa. 
column=info:regioninfo, timestamp=1434633226101, value={ENCODED = 
0927319db6bf5e128e3bec2a420819aa, NAME = 
'IntegrationTestIngest,a65c,1434633226681.0927319db6bf5e128e3bec2a420819aa.',
 STARTKEY = 'a65c', ENDKEY = 'b328'}
{noformat}

Looking at the raw scan of the meta, the timestamp for the region open of the 
already merged region is 1434633226600. This is a little after the merge 
transaction's timestamp.
{noformat}
IntegrationTestIngest,acc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec. 
column=info:seqnumDuringOpen, timestamp=1434633226600, 
value=\x00\x00\x00\x00\x00\x02\xCA\xCC
 IntegrationTestIngest,acc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec. 
column=info:server, timestamp=1434633226600, value=ddas-2-5.openstacklocal:16020
 IntegrationTestIngest,acc2,1434631353820.1bdaf759862f45d133ef77fdbda21aec. 
column=info:serverstartcode, timestamp=1434633226600, value=1434632778438
{noformat}

We need to fix it so that the merge region transaction also takes the master's 
timestamp. Similar to HBASE-13875.

When this happens, clients start to see a row in the meta with an empty 
HRegionInfo (this is because the Put done during the region open only updates 
the location information but not the HRI, and the HRI deleted during the merge 
transaction remains deleted).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: HBCK question: Exception in checkRegionConsistency() would kill HBCK

[Devaraj Das]

I think it makes sense to continue to the other regions and eventually list out 
the regions that couldn't be fixed.

From: Stephen Jiang syuanjiang...@gmail.com
Sent: Wednesday, April 22, 2015 3:52 PM
To: dev@hbase.apache.org
Subject: HBCK question: Exception in checkRegionConsistency() would kill HBCK

I am looking the code at HBaseFsck#checkRegionConsistency().  It checks
region consistency and repair the corruption if requested.  However, this
function expects some exceptions.  For example, in one aspect of region
repair, it calls HBaseFsckRepair#waitUntilAssigned(), if a region is in
transition for over 120 seconds, the timeout would throw IOException.

The problem I see is that one exception in checkRegionConsistency() would
kill entire hbck operation, because the exception would propagate.

I think the better approach is to skip the troubled region and let hbck
continue to other regions.  At the end, users only has a few regions that
needs multiple runs of hbck or manual fix.  (Maybe one exception is for
meta table, if a region in meta table is not repaired successful, we should
not continue.)

How do you think?

Thanks
Stephen

[jira] [Resolved] (HBASE-10491) RegionLocations::getRegionLocation can return unexpected replica

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10491.
-
Resolution: Later

 RegionLocations::getRegionLocation can return unexpected replica
 

 Key: HBASE-10491
 URL: https://issues.apache.org/jira/browse/HBASE-10491
 Project: HBase
  Issue Type: Sub-task
Affects Versions: hbase-10070
Reporter: Sergey Shelukhin

 The method returns first non-null replica.
 If first replica is assumed to always be non-null (discussed with Enis), then 
 this code is not necessary, it should return 0th one, maybe assert it's not 
 null.
 If that is not the case, then code may be incorrect and may return 
 non-primary to some code (locateRegion overload) that doesn't expect it.
 Perhaps method should be called getAnyRegionReplica or something like that; 
 and get(Primary?)RegionLocation should return the first.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-11634) Disallow non-atomic update operations when TIMELINE consistency is enabled

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-11634.
-
Resolution: Later

Not clear what to do here. Please reopen when it is clearer.

 Disallow non-atomic update operations when TIMELINE consistency is enabled
 --

 Key: HBASE-11634
 URL: https://issues.apache.org/jira/browse/HBASE-11634
 Project: HBase
  Issue Type: Sub-task
  Components: Client, Usability
Affects Versions: 0.99.0, hbase-10070
Reporter: Nick Dimiduk

 Something to consider for the HBASE-10070 line of changes. When a client 
 reads a timeline consistent view of data, read-update-write operations are 
 potentially destructive. When a client is accepting TIMELINE reads, we should 
 only allow server-side atomic update operations.
 In a future state, we could offer a CRDT-based data type to further extend 
 the set of write operations that are guaranteed safe under this consistency 
 model.
 I may not have the semantics of the new client API correct, but this is the 
 idea: do what we can to protect users from the major dangers of this relaxed 
 consistency model.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13460) Revise the MetaLookupPool executor-related defaults (introduced in HBASE-13036)

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-13460:
---

 Summary: Revise the MetaLookupPool executor-related defaults 
(introduced in HBASE-13036)
 Key: HBASE-13460
 URL: https://issues.apache.org/jira/browse/HBASE-13460
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das
Assignee: Devaraj Das


The value of the max threads, max tasks, etc. can be improved to not 
unnecessarily reject tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13375) Provide HBase superuser higher priority over other users in the RPC handling

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-13375:
---

 Summary: Provide HBase superuser higher priority over other users 
in the RPC handling
 Key: HBASE-13375
 URL: https://issues.apache.org/jira/browse/HBASE-13375
 Project: HBase
  Issue Type: Improvement
  Components: rpc
Reporter: Devaraj Das
 Fix For: 1.1.0


HBASE-13351 annotates Master RPCs so that RegionServer RPCs are treated with a 
higher priority compared to user RPCs (and they are handled by a separate set 
of handlers, etc.). It may be good to stretch this to users too - hbase 
superuser (configured via hbase.superuser) gets higher priority over other 
users in the RPC handling. That way the superuser can always perform 
administrative operations on the cluster even if all the normal priority 
handlers are occupied (for example, we had a situation where all the master's 
handlers were tied up with many simultaneous createTable RPC calls from 
multiple users and the master wasn't able to perform any operations initiated 
by the admin). (Discussed this some with [~enis] and [~elserj]).
Does this make sense to others?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-9738) Delete table and loadbalancer interference

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-9738.

Resolution: Fixed

Thanks for checking [~mantonov]. I think this got fixed by HBASE-10349.

 Delete table and loadbalancer interference
 --

 Key: HBASE-9738
 URL: https://issues.apache.org/jira/browse/HBASE-9738
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das
Priority: Critical
 Fix For: 2.0.0, 1.1.0


 I have noticed that when the balancer is computing a plan for region moves, 
 and a delete table is issued, there is some interference.
 1. At time t1, user deleted the table.
 2. This led to the master updating the meta table to remove the line for the 
 regioninfo for a region f2a9e2e9d70894c03f54ee5902bebee6.
 {noformat}
 2013-10-04 08:42:52,495 INFO  [MASTER_TABLE_OPERATIONS-hor15n05:6-0] 
 catalog.MetaEditor: Deleted [{ENCODED = f2a9e2e9d70894c03f54ee5902bebee6, 
 NAME = 'usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6.', 
 STARTKEY = '', ENDKEY = ''}]
 {noformat}
 3. However around the same time, the balancer kicked in, and reassigned the 
 region and made it online somewhere. It didn't check the fact (nor anyone 
 else did) that the table was indeed deleted.
 {noformat}
 2013-10-04 08:42:53,215 INFO  
 [hor15n05.gq1.ygridcore.net,6,1380869262259-BalancerChore] 
 master.HMaster: balance 
 hri=usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6., 
 src=hor15n09.gq1.ygridcore.net,60020,1380869263722, 
 dest=hor15n11.gq1.ygridcore.net,60020,1380869263682
 {noformat}
 .
 {noformat}
 2013-10-04 08:42:53,592 INFO  [AM.ZK.Worker-pool2-t829] master.RegionStates: 
 Onlined f2a9e2e9d70894c03f54ee5902bebee6 on 
 hor15n11.gq1.ygridcore.net,60020,1380869263682
 {noformat}
 4. Henceforth, all the drop tables started giving warnings like
 {noformat}
 2013-10-04 08:45:17,587 INFO  [RpcServer.handler=8,port=6] 
 master.HMaster: Client=hrt_qa//68.142.246.151 delete usertable
 2013-10-04 08:45:17,631 DEBUG [RpcServer.handler=8,port=6] 
 lock.ZKInterProcessLockBase: Acquired a lock for 
 /hbase/table-lock/usertable/write-master:600
 2013-10-04 08:45:17,637 WARN  [RpcServer.handler=8,port=6] 
 catalog.MetaReader: No serialized HRegionInfo in 
 keyvalues={usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:seqnumDuringOpen/1380876173509/Put/vlen=8/mvcc=0,
  
 usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:server/1380876173509/Put/vlen=32/mvcc=0,
  
 usertable,,1380876170581.f2a9e2e9d70894c03f54ee5902bebee6./info:serverstartcode/1380876173509/Put/vlen=8/mvcc=0}
 {noformat}
 5. The create of the same table also fails since there is still state 
 (reincarnated, maybe) about the table in the master.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13330) Region left unassigned due to AM SSH each thinking the assignment would be done by the other

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-13330:
---

 Summary: Region left unassigned due to AM  SSH each thinking the 
assignment would be done by the other
 Key: HBASE-13330
 URL: https://issues.apache.org/jira/browse/HBASE-13330
 Project: HBase
  Issue Type: Bug
  Components: master, Region Assignment
Reporter: Devaraj Das
 Fix For: 1.1.0


Here is what I found during analysis of an issue. Raising this jira and a fix 
will follow.
The TL;DR of this is that the AssignmentManager thinks the 
ServerShutdownHandler would assign the region and the ServerShutdownHandler 
thinks that the AssignmentManager would assign the region. The region 
(0d6cf37c18c54c6f4744750c6a7be837) ultimately never gets assigned. Below is an 
analysis from the logs that captures the flow of events.

1. The AssignmentManager had initially assigned this region to 
dnj1-bcpc-r3n8.example.com,60020,1425598187703
2. When the master restarted it did a scan of the meta to learn about the 
regions in the cluster. It found this region being assigned to 
dnj1-bcpc-r3n8.example.com,60020,1425598187703 from the meta record.
3. However, this server (dnj1-bcpc-r3n8.example.com,60020,1425598187703) was 
not alive anymore. So, the AssignmentManager queued up a ServerShutdownHandling 
task for this (that asynchronously executes):
{noformat}
2015-03-06 14:09:31,355 DEBUG org.apache.hadoop.hbase.master.ServerManager: 
Added=dnj1-bcpc-r3n8.example.com,60020,1425598187703 to dead servers,
 submitted shutdown handler to be executed meta=false
{noformat}

4. The AssignmentManager proceeded to read the RIT nodes from ZK. It found this 
region as well:
{noformat}
2015-03-06 14:09:31,527 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Processing 0d6cf37c18c54c6f4744750c6a7be837
in state: RS_ZK_REGION_FAILED_OPEN
{noformat}

5. The region was moved to CLOSED state:
{noformat}
2015-03-06 14:09:31,527 WARN org.apache.hadoop.hbase.master.RegionStates: 
0d6cf37c18c54c6f4744750c6a7be837 moved to CLOSED on
dnj1-bcpc-r3n2.example.com,60020,1425603618259, expected 
dnj1-bcpc-r3n8.example.com,60020,1425598187703
{noformat}
Note the reference to dnj1-bcpc-r3n2.example.com,60020,1425603618259. This 
means that the region was assigned to 
dnj1-bcpc-r3n2.example.com,60020,1425603618259 but that regionserver couldn't 
open the region for some reason, and it changed the state to 
RS_ZK_REGION_FAILED_OPEN in RIT znode on ZK.

6. After that the AssignmentManager tried to assign it again. However, the 
assignment didn't happen because the ServerShutdownHandling task queued earlier 
didn't yet execute:
{noformat}
2015-03-06 14:09:31,527 INFO org.apache.hadoop.hbase.master.AssignmentManager: 
Skip assigning 
phMonthlyVersion,\x89\x80\x00\x00,1423149098980.0d6cf37c18c54c6f4744750c6a7be837.,
 it's host dnj1-bcpc-r3n8.example.com,60020,1425598187703 is dead but not 
processed yet
{noformat}

7. Eventually the ServerShutdownHandling task executed.
{noformat}
2015-03-06 14:09:35,188 INFO 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Splitting logs 
for dnj1-bcpc-r3n8.example.com,60020,1425598187703 before assignment.
2015-03-06 14:09:35,209 INFO 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Reassigning 19 
region(s) that dnj1-bcpc-r3n8.example.com,60020,1425598187703 was
 carrying (and 0 regions(s) that were opening on this server)
2015-03-06 14:09:35,211 INFO 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Finished 
processing of shutdown of dnj1-bcpc-r3n8.example.com,60020,1425598187703
{noformat}

8. However, the ServerShutdownHandling task skipped the region in question. 
This was because this region was in RIT, and the ServerShutdownHandling task 
thinks that the AssignmentManager would assign it as part of handling the RIT 
nodes:
{noformat}
2015-03-06 14:09:35,210 INFO 
org.apache.hadoop.hbase.master.handler.ServerShutdownHandler: Skip assigning 
region in transition on other server{0d6cf37c18c54c6f4744750c6a7be837
state=CLOSED, ts=1425668971527, 
server=dnj1-bcpc-r3n2.example.com,60020,1425603618259}
{noformat}

9. At some point in the future, when the server 
dnj1-bcpc-r3n2.example.com,60020,1425603618259 dies, the ServerShutdownHandling 
for it gets queued up (from the log hbase-hbase-master-dnj1-bcpc-r3n1.log):
{noformat}
2015-03-09 11:35:10,607 INFO 
org.apache.hadoop.hbase.zookeeper.RegionServerTracker: RegionServer ephemeral 
node deleted,
processing expiration [dnj1-bcpc-r3n2.example.com,60020,1425603618259]
{noformat}

10. In RegionStates.java:serverOffline, there is a check that happens on the 
state of the region's state. Since the region is in CLOSED state, the log is 
displayed:
{noformat}
2015-03-09 11:35:15,711 WARN org.apache.hadoop.hbase.master.RegionStates: THIS 
SHOULD NOT HAPPEN: unexpected {0d6cf37c18c54c6f4744750c6a7be837 state=CLOSED, 
ts=1425668971527, server=dnj1-bcpc-r3n2.example.com

[jira] [Created] (HBASE-13192) IntegrationTestBulkLoad doesn't wait for table modification sometimes leading to spurious test failures

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-13192:
---

 Summary: IntegrationTestBulkLoad doesn't wait for table 
modification sometimes leading to spurious test failures
 Key: HBASE-13192
 URL: https://issues.apache.org/jira/browse/HBASE-13192
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das
Assignee: Devaraj Das


Noticed this issue in our internal test runs. It shows up when the test is run 
with replicas turned on. After the bulk load is complete, the test tries to 
modify the table with an intention of having the replicas load the new region 
files. But due to a bug in the way the test was waiting for the modifyTable 
operation to complete, it would not actually wait. Sometimes a replica wouldn't 
have loaded the new files yet and it might happen that the 'check' step of the 
test would go to that replica asking for data. It would give empty results 
back, and the test would fail.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13067) Fix caching of stubs to allow IP address changes of restarted remote servers

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-13067:
---

 Summary: Fix caching of stubs to allow IP address changes of 
restarted remote servers
 Key: HBASE-13067
 URL: https://issues.apache.org/jira/browse/HBASE-13067
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-13036) Meta scanner should use its own threadpool

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-13036:
---

 Summary: Meta scanner should use its own threadpool
 Key: HBASE-13036
 URL: https://issues.apache.org/jira/browse/HBASE-13036
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das
Assignee: Devaraj Das


Currently, during region location lookups, the meta is scanned and for this the 
scanner submits a request to the thread pool of the table in question. 
Sometimes, it might happen that many scan requests are simultaneously submitted 
on the table - let's say 256, and this is the max size of the thread pool. Each 
of these scan requests would in turn submit meta scan requests on the same 
thread pool, but they will be in the queue. The meta scans will not happen and 
the original table scans would lock up. The meta scans should use a different 
thread pool.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-11568) Async WAL replication for region replicas

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das reopened HBASE-11568:
-

Reopening for branch-1 commit

 Async WAL replication for region replicas
 -

 Key: HBASE-11568
 URL: https://issues.apache.org/jira/browse/HBASE-11568
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Enis Soztutar
 Fix For: 2.0.0

 Attachments: hbase-11568_v2.patch, hbase-11568_v3.patch


 As mentioned in parent issue, and design docs for phase-1 (HBASE-10070) and 
 Phase-2 (HBASE-11183), implement asynchronous WAL replication from the WAL 
 files of the primary region to the secondary region replicas. 
 The WAL replication will build upon the pluggable replication framework 
 introduced in HBASE-11367, and the distributed WAL replay. 
 Upon having some experience with the patch, we changed the design so that 
 there is only one replication queue for doing the async wal replication to 
 secondary replicas rather than having a queue per region replica. This is due 
 to the fact that, we do not want to tail the logs of every region server for 
 a single region replica. 
 Handling of flushes/compactions and memstore accounting will be handled in 
 other subtasks. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HBASE-6006) TestDelayedRpc is flaky

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-6006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-6006.

Resolution: Not a Problem

Haven't seen it failing in a while now. Resolving.

 TestDelayedRpc is flaky
 ---

 Key: HBASE-6006
 URL: https://issues.apache.org/jira/browse/HBASE-6006
 Project: HBase
  Issue Type: Bug
  Components: test
Reporter: Devaraj Das

 I noticed that TestDelayedRpc is flaky - testTooManyDelayedRpcs fails quite 
 often in my setup. Seems to me that the test needs to be fixed for it to not 
 depend on the (unpredictable) thread scheduling order in the RPC 
 server/clients that the test creates.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Reopened] (HBASE-10844) Coprocessor failure during batchmutation leaves the memstore datastructs in an inconsistent state

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das reopened HBASE-10844:
-

Thinking about it (and in offline discussions with [~jeffreyz] and [~enis]), it 
seems we should remove the 'assert' from the code that causes the RegionServer 
to abort. The reason being that coprocessors could fail. The assert was 
introduced in HBASE-10514.
{noformat}
 // close each store in parallel
 for (final Store store : stores.values()) {
+  assert abort? true: store.getFlushableSize() == 0;
{noformat}
Thoughts?

 Coprocessor failure during batchmutation leaves the memstore datastructs in 
 an inconsistent state
 -

 Key: HBASE-10844
 URL: https://issues.apache.org/jira/browse/HBASE-10844
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Devaraj Das
Assignee: Devaraj Das
 Attachments: 10844-1.txt


 Observed this in the testing with Phoenix. The test in Phoenix - 
 MutableIndexFailureIT deliberately fails the batchmutation call via the 
 installed coprocessor. But the update is not rolled back. That leaves the 
 memstore inconsistent. In particular, I observed that getFlushableSize is 
 updated before the coprocessor was called but the update is not rolled back. 
 When the region is being closed at some later point, the assert introduced in 
 HBASE-10514 in the HRegion.doClose() causes the RegionServer to shutdown 
 abnormally.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12548) Improve debuggability of IntegrationTestTimeBoundedRequestsWithRegionReplicas

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-12548:
---

 Summary: Improve debuggability of 
IntegrationTestTimeBoundedRequestsWithRegionReplicas
 Key: HBASE-12548
 URL: https://issues.apache.org/jira/browse/HBASE-12548
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: 2.0.0






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12319) Inconsistencies during region recovery due to close/open of a region during recovery

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-12319:
---

 Summary: Inconsistencies during region recovery due to close/open 
of a region during recovery
 Key: HBASE-12319
 URL: https://issues.apache.org/jira/browse/HBASE-12319
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das


In one of my test runs, I saw the following:
{noformat}
2014-10-14 13:45:30,782 DEBUG [StoreOpener-51af4bd23dc32a940ad2dd5435f00e1d-1] 
regionserver.HStore: loaded 
hdfs://hor9n01.gq1.ygridcore.net:8020/apps/hbase/data/data/default/IntegrationTestIngest/51af4bd23dc32a940ad2dd5435f00e1d/test_cf/d6df5cfe15ca41d68c619489fbde4d04,
 isReference=false, isBulkLoadResult=false, seqid=141197, majorCompaction=true
2014-10-14 13:45:30,788 DEBUG [RS_OPEN_REGION-hor9n01:60020-1] 
regionserver.HRegion: Found 3 recovered edits file(s) under 
hdfs://hor9n01.gq1.ygridcore.net:8020/apps/hbase/data/data/default/IntegrationTestIngest/51af4bd23dc32a940ad2dd5435f00e1d
.
.
2014-10-14 13:45:31,916 WARN  [RS_OPEN_REGION-hor9n01:60020-1] 
regionserver.HRegion: Null or non-existent edits file: 
hdfs://hor9n01.gq1.ygridcore.net:8020/apps/hbase/data/data/default/IntegrationTestIngest/51af4bd23dc32a940ad2dd5435f00e1d/recovered.edits/0198080
{noformat}

The above logs is from a regionserver, say RS2. From the initial analysis it 
seemed like the master asked a certain regionserver to open the region (let's 
say RS1) and for some reason asked it to close soon after. The open was still 
proceeding on RS1 but the master reassigned the region to RS2. This also 
started the recovery but it ended up seeing an inconsistent view of the 
recovered-edits files (it reports missing files as per the logs above) since 
the first regionserver (RS1) deleted some files after it completed the 
recovery. When RS2 really opens the region, it might not see the recent data 
that was written by flushes on hor9n10 during the recovery process. Reads of 
that data would have inconsistencies.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-12012) Improve cancellation for the scan RPCs

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-12012:
---

 Summary: Improve cancellation for the scan RPCs
 Key: HBASE-12012
 URL: https://issues.apache.org/jira/browse/HBASE-12012
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das


Similar to HBASE-11564 but for scans.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: The 1st HBase 0.98.6.1 patch release candidate (RC0) is available, vote closing 9/16

[Devaraj Das]

+1

On Mon, Sep 15, 2014 at 8:31 AM, Andrew Purtell apurt...@apache.org wrote:
 HBASE-11972 came in after 0.98.6 was released, involving a fix
 committed to 0.98.6, where the fix is incomplete and table creation
 functionality is affected if security is active. I think it's
 necessary to push out a patch release containing the fix for
 HBASE-11972 as 0.98.6.1 and then remove the 0.98.6 artifacts. I have
 also included on more fix, for HBASE-11963, which is a rare but
 serious thread leak in replication we encountered in production at
 Salesforce last week. Both of these changes are narrowly scoped and
 should be straightforward to assess.

 The 1st HBase 0.98.6.1 patch release candidate (RC0) is available for
 download at http://people.apache.org//~apurtell/0.98.6.1RC0/ and Maven
 artifacts are also available in the temporary repository
 https://repository.apache.org/content/repositories/orgapachehbase-1038/

 Signed with my code signing key D5365CCD.

 The issues resolved in this release are:

 HBASE-11963 Synchronize peer cluster replication connection attempts
 https://issues.apache.org/jira/browse/HBASE-11963

 HBASE-11972 The doAs user used in the update to hbase:acl table
 RPC is incorrect
 https://issues.apache.org/jira/browse/HBASE-11972

 I would like to run a short vote for ~24 hours for this patch release,
 therefore requesting three PMC vote +1/-1 by midnight Pacific Time
 (00:00 -0800 GMT) on Tuesday September 16.

 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet
 Hein (via Tom White)

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Created] (HBASE-11972) The doAs user used in the update to hbase:acl table RPC is incorrect

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-11972:
---

 Summary: The doAs user used in the update to hbase:acl table RPC 
is incorrect
 Key: HBASE-11972
 URL: https://issues.apache.org/jira/browse/HBASE-11972
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das
Assignee: Devaraj Das
Priority: Critical


This is a follow-up to HBASE-11886. I missed one doAs in the patch. We 
discovered the issue in our internal testing with security ON.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HBASE-11886) The creator of the table should have all permissions on the table

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-11886:
---

 Summary: The creator of the table should have all permissions on 
the table
 Key: HBASE-11886
 URL: https://issues.apache.org/jira/browse/HBASE-11886
 Project: HBase
  Issue Type: Bug
Affects Versions: 0.98.6, 0.98.3
Reporter: Devaraj Das
Priority: Critical
 Fix For: 1.0.0


In our testing of 0.98.4 with security ON, we found that table creator doesn't 
have RWXCA on the created table. Instead, the user representing the HBase 
daemon gets all permissions. Due to this the table creator can't write to the 
table he just created. I am suspecting HBASE-11275 introduced the problem.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: ​The ​2nd HBase 0.98.6​ ​release candidate (RC​1) is available, vote closing 9/9/2014

[Devaraj Das]

I meant to send this mail earlier in the evening but wasn't able to.. Apologies.

Andrew could you take a look at HBASE-11886 and see if it should be
fixed in this release. I think it's a serious usability issue but will
defer to your call.

On Tue, Sep 2, 2014 at 10:17 PM, Andrew Purtell apurt...@apache.org wrote:

 The 2nd HBase 0.98.6 release candidate (RC1) is available for download at


 https://people.apache.org/~apurtell/0.98.6RC1/ and Maven artifacts are also
 available in the temporary repository
 https://repository.apache.org/content/repositories/orgapachehbase-1034

 Signed with my code signing key D5365CCD.

 The issues resolved in this release can be found here:
 https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12310753version=12327376

 This candidate contains a significant backported change, ZK-less
 region assignment (HBASE-11546), thanks to the heavy lifting of Virag
 Kothari (and original contributors Jimmy Xiang and Virag Kothari). The new
 code will not be active unless the configuration
 setting hbase.assignment.usezk is set to 'false'. If possible please
 consider testing with this configuration change on a cluster and/or with
 the integration tests.

 Please try out the candidate and vote +1/-1 by midnight Pacific Time (00:00
 -0800 GMT) on September 9 on whether or not we should release this as
 0.98.6. Three +1 votes from PMC will be required to release.


 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)



 --
 Best regards,

- Andy

 Problems worthy of attack prove their worth by hitting back. - Piet Hein
 (via Tom White)

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Dropping support for JDK6 in Apache Hadoop

[Devaraj Das]

I think we are good on HBase side, at least on the 1.x+ branches. We
are dropping support for 1.6 from HBase-1.0 onwards.

On Tue, Aug 19, 2014 at 10:52 AM, Arun C Murthy a...@hortonworks.com wrote:
 [Apologies for the wide distribution.]

 Dear HBase/Hive/Pig/Oozie communities,

  We, over at Hadoop are considering dropping support for JDK6 this year.

  As you maybe aware we just released hadoop-2.5.0 and are now considering 
 making the next release i.e. hadoop-2.6.0 the *last* release of Apache Hadoop 
 which supports JDK6. This means, from hadoop-2.7.0 onwards we will not 
 support JDK6 anymore and we *may* start relying on JDK7-specific apis.

  Now, the above releases a proposal and we do not want to pull the trigger 
 without talking to projects downstream - hence the request for you feedback.

  Please feel free to forward this to other communities you might deem to be 
 at risk from this too.

 thanks,
 Arun


 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the reader
 of this message is not the intended recipient, you are hereby notified that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender immediately
 and delete it from your system. Thank You.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: Failing region replica tests in trunk

[Devaraj Das]

Yeah the replica tests issues are due to turning on the zk-less
assignment by default.

If HBASE-11611 addresses the issues to do with zk-less assignments +
replicas, let's wait for that.

On Wed, Aug 6, 2014 at 3:10 PM, Stack st...@duboce.net wrote:
 Lets see if we see the above fail there where there is no zkless assign.

 Looking at HBase-1.0 builds, the replica tests seem to generally work so
 this is a master/patch build issue only.

 St.Ack


 On Wed, Aug 6, 2014 at 2:55 PM, Stack st...@duboce.net wrote:

 The following tests have been failing with regularity on patch builds:

   org.apache.hadoop.hbase.master.TestRestartCluster
   org.apache.hadoop.hbase.client.TestReplicasClient
   org.apache.hadoop.hbase.regionserver.TestRegionReplicas
   org.apache.hadoop.hbase.master.
 TestMasterOperationsForRegionReplicas

 The issue is zkless assignment and read replicas clash?  They don't work
 together?

 Whats the thoughts on getting the tests passing in master branch?  Wait on
 the patch-of-the-year HBASE-1611?

 I've just added a branch-1 build to jenkins.  Lets see if we see the above
 fail there where there is no zkless assign.
 Thanks,
 St.Ack

 P.S. https://issues.apache.org/jira/browse/BUILDS-4 is tracking the
 regular TestIPC et al. fails -- some issue in name resolve on build machine.




-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Reopened] (HBASE-11293) Master and Region servers fail to start when hbase.master.ipc.address=0.0.0.0, hbase.regionserver.ipc.address=0.0.0.0 and Kerberos is enabled

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das reopened HBASE-11293:
-


[~miharp], this is not done yet. Trunk needs to have a patch.

 Master and Region servers fail to start when 
 hbase.master.ipc.address=0.0.0.0, hbase.regionserver.ipc.address=0.0.0.0 and 
 Kerberos is enabled
 -

 Key: HBASE-11293
 URL: https://issues.apache.org/jira/browse/HBASE-11293
 Project: HBase
  Issue Type: Bug
Reporter: Michael Harp
Assignee: Devaraj Das
 Attachments: 11293-1.txt


 Setting 
 {code}
 hbase.master.ipc.address=0.0.0.0
 hbase.regionserver.ipc.address=0.0.0.0
 {code}
 causes the _HOST substitution in hbase/_h...@example.com to result in 
 hbase/0:0:0:0:0:0:0:0...@example.com which in turn causes kerberos 
 authentication to fail.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11344) Hide row keys and such from the web UIs

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-11344:
---

 Summary: Hide row keys and such from the web UIs
 Key: HBASE-11344
 URL: https://issues.apache.org/jira/browse/HBASE-11344
 Project: HBase
  Issue Type: Improvement
Reporter: Devaraj Das
 Fix For: 0.99.0


The table details on the master UI lists the start row keys of the regions. The 
row keys might have sensitive data. We should hide them based on whether or not 
the user accessing has the required authorization to view the table.. To start 
with, we could make the display of row keys and such based on a configuration 
being true or false. If it is false, such potentially sensitive data is never 
displayed on the web UI.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: [VOTE] Merge branch HBASE-10070 to trunk

[Devaraj Das]

+1

On Fri, Jun 6, 2014 at 2:13 PM, Andrew Purtell apurt...@apache.org wrote:
 +1, thanks Enis


 On Fri, Jun 6, 2014 at 1:46 PM, Enis Söztutar enis@gmail.com wrote:

 Sorry, I was mostly out for HadoopSummit.

 Yes, the git flow would be very similar to what you propose:

  $ git checkout HBASE-10070
  $ git rebase --ignore-date master
  (fixups, git add, git rebase --continue, etc, etc, etc)
  $ git checkout master

  $ git push origin HBASE-10070 HBASE-10070-rebase-date   (optionally)
  $ git merge HBASE-10070

 We can either go --ignore-date or not depending on what we want. If needed
 I am fine with pushing the rebased master branch for review to main repo
 before the merge to another branch. If not, I can just rebase the branch
 locally and merge + push to main repo.

 Creating final patches and attaching them to jira might be cumbersome. If
 we do the rebased-branch on repo, we might not need it. But if we need that
 for review, I can do it.

 Thanks,
 Enis



 On Wed, Jun 4, 2014 at 10:48 AM, Andrew Purtell apurt...@apache.org
 wrote:

  I realize this is a vote thread but I need a satisfactory answer to the
  below inquiries before feeling comfortable casting a vote. Or perhaps
 that
  means we need to cancel this vote and move back to discussion.
 
 
  On Tue, Jun 3, 2014 at 11:17 AM, Andrew Purtell apurt...@apache.org
  wrote:
 
   Also after the merge process is completed, do you plan to use git
   format-patch to break out the per-JIRA changes into updated patches for
   those JIRAs representing in effect the final commit?
  
  
   On Tue, Jun 3, 2014 at 11:16 AM, Andrew Purtell apurt...@apache.org
   wrote:
  
  
   On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar e...@apache.org
 wrote:
  
   This VOTE is for merging back the remaining changes in branch to
 trunk.
  If
   passes, we will rebase the branch on top of current trunk, in which
 we
   will
   keep the commit-per-issue log history. After that we will do a git
  merge
   for the branch keeping the history clean and not squashing the
  commits. I
   expect rebasing to be straightforward, however with some manual
  conflict
   resolution. After the merge we'll keep running the tests to make sure
   everything is ok.
  
  
   Just to clarify that would look something like this:
  
$ git checkout HBASE-10070
$ git rebase --ignore-date master
(fixups, git add, git rebase --continue, etc, etc, etc)
$ git checkout master
$ git merge HBASE-10070
  
   ?
  
   That sounds good to me, the final merge should be a fast forward
 merge.
  
   Use of ' --ignore-date' could be mildly controversial. It's not
 strictly
   necessary because the commits for 10070 will appear grouped in
 history,
  but
   then dates on commits will be discontiguous in that section of
 history.
  I
   suggest using that option so the order of commits and dates sort the
  same
   on master.
  
  
   On Mon, Jun 2, 2014 at 2:24 PM, Enis Söztutar e...@apache.org
 wrote:
  
   Hi,
  
   Last week we started some discussion[4] for merging branch
  hbase-10070[1]
   into trunk. It seems like the consensus there is to do the merge
 sooner
   rather than later.
  
  
   We had branched hbase-10070 in Feb out of trunk[5]. The branch
 contains
   55
   jiras committed[2]. Out of these 55, 15 has already been committed to
   trunk
   and backported to hbase-10070 branch[3].
  
   This VOTE is for merging back the remaining changes in branch to
 trunk.
   If
   passes, we will rebase the branch on top of current trunk, in which
 we
   will
   keep the commit-per-issue log history. After that we will do a git
  merge
   for the branch keeping the history clean and not squashing the
  commits. I
   expect rebasing to be straightforward, however with some manual
  conflict
   resolution. After the merge we'll keep running the tests to make sure
   everything is ok.
  
   An overview of the changes, and the status of the work can be found
  under
   [4], [6] and [7].In summary, with the code in branch, you can create
   tables
   with region replicas, do gets / multi gets and scans using TIMELINE
   consistency with high availability. Region replicas periodically scan
  the
   files of the primary and pick up flushed / committed files. The RPC
   paths /
   assignment, balancing etc are pretty stable. However some more
   performance
   analysis and tuning is needed. Phase 2 work is being worked on under
   HBASE-11183, and we have some working prototype for async-replicating
  and
   region splits. However, we believe even without those features, this
  work
   is useable (especially for read-only/bulk load tables) , and can be
   released as an experimental feature in 1.0.
  
   Please indicate your choice:
  
   [ ] +1 on yes, merge branch hbase-10070 to trunk.
   [ ] 0 on don't care
   [ ] -1 don't merge, because ...
  
   I'll keep the vote running for 7 days, and close it Mon 9th of June,
  PDT.
  
   Here is my 

[jira] [Created] (HBASE-11261) Handle splitting of regions that have region_replication greater than one

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-11261:
---

 Summary: Handle splitting of regions that have region_replication 
greater than one
 Key: HBASE-11261
 URL: https://issues.apache.org/jira/browse/HBASE-11261
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das






--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10818) Add integration test for bulkload with replicas

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10818.
-

Resolution: Fixed

Committed to the branch. Thanks Nick for the initial patches.

 Add integration test for bulkload with replicas
 ---

 Key: HBASE-10818
 URL: https://issues.apache.org/jira/browse/HBASE-10818
 Project: HBase
  Issue Type: Sub-task
Affects Versions: hbase-10070
Reporter: Nick Dimiduk
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10818-7.txt, HBASE-10818.00.patch, HBASE-10818.01.patch, 
 HBASE-10818.02.patch, IntegrationTestBulkLoad_replicas.log


 Should verify bulkload is not affected by region replicas.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-11214) Fixes for scans on a replicated table

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-11214?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-11214.
-

Resolution: Fixed

Committed to hbase-10070 branch. Thanks for the review, Enis.

 Fixes for scans on a replicated table
 -

 Key: HBASE-11214
 URL: https://issues.apache.org/jira/browse/HBASE-11214
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 11214-1.txt, 11214-2.txt


 During testing with the IT in HBASE-10818, found an issue to do with how 
 close of scanners was handled. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: [DISCUSSION] Update on HBASE-10070 / Merge into trunk

[Devaraj Das]

+1 for merge to trunk now.

On Wed, May 21, 2014 at 5:08 PM, Enis Söztutar e...@apache.org wrote:
 Hi,

 We would like to give an update on the status of HBASE-10070 work, and open
 up discussion for how we can do further development.

 We seem to be at a point where we have the core functionality of the
 region replica, as described in HBASE-10070 working. As pointed out
 under the section Development Phases in the design doc posted on the
 jira HBASE-10070, this work was divided into two broad phases. The first
 phase introduces region replicas concept, the new consistency model, and
 corresponding RPC implementations. All of the issues for Phase 1 can be
 found under [3]. Phase 2 is still in the works, and contains the proposed
 changes listed under [4].

 With all the issues committed in HBASE-10070 branch in svn, we think that
 the phase-1 is complete. The user documentation on HBASE-10513 gives an
 accurate picture of what has been done in phase-1 and what the impact of
 using this feature is, APIs etc. We have added
 a couple of IT tests as part of this work and we have tested the work
 we did in phase-1 of the project quite extensively in Hortonworks'
 infrastructure.

 In summary, with the code in branch, you can create tables with region
 replicas, do gets / multi gets and scans using TIMELINE consistency with
 high availability. Region replicas periodically scan the files of the
 primary and pick up flushed / committed files. The RPC paths / assignment,
 balancing etc are pretty stable. However some more performance analysis and
 tuning is needed. More information can be found in [1] and [2].


 As a reminder, the development has been happening in the branch -
 hbase-10070 (https://github.com/apache/hbase/tree/hbase-10070). We have
 been pretty diligent about more than one committer's +1 on the branch
 commits and for almost all the subtasks in HBASE-10070 there is more than
 one +1 except for test fix issues or more trivial issues, where there maybe
  only one +1.  Enis/Nicolas/Sergey/Devaraj/Nick are the main contributors
 of code in the phase-1 and many patches have been reviewed already by
 people outside
 this group (mainly Stack, Jimmy)

 For Phase 2, we think that we can deliver on the proposed design
 incrementally over the next couple of months. However, we think that it
 might be ok to merge the changes from phase 1 first, then do a
 commit-as-you-go approach for remaining items. Therefore, we would like to
 propose  to merge the branch to trunk, and continue the work over there.
 This might also result in more reviews.

 Alternatively, we can continue the work in the branch, and do the merge at
 the end of Phase 2, but that will make the review process a bit more
 tricky, which is why we would like the merge sooner.

 What do you think? Comments, concerns?

 [1]
 https://issues.apache.org/jira/secure/attachment/12644237/hbase-10513_v1.patch
 [2]
 http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
 [3] https://issues.apache.org/jira/browse/HBASE-10070
 [4] https://issues.apache.org/jira/browse/HBASE-11183

 Thanks,

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Created] (HBASE-11214) Fixes for scans on a replicated table

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-11214:
---

 Summary: Fixes for scans on a replicated table
 Key: HBASE-11214
 URL: https://issues.apache.org/jira/browse/HBASE-11214
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das


During testing with the IT in HBASE-10818, found an issue to do with how 
close of scanners was handled. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-11149) Wire encryption is broken

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-11149:
---

 Summary: Wire encryption is broken
 Key: HBASE-11149
 URL: https://issues.apache.org/jira/browse/HBASE-11149
 Project: HBase
  Issue Type: Bug
  Components: IPC/RPC
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: 0.99.0
 Attachments: 11149-1.txt

Upon some testing with the QOP configuration (hbase.rpc.protection), discovered 
that RPC doesn't work with integrity and privacy values for the 
configuration key. I was using 0.98.x for testing but I believe the issue is 
there in trunk as well (haven't checked 0.96 and 0.94).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10844) Coprocessor failure during batchmutation leaves the memstore datastructs in an inconsistent state

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10844:
---

 Summary: Coprocessor failure during batchmutation leaves the 
memstore datastructs in an inconsistent state
 Key: HBASE-10844
 URL: https://issues.apache.org/jira/browse/HBASE-10844
 Project: HBase
  Issue Type: Bug
  Components: regionserver
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: 0.98.2


Observed this in the testing with Phoenix. The test in Phoenix - 
MutableIndexFailureIT deliberately fails the batchmutation call via the 
installed coprocessor. But the update is not rolled back. That leaves the 
memstore inconsistent. In particular, I observed that getFlushableSize is 
updated before the coprocessor was called but the update is not rolled back. 
When the region is being closed at some later point, the assert introduced in 
HBASE-10514 in the HRegion.doClose() causes the RegionServer to shutdown 
abnormally.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10743) Replica map update is problematic in RegionStates

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10743:
---

 Summary: Replica map update is problematic in RegionStates
 Key: HBASE-10743
 URL: https://issues.apache.org/jira/browse/HBASE-10743
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Attachments: 10743-1.txt

In the RegionStates.regionOnline method and in the successive method calls, the 
{defaultReplicaToOtherReplicas} is updated twice in succession - one adds and 
another deletes a replica. This is a problem. The delete should not happen in 
this code path.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10616) Integration test for multi-get calls

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10616.
-

Resolution: Fixed

 Integration test for multi-get calls
 

 Key: HBASE-10616
 URL: https://issues.apache.org/jira/browse/HBASE-10616
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10616-1.txt, 10616-2.2.txt, 10616-3.txt, 10616-4.1.txt, 
 10616-4.txt


 HBASE-10572 adds a test that does 'get' from client. We should add a similar 
 test for batch gets.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10743) Replica map update is problematic in RegionStates

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10743.
-

Resolution: Fixed

Thanks for the quick review, Enis. Committed.

 Replica map update is problematic in RegionStates
 -

 Key: HBASE-10743
 URL: https://issues.apache.org/jira/browse/HBASE-10743
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10743-1.txt


 In the RegionStates.regionOnline method and in the successive method calls, 
 the {defaultReplicaToOtherReplicas} is updated twice in succession - one adds 
 and another deletes a replica. This is a problem. The delete should not 
 happen in this code path.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10729) Enable table doesn't balance out replicas evenly if the replicas were unassigned earlier

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10729:
---

 Summary: Enable table doesn't balance out replicas evenly if the 
replicas were unassigned earlier
 Key: HBASE-10729
 URL: https://issues.apache.org/jira/browse/HBASE-10729
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das


Enable table doesn't assign out replicas keeping availability in mind, if the 
replicas were unassigned before the table was disabled. For example, when a 
snapshot is restored and then the table is enabled, the replicas are unevenly 
assigned. The reason for this is that the the enable table invokes randomAssign 
that assigns one region at a time. Since the master doesn't have any 
information about the unassigned replicas, the calls to randomAssign can't do 
any availability checks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10734) Fix RegionStates.getRegionAssignments to not add duplicate regions

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10734:
---

 Summary: Fix RegionStates.getRegionAssignments to not add 
duplicate regions
 Key: HBASE-10734
 URL: https://issues.apache.org/jira/browse/HBASE-10734
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das


RegionStates.getRegionAssignments(CollectionHRI) erroneously adds the default 
replica twice to the value list in the returned map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10729) Enable table doesn't balance out replicas evenly if the replicas were unassigned earlier

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10729.
-

Resolution: Fixed

 Enable table doesn't balance out replicas evenly if the replicas were 
 unassigned earlier
 

 Key: HBASE-10729
 URL: https://issues.apache.org/jira/browse/HBASE-10729
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10729-1.txt


 Enable table doesn't assign out replicas keeping availability in mind, if the 
 replicas were unassigned before the table was disabled. For example, when a 
 snapshot is restored and then the table is enabled, the replicas are unevenly 
 assigned. The reason for this is that the the enable table invokes 
 randomAssign that assigns one region at a time. Since the master doesn't have 
 any information about the unassigned replicas, the calls to randomAssign 
 can't do any availability checks.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: Please welcome our newest committer, Mr. Honghua Feng

[Devaraj Das]

Congrats, Honghua.

On Wed, Mar 12, 2014 at 3:38 PM, Enis Söztutar e...@apache.org wrote:
 Welcome!


 On Wed, Mar 12, 2014 at 3:33 PM, Nick Dimiduk ndimi...@gmail.com wrote:

 Well done!


 On Wed, Mar 12, 2014 at 3:30 PM, Elliott Clark ecl...@apache.org wrote:

  Welcome aboard.  Congratulations
 
 
  On Wed, Mar 12, 2014 at 3:11 PM, Gary Helmling ghelml...@gmail.com
  wrote:
 
   Congrats!
  
  
   On Wed, Mar 12, 2014 at 3:06 PM, Jimmy Xiang jxi...@cloudera.com
  wrote:
  
Congrats!
   
   
On Wed, Mar 12, 2014 at 3:02 PM, Jonathan Hsieh j...@cloudera.com
   wrote:
   
 Good stuff!


 On Wed, Mar 12, 2014 at 3:00 PM, Jean-Marc Spaggiari 
 jean-m...@spaggiari.org wrote:

  Nice work! Well done budy!!! Looking forward to join the team too
  ;)
 
  JM
 
 
  2014-03-12 17:57 GMT-04:00 Jesse Yates jesse.k.ya...@gmail.com
 :
 
   Congrats!
   On Mar 12, 2014 2:55 PM, Ted Yu yuzhih...@gmail.com wrote:
  
Good job, 宏华.
   
Congratulations.
   
   
On Wed, Mar 12, 2014 at 2:52 PM, Stack st...@duboce.net
  wrote:
   
 Please welcome our latest committer, 冯宏华.  He has
 contributed
some
   sweet
 patches since he started hanging out in these parts.  Keep
 up
   the
  great
 work Honghua.

 St.Ack

   
  
 



 --
 // Jonathan Hsieh (shay)
 // HBase Tech Lead, Software Engineer, Cloudera
 // j...@cloudera.com // @jmhsieh

   
  
 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Resolved] (HBASE-10734) Fix RegionStates.getRegionAssignments to not add duplicate regions

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10734.
-

Resolution: Fixed

Committed.

 Fix RegionStates.getRegionAssignments to not add duplicate regions
 --

 Key: HBASE-10734
 URL: https://issues.apache.org/jira/browse/HBASE-10734
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10734-1.txt


 RegionStates.getRegionAssignments(CollectionHRI) erroneously adds the 
 default replica twice to the value list in the returned map.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10672) Table snapshot should handle tables whose REGION_REPLICATION is greater than one

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10672.
-

Resolution: Fixed

Committed. Thanks for the reviews.

 Table snapshot should handle tables whose REGION_REPLICATION is greater than 
 one
 

 Key: HBASE-10672
 URL: https://issues.apache.org/jira/browse/HBASE-10672
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10672-1.1.txt, 10672-1.2.txt


 If a table has more than one region replica, then snapshot utility - 
 take-snapshot, clone-snapshot etc crashes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Resolved] (HBASE-10620) LoadBalancer.needsBalance() should check for co-located region replicas as well

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10620.
-

   Resolution: Fixed
Fix Version/s: hbase-10070

Committed. Thanks for the reviews.

 LoadBalancer.needsBalance() should check for co-located region replicas as 
 well
 ---

 Key: HBASE-10620
 URL: https://issues.apache.org/jira/browse/HBASE-10620
 Project: HBase
  Issue Type: Sub-task
Reporter: Enis Soztutar
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10620-1.txt, 10620-2.txt, 10620-3.txt


 This is a left over TODO from reviews of HBASE-10351. 
 LB.needsBalance() does some basic checking before running the LB.balance() 
 method. We need to check whether there are co-located regions in this method 
 so that the balancer can increase availability. 



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10672) Table snapshot should handle tables whose REGION_REPLICATION is greater than one

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10672:
---

 Summary: Table snapshot should handle tables whose 
REGION_REPLICATION is greater than one
 Key: HBASE-10672
 URL: https://issues.apache.org/jira/browse/HBASE-10672
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das


If a table has more than one region replica, then snapshot utility - 
take-snapshot, clone-snapshot etc crashes.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10674) HBCK should be updated to do replica related checks

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10674:
---

 Summary: HBCK should be updated to do replica related checks
 Key: HBASE-10674
 URL: https://issues.apache.org/jira/browse/HBASE-10674
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das


HBCK should be updated to have a check for whether the replicas are assigned to 
the right machines (default and non-default replicas ideally should not be in 
the same server if there is more than one server in the cluster and such 
scenarios).



--
This message was sent by Atlassian JIRA
(v6.2#6252)

[jira] [Created] (HBASE-10633) StoreFileRefresherChore throws ConcurrentModificationException sometimes

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10633:
---

 Summary: StoreFileRefresherChore throws 
ConcurrentModificationException sometimes
 Key: HBASE-10633
 URL: https://issues.apache.org/jira/browse/HBASE-10633
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (HBASE-10634) Multiget doesn't fully work

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10634:
---

 Summary: Multiget doesn't fully work
 Key: HBASE-10634
 URL: https://issues.apache.org/jira/browse/HBASE-10634
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HBASE-10633) StoreFileRefresherChore throws ConcurrentModificationException sometimes

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10633.
-

   Resolution: Fixed
Fix Version/s: hbase-10070

Committed. Thanks for the quick review [~enis].

 StoreFileRefresherChore throws ConcurrentModificationException sometimes
 

 Key: HBASE-10633
 URL: https://issues.apache.org/jira/browse/HBASE-10633
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10633-1.txt






--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HBASE-10362) HBCK changes for supporting region replicas

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10362.
-

   Resolution: Fixed
Fix Version/s: (was: 0.99.0)
   hbase-10070

Thanks, guys for the review. Committed to hbase-10070 (and updated the fix 
version too, [~ndimiduk]).

 HBCK changes for supporting region replicas
 ---

 Key: HBASE-10362
 URL: https://issues.apache.org/jira/browse/HBASE-10362
 Project: HBase
  Issue Type: Sub-task
  Components: hbck
Reporter: Enis Soztutar
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10362-1.txt, 10362-2.txt, 10362-3.txt


 We should support region replicas in HBCK. The changes are probably not that 
 intrusive. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Created] (HBASE-10616) Integration test for multi-get calls

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10616:
---

 Summary: Integration test for multi-get calls
 Key: HBASE-10616
 URL: https://issues.apache.org/jira/browse/HBASE-10616
 Project: HBase
  Issue Type: Sub-task
Reporter: Devaraj Das


HBASE-10572 adds a test that does 'get' from client. We should add a similar 
test for batch gets.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

[jira] [Resolved] (HBASE-10361) Enable/AlterTable support for region replicas

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-10361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-10361.
-

   Resolution: Fixed
Fix Version/s: (was: 0.99.0)
   hbase-10070

Committed to the hbase-10070 branch. Thanks for the reviews.

 Enable/AlterTable support for region replicas
 -

 Key: HBASE-10361
 URL: https://issues.apache.org/jira/browse/HBASE-10361
 Project: HBase
  Issue Type: Sub-task
  Components: master
Reporter: Enis Soztutar
Assignee: Devaraj Das
 Fix For: hbase-10070

 Attachments: 10361-1.txt, 10361-3.txt, 10361-4-add.txt, 10361-4.txt, 
 10361-5.txt


 Add support for region replicas in master operations enable table and modify 
 table.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Re: [PROPOSAL] HBASE-10070 branch

[Devaraj Das]

Some responses inline. Thanks for the inputs.

On Wed, Jan 15, 2014 at 11:17 AM, Stack st...@duboce.net wrote:
 On Wed, Jan 15, 2014 at 12:44 AM, Enis Söztutar e...@hortonworks.comwrote:

 Hi,

 I just wanted to give some updates on the HBASE-10070 efforts from the
 technical side, and development side, and propose a branch.

 From the technical side:
 The changes for region replicas phase 1 are becoming more mature and
 stable, and most of the base changes are starting to become good
 candidates for review. The code has been rebased to trunk, and the main
 working repo has been moved to the HBASE-10070 branch at
 https://github.com/enis/hbase/tree/hbase-10070.

 An overview of the changes that is working include:
  - HRegionInfo  MetaReader  MetaEditor changes for support region
 replicas
  - HTableDescriptor changes and shell changes for supporting
 REGION_REPLICATION
  - WebUI changes to display whether a region is a replica or not
  - AssignmentManager changes coupled with RegionStates  Master changes to
 create and assign replicas, alter table, enable table, etc support.



 Thanks for the writeup.

 I am late to the game so take my comments w/ a grain of salt -- I'll take a
 look at HBASE-10070 -- but high-level do we have to go the read replicas
 route?  IMO, having our current already-strained AssignmentManager code
 base manage three replicas instead of one will ensure that Jimmy Xiang and
 Jeffrey Zhong do nothing else for the next year or two but work on the new
 interesting use cases introduced by this new level of complexity put upon a
 system that has just achieved a hard-won stability.


Stack, the model is that the replicas (HRegionInfo with an added field
'replicaId') are treated just as any other region in the AM. You can
see the code - it's not adding much at all in terms of new code to
handle replicas.

 A few of us chatting offline -- Jimmy, Jon, Elliott, and I -- were
 wondering if you couldn't solve this read replicas in a more hbase 'native'
 way* by just bringing up three tables -- a main table and then two snapshot
 clones with the clones refreshed on a period (via snapshot or via
 in-cluster replication) --  and then a shim on top of an HBase client would
 read from the main table until failure and then from a snapshot until the
 main came back.  Reads from snapshot tables could be marked 'stale'.  You'd
 have to modify the balancer so the tables -- or at least their regions --
 were physically distinct... you might be able just have the three tables
 each in a different namespace.


At a high level, considering all the work that would be needed in the
client (for it to be able to be aware of the primary and the snapshot
regions) and in the master (to do with managing the placements of the
regions), I am not convinced. Also, consider that you will be taking a
lot of snapshots and adding to the filesystem's load for the file
creations.

 Or how much more work would it take to follow the route our Facebook
 brothers and sisters have taken doing quorum reads and writes incluster?


If you talking about Facebook's work that is talked about in
HBASE-7509, the quorum reads is something that we will benefit from,
and that will help the filesystem side of the story, but we still need
multiple (redundant) regions for the hbase side. If a region is not
reachable, the client could go to another replica for the region...

 * When I say 'native' way in the above, what I mean by this is that HBase
 has always been about giving clients a 'consistent' view -- at least when
 the query is to the source cluster.  Introducing talk and APIs that talk of
 'eventual consistency' muddies our story.



As we have discussed in the jira, there are use cases. And it's
optional - all the APIs provide 'consistency' by default (status quo).


 These are some of the remaining things that we are currently working on:
  - RPC failover support for multi-gets
  - RPC failover support for scans
  - RPC cancellation


 This all sounds great.  I was sort of hoping we wouldn't have to do stuff
 like cancellation ourselves though.  Was hoping we could take on an already
 done 'rpc' engine that did this kind of stuff for us.

 ...



 Development side:
 As discussed in the issue design doc

 https://issues.apache.org/jira/secure/attachment/12616659/HighAvailabilityDesignforreadsApachedoc.pdf
 Apache
 code development process section, at this time we would like to
 propose:
  (1) Creation of HBASE-10070 branch in svn which will be a fork of trunk as
 of the date branch is created. All of the target authors (me, Devaraj,
 Nicolas, Sergey) are already committers. I do not remember whether our
 bylaws require votes on creating branches.


 We don't have bylaws.  It is my understanding that any committer can freely
 make branches and I see nothing wrong w/ this.



Great


  (2) The branch will only contain commits that have been reviewed and +1'ed
 from 2 other committers other than the patch author. Every commit in this
 

Re: [PROPOSAL] HBASE-10070 branch

[Devaraj Das]

On Wed, Jan 15, 2014 at 4:43 PM, Elliott Clark ecl...@apache.org wrote:
 On Wed, Jan 15, 2014 at 3:57 PM, Enis Söztutar enis@gmail.com wrote:

 I am afraid, it is not coprocessors or current set of plugins only. We need
 changes in the
 RPC, meta, region server, LB and master. Since we cannot easily get hooks
 into all these in
 a clean manner, implementing this purely outside would be next to
 impossible.


 I'm pretty unconvinced that this is the correct way forward.  It seems to
 introduce a lot of risk without a lot of gain.  Right now to me the 100%
 correct way forward is through paxos.  That's a lot of work but it has the
 most payoff in the end.  It will allow much faster recovery, much easier
 read sharding, it allows the greatest flexibility on IO.


Elliott, if I am not mistaken, we will need the replica management
work for the Paxos case as well. A lot of the work done in HBASE-10070
(to start with, the master/loadbalancer side of the region-replica
management work) would be leveraged if we choose to implement Paxos.

 On the other end of the spectrum is something like MySQL/Postgres read
 slaves (either tables or clusters).  Read slaves built on top of what's
 currently there seem to give all of the benefits of read slaves built into
 the current HBase without all of the risk. Sharding on top of the already
 built datastore is a pretty well known and well understood problem.  There
 are lots of great example of making this scale to pretty insane heights.
  You lose very little flexibility and incur almost not risk to the
 stability of HBase.

We have gone over this point before. We are trying to address the
issue within a single cluster. We don't want to create more storage
overhead if we can help it (which we would have if we did
intra-cluster replication).
Again the default behavior of single replica per region, etc. is kept
intact. This should be true even from the stability point of view.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Created] (HBASE-10333) Assignments are not retained on a cluster start

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-10333:
---

 Summary: Assignments are not retained on a cluster start
 Key: HBASE-10333
 URL: https://issues.apache.org/jira/browse/HBASE-10333
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das


When a cluster is fully shutdown and then started up again with 
hbase.master.startup.retainassign set to true, I noticed that the assignments 
are not retained. Upon digging, it seems like HBASE-10101 made a change due to 
which the server holding the META previously is added to dead-servers (in 
_HMaster.assignMeta_). Later on, this makes the AssignmentManager think that 
the master recovered from a failure as opposed to a fresh cluster start (the 
ServerManager.deadServers list is not empty in the check within 
_AssignmentManager.processDeadServersAndRegionsInTransition_)



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Re: Integration Test Slow Down

[Devaraj Das]

Elliott, do you know how much it takes without chaosmonkey (no failures at
all). That would be another good data point I think.


On Thu, Dec 5, 2013 at 2:52 PM, Elliott Clark ecl...@apache.org wrote:

 So I went and checked on the run times of Cloudera's internal integration
 test jenkins runs, and I noticed that recently the
 IntegrationTestBigLinkedList runs started running much slower.

 With the slow monkey:
 5 hr 58 min - 11 hr

 With the calm monkey:
 4 hr 36 min - 6 hr 24 min

 IntegrationTestBulkLoad also has a slow down but it's much less dramatic

 22 min - 26 min

 This slow down doesn't seem to manifest itself with TestLoadAndVerify at
 all.  But it has been pretty consistent since the build containing the
 following commits:

 Changes
 HBASE-9949 Fix the race condition between Compaction and StoreScanner.init
 HBASE-9949 Addendum removes readpoint parameter for getScanner()
 HBASE-9949 Revert due to TestHRegion failure
 HBASE-9869 Optimize HConnectionManager#getCachedLocation
 HBASE-9983 Lower the memory footprint of HRegionLocation
 HBASE-9869 Optimize HConnectionManager#getCachedLocation - remove
 SoftValueSortedMap


 To me the first three commits look like they cancel each so that leaves the
 last three commits as the most likely culprits.  Does anyone have a thought
 on whey these would be causing problems with ITBLL ?


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [Shadow Regions / Read Replicas ] Wal per region?

[Devaraj Das]

On Mon, Dec 2, 2013 at 10:20 PM, Jonathan Hsieh j...@cloudera.com wrote:

  Deveraj:
  Jonathan Hsieh, WAL per region (WALpr) would give you the locality (and
 hence HDFS short
  circuit) of reads if you were to couple it with the favored nodes. The
 cost is of course more WAL
  files... In the current situation (no WALpr) it would create quite some
 traffic cross machine, no?

 I think we all agree that wal per region isn't efficient on today's
 spinning hard drive world where we are limited to a relatively low budget
 or seeks (though may be better in the future with SSD's).


Yes, agree on this. My thought is also in the direction that with SSDs the
issue of many files per RS would be hopefully not an issue.


 With this in mind, I actually I making the case that we would group the all
 the regions from RS-A onto the same set of preferred regions servers.  This
 way we only need to have one or two other RS's tailing the RS.

 So for example, if region X, Y and Z were on RS-A and its hlog, the shadow
 region memstores for X, Y, and Z would be assigned to the same one or two
 other RSs.  Ideally this would be where the HLog files replicas have
 locality (helped by favored nodes/block affinity).  Doing this, we hold the
 number of readers on the active hlogs to a constant number, do not add any
 new cross machine traffic (though tailing currently has costs on the NN).


Yes, we did consider this but the issue is how much complex would the
failure handling be in order to maintain the grouping of the regions. So,
for example, if RS-A goes down, would the master be able to choose another
RS-A' quickly to maintain the grouping of the regions in RS-A. Or do we
then fallback to the regular single-region assignments and have the
balancer group the regions back... What's the grouping size. The same
issues apply to the assignments of the shadows.

But having said that, I agree that if the WALpr is expensive on non-SSD
hardware for example, we need to address the grouping of regions issues.


 One inefficiency we have is that if there is a single log per RS, we end up
 reading all the logs to tables that may not have the shadow feature
 enabled.  However, with HBase multi-wals coming, one strategy is to shard
 wals to a number on the order of the number of disks on a machine (12-24
 these days).  I think the a wal per namespaces (this could be used to have
 a wal per table) of the hlog would make sense.  This way of shardind the
 hlog would reduce the amount of reading of irrelevant log entries on a log
 tailing scheme. It would have the added benefit of reducing the log
 splitting work reducing MTTR and allowing for recovery priorities if the
 primaries and shadows also go down.  (this is an generalization of the
 separate out the META into a separate log idea).

 Jon.

 --
 // Jonathan Hsieh (shay)
 // Software Engineer, Cloudera
 // j...@cloudera.com


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [Shadow Regions / Read Replicas ]

[Devaraj Das]

On Tue, Dec 3, 2013 at 11:07 AM, Enis Söztutar e...@apache.org wrote:

 Thanks Jon for bringing this to dev@.


 On Mon, Dec 2, 2013 at 10:01 PM, Jonathan Hsieh j...@cloudera.com wrote:

  Fundamentally, I'd prefer focusing on making HBase HBasier instead of
  tackling a feature that other systems architecturally can do better
  (inconsistent reads).   I consider consistent reads/writes being one of
  HBase's defining features. That said, I think read replicas makes sense
 and
  is a nice feature to have.
 

 Our design proposal has a specific use case goal, and hopefully we can
 demonstrate the
 benefits of having this in HBase so that even more pieces can be built on
 top of this. Plus I imagine this will
 be a widely used feature for read-only tables or bulk loaded tables. We are
 not
 proposing of reworking strong consistency semantics or major architectural
 changes. I think by
 having the tables to be defined with replication count, and the proposed
 client API changes (Consistency definition)
 plugs well into the HBase model rather well.



The good part is that the proposed architecture and the underlying
implementation can be extended to provide strong consistency semantics in
the presence of shadows. I guess much of the work that is being proposed
here would be needed even there.


 
  A few thoughts:
  - Both approaches need to have more failure handling cases thought out.
 

 Agreed. We cover most of the assignment / master side of things in the doc
 though.


  - I think the two goals are both worthy on their own each with their own
  optimal points.  We should in the design makes sure that we can support
  both goals.
 

 I think our proposal is consistent with your doc, and we have considered
 secondary region promotion
 in the future section. It would be good if you can review and comment on
 whether you see any points
 missing.


  - I want to making sure the proposed design have a path for optimal
  fast-consistent read-recovery.
 

 We think that it is, but it is a secondary goal for the initial work. I
 don't see any reason why secondary
 promotion cannot be build on top of this, once the branch is in a better
 state.


 
  Jon.
 
 
  On Mon, Dec 2, 2013 at 9:54 PM, Jonathan Hsieh j...@cloudera.com wrote:
 
   HBASE-10070 [1]  looks to be heading into a discussion more apt for the
   mailing list than in the jira. Moving this to the dev list for threaded
   discussion.  I'll start a few threads by replying to this thread with
   edited titles
  
   [1] https://issues.apache.org/jira/browse/HBASE-10070
  
   --
   // Jonathan Hsieh (shay)
   // Software Engineer, Cloudera
   // j...@cloudera.com
  
  
 
 
 
  --
  // Jonathan Hsieh (shay)
  // Software Engineer, Cloudera
  // j...@cloudera.com
 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [Shadow Regions / Read Replicas ]

[Devaraj Das]

On Tue, Dec 3, 2013 at 12:31 PM, Vladimir Rodionov
vladrodio...@gmail.comwrote:

 The downside:

 - Double/Triple memstore usage
 - Increased block cache usage (effectively, block cache will have 50%
 capacity may be less)

 These downsides are pretty serious ones. This will result:

 1. in decreased overall performance due to decreased efficient block cache
 size
  2. In more frequent memstore flushes - this will affect compaction and
 write tput.


The thing is that this is configurable on a per table basis. Depending on
the hardware characteristics one may choose to not have more than one
replica per region.. Certain classes of applications + cluster combination
can still benefit from this.



 I do not believe that  HBase 'large' MTTR does not allow to meet 99% SLA.
 of 10-20ms unless your RSs go down 2-3 times a day for several minutes each
 time. You have to analyze first why are you having so frequent failures,
 than fix the root source of the problem. Its possible to reduce 'detection'
 phase in MTTR process to couple seconds either by using external beacon
 process (as I suggested already) or by rewriting some code inside HBase and
 NameNode to move all data out from Java heap to off-heap and reducing
 GC-induced timeouts from 30 sec to 1-2 sec max. Its tough, but doable. The
 result: you will decrease MTTR by 50% at least w/o sacrificing the overall
 cluster performance.

 I think, its RS and NN large heaps   and frequent s-t-w GC  activities
 prevents meeting strict SLA - not occasional server failures.



Possibly. Better MTTR and handling of GC issues will continue - no doubt.
But still there is that window of time when certain regions are
unavailable. We want to address that for applications that can tolerate
eventual consistency.



 On Tue, Dec 3, 2013 at 11:51 AM, Jonathan Hsieh j...@cloudera.com wrote:

  To keep the discussion focused on the design goals, I'm going start
  referring to enis and deveraj's eventually consistent read replicas as
 the
  *read replica* design, and consistent fast read recovery mechanism based
 on
  shadowing/tailing the wals as *shadow regions* or *shadow memstores*.
  Can
  we agree on nomenclature?
 
 
  On Tue, Dec 3, 2013 at 11:07 AM, Enis Söztutar e...@apache.org wrote:
 
   Thanks Jon for bringing this to dev@.
  
  
   On Mon, Dec 2, 2013 at 10:01 PM, Jonathan Hsieh j...@cloudera.com
  wrote:
  
Fundamentally, I'd prefer focusing on making HBase HBasier instead
 of
tackling a feature that other systems architecturally can do better
(inconsistent reads).   I consider consistent reads/writes being one
 of
HBase's defining features. That said, I think read replicas makes
 sense
   and
is a nice feature to have.
   
  
   Our design proposal has a specific use case goal, and hopefully we can
   demonstrate the
   benefits of having this in HBase so that even more pieces can be built
 on
   top of this. Plus I imagine this will
   be a widely used feature for read-only tables or bulk loaded tables. We
  are
   not
   proposing of reworking strong consistency semantics or major
  architectural
   changes. I think by
   having the tables to be defined with replication count, and the
 proposed
   client API changes (Consistency definition)
   plugs well into the HBase model rather well.
  
  
  I do agree think that without any recent updating mechanism, we are
  limiting this usefulness of this feature to essentially *only* the
  read-only or bulk load only tables.  Recency if there were any
  edits/updates would be severely lagging (by default potentially an hour)
  especially in cases where there are only a few edits to a primarily bulk
  loaded table.  This limitation is not mentioned in the tradeoffs or
  requirements (or a non-requirements section) definitely should be listed
  there.
 
  With the current design it might be best to have a flag on the table
 which
  marks it read-only or bulk-load only so that it only gets used by users
  when the table is in that mode?  (and maybe an escape hatch for power
  users).
 
  [snip]
  
   - I think the two goals are both worthy on their own each with their
 own
optimal points.  We should in the design makes sure that we can
 support
both goals.
   
  
   I think our proposal is consistent with your doc, and we have
 considered
   secondary region promotion
   in the future section. It would be good if you can review and comment
 on
   whether you see any points
   missing.
  
  
   I definitely will. At the moment, I think the hybrid for the
 wals/hlogs I
  suggested in the other thread seems to be an optimal solution considering
  locality.  Though feasible is obviously more complex than just one
 approach
  alone.
 
 
- I want to making sure the proposed design have a path for optimal
fast-consistent read-recovery.
   
  
   We think that it is, but it is a secondary goal for the initial work. I
   don't see any reason why secondary
   promotion cannot be build on top of 

Re: [Shadow Regions / Read Replicas ]

[Devaraj Das]

On Tue, Dec 3, 2013 at 6:47 PM, Jonathan Hsieh j...@cloudera.com wrote:

 On Tue, Dec 3, 2013 at 2:04 PM, Enis Söztutar enis@gmail.com wrote:

  On Tue, Dec 3, 2013 at 11:51 AM, Jonathan Hsieh j...@cloudera.com
 wrote:
   
   On Tue, Dec 3, 2013 at 11:07 AM, Enis Söztutar e...@apache.org
 wrote:
  
Thanks Jon for bringing this to dev@.
   
   
On Mon, Dec 2, 2013 at 10:01 PM, Jonathan Hsieh j...@cloudera.com
   wrote:
   
 Fundamentally, I'd prefer focusing on making HBase HBasier
 instead
  of
 tackling a feature that other systems architecturally can do better
 (inconsistent reads).   I consider consistent reads/writes being
 one
  of
 HBase's defining features. That said, I think read replicas makes
  sense
and
 is a nice feature to have.

   
Our design proposal has a specific use case goal, and hopefully we
 can
demonstrate the
benefits of having this in HBase so that even more pieces can be
 built
  on
top of this. Plus I imagine this will
be a widely used feature for read-only tables or bulk loaded tables.
 We
   are
not
proposing of reworking strong consistency semantics or major
   architectural
changes. I think by
having the tables to be defined with replication count, and the
  proposed
client API changes (Consistency definition)
plugs well into the HBase model rather well.
   
   
   I do agree think that without any recent updating mechanism, we are
   limiting this usefulness of this feature to essentially *only* the
   read-only or bulk load only tables.  Recency if there were any
   edits/updates would be severely lagging (by default potentially an
 hour)
   especially in cases where there are only a few edits to a primarily
 bulk
   loaded table.  This limitation is not mentioned in the tradeoffs or
   requirements (or a non-requirements section) definitely should be
 listed
   there.
  
 
  Obviously the amount of lag you would observe depends on whether you are
  using
  Region snapshots, WAL-Tailing or Async wal replication. I think
 there
  are still
  use cases where you can live with 1 hour old stale reads, so that
 Region
  snapshots
  is not *just* for read-only tables. I'll add these to the tradeoff's
  section.
 

 Thanks for adding it there -- I really think it is a big headline caveat on
 my expectation of eventual consistency.  Other systems out there that
 give you eventually consistency on the millisecond level for most cases,
 while this initial implementation would has eventual mean 10's of minutes
 or even handfuls of minutes behind (with the snapshots flush mechanism)!


But that's just how the implementation is broken up currently. When WAL
tailing is implemented, we will be close, maybe, in the order of seconds
behind.


 There are a handful of other things in the phase one part of the
 implementation section that limit the usefulness of the feature to a
 certain kind of constrained hbase user.  I'll start another thread for
 those.


Cool. The one thing I just realized is that we might have some additional
work to handle security issues for the shadow regions.



 
  We are proposing to implement Region snapshots first and Async wal
  replication second.
  As argued, I think wal-tailing only makes sense with WALpr so, that work
 is
  left until after we have WAL
  per region.
 
 
 This is our main disagreement -- I'm not convinced that wal tailing only
 making sense for the wal per region hlog implementation.  Instead of
 bouncing around hypotheticals, it sounds like I'll be doing more
 experiments to prove it to myself and to convince you. :)



Thanks :-) Async WAL replication approach outlined in the doc does not
require WALpr and also has the advantage that the source itself can direct
the edits to specific other regionservers hosting the replicas in question.


 
  
   With the current design it might be best to have a flag on the table
  which
   marks it read-only or bulk-load only so that it only gets used by users
   when the table is in that mode?  (and maybe an escape hatch for power
   users).
  
 
  I think we have a read-only flag already. We might not have bulk-load
 only
  flag though. Makes sense to add it
  if we want to restrict allowing bulk loads but preventing writes.
 
  Great.

 
  
   [snip]
   
- I think the two goals are both worthy on their own each with their
  own
 optimal points.  We should in the design makes sure that we can
  support
 both goals.

   
I think our proposal is consistent with your doc, and we have
  considered
secondary region promotion
in the future section. It would be good if you can review and comment
  on
whether you see any points
missing.
   
   
I definitely will. At the moment, I think the hybrid for the
  wals/hlogs I
   suggested in the other thread seems to be an optimal solution
 considering
   locality.  Though feasible is obviously more complex than just one
  approach
   alone.
  
 

Re: [Shadow Regions / Read Replicas ] External replication disqualified?

[Devaraj Das]

On Tue, Dec 3, 2013 at 6:49 PM, Jonathan Hsieh j...@cloudera.com wrote:

 The read replicas doc mentions something a little more intrusive in the 3
 options section but doesn't seem to disqualify it.


I don't quite see what you are referring to actually... Can you please
copy-paste a relevant line from the design doc.


 Relatedly just as another strawman, for the mostly read only use case and
 bulk load only usecases, why not use normal replication against two
 clusters in the same HDFS / datacenter and add a bulk load replication
 feature?


We considered this and the issue is that the resource usage on the HDFS
would be doubled (for the store files) for the two replica case.


 We'd get latency in the seconds (closer to my expected definition of
 eventual consistency)

 Jon

 On Tue, Dec 3, 2013 at 6:47 PM, Jonathan Hsieh j...@cloudera.com wrote:

 
 
  On Tue, Dec 3, 2013 at 2:04 PM, Enis Söztutar enis@gmail.com
 wrote:
 
  On Tue, Dec 3, 2013 at 11:51 AM, Jonathan Hsieh j...@cloudera.com
  wrote:
   
   On Tue, Dec 3, 2013 at 11:07 AM, Enis Söztutar e...@apache.org
 wrote:
  
Thanks Jon for bringing this to dev@.
   
   
On Mon, Dec 2, 2013 at 10:01 PM, Jonathan Hsieh j...@cloudera.com
   wrote:
   
 Fundamentally, I'd prefer focusing on making HBase HBasier
  instead of
 tackling a feature that other systems architecturally can do
 better
 (inconsistent reads).   I consider consistent reads/writes being
  one of
 HBase's defining features. That said, I think read replicas makes
  sense
and
 is a nice feature to have.

   
Our design proposal has a specific use case goal, and hopefully we
 can
demonstrate the
benefits of having this in HBase so that even more pieces can be
  built on
top of this. Plus I imagine this will
be a widely used feature for read-only tables or bulk loaded tables.
  We
   are
not
proposing of reworking strong consistency semantics or major
   architectural
changes. I think by
having the tables to be defined with replication count, and the
  proposed
client API changes (Consistency definition)
plugs well into the HBase model rather well.
   
   
   I do agree think that without any recent updating mechanism, we are
   limiting this usefulness of this feature to essentially *only* the
   read-only or bulk load only tables.  Recency if there were any
   edits/updates would be severely lagging (by default potentially an
 hour)
   especially in cases where there are only a few edits to a primarily
 bulk
   loaded table.  This limitation is not mentioned in the tradeoffs or
   requirements (or a non-requirements section) definitely should be
 listed
   there.
  
 
  Obviously the amount of lag you would observe depends on whether you are
  using
  Region snapshots, WAL-Tailing or Async wal replication. I think
  there
  are still
  use cases where you can live with 1 hour old stale reads, so that
 Region
  snapshots
  is not *just* for read-only tables. I'll add these to the tradeoff's
  section.
 
 
  Thanks for adding it there -- I really think it is a big headline caveat
  on my expectation of eventual consistency.  Other systems out there
 that
  give you eventually consistency on the millisecond level for most cases,
  while this initial implementation would has eventual mean 10's of minutes
  or even handfuls of minutes behind (with the snapshots flush mechanism)!
 
  There are a handful of other things in the phase one part of the
  implementation section that limit the usefulness of the feature to a
  certain kind of constrained hbase user.  I'll start another thread for
  those.
 
 
 
  We are proposing to implement Region snapshots first and Async wal
  replication second.
  As argued, I think wal-tailing only makes sense with WALpr so, that work
  is
  left until after we have WAL
  per region.
 
 
  This is our main disagreement -- I'm not convinced that wal tailing only
  making sense for the wal per region hlog implementation.  Instead of
  bouncing around hypotheticals, it sounds like I'll be doing more
  experiments to prove it to myself and to convince you. :)
 
 
 
  
   With the current design it might be best to have a flag on the table
  which
   marks it read-only or bulk-load only so that it only gets used by
 users
   when the table is in that mode?  (and maybe an escape hatch for
 power
   users).
  
 
  I think we have a read-only flag already. We might not have bulk-load
 only
  flag though. Makes sense to add it
  if we want to restrict allowing bulk loads but preventing writes.
 
  Great.
 
 
  
   [snip]
   
- I think the two goals are both worthy on their own each with their
  own
 optimal points.  We should in the design makes sure that we can
  support
 both goals.

   
I think our proposal is consistent with your doc, and we have
  considered
secondary region promotion
in the future section. It would be good if you can review and
 

[jira] [Created] (HBASE-9866) Support the mode where REST server authorizes proxy users

[Devaraj Das (JIRA)]

Devaraj Das created HBASE-9866:
--

 Summary: Support the mode where REST server authorizes proxy users
 Key: HBASE-9866
 URL: https://issues.apache.org/jira/browse/HBASE-9866
 Project: HBase
  Issue Type: Improvement
Reporter: Devaraj Das
Assignee: Devaraj Das
 Fix For: 0.96.1


In one use case, someone was trying to authorize with the REST server as a 
proxy user. That mode is not supported today. 
The curl request would be something like (assuming SPNEGO auth) - 
{noformat}
curl -i --negotiate -u : http://HOST:PORT/version/cluster?doas=USER
{noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

[jira] [Resolved] (HBASE-9777) Two consecutive RS crashes could lead to their SSH stepping on each other's toes and cause master abort

[Devaraj Das (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HBASE-9777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Devaraj Das resolved HBASE-9777.


Resolution: Duplicate

HBASE-9773 should fix this. Resolving.

 Two consecutive RS crashes could lead to their SSH stepping on each other's 
 toes and cause master abort
 ---

 Key: HBASE-9777
 URL: https://issues.apache.org/jira/browse/HBASE-9777
 Project: HBase
  Issue Type: Bug
Reporter: Devaraj Das

 Here is the sequence of events (with a version of 0.96 very close to RC5 
 version created on 10/11):
 1. Master assigns regions to some server RS1. One particular region is 
 300d71b112325d43b99b6148ec7bc5b3
 2. RS1 crashes
 3. Master tries to bulk-reassign (this has retries as well) the regions to 
 other RSs. Let's say one of them is RS2.
 {noformat}
 2013-10-14 21:16:22,218 INFO  
 [hor13n02.gq1.ygridcore.net,6,1381784464025-GeneralBulkAssigner-0] 
 master.RegionStates: Transitioned {300d71b112325d43b99b6148ec7bc5b3 
 state=OFFLINE, ts=1381785382125, server=null} to 
 {300d71b112325d43b99b6148ec7bc5b3 state=PENDING_OPEN, ts=1381785382218, 
 server=hor13n04.gq1.ygridcore.net,60020,1381784772417}
 {noformat}
 4. RS2 crashes
 5. The ServerShutdownHandler for RS2 is executed, and it tries to reassign 
 the regions.
 {noformat}
 2013-10-14 21:16:32,185 INFO  [MASTER_SERVER_OPERATIONS-hor13n02:6-3] 
 master.RegionStates: Found opening region {300d71b112325d43b99b6148ec7bc5b3 
 state=PENDING_OPEN, ts=1381785382218, 
 server=hor13n04.gq1.ygridcore.net,60020,1381784772417} to be reassigned by 
 SSH for hor13n04.gq1.ygridcore.net,60020,1381784772417
 {noformat}
 6. (5) succeeds. The region states are made OPEN.
 7. The retry from (3) kicks in 
 {noformat}
 2013-10-14 21:16:22,222 INFO  [MASTER_SERVER_OPERATIONS-hor13n02:6-1] 
 master.GeneralBulkAssigner: Failed assigning 52 regions to server 
 hor13n04.gq1.ygridcore.net,60020,1381784772417, reassigning them
 {noformat}
 8. The retry finds some region state as OPEN, and the master aborts with the 
 stack trace:
 {noformat}
 2013-10-14 21:16:34,342 FATAL AM.-pool1-t46 master.HMaster: Unexpected state :
 {300d71b112325d43b99b6148ec7bc5b3 state=OPEN, ts=1381785392864, 
 server=hor13n08.gq1.ygridcore.net,60020,1381785385596} .. Cannot transit it 
 to OFFLINE.
 java.lang.IllegalStateException: Unexpected state : 
 {300d71b112325d43b99b6148ec7bc5b3 state=OPEN, ts=1381785392864, 
 server=hor13n08.gq1.ygridcore.net,60020,1381785385596}
 .. Cannot transit it to OFFLINE.
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.setOfflineInZooKeeper(AssignmentManager.java:2074)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1855)
 at 
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1449)
 at org.apache.hadoop.hbase.master.AssignCallable.call(AssignCallable.java:45)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
 at java.util.concurrent.FutureTask.run(FutureTask.java:138)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
 at java.lang.Thread.run(Thread.java:662)
 {noformat}



--
This message was sent by Atlassian JIRA
(v6.1#6144)