Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

or people to use (and to free up our
> strained
> > > dev
> > > > > > >>>>> resources), but what good is some feature if the docs are
> > > > > > >>>>> missing/incomplete?
> > > > > > >>>>>
> > > > > > >>>>> I think I could stomach the docs being inaccurate (with a
> > clear
> > > > > > >>>>> disclaimer
> > > > > > >>>>> that the chapter is incomplete -- that's a 5min task).
> But, I
> > > > > think I
> > > > > > >>>>> need
> > > > > > >>>>> an answer about how the feature handles our common dist-sys
> > > > > category
> > > > > > of
> > > > > > >>>>> problems before I can consider whether I'm ok with the
> > feature
> > > > > > hitting
> > > > > > >>>>> 2.0...
> > > > > > >>>>>
> > > > > > >>>>> I'll also try to throw up a few nodes and play with it to
> > > address
> > > > > the
> > > > > > >>>>> problem as an (ignorant) user ;)
> > > > > > >>>>>
> > > > > > >>>>>
> > > > > > >>>>> Andrew Purtell wrote:
> > > > > > >>>>>
> > > > > > >>>>> I don't like that issues were identified as "blockers" but
> > now
> > > > > there
> > > > > > is
> > > > > > >>>>>
> > > > > > >>>>>> an
> > > > > > >>>>>> attempt to walk that back.
> > > > > > >>>>>>
> > > > > > >>>>>> I don't like that development of this feature has lingered
> > > for a
> > > > > > long
> > > > > > >>>>>> time
> > > > > > >>>>>> in this unfinished state when this work could have been
> done
> > > by
> > > > > now,
> > > > > > >>>>>> now
> > > > > > >>>>>> that we are trying to get a 2.0 out the door. Because this
> > is
> > > a
> > > > > > >>>>>> volunteer
> > > > > > >>>>>> project I cannot make any demand that it should be done,
> > but I
> > > > can
> > > > > > >>>>>> certainly look at the current state and be nonplussed.
> This
> > > will
> > > > > be
> > > > > > >>>>>> yet
> > > > > > >>>>>> another half finished thing in 2.0 when this merge
> happens.
> > > > > Promises
> > > > > > >>>>>> to
> > > > > > >>>>>> finish the unfinished work are nice but not currency.
> > Commits
> > > > are
> > > > > > >>>>>> currency.
> > > > > > >>>>>> I hope at least the fault tolerance changes can be
> completed
> > > and
> > > > > > >>>>>> committed
> > > > > > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release
> > to
> > > > slip
> > > > > > >>>>>> further.
> > > > > > >>>>>>
> > > > > > >>>>>> Also, marking something experimental should be done on the
> > > > merits
> > > > > of
> > > > > > >>>>>> that
> > > > > > >>>>>> evaluation, not simply to justify dropping unfinished work
> > > into
> > > > a
> > > > > > >>>>>> release
> > > > > > >>>>>> branch.
> > > > > > >>>>>>
> > > > > > >>>>>> I will change my vote to -0.
> > > > > > >>>>>>
> > > > > > >>>>>>
> > > > > > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<
> > > e...@apache.org>
> > > > > > >>>>>>   wrote:
> > > > > 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Stack]

r convenience.
> > > > > >>>>
> > > > > >>>> Once this work is merged in, how is HBASE-15227 not a blocker
> > for
> > > > 2.0?
> > > > > >>>> Because Vlad offered to reduce its severity to make me feel
> > > better?
> > > > > >>>> Currently the description on the issue is "System must be
> > tolerant
> > > > to
> > > > > >>>> faults. Backup operations MUST be atomic (no partial
> completion
> > > > state
> > > > > in
> > > > > >>>> system table)" Sounds like a blocker to me, indeed. It is an
> > > honest
> > > > > >>>> assessment and I don't think anyone is doing the community a
> > favor
> > > > by
> > > > > >>>> trying to walk that back.
> > > > > >>>>
> > > > > >>>>
> > > > > >>>> On Tue, Mar 14, 2017 at 1:57 PM, Josh Elser<els...@apache.org
> >
> > > > >  wrote:
> > > > > >>>>
> > > > > >>>> I took a moment to read through the "blockers" as originally
> > > > > identified
> > > > > >>>> by
> > > > > >>>>
> > > > > >>>>> Vlad, and (to echo Enis' take) I read the majority of them as
> > > being
> > > > > >>>>> blockers not for the next release, but for a "full-fledged
> > > > feature".
> > > > > >>>>> I'm
> > > > > >>>>> going to intentional avoid addressing the discussion of
> > shipping
> > > > > >>>>> partial
> > > > > >>>>> features (related, but not relevant at the moment).
> > > > > >>>>>
> > > > > >>>>> HBASE-15227 is actually the one that bothers me the most,
> with
> > > > > >>>>> HBASE-17133
> > > > > >>>>> coming in close behind. Vlad, is there any documentation you
> > can
> > > > > point
> > > > > >>>>> me
> > > > > >>>>> to about what the current issues are with the current
> > > > implementation?
> > > > > >>>>> For
> > > > > >>>>> example, what happens now if the system has some kind of
> > "partial
> > > > > >>>>> completion state"?
> > > > > >>>>>
> > > > > >>>>> Documentation is one of those that is really hard to judge.
> We
> > > want
> > > > > to
> > > > > >>>>> get
> > > > > >>>>> this code out for people to use (and to free up our strained
> > dev
> > > > > >>>>> resources), but what good is some feature if the docs are
> > > > > >>>>> missing/incomplete?
> > > > > >>>>>
> > > > > >>>>> I think I could stomach the docs being inaccurate (with a
> clear
> > > > > >>>>> disclaimer
> > > > > >>>>> that the chapter is incomplete -- that's a 5min task). But, I
> > > > think I
> > > > > >>>>> need
> > > > > >>>>> an answer about how the feature handles our common dist-sys
> > > > category
> > > > > of
> > > > > >>>>> problems before I can consider whether I'm ok with the
> feature
> > > > > hitting
> > > > > >>>>> 2.0...
> > > > > >>>>>
> > > > > >>>>> I'll also try to throw up a few nodes and play with it to
> > address
> > > > the
> > > > > >>>>> problem as an (ignorant) user ;)
> > > > > >>>>>
> > > > > >>>>>
> > > > > >>>>> Andrew Purtell wrote:
> > > > > >>>>>
> > > > > >>>>> I don't like that issues were identified as "blockers" but
> now
> > > > there
> > > > > is
> > > > > >>>>>
> > > > > >>>>>> an
> > > > > >>>>>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

uot; as originally
> > > > identified
> > > > >>>> by
> > > > >>>>
> > > > >>>>> Vlad, and (to echo Enis' take) I read the majority of them as
> > being
> > > > >>>>> blockers not for the next release, but for a "full-fledged
> > > feature".
> > > > >>>>> I'm
> > > > >>>>> going to intentional avoid addressing the discussion of
> shipping
> > > > >>>>> partial
> > > > >>>>> features (related, but not relevant at the moment).
> > > > >>>>>
> > > > >>>>> HBASE-15227 is actually the one that bothers me the most, with
> > > > >>>>> HBASE-17133
> > > > >>>>> coming in close behind. Vlad, is there any documentation you
> can
> > > > point
> > > > >>>>> me
> > > > >>>>> to about what the current issues are with the current
> > > implementation?
> > > > >>>>> For
> > > > >>>>> example, what happens now if the system has some kind of
> "partial
> > > > >>>>> completion state"?
> > > > >>>>>
> > > > >>>>> Documentation is one of those that is really hard to judge. We
> > want
> > > > to
> > > > >>>>> get
> > > > >>>>> this code out for people to use (and to free up our strained
> dev
> > > > >>>>> resources), but what good is some feature if the docs are
> > > > >>>>> missing/incomplete?
> > > > >>>>>
> > > > >>>>> I think I could stomach the docs being inaccurate (with a clear
> > > > >>>>> disclaimer
> > > > >>>>> that the chapter is incomplete -- that's a 5min task). But, I
> > > think I
> > > > >>>>> need
> > > > >>>>> an answer about how the feature handles our common dist-sys
> > > category
> > > > of
> > > > >>>>> problems before I can consider whether I'm ok with the feature
> > > > hitting
> > > > >>>>> 2.0...
> > > > >>>>>
> > > > >>>>> I'll also try to throw up a few nodes and play with it to
> address
> > > the
> > > > >>>>> problem as an (ignorant) user ;)
> > > > >>>>>
> > > > >>>>>
> > > > >>>>> Andrew Purtell wrote:
> > > > >>>>>
> > > > >>>>> I don't like that issues were identified as "blockers" but now
> > > there
> > > > is
> > > > >>>>>
> > > > >>>>>> an
> > > > >>>>>> attempt to walk that back.
> > > > >>>>>>
> > > > >>>>>> I don't like that development of this feature has lingered
> for a
> > > > long
> > > > >>>>>> time
> > > > >>>>>> in this unfinished state when this work could have been done
> by
> > > now,
> > > > >>>>>> now
> > > > >>>>>> that we are trying to get a 2.0 out the door. Because this is
> a
> > > > >>>>>> volunteer
> > > > >>>>>> project I cannot make any demand that it should be done, but I
> > can
> > > > >>>>>> certainly look at the current state and be nonplussed. This
> will
> > > be
> > > > >>>>>> yet
> > > > >>>>>> another half finished thing in 2.0 when this merge happens.
> > > Promises
> > > > >>>>>> to
> > > > >>>>>> finish the unfinished work are nice but not currency. Commits
> > are
> > > > >>>>>> currency.
> > > > >>>>>> I hope at least the fault tolerance changes can be completed
> and
> > > > >>>>>> committed
> > > > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to
> > slip
> > > > >>>>>> further.
> > > > >>&

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Stack]

gt; > > >>>>> example, what happens now if the system has some kind of "partial
> > > >>>>> completion state"?
> > > >>>>>
> > > >>>>> Documentation is one of those that is really hard to judge. We
> want
> > > to
> > > >>>>> get
> > > >>>>> this code out for people to use (and to free up our strained dev
> > > >>>>> resources), but what good is some feature if the docs are
> > > >>>>> missing/incomplete?
> > > >>>>>
> > > >>>>> I think I could stomach the docs being inaccurate (with a clear
> > > >>>>> disclaimer
> > > >>>>> that the chapter is incomplete -- that's a 5min task). But, I
> > think I
> > > >>>>> need
> > > >>>>> an answer about how the feature handles our common dist-sys
> > category
> > > of
> > > >>>>> problems before I can consider whether I'm ok with the feature
> > > hitting
> > > >>>>> 2.0...
> > > >>>>>
> > > >>>>> I'll also try to throw up a few nodes and play with it to address
> > the
> > > >>>>> problem as an (ignorant) user ;)
> > > >>>>>
> > > >>>>>
> > > >>>>> Andrew Purtell wrote:
> > > >>>>>
> > > >>>>> I don't like that issues were identified as "blockers" but now
> > there
> > > is
> > > >>>>>
> > > >>>>>> an
> > > >>>>>> attempt to walk that back.
> > > >>>>>>
> > > >>>>>> I don't like that development of this feature has lingered for a
> > > long
> > > >>>>>> time
> > > >>>>>> in this unfinished state when this work could have been done by
> > now,
> > > >>>>>> now
> > > >>>>>> that we are trying to get a 2.0 out the door. Because this is a
> > > >>>>>> volunteer
> > > >>>>>> project I cannot make any demand that it should be done, but I
> can
> > > >>>>>> certainly look at the current state and be nonplussed. This will
> > be
> > > >>>>>> yet
> > > >>>>>> another half finished thing in 2.0 when this merge happens.
> > Promises
> > > >>>>>> to
> > > >>>>>> finish the unfinished work are nice but not currency. Commits
> are
> > > >>>>>> currency.
> > > >>>>>> I hope at least the fault tolerance changes can be completed and
> > > >>>>>> committed
> > > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to
> slip
> > > >>>>>> further.
> > > >>>>>>
> > > >>>>>> Also, marking something experimental should be done on the
> merits
> > of
> > > >>>>>> that
> > > >>>>>> evaluation, not simply to justify dropping unfinished work into
> a
> > > >>>>>> release
> > > >>>>>> branch.
> > > >>>>>>
> > > >>>>>> I will change my vote to -0.
> > > >>>>>>
> > > >>>>>>
> > > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
> > > >>>>>>   wrote:
> > > >>>>>>
> > > >>>>>> I think there is some misconception of using the term "blockers"
> > for
> > > >>>>>>
> > > >>>>>> referring to those jiras. My understanding is that those three
> > jiras
> > > >>>>>>> are
> > > >>>>>>> blockers for the backup functionality to be more mature and
> more
> > > >>>>>>> usable.
> > > >>>>>>> But they are not release blockers. Let's say we merged the code
> > in,
> > > >>>>>>> and
> > > >>>>>>> for
> > > >>>>>>> some reason those did not get addressed in time. We can still
> do
> > > the
> > >

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

 ;)
> > >>>>>
> > >>>>>
> > >>>>> Andrew Purtell wrote:
> > >>>>>
> > >>>>> I don't like that issues were identified as "blockers" but now
> there
> > is
> > >>>>>
> > >>>>>> an
> > >>>>>> attempt to walk that back.
> > >>>>>>
> > >>>>>> I don't like that development of this feature has lingered for a
> > long
> > >>>>>> time
> > >>>>>> in this unfinished state when this work could have been done by
> now,
> > >>>>>> now
> > >>>>>> that we are trying to get a 2.0 out the door. Because this is a
> > >>>>>> volunteer
> > >>>>>> project I cannot make any demand that it should be done, but I can
> > >>>>>> certainly look at the current state and be nonplussed. This will
> be
> > >>>>>> yet
> > >>>>>> another half finished thing in 2.0 when this merge happens.
> Promises
> > >>>>>> to
> > >>>>>> finish the unfinished work are nice but not currency. Commits are
> > >>>>>> currency.
> > >>>>>> I hope at least the fault tolerance changes can be completed and
> > >>>>>> committed
> > >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip
> > >>>>>> further.
> > >>>>>>
> > >>>>>> Also, marking something experimental should be done on the merits
> of
> > >>>>>> that
> > >>>>>> evaluation, not simply to justify dropping unfinished work into a
> > >>>>>> release
> > >>>>>> branch.
> > >>>>>>
> > >>>>>> I will change my vote to -0.
> > >>>>>>
> > >>>>>>
> > >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
> > >>>>>>   wrote:
> > >>>>>>
> > >>>>>> I think there is some misconception of using the term "blockers"
> for
> > >>>>>>
> > >>>>>> referring to those jiras. My understanding is that those three
> jiras
> > >>>>>>> are
> > >>>>>>> blockers for the backup functionality to be more mature and more
> > >>>>>>> usable.
> > >>>>>>> But they are not release blockers. Let's say we merged the code
> in,
> > >>>>>>> and
> > >>>>>>> for
> > >>>>>>> some reason those did not get addressed in time. We can still do
> > the
> > >>>>>>> 2.0
> > >>>>>>> release without having to wait for the commits. We can instead
> mark
> > >>>>>>> the
> > >>>>>>> "backup" feature as experimental with known issues and go on with
> > the
> > >>>>>>> release. In that sense they are not real release blockers.
> > >>>>>>>
> > >>>>>>> We are proposing the merge at this time because of the above that
> > >>>>>>> maintaining this in a branch is becoming extremely costly and not
> > >>>>>>> productive for the HBase community. Realistically, we cannot have
> > the
> > >>>>>>> luxury of having to wait another couple of months and doing yet
> > >>>>>>> another
> > >>>>>>> giant round of reviews because the code base is a moving target.
> > >>>>>>>
> > >>>>>>> Enis
> > >>>>>>>
> > >>>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<
> d...@hortonworks.com>
> > >>>>>>> wrote:
> > >>>>>>>
> > >>>>>>> Vlad, on the first point, I think what Stack is saying is that
> > >>>>>>> creating
> > >>>>>>>
> > >>>>>>> the new branch (as Ted did) ignores the feedback incorporated
> thus
> > >>>>>>>> far
> > >>>>>>>> in
> > &g

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Stack]

>>>>>> I hope at least the fault tolerance changes can be completed and
> >>>>>> committed
> >>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip
> >>>>>> further.
> >>>>>>
> >>>>>> Also, marking something experimental should be done on the merits of
> >>>>>> that
> >>>>>> evaluation, not simply to justify dropping unfinished work into a
> >>>>>> release
> >>>>>> branch.
> >>>>>>
> >>>>>> I will change my vote to -0.
> >>>>>>
> >>>>>>
> >>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
> >>>>>>   wrote:
> >>>>>>
> >>>>>> I think there is some misconception of using the term "blockers" for
> >>>>>>
> >>>>>> referring to those jiras. My understanding is that those three jiras
> >>>>>>> are
> >>>>>>> blockers for the backup functionality to be more mature and more
> >>>>>>> usable.
> >>>>>>> But they are not release blockers. Let's say we merged the code in,
> >>>>>>> and
> >>>>>>> for
> >>>>>>> some reason those did not get addressed in time. We can still do
> the
> >>>>>>> 2.0
> >>>>>>> release without having to wait for the commits. We can instead mark
> >>>>>>> the
> >>>>>>> "backup" feature as experimental with known issues and go on with
> the
> >>>>>>> release. In that sense they are not real release blockers.
> >>>>>>>
> >>>>>>> We are proposing the merge at this time because of the above that
> >>>>>>> maintaining this in a branch is becoming extremely costly and not
> >>>>>>> productive for the HBase community. Realistically, we cannot have
> the
> >>>>>>> luxury of having to wait another couple of months and doing yet
> >>>>>>> another
> >>>>>>> giant round of reviews because the code base is a moving target.
> >>>>>>>
> >>>>>>> Enis
> >>>>>>>
> >>>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
> >>>>>>> wrote:
> >>>>>>>
> >>>>>>> Vlad, on the first point, I think what Stack is saying is that
> >>>>>>> creating
> >>>>>>>
> >>>>>>> the new branch (as Ted did) ignores the feedback incorporated thus
> >>>>>>>> far
> >>>>>>>> in
> >>>>>>>> the iterations of the mega-patch. That's a wrong way to go.
> >>>>>>>> On the separation into a backup module, again, that was reverted
> to
> >>>>>>>> ease
> >>>>>>>> reviews of the mega-patch, and was noted as work to be done
> later. I
> >>>>>>>>
> >>>>>>>> think
> >>>>>>>>
> >>>>>>> Stack just wants to make the list of remaining work more complete
> by
> >>>>>>>
> >>>>>>>> citing
> >>>>>>>>
> >>>>>>> that as pending work.
> >>>>>>>
> >>>>>>>> 
> >>>>>>>> From: Vladimir Rodionov<vladrodio...@gmail.com>
> >>>>>>>> Sent: Monday, March 13, 2017 3:09 PM
> >>>>>>>> To: dev@hbase.apache.org
> >>>>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote
> >>>>>>>> closing
> >>>>>>>> 3/11/2017
> >>>>>>>>
> >>>>>>>> It ignores the feedback
> >>>>>>>>
> >>>>>>>> If I "ignore" feedback, I put my comment - why? I am always  open
> >>>>>>>>> for
> >>>>>>>>>
> >>>>>>>>> further discussions. If reviewer does not insist on a particular
>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

ply to justify dropping unfinished work into a
>>>>>>> release
>>>>>>> branch.
>>>>>>>
>>>>>>> I will change my vote to -0.
>>>>>>>
>>>>>>>
>>>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
>>>>>>>   wrote:
>>>>>>>
>>>>>>> I think there is some misconception of using the term "blockers" for
>>>>>>>
>>>>>>> referring to those jiras. My understanding is that those three jiras
>>>>>>>> are
>>>>>>>> blockers for the backup functionality to be more mature and more
>>>>>>>> usable.
>>>>>>>> But they are not release blockers. Let's say we merged the code in,
>>>>>>>> and
>>>>>>>> for
>>>>>>>> some reason those did not get addressed in time. We can still do the
>>>>>>>> 2.0
>>>>>>>> release without having to wait for the commits. We can instead mark
>>>>>>>> the
>>>>>>>> "backup" feature as experimental with known issues and go on with
>>>>>>>> the
>>>>>>>> release. In that sense they are not real release blockers.
>>>>>>>>
>>>>>>>> We are proposing the merge at this time because of the above that
>>>>>>>> maintaining this in a branch is becoming extremely costly and not
>>>>>>>> productive for the HBase community. Realistically, we cannot have
>>>>>>>> the
>>>>>>>> luxury of having to wait another couple of months and doing yet
>>>>>>>> another
>>>>>>>> giant round of reviews because the code base is a moving target.
>>>>>>>>
>>>>>>>> Enis
>>>>>>>>
>>>>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> Vlad, on the first point, I think what Stack is saying is that
>>>>>>>> creating
>>>>>>>>
>>>>>>>> the new branch (as Ted did) ignores the feedback incorporated thus
>>>>>>>>> far
>>>>>>>>> in
>>>>>>>>> the iterations of the mega-patch. That's a wrong way to go.
>>>>>>>>> On the separation into a backup module, again, that was reverted to
>>>>>>>>> ease
>>>>>>>>> reviews of the mega-patch, and was noted as work to be done later.
>>>>>>>>> I
>>>>>>>>>
>>>>>>>>> think
>>>>>>>>>
>>>>>>>> Stack just wants to make the list of remaining work more complete by
>>>>>>>>
>>>>>>>>> citing
>>>>>>>>>
>>>>>>>> that as pending work.
>>>>>>>>
>>>>>>>>> 
>>>>>>>>> From: Vladimir Rodionov<vladrodio...@gmail.com>
>>>>>>>>> Sent: Monday, March 13, 2017 3:09 PM
>>>>>>>>> To: dev@hbase.apache.org
>>>>>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote
>>>>>>>>> closing
>>>>>>>>> 3/11/2017
>>>>>>>>>
>>>>>>>>> It ignores the feedback
>>>>>>>>>
>>>>>>>>> If I "ignore" feedback, I put my comment - why? I am always  open
>>>>>>>>>> for
>>>>>>>>>>
>>>>>>>>>> further discussions. If reviewer does not insist on a particular
>>>>>>>>> request
>>>>>>>>>
>>>>>>>>> -
>>>>>>>>>
>>>>>>>> it will be dropped. I think it is fair.
>>>>>>>>
>>>>>>>>> he list is incomplete because a bunch of
>>>>>>>>>
>>>>>>>>> follow-ons came of the review cycle

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

 don't think anyone is doing the community a favor by
>>>> trying to walk that back.
>>>>
>>>>
>>>> On Tue, Mar 14, 2017 at 1:57 PM, Josh Elser<els...@apache.org>   wrote:
>>>>
>>>> I took a moment to read through the "blockers" as originally identified
>>>> by
>>>>
>>>>> Vlad, and (to echo Enis' take) I read the majority of them as being
>>>>> blockers not for the next release, but for a "full-fledged feature".
>>>>> I'm
>>>>> going to intentional avoid addressing the discussion of shipping
>>>>> partial
>>>>> features (related, but not relevant at the moment).
>>>>>
>>>>> HBASE-15227 is actually the one that bothers me the most, with
>>>>> HBASE-17133
>>>>> coming in close behind. Vlad, is there any documentation you can point
>>>>> me
>>>>> to about what the current issues are with the current implementation?
>>>>> For
>>>>> example, what happens now if the system has some kind of "partial
>>>>> completion state"?
>>>>>
>>>>> Documentation is one of those that is really hard to judge. We want to
>>>>> get
>>>>> this code out for people to use (and to free up our strained dev
>>>>> resources), but what good is some feature if the docs are
>>>>> missing/incomplete?
>>>>>
>>>>> I think I could stomach the docs being inaccurate (with a clear
>>>>> disclaimer
>>>>> that the chapter is incomplete -- that's a 5min task). But, I think I
>>>>> need
>>>>> an answer about how the feature handles our common dist-sys category of
>>>>> problems before I can consider whether I'm ok with the feature hitting
>>>>> 2.0...
>>>>>
>>>>> I'll also try to throw up a few nodes and play with it to address the
>>>>> problem as an (ignorant) user ;)
>>>>>
>>>>>
>>>>> Andrew Purtell wrote:
>>>>>
>>>>> I don't like that issues were identified as "blockers" but now there is
>>>>>
>>>>>> an
>>>>>> attempt to walk that back.
>>>>>>
>>>>>> I don't like that development of this feature has lingered for a long
>>>>>> time
>>>>>> in this unfinished state when this work could have been done by now,
>>>>>> now
>>>>>> that we are trying to get a 2.0 out the door. Because this is a
>>>>>> volunteer
>>>>>> project I cannot make any demand that it should be done, but I can
>>>>>> certainly look at the current state and be nonplussed. This will be
>>>>>> yet
>>>>>> another half finished thing in 2.0 when this merge happens. Promises
>>>>>> to
>>>>>> finish the unfinished work are nice but not currency. Commits are
>>>>>> currency.
>>>>>> I hope at least the fault tolerance changes can be completed and
>>>>>> committed
>>>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip
>>>>>> further.
>>>>>>
>>>>>> Also, marking something experimental should be done on the merits of
>>>>>> that
>>>>>> evaluation, not simply to justify dropping unfinished work into a
>>>>>> release
>>>>>> branch.
>>>>>>
>>>>>> I will change my vote to -0.
>>>>>>
>>>>>>
>>>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
>>>>>>   wrote:
>>>>>>
>>>>>> I think there is some misconception of using the term "blockers" for
>>>>>>
>>>>>> referring to those jiras. My understanding is that those three jiras
>>>>>>> are
>>>>>>> blockers for the backup functionality to be more mature and more
>>>>>>> usable.
>>>>>>> But they are not release blockers. Let's say we merged the code in,
>>>>>>> and
>>>>>>> for
>>>>>>> some reason those did not get addressed in time. We can still do the
>>>>>>> 2.0
>>>>>>> release without ha

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Josh Elser]

isclaimer
that the chapter is incomplete -- that's a 5min task). But, I think I
need
an answer about how the feature handles our common dist-sys category of
problems before I can consider whether I'm ok with the feature hitting
2.0...

I'll also try to throw up a few nodes and play with it to address the
problem as an (ignorant) user ;)


Andrew Purtell wrote:

I don't like that issues were identified as "blockers" but now there is

an
attempt to walk that back.

I don't like that development of this feature has lingered for a long
time
in this unfinished state when this work could have been done by now, now
that we are trying to get a 2.0 out the door. Because this is a
volunteer
project I cannot make any demand that it should be done, but I can
certainly look at the current state and be nonplussed. This will be yet
another half finished thing in 2.0 when this merge happens. Promises to
finish the unfinished work are nice but not currency. Commits are
currency.
I hope at least the fault tolerance changes can be completed and
committed
before we spin a 2.0 RC, and without causing a 2.0 release to slip
further.

Also, marking something experimental should be done on the merits of
that
evaluation, not simply to justify dropping unfinished work into a
release
branch.

I will change my vote to -0.


On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
  wrote:

I think there is some misconception of using the term "blockers" for


referring to those jiras. My understanding is that those three jiras
are
blockers for the backup functionality to be more mature and more
usable.
But they are not release blockers. Let's say we merged the code in, and
for
some reason those did not get addressed in time. We can still do the
2.0
release without having to wait for the commits. We can instead mark the
"backup" feature as experimental with known issues and go on with the
release. In that sense they are not real release blockers.

We are proposing the merge at this time because of the above that
maintaining this in a branch is becoming extremely costly and not
productive for the HBase community. Realistically, we cannot have the
luxury of having to wait another couple of months and doing yet another
giant round of reviews because the code base is a moving target.

Enis

On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
wrote:

Vlad, on the first point, I think what Stack is saying is that creating


the new branch (as Ted did) ignores the feedback incorporated thus far
in
the iterations of the mega-patch. That's a wrong way to go.
On the separation into a backup module, again, that was reverted to
ease
reviews of the mega-patch, and was noted as work to be done later. I

think

Stack just wants to make the list of remaining work more complete by

citing

that as pending work.


From: Vladimir Rodionov<vladrodio...@gmail.com>
Sent: Monday, March 13, 2017 3:09 PM
To: dev@hbase.apache.org
Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
3/11/2017

It ignores the feedback


If I "ignore" feedback, I put my comment - why? I am always  open for


further discussions. If reviewer does not insist on a particular
request

-

it will be dropped. I think it is fair.

he list is incomplete because a bunch of


follow-ons came of the review cycle (including moving backup/restore

out

of

core to live in its own module).

For those who were not following our lengthy conversation on a review


board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>wrote:

On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>wrote:


On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>
  wrote:


HBASE-14123 branch has been created, with Vlad's mega patch v61.


The patch put up for VOTE here was done on a branch. The call to


merge

seems to have been premature given the many cycles of review and test


that

happened subsequent (The cycles burned a bunch of dev resource).

The patch as is is now in a state where it is too big for our infra;

rb

and JIRA are creaking under the size and # of iterations.


Adding finish of new JIRAs to this merge implies a new round of

review

and


test of an already massive patch. Who is going to do this work?

Going back to a new branch seems wrong route to take.

St.Ack



To be more explicit, this patch was developed on a branch and then a


bunch

of dev resources were burned getting it into a state where it could be

merged to master. Going back to a branch to bulk up the merge so it
includes more JIRAs than the many it already incorporates is the
wrong
direction for us to be headed in. It ignores the feedback given and
the
work done by Vladimir slimming down an already over-broad

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Andrew Purtell]

t not currency. Commits are
>>>> currency.
>>>> I hope at least the fault tolerance changes can be completed and
>>>> committed
>>>> before we spin a 2.0 RC, and without causing a 2.0 release to slip
>>>> further.
>>>>
>>>> Also, marking something experimental should be done on the merits of
>>>> that
>>>> evaluation, not simply to justify dropping unfinished work into a
>>>> release
>>>> branch.
>>>>
>>>> I will change my vote to -0.
>>>>
>>>>
>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
>>>>  wrote:
>>>>
>>>> I think there is some misconception of using the term "blockers" for
>>>>
>>>>> referring to those jiras. My understanding is that those three jiras
>>>>> are
>>>>> blockers for the backup functionality to be more mature and more
>>>>> usable.
>>>>> But they are not release blockers. Let's say we merged the code in, and
>>>>> for
>>>>> some reason those did not get addressed in time. We can still do the
>>>>> 2.0
>>>>> release without having to wait for the commits. We can instead mark the
>>>>> "backup" feature as experimental with known issues and go on with the
>>>>> release. In that sense they are not real release blockers.
>>>>>
>>>>> We are proposing the merge at this time because of the above that
>>>>> maintaining this in a branch is becoming extremely costly and not
>>>>> productive for the HBase community. Realistically, we cannot have the
>>>>> luxury of having to wait another couple of months and doing yet another
>>>>> giant round of reviews because the code base is a moving target.
>>>>>
>>>>> Enis
>>>>>
>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
>>>>> wrote:
>>>>>
>>>>> Vlad, on the first point, I think what Stack is saying is that creating
>>>>>
>>>>>> the new branch (as Ted did) ignores the feedback incorporated thus far
>>>>>> in
>>>>>> the iterations of the mega-patch. That's a wrong way to go.
>>>>>> On the separation into a backup module, again, that was reverted to
>>>>>> ease
>>>>>> reviews of the mega-patch, and was noted as work to be done later. I
>>>>>>
>>>>>> think
>>>>>
>>>>> Stack just wants to make the list of remaining work more complete by
>>>>>>
>>>>>> citing
>>>>>
>>>>> that as pending work.
>>>>>> 
>>>>>> From: Vladimir Rodionov<vladrodio...@gmail.com>
>>>>>> Sent: Monday, March 13, 2017 3:09 PM
>>>>>> To: dev@hbase.apache.org
>>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
>>>>>> 3/11/2017
>>>>>>
>>>>>>It ignores the feedback
>>>>>>
>>>>>>> If I "ignore" feedback, I put my comment - why? I am always  open for
>>>>>>>
>>>>>> further discussions. If reviewer does not insist on a particular
>>>>>> request
>>>>>>
>>>>>> -
>>>>>
>>>>> it will be dropped. I think it is fair.
>>>>>>
>>>>>> he list is incomplete because a bunch of
>>>>>>
>>>>>>> follow-ons came of the review cycle (including moving backup/restore
>>>>>>>>
>>>>>>>> out
>>>>>>>
>>>>>> of
>>>>>>
>>>>>> core to live in its own module).
>>>>>>> For those who were not following our lengthy conversation on a review
>>>>>>>
>>>>>> board, separation of a backup code into a separate module has been
>>>>>> done last year, but has been reverted back by request of a reviewer.
>>>>>>
>>>>>>
>>>>>> -Vladimir
>>>>>>
>>>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>   wrote:
>>>>>>
>>>>>> On 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Josh Elser]

oductive for the HBase community. Realistically, we cannot have the
luxury of having to wait another couple of months and doing yet another
giant round of reviews because the code base is a moving target.

Enis

On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
wrote:

Vlad, on the first point, I think what Stack is saying is that creating

the new branch (as Ted did) ignores the feedback incorporated thus far
in
the iterations of the mega-patch. That's a wrong way to go.
On the separation into a backup module, again, that was reverted to ease
reviews of the mega-patch, and was noted as work to be done later. I


think


Stack just wants to make the list of remaining work more complete by


citing


that as pending work.

From: Vladimir Rodionov<vladrodio...@gmail.com>
Sent: Monday, March 13, 2017 3:09 PM
To: dev@hbase.apache.org
Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
3/11/2017

   It ignores the feedback

If I "ignore" feedback, I put my comment - why? I am always  open for

further discussions. If reviewer does not insist on a particular request


-


it will be dropped. I think it is fair.

he list is incomplete because a bunch of

follow-ons came of the review cycle (including moving backup/restore


out

of


core to live in its own module).
For those who were not following our lengthy conversation on a review

board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>   wrote:

On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>   wrote:

On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>   wrote:

HBASE-14123 branch has been created, with Vlad's mega patch v61.

The patch put up for VOTE here was done on a branch. The call to

merge

seems to have been premature given the many cycles of review and test

that


happened subsequent (The cycles burned a bunch of dev resource).

The patch as is is now in a state where it is too big for our infra;


rb

and JIRA are creaking under the size and # of iterations.

Adding finish of new JIRAs to this merge implies a new round of


review

and

test of an already massive patch. Who is going to do this work?

Going back to a new branch seems wrong route to take.

St.Ack



To be more explicit, this patch was developed on a branch and then a

bunch


of dev resources were burned getting it into a state where it could be
merged to master. Going back to a branch to bulk up the merge so it
includes more JIRAs than the many it already incorporates is the wrong
direction for us to be headed in. It ignores the feedback given and the
work done by Vladimir slimming down an already over-broad scope. It is


also


predicated on abundant review and testing resource being on tap to


cycle
on


a feature that is useful, but non-core.

The patch is ready for merge IMO. Geoffrey makes a nice list of what is
still to do though IIRC, the list is incomplete because a bunch of
follow-ons came of the review cycle (including moving backup/restore


out
of


core to live in its own module).

The patch needs three votes to merge. I am not against merge but I am


not
voting for the patch because I do have any more time to spend on this

non-core feature and feel that a vote will have me assume a


responsibility


I will not shirk.

S





FYI

On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu<yuzhih...@gmail.com>


wrote:

Thanks for the feedback, Andrew.

How about the following plan:

create branch HBASE-14123 off of master with mega patch v61 as the


first

commit (reviewed by Stack and Enis)

Vlad and I continue development (the 3 blockers) based on


HBASE-14123

   branch

when all of the blockers get +1 and merged into HBASE-14123


branch,

we


propose to community for merging into branch-2 (master branch, if

branch-2


doesn't materialize for whatever reason) again

Cheers


On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell<


apurt...@apache.org>

wrote:

Thanks for the offer but I like that you were honest about
compiling

a


list

of issues that you thought were blockers for release. Since this


proposal
is a merge into 2.0, and we are trying to release 2.0, I am -1 on
this

merge until those blockers are addressed.

I had a look at the list.

I think the documentation issue is important but not actually a


blocker.
That may be a controversial opinion, but documentation can be
back-filled
worst case. So take HBASE-17133 off the list.

Remaining are effectively HBASE-14417, HBASE-14141, and


HBASE-15227.

They

all have patches attached to the respective JIRAs so completing
this

work

won't be onerous. Get these committed and I will lift my -1. The
others

who

voted +1 on this thread surely can help with that.

Thanks.


On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov<
v

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

for
> >>> referring to those jiras. My understanding is that those three jiras
> are
> >>> blockers for the backup functionality to be more mature and more
> usable.
> >>> But they are not release blockers. Let's say we merged the code in, and
> >>> for
> >>> some reason those did not get addressed in time. We can still do the
> 2.0
> >>> release without having to wait for the commits. We can instead mark the
> >>> "backup" feature as experimental with known issues and go on with the
> >>> release. In that sense they are not real release blockers.
> >>>
> >>> We are proposing the merge at this time because of the above that
> >>> maintaining this in a branch is becoming extremely costly and not
> >>> productive for the HBase community. Realistically, we cannot have the
> >>> luxury of having to wait another couple of months and doing yet another
> >>> giant round of reviews because the code base is a moving target.
> >>>
> >>> Enis
> >>>
> >>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
> >>> wrote:
> >>>
> >>> Vlad, on the first point, I think what Stack is saying is that creating
> >>>> the new branch (as Ted did) ignores the feedback incorporated thus far
> >>>> in
> >>>> the iterations of the mega-patch. That's a wrong way to go.
> >>>> On the separation into a backup module, again, that was reverted to
> ease
> >>>> reviews of the mega-patch, and was noted as work to be done later. I
> >>>>
> >>> think
> >>>
> >>>> Stack just wants to make the list of remaining work more complete by
> >>>>
> >>> citing
> >>>
> >>>> that as pending work.
> >>>> 
> >>>> From: Vladimir Rodionov<vladrodio...@gmail.com>
> >>>> Sent: Monday, March 13, 2017 3:09 PM
> >>>> To: dev@hbase.apache.org
> >>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
> >>>> 3/11/2017
> >>>>
> >>>>   It ignores the feedback
> >>>>>>
> >>>>> If I "ignore" feedback, I put my comment - why? I am always  open for
> >>>> further discussions. If reviewer does not insist on a particular
> request
> >>>>
> >>> -
> >>>
> >>>> it will be dropped. I think it is fair.
> >>>>
> >>>> he list is incomplete because a bunch of
> >>>>>> follow-ons came of the review cycle (including moving backup/restore
> >>>>>>
> >>>>> out
> >>>
> >>>> of
> >>>>
> >>>>> core to live in its own module).
> >>>>>>
> >>>>> For those who were not following our lengthy conversation on a review
> >>>> board, separation of a backup code into a separate module has been
> >>>> done last year, but has been reverted back by request of a reviewer.
> >>>>
> >>>>
> >>>> -Vladimir
> >>>>
> >>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>  wrote:
> >>>>
> >>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>  wrote:
> >>>>>
> >>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>  wrote:
> >>>>>>
> >>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> >>>>>>>
> >>>>>>> The patch put up for VOTE here was done on a branch. The call to
> >>>>>>
> >>>>> merge
> >>>
> >>>> seems to have been premature given the many cycles of review and test
> >>>>>>
> >>>>> that
> >>>>>
> >>>>>> happened subsequent (The cycles burned a bunch of dev resource).
> >>>>>>
> >>>>>> The patch as is is now in a state where it is too big for our infra;
> >>>>>>
> >>>>> rb
> >>>
> >>>> and JIRA are creaking under the size and # of iterations.
> >>>>>>
> >>>>>> Adding finish of new JIRAs to this merge implies a new round of
> >>>>&

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Andrew Purtell]

ing yet another
>>> giant round of reviews because the code base is a moving target.
>>>
>>> Enis
>>>
>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
>>> wrote:
>>>
>>> Vlad, on the first point, I think what Stack is saying is that creating
>>>> the new branch (as Ted did) ignores the feedback incorporated thus far
>>>> in
>>>> the iterations of the mega-patch. That's a wrong way to go.
>>>> On the separation into a backup module, again, that was reverted to ease
>>>> reviews of the mega-patch, and was noted as work to be done later. I
>>>>
>>> think
>>>
>>>> Stack just wants to make the list of remaining work more complete by
>>>>
>>> citing
>>>
>>>> that as pending work.
>>>> 
>>>> From: Vladimir Rodionov<vladrodio...@gmail.com>
>>>> Sent: Monday, March 13, 2017 3:09 PM
>>>> To: dev@hbase.apache.org
>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
>>>> 3/11/2017
>>>>
>>>>   It ignores the feedback
>>>>>>
>>>>> If I "ignore" feedback, I put my comment - why? I am always  open for
>>>> further discussions. If reviewer does not insist on a particular request
>>>>
>>> -
>>>
>>>> it will be dropped. I think it is fair.
>>>>
>>>> he list is incomplete because a bunch of
>>>>>> follow-ons came of the review cycle (including moving backup/restore
>>>>>>
>>>>> out
>>>
>>>> of
>>>>
>>>>> core to live in its own module).
>>>>>>
>>>>> For those who were not following our lengthy conversation on a review
>>>> board, separation of a backup code into a separate module has been
>>>> done last year, but has been reverted back by request of a reviewer.
>>>>
>>>>
>>>> -Vladimir
>>>>
>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>  wrote:
>>>>
>>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>  wrote:
>>>>>
>>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>  wrote:
>>>>>>
>>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61.
>>>>>>>
>>>>>>> The patch put up for VOTE here was done on a branch. The call to
>>>>>>
>>>>> merge
>>>
>>>> seems to have been premature given the many cycles of review and test
>>>>>>
>>>>> that
>>>>>
>>>>>> happened subsequent (The cycles burned a bunch of dev resource).
>>>>>>
>>>>>> The patch as is is now in a state where it is too big for our infra;
>>>>>>
>>>>> rb
>>>
>>>> and JIRA are creaking under the size and # of iterations.
>>>>>>
>>>>>> Adding finish of new JIRAs to this merge implies a new round of
>>>>>>
>>>>> review
>>>
>>>> and
>>>>>
>>>>>> test of an already massive patch. Who is going to do this work?
>>>>>>
>>>>>> Going back to a new branch seems wrong route to take.
>>>>>>
>>>>>> St.Ack
>>>>>>
>>>>>>
>>>>>>
>>>>>> To be more explicit, this patch was developed on a branch and then a
>>>>>
>>>> bunch
>>>>
>>>>> of dev resources were burned getting it into a state where it could be
>>>>> merged to master. Going back to a branch to bulk up the merge so it
>>>>> includes more JIRAs than the many it already incorporates is the wrong
>>>>> direction for us to be headed in. It ignores the feedback given and the
>>>>> work done by Vladimir slimming down an already over-broad scope. It is
>>>>>
>>>> also
>>>>
>>>>> predicated on abundant review and testing resource being on tap to
>>>>>
>>>> cycle
>>>
>>>> on
>>>>
>>>>> a feature that is useful, but non-core.
>>>>>
>>>>> The patch is ready for merge IMO. Geof

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

>>>>
>>>>
>>>> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>
>>>>  wrote:
>>>>
>>>> I think there is some misconception of using the term "blockers" for
>>>>
>>>>> referring to those jiras. My understanding is that those three jiras
>>>>> are
>>>>> blockers for the backup functionality to be more mature and more
>>>>> usable.
>>>>> But they are not release blockers. Let's say we merged the code in, and
>>>>> for
>>>>> some reason those did not get addressed in time. We can still do the
>>>>> 2.0
>>>>> release without having to wait for the commits. We can instead mark the
>>>>> "backup" feature as experimental with known issues and go on with the
>>>>> release. In that sense they are not real release blockers.
>>>>>
>>>>> We are proposing the merge at this time because of the above that
>>>>> maintaining this in a branch is becoming extremely costly and not
>>>>> productive for the HBase community. Realistically, we cannot have the
>>>>> luxury of having to wait another couple of months and doing yet another
>>>>> giant round of reviews because the code base is a moving target.
>>>>>
>>>>> Enis
>>>>>
>>>>> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>
>>>>> wrote:
>>>>>
>>>>> Vlad, on the first point, I think what Stack is saying is that creating
>>>>>
>>>>>> the new branch (as Ted did) ignores the feedback incorporated thus far
>>>>>> in
>>>>>> the iterations of the mega-patch. That's a wrong way to go.
>>>>>> On the separation into a backup module, again, that was reverted to
>>>>>> ease
>>>>>> reviews of the mega-patch, and was noted as work to be done later. I
>>>>>>
>>>>>> think
>>>>>
>>>>> Stack just wants to make the list of remaining work more complete by
>>>>>>
>>>>>> citing
>>>>>
>>>>> that as pending work.
>>>>>> 
>>>>>> From: Vladimir Rodionov<vladrodio...@gmail.com>
>>>>>> Sent: Monday, March 13, 2017 3:09 PM
>>>>>> To: dev@hbase.apache.org
>>>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
>>>>>> 3/11/2017
>>>>>>
>>>>>>It ignores the feedback
>>>>>>
>>>>>>> If I "ignore" feedback, I put my comment - why? I am always  open for
>>>>>>>
>>>>>> further discussions. If reviewer does not insist on a particular
>>>>>> request
>>>>>>
>>>>>> -
>>>>>
>>>>> it will be dropped. I think it is fair.
>>>>>>
>>>>>> he list is incomplete because a bunch of
>>>>>>
>>>>>>> follow-ons came of the review cycle (including moving backup/restore
>>>>>>>>
>>>>>>>> out
>>>>>>>
>>>>>> of
>>>>>>
>>>>>> core to live in its own module).
>>>>>>> For those who were not following our lengthy conversation on a review
>>>>>>>
>>>>>> board, separation of a backup code into a separate module has been
>>>>>> done last year, but has been reverted back by request of a reviewer.
>>>>>>
>>>>>>
>>>>>> -Vladimir
>>>>>>
>>>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>   wrote:
>>>>>>
>>>>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>   wrote:
>>>>>>
>>>>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>
>>>>>>>  wrote:
>>>>>>>
>>>>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61.
>>>>>>>>
>>>>>>>>> The patch put up for VOTE here was done on a branch. The call to
>>>>>>>>>
>>>>>>>> merge
>>>>>>>
>>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Josh Elser]

to make the list of remaining work more complete by


citing


that as pending work.

From: Vladimir Rodionov<vladrodio...@gmail.com>
Sent: Monday, March 13, 2017 3:09 PM
To: dev@hbase.apache.org
Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
3/11/2017

   It ignores the feedback

If I "ignore" feedback, I put my comment - why? I am always  open for

further discussions. If reviewer does not insist on a particular request


-


it will be dropped. I think it is fair.

he list is incomplete because a bunch of

follow-ons came of the review cycle (including moving backup/restore


out

of


core to live in its own module).
For those who were not following our lengthy conversation on a review

board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>   wrote:

On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>   wrote:

On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>   wrote:

HBASE-14123 branch has been created, with Vlad's mega patch v61.

The patch put up for VOTE here was done on a branch. The call to

merge

seems to have been premature given the many cycles of review and test

that


happened subsequent (The cycles burned a bunch of dev resource).

The patch as is is now in a state where it is too big for our infra;


rb

and JIRA are creaking under the size and # of iterations.

Adding finish of new JIRAs to this merge implies a new round of


review

and

test of an already massive patch. Who is going to do this work?

Going back to a new branch seems wrong route to take.

St.Ack



To be more explicit, this patch was developed on a branch and then a

bunch


of dev resources were burned getting it into a state where it could be
merged to master. Going back to a branch to bulk up the merge so it
includes more JIRAs than the many it already incorporates is the wrong
direction for us to be headed in. It ignores the feedback given and the
work done by Vladimir slimming down an already over-broad scope. It is


also


predicated on abundant review and testing resource being on tap to


cycle
on


a feature that is useful, but non-core.

The patch is ready for merge IMO. Geoffrey makes a nice list of what is
still to do though IIRC, the list is incomplete because a bunch of
follow-ons came of the review cycle (including moving backup/restore


out
of


core to live in its own module).

The patch needs three votes to merge. I am not against merge but I am


not
voting for the patch because I do have any more time to spend on this

non-core feature and feel that a vote will have me assume a


responsibility


I will not shirk.

S





FYI

On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu<yuzhih...@gmail.com>


wrote:

Thanks for the feedback, Andrew.

How about the following plan:

create branch HBASE-14123 off of master with mega patch v61 as the


first

commit (reviewed by Stack and Enis)

Vlad and I continue development (the 3 blockers) based on


HBASE-14123

   branch

when all of the blockers get +1 and merged into HBASE-14123


branch,

we


propose to community for merging into branch-2 (master branch, if

branch-2


doesn't materialize for whatever reason) again

Cheers


On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell<


apurt...@apache.org>

wrote:

Thanks for the offer but I like that you were honest about
compiling

a


list

of issues that you thought were blockers for release. Since this


proposal
is a merge into 2.0, and we are trying to release 2.0, I am -1 on
this

merge until those blockers are addressed.

I had a look at the list.

I think the documentation issue is important but not actually a


blocker.
That may be a controversial opinion, but documentation can be
back-filled
worst case. So take HBASE-17133 off the list.

Remaining are effectively HBASE-14417, HBASE-14141, and


HBASE-15227.

They

all have patches attached to the respective JIRAs so completing
this

work

won't be onerous. Get these committed and I will lift my -1. The
others

who

voted +1 on this thread surely can help with that.

Thanks.


On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov<
vladrodio...@gmail.com>
wrote:

No problem I will downgrade Blockers to Majors if it scares
you,

Andrew



Sent from my iPhone

On Mar 10, 2017, at 1:52 PM, Andrew Purtell<
apurt...@apache.org

wrote:

​I know the merge of this feature has lagged substantially. I
think

that

is


regrettable but on another thread we are lamenting that 2.0


is

already

late. Unless I misunderstand, this is a proposal to merge

something

with

known blockers into trunk before we branch it for 2.0 which
will

effectively prevent that release because these blockers will

be

there. I

am


inclined to veto. Probably we should not propose branch


merges

into

code

we


ar

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

;> the new branch (as Ted did) ignores the feedback incorporated thus far
>>>> in
>>>> the iterations of the mega-patch. That's a wrong way to go.
>>>> On the separation into a backup module, again, that was reverted to ease
>>>> reviews of the mega-patch, and was noted as work to be done later. I
>>>>
>>> think
>>>
>>>> Stack just wants to make the list of remaining work more complete by
>>>>
>>> citing
>>>
>>>> that as pending work.
>>>> 
>>>> From: Vladimir Rodionov<vladrodio...@gmail.com>
>>>> Sent: Monday, March 13, 2017 3:09 PM
>>>> To: dev@hbase.apache.org
>>>> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
>>>> 3/11/2017
>>>>
>>>>   It ignores the feedback
>>>>>>
>>>>> If I "ignore" feedback, I put my comment - why? I am always  open for
>>>> further discussions. If reviewer does not insist on a particular request
>>>>
>>> -
>>>
>>>> it will be dropped. I think it is fair.
>>>>
>>>> he list is incomplete because a bunch of
>>>>>> follow-ons came of the review cycle (including moving backup/restore
>>>>>>
>>>>> out
>>>
>>>> of
>>>>
>>>>> core to live in its own module).
>>>>>>
>>>>> For those who were not following our lengthy conversation on a review
>>>> board, separation of a backup code into a separate module has been
>>>> done last year, but has been reverted back by request of a reviewer.
>>>>
>>>>
>>>> -Vladimir
>>>>
>>>> On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>  wrote:
>>>>
>>>> On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>  wrote:
>>>>>
>>>>> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>  wrote:
>>>>>>
>>>>>> HBASE-14123 branch has been created, with Vlad's mega patch v61.
>>>>>>>
>>>>>>> The patch put up for VOTE here was done on a branch. The call to
>>>>>>
>>>>> merge
>>>
>>>> seems to have been premature given the many cycles of review and test
>>>>>>
>>>>> that
>>>>>
>>>>>> happened subsequent (The cycles burned a bunch of dev resource).
>>>>>>
>>>>>> The patch as is is now in a state where it is too big for our infra;
>>>>>>
>>>>> rb
>>>
>>>> and JIRA are creaking under the size and # of iterations.
>>>>>>
>>>>>> Adding finish of new JIRAs to this merge implies a new round of
>>>>>>
>>>>> review
>>>
>>>> and
>>>>>
>>>>>> test of an already massive patch. Who is going to do this work?
>>>>>>
>>>>>> Going back to a new branch seems wrong route to take.
>>>>>>
>>>>>> St.Ack
>>>>>>
>>>>>>
>>>>>>
>>>>>> To be more explicit, this patch was developed on a branch and then a
>>>>>
>>>> bunch
>>>>
>>>>> of dev resources were burned getting it into a state where it could be
>>>>> merged to master. Going back to a branch to bulk up the merge so it
>>>>> includes more JIRAs than the many it already incorporates is the wrong
>>>>> direction for us to be headed in. It ignores the feedback given and the
>>>>> work done by Vladimir slimming down an already over-broad scope. It is
>>>>>
>>>> also
>>>>
>>>>> predicated on abundant review and testing resource being on tap to
>>>>>
>>>> cycle
>>>
>>>> on
>>>>
>>>>> a feature that is useful, but non-core.
>>>>>
>>>>> The patch is ready for merge IMO. Geoffrey makes a nice list of what is
>>>>> still to do though IIRC, the list is incomplete because a bunch of
>>>>> follow-ons came of the review cycle (including moving backup/restore
>>>>>
>>>> out
>>>
>>>> of
>>>>
>>>>> core to live in its own m

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Josh Elser]

I took a moment to read through the "blockers" as originally identified 
by Vlad, and (to echo Enis' take) I read the majority of them as being 
blockers not for the next release, but for a "full-fledged feature". I'm 
going to intentional avoid addressing the discussion of shipping partial 
features (related, but not relevant at the moment).


HBASE-15227 is actually the one that bothers me the most, with 
HBASE-17133 coming in close behind. Vlad, is there any documentation you 
can point me to about what the current issues are with the current 
implementation? For example, what happens now if the system has some 
kind of "partial completion state"?


Documentation is one of those that is really hard to judge. We want to 
get this code out for people to use (and to free up our strained dev 
resources), but what good is some feature if the docs are 
missing/incomplete?


I think I could stomach the docs being inaccurate (with a clear 
disclaimer that the chapter is incomplete -- that's a 5min task). But, I 
think I need an answer about how the feature handles our common dist-sys 
category of problems before I can consider whether I'm ok with the 
feature hitting 2.0...


I'll also try to throw up a few nodes and play with it to address the 
problem as an (ignorant) user ;)


Andrew Purtell wrote:

I don't like that issues were identified as "blockers" but now there is an
attempt to walk that back.

I don't like that development of this feature has lingered for a long time
in this unfinished state when this work could have been done by now, now
that we are trying to get a 2.0 out the door. Because this is a volunteer
project I cannot make any demand that it should be done, but I can
certainly look at the current state and be nonplussed. This will be yet
another half finished thing in 2.0 when this merge happens. Promises to
finish the unfinished work are nice but not currency. Commits are currency.
I hope at least the fault tolerance changes can be completed and committed
before we spin a 2.0 RC, and without causing a 2.0 release to slip further.

Also, marking something experimental should be done on the merits of that
evaluation, not simply to justify dropping unfinished work into a release
branch.

I will change my vote to -0.


On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar<e...@apache.org>  wrote:


I think there is some misconception of using the term "blockers" for
referring to those jiras. My understanding is that those three jiras are
blockers for the backup functionality to be more mature and more usable.
But they are not release blockers. Let's say we merged the code in, and for
some reason those did not get addressed in time. We can still do the 2.0
release without having to wait for the commits. We can instead mark the
"backup" feature as experimental with known issues and go on with the
release. In that sense they are not real release blockers.

We are proposing the merge at this time because of the above that
maintaining this in a branch is becoming extremely costly and not
productive for the HBase community. Realistically, we cannot have the
luxury of having to wait another couple of months and doing yet another
giant round of reviews because the code base is a moving target.

Enis

On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das<d...@hortonworks.com>  wrote:


Vlad, on the first point, I think what Stack is saying is that creating
the new branch (as Ted did) ignores the feedback incorporated thus far in
the iterations of the mega-patch. That's a wrong way to go.
On the separation into a backup module, again, that was reverted to ease
reviews of the mega-patch, and was noted as work to be done later. I

think

Stack just wants to make the list of remaining work more complete by

citing

that as pending work.

From: Vladimir Rodionov<vladrodio...@gmail.com>
Sent: Monday, March 13, 2017 3:09 PM
To: dev@hbase.apache.org
Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
3/11/2017


  It ignores the feedback

If I "ignore" feedback, I put my comment - why? I am always  open for
further discussions. If reviewer does not insist on a particular request

-

it will be dropped. I think it is fair.


he list is incomplete because a bunch of
follow-ons came of the review cycle (including moving backup/restore

out

of

core to live in its own module).

For those who were not following our lengthy conversation on a review
board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack<st...@duboce.net>  wrote:


On Fri, Mar 10, 2017 at 9:09 PM, Stack<st...@duboce.net>  wrote:


On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu<yuzhih...@gmail.com>  wrote:


HBASE-14123 branch has been created, with Vlad's mega patch v61.


T

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

As of 6:06pm PST, I still saw these branches on
https://github.com/apache/hbase/

Maybe there is some delay.

Thanks Andrew.

On Mon, Mar 13, 2017 at 5:59 PM, Andrew Purtell <apurt...@apache.org> wrote:

> That's because this was pushed with a different name. It's gone now.
>
> apurtell@onyx:~/src/hbase$ git branch -a | grep 14123
>   remotes/origin/14123
> apurtell@onyx:~/src/hbase$ git push origin :14123
> Username for 'https://git-wip-us.apache.org': apurtell
> Password for 'https://apurt...@git-wip-us.apache.org':
> To https://git-wip-us.apache.org/repos/asf/hbase.git
>  - [deleted] 14123
>
>
> On Mon, Mar 13, 2017 at 4:13 PM, Ted Yu <yuzhih...@gmail.com> wrote:
>
> > I tried to delete the HBASE-14123 branch using commands I found on
> > http://stackoverflow.com/questions/2003505/how-to-
> > delete-a-git-branch-both-locally-and-remotely
> >
> > Not sure if there is lag on github side:
> >
> > $ git push origin :origin/HBASE-14123
> > error: unable to delete 'origin/HBASE-14123': remote ref does not exist
> > error: failed to push some refs to '
> > https://git-wip-us.apache.org/repos/asf/hbase.git'
> >
> > FYI
> >
> > On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar <e...@apache.org> wrote:
> >
> > > I think there is some misconception of using the term "blockers" for
> > > referring to those jiras. My understanding is that those three jiras
> are
> > > blockers for the backup functionality to be more mature and more
> usable.
> > > But they are not release blockers. Let's say we merged the code in, and
> > for
> > > some reason those did not get addressed in time. We can still do the
> 2.0
> > > release without having to wait for the commits. We can instead mark the
> > > "backup" feature as experimental with known issues and go on with the
> > > release. In that sense they are not real release blockers.
> > >
> > > We are proposing the merge at this time because of the above that
> > > maintaining this in a branch is becoming extremely costly and not
> > > productive for the HBase community. Realistically, we cannot have the
> > > luxury of having to wait another couple of months and doing yet another
> > > giant round of reviews because the code base is a moving target.
> > >
> > > Enis
> > >
> > > On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das <d...@hortonworks.com>
> > wrote:
> > >
> > > > Vlad, on the first point, I think what Stack is saying is that
> creating
> > > > the new branch (as Ted did) ignores the feedback incorporated thus
> far
> > in
> > > > the iterations of the mega-patch. That's a wrong way to go.
> > > > On the separation into a backup module, again, that was reverted to
> > ease
> > > > reviews of the mega-patch, and was noted as work to be done later. I
> > > think
> > > > Stack just wants to make the list of remaining work more complete by
> > > citing
> > > > that as pending work.
> > > > 
> > > > From: Vladimir Rodionov <vladrodio...@gmail.com>
> > > > Sent: Monday, March 13, 2017 3:09 PM
> > > > To: dev@hbase.apache.org
> > > > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote
> closing
> > > > 3/11/2017
> > > >
> > > > >>  It ignores the feedback
> > > >
> > > > If I "ignore" feedback, I put my comment - why? I am always  open for
> > > > further discussions. If reviewer does not insist on a particular
> > request
> > > -
> > > > it will be dropped. I think it is fair.
> > > >
> > > > >> he list is incomplete because a bunch of
> > > > >> follow-ons came of the review cycle (including moving
> backup/restore
> > > out
> > > > of
> > > > >> core to live in its own module).
> > > >
> > > > For those who were not following our lengthy conversation on a review
> > > > board, separation of a backup code into a separate module has been
> > > > done last year, but has been reverted back by request of a reviewer.
> > > >
> > > >
> > > > -Vladimir
> > > >
> > > > On Mon, Mar 13, 2017 at 2:23 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote:
> > >

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Andrew Purtell]

That's because this was pushed with a different name. It's gone now.

apurtell@onyx:~/src/hbase$ git branch -a | grep 14123
  remotes/origin/14123
apurtell@onyx:~/src/hbase$ git push origin :14123
Username for 'https://git-wip-us.apache.org': apurtell
Password for 'https://apurt...@git-wip-us.apache.org':
To https://git-wip-us.apache.org/repos/asf/hbase.git
 - [deleted] 14123


On Mon, Mar 13, 2017 at 4:13 PM, Ted Yu <yuzhih...@gmail.com> wrote:

> I tried to delete the HBASE-14123 branch using commands I found on
> http://stackoverflow.com/questions/2003505/how-to-
> delete-a-git-branch-both-locally-and-remotely
>
> Not sure if there is lag on github side:
>
> $ git push origin :origin/HBASE-14123
> error: unable to delete 'origin/HBASE-14123': remote ref does not exist
> error: failed to push some refs to '
> https://git-wip-us.apache.org/repos/asf/hbase.git'
>
> FYI
>
> On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar <e...@apache.org> wrote:
>
> > I think there is some misconception of using the term "blockers" for
> > referring to those jiras. My understanding is that those three jiras are
> > blockers for the backup functionality to be more mature and more usable.
> > But they are not release blockers. Let's say we merged the code in, and
> for
> > some reason those did not get addressed in time. We can still do the 2.0
> > release without having to wait for the commits. We can instead mark the
> > "backup" feature as experimental with known issues and go on with the
> > release. In that sense they are not real release blockers.
> >
> > We are proposing the merge at this time because of the above that
> > maintaining this in a branch is becoming extremely costly and not
> > productive for the HBase community. Realistically, we cannot have the
> > luxury of having to wait another couple of months and doing yet another
> > giant round of reviews because the code base is a moving target.
> >
> > Enis
> >
> > On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das <d...@hortonworks.com>
> wrote:
> >
> > > Vlad, on the first point, I think what Stack is saying is that creating
> > > the new branch (as Ted did) ignores the feedback incorporated thus far
> in
> > > the iterations of the mega-patch. That's a wrong way to go.
> > > On the separation into a backup module, again, that was reverted to
> ease
> > > reviews of the mega-patch, and was noted as work to be done later. I
> > think
> > > Stack just wants to make the list of remaining work more complete by
> > citing
> > > that as pending work.
> > > 
> > > From: Vladimir Rodionov <vladrodio...@gmail.com>
> > > Sent: Monday, March 13, 2017 3:09 PM
> > > To: dev@hbase.apache.org
> > > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
> > > 3/11/2017
> > >
> > > >>  It ignores the feedback
> > >
> > > If I "ignore" feedback, I put my comment - why? I am always  open for
> > > further discussions. If reviewer does not insist on a particular
> request
> > -
> > > it will be dropped. I think it is fair.
> > >
> > > >> he list is incomplete because a bunch of
> > > >> follow-ons came of the review cycle (including moving backup/restore
> > out
> > > of
> > > >> core to live in its own module).
> > >
> > > For those who were not following our lengthy conversation on a review
> > > board, separation of a backup code into a separate module has been
> > > done last year, but has been reverted back by request of a reviewer.
> > >
> > >
> > > -Vladimir
> > >
> > > On Mon, Mar 13, 2017 at 2:23 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote:
> > > >
> > > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhih...@gmail.com>
> wrote:
> > > > >
> > > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> > > > >>
> > > > >
> > > > > The patch put up for VOTE here was done on a branch. The call to
> > merge
> > > > > seems to have been premature given the many cycles of review and
> test
> > > > that
> > > > > happened subsequent (The cycles burned a bunch of dev resource).
> > > > >
> > > > > The patch as is is now in a state where it is too bi

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Andrew Purtell]

I don't like that issues were identified as "blockers" but now there is an
attempt to walk that back.

I don't like that development of this feature has lingered for a long time
in this unfinished state when this work could have been done by now, now
that we are trying to get a 2.0 out the door. Because this is a volunteer
project I cannot make any demand that it should be done, but I can
certainly look at the current state and be nonplussed. This will be yet
another half finished thing in 2.0 when this merge happens. Promises to
finish the unfinished work are nice but not currency. Commits are currency.
I hope at least the fault tolerance changes can be completed and committed
before we spin a 2.0 RC, and without causing a 2.0 release to slip further.

Also, marking something experimental should be done on the merits of that
evaluation, not simply to justify dropping unfinished work into a release
branch.

I will change my vote to -0.


On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar <e...@apache.org> wrote:

> I think there is some misconception of using the term "blockers" for
> referring to those jiras. My understanding is that those three jiras are
> blockers for the backup functionality to be more mature and more usable.
> But they are not release blockers. Let's say we merged the code in, and for
> some reason those did not get addressed in time. We can still do the 2.0
> release without having to wait for the commits. We can instead mark the
> "backup" feature as experimental with known issues and go on with the
> release. In that sense they are not real release blockers.
>
> We are proposing the merge at this time because of the above that
> maintaining this in a branch is becoming extremely costly and not
> productive for the HBase community. Realistically, we cannot have the
> luxury of having to wait another couple of months and doing yet another
> giant round of reviews because the code base is a moving target.
>
> Enis
>
> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das <d...@hortonworks.com> wrote:
>
> > Vlad, on the first point, I think what Stack is saying is that creating
> > the new branch (as Ted did) ignores the feedback incorporated thus far in
> > the iterations of the mega-patch. That's a wrong way to go.
> > On the separation into a backup module, again, that was reverted to ease
> > reviews of the mega-patch, and was noted as work to be done later. I
> think
> > Stack just wants to make the list of remaining work more complete by
> citing
> > that as pending work.
> > ____
> > From: Vladimir Rodionov <vladrodio...@gmail.com>
> > Sent: Monday, March 13, 2017 3:09 PM
> > To: dev@hbase.apache.org
> > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
> > 3/11/2017
> >
> > >>  It ignores the feedback
> >
> > If I "ignore" feedback, I put my comment - why? I am always  open for
> > further discussions. If reviewer does not insist on a particular request
> -
> > it will be dropped. I think it is fair.
> >
> > >> he list is incomplete because a bunch of
> > >> follow-ons came of the review cycle (including moving backup/restore
> out
> > of
> > >> core to live in its own module).
> >
> > For those who were not following our lengthy conversation on a review
> > board, separation of a backup code into a separate module has been
> > done last year, but has been reverted back by request of a reviewer.
> >
> >
> > -Vladimir
> >
> > On Mon, Mar 13, 2017 at 2:23 PM, Stack <st...@duboce.net> wrote:
> >
> > > On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > > >
> > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> > > >>
> > > >
> > > > The patch put up for VOTE here was done on a branch. The call to
> merge
> > > > seems to have been premature given the many cycles of review and test
> > > that
> > > > happened subsequent (The cycles burned a bunch of dev resource).
> > > >
> > > > The patch as is is now in a state where it is too big for our infra;
> rb
> > > > and JIRA are creaking under the size and # of iterations.
> > > >
> > > > Adding finish of new JIRAs to this merge implies a new round of
> review
> > > and
> > > > test of an already massive patch. Who is going to do this work?
> > > >
> > > > Going bac

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

I tried to delete the HBASE-14123 branch using commands I found on
http://stackoverflow.com/questions/2003505/how-to-delete-a-git-branch-both-locally-and-remotely

Not sure if there is lag on github side:

$ git push origin :origin/HBASE-14123
error: unable to delete 'origin/HBASE-14123': remote ref does not exist
error: failed to push some refs to '
https://git-wip-us.apache.org/repos/asf/hbase.git'

FYI

On Mon, Mar 13, 2017 at 4:05 PM, Enis Söztutar <e...@apache.org> wrote:

> I think there is some misconception of using the term "blockers" for
> referring to those jiras. My understanding is that those three jiras are
> blockers for the backup functionality to be more mature and more usable.
> But they are not release blockers. Let's say we merged the code in, and for
> some reason those did not get addressed in time. We can still do the 2.0
> release without having to wait for the commits. We can instead mark the
> "backup" feature as experimental with known issues and go on with the
> release. In that sense they are not real release blockers.
>
> We are proposing the merge at this time because of the above that
> maintaining this in a branch is becoming extremely costly and not
> productive for the HBase community. Realistically, we cannot have the
> luxury of having to wait another couple of months and doing yet another
> giant round of reviews because the code base is a moving target.
>
> Enis
>
> On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das <d...@hortonworks.com> wrote:
>
> > Vlad, on the first point, I think what Stack is saying is that creating
> > the new branch (as Ted did) ignores the feedback incorporated thus far in
> > the iterations of the mega-patch. That's a wrong way to go.
> > On the separation into a backup module, again, that was reverted to ease
> > reviews of the mega-patch, and was noted as work to be done later. I
> think
> > Stack just wants to make the list of remaining work more complete by
> citing
> > that as pending work.
> > ____
> > From: Vladimir Rodionov <vladrodio...@gmail.com>
> > Sent: Monday, March 13, 2017 3:09 PM
> > To: dev@hbase.apache.org
> > Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
> > 3/11/2017
> >
> > >>  It ignores the feedback
> >
> > If I "ignore" feedback, I put my comment - why? I am always  open for
> > further discussions. If reviewer does not insist on a particular request
> -
> > it will be dropped. I think it is fair.
> >
> > >> he list is incomplete because a bunch of
> > >> follow-ons came of the review cycle (including moving backup/restore
> out
> > of
> > >> core to live in its own module).
> >
> > For those who were not following our lengthy conversation on a review
> > board, separation of a backup code into a separate module has been
> > done last year, but has been reverted back by request of a reviewer.
> >
> >
> > -Vladimir
> >
> > On Mon, Mar 13, 2017 at 2:23 PM, Stack <st...@duboce.net> wrote:
> >
> > > On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote:
> > >
> > > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > > >
> > > >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> > > >>
> > > >
> > > > The patch put up for VOTE here was done on a branch. The call to
> merge
> > > > seems to have been premature given the many cycles of review and test
> > > that
> > > > happened subsequent (The cycles burned a bunch of dev resource).
> > > >
> > > > The patch as is is now in a state where it is too big for our infra;
> rb
> > > > and JIRA are creaking under the size and # of iterations.
> > > >
> > > > Adding finish of new JIRAs to this merge implies a new round of
> review
> > > and
> > > > test of an already massive patch. Who is going to do this work?
> > > >
> > > > Going back to a new branch seems wrong route to take.
> > > >
> > > > St.Ack
> > > >
> > > >
> > > >
> > > To be more explicit, this patch was developed on a branch and then a
> > bunch
> > > of dev resources were burned getting it into a state where it could be
> > > merged to master. Going back to a branch to bulk up the merge so it
> > > includes more JIRAs than the many it already incorporates is the wrong
> > > direction for us to be headed in

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Enis Söztutar]

I think there is some misconception of using the term "blockers" for
referring to those jiras. My understanding is that those three jiras are
blockers for the backup functionality to be more mature and more usable.
But they are not release blockers. Let's say we merged the code in, and for
some reason those did not get addressed in time. We can still do the 2.0
release without having to wait for the commits. We can instead mark the
"backup" feature as experimental with known issues and go on with the
release. In that sense they are not real release blockers.

We are proposing the merge at this time because of the above that
maintaining this in a branch is becoming extremely costly and not
productive for the HBase community. Realistically, we cannot have the
luxury of having to wait another couple of months and doing yet another
giant round of reviews because the code base is a moving target.

Enis

On Mon, Mar 13, 2017 at 3:46 PM, Devaraj Das <d...@hortonworks.com> wrote:

> Vlad, on the first point, I think what Stack is saying is that creating
> the new branch (as Ted did) ignores the feedback incorporated thus far in
> the iterations of the mega-patch. That's a wrong way to go.
> On the separation into a backup module, again, that was reverted to ease
> reviews of the mega-patch, and was noted as work to be done later. I think
> Stack just wants to make the list of remaining work more complete by citing
> that as pending work.
> 
> From: Vladimir Rodionov <vladrodio...@gmail.com>
> Sent: Monday, March 13, 2017 3:09 PM
> To: dev@hbase.apache.org
> Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing
> 3/11/2017
>
> >>  It ignores the feedback
>
> If I "ignore" feedback, I put my comment - why? I am always  open for
> further discussions. If reviewer does not insist on a particular request -
> it will be dropped. I think it is fair.
>
> >> he list is incomplete because a bunch of
> >> follow-ons came of the review cycle (including moving backup/restore out
> of
> >> core to live in its own module).
>
> For those who were not following our lengthy conversation on a review
> board, separation of a backup code into a separate module has been
> done last year, but has been reverted back by request of a reviewer.
>
>
> -Vladimir
>
> On Mon, Mar 13, 2017 at 2:23 PM, Stack <st...@duboce.net> wrote:
>
> > On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote:
> >
> > > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> > >
> > >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> > >>
> > >
> > > The patch put up for VOTE here was done on a branch. The call to merge
> > > seems to have been premature given the many cycles of review and test
> > that
> > > happened subsequent (The cycles burned a bunch of dev resource).
> > >
> > > The patch as is is now in a state where it is too big for our infra; rb
> > > and JIRA are creaking under the size and # of iterations.
> > >
> > > Adding finish of new JIRAs to this merge implies a new round of review
> > and
> > > test of an already massive patch. Who is going to do this work?
> > >
> > > Going back to a new branch seems wrong route to take.
> > >
> > > St.Ack
> > >
> > >
> > >
> > To be more explicit, this patch was developed on a branch and then a
> bunch
> > of dev resources were burned getting it into a state where it could be
> > merged to master. Going back to a branch to bulk up the merge so it
> > includes more JIRAs than the many it already incorporates is the wrong
> > direction for us to be headed in. It ignores the feedback given and the
> > work done by Vladimir slimming down an already over-broad scope. It is
> also
> > predicated on abundant review and testing resource being on tap to cycle
> on
> > a feature that is useful, but non-core.
> >
> > The patch is ready for merge IMO. Geoffrey makes a nice list of what is
> > still to do though IIRC, the list is incomplete because a bunch of
> > follow-ons came of the review cycle (including moving backup/restore out
> of
> > core to live in its own module).
> >
> > The patch needs three votes to merge. I am not against merge but I am not
> > voting for the patch because I do have any more time to spend on this
> > non-core feature and feel that a vote will have me assume a
> responsibility
> > I will not shirk.
> >
> > S
> >
> >
> >
> > >
> >

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Devaraj Das]

Vlad, on the first point, I think what Stack is saying is that creating the new 
branch (as Ted did) ignores the feedback incorporated thus far in the 
iterations of the mega-patch. That's a wrong way to go.
On the separation into a backup module, again, that was reverted to ease 
reviews of the mega-patch, and was noted as work to be done later. I think 
Stack just wants to make the list of remaining work more complete by citing 
that as pending work.

From: Vladimir Rodionov <vladrodio...@gmail.com>
Sent: Monday, March 13, 2017 3:09 PM
To: dev@hbase.apache.org
Subject: Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

>>  It ignores the feedback

If I "ignore" feedback, I put my comment - why? I am always  open for
further discussions. If reviewer does not insist on a particular request -
it will be dropped. I think it is fair.

>> he list is incomplete because a bunch of
>> follow-ons came of the review cycle (including moving backup/restore out
of
>> core to live in its own module).

For those who were not following our lengthy conversation on a review
board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack <st...@duboce.net> wrote:

> On Fri, Mar 10, 2017 at 9:09 PM, Stack <st...@duboce.net> wrote:
>
> > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> >
> >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> >>
> >
> > The patch put up for VOTE here was done on a branch. The call to merge
> > seems to have been premature given the many cycles of review and test
> that
> > happened subsequent (The cycles burned a bunch of dev resource).
> >
> > The patch as is is now in a state where it is too big for our infra; rb
> > and JIRA are creaking under the size and # of iterations.
> >
> > Adding finish of new JIRAs to this merge implies a new round of review
> and
> > test of an already massive patch. Who is going to do this work?
> >
> > Going back to a new branch seems wrong route to take.
> >
> > St.Ack
> >
> >
> >
> To be more explicit, this patch was developed on a branch and then a bunch
> of dev resources were burned getting it into a state where it could be
> merged to master. Going back to a branch to bulk up the merge so it
> includes more JIRAs than the many it already incorporates is the wrong
> direction for us to be headed in. It ignores the feedback given and the
> work done by Vladimir slimming down an already over-broad scope. It is also
> predicated on abundant review and testing resource being on tap to cycle on
> a feature that is useful, but non-core.
>
> The patch is ready for merge IMO. Geoffrey makes a nice list of what is
> still to do though IIRC, the list is incomplete because a bunch of
> follow-ons came of the review cycle (including moving backup/restore out of
> core to live in its own module).
>
> The patch needs three votes to merge. I am not against merge but I am not
> voting for the patch because I do have any more time to spend on this
> non-core feature and feel that a vote will have me assume a responsibility
> I will not shirk.
>
> S
>
>
>
> >
> >
> >
> >> FYI
> >>
> >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu <yuzhih...@gmail.com> wrote:
> >>
> >> > Thanks for the feedback, Andrew.
> >> >
> >> > How about the following plan:
> >> >
> >> > create branch HBASE-14123 off of master with mega patch v61 as the
> first
> >> > commit (reviewed by Stack and Enis)
> >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123
> >> >  branch
> >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we
> >> > propose to community for merging into branch-2 (master branch, if
> >> branch-2
> >> > doesn't materialize for whatever reason) again
> >> >
> >> > Cheers
> >> >
> >> >
> >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell <apurt...@apache.org>
> >> > wrote:
> >> >
> >> >> Thanks for the offer but I like that you were honest about compiling
> a
> >> >> list
> >> >> of issues that you thought were blockers for release. Since this
> >> proposal
> >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on
> this
> >> >> merge until those blo

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

>>  It ignores the feedback

If I "ignore" feedback, I put my comment - why? I am always  open for
further discussions. If reviewer does not insist on a particular request -
it will be dropped. I think it is fair.

>> he list is incomplete because a bunch of
>> follow-ons came of the review cycle (including moving backup/restore out
of
>> core to live in its own module).

For those who were not following our lengthy conversation on a review
board, separation of a backup code into a separate module has been
done last year, but has been reverted back by request of a reviewer.


-Vladimir

On Mon, Mar 13, 2017 at 2:23 PM, Stack  wrote:

> On Fri, Mar 10, 2017 at 9:09 PM, Stack  wrote:
>
> > On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu  wrote:
> >
> >> HBASE-14123 branch has been created, with Vlad's mega patch v61.
> >>
> >
> > The patch put up for VOTE here was done on a branch. The call to merge
> > seems to have been premature given the many cycles of review and test
> that
> > happened subsequent (The cycles burned a bunch of dev resource).
> >
> > The patch as is is now in a state where it is too big for our infra; rb
> > and JIRA are creaking under the size and # of iterations.
> >
> > Adding finish of new JIRAs to this merge implies a new round of review
> and
> > test of an already massive patch. Who is going to do this work?
> >
> > Going back to a new branch seems wrong route to take.
> >
> > St.Ack
> >
> >
> >
> To be more explicit, this patch was developed on a branch and then a bunch
> of dev resources were burned getting it into a state where it could be
> merged to master. Going back to a branch to bulk up the merge so it
> includes more JIRAs than the many it already incorporates is the wrong
> direction for us to be headed in. It ignores the feedback given and the
> work done by Vladimir slimming down an already over-broad scope. It is also
> predicated on abundant review and testing resource being on tap to cycle on
> a feature that is useful, but non-core.
>
> The patch is ready for merge IMO. Geoffrey makes a nice list of what is
> still to do though IIRC, the list is incomplete because a bunch of
> follow-ons came of the review cycle (including moving backup/restore out of
> core to live in its own module).
>
> The patch needs three votes to merge. I am not against merge but I am not
> voting for the patch because I do have any more time to spend on this
> non-core feature and feel that a vote will have me assume a responsibility
> I will not shirk.
>
> S
>
>
>
> >
> >
> >
> >> FYI
> >>
> >> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu  wrote:
> >>
> >> > Thanks for the feedback, Andrew.
> >> >
> >> > How about the following plan:
> >> >
> >> > create branch HBASE-14123 off of master with mega patch v61 as the
> first
> >> > commit (reviewed by Stack and Enis)
> >> > Vlad and I continue development (the 3 blockers) based on HBASE-14123
> >> >  branch
> >> > when all of the blockers get +1 and merged into HBASE-14123 branch, we
> >> > propose to community for merging into branch-2 (master branch, if
> >> branch-2
> >> > doesn't materialize for whatever reason) again
> >> >
> >> > Cheers
> >> >
> >> >
> >> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell 
> >> > wrote:
> >> >
> >> >> Thanks for the offer but I like that you were honest about compiling
> a
> >> >> list
> >> >> of issues that you thought were blockers for release. Since this
> >> proposal
> >> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on
> this
> >> >> merge until those blockers are addressed.
> >> >>
> >> >> I had a look at the list.
> >> >>
> >> >> I think the documentation issue is important but not actually a
> >> blocker.
> >> >> That may be a controversial opinion, but documentation can be
> >> back-filled
> >> >> worst case. So take HBASE-17133 off the list.
> >> >>
> >> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227.
> >> They
> >> >> all have patches attached to the respective JIRAs so completing this
> >> work
> >> >> won't be onerous. Get these committed and I will lift my -1. The
> others
> >> >> who
> >> >> voted +1 on this thread surely can help with that.
> >> >>
> >> >> Thanks.
> >> >>
> >> >>
> >> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov <
> >> >> vladrodio...@gmail.com>
> >> >> wrote:
> >> >>
> >> >> > No problem I will downgrade Blockers to Majors if it scares you,
> >> Andrew
> >> >> 
> >> >> >
> >> >> > Sent from my iPhone
> >> >> >
> >> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell  >
> >> >> wrote:
> >> >> > >
> >> >> > > ​I know the merge of this feature has lagged substantially. I
> think
> >> >> that
> >> >> > is
> >> >> > > regrettable but on another thread we are lamenting that 2.0 is
> >> already
> >> >> > > late. Unless I misunderstand, this is a proposal to merge
> something
> >> >> with
> >> >> > > known blockers into 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Stack]

On Fri, Mar 10, 2017 at 9:09 PM, Stack  wrote:

> On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu  wrote:
>
>> HBASE-14123 branch has been created, with Vlad's mega patch v61.
>>
>
> The patch put up for VOTE here was done on a branch. The call to merge
> seems to have been premature given the many cycles of review and test that
> happened subsequent (The cycles burned a bunch of dev resource).
>
> The patch as is is now in a state where it is too big for our infra; rb
> and JIRA are creaking under the size and # of iterations.
>
> Adding finish of new JIRAs to this merge implies a new round of review and
> test of an already massive patch. Who is going to do this work?
>
> Going back to a new branch seems wrong route to take.
>
> St.Ack
>
>
>
To be more explicit, this patch was developed on a branch and then a bunch
of dev resources were burned getting it into a state where it could be
merged to master. Going back to a branch to bulk up the merge so it
includes more JIRAs than the many it already incorporates is the wrong
direction for us to be headed in. It ignores the feedback given and the
work done by Vladimir slimming down an already over-broad scope. It is also
predicated on abundant review and testing resource being on tap to cycle on
a feature that is useful, but non-core.

The patch is ready for merge IMO. Geoffrey makes a nice list of what is
still to do though IIRC, the list is incomplete because a bunch of
follow-ons came of the review cycle (including moving backup/restore out of
core to live in its own module).

The patch needs three votes to merge. I am not against merge but I am not
voting for the patch because I do have any more time to spend on this
non-core feature and feel that a vote will have me assume a responsibility
I will not shirk.

S



>
>
>
>> FYI
>>
>> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu  wrote:
>>
>> > Thanks for the feedback, Andrew.
>> >
>> > How about the following plan:
>> >
>> > create branch HBASE-14123 off of master with mega patch v61 as the first
>> > commit (reviewed by Stack and Enis)
>> > Vlad and I continue development (the 3 blockers) based on HBASE-14123
>> >  branch
>> > when all of the blockers get +1 and merged into HBASE-14123 branch, we
>> > propose to community for merging into branch-2 (master branch, if
>> branch-2
>> > doesn't materialize for whatever reason) again
>> >
>> > Cheers
>> >
>> >
>> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell 
>> > wrote:
>> >
>> >> Thanks for the offer but I like that you were honest about compiling a
>> >> list
>> >> of issues that you thought were blockers for release. Since this
>> proposal
>> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this
>> >> merge until those blockers are addressed.
>> >>
>> >> I had a look at the list.
>> >>
>> >> I think the documentation issue is important but not actually a
>> blocker.
>> >> That may be a controversial opinion, but documentation can be
>> back-filled
>> >> worst case. So take HBASE-17133 off the list.
>> >>
>> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227.
>> They
>> >> all have patches attached to the respective JIRAs so completing this
>> work
>> >> won't be onerous. Get these committed and I will lift my -1. The others
>> >> who
>> >> voted +1 on this thread surely can help with that.
>> >>
>> >> Thanks.
>> >>
>> >>
>> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov <
>> >> vladrodio...@gmail.com>
>> >> wrote:
>> >>
>> >> > No problem I will downgrade Blockers to Majors if it scares you,
>> Andrew
>> >> 
>> >> >
>> >> > Sent from my iPhone
>> >> >
>> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell 
>> >> wrote:
>> >> > >
>> >> > > ​I know the merge of this feature has lagged substantially. I think
>> >> that
>> >> > is
>> >> > > regrettable but on another thread we are lamenting that 2.0 is
>> already
>> >> > > late. Unless I misunderstand, this is a proposal to merge something
>> >> with
>> >> > > known blockers into trunk before we branch it for 2.0 which will
>> >> > > effectively prevent that release because these blockers will be
>> >> there. I
>> >> > am
>> >> > > inclined to veto. Probably we should not propose branch merges into
>> >> code
>> >> > we
>> >> > > are trying to get out the door with known blockers. Why not do that
>> >> work
>> >> > > first? It seems an obvious question. Perhaps I am missing
>> something.
>> >> > >
>> >> > > If we can branch for 2.0 now and then merge this, and not into the
>> 2.0
>> >> > > branch, I would vote +1 for branch merge even with known blockers
>> >> > pending.
>> >> > > ​
>> >> > >
>> >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
>> >> > vladrodio...@gmail.com>
>> >> > > wrote:
>> >> > >
>> >> > >> They are not blockers for merge - only for 2.0. GA
>> >> > >> As I said already the feature is usable right now
>> >> > >> We would like to continue 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Stack]

On Fri, Mar 10, 2017 at 6:01 PM, Ted Yu  wrote:

> HBASE-14123 branch has been created, with Vlad's mega patch v61.
>

The patch put up for VOTE here was done on a branch. The call to merge
seems to have been premature given the many cycles of review and test that
happened subsequent (The cycles burned a bunch of dev resource).

The patch as is is now in a state where it is too big for our infra; rb and
JIRA are creaking under the size and # of iterations.

Adding finish of new JIRAs to this merge implies a new round of review and
test of an already massive patch. Who is going to do this work?

Going back to a new branch seems wrong route to take.

St.Ack





> FYI
>
> On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu  wrote:
>
> > Thanks for the feedback, Andrew.
> >
> > How about the following plan:
> >
> > create branch HBASE-14123 off of master with mega patch v61 as the first
> > commit (reviewed by Stack and Enis)
> > Vlad and I continue development (the 3 blockers) based on HBASE-14123
> >  branch
> > when all of the blockers get +1 and merged into HBASE-14123 branch, we
> > propose to community for merging into branch-2 (master branch, if
> branch-2
> > doesn't materialize for whatever reason) again
> >
> > Cheers
> >
> >
> > On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell 
> > wrote:
> >
> >> Thanks for the offer but I like that you were honest about compiling a
> >> list
> >> of issues that you thought were blockers for release. Since this
> proposal
> >> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this
> >> merge until those blockers are addressed.
> >>
> >> I had a look at the list.
> >>
> >> I think the documentation issue is important but not actually a blocker.
> >> That may be a controversial opinion, but documentation can be
> back-filled
> >> worst case. So take HBASE-17133 off the list.
> >>
> >> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227.
> They
> >> all have patches attached to the respective JIRAs so completing this
> work
> >> won't be onerous. Get these committed and I will lift my -1. The others
> >> who
> >> voted +1 on this thread surely can help with that.
> >>
> >> Thanks.
> >>
> >>
> >> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov <
> >> vladrodio...@gmail.com>
> >> wrote:
> >>
> >> > No problem I will downgrade Blockers to Majors if it scares you,
> Andrew
> >> 
> >> >
> >> > Sent from my iPhone
> >> >
> >> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell 
> >> wrote:
> >> > >
> >> > > ​I know the merge of this feature has lagged substantially. I think
> >> that
> >> > is
> >> > > regrettable but on another thread we are lamenting that 2.0 is
> already
> >> > > late. Unless I misunderstand, this is a proposal to merge something
> >> with
> >> > > known blockers into trunk before we branch it for 2.0 which will
> >> > > effectively prevent that release because these blockers will be
> >> there. I
> >> > am
> >> > > inclined to veto. Probably we should not propose branch merges into
> >> code
> >> > we
> >> > > are trying to get out the door with known blockers. Why not do that
> >> work
> >> > > first? It seems an obvious question. Perhaps I am missing something.
> >> > >
> >> > > If we can branch for 2.0 now and then merge this, and not into the
> 2.0
> >> > > branch, I would vote +1 for branch merge even with known blockers
> >> > pending.
> >> > > ​
> >> > >
> >> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
> >> > vladrodio...@gmail.com>
> >> > > wrote:
> >> > >
> >> > >> They are not blockers for merge - only for 2.0. GA
> >> > >> As I said already the feature is usable right now
> >> > >> We would like to continue working on master and we would like to
> see
> >> a
> >> > >> commitment from community
> >> > >>
> >> > >> Sent from my iPhone
> >> > >>
> >> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell 
> >> > wrote:
> >> > >>
> >> >  Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> >> > >>>
> >> > >>> If we have identified blockers, why merge this before they are in?
> >> > >>> Otherwise we can't release 2.0, and it is overdue.
> >> > >>>
> >> > >>>
> >> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
> >> > >> vladrodio...@gmail.com>
> >> > >>> wrote:
> >> > >>>
> >> >  Hello, HBase folks
> >> > 
> >> >  For your consideration today is Backup/Restore feature for Apache
> >> > HBAse
> >> >  2.0.
> >> >  Backup code is available as a mega patch in HBASE-14123 (v61),
> >> applies
> >> >  cleanly to the current master, all test PASS, patch has no other
> >> > issues.
> >> > 
> >> >  The patch has gone through numerous rounds of code reviews and
> has
> >> > >> probably
> >> >  the most lengthy discussion thread on Apache JIRA (HBASE-14123)
> :)
> >> > 
> >> >  The work has been split into 3 phases (HBASE-14030, 14123, 14414)
> >> Two
> >> 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

HBASE-14123 branch has been created, with Vlad's mega patch v61.

FYI

On Fri, Mar 10, 2017 at 3:30 PM, Ted Yu  wrote:

> Thanks for the feedback, Andrew.
>
> How about the following plan:
>
> create branch HBASE-14123 off of master with mega patch v61 as the first
> commit (reviewed by Stack and Enis)
> Vlad and I continue development (the 3 blockers) based on HBASE-14123
>  branch
> when all of the blockers get +1 and merged into HBASE-14123 branch, we
> propose to community for merging into branch-2 (master branch, if branch-2
> doesn't materialize for whatever reason) again
>
> Cheers
>
>
> On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell 
> wrote:
>
>> Thanks for the offer but I like that you were honest about compiling a
>> list
>> of issues that you thought were blockers for release. Since this proposal
>> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this
>> merge until those blockers are addressed.
>>
>> I had a look at the list.
>>
>> I think the documentation issue is important but not actually a blocker.
>> That may be a controversial opinion, but documentation can be back-filled
>> worst case. So take HBASE-17133 off the list.
>>
>> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They
>> all have patches attached to the respective JIRAs so completing this work
>> won't be onerous. Get these committed and I will lift my -1. The others
>> who
>> voted +1 on this thread surely can help with that.
>>
>> Thanks.
>>
>>
>> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov <
>> vladrodio...@gmail.com>
>> wrote:
>>
>> > No problem I will downgrade Blockers to Majors if it scares you, Andrew
>> 
>> >
>> > Sent from my iPhone
>> >
>> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell 
>> wrote:
>> > >
>> > > ​I know the merge of this feature has lagged substantially. I think
>> that
>> > is
>> > > regrettable but on another thread we are lamenting that 2.0 is already
>> > > late. Unless I misunderstand, this is a proposal to merge something
>> with
>> > > known blockers into trunk before we branch it for 2.0 which will
>> > > effectively prevent that release because these blockers will be
>> there. I
>> > am
>> > > inclined to veto. Probably we should not propose branch merges into
>> code
>> > we
>> > > are trying to get out the door with known blockers. Why not do that
>> work
>> > > first? It seems an obvious question. Perhaps I am missing something.
>> > >
>> > > If we can branch for 2.0 now and then merge this, and not into the 2.0
>> > > branch, I would vote +1 for branch merge even with known blockers
>> > pending.
>> > > ​
>> > >
>> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
>> > vladrodio...@gmail.com>
>> > > wrote:
>> > >
>> > >> They are not blockers for merge - only for 2.0. GA
>> > >> As I said already the feature is usable right now
>> > >> We would like to continue working on master and we would like to see
>> a
>> > >> commitment from community
>> > >>
>> > >> Sent from my iPhone
>> > >>
>> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell 
>> > wrote:
>> > >>
>> >  Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>> > >>>
>> > >>> If we have identified blockers, why merge this before they are in?
>> > >>> Otherwise we can't release 2.0, and it is overdue.
>> > >>>
>> > >>>
>> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
>> > >> vladrodio...@gmail.com>
>> > >>> wrote:
>> > >>>
>> >  Hello, HBase folks
>> > 
>> >  For your consideration today is Backup/Restore feature for Apache
>> > HBAse
>> >  2.0.
>> >  Backup code is available as a mega patch in HBASE-14123 (v61),
>> applies
>> >  cleanly to the current master, all test PASS, patch has no other
>> > issues.
>> > 
>> >  The patch has gone through numerous rounds of code reviews and has
>> > >> probably
>> >  the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
>> > 
>> >  The work has been split into 3 phases (HBASE-14030, 14123, 14414)
>> Two
>> > >> first
>> >  are complete, third one is still in progress.
>> > 
>> > 
>> >  *** Summary of work HBASE-14123
>> > 
>> >  The new feature introduces new command-line extensions to the hbase
>> > >> command
>> >  and, from the client side, is accessible through command-line only
>> >  Operations:
>> >  * Create full backup on a list of tables or backup set
>> >  * Create incremental backup image for table list or backup set
>> >  * Restore list of tables from a given backup image
>> >  * Show current backup progress
>> >  * Delete backup image and all related images
>> >  * Show history of backups
>> >  * Backup set operations: create backup set, add/remove table
>> to/from
>> > >> backup
>> >  set, etc
>> > 
>> >  In the current implementation, the feature is already usable,
>> meaning
>> > >> 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

Thanks for the feedback, Andrew.

How about the following plan:

create branch HBASE-14123 off of master with mega patch v61 as the first
commit (reviewed by Stack and Enis)
Vlad and I continue development (the 3 blockers) based on HBASE-14123 branch
when all of the blockers get +1 and merged into HBASE-14123 branch, we
propose to community for merging into branch-2 (master branch, if branch-2
doesn't materialize for whatever reason) again

Cheers


On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell  wrote:

> Thanks for the offer but I like that you were honest about compiling a list
> of issues that you thought were blockers for release. Since this proposal
> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this
> merge until those blockers are addressed.
>
> I had a look at the list.
>
> I think the documentation issue is important but not actually a blocker.
> That may be a controversial opinion, but documentation can be back-filled
> worst case. So take HBASE-17133 off the list.
>
> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They
> all have patches attached to the respective JIRAs so completing this work
> won't be onerous. Get these committed and I will lift my -1. The others who
> voted +1 on this thread surely can help with that.
>
> Thanks.
>
>
> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov  >
> wrote:
>
> > No problem I will downgrade Blockers to Majors if it scares you, Andrew
> 
> >
> > Sent from my iPhone
> >
> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell 
> wrote:
> > >
> > > ​I know the merge of this feature has lagged substantially. I think
> that
> > is
> > > regrettable but on another thread we are lamenting that 2.0 is already
> > > late. Unless I misunderstand, this is a proposal to merge something
> with
> > > known blockers into trunk before we branch it for 2.0 which will
> > > effectively prevent that release because these blockers will be there.
> I
> > am
> > > inclined to veto. Probably we should not propose branch merges into
> code
> > we
> > > are trying to get out the door with known blockers. Why not do that
> work
> > > first? It seems an obvious question. Perhaps I am missing something.
> > >
> > > If we can branch for 2.0 now and then merge this, and not into the 2.0
> > > branch, I would vote +1 for branch merge even with known blockers
> > pending.
> > > ​
> > >
> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
> > vladrodio...@gmail.com>
> > > wrote:
> > >
> > >> They are not blockers for merge - only for 2.0. GA
> > >> As I said already the feature is usable right now
> > >> We would like to continue working on master and we would like to see a
> > >> commitment from community
> > >>
> > >> Sent from my iPhone
> > >>
> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell 
> > wrote:
> > >>
> >  Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> > >>>
> > >>> If we have identified blockers, why merge this before they are in?
> > >>> Otherwise we can't release 2.0, and it is overdue.
> > >>>
> > >>>
> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
> > >> vladrodio...@gmail.com>
> > >>> wrote:
> > >>>
> >  Hello, HBase folks
> > 
> >  For your consideration today is Backup/Restore feature for Apache
> > HBAse
> >  2.0.
> >  Backup code is available as a mega patch in HBASE-14123 (v61),
> applies
> >  cleanly to the current master, all test PASS, patch has no other
> > issues.
> > 
> >  The patch has gone through numerous rounds of code reviews and has
> > >> probably
> >  the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
> > 
> >  The work has been split into 3 phases (HBASE-14030, 14123, 14414)
> Two
> > >> first
> >  are complete, third one is still in progress.
> > 
> > 
> >  *** Summary of work HBASE-14123
> > 
> >  The new feature introduces new command-line extensions to the hbase
> > >> command
> >  and, from the client side, is accessible through command-line only
> >  Operations:
> >  * Create full backup on a list of tables or backup set
> >  * Create incremental backup image for table list or backup set
> >  * Restore list of tables from a given backup image
> >  * Show current backup progress
> >  * Delete backup image and all related images
> >  * Show history of backups
> >  * Backup set operations: create backup set, add/remove table to/from
> > >> backup
> >  set, etc
> > 
> >  In the current implementation, the feature is already usable,
> meaning
> > >> that
> >  users can backup tables and restore them using provided command-line
> > >> tools.
> >  Both: full and incremental backups are supported.
> >  This work is based on original work of IBM team (HBASE-7912). The
> full
> > >> list
> >  of JIRAs included in this mega patch can 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Geoffrey Jacoby]

I have no vote here, but I'd argue that HBASE-14417 and HBASE-14141
shouldn't be blockers. I agree that HBASE-15227 to add fault tolerance is a
blocker.

HBASE-14417 is support for incrementally backing up bulk loaded rows.
That's an important feature, but if you don't use bulk loads, or don't care
about _incremental_ backups of bulk loads, you'd be able to use backup
quite happily without it. Even if you bulk load occasionally, you can do a
full backup afterward. In the meantime, the docs and help text would just
need to make the limitation clear in big bold letters. :-)

HBASE-14141 is allowing HBase Backup to filter out unnecessary data from
its incremental backups. Since the backup tool allows you to specify that
only certain tables be backed up, incremental backups at WAL granularity
will accidentally backup some rows from unneeded tables. This doesn't
affect the correctness of the to-be-backed-up tables backups or restores,
only its storage cost. The feature works, but there's an important storage
optimization to be done.

As someone who works on clusters that don't use bulk load, and would want
to backup all tables, neither of these seems like a showstopper. However,
if HBASE-14141 would be a breaking change to the backup format, then that
would change my mind about it being a blocker.

Geoffrey



On Fri, Mar 10, 2017 at 3:01 PM, Andrew Purtell  wrote:

> Thanks for the offer but I like that you were honest about compiling a list
> of issues that you thought were blockers for release. Since this proposal
> is a merge into 2.0, and we are trying to release 2.0, I am -1 on this
> merge until those blockers are addressed.
>
> I had a look at the list.
>
> I think the documentation issue is important but not actually a blocker.
> That may be a controversial opinion, but documentation can be back-filled
> worst case. So take HBASE-17133 off the list.
>
> Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They
> all have patches attached to the respective JIRAs so completing this work
> won't be onerous. Get these committed and I will lift my -1. The others who
> voted +1 on this thread surely can help with that.
>
> Thanks.
>
>
> On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov  >
> wrote:
>
> > No problem I will downgrade Blockers to Majors if it scares you, Andrew
> 
> >
> > Sent from my iPhone
> >
> > > On Mar 10, 2017, at 1:52 PM, Andrew Purtell 
> wrote:
> > >
> > > ​I know the merge of this feature has lagged substantially. I think
> that
> > is
> > > regrettable but on another thread we are lamenting that 2.0 is already
> > > late. Unless I misunderstand, this is a proposal to merge something
> with
> > > known blockers into trunk before we branch it for 2.0 which will
> > > effectively prevent that release because these blockers will be there.
> I
> > am
> > > inclined to veto. Probably we should not propose branch merges into
> code
> > we
> > > are trying to get out the door with known blockers. Why not do that
> work
> > > first? It seems an obvious question. Perhaps I am missing something.
> > >
> > > If we can branch for 2.0 now and then merge this, and not into the 2.0
> > > branch, I would vote +1 for branch merge even with known blockers
> > pending.
> > > ​
> > >
> > > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
> > vladrodio...@gmail.com>
> > > wrote:
> > >
> > >> They are not blockers for merge - only for 2.0. GA
> > >> As I said already the feature is usable right now
> > >> We would like to continue working on master and we would like to see a
> > >> commitment from community
> > >>
> > >> Sent from my iPhone
> > >>
> > >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell 
> > wrote:
> > >>
> >  Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> > >>>
> > >>> If we have identified blockers, why merge this before they are in?
> > >>> Otherwise we can't release 2.0, and it is overdue.
> > >>>
> > >>>
> > >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
> > >> vladrodio...@gmail.com>
> > >>> wrote:
> > >>>
> >  Hello, HBase folks
> > 
> >  For your consideration today is Backup/Restore feature for Apache
> > HBAse
> >  2.0.
> >  Backup code is available as a mega patch in HBASE-14123 (v61),
> applies
> >  cleanly to the current master, all test PASS, patch has no other
> > issues.
> > 
> >  The patch has gone through numerous rounds of code reviews and has
> > >> probably
> >  the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
> > 
> >  The work has been split into 3 phases (HBASE-14030, 14123, 14414)
> Two
> > >> first
> >  are complete, third one is still in progress.
> > 
> > 
> >  *** Summary of work HBASE-14123
> > 
> >  The new feature introduces new command-line extensions to the hbase
> > >> command
> >  and, from the client side, is accessible through 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Andrew Purtell]

Thanks for the offer but I like that you were honest about compiling a list
of issues that you thought were blockers for release. Since this proposal
is a merge into 2.0, and we are trying to release 2.0, I am -1 on this
merge until those blockers are addressed.

I had a look at the list.

I think the documentation issue is important but not actually a blocker.
That may be a controversial opinion, but documentation can be back-filled
worst case. So take HBASE-17133 off the list.

Remaining are effectively HBASE-14417, HBASE-14141, and HBASE-15227. They
all have patches attached to the respective JIRAs so completing this work
won't be onerous. Get these committed and I will lift my -1. The others who
voted +1 on this thread surely can help with that.

Thanks.


On Fri, Mar 10, 2017 at 2:32 PM, Vladimir Rodionov 
wrote:

> No problem I will downgrade Blockers to Majors if it scares you, Andrew 
>
> Sent from my iPhone
>
> > On Mar 10, 2017, at 1:52 PM, Andrew Purtell  wrote:
> >
> > ​I know the merge of this feature has lagged substantially. I think that
> is
> > regrettable but on another thread we are lamenting that 2.0 is already
> > late. Unless I misunderstand, this is a proposal to merge something with
> > known blockers into trunk before we branch it for 2.0 which will
> > effectively prevent that release because these blockers will be there. I
> am
> > inclined to veto. Probably we should not propose branch merges into code
> we
> > are trying to get out the door with known blockers. Why not do that work
> > first? It seems an obvious question. Perhaps I am missing something.
> >
> > If we can branch for 2.0 now and then merge this, and not into the 2.0
> > branch, I would vote +1 for branch merge even with known blockers
> pending.
> > ​
> >
> > On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov <
> vladrodio...@gmail.com>
> > wrote:
> >
> >> They are not blockers for merge - only for 2.0. GA
> >> As I said already the feature is usable right now
> >> We would like to continue working on master and we would like to see a
> >> commitment from community
> >>
> >> Sent from my iPhone
> >>
> >> On Mar 10, 2017, at 11:16 AM, Andrew Purtell 
> wrote:
> >>
>  Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> >>>
> >>> If we have identified blockers, why merge this before they are in?
> >>> Otherwise we can't release 2.0, and it is overdue.
> >>>
> >>>
> >>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
> >> vladrodio...@gmail.com>
> >>> wrote:
> >>>
>  Hello, HBase folks
> 
>  For your consideration today is Backup/Restore feature for Apache
> HBAse
>  2.0.
>  Backup code is available as a mega patch in HBASE-14123 (v61), applies
>  cleanly to the current master, all test PASS, patch has no other
> issues.
> 
>  The patch has gone through numerous rounds of code reviews and has
> >> probably
>  the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
> 
>  The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
> >> first
>  are complete, third one is still in progress.
> 
> 
>  *** Summary of work HBASE-14123
> 
>  The new feature introduces new command-line extensions to the hbase
> >> command
>  and, from the client side, is accessible through command-line only
>  Operations:
>  * Create full backup on a list of tables or backup set
>  * Create incremental backup image for table list or backup set
>  * Restore list of tables from a given backup image
>  * Show current backup progress
>  * Delete backup image and all related images
>  * Show history of backups
>  * Backup set operations: create backup set, add/remove table to/from
> >> backup
>  set, etc
> 
>  In the current implementation, the feature is already usable, meaning
> >> that
>  users can backup tables and restore them using provided command-line
> >> tools.
>  Both: full and incremental backups are supported.
>  This work is based on original work of IBM team (HBASE-7912). The full
> >> list
>  of JIRAs included in this mega patch can be found in three umbrella
> >> JIRAs:
>  HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3
> -
> >> all
>  resolved ones made it into the patch)
> 
>  *** What are the remaining work items
> 
>  All remaining items can be found in Phase 3 umbrella JIRA:
> HBASE-14414.
>  They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
>  Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> 
>  * BLOCKER
> 
>  * HBASE-14417 Incremental backup and bulk loading ( Patch available)
>  * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
>  * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
>  include only edits from backup tables (Patch available)
> 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

No problem I will downgrade Blockers to Majors if it scares you, Andrew 

Sent from my iPhone

> On Mar 10, 2017, at 1:52 PM, Andrew Purtell  wrote:
> 
> ​I know the merge of this feature has lagged substantially. I think that is
> regrettable but on another thread we are lamenting that 2.0 is already
> late. Unless I misunderstand, this is a proposal to merge something with
> known blockers into trunk before we branch it for 2.0 which will
> effectively prevent that release because these blockers will be there. I am
> inclined to veto. Probably we should not propose branch merges into code we
> are trying to get out the door with known blockers. Why not do that work
> first? It seems an obvious question. Perhaps I am missing something.
> 
> If we can branch for 2.0 now and then merge this, and not into the 2.0
> branch, I would vote +1 for branch merge even with known blockers pending.
> ​
> 
> On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov 
> wrote:
> 
>> They are not blockers for merge - only for 2.0. GA
>> As I said already the feature is usable right now
>> We would like to continue working on master and we would like to see a
>> commitment from community
>> 
>> Sent from my iPhone
>> 
>> On Mar 10, 2017, at 11:16 AM, Andrew Purtell  wrote:
>> 
 Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>>> 
>>> If we have identified blockers, why merge this before they are in?
>>> Otherwise we can't release 2.0, and it is overdue.
>>> 
>>> 
>>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
>> vladrodio...@gmail.com>
>>> wrote:
>>> 
 Hello, HBase folks
 
 For your consideration today is Backup/Restore feature for Apache HBAse
 2.0.
 Backup code is available as a mega patch in HBASE-14123 (v61), applies
 cleanly to the current master, all test PASS, patch has no other issues.
 
 The patch has gone through numerous rounds of code reviews and has
>> probably
 the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
 
 The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
>> first
 are complete, third one is still in progress.
 
 
 *** Summary of work HBASE-14123
 
 The new feature introduces new command-line extensions to the hbase
>> command
 and, from the client side, is accessible through command-line only
 Operations:
 * Create full backup on a list of tables or backup set
 * Create incremental backup image for table list or backup set
 * Restore list of tables from a given backup image
 * Show current backup progress
 * Delete backup image and all related images
 * Show history of backups
 * Backup set operations: create backup set, add/remove table to/from
>> backup
 set, etc
 
 In the current implementation, the feature is already usable, meaning
>> that
 users can backup tables and restore them using provided command-line
>> tools.
 Both: full and incremental backups are supported.
 This work is based on original work of IBM team (HBASE-7912). The full
>> list
 of JIRAs included in this mega patch can be found in three umbrella
>> JIRAs:
 HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 -
>> all
 resolved ones made it into the patch)
 
 *** What are the remaining work items
 
 All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
 They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
 Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
 
 * BLOCKER
 
 * HBASE-14417 Incremental backup and bulk loading ( Patch available)
 * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
 * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
 include only edits from backup tables (Patch available)
 * HBASE-17133 Backup documentation
 * HBASE-15227 Fault tolerance support
 
 * CRITICAL
 
 * HBASE-16465 Disable split/merges during backup
 
 We have umbrella JIRA (HBASE-14414) to track all the remaining work
 All the BLOCKER and CRITICAL JIRAs currently in open state will be
 implemented by 2.0 release time. Some MAJOR too, but it depends on
>> resource
 availability
 The former development branch (HBASE-7912) is obsolete and will be
 closed/deleted after the merge.
 We want backup to be a GA feature in 2.0
 We are going to support full backward compatibility for backup tool in
>> 2.0
 and onwards.
 
  Configuration
 
 Backup is disabled, by default. To enable it, the following
>> configuration
 properties must be added to hbase-site.xml:
 
 hbase.backup.enable=true
 hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
 apache.hadoop.hbase.backup.master.BackupLogCleaner
 hbase.procedure.master.classes=YOUR_CLASSES,org.
 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Andrew Purtell]

​I know the merge of this feature has lagged substantially. I think that is
regrettable but on another thread we are lamenting that 2.0 is already
late. Unless I misunderstand, this is a proposal to merge something with
known blockers into trunk before we branch it for 2.0 which will
effectively prevent that release because these blockers will be there. I am
inclined to veto. Probably we should not propose branch merges into code we
are trying to get out the door with known blockers. Why not do that work
first? It seems an obvious question. Perhaps I am missing something.

If we can branch for 2.0 now and then merge this, and not into the 2.0
branch, I would vote +1 for branch merge even with known blockers pending.
​

On Fri, Mar 10, 2017 at 1:42 PM, Vladimir Rodionov 
wrote:

> They are not blockers for merge - only for 2.0. GA
> As I said already the feature is usable right now
> We would like to continue working on master and we would like to see a
> commitment from community
>
> Sent from my iPhone
>
> On Mar 10, 2017, at 11:16 AM, Andrew Purtell  wrote:
>
> >> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> >
> > If we have identified blockers, why merge this before they are in?
> > Otherwise we can't release 2.0, and it is overdue.
> >
> >
> > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
> vladrodio...@gmail.com>
> > wrote:
> >
> >> Hello, HBase folks
> >>
> >> For your consideration today is Backup/Restore feature for Apache HBAse
> >> 2.0.
> >> Backup code is available as a mega patch in HBASE-14123 (v61), applies
> >> cleanly to the current master, all test PASS, patch has no other issues.
> >>
> >> The patch has gone through numerous rounds of code reviews and has
> probably
> >> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
> >>
> >> The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
> first
> >> are complete, third one is still in progress.
> >>
> >>
> >> *** Summary of work HBASE-14123
> >>
> >> The new feature introduces new command-line extensions to the hbase
> command
> >> and, from the client side, is accessible through command-line only
> >> Operations:
> >> * Create full backup on a list of tables or backup set
> >> * Create incremental backup image for table list or backup set
> >> * Restore list of tables from a given backup image
> >> * Show current backup progress
> >> * Delete backup image and all related images
> >> * Show history of backups
> >> * Backup set operations: create backup set, add/remove table to/from
> backup
> >> set, etc
> >>
> >> In the current implementation, the feature is already usable, meaning
> that
> >> users can backup tables and restore them using provided command-line
> tools.
> >> Both: full and incremental backups are supported.
> >> This work is based on original work of IBM team (HBASE-7912). The full
> list
> >> of JIRAs included in this mega patch can be found in three umbrella
> JIRAs:
> >> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 -
> all
> >> resolved ones made it into the patch)
> >>
> >> *** What are the remaining work items
> >>
> >> All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
> >> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> >> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> >>
> >> * BLOCKER
> >>
> >> * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> >> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> >> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
> >> include only edits from backup tables (Patch available)
> >> * HBASE-17133 Backup documentation
> >> * HBASE-15227 Fault tolerance support
> >>
> >> * CRITICAL
> >>
> >> * HBASE-16465 Disable split/merges during backup
> >>
> >> We have umbrella JIRA (HBASE-14414) to track all the remaining work
> >> All the BLOCKER and CRITICAL JIRAs currently in open state will be
> >> implemented by 2.0 release time. Some MAJOR too, but it depends on
> resource
> >> availability
> >> The former development branch (HBASE-7912) is obsolete and will be
> >> closed/deleted after the merge.
> >> We want backup to be a GA feature in 2.0
> >> We are going to support full backward compatibility for backup tool in
> 2.0
> >> and onwards.
> >>
> >>  Configuration
> >>
> >> Backup is disabled, by default. To enable it, the following
> configuration
> >> properties must be added to hbase-site.xml:
> >>
> >> hbase.backup.enable=true
> >> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> >> apache.hadoop.hbase.backup.master.BackupLogCleaner
> >> hbase.procedure.master.classes=YOUR_CLASSES,org.
> >> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> >> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> >> apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
> >> nager
> >>
> >>
> >> I would like to thank IBM 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

They are not blockers for merge - only for 2.0. GA
As I said already the feature is usable right now 
We would like to continue working on master and we would like to see a 
commitment from community

Sent from my iPhone

On Mar 10, 2017, at 11:16 AM, Andrew Purtell  wrote:

>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> 
> If we have identified blockers, why merge this before they are in?
> Otherwise we can't release 2.0, and it is overdue.
> 
> 
> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov 
> wrote:
> 
>> Hello, HBase folks
>> 
>> For your consideration today is Backup/Restore feature for Apache HBAse
>> 2.0.
>> Backup code is available as a mega patch in HBASE-14123 (v61), applies
>> cleanly to the current master, all test PASS, patch has no other issues.
>> 
>> The patch has gone through numerous rounds of code reviews and has probably
>> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
>> 
>> The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first
>> are complete, third one is still in progress.
>> 
>> 
>> *** Summary of work HBASE-14123
>> 
>> The new feature introduces new command-line extensions to the hbase command
>> and, from the client side, is accessible through command-line only
>> Operations:
>> * Create full backup on a list of tables or backup set
>> * Create incremental backup image for table list or backup set
>> * Restore list of tables from a given backup image
>> * Show current backup progress
>> * Delete backup image and all related images
>> * Show history of backups
>> * Backup set operations: create backup set, add/remove table to/from backup
>> set, etc
>> 
>> In the current implementation, the feature is already usable, meaning that
>> users can backup tables and restore them using provided command-line tools.
>> Both: full and incremental backups are supported.
>> This work is based on original work of IBM team (HBASE-7912). The full list
>> of JIRAs included in this mega patch can be found in three umbrella JIRAs:
>> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all
>> resolved ones made it into the patch)
>> 
>> *** What are the remaining work items
>> 
>> All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
>> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
>> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>> 
>> * BLOCKER
>> 
>> * HBASE-14417 Incremental backup and bulk loading ( Patch available)
>> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
>> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
>> include only edits from backup tables (Patch available)
>> * HBASE-17133 Backup documentation
>> * HBASE-15227 Fault tolerance support
>> 
>> * CRITICAL
>> 
>> * HBASE-16465 Disable split/merges during backup
>> 
>> We have umbrella JIRA (HBASE-14414) to track all the remaining work
>> All the BLOCKER and CRITICAL JIRAs currently in open state will be
>> implemented by 2.0 release time. Some MAJOR too, but it depends on resource
>> availability
>> The former development branch (HBASE-7912) is obsolete and will be
>> closed/deleted after the merge.
>> We want backup to be a GA feature in 2.0
>> We are going to support full backward compatibility for backup tool in 2.0
>> and onwards.
>> 
>>  Configuration
>> 
>> Backup is disabled, by default. To enable it, the following configuration
>> properties must be added to hbase-site.xml:
>> 
>> hbase.backup.enable=true
>> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
>> apache.hadoop.hbase.backup.master.BackupLogCleaner
>> hbase.procedure.master.classes=YOUR_CLASSES,org.
>> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
>> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
>> apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
>> nager
>> 
>> 
>> I would like to thank IBM team and Jerry He for original work,
>> 
>> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
>> 
>> Special thanks to Ted Yu for his co-development work.
>> 
>> References:
>> 
>> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains
>> design doc)
>> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
>> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
>> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
>> 
>> Please  vote +1/-1 by midnight Pacific Time (00:00
>> -0800 GMT) on March 11th  ​on whether or not we should merge this into the
>> current master.
>> 
>> -Vladimir Rodionov
>> 
> 
> 
> 
> -- 
> Best regards,
> 
>   - Andy
> 
> If you are given a choice, you believe you have acted freely. - Raymond
> Teller (via Peter Watts)

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Andrew Purtell]

> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.

If we have identified blockers, why merge this before they are in?
Otherwise we can't release 2.0, and it is overdue.


On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov 
wrote:

> Hello, HBase folks
>
> For your consideration today is Backup/Restore feature for Apache HBAse
> 2.0.
> Backup code is available as a mega patch in HBASE-14123 (v61), applies
> cleanly to the current master, all test PASS, patch has no other issues.
>
> The patch has gone through numerous rounds of code reviews and has probably
> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
>
> The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first
> are complete, third one is still in progress.
>
>
> *** Summary of work HBASE-14123
>
> The new feature introduces new command-line extensions to the hbase command
> and, from the client side, is accessible through command-line only
> Operations:
> * Create full backup on a list of tables or backup set
> * Create incremental backup image for table list or backup set
> * Restore list of tables from a given backup image
> * Show current backup progress
> * Delete backup image and all related images
> * Show history of backups
> * Backup set operations: create backup set, add/remove table to/from backup
> set, etc
>
> In the current implementation, the feature is already usable, meaning that
> users can backup tables and restore them using provided command-line tools.
> Both: full and incremental backups are supported.
> This work is based on original work of IBM team (HBASE-7912). The full list
> of JIRAs included in this mega patch can be found in three umbrella JIRAs:
> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all
> resolved ones made it into the patch)
>
> *** What are the remaining work items
>
> All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>
> * BLOCKER
>
> * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
> include only edits from backup tables (Patch available)
> * HBASE-17133 Backup documentation
> * HBASE-15227 Fault tolerance support
>
> * CRITICAL
>
> * HBASE-16465 Disable split/merges during backup
>
> We have umbrella JIRA (HBASE-14414) to track all the remaining work
> All the BLOCKER and CRITICAL JIRAs currently in open state will be
> implemented by 2.0 release time. Some MAJOR too, but it depends on resource
> availability
> The former development branch (HBASE-7912) is obsolete and will be
> closed/deleted after the merge.
> We want backup to be a GA feature in 2.0
> We are going to support full backward compatibility for backup tool in 2.0
> and onwards.
>
>  Configuration
>
> Backup is disabled, by default. To enable it, the following configuration
> properties must be added to hbase-site.xml:
>
> hbase.backup.enable=true
> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> apache.hadoop.hbase.backup.master.BackupLogCleaner
> hbase.procedure.master.classes=YOUR_CLASSES,org.
> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
> nager
>
>
> I would like to thank IBM team and Jerry He for original work,
>
> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
>
> Special thanks to Ted Yu for his co-development work.
>
> References:
>
> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains
> design doc)
> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
>
> Please  vote +1/-1 by midnight Pacific Time (00:00
> -0800 GMT) on March 11th  ​on whether or not we should merge this into the
> current master.
>
> -Vladimir Rodionov
>



-- 
Best regards,

   - Andy

If you are given a choice, you believe you have acted freely. - Raymond
Teller (via Peter Watts)

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

Still need one more +1

On Thu, Mar 9, 2017 at 11:27 PM, Vladimir Rodionov 
wrote:

> bump
>
> On Thu, Mar 9, 2017 at 10:11 AM, Ted Yu  wrote:
>
>> +1 from me as well.
>>
>> On Wed, Mar 8, 2017 at 3:48 PM, Enis Söztutar  wrote:
>>
>> > Thanks Vladimir for the write up and the work. Glad to see progress.
>> >
>> > Here is my +1. I'm pretty sure we can get the blockers in before the 2.0
>> > timeframe with the momentum, so it is a good idea to merge now so that
>> > development can continue in master, and there is more exposure for
>> testing,
>> > etc.
>> >
>> > Enis
>> >
>> > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
>> vladrodio...@gmail.com>
>> > wrote:
>> >
>> > > Hello, HBase folks
>> > >
>> > > For your consideration today is Backup/Restore feature for Apache
>> HBAse
>> > > 2.0.
>> > > Backup code is available as a mega patch in HBASE-14123 (v61), applies
>> > > cleanly to the current master, all test PASS, patch has no other
>> issues.
>> > >
>> > > The patch has gone through numerous rounds of code reviews and has
>> > probably
>> > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
>> > >
>> > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
>> > first
>> > > are complete, third one is still in progress.
>> > >
>> > >
>> > > *** Summary of work HBASE-14123
>> > >
>> > > The new feature introduces new command-line extensions to the hbase
>> > command
>> > > and, from the client side, is accessible through command-line only
>> > > Operations:
>> > > * Create full backup on a list of tables or backup set
>> > > * Create incremental backup image for table list or backup set
>> > > * Restore list of tables from a given backup image
>> > > * Show current backup progress
>> > > * Delete backup image and all related images
>> > > * Show history of backups
>> > > * Backup set operations: create backup set, add/remove table to/from
>> > backup
>> > > set, etc
>> > >
>> > > In the current implementation, the feature is already usable, meaning
>> > that
>> > > users can backup tables and restore them using provided command-line
>> > tools.
>> > > Both: full and incremental backups are supported.
>> > > This work is based on original work of IBM team (HBASE-7912). The full
>> > list
>> > > of JIRAs included in this mega patch can be found in three umbrella
>> > JIRAs:
>> > > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3
>> -
>> > all
>> > > resolved ones made it into the patch)
>> > >
>> > > *** What are the remaining work items
>> > >
>> > > All remaining items can be found in Phase 3 umbrella JIRA:
>> HBASE-14414.
>> > > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
>> > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>> > >
>> > > * BLOCKER
>> > >
>> > > * HBASE-14417 Incremental backup and bulk loading ( Patch available)
>> > > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
>> > > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
>> > > include only edits from backup tables (Patch available)
>> > > * HBASE-17133 Backup documentation
>> > > * HBASE-15227 Fault tolerance support
>> > >
>> > > * CRITICAL
>> > >
>> > > * HBASE-16465 Disable split/merges during backup
>> > >
>> > > We have umbrella JIRA (HBASE-14414) to track all the remaining work
>> > > All the BLOCKER and CRITICAL JIRAs currently in open state will be
>> > > implemented by 2.0 release time. Some MAJOR too, but it depends on
>> > resource
>> > > availability
>> > > The former development branch (HBASE-7912) is obsolete and will be
>> > > closed/deleted after the merge.
>> > > We want backup to be a GA feature in 2.0
>> > > We are going to support full backward compatibility for backup tool in
>> > 2.0
>> > > and onwards.
>> > >
>> > >  Configuration
>> > >
>> > > Backup is disabled, by default. To enable it, the following
>> configuration
>> > > properties must be added to hbase-site.xml:
>> > >
>> > > hbase.backup.enable=true
>> > > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
>> > > apache.hadoop.hbase.backup.master.BackupLogCleaner
>> > > hbase.procedure.master.classes=YOUR_CLASSES,org.
>> > > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
>> > > hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
>> > > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerP
>> rocedureMa
>> > > nager
>> > >
>> > >
>> > > I would like to thank IBM team and Jerry He for original work,
>> > >
>> > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
>> > >
>> > > Special thanks to Ted Yu for his co-development work.
>> > >
>> > > References:
>> > >
>> > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM,
>> contains
>> > > design doc)
>> > > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
>> > > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
>> > > 

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

bump

On Thu, Mar 9, 2017 at 10:11 AM, Ted Yu  wrote:

> +1 from me as well.
>
> On Wed, Mar 8, 2017 at 3:48 PM, Enis Söztutar  wrote:
>
> > Thanks Vladimir for the write up and the work. Glad to see progress.
> >
> > Here is my +1. I'm pretty sure we can get the blockers in before the 2.0
> > timeframe with the momentum, so it is a good idea to merge now so that
> > development can continue in master, and there is more exposure for
> testing,
> > etc.
> >
> > Enis
> >
> > On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov <
> vladrodio...@gmail.com>
> > wrote:
> >
> > > Hello, HBase folks
> > >
> > > For your consideration today is Backup/Restore feature for Apache HBAse
> > > 2.0.
> > > Backup code is available as a mega patch in HBASE-14123 (v61), applies
> > > cleanly to the current master, all test PASS, patch has no other
> issues.
> > >
> > > The patch has gone through numerous rounds of code reviews and has
> > probably
> > > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
> > >
> > > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
> > first
> > > are complete, third one is still in progress.
> > >
> > >
> > > *** Summary of work HBASE-14123
> > >
> > > The new feature introduces new command-line extensions to the hbase
> > command
> > > and, from the client side, is accessible through command-line only
> > > Operations:
> > > * Create full backup on a list of tables or backup set
> > > * Create incremental backup image for table list or backup set
> > > * Restore list of tables from a given backup image
> > > * Show current backup progress
> > > * Delete backup image and all related images
> > > * Show history of backups
> > > * Backup set operations: create backup set, add/remove table to/from
> > backup
> > > set, etc
> > >
> > > In the current implementation, the feature is already usable, meaning
> > that
> > > users can backup tables and restore them using provided command-line
> > tools.
> > > Both: full and incremental backups are supported.
> > > This work is based on original work of IBM team (HBASE-7912). The full
> > list
> > > of JIRAs included in this mega patch can be found in three umbrella
> > JIRAs:
> > > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 -
> > all
> > > resolved ones made it into the patch)
> > >
> > > *** What are the remaining work items
> > >
> > > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
> > > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> > > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> > >
> > > * BLOCKER
> > >
> > > * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> > > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> > > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
> > > include only edits from backup tables (Patch available)
> > > * HBASE-17133 Backup documentation
> > > * HBASE-15227 Fault tolerance support
> > >
> > > * CRITICAL
> > >
> > > * HBASE-16465 Disable split/merges during backup
> > >
> > > We have umbrella JIRA (HBASE-14414) to track all the remaining work
> > > All the BLOCKER and CRITICAL JIRAs currently in open state will be
> > > implemented by 2.0 release time. Some MAJOR too, but it depends on
> > resource
> > > availability
> > > The former development branch (HBASE-7912) is obsolete and will be
> > > closed/deleted after the merge.
> > > We want backup to be a GA feature in 2.0
> > > We are going to support full backward compatibility for backup tool in
> > 2.0
> > > and onwards.
> > >
> > >  Configuration
> > >
> > > Backup is disabled, by default. To enable it, the following
> configuration
> > > properties must be added to hbase-site.xml:
> > >
> > > hbase.backup.enable=true
> > > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> > > apache.hadoop.hbase.backup.master.BackupLogCleaner
> > > hbase.procedure.master.classes=YOUR_CLASSES,org.
> > > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> > > hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> > > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
> > > nager
> > >
> > >
> > > I would like to thank IBM team and Jerry He for original work,
> > >
> > > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
> > >
> > > Special thanks to Ted Yu for his co-development work.
> > >
> > > References:
> > >
> > > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM,
> contains
> > > design doc)
> > > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
> > > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
> > > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
> > >
> > > Please  vote +1/-1 by midnight Pacific Time (00:00
> > > -0800 GMT) on March 11th  ​on whether or not we should merge this into
> > the
> > > current master.
> > >
> > > -Vladimir Rodionov
> > >
> >

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

bump

On Wed, Mar 8, 2017 at 3:52 PM, Vladimir Rodionov 
wrote:

> No problem, we can extend deadline.
>
> On Wed, Mar 8, 2017 at 3:31 PM, Ted Yu  wrote:
>
>> March 11th is on weekend.
>>
>> Do you want to give people who haven't looked at the mega patch in depth
>> some more time ?
>>
>> Cheers
>>
>> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov > >
>> wrote:
>>
>> > Hello, HBase folks
>> >
>> > For your consideration today is Backup/Restore feature for Apache HBAse
>> > 2.0.
>> > Backup code is available as a mega patch in HBASE-14123 (v61), applies
>> > cleanly to the current master, all test PASS, patch has no other issues.
>> >
>> > The patch has gone through numerous rounds of code reviews and has
>> probably
>> > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
>> >
>> > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
>> first
>> > are complete, third one is still in progress.
>> >
>> >
>> > *** Summary of work HBASE-14123
>> >
>> > The new feature introduces new command-line extensions to the hbase
>> command
>> > and, from the client side, is accessible through command-line only
>> > Operations:
>> > * Create full backup on a list of tables or backup set
>> > * Create incremental backup image for table list or backup set
>> > * Restore list of tables from a given backup image
>> > * Show current backup progress
>> > * Delete backup image and all related images
>> > * Show history of backups
>> > * Backup set operations: create backup set, add/remove table to/from
>> backup
>> > set, etc
>> >
>> > In the current implementation, the feature is already usable, meaning
>> that
>> > users can backup tables and restore them using provided command-line
>> tools.
>> > Both: full and incremental backups are supported.
>> > This work is based on original work of IBM team (HBASE-7912). The full
>> list
>> > of JIRAs included in this mega patch can be found in three umbrella
>> JIRAs:
>> > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 -
>> all
>> > resolved ones made it into the patch)
>> >
>> > *** What are the remaining work items
>> >
>> > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
>> > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
>> > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>> >
>> > * BLOCKER
>> >
>> > * HBASE-14417 Incremental backup and bulk loading ( Patch available)
>> > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
>> > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
>> > include only edits from backup tables (Patch available)
>> > * HBASE-17133 Backup documentation
>> > * HBASE-15227 Fault tolerance support
>> >
>> > * CRITICAL
>> >
>> > * HBASE-16465 Disable split/merges during backup
>> >
>> > We have umbrella JIRA (HBASE-14414) to track all the remaining work
>> > All the BLOCKER and CRITICAL JIRAs currently in open state will be
>> > implemented by 2.0 release time. Some MAJOR too, but it depends on
>> resource
>> > availability
>> > The former development branch (HBASE-7912) is obsolete and will be
>> > closed/deleted after the merge.
>> > We want backup to be a GA feature in 2.0
>> > We are going to support full backward compatibility for backup tool in
>> 2.0
>> > and onwards.
>> >
>> >  Configuration
>> >
>> > Backup is disabled, by default. To enable it, the following
>> configuration
>> > properties must be added to hbase-site.xml:
>> >
>> > hbase.backup.enable=true
>> > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
>> > apache.hadoop.hbase.backup.master.BackupLogCleaner
>> > hbase.procedure.master.classes=YOUR_CLASSES,org.
>> > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
>> > hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
>> > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
>> > nager
>> >
>> >
>> > I would like to thank IBM team and Jerry He for original work,
>> >
>> > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
>> >
>> > Special thanks to Ted Yu for his co-development work.
>> >
>> > References:
>> >
>> > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM,
>> contains
>> > design doc)
>> > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
>> > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
>> > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
>> >
>> > Please  vote +1/-1 by midnight Pacific Time (00:00
>> > -0800 GMT) on March 11th  ​on whether or not we should merge this into
>> the
>> > current master.
>> >
>> > -Vladimir Rodionov
>> >
>>
>
>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

+1 from me as well.

On Wed, Mar 8, 2017 at 3:48 PM, Enis Söztutar  wrote:

> Thanks Vladimir for the write up and the work. Glad to see progress.
>
> Here is my +1. I'm pretty sure we can get the blockers in before the 2.0
> timeframe with the momentum, so it is a good idea to merge now so that
> development can continue in master, and there is more exposure for testing,
> etc.
>
> Enis
>
> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov 
> wrote:
>
> > Hello, HBase folks
> >
> > For your consideration today is Backup/Restore feature for Apache HBAse
> > 2.0.
> > Backup code is available as a mega patch in HBASE-14123 (v61), applies
> > cleanly to the current master, all test PASS, patch has no other issues.
> >
> > The patch has gone through numerous rounds of code reviews and has
> probably
> > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
> >
> > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
> first
> > are complete, third one is still in progress.
> >
> >
> > *** Summary of work HBASE-14123
> >
> > The new feature introduces new command-line extensions to the hbase
> command
> > and, from the client side, is accessible through command-line only
> > Operations:
> > * Create full backup on a list of tables or backup set
> > * Create incremental backup image for table list or backup set
> > * Restore list of tables from a given backup image
> > * Show current backup progress
> > * Delete backup image and all related images
> > * Show history of backups
> > * Backup set operations: create backup set, add/remove table to/from
> backup
> > set, etc
> >
> > In the current implementation, the feature is already usable, meaning
> that
> > users can backup tables and restore them using provided command-line
> tools.
> > Both: full and incremental backups are supported.
> > This work is based on original work of IBM team (HBASE-7912). The full
> list
> > of JIRAs included in this mega patch can be found in three umbrella
> JIRAs:
> > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 -
> all
> > resolved ones made it into the patch)
> >
> > *** What are the remaining work items
> >
> > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
> > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> >
> > * BLOCKER
> >
> > * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
> > include only edits from backup tables (Patch available)
> > * HBASE-17133 Backup documentation
> > * HBASE-15227 Fault tolerance support
> >
> > * CRITICAL
> >
> > * HBASE-16465 Disable split/merges during backup
> >
> > We have umbrella JIRA (HBASE-14414) to track all the remaining work
> > All the BLOCKER and CRITICAL JIRAs currently in open state will be
> > implemented by 2.0 release time. Some MAJOR too, but it depends on
> resource
> > availability
> > The former development branch (HBASE-7912) is obsolete and will be
> > closed/deleted after the merge.
> > We want backup to be a GA feature in 2.0
> > We are going to support full backward compatibility for backup tool in
> 2.0
> > and onwards.
> >
> >  Configuration
> >
> > Backup is disabled, by default. To enable it, the following configuration
> > properties must be added to hbase-site.xml:
> >
> > hbase.backup.enable=true
> > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> > apache.hadoop.hbase.backup.master.BackupLogCleaner
> > hbase.procedure.master.classes=YOUR_CLASSES,org.
> > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> > hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
> > nager
> >
> >
> > I would like to thank IBM team and Jerry He for original work,
> >
> > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
> >
> > Special thanks to Ted Yu for his co-development work.
> >
> > References:
> >
> > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains
> > design doc)
> > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
> > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
> > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
> >
> > Please  vote +1/-1 by midnight Pacific Time (00:00
> > -0800 GMT) on March 11th  ​on whether or not we should merge this into
> the
> > current master.
> >
> > -Vladimir Rodionov
> >
>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

No problem, we can extend deadline.

On Wed, Mar 8, 2017 at 3:31 PM, Ted Yu  wrote:

> March 11th is on weekend.
>
> Do you want to give people who haven't looked at the mega patch in depth
> some more time ?
>
> Cheers
>
> On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov 
> wrote:
>
> > Hello, HBase folks
> >
> > For your consideration today is Backup/Restore feature for Apache HBAse
> > 2.0.
> > Backup code is available as a mega patch in HBASE-14123 (v61), applies
> > cleanly to the current master, all test PASS, patch has no other issues.
> >
> > The patch has gone through numerous rounds of code reviews and has
> probably
> > the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
> >
> > The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two
> first
> > are complete, third one is still in progress.
> >
> >
> > *** Summary of work HBASE-14123
> >
> > The new feature introduces new command-line extensions to the hbase
> command
> > and, from the client side, is accessible through command-line only
> > Operations:
> > * Create full backup on a list of tables or backup set
> > * Create incremental backup image for table list or backup set
> > * Restore list of tables from a given backup image
> > * Show current backup progress
> > * Delete backup image and all related images
> > * Show history of backups
> > * Backup set operations: create backup set, add/remove table to/from
> backup
> > set, etc
> >
> > In the current implementation, the feature is already usable, meaning
> that
> > users can backup tables and restore them using provided command-line
> tools.
> > Both: full and incremental backups are supported.
> > This work is based on original work of IBM team (HBASE-7912). The full
> list
> > of JIRAs included in this mega patch can be found in three umbrella
> JIRAs:
> > HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 -
> all
> > resolved ones made it into the patch)
> >
> > *** What are the remaining work items
> >
> > All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
> > They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> > Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
> >
> > * BLOCKER
> >
> > * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> > * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> > * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
> > include only edits from backup tables (Patch available)
> > * HBASE-17133 Backup documentation
> > * HBASE-15227 Fault tolerance support
> >
> > * CRITICAL
> >
> > * HBASE-16465 Disable split/merges during backup
> >
> > We have umbrella JIRA (HBASE-14414) to track all the remaining work
> > All the BLOCKER and CRITICAL JIRAs currently in open state will be
> > implemented by 2.0 release time. Some MAJOR too, but it depends on
> resource
> > availability
> > The former development branch (HBASE-7912) is obsolete and will be
> > closed/deleted after the merge.
> > We want backup to be a GA feature in 2.0
> > We are going to support full backward compatibility for backup tool in
> 2.0
> > and onwards.
> >
> >  Configuration
> >
> > Backup is disabled, by default. To enable it, the following configuration
> > properties must be added to hbase-site.xml:
> >
> > hbase.backup.enable=true
> > hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> > apache.hadoop.hbase.backup.master.BackupLogCleaner
> > hbase.procedure.master.classes=YOUR_CLASSES,org.
> > apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> > hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> > apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
> > nager
> >
> >
> > I would like to thank IBM team and Jerry He for original work,
> >
> > Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
> >
> > Special thanks to Ted Yu for his co-development work.
> >
> > References:
> >
> > https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains
> > design doc)
> > https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
> > https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
> > https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
> >
> > Please  vote +1/-1 by midnight Pacific Time (00:00
> > -0800 GMT) on March 11th  ​on whether or not we should merge this into
> the
> > current master.
> >
> > -Vladimir Rodionov
> >
>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Enis Söztutar]

Thanks Vladimir for the write up and the work. Glad to see progress.

Here is my +1. I'm pretty sure we can get the blockers in before the 2.0
timeframe with the momentum, so it is a good idea to merge now so that
development can continue in master, and there is more exposure for testing,
etc.

Enis

On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov 
wrote:

> Hello, HBase folks
>
> For your consideration today is Backup/Restore feature for Apache HBAse
> 2.0.
> Backup code is available as a mega patch in HBASE-14123 (v61), applies
> cleanly to the current master, all test PASS, patch has no other issues.
>
> The patch has gone through numerous rounds of code reviews and has probably
> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
>
> The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first
> are complete, third one is still in progress.
>
>
> *** Summary of work HBASE-14123
>
> The new feature introduces new command-line extensions to the hbase command
> and, from the client side, is accessible through command-line only
> Operations:
> * Create full backup on a list of tables or backup set
> * Create incremental backup image for table list or backup set
> * Restore list of tables from a given backup image
> * Show current backup progress
> * Delete backup image and all related images
> * Show history of backups
> * Backup set operations: create backup set, add/remove table to/from backup
> set, etc
>
> In the current implementation, the feature is already usable, meaning that
> users can backup tables and restore them using provided command-line tools.
> Both: full and incremental backups are supported.
> This work is based on original work of IBM team (HBASE-7912). The full list
> of JIRAs included in this mega patch can be found in three umbrella JIRAs:
> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all
> resolved ones made it into the patch)
>
> *** What are the remaining work items
>
> All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>
> * BLOCKER
>
> * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
> include only edits from backup tables (Patch available)
> * HBASE-17133 Backup documentation
> * HBASE-15227 Fault tolerance support
>
> * CRITICAL
>
> * HBASE-16465 Disable split/merges during backup
>
> We have umbrella JIRA (HBASE-14414) to track all the remaining work
> All the BLOCKER and CRITICAL JIRAs currently in open state will be
> implemented by 2.0 release time. Some MAJOR too, but it depends on resource
> availability
> The former development branch (HBASE-7912) is obsolete and will be
> closed/deleted after the merge.
> We want backup to be a GA feature in 2.0
> We are going to support full backward compatibility for backup tool in 2.0
> and onwards.
>
>  Configuration
>
> Backup is disabled, by default. To enable it, the following configuration
> properties must be added to hbase-site.xml:
>
> hbase.backup.enable=true
> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> apache.hadoop.hbase.backup.master.BackupLogCleaner
> hbase.procedure.master.classes=YOUR_CLASSES,org.
> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
> nager
>
>
> I would like to thank IBM team and Jerry He for original work,
>
> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
>
> Special thanks to Ted Yu for his co-development work.
>
> References:
>
> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains
> design doc)
> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
>
> Please  vote +1/-1 by midnight Pacific Time (00:00
> -0800 GMT) on March 11th  ​on whether or not we should merge this into the
> current master.
>
> -Vladimir Rodionov
>

Re: [VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Ted Yu]

March 11th is on weekend.

Do you want to give people who haven't looked at the mega patch in depth
some more time ?

Cheers

On Wed, Mar 8, 2017 at 1:32 PM, Vladimir Rodionov 
wrote:

> Hello, HBase folks
>
> For your consideration today is Backup/Restore feature for Apache HBAse
> 2.0.
> Backup code is available as a mega patch in HBASE-14123 (v61), applies
> cleanly to the current master, all test PASS, patch has no other issues.
>
> The patch has gone through numerous rounds of code reviews and has probably
> the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)
>
> The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first
> are complete, third one is still in progress.
>
>
> *** Summary of work HBASE-14123
>
> The new feature introduces new command-line extensions to the hbase command
> and, from the client side, is accessible through command-line only
> Operations:
> * Create full backup on a list of tables or backup set
> * Create incremental backup image for table list or backup set
> * Restore list of tables from a given backup image
> * Show current backup progress
> * Delete backup image and all related images
> * Show history of backups
> * Backup set operations: create backup set, add/remove table to/from backup
> set, etc
>
> In the current implementation, the feature is already usable, meaning that
> users can backup tables and restore them using provided command-line tools.
> Both: full and incremental backups are supported.
> This work is based on original work of IBM team (HBASE-7912). The full list
> of JIRAs included in this mega patch can be found in three umbrella JIRAs:
> HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all
> resolved ones made it into the patch)
>
> *** What are the remaining work items
>
> All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
> They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
> Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.
>
> * BLOCKER
>
> * HBASE-14417 Incremental backup and bulk loading ( Patch available)
> * HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
> * HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
> include only edits from backup tables (Patch available)
> * HBASE-17133 Backup documentation
> * HBASE-15227 Fault tolerance support
>
> * CRITICAL
>
> * HBASE-16465 Disable split/merges during backup
>
> We have umbrella JIRA (HBASE-14414) to track all the remaining work
> All the BLOCKER and CRITICAL JIRAs currently in open state will be
> implemented by 2.0 release time. Some MAJOR too, but it depends on resource
> availability
> The former development branch (HBASE-7912) is obsolete and will be
> closed/deleted after the merge.
> We want backup to be a GA feature in 2.0
> We are going to support full backward compatibility for backup tool in 2.0
> and onwards.
>
>  Configuration
>
> Backup is disabled, by default. To enable it, the following configuration
> properties must be added to hbase-site.xml:
>
> hbase.backup.enable=true
> hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.
> apache.hadoop.hbase.backup.master.BackupLogCleaner
> hbase.procedure.master.classes=YOUR_CLASSES,org.
> apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
> hbase.procedure.regionserver.classes=YOUR_CLASSES,org.
> apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureMa
> nager
>
>
> I would like to thank IBM team and Jerry He for original work,
>
> Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews
>
> Special thanks to Ted Yu for his co-development work.
>
> References:
>
> https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains
> design doc)
> https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
> https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
> https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)
>
> Please  vote +1/-1 by midnight Pacific Time (00:00
> -0800 GMT) on March 11th  ​on whether or not we should merge this into the
> current master.
>
> -Vladimir Rodionov
>

[VOTE] Backup/Restore feature for HBase 2.0, vote closing 3/11/2017

[Vladimir Rodionov]

Hello, HBase folks

For your consideration today is Backup/Restore feature for Apache HBAse 2.0.
Backup code is available as a mega patch in HBASE-14123 (v61), applies
cleanly to the current master, all test PASS, patch has no other issues.

The patch has gone through numerous rounds of code reviews and has probably
the most lengthy discussion thread on Apache JIRA (HBASE-14123) :)

The work has been split into 3 phases (HBASE-14030, 14123, 14414) Two first
are complete, third one is still in progress.


*** Summary of work HBASE-14123

The new feature introduces new command-line extensions to the hbase command
and, from the client side, is accessible through command-line only
Operations:
* Create full backup on a list of tables or backup set
* Create incremental backup image for table list or backup set
* Restore list of tables from a given backup image
* Show current backup progress
* Delete backup image and all related images
* Show history of backups
* Backup set operations: create backup set, add/remove table to/from backup
set, etc

In the current implementation, the feature is already usable, meaning that
users can backup tables and restore them using provided command-line tools.
Both: full and incremental backups are supported.
This work is based on original work of IBM team (HBASE-7912). The full list
of JIRAs included in this mega patch can be found in three umbrella JIRAs:
HBASE-14030 (Phase 1), HBASE-14123 (Phase 2) and HBASE-14414 (Phase 3 - all
resolved ones made it into the patch)

*** What are the remaining work items

All remaining items can be found in Phase 3 umbrella JIRA: HBASE-14414.
They are split into 3 groups: BLOCKER, CRITICAL, MAJOR
Only BLOCKERs and CRITICALs are guaranteed for HBase 2.0 release.

* BLOCKER

* HBASE-14417 Incremental backup and bulk loading ( Patch available)
* HBASE-14135 HBase Backup/Restore Phase 3: Merge backup images
* HBASE-14141 HBase Backup/Restore Phase 3: Filter WALs on backup to
include only edits from backup tables (Patch available)
* HBASE-17133 Backup documentation
* HBASE-15227 Fault tolerance support

* CRITICAL

* HBASE-16465 Disable split/merges during backup

We have umbrella JIRA (HBASE-14414) to track all the remaining work
All the BLOCKER and CRITICAL JIRAs currently in open state will be
implemented by 2.0 release time. Some MAJOR too, but it depends on resource
availability
The former development branch (HBASE-7912) is obsolete and will be
closed/deleted after the merge.
We want backup to be a GA feature in 2.0
We are going to support full backward compatibility for backup tool in 2.0
and onwards.

 Configuration

Backup is disabled, by default. To enable it, the following configuration
properties must be added to hbase-site.xml:

hbase.backup.enable=true
hbase.master.logcleaner.plugins=YOUR_PLUGINS,org.apache.hadoop.hbase.backup.master.BackupLogCleaner
hbase.procedure.master.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.master.LogRollMasterProcedureManager
hbase.procedure.regionserver.classes=YOUR_CLASSES,org.apache.hadoop.hbase.backup.regionserver.LogRollRegionServerProcedureManager


I would like to thank IBM team and Jerry He for original work,

Enis, Ted, Stack, Matteo, Jerry for time spent on code reviews

Special thanks to Ted Yu for his co-development work.

References:

https://issues.apache.org/jira/browse/HBASE-7912 (original IBM, contains
design doc)
https://issues.apache.org/jira/browse/HBASE-14030 (Phase 1)
https://issues.apache.org/jira/browse/HBASE-14123 (Phase 2)
https://issues.apache.org/jira/browse/HBASE-14414 (Phase 3)

Please  vote +1/-1 by midnight Pacific Time (00:00
-0800 GMT) on March 11th  ​on whether or not we should merge this into the
current master.

-Vladimir Rodionov