For what it's worth, I think AMv2 is the main reason to have a 2.0 in the first place, so I would both agree it needs a lot more testing and yet I would want us to have a 2.0 release as the vehicle for getting that to happen. For other features without testing from a number of parties or at scale the value proposition is less clear and it's fine by me for the RM to set them aside for future releases.
Also, I can relay that there is some interest where I work in utilizing HBASE-7912 but the impression we have is it is unfinished and untested. So for now we are ignoring it and continuing with home grown solutions. Part of the problem is fault tolerance was left to the last phase(s) and yet it is an essential property for adoption for serious work. The best way to resolve this IMHO is for the developers of this feature to complete those unfinished JIRAs, especially concerning resilience to failures. > On Sep 9, 2017, at 4:11 PM, Vladimir Rodionov <vladrodio...@gmail.com> wrote: > > Hmm, the next on your list (of kicked out from branch v2) should be AM v2 I > presume? > > -Vlad > >> On Sat, Sep 9, 2017 at 4:04 PM, stack <saint....@gmail.com> wrote: >> >> In spite of repeated requests for eng summary of state of this feature -- >> summary of what is in 2.0, what is not, what the capabilities are, how well >> it has been tested and at what scale -- all I get, when the requests are >> not ignored, are pointers to lists of ill-describing jiras and some pending >> user facing doc update. >> >> For other features, mob or region server groups, I know that they have been >> running at scale in production for as much as a year and more. I have some >> confidence these items basically work. For backup/restore I have no such >> sense even after spending time in review and trying to use the feature. >> >> As release manager, I have say over what makes it into a release. Unless >> the work is done to convince me that backup/restore is more than a lump of >> code and a few unit tests that can pass on some fellows laptop, I am going >> to kick it out of branch-2. Let the feature harden more in master branch >> before it ships in a release. >> >> S >> >> On Sep 8, 2017 10:59 PM, "Vladimir Rodionov" <vladrodio...@gmail.com> >> wrote: >> >>>>> Have I grasped the state of things correctly, Vlad? >>> >>> Josh, the only thing which is still pending is doc update. All other >>> features are good to have but not a blockers for 2.0 release. >>> >>> -Vlad >>> >>> On Fri, Sep 8, 2017 at 10:42 PM, Vladimir Rodionov < >> vladrodio...@gmail.com >>>> >>> wrote: >>> >>>>>> What testing and at what >>>>>> scale has testing been done? >>>> >>>> Do we have have that for other features? >>>> >>>> >>>> On Fri, Sep 8, 2017 at 10:41 PM, Vladimir Rodionov < >>> vladrodio...@gmail.com >>>>> wrote: >>>> >>>>>>> It asks: "How do I figure what of backup/restore feature is going >> to >>>>> be in >>>>>>> hbase-2.0.0? >>>>> >>>>> Hmm, wait for doc update. >>>>> >>>>> >>>>>> On Fri, Sep 8, 2017 at 2:39 PM, Stack <st...@duboce.net> wrote: >>>>>> >>>>>> HBASE-14414 is a JIRA with a list of random seeming issues w/ >>>>>> non-descript >>>>>> summaries: "Add nonce support to TableBackupProcedure, BackupID must >>>>>> include backup set name, ...". The last comment in that issue is from >>>>>> July. >>>>>> It asks: "How do I figure what of backup/restore feature is going to >> be >>>>>> in >>>>>> hbase-2.0.0? Thanks Vladimir Rodionov >>>>>> <https://issues.apache.org/jira/secure/ViewProfile.jspa? >> name=vrodionov >>>>>>> ." >>>>>> to which there is no answer. Doc update is TODO. >>>>>> >>>>>> Where is the summary of the capability in hbase-2? What testing and >> at >>>>>> what >>>>>> scale has testing been done? Is this 'stable or experimental'? If I >>> can't >>>>>> get basic info on this feature though I ask repeatedly, what hope >> does >>>>>> the >>>>>> poor old operator have? >>>>>> >>>>>> St.Ack >>>>>> >>>>>> >>>>>> On Fri, Sep 8, 2017 at 1:59 PM, Vladimir Rodionov < >>>>>> vladrodio...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> HBASE-14414 >>>>>>> >>>>>>>> On Fri, Sep 8, 2017 at 1:14 PM, Stack <st...@duboce.net> wrote: >>>>>>>> >>>>>>>> Where do I go to get the current status of this feature? Looking >> in >>>>>> JIRA >>>>>>> I >>>>>>>> see loads of issues open against backup including some against >>>>>>> hbase-2.0.0 >>>>>>>> and no progress being made that I can discern. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> S >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> On Wed, Nov 23, 2016 at 8:52 AM, Stack <st...@duboce.net> wrote: >>>>>>>>> >>>>>>>>> On Tue, Nov 22, 2016 at 6:48 PM, Stack <st...@duboce.net> >> wrote: >>>>>>>>> >>>>>>>>>> On Tue, Nov 22, 2016 at 3:17 PM, Vladimir Rodionov < >>>>>>>>>> vladrodio...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>>>> and/or he answered most of the review feedback >>>>>>>>>>> >>>>>>>>>>> No, questions are still open, but I do not see any blockers >> and >>>>>> we >>>>>>> have >>>>>>>>>>> HBASE-16940 to address these questions. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> Agree. No blockers but stuff that should be dealt with (No one >>>>>> will >>>>>>> pay >>>>>>>>>> me any attention once merge goes in -- smile). >>>>>>>>>> >>>>>>>>>> >>>>>>>>> Let me clarify the above. I want review addressed before merge >>>>>> happens. >>>>>>>>> Sorry if any confusion. >>>>>>>>> St.Ack >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>> St.Ack >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>> On Tue, Nov 22, 2016 at 3:04 PM, Devaraj Das < >>>>>> d...@hortonworks.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi Stack, hats off to you for spending so much time on >> this! >>>>>>> Thanks! >>>>>>>>>>> From >>>>>>>>>>>> my understanding, Vlad has raised follow-up jiras for the >>>>>> issues >>>>>>> you >>>>>>>>>>>> raised, and/or he answered most of the review feedback. So, >>> do >>>>>> you >>>>>>>>>>> think we >>>>>>>>>>>> could do a merge vote now? >>>>>>>>>>>> Devaraj. >>>>>>>>>>>> ________________________________________ >>>>>>>>>>>> From: Vladimir Rodionov <vladrodio...@gmail.com> >>>>>>>>>>>> Sent: Monday, November 21, 2016 8:34 PM >>>>>>>>>>>> To: dev@hbase.apache.org >>>>>>>>>>>> Subject: Re: [DISCUSSION] Merge Backup / Restore - Branch >>>>>>> HBASE-7912 >>>>>>>>>>>> >>>>>>>>>>>>>> I have spent a good bit of time reviewing and testing >> this >>>>>>>> feature. >>>>>>>>>>> I >>>>>>>>>>>> would >>>>>>>>>>>>>> like my review and concerns addressed and I'd like it to >>> be >>>>>>> clear >>>>>>>>>>> how; >>>>>>>>>>>>>> either explicit follow-on issues, pointers to where in >> the >>>>>> patch >>>>>>>> or >>>>>>>>>>> doc >>>>>>>>>>>> my >>>>>>>>>>>>>> remarks have been catered to, etc. Until then, I am >>> against >>>>>>>> commit. >>>>>>>>>>>> >>>>>>>>>>>> Stack, mega patch review comments will be addressed in the >>>>>>> dedicated >>>>>>>>>>> JIRA: >>>>>>>>>>>> HBASE-16940 >>>>>>>>>>>> I have open several other JIRAs to address your other >>> comments >>>>>> (not >>>>>>>> on >>>>>>>>>>>> review board). >>>>>>>>>>>> >>>>>>>>>>>> Details are here (end of the thread): >>>>>>>>>>>> https://issues.apache.org/jira/browse/HBASE-14123 >>>>>>>>>>>> >>>>>>>>>>>> Let me know what else should we do to move merge forward. >>>>>>>>>>>> >>>>>>>>>>>> -Vlad >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Nov 18, 2016 at 4:54 PM, Stack <st...@duboce.net> >>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:53 PM, Ted Yu < >>> yuzhih...@gmail.com >>>>>>> >>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, Matteo. >>>>>>>>>>>>>> >>>>>>>>>>>>>> bq. restore is not clear if given an incremental id it >>>>>> will do >>>>>>>> the >>>>>>>>>>> full >>>>>>>>>>>>>> restore from full up to that point or if i need to >> apply >>>>>>> manually >>>>>>>>>>>>>> everything >>>>>>>>>>>>>> >>>>>>>>>>>>>> The restore takes into consideration of the dependent >>>>>>> backup(s). >>>>>>>>>>>>>> So there is no need to apply preceding backup(s) >>> manually. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> I ask this question on the issue. It is not clear from >> the >>>>>> usage >>>>>>> or >>>>>>>>>>> doc >>>>>>>>>>>> how >>>>>>>>>>>>> to run a restore from incremental. Can you fix in doc and >>>>>> usage >>>>>>> how >>>>>>>>>>> so I >>>>>>>>>>>>> can be clear and try it. Currently I am stuck verifying a >>>>>> round >>>>>>>> trip >>>>>>>>>>>> backup >>>>>>>>>>>>> restore made of incrementals. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks, >>>>>>>>>>>>> S >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Nov 18, 2016 at 3:48 PM, Matteo Bertozzi < >>>>>>>>>>>>> theo.berto...@gmail.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> I did one last pass to the mega patch. I don't see >>>>>> anything >>>>>>>> major >>>>>>>>>>>> that >>>>>>>>>>>>>>> should block the merge. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> - most of the code is isolated in the backup package >>>>>>>>>>>>>>> - all the backup code is client side >>>>>>>>>>>>>>> - there are few changes to the server side, mainly >> for >>>>>>>> cleaners, >>>>>>>>>>> wal >>>>>>>>>>>>>>> rolling and similar (which is ok) >>>>>>>>>>>>>>> - there is a good number of tests, and an integration >>>>>> test >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> the code seems to have still some left overs from the >>> old >>>>>>>>>>>>> implementation, >>>>>>>>>>>>>>> and some stuff needs a cleanup. but I don't think >> this >>>>>> should >>>>>>>> be >>>>>>>>>>> used >>>>>>>>>>>>> as >>>>>>>>>>>>>> an >>>>>>>>>>>>>>> argument to block the merge. I think the guys will >> keep >>>>>>> working >>>>>>>>>>> on >>>>>>>>>>>> this >>>>>>>>>>>>>> and >>>>>>>>>>>>>>> they may also get help of others once the patch is in >>>>>> master. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I still have my concerns about the current >> limitations, >>>>>> but >>>>>>>>>>> these are >>>>>>>>>>>>>>> things already planned for phase 3, so some of this >>>>>> stuff may >>>>>>>>>>> even be >>>>>>>>>>>>> in >>>>>>>>>>>>>>> the final 2.0. >>>>>>>>>>>>>>> but as long as we have a "current limitations" >> section >>>>>> in the >>>>>>>>>>> user >>>>>>>>>>>>> guide >>>>>>>>>>>>>>> mentioning important stuff like the ones below, I'm >> ok >>>>>> with >>>>>>> it. >>>>>>>>>>>>>>> - if you write to the table with >> Durability.SKIP_WALS >>>>>> your >>>>>>>> data >>>>>>>>>>> will >>>>>>>>>>>>> not >>>>>>>>>>>>>>> be in the incremental-backup >>>>>>>>>>>>>>> - if you bulkload files that data will not be in the >>>>>>>> incremental >>>>>>>>>>>>> backup >>>>>>>>>>>>>>> (HBASE-14417) >>>>>>>>>>>>>>> - the incremental backup will not only contains the >>>>>> data of >>>>>>>> the >>>>>>>>>>>> table >>>>>>>>>>>>>> you >>>>>>>>>>>>>>> specified but also the regions from other tables that >>>>>> are on >>>>>>>> the >>>>>>>>>>> same >>>>>>>>>>>>> set >>>>>>>>>>>>>>> of RSs (HBASE-14141) ...maybe a note about security >>>>>> around >>>>>>> this >>>>>>>>>>> topic >>>>>>>>>>>>>>> - the incremental backup will not contains just the >>>>>> "latest >>>>>>>> row" >>>>>>>>>>>>> between >>>>>>>>>>>>>>> backup A and B, but it will also contains all the >>> updates >>>>>>>>>>> occurred in >>>>>>>>>>>>>>> between. but the restore does not allow you to >> restore >>>>>> up to >>>>>>> a >>>>>>>>>>>> certain >>>>>>>>>>>>>>> point in time, the restore will always be up to the >>>>>> "latest >>>>>>>>>>> backup >>>>>>>>>>>>>> point". >>>>>>>>>>>>>>> - you should limit the number of "incremental" up >> to N >>>>>> (or >>>>>>>> maybe >>>>>>>>>>>>> SIZE), >>>>>>>>>>>>>> to >>>>>>>>>>>>>>> avoid replay time becoming the bottleneck. >>> (HBASE-14135) >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I'll be ok even with the above not being in the final >>>>>> 2.0, >>>>>>>>>>>>>>> but i'd like to see as blocker for the final 2.0 (not >>> the >>>>>>>> merge) >>>>>>>>>>>>>>> - the backup code moved in an hbase-backup module >>>>>>>>>>>>>>> - and some more work around tools, especially to try >>> to >>>>>>> unify >>>>>>>>>>> and >>>>>>>>>>>> make >>>>>>>>>>>>>>> simple the backup experience (simple example: in some >>>>>> case >>>>>>>> there >>>>>>>>>>> is a >>>>>>>>>>>>>>> backup_id argument in others a backupId argument. or >>>>>> things >>>>>>>>>>> like.. >>>>>>>>>>>>>> restore >>>>>>>>>>>>>>> is not clear if given an incremental id it will do >> the >>>>>> full >>>>>>>>>>> restore >>>>>>>>>>>>> from >>>>>>>>>>>>>>> full up to that point or if i need to apply manually >>>>>>>> everything). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> in conclusion, I think we can open a merge vote. I'll >>> be >>>>>> +1 >>>>>>> on >>>>>>>>>>> it, >>>>>>>>>>>> and >>>>>>>>>>>>> I >>>>>>>>>>>>>>> think we should try to reject -1 with just a "code >>>>>> cleanup" >>>>>>>>>>>> motivation, >>>>>>>>>>>>>>> since there will still be work going on on the code >>>>>> after the >>>>>>>>>>> merge. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Matteo >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On Sun, Nov 6, 2016 at 10:54 PM, Devaraj Das < >>>>>>>>>>> d...@hortonworks.com> >>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Stack and others, anything else on the patch? Merge >>> to >>>>>>> master >>>>>>>>>>> now? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >>
