Or you can apply 14123-master.v29.full.txt (HBASE-14123) to current master.
-Vlad On Tue, Oct 11, 2016 at 1:50 PM, Vladimir Rodionov <vladrodio...@gmail.com> wrote: > Michael, > > Its in HBASE-7912 > > This is tip of git log: > > commit a072f6f49a26a7259ff2aaef6cb56d85eb592482 > Author: Frank Welsch <fwel...@jps.net> > Date: Fri Sep 23 18:00:42 2016 -0400 > > HBASE-16574 Book updates for backup and restore > > commit b14e2ab1c24e65ff88dd4c579acf83cb4ed0605e > Author: tedyu <yuzhih...@gmail.com> > Date: Wed Oct 5 16:29:40 2016 -0700 > > HBASE-16727 Backup refactoring: remove MR dependencies from HMaster > (Vladimir Rodionov) > > -Vlad > > On Tue, Oct 11, 2016 at 1:44 PM, Stack <st...@duboce.net> wrote: > >> Which branch do I check out to try it? HBASE-7912 is not it. I don't see >> an >> HBASE-16727... >> Thanks, >> M >> >> >> On Tue, Oct 11, 2016 at 11:41 AM, Vladimir Rodionov < >> vladrodio...@gmail.com> >> wrote: >> >> > The last patch is on review board: >> > https://reviews.apache.org/r/52748 >> > >> > On Mon, Oct 10, 2016 at 3:52 PM, Vladimir Rodionov < >> vladrodio...@gmail.com >> > > >> > wrote: >> > >> > > >> How hard to put in an hbase-backup module? hbase-server is fat >> enough >> > > >> already. Could be done as a follow-up. >> > > >> > > https://issues.apache.org/jira/browse/HBASE-16727? >> > > focusedCommentId=15531237&page=com.atlassian.jira. >> > > plugin.system.issuetabpanels:comment-tabpanel#comment-15531237 >> > > >> > > Can we do merge first? Then we can discuss separate module. >> > > >> > > -Vlad >> > > >> > > On Mon, Oct 10, 2016 at 3:44 PM, Ted Yu <yuzhih...@gmail.com> wrote: >> > > >> > >> Looks like the first quote was cut off. >> > >> The original sentence was: >> > >> >> > >> bq. no mapreduce job launched from master or region server. >> > >> >> > >> mapreduce job is launched from the node where command line tool is >> run. >> > >> >> > >> On Mon, Oct 10, 2016 at 3:38 PM, Stack <st...@duboce.net> wrote: >> > >> >> > >> > bq. launched from master or region server. >> > >> > >> > >> > What does this mean please? Has to be run from Master or >> RegionServer? >> > >> Can >> > >> > it be run from another node altogether? >> > >> > >> > >> > On Mon, Oct 10, 2016 at 1:44 PM, Vladimir Rodionov < >> > >> vladrodio...@gmail.com >> > >> > > >> > >> > wrote: >> > >> > >> > >> > > >> mapreduce dependency has been moved to client side - no >> mapreduce >> > >> job >> > >> > > >> > >> > > 1. We have no code in the client module anymore, due to >> dependency >> > on >> > >> > > internal server API (HFile and WAL access). >> > >> > > 2. Backup/ restore are client - driven operations, but all the >> code >> > >> > resides >> > >> > > in the server module >> > >> > > >> > >> > >> > >> > How hard to put in an hbase-backup module? hbase-server is fat >> enough >> > >> > already. Could be done as a follow-up. >> > >> > >> > >> > Thanks, >> > >> > St.Ack >> > >> > >> > >> > >> > >> > >> > >> > > 3. No MR in Master, no procedure - driven execution. >> > >> > > 4. Old good MR from command-line. >> > >> > > 5. Security was simplified and now only super-user is allowed to >> run >> > >> > > backup/restores. >> > >> > > 6. HBase Backup API was gone due to 1. Now only command-line >> access >> > to >> > >> > > backup tools. >> > >> > > >> > >> > > These consequences of refactoring has been discussed in >> HBASE-16727. >> > >> > > >> > >> > > -Vlad >> > >> > > >> > >> > > >> > >> > > >> > >> > > >> > >> > > On Mon, Oct 10, 2016 at 1:31 PM, Ted Yu <yuzhih...@gmail.com> >> > wrote: >> > >> > > >> > >> > > > Reviving this thread. >> > >> > > > >> > >> > > > The following has taken place: >> > >> > > > >> > >> > > > mapreduce dependency has been moved to client side - no >> mapreduce >> > >> job >> > >> > > > launched from master or region server. >> > >> > > > document patch (HBASE-16574) has been integrated. >> > >> > > > Updated mega patch has been attached to HBASE-14123: this >> covers >> > the >> > >> > > > refactor in #1 above and the protobuf 3 merge. >> > >> > > > >> > >> > > > If community has more feedback on the merge proposal, I would >> love >> > >> to >> > >> > > hear >> > >> > > > it. >> > >> > > > >> > >> > > > Thanks >> > >> > > > >> > >> > > > On Thu, Sep 22, 2016 at 10:31 AM, Sean Busbey < >> > bus...@cloudera.com> >> > >> > > wrote: >> > >> > > > >> > >> > > > > I'd like to see the docs proposed on HBASE-16574 integrated >> into >> > >> our >> > >> > > > > project's documentation prior to merge. >> > >> > > > > >> > >> > > > > On Thu, Sep 22, 2016 at 9:02 AM, Ted Yu <yuzhih...@gmail.com >> > >> > >> wrote: >> > >> > > > > > This feature can be marked experimental due to some >> > limitations >> > >> > such >> > >> > > as >> > >> > > > > > security. >> > >> > > > > > >> > >> > > > > > Your previous round of comments have been addressed. >> > >> > > > > > Command line tool has gone through: >> > >> > > > > > >> > >> > > > > > HBASE-16620 Fix backup command-line tool usability issues >> > >> > > > > > HBASE-16655 hbase backup describe with incorrect backup id >> > >> results >> > >> > in >> > >> > > > NPE >> > >> > > > > > >> > >> > > > > > The updated doc has been attached to HBASE-16574. >> > >> > > > > > >> > >> > > > > > Cheers >> > >> > > > > > >> > >> > > > > > On Thu, Sep 22, 2016 at 8:53 AM, Stack <st...@duboce.net> >> > >> wrote: >> > >> > > > > > >> > >> > > > > >> On Wed, Sep 21, 2016 at 7:43 AM, Ted Yu < >> yuzhih...@gmail.com >> > > >> > >> > > wrote: >> > >> > > > > >> >> > >> > > > > >> > Are there more (review) comments ? >> > >> > > > > >> > >> > >> > > > > >> > >> > >> > > > > >> Are outstanding comments addressed? >> > >> > > > > >> >> > >> > > > > >> I don't see answer to my 'is this experimental/will it be >> > >> marked >> > >> > > > > >> experimental' question. >> > >> > > > > >> >> > >> > > > > >> I ran into some issues trying to use the feature and >> > suggested >> > >> > that >> > >> > > a >> > >> > > > > >> feature likes this needs polish else it'll just rot, >> unused. >> > >> Has >> > >> > > > polish >> > >> > > > > >> been applied? All ready for another 'user' test? Suggest >> that >> > >> you >> > >> > > > update >> > >> > > > > >> here going forward for the benefit of those trying to >> follow >> > >> along >> > >> > > and >> > >> > > > > who >> > >> > > > > >> are not watching JIRA change fly-by. >> > >> > > > > >> >> > >> > > > > >> It looks like doc got a revision -- I have to check -- to >> > take >> > >> on >> > >> > > > > >> suggestion made above but again, suggest, that this thread >> > gets >> > >> > > > updated. >> > >> > > > > >> >> > >> > > > > >> Thanks, >> > >> > > > > >> St.Ack >> > >> > > > > >> >> > >> > > > > >> >> > >> > > > > >> >> > >> > > > > >> > Thanks >> > >> > > > > >> > >> > >> > > > > >> > On Tue, Sep 20, 2016 at 10:02 AM, Devaraj Das < >> > >> > > d...@hortonworks.com >> > >> > > > > >> > >> > > > > >> > wrote: >> > >> > > > > >> > >> > >> > > > > >> > > Just reviving this thread. Thanks Sean, Stack, Dima, >> and >> > >> > others >> > >> > > > for >> > >> > > > > the >> > >> > > > > >> > > thorough reviews and testing. Thanks Ted and Vlad for >> > >> taking >> > >> > > care >> > >> > > > of >> > >> > > > > >> the >> > >> > > > > >> > > feedback. Are we all good to do the merge now? Rather >> do >> > >> > sooner >> > >> > > > than >> > >> > > > > >> > later. >> > >> > > > > >> > > ________________________________________ >> > >> > > > > >> > > From: saint....@gmail.com <saint....@gmail.com> on >> > behalf >> > >> of >> > >> > > > Stack >> > >> > > > > < >> > >> > > > > >> > > st...@duboce.net> >> > >> > > > > >> > > Sent: Monday, September 12, 2016 1:18 PM >> > >> > > > > >> > > To: HBase Dev List >> > >> > > > > >> > > Subject: Re: [DISCUSSION] Merge Backup / Restore - >> Branch >> > >> > > > HBASE-7912 >> > >> > > > > >> > > >> > >> > > > > >> > > On Mon, Sep 12, 2016 at 12:19 PM, Ted Yu < >> > >> yuzhih...@gmail.com >> > >> > > >> > >> > > > > wrote: >> > >> > > > > >> > > >> > >> > > > > >> > > > Mega patch (rev 18) is on HBASE-14123. >> > >> > > > > >> > > > >> > >> > > > > >> > > > Please comment on HBASE-14123 on how you want to >> > review. >> > >> > > > > >> > > > >> > >> > > > > >> > > >> > >> > > > > >> > > >> > >> > > > > >> > > Yeah. That was my lost tab. Last rb was 6 months ago. >> > >> Suggest >> > >> > > > > updating >> > >> > > > > >> > it. >> > >> > > > > >> > > RB is pretty good for review. Patch is only 1.5M so >> > should >> > >> be >> > >> > > > fine. >> > >> > > > > >> > > >> > >> > > > > >> > > St.Ack >> > >> > > > > >> > > >> > >> > > > > >> > > >> > >> > > > > >> > > > >> > >> > > > > >> > > > Thanks >> > >> > > > > >> > > > >> > >> > > > > >> > > > On Mon, Sep 12, 2016 at 12:15 PM, Stack < >> > >> st...@duboce.net> >> > >> > > > wrote: >> > >> > > > > >> > > > >> > >> > > > > >> > > > > On review of the 'patch', do I just compare the >> > branch >> > >> to >> > >> > > > > master or >> > >> > > > > >> > is >> > >> > > > > >> > > > > there a megapatch posted somewhere (I think I saw >> one >> > >> but >> > >> > it >> > >> > > > > seemed >> > >> > > > > >> > > stale >> > >> > > > > >> > > > > and then I 'lost' the tab). Sorry for dumb >> question. >> > >> > > > > >> > > > > St.Ack >> > >> > > > > >> > > > > >> > >> > > > > >> > > > > On Mon, Sep 12, 2016 at 12:01 PM, Stack < >> > >> st...@duboce.net >> > >> > > >> > >> > > > > wrote: >> > >> > > > > >> > > > > >> > >> > > > > >> > > > > > Late to the game. A few comments after rereading >> > this >> > >> > > thread >> > >> > > > > as a >> > >> > > > > >> > > > 'user'. >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > + Before merge, a user-facing feature like this >> > >> should >> > >> > > work >> > >> > > > > (If >> > >> > > > > >> > this >> > >> > > > > >> > > is >> > >> > > > > >> > > > > "higher-bar >> > >> > > > > >> > > > > > for new features", bring it on -- smile). >> > >> > > > > >> > > > > > + As a user, I tried the branch with tools after >> > >> > reviewing >> > >> > > > the >> > >> > > > > >> > > > > just-posted >> > >> > > > > >> > > > > > doc. I had an 'interesting' experience (left >> > >> comments up >> > >> > > on >> > >> > > > > >> > issue). I >> > >> > > > > >> > > > > think >> > >> > > > > >> > > > > > the tooling/doc. important to get right. If it >> > breaks >> > >> > > easily >> > >> > > > > or >> > >> > > > > >> is >> > >> > > > > >> > > > > > inconsistent (or lacks 'polish'), operators will >> > >> judge >> > >> > the >> > >> > > > > whole >> > >> > > > > >> > > > > > backup/restore tooling chain as not trustworthy >> and >> > >> > > abandon >> > >> > > > > it. >> > >> > > > > >> > Lets >> > >> > > > > >> > > > not >> > >> > > > > >> > > > > > have this happen to this feature. >> > >> > > > > >> > > > > > + Matteo's suggestion (with a helpful starter >> list) >> > >> that >> > >> > > > there >> > >> > > > > >> > needs >> > >> > > > > >> > > to >> > >> > > > > >> > > > > be >> > >> > > > > >> > > > > > explicit qualification on what is actually being >> > >> > delivered >> > >> > > > -- >> > >> > > > > >> > > > including a >> > >> > > > > >> > > > > > listing of limitations (some look serious such >> as >> > >> data >> > >> > > bleed >> > >> > > > > from >> > >> > > > > >> > > other >> > >> > > > > >> > > > > > regions in WALs, but maybe I don't care for my >> use >> > >> > > case...) >> > >> > > > -- >> > >> > > > > >> > needs >> > >> > > > > >> > > to >> > >> > > > > >> > > > > > accompany the merge. Lets fold them into the >> user >> > >> doc. >> > >> > in >> > >> > > > the >> > >> > > > > >> > > technical >> > >> > > > > >> > > > > > overview area as suggested so user expectations >> are >> > >> > > properly >> > >> > > > > >> > managed >> > >> > > > > >> > > > > > (otherwise, they expect the world and will just >> > give >> > >> up >> > >> > > when >> > >> > > > > we >> > >> > > > > >> > fall >> > >> > > > > >> > > > > > short). Vladimir did a list of what is in each >> of >> > the >> > >> > > phases >> > >> > > > > >> above >> > >> > > > > >> > > > which >> > >> > > > > >> > > > > > would serve as a good start. >> > >> > > > > >> > > > > > + Is this feature 'experimental' (Matteo asks >> > above). >> > >> > I'd >> > >> > > > > prefer >> > >> > > > > >> it >> > >> > > > > >> > > is >> > >> > > > > >> > > > > > not. If it is, it should be labelled all over >> that >> > >> it is >> > >> > > > so. I >> > >> > > > > >> see >> > >> > > > > >> > > > > current >> > >> > > > > >> > > > > > state called out as a '... technical preview >> > >> feature'. >> > >> > > Does >> > >> > > > > this >> > >> > > > > >> > mean >> > >> > > > > >> > > > > > not-for-users? >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > St.Ack >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > On Mon, Sep 12, 2016 at 8:03 AM, Ted Yu < >> > >> > > > yuzhih...@gmail.com> >> > >> > > > > >> > wrote: >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > >> Sean: >> > >> > > > > >> > > > > >> Do you have more comments ? >> > >> > > > > >> > > > > >> >> > >> > > > > >> > > > > >> Cheers >> > >> > > > > >> > > > > >> >> > >> > > > > >> > > > > >> On Fri, Sep 9, 2016 at 1:42 PM, Vladimir >> Rodionov >> > < >> > >> > > > > >> > > > > vladrodio...@gmail.com >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> wrote: >> > >> > > > > >> > > > > >> >> > >> > > > > >> > > > > >> > Sean, >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > Backup/Restore can fail due to various >> reasons: >> > >> > network >> > >> > > > > outage >> > >> > > > > >> > > > > (cluster >> > >> > > > > >> > > > > >> > wide), various time-outs in HBase and HDFS >> > layer, >> > >> M/R >> > >> > > > > failure >> > >> > > > > >> > due >> > >> > > > > >> > > to >> > >> > > > > >> > > > > >> "HDFS >> > >> > > > > >> > > > > >> > exceeded quota", user error (manual deletion >> of >> > >> data) >> > >> > > and >> > >> > > > > so >> > >> > > > > >> on >> > >> > > > > >> > so >> > >> > > > > >> > > > on. >> > >> > > > > >> > > > > >> That >> > >> > > > > >> > > > > >> > is impossible to enumerate all possible >> types of >> > >> > > failures >> > >> > > > > in a >> > >> > > > > >> > > > > >> distributed >> > >> > > > > >> > > > > >> > system - that is not our goal/task. >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > We focus completely on backup system table >> > >> > consistency >> > >> > > > in a >> > >> > > > > >> > > presence >> > >> > > > > >> > > > > of >> > >> > > > > >> > > > > >> any >> > >> > > > > >> > > > > >> > type of failure. That is what I call >> "tolerance >> > to >> > >> > > > > failures". >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > On a failure: >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > BACKUP. All backup system information (prior >> to >> > >> > backup) >> > >> > > > > will >> > >> > > > > >> be >> > >> > > > > >> > > > > restored >> > >> > > > > >> > > > > >> > and all temporary data, related to a failed >> > >> session, >> > >> > in >> > >> > > > > HDFS >> > >> > > > > >> > will >> > >> > > > > >> > > be >> > >> > > > > >> > > > > >> > deleted >> > >> > > > > >> > > > > >> > RESTORE. We do not care about system data, >> > because >> > >> > > > restore >> > >> > > > > >> does >> > >> > > > > >> > > not >> > >> > > > > >> > > > > >> change >> > >> > > > > >> > > > > >> > it. Temporary data in HDFS will be cleaned up >> > and >> > >> > table >> > >> > > > > will >> > >> > > > > >> be >> > >> > > > > >> > > in a >> > >> > > > > >> > > > > >> state >> > >> > > > > >> > > > > >> > back to where it was before operation >> started. >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > This is what user should expect in case of a >> > >> failure. >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > -Vlad >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > -Vlad >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > On Fri, Sep 9, 2016 at 12:56 PM, Sean Busbey >> < >> > >> > > > > >> bus...@apache.org >> > >> > > > > >> > > >> > >> > > > > >> > > > > wrote: >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> > > Failing in a consistent way, with docs that >> > >> explain >> > >> > > the >> > >> > > > > >> > various >> > >> > > > > >> > > > > >> > > expected failures would be sufficient. >> > >> > > > > >> > > > > >> > > >> > >> > > > > >> > > > > >> > > On Fri, Sep 9, 2016 at 12:16 PM, Vladimir >> > >> Rodionov >> > >> > > > > >> > > > > >> > > <vladrodio...@gmail.com> wrote: >> > >> > > > > >> > > > > >> > > > Do not worry Sean, doc is coming today >> as a >> > >> > preview >> > >> > > > and >> > >> > > > > >> our >> > >> > > > > >> > > > writer >> > >> > > > > >> > > > > >> > Frank >> > >> > > > > >> > > > > >> > > > will be working on a putting it into >> Apache >> > >> > repo. >> > >> > > > > >> Timeline >> > >> > > > > >> > > > > depends >> > >> > > > > >> > > > > >> on >> > >> > > > > >> > > > > >> > > > Franks schedule but I hope we will get it >> > >> rather >> > >> > > > sooner >> > >> > > > > >> than >> > >> > > > > >> > > > > later. >> > >> > > > > >> > > > > >> > > > >> > >> > > > > >> > > > > >> > > > As for failure testing, we are focusing >> only >> > >> on a >> > >> > > > > >> consistent >> > >> > > > > >> > > > state >> > >> > > > > >> > > > > >> of >> > >> > > > > >> > > > > >> > > > backup system data in a presence of any >> type >> > >> of >> > >> > > > > failures, >> > >> > > > > >> We >> > >> > > > > >> > > are >> > >> > > > > >> > > > > not >> > >> > > > > >> > > > > >> > > going >> > >> > > > > >> > > > > >> > > > to implement anything more "fancy", than >> > >> that. >> > >> > We >> > >> > > > > allow >> > >> > > > > >> > both: >> > >> > > > > >> > > > > >> backup >> > >> > > > > >> > > > > >> > and >> > >> > > > > >> > > > > >> > > > restore to fail. What we do not allow is >> to >> > >> have >> > >> > > > system >> > >> > > > > >> data >> > >> > > > > >> > > > > >> corrupted. >> > >> > > > > >> > > > > >> > > > Will it suffice for you? Do you have any >> > other >> > >> > > > > concerns, >> > >> > > > > >> you >> > >> > > > > >> > > > want >> > >> > > > > >> > > > > >> us to >> > >> > > > > >> > > > > >> > > > address? >> > >> > > > > >> > > > > >> > > > >> > >> > > > > >> > > > > >> > > > -Vlad >> > >> > > > > >> > > > > >> > > > >> > >> > > > > >> > > > > >> > > > >> > >> > > > > >> > > > > >> > > > On Fri, Sep 9, 2016 at 10:56 AM, Sean >> > Busbey < >> > >> > > > > >> > > bus...@apache.org >> > >> > > > > >> > > > > >> > >> > > > > >> > > > > >> > wrote: >> > >> > > > > >> > > > > >> > > > >> > >> > > > > >> > > > > >> > > >> "docs will come to Apache soon" does not >> > >> address >> > >> > > my >> > >> > > > > >> concern >> > >> > > > > >> > > > > around >> > >> > > > > >> > > > > >> > docs >> > >> > > > > >> > > > > >> > > at >> > >> > > > > >> > > > > >> > > >> all, unless said docs have already made >> it >> > >> into >> > >> > > the >> > >> > > > > >> project >> > >> > > > > >> > > > > repo. I >> > >> > > > > >> > > > > >> > > don't >> > >> > > > > >> > > > > >> > > >> want third party resources for using a >> > major >> > >> and >> > >> > > > > >> important >> > >> > > > > >> > > > > feature >> > >> > > > > >> > > > > >> of >> > >> > > > > >> > > > > >> > > the >> > >> > > > > >> > > > > >> > > >> project, I want us to provide end users >> > with >> > >> > what >> > >> > > > they >> > >> > > > > >> need >> > >> > > > > >> > > to >> > >> > > > > >> > > > > get >> > >> > > > > >> > > > > >> the >> > >> > > > > >> > > > > >> > > job >> > >> > > > > >> > > > > >> > > >> done. >> > >> > > > > >> > > > > >> > > >> >> > >> > > > > >> > > > > >> > > >> I see some calls for patience on the >> > failure >> > >> > > > testing, >> > >> > > > > but >> > >> > > > > >> > the >> > >> > > > > >> > > > > >> appeal >> > >> > > > > >> > > > > >> > to >> > >> > > > > >> > > > > >> > > us >> > >> > > > > >> > > > > >> > > >> having done a bad job of requiring >> proper >> > >> tests >> > >> > of >> > >> > > > > >> previous >> > >> > > > > >> > > > > >> features >> > >> > > > > >> > > > > >> > > just >> > >> > > > > >> > > > > >> > > >> makes me more concerned about not >> getting >> > >> them >> > >> > > > here. I >> > >> > > > > >> > don't >> > >> > > > > >> > > > want >> > >> > > > > >> > > > > >> to >> > >> > > > > >> > > > > >> > set >> > >> > > > > >> > > > > >> > > >> yet another bad example that will then >> be >> > >> > pointed >> > >> > > to >> > >> > > > > in >> > >> > > > > >> the >> > >> > > > > >> > > > > future. >> > >> > > > > >> > > > > >> > > >> >> > >> > > > > >> > > > > >> > > >> On Sep 8, 2016 10:50, "Ted Yu" < >> > >> > > yuzhih...@gmail.com >> > >> > > > > >> > >> > > > > >> > wrote: >> > >> > > > > >> > > > > >> > > >> >> > >> > > > > >> > > > > >> > > >> > Is there any concern which is not >> > >> addressed ? >> > >> > > > > >> > > > > >> > > >> > >> > >> > > > > >> > > > > >> > > >> > Do we need another Vote thread ? >> > >> > > > > >> > > > > >> > > >> > >> > >> > > > > >> > > > > >> > > >> > Thanks >> > >> > > > > >> > > > > >> > > >> > >> > >> > > > > >> > > > > >> > > >> > On Thu, Sep 8, 2016 at 9:21 AM, Andrew >> > >> > Purtell < >> > >> > > > > >> > > > > >> apurt...@apache.org >> > >> > > > > >> > > > > >> > > >> > >> > > > > >> > > > > >> > > >> > wrote: >> > >> > > > > >> > > > > >> > > >> > >> > >> > > > > >> > > > > >> > > >> > > Vlad, >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > I apologize for using the term >> > >> 'half-baked' >> > >> > > in a >> > >> > > > > way >> > >> > > > > >> > that >> > >> > > > > >> > > > > could >> > >> > > > > >> > > > > >> > > seem a >> > >> > > > > >> > > > > >> > > >> > > description of HBASE-7912. I meant >> that >> > >> as a >> > >> > > > > general >> > >> > > > > >> > > > > >> hypothetical. >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > On Wed, Sep 7, 2016 at 9:36 AM, >> > Vladimir >> > >> > > > Rodionov >> > >> > > > > < >> > >> > > > > >> > > > > >> > > >> > vladrodio...@gmail.com> >> > >> > > > > >> > > > > >> > > >> > > wrote: >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > > >> I'm not sure that "There is >> > already >> > >> > lots >> > >> > > of >> > >> > > > > >> > > half-baked >> > >> > > > > >> > > > > >> code >> > >> > > > > >> > > > > >> > in >> > >> > > > > >> > > > > >> > > the >> > >> > > > > >> > > > > >> > > >> > > > branch, >> > >> > > > > >> > > > > >> > > >> > > > so what's the harm in adding >> more?" >> > >> > > > > >> > > > > >> > > >> > > > >> > >> > > > > >> > > > > >> > > >> > > > I meant - not production - ready >> yet. >> > >> This >> > >> > > is >> > >> > > > > 2.0 >> > >> > > > > >> > > > > development >> > >> > > > > >> > > > > >> > > branch >> > >> > > > > >> > > > > >> > > >> > and, >> > >> > > > > >> > > > > >> > > >> > > > hence many features are in works, >> > >> > > > > >> > > > > >> > > >> > > > not being tested well etc. I do >> not >> > >> > consider >> > >> > > > > backup >> > >> > > > > >> > as >> > >> > > > > >> > > > half >> > >> > > > > >> > > > > >> > baked >> > >> > > > > >> > > > > >> > > >> > > feature - >> > >> > > > > >> > > > > >> > > >> > > > it has passed our internal QA and >> has >> > >> very >> > >> > > > good >> > >> > > > > >> doc, >> > >> > > > > >> > > > which >> > >> > > > > >> > > > > we >> > >> > > > > >> > > > > >> > will >> > >> > > > > >> > > > > >> > > >> > > provide >> > >> > > > > >> > > > > >> > > >> > > > to Apache shortly. >> > >> > > > > >> > > > > >> > > >> > > > >> > >> > > > > >> > > > > >> > > >> > > > -Vlad >> > >> > > > > >> > > > > >> > > >> > > > >> > >> > > > > >> > > > > >> > > >> > > > On Wed, Sep 7, 2016 at 9:13 AM, >> > Andrew >> > >> > > > Purtell < >> > >> > > > > >> > > > > >> > > apurt...@apache.org> >> > >> > > > > >> > > > > >> > > >> > > > wrote: >> > >> > > > > >> > > > > >> > > >> > > > >> > >> > > > > >> > > > > >> > > >> > > > > We shouldn't admit half baked >> > changes >> > >> > that >> > >> > > > > won't >> > >> > > > > >> be >> > >> > > > > >> > > > > >> finished. >> > >> > > > > >> > > > > >> > > >> However >> > >> > > > > >> > > > > >> > > >> > > in >> > >> > > > > >> > > > > >> > > >> > > > > this case the crew working on >> this >> > >> > feature >> > >> > > > are >> > >> > > > > >> long >> > >> > > > > >> > > > > timers >> > >> > > > > >> > > > > >> and >> > >> > > > > >> > > > > >> > > less >> > >> > > > > >> > > > > >> > > >> > > > likely >> > >> > > > > >> > > > > >> > > >> > > > > than just about anyone to leave >> > >> > something >> > >> > > > in a >> > >> > > > > >> half >> > >> > > > > >> > > > baked >> > >> > > > > >> > > > > >> > > state. Of >> > >> > > > > >> > > > > >> > > >> > > > course >> > >> > > > > >> > > > > >> > > >> > > > > there is no guarantee how >> anything >> > >> will >> > >> > > turn >> > >> > > > > out, >> > >> > > > > >> > > but I >> > >> > > > > >> > > > > am >> > >> > > > > >> > > > > >> > > willing >> > >> > > > > >> > > > > >> > > >> to >> > >> > > > > >> > > > > >> > > >> > > > take >> > >> > > > > >> > > > > >> > > >> > > > > a little on faith if they feel >> > their >> > >> > best >> > >> > > > path >> > >> > > > > >> > > forward >> > >> > > > > >> > > > > now >> > >> > > > > >> > > > > >> is >> > >> > > > > >> > > > > >> > to >> > >> > > > > >> > > > > >> > > >> > merge >> > >> > > > > >> > > > > >> > > >> > > to >> > >> > > > > >> > > > > >> > > >> > > > > trunk. I only wish I had >> bandwidth >> > to >> > >> > have >> > >> > > > > done >> > >> > > > > >> > some >> > >> > > > > >> > > > real >> > >> > > > > >> > > > > >> > > kicking >> > >> > > > > >> > > > > >> > > >> of >> > >> > > > > >> > > > > >> > > >> > > the >> > >> > > > > >> > > > > >> > > >> > > > > tires by now. Maybe this week. >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > (Yes, I'm using some of that >> time >> > for >> > >> > this >> > >> > > > > email >> > >> > > > > >> > :-) >> > >> > > > > >> > > > but >> > >> > > > > >> > > > > I >> > >> > > > > >> > > > > >> > type >> > >> > > > > >> > > > > >> > > >> > fast.) >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > That said, I would like to >> agitate >> > >> for >> > >> > > > making >> > >> > > > > 2.0 >> > >> > > > > >> > > more >> > >> > > > > >> > > > > real >> > >> > > > > >> > > > > >> > and >> > >> > > > > >> > > > > >> > > >> spend >> > >> > > > > >> > > > > >> > > >> > > > some >> > >> > > > > >> > > > > >> > > >> > > > > time on it now that I'm winding >> > down >> > >> > with >> > >> > > > > 0.98. I >> > >> > > > > >> > > think >> > >> > > > > >> > > > > >> that >> > >> > > > > >> > > > > >> > > means >> > >> > > > > >> > > > > >> > > >> > > > > branching for 2.0 real soon now >> and >> > >> even >> > >> > > > > evicting >> > >> > > > > >> > > > things >> > >> > > > > >> > > > > >> from >> > >> > > > > >> > > > > >> > > 2.0 >> > >> > > > > >> > > > > >> > > >> > > branch >> > >> > > > > >> > > > > >> > > >> > > > > that aren't finished or stable, >> > >> leaving >> > >> > > them >> > >> > > > > only >> > >> > > > > >> > > once >> > >> > > > > >> > > > > >> again >> > >> > > > > >> > > > > >> > in >> > >> > > > > >> > > > > >> > > the >> > >> > > > > >> > > > > >> > > >> > > > master >> > >> > > > > >> > > > > >> > > >> > > > > branch. Or, maybe just evicting >> > them. >> > >> > > Let's >> > >> > > > > take >> > >> > > > > >> it >> > >> > > > > >> > > > case >> > >> > > > > >> > > > > by >> > >> > > > > >> > > > > >> > > case. >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > I think this feature can come in >> > >> > > relatively >> > >> > > > > >> safely. >> > >> > > > > >> > > As >> > >> > > > > >> > > > > >> added >> > >> > > > > >> > > > > >> > > >> > insurance, >> > >> > > > > >> > > > > >> > > >> > > > > let's admit the possibility it >> > could >> > >> be >> > >> > > > > reverted >> > >> > > > > >> on >> > >> > > > > >> > > the >> > >> > > > > >> > > > > 2.0 >> > >> > > > > >> > > > > >> > > branch >> > >> > > > > >> > > > > >> > > >> if >> > >> > > > > >> > > > > >> > > >> > > > folks >> > >> > > > > >> > > > > >> > > >> > > > > working on stabilizing 2.0 >> decide >> > to >> > >> > evict >> > >> > > > it >> > >> > > > > >> > because >> > >> > > > > >> > > > it >> > >> > > > > >> > > > > is >> > >> > > > > >> > > > > >> > > >> > unfinished >> > >> > > > > >> > > > > >> > > >> > > or >> > >> > > > > >> > > > > >> > > >> > > > > unstable, because that certainly >> > can >> > >> > > > happen. I >> > >> > > > > >> > would >> > >> > > > > >> > > > > >> expect if >> > >> > > > > >> > > > > >> > > talk >> > >> > > > > >> > > > > >> > > >> > > like >> > >> > > > > >> > > > > >> > > >> > > > > that starts, we'd get help >> > finishing >> > >> or >> > >> > > > > >> stabilizing >> > >> > > > > >> > > > > what's >> > >> > > > > >> > > > > >> > under >> > >> > > > > >> > > > > >> > > >> > > > discussion >> > >> > > > > >> > > > > >> > > >> > > > > for revert. Or, we'd have a >> revert. >> > >> > Either >> > >> > > > way >> > >> > > > > >> the >> > >> > > > > >> > > > > outcome >> > >> > > > > >> > > > > >> is >> > >> > > > > >> > > > > >> > > >> > > acceptable. >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > On Wed, Sep 7, 2016 at 8:56 AM, >> > Dima >> > >> > > Spivak >> > >> > > > < >> > >> > > > > >> > > > > >> > > dimaspi...@apache.org >> > >> > > > > >> > > > > >> > > >> > >> > >> > > > > >> > > > > >> > > >> > > > wrote: >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > I'm not sure that "There is >> > already >> > >> > lots >> > >> > > > of >> > >> > > > > >> > > > half-baked >> > >> > > > > >> > > > > >> code >> > >> > > > > >> > > > > >> > in >> > >> > > > > >> > > > > >> > > >> the >> > >> > > > > >> > > > > >> > > >> > > > > branch, >> > >> > > > > >> > > > > >> > > >> > > > > > so what's the harm in adding >> > more?" >> > >> > is a >> > >> > > > > good >> > >> > > > > >> > code >> > >> > > > > >> > > > > commit >> > >> > > > > >> > > > > >> > > >> > philosophy >> > >> > > > > >> > > > > >> > > >> > > > for >> > >> > > > > >> > > > > >> > > >> > > > > a >> > >> > > > > >> > > > > >> > > >> > > > > > fault-tolerant distributed >> data >> > >> store. >> > >> > > ;) >> > >> > > > > >> > > > > >> > > >> > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > More seriously, a lack of test >> > >> > coverage >> > >> > > > for >> > >> > > > > >> > > existing >> > >> > > > > >> > > > > >> > features >> > >> > > > > >> > > > > >> > > >> > > shouldn't >> > >> > > > > >> > > > > >> > > >> > > > > be >> > >> > > > > >> > > > > >> > > >> > > > > > used as justification for >> > >> introducing >> > >> > > new >> > >> > > > > >> > features >> > >> > > > > >> > > > with >> > >> > > > > >> > > > > >> the >> > >> > > > > >> > > > > >> > > same >> > >> > > > > >> > > > > >> > > >> > > > > > shortcomings. Ultimately, it's >> > the >> > >> end >> > >> > > > user >> > >> > > > > who >> > >> > > > > >> > > will >> > >> > > > > >> > > > > feel >> > >> > > > > >> > > > > >> > the >> > >> > > > > >> > > > > >> > > >> pain, >> > >> > > > > >> > > > > >> > > >> > > so >> > >> > > > > >> > > > > >> > > >> > > > > > shouldn't we do everything we >> can >> > >> to >> > >> > > > > mitigate >> > >> > > > > >> > that? >> > >> > > > > >> > > > > >> > > >> > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > -Dima >> > >> > > > > >> > > > > >> > > >> > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > On Wed, Sep 7, 2016 at 8:46 >> AM, >> > >> > Vladimir >> > >> > > > > >> > Rodionov < >> > >> > > > > >> > > > > >> > > >> > > > > vladrodio...@gmail.com> >> > >> > > > > >> > > > > >> > > >> > > > > > wrote: >> > >> > > > > >> > > > > >> > > >> > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > Sean, >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > * have docs >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > Agree. We have a doc and >> backup >> > >> is >> > >> > the >> > >> > > > > most >> > >> > > > > >> > > > > documented >> > >> > > > > >> > > > > >> > > feature >> > >> > > > > >> > > > > >> > > >> > :), >> > >> > > > > >> > > > > >> > > >> > > we >> > >> > > > > >> > > > > >> > > >> > > > > > will >> > >> > > > > >> > > > > >> > > >> > > > > > > release it shortly to >> Apache. >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > * have sunny-day correctness >> > >> tests >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > Feature has close to 60 >> test >> > >> cases, >> > >> > > > which >> > >> > > > > >> run >> > >> > > > > >> > > for >> > >> > > > > >> > > > > >> approx >> > >> > > > > >> > > > > >> > 30 >> > >> > > > > >> > > > > >> > > >> min. >> > >> > > > > >> > > > > >> > > >> > > We >> > >> > > > > >> > > > > >> > > >> > > > > can >> > >> > > > > >> > > > > >> > > >> > > > > > > add more, if community do >> not >> > >> mind >> > >> > :) >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > * have >> > >> > correctness-in-face-of-failure >> > >> > > > > tests >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > Any examples of these tests >> in >> > >> > > existing >> > >> > > > > >> > features? >> > >> > > > > >> > > > In >> > >> > > > > >> > > > > >> > works, >> > >> > > > > >> > > > > >> > > we >> > >> > > > > >> > > > > >> > > >> > > have a >> > >> > > > > >> > > > > >> > > >> > > > > > clear >> > >> > > > > >> > > > > >> > > >> > > > > > > understanding of what >> should be >> > >> done >> > >> > > by >> > >> > > > > the >> > >> > > > > >> > time >> > >> > > > > >> > > of >> > >> > > > > >> > > > > 2.0 >> > >> > > > > >> > > > > >> > > >> release. >> > >> > > > > >> > > > > >> > > >> > > > > > > That is very close goal for >> us, >> > >> to >> > >> > > > verify >> > >> > > > > IT >> > >> > > > > >> > > monkey >> > >> > > > > >> > > > > for >> > >> > > > > >> > > > > >> > > >> existing >> > >> > > > > >> > > > > >> > > >> > > > code. >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > * don't rely on things >> outside >> > of >> > >> > > HBase >> > >> > > > > for >> > >> > > > > >> > > normal >> > >> > > > > >> > > > > >> > operation >> > >> > > > > >> > > > > >> > > >> > (okay >> > >> > > > > >> > > > > >> > > >> > > > for >> > >> > > > > >> > > > > >> > > >> > > > > > > advanced operation) >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > We do not. >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > Enormous time has been spent >> > >> already >> > >> > > on >> > >> > > > > the >> > >> > > > > >> > > > > development >> > >> > > > > >> > > > > >> > and >> > >> > > > > >> > > > > >> > > >> > testing >> > >> > > > > >> > > > > >> > > >> > > > the >> > >> > > > > >> > > > > >> > > >> > > > > > > feature, it has passed our >> > >> internal >> > >> > > > tests >> > >> > > > > and >> > >> > > > > >> > > many >> > >> > > > > >> > > > > >> rounds >> > >> > > > > >> > > > > >> > of >> > >> > > > > >> > > > > >> > > >> code >> > >> > > > > >> > > > > >> > > >> > > > > reviews >> > >> > > > > >> > > > > >> > > >> > > > > > > by HBase committers. We do >> not >> > >> mind >> > >> > if >> > >> > > > > >> someone >> > >> > > > > >> > > from >> > >> > > > > >> > > > > >> HBase >> > >> > > > > >> > > > > >> > > >> > community >> > >> > > > > >> > > > > >> > > >> > > > > > > (outside of HW) will review >> the >> > >> > code, >> > >> > > > but >> > >> > > > > it >> > >> > > > > >> > will >> > >> > > > > >> > > > > >> probably >> > >> > > > > >> > > > > >> > > >> takes >> > >> > > > > >> > > > > >> > > >> > > > > forever >> > >> > > > > >> > > > > >> > > >> > > > > > to >> > >> > > > > >> > > > > >> > > >> > > > > > > wait for volunteer?, the >> > feature >> > >> is >> > >> > > > quite >> > >> > > > > >> large >> > >> > > > > >> > > > (1MB+ >> > >> > > > > >> > > > > >> > > >> cumulative >> > >> > > > > >> > > > > >> > > >> > > > patch) >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > 2.0 branch is full of half >> > baked >> > >> > > > features, >> > >> > > > > >> most >> > >> > > > > >> > > of >> > >> > > > > >> > > > > them >> > >> > > > > >> > > > > >> > are >> > >> > > > > >> > > > > >> > > in >> > >> > > > > >> > > > > >> > > >> > > active >> > >> > > > > >> > > > > >> > > >> > > > > > > development, therefore I am >> not >> > >> > > > following >> > >> > > > > you >> > >> > > > > >> > > here, >> > >> > > > > >> > > > > >> Sean? >> > >> > > > > >> > > > > >> > > Why >> > >> > > > > >> > > > > >> > > >> > > > > HBASE-7912 >> > >> > > > > >> > > > > >> > > >> > > > > > is >> > >> > > > > >> > > > > >> > > >> > > > > > > not good enough yet to be >> > >> integrated >> > >> > > > into >> > >> > > > > 2.0 >> > >> > > > > >> > > > branch? >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > -Vlad >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > On Wed, Sep 7, 2016 at 8:23 >> AM, >> > >> Sean >> > >> > > > > Busbey < >> > >> > > > > >> > > > > >> > > bus...@apache.org >> > >> > > > > >> > > > > >> > > >> > >> > >> > > > > >> > > > > >> > > >> > > > wrote: >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > On Tue, Sep 6, 2016 at >> 10:36 >> > >> PM, >> > >> > > Josh >> > >> > > > > >> Elser < >> > >> > > > > >> > > > > >> > > >> > > josh.el...@gmail.com> >> > >> > > > > >> > > > > >> > > >> > > > > > > wrote: >> > >> > > > > >> > > > > >> > > >> > > > > > > > > So, the answer to Sean's >> > >> > original >> > >> > > > > >> question >> > >> > > > > >> > is >> > >> > > > > >> > > > "as >> > >> > > > > >> > > > > >> > > robust as >> > >> > > > > >> > > > > >> > > >> > > > > snapshots >> > >> > > > > >> > > > > >> > > >> > > > > > > > > presently are"? >> > >> (independence of >> > >> > > > > >> > > backup/restore >> > >> > > > > >> > > > > >> > failure >> > >> > > > > >> > > > > >> > > >> > > tolerance >> > >> > > > > >> > > > > >> > > >> > > > > > from >> > >> > > > > >> > > > > >> > > >> > > > > > > > > snapshot failure >> tolerance) >> > >> > > > > >> > > > > >> > > >> > > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > > Is this just a question >> WRT >> > >> > > context >> > >> > > > of >> > >> > > > > >> the >> > >> > > > > >> > > > > change, >> > >> > > > > >> > > > > >> or >> > >> > > > > >> > > > > >> > > is it >> > >> > > > > >> > > > > >> > > >> > > means >> > >> > > > > >> > > > > >> > > >> > > > > > for a >> > >> > > > > >> > > > > >> > > >> > > > > > > > veto >> > >> > > > > >> > > > > >> > > >> > > > > > > > > from you, Sean? Just >> trying >> > >> to >> > >> > > make >> > >> > > > > sure >> > >> > > > > >> > I'm >> > >> > > > > >> > > > > >> following >> > >> > > > > >> > > > > >> > > >> along >> > >> > > > > >> > > > > >> > > >> > > > > > > adequately. >> > >> > > > > >> > > > > >> > > >> > > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > I'd say ATM I'm -0, >> bordering >> > >> on >> > >> > -1 >> > >> > > > but >> > >> > > > > not >> > >> > > > > >> > for >> > >> > > > > >> > > > > >> reasons >> > >> > > > > >> > > > > >> > I >> > >> > > > > >> > > > > >> > > can >> > >> > > > > >> > > > > >> > > >> > > > > > articulate >> > >> > > > > >> > > > > >> > > >> > > > > > > > well. >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > Here's an attempt. >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > We've been trying to move, >> > as a >> > >> > > > > community, >> > >> > > > > >> > > > towards >> > >> > > > > >> > > > > >> > > minimizing >> > >> > > > > >> > > > > >> > > >> > > risk >> > >> > > > > >> > > > > >> > > >> > > > to >> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream folks by >> getting >> > >> > > "complete >> > >> > > > > >> enough >> > >> > > > > >> > > for >> > >> > > > > >> > > > > use" >> > >> > > > > >> > > > > >> > > gates >> > >> > > > > >> > > > > >> > > >> in >> > >> > > > > >> > > > > >> > > >> > > > place >> > >> > > > > >> > > > > >> > > >> > > > > > > > before we introduce new >> > >> features. >> > >> > > This >> > >> > > > > was >> > >> > > > > >> > > > spurred >> > >> > > > > >> > > > > >> by a >> > >> > > > > >> > > > > >> > > some >> > >> > > > > >> > > > > >> > > >> > > > features >> > >> > > > > >> > > > > >> > > >> > > > > > > > getting in half-baked and >> > never >> > >> > > making >> > >> > > > > it >> > >> > > > > >> to >> > >> > > > > >> > > "can >> > >> > > > > >> > > > > >> really >> > >> > > > > >> > > > > >> > > use" >> > >> > > > > >> > > > > >> > > >> > > > status >> > >> > > > > >> > > > > >> > > >> > > > > > > > (I'm thinking of >> distributed >> > >> log >> > >> > > > replay >> > >> > > > > and >> > >> > > > > >> > the >> > >> > > > > >> > > > > >> zk-less >> > >> > > > > >> > > > > >> > > >> > > assignment >> > >> > > > > >> > > > > >> > > >> > > > > > > > stuff, I don't recall if >> > there >> > >> was >> > >> > > > > more). >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > The gates, generally, >> > included >> > >> > > things >> > >> > > > > like: >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > * have docs >> > >> > > > > >> > > > > >> > > >> > > > > > > > * have sunny-day >> correctness >> > >> tests >> > >> > > > > >> > > > > >> > > >> > > > > > > > * have >> > >> > > correctness-in-face-of-failure >> > >> > > > > tests >> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't rely on things >> > outside >> > >> of >> > >> > > > HBase >> > >> > > > > for >> > >> > > > > >> > > > normal >> > >> > > > > >> > > > > >> > > operation >> > >> > > > > >> > > > > >> > > >> > > (okay >> > >> > > > > >> > > > > >> > > >> > > > > for >> > >> > > > > >> > > > > >> > > >> > > > > > > > advanced operation) >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > As an example, we kept the >> > MOB >> > >> > work >> > >> > > > off >> > >> > > > > in >> > >> > > > > >> a >> > >> > > > > >> > > > branch >> > >> > > > > >> > > > > >> and >> > >> > > > > >> > > > > >> > > out >> > >> > > > > >> > > > > >> > > >> of >> > >> > > > > >> > > > > >> > > >> > > > master >> > >> > > > > >> > > > > >> > > >> > > > > > > > until it could pass these >> > >> > criteria. >> > >> > > > The >> > >> > > > > big >> > >> > > > > >> > > > > exemption >> > >> > > > > >> > > > > >> > > we've >> > >> > > > > >> > > > > >> > > >> had >> > >> > > > > >> > > > > >> > > >> > > to >> > >> > > > > >> > > > > >> > > >> > > > > > > > this was the hbase-spark >> > >> > > integration, >> > >> > > > > where >> > >> > > > > >> > we >> > >> > > > > >> > > > all >> > >> > > > > >> > > > > >> > agreed >> > >> > > > > >> > > > > >> > > it >> > >> > > > > >> > > > > >> > > >> > > could >> > >> > > > > >> > > > > >> > > >> > > > > > > > land in master because it >> was >> > >> very >> > >> > > > well >> > >> > > > > >> > > isolated >> > >> > > > > >> > > > > (the >> > >> > > > > >> > > > > >> > > slide >> > >> > > > > >> > > > > >> > > >> > away >> > >> > > > > >> > > > > >> > > >> > > > from >> > >> > > > > >> > > > > >> > > >> > > > > > > > including docs as a >> > first-class >> > >> > part >> > >> > > > of >> > >> > > > > >> > > building >> > >> > > > > >> > > > up >> > >> > > > > >> > > > > >> that >> > >> > > > > >> > > > > >> > > >> > > > integration >> > >> > > > > >> > > > > >> > > >> > > > > > > > has led me to doubt the >> > wisdom >> > >> of >> > >> > > this >> > >> > > > > >> > > decision). >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > We've also been treating >> > >> inclusion >> > >> > > in >> > >> > > > a >> > >> > > > > >> > > "probably >> > >> > > > > >> > > > > >> will >> > >> > > > > >> > > > > >> > be >> > >> > > > > >> > > > > >> > > >> > > released >> > >> > > > > >> > > > > >> > > >> > > > to >> > >> > > > > >> > > > > >> > > >> > > > > > > > downstream" branches as a >> > >> higher >> > >> > > bar, >> > >> > > > > >> > requiring >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't moderately impact >> > >> > > performance >> > >> > > > > when >> > >> > > > > >> > the >> > >> > > > > >> > > > > >> feature >> > >> > > > > >> > > > > >> > > isn't >> > >> > > > > >> > > > > >> > > >> in >> > >> > > > > >> > > > > >> > > >> > > use >> > >> > > > > >> > > > > >> > > >> > > > > > > > * don't severely impact >> > >> > performance >> > >> > > > when >> > >> > > > > >> the >> > >> > > > > >> > > > > feature >> > >> > > > > >> > > > > >> is >> > >> > > > > >> > > > > >> > in >> > >> > > > > >> > > > > >> > > >> use >> > >> > > > > >> > > > > >> > > >> > > > > > > > * either default-to-on or >> > show >> > >> > > enough >> > >> > > > > >> demand >> > >> > > > > >> > to >> > >> > > > > >> > > > > >> believe >> > >> > > > > >> > > > > >> > a >> > >> > > > > >> > > > > >> > > >> > > > non-trivial >> > >> > > > > >> > > > > >> > > >> > > > > > > > number of folks will turn >> the >> > >> > > feature >> > >> > > > on >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > The above has kept MOB and >> > >> > > hbase-spark >> > >> > > > > >> > > > integration >> > >> > > > > >> > > > > >> out >> > >> > > > > >> > > > > >> > of >> > >> > > > > >> > > > > >> > > >> > > branch-1, >> > >> > > > > >> > > > > >> > > >> > > > > > > > presumably while they've >> > >> "gotten >> > >> > > more >> > >> > > > > >> stable" >> > >> > > > > >> > > in >> > >> > > > > >> > > > > >> master >> > >> > > > > >> > > > > >> > > from >> > >> > > > > >> > > > > >> > > >> > the >> > >> > > > > >> > > > > >> > > >> > > > odd >> > >> > > > > >> > > > > >> > > >> > > > > > > > vendor inclusion. >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > Are we going to have a 2.0 >> > >> release >> > >> > > > > before >> > >> > > > > >> the >> > >> > > > > >> > > end >> > >> > > > > >> > > > > of >> > >> > > > > >> > > > > >> the >> > >> > > > > >> > > > > >> > > >> year? >> > >> > > > > >> > > > > >> > > >> > > > We're >> > >> > > > > >> > > > > >> > > >> > > > > > > > coming up on 1.5 years >> since >> > >> the >> > >> > > > > release of >> > >> > > > > >> > > > version >> > >> > > > > >> > > > > >> 1.0; >> > >> > > > > >> > > > > >> > > >> seems >> > >> > > > > >> > > > > >> > > >> > > like >> > >> > > > > >> > > > > >> > > >> > > > > > > > it's about time, though I >> > >> haven't >> > >> > > seen >> > >> > > > > any >> > >> > > > > >> > > > concrete >> > >> > > > > >> > > > > >> > plans >> > >> > > > > >> > > > > >> > > >> this >> > >> > > > > >> > > > > >> > > >> > > > year. >> > >> > > > > >> > > > > >> > > >> > > > > > > > Presuming we are going to >> > have >> > >> one >> > >> > > by >> > >> > > > > the >> > >> > > > > >> end >> > >> > > > > >> > > of >> > >> > > > > >> > > > > the >> > >> > > > > >> > > > > >> > > year, it >> > >> > > > > >> > > > > >> > > >> > > > seems a >> > >> > > > > >> > > > > >> > > >> > > > > > > > bit close to still be >> adding >> > in >> > >> > > > > "features >> > >> > > > > >> > that >> > >> > > > > >> > > > need >> > >> > > > > >> > > > > >> > > maturing" >> > >> > > > > >> > > > > >> > > >> > on >> > >> > > > > >> > > > > >> > > >> > > > the >> > >> > > > > >> > > > > >> > > >> > > > > > > > branch. >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > > The lack of a concrete >> plan >> > for >> > >> > 2.0 >> > >> > > > > keeps >> > >> > > > > >> me >> > >> > > > > >> > > from >> > >> > > > > >> > > > > >> > > considering >> > >> > > > > >> > > > > >> > > >> > > these >> > >> > > > > >> > > > > >> > > >> > > > > > > > things blocker at the >> moment. >> > >> But >> > >> > I >> > >> > > > know >> > >> > > > > >> > first >> > >> > > > > >> > > > hand >> > >> > > > > >> > > > > >> how >> > >> > > > > >> > > > > >> > > much >> > >> > > > > >> > > > > >> > > >> > > > trouble >> > >> > > > > >> > > > > >> > > >> > > > > > > > folks have had with other >> > >> features >> > >> > > > that >> > >> > > > > >> have >> > >> > > > > >> > > gone >> > >> > > > > >> > > > > >> into >> > >> > > > > >> > > > > >> > > >> > downstream >> > >> > > > > >> > > > > >> > > >> > > > > > > > facing releases without >> > >> robustness >> > >> > > > > checks >> > >> > > > > >> > (i.e. >> > >> > > > > >> > > > > >> > > replication), >> > >> > > > > >> > > > > >> > > >> > and >> > >> > > > > >> > > > > >> > > >> > > > I'm >> > >> > > > > >> > > > > >> > > >> > > > > > > > concerned about what we're >> > >> setting >> > >> > > up >> > >> > > > if >> > >> > > > > >> 2.0 >> > >> > > > > >> > > goes >> > >> > > > > >> > > > > out >> > >> > > > > >> > > > > >> > with >> > >> > > > > >> > > > > >> > > >> this >> > >> > > > > >> > > > > >> > > >> > > > > > > > feature in its current >> state. >> > >> > > > > >> > > > > >> > > >> > > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > > >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > -- >> > >> > > > > >> > > > > >> > > >> > > > > Best regards, >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > - Andy >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > > Problems worthy of attack prove >> > their >> > >> > > worth >> > >> > > > by >> > >> > > > > >> > > hitting >> > >> > > > > >> > > > > >> back. - >> > >> > > > > >> > > > > >> > > Piet >> > >> > > > > >> > > > > >> > > >> > > Hein >> > >> > > > > >> > > > > >> > > >> > > > > (via Tom White) >> > >> > > > > >> > > > > >> > > >> > > > > >> > >> > > > > >> > > > > >> > > >> > > > >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > -- >> > >> > > > > >> > > > > >> > > >> > > Best regards, >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > - Andy >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > > Problems worthy of attack prove >> their >> > >> worth >> > >> > by >> > >> > > > > >> hitting >> > >> > > > > >> > > > back. >> > >> > > > > >> > > > > - >> > >> > > > > >> > > > > >> > Piet >> > >> > > > > >> > > > > >> > > >> Hein >> > >> > > > > >> > > > > >> > > >> > > (via Tom White) >> > >> > > > > >> > > > > >> > > >> > > >> > >> > > > > >> > > > > >> > > >> > >> > >> > > > > >> > > > > >> > > >> >> > >> > > > > >> > > > > >> > > >> > >> > > > > >> > > > > >> > >> > >> > > > > >> > > > > >> >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > > >> > >> > > > > >> > > > > >> > >> > > > > >> > > > >> > >> > > > > >> > > >> > >> > > > > >> > >> > >> > > > > >> >> > >> > > > > >> > >> > > > > >> > >> > > > > >> > >> > > > > -- >> > >> > > > > busbey >> > >> > > > > >> > >> > > > >> > >> > > >> > >> > >> > >> >> > > >> > > >> > >> > >
