> If we do time-based releases without wire and format compatibility, it would consume a lot of our energy satisfying such requirements from different companies.
Since currently we do not have guarantees of wire and format compatibility across major releases, only that of a migration step, this isn't really relevant to the question of time gated releasing or not. Adding that requirement would have to happen first. Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) >________________________________ > From: Ted Yu <yuzhih...@gmail.com> >To: dev@hbase.apache.org; Andrew Purtell <apurt...@apache.org> >Cc: lars hofhansl <lhofha...@yahoo.com>; Matt Corgan <mcor...@hotpads.com> >Sent: Monday, January 30, 2012 11:46 AM >Subject: Re: hbase 0.94.0 > > >Before long term wire and format compatibility is reached, I am -0 on time >based release. > >As Nicolas pointed out, FB wants to migrate from 0.89-fb to 0.94 >If we do time-based releases without wire and format compatibility, it would >consume a lot of our energy satisfying such requirements from different >companies. > >I propose wire and format compatibility as the main theme for 0.96 > >Cheers > > >On Mon, Jan 30, 2012 at 11:40 AM, Andrew Purtell <apurt...@apache.org> wrote: > >> Maybe the discussion should be whether 0.94 should be a time-gated or >>> feature-gated release? IMO we already have enough good stuff in there >>> on the perf front. It will be great if the checksum improvement makes >>> it, but if it doesn't, I'd rather have a release than delay for it. >> >> >>Time gated releases makes sense going forward for a few reasons, mark me down >>for that option. >> >> >> >>Best regards, >> >> - Andy >> >>Problems worthy of attack prove their worth by hitting back. - Piet Hein (via >>Tom White) >> >> >> >>----- Original Message ----- >>> From: Todd Lipcon <t...@cloudera.com> >>> To: dev@hbase.apache.org >> >>> Cc: lars hofhansl <lhofha...@yahoo.com>; Matt Corgan <mcor...@hotpads.com> >>> Sent: Monday, January 30, 2012 10:44 AM >>> Subject: Re: hbase 0.94.0 >>> >>> On Sun, Jan 29, 2012 at 7:30 AM, Ted Yu <yuzhih...@gmail.com> wrote: >>>> Initial results from HBASE-5074, support checksums in HBase block cache, >>>> look promising. >>>> >>>> Shall we discuss whether 0.94 should include it (assuming Dhruba can >>>> finish >>>> the feature in Feb.) ? >>> >>> If it's a time-based release, then there's nothing to discuss. Either >>> it's done in time, in which case it's part of 94, or it's not, in >>> which case it's part of 96, right? >>> >>> Maybe the discussion should be whether 0.94 should be a time-gated or >>> feature-gated release? IMO we already have enough good stuff in there >>> on the perf front. It will be great if the checksum improvement makes >>> it, but if it doesn't, I'd rather have a release than delay for it. >>> >>> -Todd >>> >>>> >>>> Cheers >>>> >>>> On Fri, Jan 27, 2012 at 6:27 PM, lars hofhansl <lhofha...@yahoo.com> >>> wrote: >>>> >>>>> Salesforce will jumpstart with 0.94. :) Our dev cluster has the current >>>>> trunk on it. >>>>> >>>>> OK let's just add these to 0.94: >>>>> HBASE-4218 >>>>> >>>>> HBASE-4608 >>>>> HBASE-5128 >>>>> script necessary to do HBASE-5293 in 0.96 >>>>> >>>>> >>>>> And defer the rest to 0.96, and branch 0.94 soon'ish. >>>>> Do you think you can do a 0.92 and trunk version of HBASE-5128? >>>>> >>>>> -- Lars >>>>> >>>>> >>>>> ________________________________ >>>>> From: Jonathan Hsieh <j...@cloudera.com> >>>>> To: Matt Corgan <mcor...@hotpads.com> >>>>> Cc: lars hofhansl <lhofha...@yahoo.com>; dev >>> <dev@hbase.apache.org> >>>>> Sent: Friday, January 27, 2012 6:11 PM >>>>> Subject: Re: hbase 0.94.0 >>>>> >>>>> Lars, Matt, >>>>> >>>>> I like Matt's suggestion -- as long as it is not a burden let's >>> keep it in. >>>>> If it becomes one, let's do what is best for the project. >>>>> >>>>> If Cloudera needs to do the upgrade path, we'll provide one. If it >>> doesn't >>>>> go to the apache repo, we'll most likely open source it on github >>> or >>>>> something, or "keep it on the jira" like some of the repair >>> ruby scripts. >>>>> >>>>> Question -- to the Trend/SalesForce/FB folks can you talk about your >>> plans >>>>> for upgrades? Are you guys moving from 0.89/0.90ish to 0.92 already? >>> or >>>>> are you guys going to have to do the direct upgrade to 0.94 from 0.90 >>> land >>>>> as well? >>>>> >>>>> Jon. >>>>> >>>>> >>>>> On Fri, Jan 27, 2012 at 12:06 PM, Matt Corgan >>> <mcor...@hotpads.com> wrote: >>>>> >>>>> > The reason I brought it up is now that Mikhail committed the data >>> block >>>>> > encoding I was going to take a stab at adding the prefix trie >>> encoding I >>>>> > was working on this past summer. My plan is to first make a >>> minimally >>>>> > invasive patch to prove correctness. But, after that there will >>> probably >>>>> > be some big performance gains to be had from reworking some of the >>> things >>>>> > like KeyValueScanner which I would not have the courage to get >>> working >>>>> with >>>>> > v1. >>>>> > >>>>> > So, that was why I asked, but all of that is still more >>> hypothetical than >>>>> > real, and I don't even know if the first part will be done >>> before >>>>> branching >>>>> > .94 at the end of February. Makes sense to me to not delete v1 >>> until >>>>> > there's a good reason to, which it doesn't look like we >>> have yet. If I >>>>> get >>>>> > to the point where v1 is halting progress then we can reevaluate >>> based on >>>>> > more specific issues. Maybe none of the prefix trie will even >>> make >>>>> .94... >>>>> > >>>>> > ..sent from my phone >>>>> > On Jan 27, 2012 1:55 PM, "lars hofhansl" >>> <lhofha...@yahoo.com> wrote: >>>>> > >>>>> >> Hey Jon, >>>>> >> >>>>> >> understood. Makes 0.94 hard, though. If we decide now to have >>> a 0.90 to >>>>> >> 0.94 upgrade path and then timing does not work out and nobody >>> signs up >>>>> for >>>>> >> the testing because it 0.92 is more convenient we'd have >>> gone through >>>>> this >>>>> >> for nothing. >>>>> >> >>>>> >> So... Thinking about this more I am -1 on supporting an >>> official upgrade >>>>> >> path other then from one release to the next. >>>>> >> That said, we do not have to break things intentionally. >>>>> >> >>>>> >> >>>>> >> I'm fine pushing HBASE-2600 and HFile v1 removal out of >>> 0.94... As long >>>>> >> as we won't havethe same argument for 0.96 :) >>>>> >> >>>>> >> And I am not aware of any file compatibility issues. >>>>> >> >>>>> >> >>>>> >> We can also leave the 0.92 migration code in, but not >>> officially support >>>>> >> it as 0.90 to 0.94 path. >>>>> >> Then CDH4 can make sure (if needed) that it is all working. >>>>> >> >>>>> >> >>>>> >> Does that work for you guys? >>>>> >> >>>>> >> -- Lars >>>>> >> >>>>> >> >>>>> >> >>>>> >> ________________________________ >>>>> >> From: Jonathan Hsieh <j...@cloudera.com> >>>>> >> To: dev@hbase.apache.org; lars hofhansl >>> <lhofha...@yahoo.com> >>>>> >> Sent: Friday, January 27, 2012 10:10 AM >>>>> >> Subject: Re: hbase 0.94.0 >>>>> >> >>>>> >> >>>>> >> Lars, >>>>> >> >>>>> >> The upcoming CDH4 Beta HBase will be based on the latest >>> hotness, >>>>> 0.92.0. >>>>> >> A CDH4 GA HBase will have to have an upgrade path from CDH3 >>> HBase. If >>>>> that >>>>> >> HBase is 0.92 based, we'll test that, and if timing works >>> out and we >>>>> decide >>>>> >> 0.94, we'll have to have a path (0.90->0.94) for than >>> and will test >>>>> that. >>>>> >> >>>>> >> HBASE-2600 is a big change of encoding of meta, while my >>> understanding >>>>> is >>>>> >> that 0.90->0,92 is a graceful HFile format conversion. Are >>> there are >>>>> >> things currently in trunk that further break compatiblity of >>> the file >>>>> >> format? (what jiras?) >>>>> >> >>>>> >> Jon. >>>>> >> >>>>> >> >>>>> >> On Fri, Jan 27, 2012 at 9:16 AM, lars hofhansl >>> <lhofha...@yahoo.com> >>>>> >> wrote: >>>>> >> >>>>> >> Not removing code for upgrade is fine. >>>>> >> > >>>>> >> >Todd, is Cloudera signing up for testing this path (0.90 >>> to 0.94)? >>>>> >> > >>>>> >> > >>>>> >> >Stack, what's your feeling w.r.t. to HBASE-2600, will >>> keeping the 0.90 >>>>> >> -> 0.92 migration path >>>>> >> >make the migration code for HBASE-2600 (much) more >>> complicated in 0.94? >>>>> >> > >>>>> >> > >>>>> >> >-- Lars >>>>> >> > >>>>> >> > >>>>> >> > >>>>> >> >----- Original Message ----- >>>>> >> >From: Stack <st...@duboce.net> >>>>> >> >To: dev@hbase.apache.org >>>>> >> >Cc: >>>>> >> >Sent: Friday, January 27, 2012 9:02 AM >>>>> >> >Subject: Re: hbase 0.94.0 >>>>> >> > >>>>> >> > >>>>> >> >On Fri, Jan 27, 2012 at 8:42 AM, Stack >>> <st...@duboce.net> wrote: >>>>> >> >> In this particular case, there was no explicit >>> migration step needed >>>>> >> >> going 0.90 to 0.92. Upgrading from 0.90 to 0.94 >>> should just be >>>>> >> >> running the 0.92 to 0.94 migration script (if there >>> is one). >>>>> >> >> >>>>> >> > >>>>> >> >Thinking more, the above only really holds if we keep the >>> .META. >>>>> >> >migration code that runs in 0.92 on startup on into 0.94 >>> (I have a >>>>> >> >patch here where I'm removing it... I should put this >>> bit of code >>>>> >> >back). >>>>> >> > >>>>> >> >I see Todd that you vote against removing hfile v1 in >>> 0.94. To make >>>>> >> >going from CDH3 to CDH4, we should not purge any migrating >>> code or old >>>>> >> >version support. >>>>> >> > >>>>> >> >I'd be fine w/ that. We'll need to work hard to >>> ensure it taking it >>>>> >> >on as a principal for 0.94. Ok w/ you "Iron >>> Hand" Lars? >>>>> >> > >>>>> >> >St.Ack >>>>> >> > >>>>> >> > >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> // Jonathan Hsieh (shay) >>>>> >> // Software Engineer, Cloudera >>>>> >> >>>>> >> // j...@cloudera.com >>>>> >> >>>>> > >>>>> >>>>> >>>>> -- >>>>> // Jonathan Hsieh (shay) >>>>> // Software Engineer, Cloudera >>>>> // j...@cloudera.com >>>>> >>> >>> >>> >>> -- >>> Todd Lipcon >>> Software Engineer, Cloudera >>> >> > > >
