The upgrade test looks something like: * Start up 1.4 (or 1.5) * Create tables with a variety of configurations (LZO/Snappy/GZ, Bloom Filter On/Off, Block Cache On/Off) * Load some data into these tables, enough such that flushes and compactions start to occur. * Abruptly kill all of the servers. This ensures that there are WALs around. Unfortunately, we can't have any compactions in progress at this point, since that causes a Fate Operation, and that would prevent the upgrade from completing. * Upgrade the bits, start 1.6, and wait for the upgrade steps to finish. * Check for data loss * Trigger some compactions and wait for those to finish. In a previous iteration, we discovered an issue where the relative path'd files and WALs were not being properly deleted from !0. * Check for data loss. * Restart the tablet servers to force tablets to reload (test for orphaned files). * Check for data loss.
We did not try using custom iterators. Mike On Wed, Jul 30, 2014 at 4:58 PM, Christopher <[email protected]> wrote: > Do you have a list of test cases you've tried? > (like with/without walogs, custom iterators on tables, delete markers for > the metadata table, etc.)? > > If we could consolidate some of those into a comprehensive list, that'd be > pretty useful, I'd think, to give confidence that we didn't miss anything > in the review. It might also help focus testing for 1.6.1 release after > you've pushed. > > > -- > Christopher L Tubbs II > http://gravatar.com/ctubbsii > > > On Wed, Jul 30, 2014 at 4:39 PM, Sean Busbey <[email protected]> wrote: > > > Friendly reminder that direct upgrades from 1.4 to 1.6 is under review: > > > > https://reviews.apache.org/r/23413/ > > > > Pending any additional concerns, I'll be pushing this soon. > > > > > > On Fri, Jul 11, 2014 at 2:22 PM, Sean Busbey <[email protected]> > wrote: > > > > > An initial port of the 1.4 -> 1.6 upgrade code for the current > > > 1.6.1-SNAPSHOT branch is now up: > > > > > > * https://issues.apache.org/jira/browse/ACCUMULO-2988 > > > * https://reviews.apache.org/r/23413/ > > > > > > > > > On Wed, Jun 25, 2014 at 2:36 PM, Christopher <[email protected]> > > wrote: > > > > > >> Cool. Thanks, Sean! > > >> > > >> > > >> -- > > >> Christopher L Tubbs II > > >> http://gravatar.com/ctubbsii > > >> > > >> > > >> On Wed, Jun 25, 2014 at 3:25 PM, Sean Busbey <[email protected]> > > wrote: > > >> > > >> > I'll be creating a ticket and posting a patch this week. > > >> > > > >> > > > >> > On Wed, Jun 25, 2014 at 2:21 PM, Christopher <[email protected]> > > >> wrote: > > >> > > > >> > > So, just to revisit this conversation, it seems like there is > > >> interest in > > >> > > supporting this. Is there already a ticket for it and/or somebody > > >> > > interested in doing the necessary work for 1.6.1? > > >> > > > > >> > > > > >> > > -- > > >> > > Christopher L Tubbs II > > >> > > http://gravatar.com/ctubbsii > > >> > > > > >> > > > > >> > > On Thu, Jun 19, 2014 at 11:07 AM, Mike Drob <[email protected]> > > >> wrote: > > >> > > > > >> > > > In a nutshell: stop 1.4, install 1.6, copy the WALs to HDFS > > >> > > > (ACCUMULO-2770), start 1.6 > > >> > > > > > >> > > > Mike > > >> > > > > > >> > > > > > >> > > > On Wed, Jun 18, 2014 at 5:54 PM, Drew Farris < > > [email protected] > > >> > > > >> > > > wrote: > > >> > > > > > >> > > > > Mike, > > >> > > > > > > >> > > > > So works just like upgrading from 1.5? > > >> > > > > > > >> > > > > (After 1.4 shutdown, install 1.6 and restart?) > > >> > > > > > > >> > > > > That sounds entirely reasonable. > > >> > > > > > > >> > > > > Drew > > >> > > > > On Jun 17, 2014 10:52 PM, "Mike Drob" <[email protected]> > > >> wrote: > > >> > > > > > > >> > > > > > We initially tried to set it up as a stand-alone utility but > > >> > > eventually > > >> > > > > > gave up. In order to properly do the upgrade, you > concurrently > > >> need > > >> > > to > > >> > > > > run > > >> > > > > > whatever upgrade code concurrently with a tablet server > > hosting > > >> > > > !METADATA > > >> > > > > > and a tablet server that can replay WALs. We ended up > > >> duplicating a > > >> > > lot > > >> > > > > of > > >> > > > > > logic already present in master before scrapping that plan. > An > > >> > > > > alternative > > >> > > > > > would have been to try to build on MAC, but that was also > > >> > non-trivial > > >> > > > to > > >> > > > > > deploy, so we spliced the code into the existing upgrade > path. > > >> How > > >> > do > > >> > > > you > > >> > > > > > feel about that, Drew? > > >> > > > > > > > >> > > > > > > > >> > > > > > On Tue, Jun 17, 2014 at 8:57 PM, Drew Farris < > > >> > [email protected]> > > >> > > > > > wrote: > > >> > > > > > > > >> > > > > > > I'm +1 for a utility that would allow us to go directly > from > > >> 1.4 > > >> > to > > >> > > > > 1.6. > > >> > > > > > > > > >> > > > > > > In terms of a general policy, I suggest we make this sort > of > > >> > > decision > > >> > > > > on > > >> > > > > > a > > >> > > > > > > case by case basis. My unreasonably self-centered > intuition > > >> > > suggests > > >> > > > > that > > >> > > > > > > there may be some folks that want to go from 1.4 to 1.6 > now > > >> due > > >> > to > > >> > > a > > >> > > > > > > relatively short 1.5 cycle. The need to jump multiple > > versions > > >> > like > > >> > > > > might > > >> > > > > > > not exist in the future. > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > > On Mon, Jun 16, 2014 at 5:24 PM, Sean Busbey < > > >> > [email protected]> > > >> > > > > > wrote: > > >> > > > > > > > > >> > > > > > > > In an effort to get more users off of our now > unsupported > > >> 1.4 > > >> > > > > release, > > >> > > > > > > > should we support upgrading directly to 1.6 without > going > > >> > > through a > > >> > > > > 1.5 > > >> > > > > > > > upgrade? > > >> > > > > > > > > > >> > > > > > > > More directly for those on user@: would you be more > > likely > > >> to > > >> > > > > upgrade > > >> > > > > > > off > > >> > > > > > > > of 1.4 if you could do so directly to 1.6? > > >> > > > > > > > > > >> > > > > > > > We have this working locally at Cloudera as a part of > our > > >> CDH > > >> > > > > > integration > > >> > > > > > > > (we shipped 1.4 and we're planning to ship 1.6 next). > > >> > > > > > > > > > >> > > > > > > > We can get into implementation details on a jira if > > there's > > >> > > > positive > > >> > > > > > > > consensus, but the changes weren't very complicated. > > They're > > >> > > mostly > > >> > > > > > > > > > >> > > > > > > > * forward porting and consolidating some upgrade code > > >> > > > > > > > * additions to the README for instructions > > >> > > > > > > > > > >> > > > > > > > Personally, I can see the both sides of the argument. On > > the > > >> > plus > > >> > > > > side, > > >> > > > > > > > anything to get more users off of 1.4 is a good thing. > On > > >> the > > >> > > > > negative > > >> > > > > > > > side, it means we have the 1.4 related upgrade code > > sitting > > >> in > > >> > a > > >> > > > > > > supported > > >> > > > > > > > code branch longer. > > >> > > > > > > > > > >> > > > > > > > Thoughts? > > >> > > > > > > > > > >> > > > > > > > -- > > >> > > > > > > > Sean > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > >> > > > >> > -- > > >> > Sean > > >> > > > >> > > > > > > > > > > > > -- > > > Sean > > > > > > > > > > > -- > > Sean > > >
