That seems out of scope for this issue. Can it be done as part of the work on ACCUMULO-3006 when that gets in?
On Wed, Jul 30, 2014 at 5:55 PM, <[email protected]> wrote: > Please add a check that instance.dfs.uri, although deprecated, is > configured in the upgrade and that instance.volumes is NOT configured, see > the discussion in ACCUMULO-3006. > > -----Original Message----- > From: Mike Drob [mailto:[email protected]] > Sent: Wednesday, July 30, 2014 6:12 PM > To: Accumulo Dev List > Subject: Re: [DISCUSS] Should we support upgrading 1.4 -> 1.6 w/o going > through 1.5? > > The upgrade test looks something like: > > * Start up 1.4 (or 1.5) > * Create tables with a variety of configurations (LZO/Snappy/GZ, Bloom > Filter On/Off, Block Cache On/Off) > * Load some data into these tables, enough such that flushes and > compactions start to occur. > * Abruptly kill all of the servers. This ensures that there are WALs > around. Unfortunately, we can't have any compactions in progress at this > point, since that causes a Fate Operation, and that would prevent the > upgrade from completing. > * Upgrade the bits, start 1.6, and wait for the upgrade steps to finish. > * Check for data loss > * Trigger some compactions and wait for those to finish. In a previous > iteration, we discovered an issue where the relative path'd files and WALs > were not being properly deleted from !0. > * Check for data loss. > * Restart the tablet servers to force tablets to reload (test for orphaned > files). > * Check for data loss. > > We did not try using custom iterators. > > Mike > > > On Wed, Jul 30, 2014 at 4:58 PM, Christopher <[email protected]> wrote: > > > Do you have a list of test cases you've tried? > > (like with/without walogs, custom iterators on tables, delete markers > > for the metadata table, etc.)? > > > > If we could consolidate some of those into a comprehensive list, > > that'd be pretty useful, I'd think, to give confidence that we didn't > > miss anything in the review. It might also help focus testing for > > 1.6.1 release after you've pushed. > > > > > > -- > > Christopher L Tubbs II > > http://gravatar.com/ctubbsii > > > > > > On Wed, Jul 30, 2014 at 4:39 PM, Sean Busbey <[email protected]> > wrote: > > > > > Friendly reminder that direct upgrades from 1.4 to 1.6 is under review: > > > > > > https://reviews.apache.org/r/23413/ > > > > > > Pending any additional concerns, I'll be pushing this soon. > > > > > > > > > On Fri, Jul 11, 2014 at 2:22 PM, Sean Busbey <[email protected]> > > wrote: > > > > > > > An initial port of the 1.4 -> 1.6 upgrade code for the current > > > > 1.6.1-SNAPSHOT branch is now up: > > > > > > > > * https://issues.apache.org/jira/browse/ACCUMULO-2988 > > > > * https://reviews.apache.org/r/23413/ > > > > > > > > > > > > On Wed, Jun 25, 2014 at 2:36 PM, Christopher <[email protected]> > > > wrote: > > > > > > > >> Cool. Thanks, Sean! > > > >> > > > >> > > > >> -- > > > >> Christopher L Tubbs II > > > >> http://gravatar.com/ctubbsii > > > >> > > > >> > > > >> On Wed, Jun 25, 2014 at 3:25 PM, Sean Busbey > > > >> <[email protected]> > > > wrote: > > > >> > > > >> > I'll be creating a ticket and posting a patch this week. > > > >> > > > > >> > > > > >> > On Wed, Jun 25, 2014 at 2:21 PM, Christopher > > > >> > <[email protected]> > > > >> wrote: > > > >> > > > > >> > > So, just to revisit this conversation, it seems like there is > > > >> interest in > > > >> > > supporting this. Is there already a ticket for it and/or > > > >> > > somebody interested in doing the necessary work for 1.6.1? > > > >> > > > > > >> > > > > > >> > > -- > > > >> > > Christopher L Tubbs II > > > >> > > http://gravatar.com/ctubbsii > > > >> > > > > > >> > > > > > >> > > On Thu, Jun 19, 2014 at 11:07 AM, Mike Drob > > > >> > > <[email protected]> > > > >> wrote: > > > >> > > > > > >> > > > In a nutshell: stop 1.4, install 1.6, copy the WALs to HDFS > > > >> > > > (ACCUMULO-2770), start 1.6 > > > >> > > > > > > >> > > > Mike > > > >> > > > > > > >> > > > > > > >> > > > On Wed, Jun 18, 2014 at 5:54 PM, Drew Farris < > > > [email protected] > > > >> > > > > >> > > > wrote: > > > >> > > > > > > >> > > > > Mike, > > > >> > > > > > > > >> > > > > So works just like upgrading from 1.5? > > > >> > > > > > > > >> > > > > (After 1.4 shutdown, install 1.6 and restart?) > > > >> > > > > > > > >> > > > > That sounds entirely reasonable. > > > >> > > > > > > > >> > > > > Drew > > > >> > > > > On Jun 17, 2014 10:52 PM, "Mike Drob" > > > >> > > > > <[email protected]> > > > >> wrote: > > > >> > > > > > > > >> > > > > > We initially tried to set it up as a stand-alone > > > >> > > > > > utility but > > > >> > > eventually > > > >> > > > > > gave up. In order to properly do the upgrade, you > > concurrently > > > >> need > > > >> > > to > > > >> > > > > run > > > >> > > > > > whatever upgrade code concurrently with a tablet server > > > hosting > > > >> > > > !METADATA > > > >> > > > > > and a tablet server that can replay WALs. We ended up > > > >> duplicating a > > > >> > > lot > > > >> > > > > of > > > >> > > > > > logic already present in master before scrapping that > plan. > > An > > > >> > > > > alternative > > > >> > > > > > would have been to try to build on MAC, but that was > > > >> > > > > > also > > > >> > non-trivial > > > >> > > > to > > > >> > > > > > deploy, so we spliced the code into the existing > > > >> > > > > > upgrade > > path. > > > >> How > > > >> > do > > > >> > > > you > > > >> > > > > > feel about that, Drew? > > > >> > > > > > > > > >> > > > > > > > > >> > > > > > On Tue, Jun 17, 2014 at 8:57 PM, Drew Farris < > > > >> > [email protected]> > > > >> > > > > > wrote: > > > >> > > > > > > > > >> > > > > > > I'm +1 for a utility that would allow us to go > > > >> > > > > > > directly > > from > > > >> 1.4 > > > >> > to > > > >> > > > > 1.6. > > > >> > > > > > > > > > >> > > > > > > In terms of a general policy, I suggest we make this > > > >> > > > > > > sort > > of > > > >> > > decision > > > >> > > > > on > > > >> > > > > > a > > > >> > > > > > > case by case basis. My unreasonably self-centered > > intuition > > > >> > > suggests > > > >> > > > > that > > > >> > > > > > > there may be some folks that want to go from 1.4 to > > > >> > > > > > > 1.6 > > now > > > >> due > > > >> > to > > > >> > > a > > > >> > > > > > > relatively short 1.5 cycle. The need to jump multiple > > > versions > > > >> > like > > > >> > > > > might > > > >> > > > > > > not exist in the future. > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > On Mon, Jun 16, 2014 at 5:24 PM, Sean Busbey < > > > >> > [email protected]> > > > >> > > > > > wrote: > > > >> > > > > > > > > > >> > > > > > > > In an effort to get more users off of our now > > unsupported > > > >> 1.4 > > > >> > > > > release, > > > >> > > > > > > > should we support upgrading directly to 1.6 without > > going > > > >> > > through a > > > >> > > > > 1.5 > > > >> > > > > > > > upgrade? > > > >> > > > > > > > > > > >> > > > > > > > More directly for those on user@: would you be more > > > likely > > > >> to > > > >> > > > > upgrade > > > >> > > > > > > off > > > >> > > > > > > > of 1.4 if you could do so directly to 1.6? > > > >> > > > > > > > > > > >> > > > > > > > We have this working locally at Cloudera as a part > > > >> > > > > > > > of > > our > > > >> CDH > > > >> > > > > > integration > > > >> > > > > > > > (we shipped 1.4 and we're planning to ship 1.6 next). > > > >> > > > > > > > > > > >> > > > > > > > We can get into implementation details on a jira if > > > there's > > > >> > > > positive > > > >> > > > > > > > consensus, but the changes weren't very complicated. > > > They're > > > >> > > mostly > > > >> > > > > > > > > > > >> > > > > > > > * forward porting and consolidating some upgrade > > > >> > > > > > > > code > > > >> > > > > > > > * additions to the README for instructions > > > >> > > > > > > > > > > >> > > > > > > > Personally, I can see the both sides of the > > > >> > > > > > > > argument. On > > > the > > > >> > plus > > > >> > > > > side, > > > >> > > > > > > > anything to get more users off of 1.4 is a good thing. > > On > > > >> the > > > >> > > > > negative > > > >> > > > > > > > side, it means we have the 1.4 related upgrade code > > > sitting > > > >> in > > > >> > a > > > >> > > > > > > supported > > > >> > > > > > > > code branch longer. > > > >> > > > > > > > > > > >> > > > > > > > Thoughts? > > > >> > > > > > > > > > > >> > > > > > > > -- > > > >> > > > > > > > Sean > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > >> > > > > >> > -- > > > >> > Sean > > > >> > > > > >> > > > > > > > > > > > > > > > > -- > > > > Sean > > > > > > > > > > > > > > > > -- > > > Sean > > > > > > >
