Hi Gene, Was the database fully recreated between attempts? It seems like there may be some stale indexes sitting around.
Christian On Mon, Aug 28, 2017 at 8:36 PM, Gene Paul Quevedo <[email protected]> wrote: > Hi Christian, > > I (re) tried this procedure of creating a dump from production, creating a > database on a test server, changing the engine to InnoDB for 64 tables and, > running the rb-site upgrade. > Still, it's failing somewhere, but I'm able to capture the log message > from the upgrade failure, the first after using a pristine DB copy from the > production. > > root@test:~# rb-site upgrade /var/www/netcomm-codereviews > Rebuilding directory structure > Upgrading site settings_local.py > Updating database. This may take a while. > > The log output below, including warnings and errors, > can be ignored unless upgrade fails. > > ------------------ <begin log output> ------------------ > Creating tables ... > Creating table accounts_trophy > Creating table attachments_fileattachmenthistory > Creating table diffviewer_rawfilediffdata > Creating table notifications_webhooktarget_repositories > Creating table notifications_webhooktarget > Creating table webapi_webapitoken > Upgrading Review Board from 2.0.25 to 2.5.15 > There are unapplied evolutions for accounts. > There are unapplied evolutions for attachments. > There are unapplied evolutions for diffviewer. > There are unapplied evolutions for notifications. > There are unapplied evolutions for reviews. > There are unapplied evolutions for webapi. > Adding baseline version for new models > Evolutions in notifications baseline: webhooktarget_extra_state, > webhooktarget_extra_data_null > Project signature has changed - an evolution is required > Installing custom SQL ... > Installing indexes ... > Installed 0 object(s) from 0 fixture(s) > > This may take a while. Please be patient, and do not cancel the upgrade! > > Applying database evolutions for accounts... > Applying database evolutions for attachments... > Applying database evolutions for diffviewer... > Database evolutions for diffviewer failed! > The SQL statement was: CREATE INDEX `diffviewer_filediff_4d02a56a` ON > `diffviewer_filediff` (`raw_diff_hash_id`); > The database error was: (1061, "Duplicate key name > 'diffviewer_filediff_4d02a56a'") > > CommandError: Error applying evolution for diffviewer: (1061, "Duplicate > key name 'diffviewer_filediff_4d02a56a'") > > ------------------------------------------------------------ > ----------------------------------------------- > > By the time it failed it has created at least 5 tables. Seems like it's > hard to tell what is causing the duplicate key name on table indexes. > Attaching here are the latest schema post-upgrade failure > > but hey! thanks so much for helping me out on this one :) > > -Gene > > > On Tue, 22 Aug 2017 at 17:10 Christian Hammond <[email protected]> > wrote: > >> Hi Gene, >> >> The schema looks healthy at first glance. It seems like a normal 2.0.x >> schema, and one that would be safe to upgrade to 2.5.x. >> >> I don't know what the original error was that you hit (the one in the >> original post here was due to a subsequent attempt at upgrading a database >> that already failed in a way we couldn't recover from), but one possible >> cause of failures would be the table engine type. >> >> Old versions of MySQL used MyISAM as the table type (the ENGINE= lines in >> the schema). Newer MySQL versions (from the past several years onward) >> default to InnoDB for any new tables (and 2.5.x would introduce new >> tables). If you have a mix of table types, MySQL can end up throwing >> confusing errors when attempting to create any relations between those >> tables. It's something I'd love for us to be able to catch, but it's pretty >> much a MySQL responsibility. >> >> While I don't know if you hit an error related to that, what you might >> want to try doing is converting your schema to use InnoDB for all tables. >> You can do a full database dump from the production server, change all >> ENGINE=MyISAM entries to ENGINE=InnoDB, and then load the database into the >> test server. Then attempt an upgrade, and see if it goes through >> successfully. >> >> If it doesn't succeed, show me the full output, and I'll probably be able >> to narrow it down. >> >> Christian >> >> On Sun, Aug 20, 2017 at 6:25 PM, Gene Paul Quevedo < >> [email protected]> wrote: >> >>> Oops looks like you might be right. Anyway here's our schema direct >>> coming from our production site. >>> >>> Previously we were running 2.0.20, and the upgrade went smooth if I >>> recall. >>> >>> On Monday, 21 August 2017 10:23:03 UTC+10, Gene Paul Quevedo wrote: >>>> >>>> hi Christian >>>> >>>> The schema is from 2.0.25. It was from a snapshot of our test server >>>> before the upgrade was performed. If there's a suspicion I could generate >>>> again the schema coming from ou production server, which runs 2.0.25 >>>> >>>> On Tuesday, 15 August 2017 09:49:09 UTC+10, Christian Hammond wrote: >>>>> >>>>> Hi Gene, >>>>> >>>>> Looks like there was a partial attempt at an evolution that failed >>>>> once before. The "visibility" column was applied and then something >>>>> apparently went wrong, or upgrading was cancelled. Deleting the duplicates >>>>> and running again would lead to this, unless carefully handled in a way >>>>> compatible with django-evolution's internals. >>>>> >>>>> Is this database schema the initial one applied from 2.0.25, or what >>>>> you have on the test server after the upgrade attempts? >>>>> >>>>> Christian >>>>> >>>>> On Sun, Aug 13, 2017 at 8:40 PM, Gene Paul Quevedo < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi, >>>>>> >>>>>> I'ts my first time to post here, and I hope someone can point me to >>>>>> the right way of solving this issue, even though it seems like there were >>>>>> issues like this before >>>>>> >>>>>> I'm trying to do a DB upgrade of Review from 2.0.25 to 2.5.15 on a >>>>>> test server with a copy of our Database, and seems like it has bumped >>>>>> into >>>>>> this issue >>>>>> >>>>>> Rebuilding directory structure >>>>>> Upgrading site settings_local.py >>>>>> Updating database. This may take a while. >>>>>> >>>>>> The log output below, including warnings and errors, >>>>>> can be ignored unless upgrade fails. >>>>>> >>>>>> ------------------ <begin log output> ------------------ >>>>>> Creating tables ... >>>>>> There are unapplied evolutions for accounts. >>>>>> There are unapplied evolutions for attachments. >>>>>> There are unapplied evolutions for diffviewer. >>>>>> There are unapplied evolutions for reviews. >>>>>> There are unapplied evolutions for webapi. >>>>>> Project signature has changed - an evolution is required >>>>>> Installing custom SQL ... >>>>>> Installing indexes ... >>>>>> Installed 0 object(s) from 0 fixture(s) >>>>>> >>>>>> This may take a while. Please be patient, and do not cancel the >>>>>> upgrade! >>>>>> >>>>>> Applying database evolutions for accounts... >>>>>> Database evolutions for accounts failed! >>>>>> The SQL statement was: (u'ALTER TABLE `accounts_reviewrequestvisit` >>>>>> ADD COLUMN `visibility` varchar(1) NOT NULL DEFAULT %s;', [u'V']) >>>>>> >>>>>> >>>>>> >>>>>> *The database error was: (1060, "Duplicate column name >>>>>> 'visibility'")CommandError: Error applying evolution for accounts: (1060, >>>>>> "Duplicate column name 'visibility'")* >>>>>> There are probably 4 variations of that 'duplicate column' error, and >>>>>> the other one is a 'duplicate index' . >>>>>> My initial solution was to delete the duplicates and run the upgrade >>>>>> procedure again, but seems like the duplication seems to be repeating >>>>>> (!), >>>>>> as if Django Evolution is creating those duplicate columns/indices >>>>>> >>>>>> Any ideas on how to debug this further? I'm attaching our DB schema >>>>>> here >>>>>> >>>>>> thank you! >>>>>> - Gene >>>>>> >>>>>> -- >>>>>> Supercharge your Review Board with Power Pack: >>>>>> https://www.reviewboard.org/powerpack/ >>>>>> Want us to host Review Board for you? Check out RBCommons: >>>>>> https://rbcommons.com/ >>>>>> Happy user? Let us know! https://www.reviewboard.org/users/ >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "reviewboard" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Christian Hammond >>>>> President/CEO of Beanbag <https://www.beanbaginc.com/> >>>>> Makers of Review Board <https://www.reviewboard.org/> >>>>> >>>> -- >>> Supercharge your Review Board with Power Pack: >>> https://www.reviewboard.org/powerpack/ >>> Want us to host Review Board for you? Check out RBCommons: >>> https://rbcommons.com/ >>> Happy user? Let us know! https://www.reviewboard.org/users/ >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "reviewboard" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send an >>> email to [email protected]. >>> >> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Christian Hammond >> President/CEO of Beanbag <https://www.beanbaginc.com/> >> Makers of Review Board <https://www.reviewboard.org/> >> >> -- >> Supercharge your Review Board with Power Pack: >> https://www.reviewboard.org/powerpack/ >> Want us to host Review Board for you? Check out RBCommons: >> https://rbcommons.com/ >> Happy user? Let us know! https://www.reviewboard.org/users/ >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "reviewboard" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/ >> topic/reviewboard/NZ33huj8f8A/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> > -- > Supercharge your Review Board with Power Pack: > https://www.reviewboard.org/powerpack/ > Want us to host Review Board for you? Check out RBCommons: > https://rbcommons.com/ > Happy user? Let us know! https://www.reviewboard.org/users/ > --- > You received this message because you are subscribed to the Google Groups > "reviewboard" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- Christian Hammond President/CEO of Beanbag <https://www.beanbaginc.com/> Makers of Review Board <https://www.reviewboard.org/> -- Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/ Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/ Happy user? Let us know! https://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups "reviewboard" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
