Hi Jasper, --execute is fine, and it's what rb-site runs. It's the combination of that and --hint that gets you into trouble. I'll explain what's going on.
Since we need to make changes to the schema for the database every so often, and since we support a variety of databases, we need to use a tool to let us record the changes we need to apply at a high level and manage applying those when it's time to upgrade. This tool is called django-evolution, and it's something we maintain these days. In order to be able to sanely apply changes, every single change ever must be recorded as part of the product, so that they can be scanned and matched up during the upgrade process. These are stored in "evolution files" (little bits of Python). Once an upgrade is performed, new state is recorded in a couple tables in the database containing the entire list of evolutions applied and the resulting signature of the database. In your case, at some point somebody modified the Review Board code and added new fields to some of our tables. Maybe evolution files were added, or maybe --hint --execute was used. Either way, not being part of the product meant that future upgrades had no way of factoring in these fields. Django-evolution knows it once existed but no longer does, and doesn't know what to do about this. It has no instructions on why those fields are gone or how to remove them. Every time you run --hint --execute, you're telling it "Ignore all the carefully-specified rules that were written to upgrade and migrate data, and just try to figure it out yourself." It'll do this by diffing the last stored schema and the schema made up of all the database models and fields in the product. This schema will be incomplete. We have updates that carefully set defaults for new fields on migrated data, for example. You probably don't have these applied now. That could lead to some hard-to-diagnose bug in the future. That's why, ideally, you'd temporarily add those fields back. The upgrade process would see that they're still around and wouldn't worry about them, allowing it to focus on our carefully crafted upgrade rules. Once applied, you could then brute-force just their removal with --hint --execute. Of course, if you've already ran this, it's too late to do that. It might be fine, but heads up that it might lead to future unexpected behavior. There's nothing available that can figure out whether you have the exact right schema now, and that data was sanely migrated. You'd have to examine the difference between your database schema and that of a new system, and examine all the evolution files that were applied, hand-applying any missing changes carefully to your database. Database repair operations like this are something we offer under a 1 year Premium Support contract. Christian On Friday, January 15, 2016, Jasper Chow <jasperc...@gmail.com> wrote: > Thanks for the feedback Christian > > I actually managed to get further by even smaller increments. 1.7.1 -> > 1.7.2 ->1.7.28 and finally 2.0 > The process is slow and i have to resolve things like Djblet needs to be > updated, importlib missing etc > > so it seems to be working so far - why is --execute a last resort? what > does it really do? > Unfortunately I have to do a --hint --execute EVERYTIME before a rb-site > upgrade. as rb-site upgrade gives out different conflicts every time - i > thought as long as i can upgrade once, then subsequent should be good. - ( > since i am installing using egg from > http://downloads.reviewboard.org/releases/ReviewBoard/ ) > > I looked into our cloned repository and i don't think there is any model > change, It seems all the changes were cosmetic. Is there a documentation / > certain step that i can perform to check the discrepancy systematically.? > > > > > > On Friday, January 15, 2016 at 5:12:04 AM UTC-5, Christian Hammond wrote >> >> Hi Jasper, >> >> This definitely looks to be due to custom model changes made by someone >> there. You'll need to do some special stuff to disentangle these. You >> definitely don't want to use --hint --execute is a recipe for disaster (got >> a backup?). >> >> What you're going to have to do is take those old model changes from your >> custom version of Review Board and port them back to the new version. Then >> try to do an upgrade. If that works properly, you can then remove those >> fields again, and *then* run --hint --execute, to get the database >> evolution history into a sane state. >> >> Of course, if you have stuff in those columns that you need still, they >> will be lost. We have an extra_data field on all these objects, which is >> where custom state should be stored, so you may want to figure out whether >> you want to migrate that and update whatever it is that was using these >> fields. >> >> Christian >> >> -- >> Christian Hammond - chri...@beanbaginc.com >> Review Board - https://www.reviewboard.org >> Beanbag, Inc. - https://www.beanbaginc.com >> >> On Wed, Jan 13, 2016 at 1:47 PM, Jasper Chow <jaspe...@gmail.com> wrote: >> >>> Update: >>> >>> tried rb-site manage /path/to/site evolve -- --hint --execute >>> but ends with Duplicate column error (password , or permission) >>> >>> >>> hmm. further research it may not be due to our customization , but >>> another case that the database somehow is in a bad state. >>> I recall reading from some posts then manual manipulation of database >>> schema is not recommended. >>> >>> >>> >>> >>> On Wednesday, January 13, 2016 at 1:06:43 PM UTC-5, Jasper Chow wrote: >>>> >>>> >>>> Attach evolve log ( rb-site manage /var/www/reviewboard evolve -- >>>> --hint ) for diagnosis >>>> >>>> thanks >>>> >>>> >>>> On Tuesday, January 12, 2016 at 12:58:01 PM UTC-5, Jasper Chow wrote: >>>>> >>>>> Hello >>>>> >>>>> I am having issue similar to previous reported, while running rb-site >>>>> upgrade >>>>> >>>>> https://hellosplat.com/s/beanbag/tickets/3967/ >>>>> https://www.mail-archive.com/reviewboard@googlegroups.com/msg15947.html >>>>> >>>>> >>>>> information: >>>>> on Linux (CentOS) reviewboard 1.7 -> 2.0.16 ( intended to upgrade to >>>>> latest but ran into some problem, thought I would do it with a smaller >>>>> increment ) >>>>> >>>>> >>>>> easy_install ReviewBoard==2.0.16 (successful) >>>>> rb-site upgrade [path] fails with following error: >>>>> >>>>> >>>>> >>>>> In model reviews.ReviewRequest: >>>>> Field 'related_review_number' has been deleted >>>>> In model reviews.Review >>>>> Field 'notify_only_submitter' has been deleted >>>>> In model reviews.ReviewRequestDraft >>>>> Field 'related_review_number' has been deleted >>>>> In model accounts.Profile >>>>> Field 'eclipse_diff_view' has been deleted >>>>> >>>>> >>>>> Judging by the name of the columns it seems some customization done by >>>>> previous admin here. So question is >>>>> What is the best practice to migrate these extension columns? >>>>> >>>>> >>>>> I tried >>>>> - drop the columns from the database ( and to restore them later ) , >>>>> and run upgrade again >>>>> - or used the trick provided in another post >>>>> >>>>> >>> from django_evolution.models import Version >>>>> >>> v = Version.objects.all()[0] >>>>> >>> print v >>>>> Hinted version, updated on 2015-02-12 02:23:19+00:00 >>>>> >>> v.delete() >>>>> >>> >>>>> >>>>> but it doesn't seems to help either >>>>> >>>>> Or maybe it should be other way around that I need to modify the >>>>> schema somewhere before i run the upgrade script? >>>>> >>>>> any help / suggestion is appreciated.. thank you >>>>> >>>>> >>>>> -- >>> Supercharge your Review Board with Power Pack: >>> https://www.reviewboard.org/powerpack/ >>> Want us to host Review Board for you? Check out RBCommons: >>> https://rbcommons.com/ >>> Happy user? Let us know! https://www.reviewboard.org/users/ >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "reviewboard" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to reviewboard...@googlegroups.com. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > Supercharge your Review Board with Power Pack: > https://www.reviewboard.org/powerpack/ > Want us to host Review Board for you? Check out RBCommons: > https://rbcommons.com/ > Happy user? Let us know! https://www.reviewboard.org/users/ > --- > You received this message because you are subscribed to the Google Groups > "reviewboard" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to reviewboard+unsubscr...@googlegroups.com > <javascript:_e(%7B%7D,'cvml','reviewboard%2bunsubscr...@googlegroups.com');> > . > For more options, visit https://groups.google.com/d/optout. > -- -- Christian Hammond - christ...@beanbaginc.com Review Board - https://www.reviewboard.org Beanbag, Inc. - https://www.beanbaginc.com -- Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/ Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/ Happy user? Let us know! https://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups "reviewboard" group. To unsubscribe from this group and stop receiving emails from it, send an email to reviewboard+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
