I also tried ./reviewboard/manage.py evolve
as suggested in https://www.reviewboard.org/docs/codebase/dev/tasks/database-evolutions/ but the console also says no evolution required. On Thursday, January 21, 2016 at 11:39:11 AM UTC-5, Jasper Chow wrote: > > Hi Christian > > I don't seems to be able to tell evolution recognizing my schema change. > Can you please advise what is missing? > > [Basic Info] > I have 1.7.0 installed with certain extra fields already in place > For instance > in accounts/evolution > __init__.py is customized with adding line 'eclipse_diff_view' to > the SEQUENCE > and in > eclipse_diff_view.py there is a MUTATIONS AddField ('Profile'. > 'eclipse_diff_view' , ... , ... ) > > > > What i did is trying to remove those fields in 1.7.1 and hopefully from > that point on future upgrade wont have the evolution error ( then I will > think of a way to port back the data to the new release, that is for later ) > > [Step I took] > 1 .So I added in eclipse_diff_view.py (in MUTATION below the AddField) > DeleteField ('Profile'. 'eclipse_diff_view') > > no change in __init__.py > > 2. build the egg using > > python setup.py developer then python setup.py bdist_egg > > 3. easy_install [the new dev egg] > > 4. Run rb-site manage [path] evolve -- --hint > > > > I expect evolution will suggest that the field to be deleted. , then i > will run the rb-site upgrade. But evolution seems to think there is no > change. > > > I think there may be some step missing or maybe the the steps are out of > sequence. Please advise? thanks > > Jasper > > > > > > > > > On Monday, January 18, 2016 at 2:19:31 PM UTC-5, Jasper Chow wrote: >> >> Thanks for the detailed explanation Christian . I have a clear picture >> now. really appreciated. >> >> Don't think it is a good idea to ignore the rules applied to schema and >> just -execute all the way in every release. As eventually I will run into >> much bigger problem in the future. I revisited the fields mentioned in >> --hint and able to trace down to some customization done quite a while ago. >> >> Attempting to turn the schema back to sane without losing data. But >> thanks for the info and help so far. >> >> Jasper >> >> >> >> >> >> On Saturday, January 16, 2016 at 11:00:35 AM UTC-5, Christian Hammond >> wrote: >>> >>> Hi Jasper, >>> >>> --execute is fine, and it's what rb-site runs. It's the combination of >>> that and --hint that gets you into trouble. I'll explain what's going on. >>> >>> Since we need to make changes to the schema for the database every so >>> often, and since we support a variety of databases, we need to use a tool >>> to let us record the changes we need to apply at a high level and manage >>> applying those when it's time to upgrade. This tool is called >>> django-evolution, and it's something we maintain these days. >>> >>> In order to be able to sanely apply changes, every single change ever >>> must be recorded as part of the product, so that they can be scanned and >>> matched up during the upgrade process. These are stored in "evolution >>> files" (little bits of Python). Once an upgrade is performed, new state is >>> recorded in a couple tables in the database containing the entire list of >>> evolutions applied and the resulting signature of the database. >>> >>> In your case, at some point somebody modified the Review Board code and >>> added new fields to some of our tables. Maybe evolution files were added, >>> or maybe --hint --execute was used. Either way, not being part of the >>> product meant that future upgrades had no way of factoring in these fields. >>> Django-evolution knows it once existed but no longer does, and doesn't know >>> what to do about this. It has no instructions on why those fields are gone >>> or how to remove them. >>> >>> Every time you run --hint --execute, you're telling it "Ignore all the >>> carefully-specified rules that were written to upgrade and migrate data, >>> and just try to figure it out yourself." It'll do this by diffing the last >>> stored schema and the schema made up of all the database models and fields >>> in the product. >>> >>> This schema will be incomplete. We have updates that carefully set >>> defaults for new fields on migrated data, for example. You probably don't >>> have these applied now. That could lead to some hard-to-diagnose bug in the >>> future. >>> >>> That's why, ideally, you'd temporarily add those fields back. The >>> upgrade process would see that they're still around and wouldn't worry >>> about them, allowing it to focus on our carefully crafted upgrade rules. >>> Once applied, you could then brute-force just their removal with --hint >>> --execute. Of course, if you've already ran this, it's too late to do that. >>> It might be fine, but heads up that it might lead to future unexpected >>> behavior. >>> >>> There's nothing available that can figure out whether you have the exact >>> right schema now, and that data was sanely migrated. You'd have to examine >>> the difference between your database schema and that of a new system, and >>> examine all the evolution files that were applied, hand-applying any >>> missing changes carefully to your database. Database repair operations like >>> this are something we offer under a 1 year Premium Support contract. >>> >>> Christian >>> >>> >>> On Friday, January 15, 2016, Jasper Chow <[email protected]> wrote: >>> >>>> Thanks for the feedback Christian >>>> >>>> I actually managed to get further by even smaller increments. 1.7.1 -> >>>> 1.7.2 ->1.7.28 and finally 2.0 >>>> The process is slow and i have to resolve things like Djblet needs to >>>> be updated, importlib missing etc >>>> >>>> so it seems to be working so far - why is --execute a last resort? what >>>> does it really do? >>>> Unfortunately I have to do a --hint --execute EVERYTIME before a >>>> rb-site upgrade. as rb-site upgrade gives out different conflicts every >>>> time - i thought as long as i can upgrade once, then subsequent should be >>>> good. - ( since i am installing using egg from >>>> http://downloads.reviewboard.org/releases/ReviewBoard/ ) >>>> >>>> I looked into our cloned repository and i don't think there is any >>>> model change, It seems all the changes were cosmetic. Is there a >>>> documentation / certain step that i can perform to check the discrepancy >>>> systematically.? >>>> >>>> >>>> >>>> >>>> >>>> On Friday, January 15, 2016 at 5:12:04 AM UTC-5, Christian Hammond wrote >>>>> >>>>> Hi Jasper, >>>>> >>>>> This definitely looks to be due to custom model changes made by >>>>> someone there. You'll need to do some special stuff to disentangle these. >>>>> You definitely don't want to use --hint --execute is a recipe for >>>>> disaster >>>>> (got a backup?). >>>>> >>>>> What you're going to have to do is take those old model changes from >>>>> your custom version of Review Board and port them back to the new >>>>> version. >>>>> Then try to do an upgrade. If that works properly, you can then remove >>>>> those fields again, and *then* run --hint --execute, to get the database >>>>> evolution history into a sane state. >>>>> >>>>> Of course, if you have stuff in those columns that you need still, >>>>> they will be lost. We have an extra_data field on all these objects, >>>>> which >>>>> is where custom state should be stored, so you may want to figure out >>>>> whether you want to migrate that and update whatever it is that was using >>>>> these fields. >>>>> >>>>> Christian >>>>> >>>>> -- >>>>> Christian Hammond - [email protected] >>>>> Review Board - https://www.reviewboard.org >>>>> Beanbag, Inc. - https://www.beanbaginc.com >>>>> >>>>> On Wed, Jan 13, 2016 at 1:47 PM, Jasper Chow <[email protected]> >>>>> wrote: >>>>> >>>>>> Update: >>>>>> >>>>>> tried rb-site manage /path/to/site evolve -- --hint --execute >>>>>> but ends with Duplicate column error (password , or permission) >>>>>> >>>>>> >>>>>> hmm. further research it may not be due to our customization , but >>>>>> another case that the database somehow is in a bad state. >>>>>> I recall reading from some posts then manual manipulation of database >>>>>> schema is not recommended. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wednesday, January 13, 2016 at 1:06:43 PM UTC-5, Jasper Chow wrote: >>>>>>> >>>>>>> >>>>>>> Attach evolve log ( rb-site manage /var/www/reviewboard evolve -- >>>>>>> --hint ) for diagnosis >>>>>>> >>>>>>> thanks >>>>>>> >>>>>>> >>>>>>> On Tuesday, January 12, 2016 at 12:58:01 PM UTC-5, Jasper Chow wrote: >>>>>>>> >>>>>>>> Hello >>>>>>>> >>>>>>>> I am having issue similar to previous reported, while running >>>>>>>> rb-site upgrade >>>>>>>> >>>>>>>> https://hellosplat.com/s/beanbag/tickets/3967/ >>>>>>>> >>>>>>>> https://www.mail-archive.com/[email protected]/msg15947.html >>>>>>>> >>>>>>>> >>>>>>>> information: >>>>>>>> on Linux (CentOS) reviewboard 1.7 -> 2.0.16 ( intended to upgrade >>>>>>>> to latest but ran into some problem, thought I would do it with a >>>>>>>> smaller >>>>>>>> increment ) >>>>>>>> >>>>>>>> >>>>>>>> easy_install ReviewBoard==2.0.16 (successful) >>>>>>>> rb-site upgrade [path] fails with following error: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> In model reviews.ReviewRequest: >>>>>>>> Field 'related_review_number' has been deleted >>>>>>>> In model reviews.Review >>>>>>>> Field 'notify_only_submitter' has been deleted >>>>>>>> In model reviews.ReviewRequestDraft >>>>>>>> Field 'related_review_number' has been deleted >>>>>>>> In model accounts.Profile >>>>>>>> Field 'eclipse_diff_view' has been deleted >>>>>>>> >>>>>>>> >>>>>>>> Judging by the name of the columns it seems some customization done >>>>>>>> by previous admin here. So question is >>>>>>>> What is the best practice to migrate these extension columns? >>>>>>>> >>>>>>>> >>>>>>>> I tried >>>>>>>> - drop the columns from the database ( and to restore them later ) >>>>>>>> , and run upgrade again >>>>>>>> - or used the trick provided in another post >>>>>>>> >>>>>>>> >>> from django_evolution.models import Version >>>>>>>> >>> v = Version.objects.all()[0] >>>>>>>> >>> print v >>>>>>>> Hinted version, updated on 2015-02-12 02:23:19+00:00 >>>>>>>> >>> v.delete() >>>>>>>> >>> >>>>>>>> >>>>>>>> but it doesn't seems to help either >>>>>>>> >>>>>>>> Or maybe it should be other way around that I need to modify the >>>>>>>> schema somewhere before i run the upgrade script? >>>>>>>> >>>>>>>> any help / suggestion is appreciated.. thank you >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>> Supercharge your Review Board with Power Pack: >>>>>> https://www.reviewboard.org/powerpack/ >>>>>> Want us to host Review Board for you? Check out RBCommons: >>>>>> https://rbcommons.com/ >>>>>> Happy user? Let us know! https://www.reviewboard.org/users/ >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "reviewboard" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>> Supercharge your Review Board with Power Pack: >>>> https://www.reviewboard.org/powerpack/ >>>> Want us to host Review Board for you? Check out RBCommons: >>>> https://rbcommons.com/ >>>> Happy user? Let us know! https://www.reviewboard.org/users/ >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "reviewboard" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> -- >>> -- >>> Christian Hammond - [email protected] >>> Review Board - https://www.reviewboard.org >>> Beanbag, Inc. - https://www.beanbaginc.com >>> >>> -- Supercharge your Review Board with Power Pack: https://www.reviewboard.org/powerpack/ Want us to host Review Board for you? Check out RBCommons: https://rbcommons.com/ Happy user? Let us know! https://www.reviewboard.org/users/ --- You received this message because you are subscribed to the Google Groups "reviewboard" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
