Hi Christian
I don't seems to be able to tell evolution recognizing my schema change.
Can you please advise what is missing?
[Basic Info]
I have 1.7.0 installed with certain extra fields already in place
For instance
in accounts/evolution
__init__.py is customized with adding line 'eclipse_diff_view' to
the SEQUENCE
and in
eclipse_diff_view.py there is a MUTATIONS AddField ('Profile'.
'eclipse_diff_view' , ... , ... )
What i did is trying to remove those fields in 1.7.1 and hopefully from
that point on future upgrade wont have the evolution error ( then I will
think of a way to port back the data to the new release, that is for later )
[Step I took]
1 .So I added in eclipse_diff_view.py (in MUTATION below the AddField)
DeleteField ('Profile'. 'eclipse_diff_view')
no change in __init__.py
2. build the egg using
python setup.py developer then python setup.py bdist_egg
3. easy_install [the new dev egg]
4. Run rb-site manage [path] evolve -- --hint
I expect evolution will suggest that the field to be deleted. , then i will
run the rb-site upgrade. But evolution seems to think there is no change.
I think there may be some step missing or maybe the the steps are out of
sequence. Please advise? thanks
Jasper
On Monday, January 18, 2016 at 2:19:31 PM UTC-5, Jasper Chow wrote:
>
> Thanks for the detailed explanation Christian . I have a clear picture
> now. really appreciated.
>
> Don't think it is a good idea to ignore the rules applied to schema and
> just -execute all the way in every release. As eventually I will run into
> much bigger problem in the future. I revisited the fields mentioned in
> --hint and able to trace down to some customization done quite a while ago.
>
> Attempting to turn the schema back to sane without losing data. But thanks
> for the info and help so far.
>
> Jasper
>
>
>
>
>
> On Saturday, January 16, 2016 at 11:00:35 AM UTC-5, Christian Hammond
> wrote:
>>
>> Hi Jasper,
>>
>> --execute is fine, and it's what rb-site runs. It's the combination of
>> that and --hint that gets you into trouble. I'll explain what's going on.
>>
>> Since we need to make changes to the schema for the database every so
>> often, and since we support a variety of databases, we need to use a tool
>> to let us record the changes we need to apply at a high level and manage
>> applying those when it's time to upgrade. This tool is called
>> django-evolution, and it's something we maintain these days.
>>
>> In order to be able to sanely apply changes, every single change ever
>> must be recorded as part of the product, so that they can be scanned and
>> matched up during the upgrade process. These are stored in "evolution
>> files" (little bits of Python). Once an upgrade is performed, new state is
>> recorded in a couple tables in the database containing the entire list of
>> evolutions applied and the resulting signature of the database.
>>
>> In your case, at some point somebody modified the Review Board code and
>> added new fields to some of our tables. Maybe evolution files were added,
>> or maybe --hint --execute was used. Either way, not being part of the
>> product meant that future upgrades had no way of factoring in these fields.
>> Django-evolution knows it once existed but no longer does, and doesn't know
>> what to do about this. It has no instructions on why those fields are gone
>> or how to remove them.
>>
>> Every time you run --hint --execute, you're telling it "Ignore all the
>> carefully-specified rules that were written to upgrade and migrate data,
>> and just try to figure it out yourself." It'll do this by diffing the last
>> stored schema and the schema made up of all the database models and fields
>> in the product.
>>
>> This schema will be incomplete. We have updates that carefully set
>> defaults for new fields on migrated data, for example. You probably don't
>> have these applied now. That could lead to some hard-to-diagnose bug in the
>> future.
>>
>> That's why, ideally, you'd temporarily add those fields back. The upgrade
>> process would see that they're still around and wouldn't worry about them,
>> allowing it to focus on our carefully crafted upgrade rules. Once applied,
>> you could then brute-force just their removal with --hint --execute. Of
>> course, if you've already ran this, it's too late to do that. It might be
>> fine, but heads up that it might lead to future unexpected behavior.
>>
>> There's nothing available that can figure out whether you have the exact
>> right schema now, and that data was sanely migrated. You'd have to examine
>> the difference between your database schema and that of a new system, and
>> examine all the evolution files that were applied, hand-applying any
>> missing changes carefully to your database. Database repair operations like
>> this are something we offer under a 1 year Premium Support contract.
>>
>> Christian
>>
>>
>> On Friday, January 15, 2016, Jasper Chow <[email protected]> wrote:
>>
>>> Thanks for the feedback Christian
>>>
>>> I actually managed to get further by even smaller increments. 1.7.1 ->
>>> 1.7.2 ->1.7.28 and finally 2.0
>>> The process is slow and i have to resolve things like Djblet needs to be
>>> updated, importlib missing etc
>>>
>>> so it seems to be working so far - why is --execute a last resort? what
>>> does it really do?
>>> Unfortunately I have to do a --hint --execute EVERYTIME before a
>>> rb-site upgrade. as rb-site upgrade gives out different conflicts every
>>> time - i thought as long as i can upgrade once, then subsequent should be
>>> good. - ( since i am installing using egg from
>>> http://downloads.reviewboard.org/releases/ReviewBoard/ )
>>>
>>> I looked into our cloned repository and i don't think there is any model
>>> change, It seems all the changes were cosmetic. Is there a documentation /
>>> certain step that i can perform to check the discrepancy systematically.?
>>>
>>>
>>>
>>>
>>>
>>> On Friday, January 15, 2016 at 5:12:04 AM UTC-5, Christian Hammond wrote
>>>>
>>>> Hi Jasper,
>>>>
>>>> This definitely looks to be due to custom model changes made by someone
>>>> there. You'll need to do some special stuff to disentangle these. You
>>>> definitely don't want to use --hint --execute is a recipe for disaster
>>>> (got
>>>> a backup?).
>>>>
>>>> What you're going to have to do is take those old model changes from
>>>> your custom version of Review Board and port them back to the new version.
>>>> Then try to do an upgrade. If that works properly, you can then remove
>>>> those fields again, and *then* run --hint --execute, to get the database
>>>> evolution history into a sane state.
>>>>
>>>> Of course, if you have stuff in those columns that you need still, they
>>>> will be lost. We have an extra_data field on all these objects, which is
>>>> where custom state should be stored, so you may want to figure out whether
>>>> you want to migrate that and update whatever it is that was using these
>>>> fields.
>>>>
>>>> Christian
>>>>
>>>> --
>>>> Christian Hammond - [email protected]
>>>> Review Board - https://www.reviewboard.org
>>>> Beanbag, Inc. - https://www.beanbaginc.com
>>>>
>>>> On Wed, Jan 13, 2016 at 1:47 PM, Jasper Chow <[email protected]>
>>>> wrote:
>>>>
>>>>> Update:
>>>>>
>>>>> tried rb-site manage /path/to/site evolve -- --hint --execute
>>>>> but ends with Duplicate column error (password , or permission)
>>>>>
>>>>>
>>>>> hmm. further research it may not be due to our customization , but
>>>>> another case that the database somehow is in a bad state.
>>>>> I recall reading from some posts then manual manipulation of database
>>>>> schema is not recommended.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wednesday, January 13, 2016 at 1:06:43 PM UTC-5, Jasper Chow wrote:
>>>>>>
>>>>>>
>>>>>> Attach evolve log ( rb-site manage /var/www/reviewboard evolve --
>>>>>> --hint ) for diagnosis
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>>
>>>>>> On Tuesday, January 12, 2016 at 12:58:01 PM UTC-5, Jasper Chow wrote:
>>>>>>>
>>>>>>> Hello
>>>>>>>
>>>>>>> I am having issue similar to previous reported, while running
>>>>>>> rb-site upgrade
>>>>>>>
>>>>>>> https://hellosplat.com/s/beanbag/tickets/3967/
>>>>>>>
>>>>>>> https://www.mail-archive.com/[email protected]/msg15947.html
>>>>>>>
>>>>>>>
>>>>>>> information:
>>>>>>> on Linux (CentOS) reviewboard 1.7 -> 2.0.16 ( intended to upgrade to
>>>>>>> latest but ran into some problem, thought I would do it with a smaller
>>>>>>> increment )
>>>>>>>
>>>>>>>
>>>>>>> easy_install ReviewBoard==2.0.16 (successful)
>>>>>>> rb-site upgrade [path] fails with following error:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> In model reviews.ReviewRequest:
>>>>>>> Field 'related_review_number' has been deleted
>>>>>>> In model reviews.Review
>>>>>>> Field 'notify_only_submitter' has been deleted
>>>>>>> In model reviews.ReviewRequestDraft
>>>>>>> Field 'related_review_number' has been deleted
>>>>>>> In model accounts.Profile
>>>>>>> Field 'eclipse_diff_view' has been deleted
>>>>>>>
>>>>>>>
>>>>>>> Judging by the name of the columns it seems some customization done
>>>>>>> by previous admin here. So question is
>>>>>>> What is the best practice to migrate these extension columns?
>>>>>>>
>>>>>>>
>>>>>>> I tried
>>>>>>> - drop the columns from the database ( and to restore them later ) ,
>>>>>>> and run upgrade again
>>>>>>> - or used the trick provided in another post
>>>>>>>
>>>>>>> >>> from django_evolution.models import Version
>>>>>>> >>> v = Version.objects.all()[0]
>>>>>>> >>> print v
>>>>>>> Hinted version, updated on 2015-02-12 02:23:19+00:00
>>>>>>> >>> v.delete()
>>>>>>> >>>
>>>>>>>
>>>>>>> but it doesn't seems to help either
>>>>>>>
>>>>>>> Or maybe it should be other way around that I need to modify the
>>>>>>> schema somewhere before i run the upgrade script?
>>>>>>>
>>>>>>> any help / suggestion is appreciated.. thank you
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>> Supercharge your Review Board with Power Pack:
>>>>> https://www.reviewboard.org/powerpack/
>>>>> Want us to host Review Board for you? Check out RBCommons:
>>>>> https://rbcommons.com/
>>>>> Happy user? Let us know! https://www.reviewboard.org/users/
>>>>> ---
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "reviewboard" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to [email protected].
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> --
>>> Supercharge your Review Board with Power Pack:
>>> https://www.reviewboard.org/powerpack/
>>> Want us to host Review Board for you? Check out RBCommons:
>>> https://rbcommons.com/
>>> Happy user? Let us know! https://www.reviewboard.org/users/
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "reviewboard" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to [email protected].
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>> --
>> --
>> Christian Hammond - [email protected]
>> Review Board - https://www.reviewboard.org
>> Beanbag, Inc. - https://www.beanbaginc.com
>>
>>
--
Supercharge your Review Board with Power Pack:
https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons:
https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
---
You received this message because you are subscribed to the Google Groups
"reviewboard" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
