Hi Christian 

I don't seems to be able to tell evolution recognizing my schema change. 
Can you please advise what is missing?

[Basic Info]
I have 1.7.0 installed with certain extra fields already in place 
For instance 
in accounts/evolution 
__init__.py        is customized with adding line 'eclipse_diff_view' to 
the SEQUENCE
and in 
eclipse_diff_view.py  there is a MUTATIONS    AddField ('Profile'. 
'eclipse_diff_view' , ... , ... )



What i did is trying to remove those fields in 1.7.1 and hopefully from 
that point on future upgrade wont have the evolution error ( then I will 
think of a way to port back the data to the new release, that is for later )

[Step I took]
1 .So I added in  eclipse_diff_view.py   (in MUTATION below the AddField)
DeleteField  ('Profile'. 'eclipse_diff_view')

no change in __init__.py

2. build the egg using 

python setup.py developer  then python setup.py bdist_egg

3. easy_install [the new dev egg]

4. Run rb-site manage [path] evolve -- --hint



I expect evolution will suggest that the field to be deleted. , then i will 
run the rb-site upgrade. But evolution seems to think there is no change. 


I think there may be some step missing or maybe the the steps are out of 
sequence. Please advise? thanks

Jasper






 

On Monday, January 18, 2016 at 2:19:31 PM UTC-5, Jasper Chow wrote:
>
> Thanks for the detailed explanation Christian . I have a clear picture 
> now. really appreciated. 
>
> Don't think it is a good idea to ignore the rules applied to schema and 
> just -execute all the way in every release.  As eventually I will run into 
> much bigger problem in the future. I revisited the fields mentioned in 
> --hint and able to trace down to some customization done quite a while ago. 
>
> Attempting to turn the schema back to sane without losing data. But thanks 
> for the info and help so far. 
>
> Jasper
>
>
>
>
>
> On Saturday, January 16, 2016 at 11:00:35 AM UTC-5, Christian Hammond 
> wrote:
>>
>> Hi Jasper,
>>
>> --execute is fine, and it's what rb-site runs. It's the combination of 
>> that and --hint that gets you into trouble. I'll explain what's going on.
>>
>> Since we need to make changes to the schema for the database every so 
>> often, and since we support a variety of databases, we need to use a tool 
>> to let us record the changes we need to apply at a high level and manage 
>> applying those when it's time to upgrade. This tool is called 
>> django-evolution, and it's something we maintain these days.
>>
>> In order to be able to sanely apply changes, every single change ever 
>> must be recorded as part of the product, so that they can be scanned and 
>> matched up during the upgrade process. These are stored in "evolution 
>> files" (little bits of Python). Once an upgrade is performed, new state is 
>> recorded in a couple tables in the database containing the entire list of 
>> evolutions applied and the resulting signature of the database.
>>
>> In your case, at some point somebody modified the Review Board code and 
>> added new fields to some of our tables. Maybe evolution files were added, 
>> or maybe --hint --execute was used. Either way, not being part of the 
>> product meant that future upgrades had no way of factoring in these fields. 
>> Django-evolution knows it once existed but no longer does, and doesn't know 
>> what to do about this. It has no instructions on why those fields are gone 
>> or how to remove them.
>>
>> Every time you run --hint --execute, you're telling it "Ignore all the 
>> carefully-specified rules that were written to upgrade and migrate data, 
>> and just try to figure it out yourself." It'll do this by diffing the last 
>> stored schema and the schema made up of all the database models and fields 
>> in the product.
>>
>> This schema will be incomplete. We have updates that carefully set 
>> defaults for new fields on migrated data, for example. You probably don't 
>> have these applied now. That could lead to some hard-to-diagnose bug in the 
>> future.
>>
>> That's why, ideally, you'd temporarily add those fields back. The upgrade 
>> process would see that they're still around and wouldn't worry about them, 
>> allowing it to focus on our carefully crafted upgrade rules. Once applied, 
>> you could then brute-force just their removal with --hint --execute. Of 
>> course, if you've already ran this, it's too late to do that. It might be 
>> fine, but heads up that it might lead to future unexpected behavior.
>>
>> There's nothing available that can figure out whether you have the exact 
>> right schema now, and that data was sanely migrated. You'd have to examine 
>> the difference between your database schema and that of a new system, and 
>> examine all the evolution files that were applied, hand-applying any 
>> missing changes carefully to your database. Database repair operations like 
>> this are something we offer under a 1 year Premium Support contract.
>>
>> Christian
>>
>>
>> On Friday, January 15, 2016, Jasper Chow <jaspe...@gmail.com> wrote:
>>
>>> Thanks for the feedback Christian 
>>>
>>> I actually managed to get further by even smaller increments. 1.7.1 -> 
>>> 1.7.2 ->1.7.28 and finally 2.0   
>>> The process is slow and i have to resolve things like Djblet needs to be 
>>> updated, importlib missing etc
>>>
>>> so it seems to be working so far - why is --execute a last resort? what 
>>> does it really do?
>>> Unfortunately I have to do  a --hint --execute EVERYTIME before a 
>>> rb-site upgrade.   as rb-site upgrade gives out different conflicts every 
>>> time - i thought as long as i can upgrade once, then subsequent should be 
>>> good. - ( since i am installing using egg from 
>>> http://downloads.reviewboard.org/releases/ReviewBoard/ )  
>>>
>>> I looked into our cloned repository and i don't think there is any model 
>>> change, It seems all the changes were cosmetic. Is there a documentation / 
>>> certain step that i can perform to check the discrepancy systematically.?  
>>>
>>>
>>>
>>>
>>>
>>> On Friday, January 15, 2016 at 5:12:04 AM UTC-5, Christian Hammond wrote
>>>>
>>>> Hi Jasper,
>>>>
>>>> This definitely looks to be due to custom model changes made by someone 
>>>> there. You'll need to do some special stuff to disentangle these. You 
>>>> definitely don't want to use --hint --execute is a recipe for disaster 
>>>> (got 
>>>> a backup?).
>>>>
>>>> What you're going to have to do is take those old model changes from 
>>>> your custom version of Review Board and port them back to the new version. 
>>>> Then try to do an upgrade. If that works properly, you can then remove 
>>>> those fields again, and *then* run --hint --execute, to get the database 
>>>> evolution history into a sane state.
>>>>
>>>> Of course, if you have stuff in those columns that you need still, they 
>>>> will be lost. We have an extra_data field on all these objects, which is 
>>>> where custom state should be stored, so you may want to figure out whether 
>>>> you want to migrate that and update whatever it is that was using these 
>>>> fields.
>>>>
>>>> Christian
>>>>
>>>> -- 
>>>> Christian Hammond - chri...@beanbaginc.com
>>>> Review Board - https://www.reviewboard.org
>>>> Beanbag, Inc. - https://www.beanbaginc.com
>>>>
>>>> On Wed, Jan 13, 2016 at 1:47 PM, Jasper Chow <jaspe...@gmail.com> 
>>>> wrote:
>>>>
>>>>> Update:
>>>>>
>>>>> tried rb-site manage /path/to/site evolve -- --hint --execute
>>>>> but ends with Duplicate column error (password , or permission)
>>>>>
>>>>>
>>>>> hmm. further research it may not be due to our customization , but 
>>>>> another case that the database somehow is in a bad state.  
>>>>> I recall reading from some posts then manual manipulation of database 
>>>>> schema is not recommended. 
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wednesday, January 13, 2016 at 1:06:43 PM UTC-5, Jasper Chow wrote:
>>>>>>
>>>>>>
>>>>>> Attach evolve log (  rb-site manage /var/www/reviewboard evolve -- 
>>>>>> --hint )  for diagnosis
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>>
>>>>>> On Tuesday, January 12, 2016 at 12:58:01 PM UTC-5, Jasper Chow wrote:
>>>>>>>
>>>>>>> Hello 
>>>>>>>
>>>>>>> I am having issue similar to previous reported, while running 
>>>>>>> rb-site upgrade
>>>>>>>
>>>>>>> https://hellosplat.com/s/beanbag/tickets/3967/
>>>>>>>
>>>>>>> https://www.mail-archive.com/reviewboard@googlegroups.com/msg15947.html
>>>>>>>
>>>>>>>
>>>>>>> information:
>>>>>>> on Linux (CentOS) reviewboard 1.7 -> 2.0.16 ( intended to upgrade to 
>>>>>>> latest but ran into some problem, thought I would do it with a smaller 
>>>>>>> increment )
>>>>>>>
>>>>>>>
>>>>>>> easy_install ReviewBoard==2.0.16   (successful)
>>>>>>> rb-site upgrade [path]      fails with following error:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> In model reviews.ReviewRequest: 
>>>>>>> Field 'related_review_number' has been deleted
>>>>>>> In model reviews.Review
>>>>>>> Field 'notify_only_submitter' has been deleted
>>>>>>> In model reviews.ReviewRequestDraft
>>>>>>> Field 'related_review_number' has been deleted
>>>>>>> In model accounts.Profile
>>>>>>> Field 'eclipse_diff_view' has been deleted
>>>>>>>
>>>>>>>
>>>>>>> Judging by the name of the columns it seems some customization done 
>>>>>>> by previous admin here. So question is
>>>>>>> What is the best practice to  migrate these extension columns?
>>>>>>>
>>>>>>>
>>>>>>> I tried 
>>>>>>> - drop the columns from the database ( and to restore them later ) , 
>>>>>>> and run upgrade again 
>>>>>>> - or used the trick provided in another post 
>>>>>>>
>>>>>>> >>> from django_evolution.models import Version
>>>>>>> >>> v = Version.objects.all()[0]
>>>>>>> >>> print v
>>>>>>> Hinted version, updated on 2015-02-12 02:23:19+00:00
>>>>>>> >>> v.delete()
>>>>>>> >>> 
>>>>>>>
>>>>>>>  but it doesn't seems to help either 
>>>>>>>
>>>>>>> Or maybe it should be other way around that  I need to modify the 
>>>>>>> schema somewhere before i run the upgrade script?
>>>>>>>
>>>>>>> any help / suggestion is appreciated.. thank you
>>>>>>>
>>>>>>>
>>>>>>> -- 
>>>>> Supercharge your Review Board with Power Pack: 
>>>>> https://www.reviewboard.org/powerpack/
>>>>> Want us to host Review Board for you? Check out RBCommons: 
>>>>> https://rbcommons.com/
>>>>> Happy user? Let us know! https://www.reviewboard.org/users/
>>>>> --- 
>>>>> You received this message because you are subscribed to the Google 
>>>>> Groups "reviewboard" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send 
>>>>> an email to reviewboard...@googlegroups.com.
>>>>> For more options, visit https://groups.google.com/d/optout.
>>>>>
>>>>
>>>> -- 
>>> Supercharge your Review Board with Power Pack: 
>>> https://www.reviewboard.org/powerpack/
>>> Want us to host Review Board for you? Check out RBCommons: 
>>> https://rbcommons.com/
>>> Happy user? Let us know! https://www.reviewboard.org/users/
>>> --- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "reviewboard" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to reviewboard+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>
>>
>> -- 
>> -- 
>> Christian Hammond - chri...@beanbaginc.com
>> Review Board - https://www.reviewboard.org
>> Beanbag, Inc. - https://www.beanbaginc.com
>>
>>

-- 
Supercharge your Review Board with Power Pack: 
https://www.reviewboard.org/powerpack/
Want us to host Review Board for you? Check out RBCommons: 
https://rbcommons.com/
Happy user? Let us know! https://www.reviewboard.org/users/
--- 
You received this message because you are subscribed to the Google Groups 
"reviewboard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to reviewboard+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to