Yeah, it could be scripted, of course, to re-create the environment. It's just a complex setup and I haven't spent time writing any automation scripts to re-create state due to focusing on features I'm building. Hopefully someday I'll get around to that.
On Thu, Mar 6, 2014 at 6:15 AM, Daan Hoogland <daan.hoogl...@gmail.com>wrote: > Ah, yes and it is an answer to Mike's initial question indeed. It is > not sufficient for what Miguel describes however. > > On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira > <mferre...@schubergphilis.com> wrote: > > Hi all, > > > > I've been looking at the way DB related code (SQL scripts and Java > classes) are updated and what is the impact of those updates on a live > cloudstack DB. > > By the way, my intention is to support a per-commit DB upgrade of a > running system. > > On Thu, Mar 6, 2014 at 2:02 PM, Donal Lafferty > <donal.laffe...@citrix.com> wrote: > > The thread started with a discussion of problems with out of date > databases. Mike pulled from master, rebuilt, and found out his testbed's > database was out of date and no longer works. > > > > We've tried to find a database solution, but out of databases are caused > by more than schema changes. The data in the tables and other pieces of > CloudStack can change after a commit, especially towards the end of release > when everyone's feature is being merged. > > > > With 4.2, I dealt with this issue by writing a script that rebuilt my > testbed by replaying the API calls I made to set it up in the first place. > > > > Is this a solution you've considered for your dev environment? > > > > DL > > > >> -----Original Message----- > >> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > >> Sent: 06 March 2014 12:50 > >> To: dev > >> Cc: Daan Hoogland > >> Subject: Re: [DISCUSS] Checking in code that will break others' > environments > >> > >> No, but how do you mean with respect to this thread? > >> > >> On Thu, Mar 6, 2014 at 1:48 PM, Donal Lafferty < > donal.laffe...@citrix.com> > >> wrote: > >> > Hi Daan, > >> > > >> > Is there anything stopping you from scripting the configuration of > >> > your CloudStack testbed? E.g. with Marvin or CloudMonkey > >> > > >> > DL > >> > > >> >> -----Original Message----- > >> >> From: Daan Hoogland [mailto:daan.hoogl...@gmail.com] > >> >> Sent: 06 March 2014 09:53 > >> >> To: dev > >> >> Cc: Daan Hoogland > >> >> Subject: Re: [DISCUSS] Checking in code that will break others' > >> >> environments > >> >> > >> >> I totally agree with the incremental approach. I am a fascist at time > >> >> because i would even want people to add downgrade scripts to any db > >> >> change they make. Having them not adjust their sql is a good first > step, > >> though. > >> >> > >> >> On Thu, Mar 6, 2014 at 10:43 AM, Miguel Ferreira > >> >> <mferre...@schubergphilis.com> wrote: > >> >> > Hi all, > >> >> > > >> >> > I've been looking at the way DB related code (SQL scripts and Java > >> >> > classes) > >> >> are updated and what is the impact of those updates on a live > cloudstack > >> DB. > >> >> > By the way, my intention is to support a per-commit DB upgrade of a > >> >> running system. > >> >> > > >> >> > Anyway, I completely agree that sending an email warning people > >> >> > about > >> >> changes that can potentially break running environments is a good > >> >> thing, and can save a lot of time and headaches. > >> >> > I also like the idea of Rajani of using tools to help in the DB > >> >> > migration > >> >> process. > >> >> > However, I would like to put forward another idea: what about > >> >> > making sure > >> >> that the changes to the DB (schema and data) are always incremental? > >> >> > > >> >> > I mean, once a SQL statement (say A) is added to a script it could > >> >> > be kept as > >> >> is, and subsequent modifications cloud be made via new statements > >> >> (say B), instead of adapting A. > >> >> > This would allow people to upgrade their databases to the point > >> >> > they ran > >> >> statement A, and afterwards upgrade it again to the point they ran > >> >> statement B. > >> >> > The same principle could also be applied to the Java classes that > >> >> > do data > >> >> migration, but maybe there it might be a bit more involved. > >> >> > > >> >> > Cheers, > >> >> > Miguel > >> >> > > >> >> > -----Original Message----- > >> >> > From: Koushik Das [mailto:koushik....@citrix.com] > >> >> > Sent: donderdag 6 maart 2014 8:25 > >> >> > To: <dev@cloudstack.apache.org> > >> >> > Subject: Re: [DISCUSS] Checking in code that will break others' > >> >> > environments > >> >> > > >> >> > Before doing a git pull, I generally check the sql schema changes > >> >> > and run > >> >> the delta manually on my existing setup. In most of the cases that > >> >> works for me without having to redeploy the db. > >> >> > > >> >> > -Koushik > >> >> > > >> >> > On 06-Mar-2014, at 11:43 AM, Mike Tutkowski > >> >> <mike.tutkow...@solidfire.com> wrote: > >> >> > > >> >> >> Yeah, I definitely just meant a "heads up" during development if > >> >> >> you are going to change something that will break other people's > >> >> >> environments who update. If these people know in advance, they may > >> >> >> choose to postpone an update until they are at a better point. > >> >> >> > >> >> >> > >> >> >> On Wed, Mar 5, 2014 at 11:01 PM, Rajani Karuturi > >> >> >> <rajani.karut...@citrix.com > >> >> >>> wrote: > >> >> >> > >> >> >>> Across versions db migration is taken care. I think this is bound > >> >> >>> to occur while working on a release, if multiple people work on > >> >> >>> the same branch with different work-in-progress features. > >> >> >>> > >> >> >>> Could we move to flyway or liquibase which can take care of db > >> >> >>> versioning and migration? > >> >> >>> > >> >> >>> > >> >> >>> ~Rajani > >> >> >>> > >> >> >>> > >> >> >>> > >> >> >>> On 06-Mar-2014, at 2:08 am, Mike Tutkowski > >> >> >>> <mike.tutkow...@solidfire.com> > >> >> >>> wrote: > >> >> >>> > >> >> >>>> Yeah, in this case, I'm not referring to erroneous code that > >> >> >>>> breaks a person's environment (since hopefully the person > >> >> >>>> wouldn't have knowingly checked in such code), but rather, say, > >> >> >>>> DB-type changes that improve the system, but break current > >> setups. > >> >> >>>> > >> >> >>>> Just a heads-up e-mail with some easily identifiable tag. > >> >> >>>> > >> >> >>>> Can anyone think of a good tag for this? It's not always DB > >> >> >>>> related, so > >> >> >>> we > >> >> >>>> might want the tag to be more general. > >> >> >>>> > >> >> >>>> > >> >> >>>> On Wed, Mar 5, 2014 at 1:28 PM, Ian Duffy <i...@ianduffy.ie> > >> wrote: > >> >> >>>> > >> >> >>>>> +1 to this. > >> >> >>>>> > >> >> >>>>> Having the build suddenly break due to a git pull has been very > >> >> >>> annoying! > >> >> >>>>> I usually end up searching through the commit log and doing a > >> >> >>>>> resets until I find a commit where it works. Then waiting > >> >> >>>>> awhile until I do a git pull again and hoping the code was > fixed. > >> >> >>>>> > >> >> >>>>> On 5 March 2014 20:19, Mike Tutkowski > >> >> >>>>> <mike.tutkow...@solidfire.com> > >> >> >>>>> wrote: > >> >> >>>>>> Hi, > >> >> >>>>>> > >> >> >>>>>> I encountered a bit of a problem this morning and thought I > >> >> >>>>>> would bring > >> >> >>>>> it > >> >> >>>>>> up for discussion. > >> >> >>>>>> > >> >> >>>>>> If we already have a policy around this, please let me know. > >> >> >>>>>> > >> >> >>>>>> So, I fetched the latest and rebased my local 4.4 development > >> >> >>>>>> branch on > >> >> >>>>> top > >> >> >>>>>> of master. This all went just fine. > >> >> >>>>>> > >> >> >>>>>> When I rebuilt and re-started the CS Management Server, I soon > >> >> >>> realized I > >> >> >>>>>> could no longer log in from the GUI. > >> >> >>>>>> > >> >> >>>>>> As it turns out, the DB schema had been updated and so my > >> >> >>>>>> database was > >> >> >>>>> out > >> >> >>>>>> of date. The code was querying for fields that didn't exist > in my > >> DB. > >> >> >>>>>> > >> >> >>>>>> As far as I know, the easiest way to get around this is to > >> >> >>>>>> destroy my current cloud, run the script to re-build my > >> >> >>>>>> database, then re-create > >> >> >>> my > >> >> >>>>>> cloud, which is somewhat time consuming. > >> >> >>>>>> > >> >> >>>>>> Do we have a process in place currently in which we ask those > >> >> >>>>>> who make > >> >> >>>>> such > >> >> >>>>>> changes to send out a notification e-mail to dev@ to give > >> >> >>>>>> people a > >> >> >>>>> heads up > >> >> >>>>>> that updating will lead to such issues? On previous projects, > >> >> >>>>>> we would > >> >> >>>>> send > >> >> >>>>>> out an e-mail and then people could be aware to only update if > >> >> >>>>>> they > >> >> >>> were > >> >> >>>>>> prepared for such re-work. > >> >> >>>>>> > >> >> >>>>>> To be clear here, I'm not meaning to pick on anyone in > >> >> >>> particular...this > >> >> >>>>>> has happened several times over the course of my CloudStack > >> >> >>>>>> development > >> >> >>>>> and > >> >> >>>>>> I expect that I, too, have checked in such code (without > >> >> >>>>>> sending out a relevant e-mail) that lead people to have to > >> >> >>>>>> perform such a complete re-build action un-expectedly. > >> >> >>>>>> > >> >> >>>>>> What do people think about this? Maybe we should just add an > >> >> >>>>>> e-mail tag > >> >> >>>>> or > >> >> >>>>>> something and point people to the relevant commit? > >> >> >>>>>> > >> >> >>>>>> Thanks! > >> >> >>>>>> > >> >> >>>>>> -- > >> >> >>>>>> *Mike Tutkowski* > >> >> >>>>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >> >>>>>> e: mike.tutkow...@solidfire.com > >> >> >>>>>> o: 303.746.7302 > >> >> >>>>>> Advancing the way the world uses the > >> >> >>>>>> cloud<http://solidfire.com/solution/overview/?video=play> > >> >> >>>>>> *(tm)* > >> >> >>>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> > >> >> >>>> -- > >> >> >>>> *Mike Tutkowski* > >> >> >>>> *Senior CloudStack Developer, SolidFire Inc.* > >> >> >>>> e: mike.tutkow...@solidfire.com > >> >> >>>> o: 303.746.7302 > >> >> >>>> Advancing the way the world uses the > >> >> >>>> cloud<http://solidfire.com/solution/overview/?video=play> > >> >> >>>> *(tm)* > >> >> >>> > >> >> >>> > >> >> >> > >> >> >> > >> >> >> -- > >> >> >> *Mike Tutkowski* > >> >> >> *Senior CloudStack Developer, SolidFire Inc.* > >> >> >> e: mike.tutkow...@solidfire.com > >> >> >> o: 303.746.7302 > >> >> >> Advancing the way the world uses the > >> >> >> cloud<http://solidfire.com/solution/overview/?video=play> > >> >> >> *(tm)* > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> Daan > >> > >> > >> > >> -- > >> Daan > > > > -- > Daan > -- *Mike Tutkowski* *Senior CloudStack Developer, SolidFire Inc.* e: mike.tutkow...@solidfire.com o: 303.746.7302 Advancing the way the world uses the cloud<http://solidfire.com/solution/overview/?video=play> *(tm)*
