Hi, I want to suggest a slightly different approach. Goose is the brain managing the DB upgrade, right? The data patches are part of the DB evolution, but today we can't use goose to run them because we have seeds.sql in the middle. What I suggest is turning seeds.sql into another migration script, resulting in the following procedure: 0) (In clean installations) Tables creation 1) Goose migrations: 1a) Schema changes 1b) Data seeding (seeds.sql) 1c) Data changes
Going forward, I believe a data change migration script will be attached to most schema changes, instead of populating the DB in the seeds.sql. The benefits come from the fact that for future changes, the order presented above (1a, 1b, 1c) is not strict. Future schema/data changes are expressed in a single migration script, containing all relevant operations. This ensures that whatever change is needed (schema/data/both), and whatever the change depends on, it can be handled by a single Goose migration script. What do you think? On Fri, Jun 9, 2017 at 6:35 PM, Dewayne Richardson <[email protected]> wrote: > Yea, it's just a new feature to admin.pl to support data conversions, to > keep the migrations clean. Derek and I have been working through it. > > -Dew > > On Thu, Jun 8, 2017 at 7:40 AM, Jeremy Mitchell <[email protected]> > wrote: > > > This seems to make sense to me but honestly, I'd probably defer to > Dewayne. > > > > In theory, it would be nice if migrations only included "structural" > > changes (new tables, columns, changing column types or not null, etc) > and > > seeds focused on the "base" (or the minimum required) static data > required > > of TO (types, statuses, roles, etc) and then yea, putting data fixing or > > data massaging as the last step makes sense to me. But you know what they > > say about theory... > > > > +1 > > > > Jeremy > > > > On Wed, Jun 7, 2017 at 8:41 AM, Gelinas, Derek < > [email protected]> > > wrote: > > > > > I'm adding a feature to traffic ops that creates a new column in > > > steering_target called type, that is populated with type ids from the > > type > > > table. Using admin.pl upgrade, the column is created in migrations, > and > > > the two types for this table are populated by seeds.sql. None of this > is > > > out of the ordinary. Unfortunately I also need to populate the type > > column > > > based on data that isn't in there until after seeds.sql is run, so I > > can't > > > place this into the migration. Seeds.sql needs to run after the > > migration > > > due to any structural changes that happen there. > > > > > > Dewayne and I have discussed this a bit this morning, and we're > thinking > > > the best solution might be a third step, run after seeds.sql, called > > > patches.sql. This would be specifically for data fixes like in this > use > > > case. The order would be as follows: > > > > > > migration - structure > > > seeds - static data > > > patches - data fixes > > > > > > Thoughts? > > > > > > Derek > > > -- *Naama Shoresh* Qwilt | Work: +972-72-2221706 | Mobile: +972-52-3401999 | [email protected]
