+1 On Tue, Apr 16, 2013 at 8:36 PM, Denis Gervalle <[email protected]> wrote: > Hi devs, > > I have filed today a blocker issue: http://jira.xwiki.org/browse/XWIKI-9041 > This serious issue is the result of our current schema migration process. > > Currently, managing schema changes is done at two level: > > 1) A automatic script is computed by hibernate, comparing the current DB > schema, and the targeted schema extracted from mapping files. > 2) DataMigration may change schema using liquibase (git like solution for > managing schema changes) > > According to the Hibernate team, 1) should only be used for development > purpose. The Hibernate team clearly discourage usage of 1) in production, > since it does not cover every use cases. For example, it does not: > - drop column, table, etc... > - change datatype > - add constrainst > ... > However, this was the first, and until 4.x, the only way we had to manage > our database schema. This is why we have introduced 2) in 4.0, to be able > to change more information in our schema. Since 4.3, 2) was even be > splitted in two steps, one before and one after step 1) for partially > solving similar issue to http://jira.xwiki.org/browse/XWIKI-9041. > > While 1) is an uncontrollable monolithic step, 2) is like the underlying > data migration, a chronologic process, that apply change in order. These > two methods does not really work well join together, since parts of step 1) > may need some part of step 2) to be done, and the reverse. > > For example, migration of IDs from 32bits to 64bits bases its schema > changes on the hibernate mapping, and need the schema to be updated by > hibernate first. A new table like xwikistringlists introduced in 5.0M2 > implies the creation of new constraints against existing IDs, and therefore > need the change of datatype to be processed first. There is no way out of > that situation, and this is the cause of > http://jira.xwiki.org/browse/XWIKI-9041. > > Since we do not do a lot of schema updates during a release cycle, the > current situation has been manageable during the 4.x cycle. But, to avoid > XWIKI-9041, it implies that user must not skip a major release during > migration. This is a possible option, but it may implies some refactoring > of existing migration inside a given release cycle, and probably some > limitations as well. > > After brainstorming about the situation with Thomas, we have concluded that > it would be better to get rid of step 1), to control precisely the > evolution of the schema. However, we do not want to cause a breakage for > existing users. So here what we proposed: > > A) For any existing database having a version lower than 50000, apply the > automatic update of schema using hibernate, but based on the mapping file > of the latest 4.5.x release, and followed by B) > B) Any changes in mapping file done in 5.x or later (implying schema > changes) should be backed by a corresponding DataMigration that provide > Liquibase changelogs to reflect those changes on the database. > C) Newly created database are setup using the latest mapping file by > hibernate (since we manage versioning of our database, old liquibase change > logs will never be applied to these new db) > D) Advanced user will be allowed to also provide liquibase file, to manage > their schema updated properly (existing hidden feature), but... > E) The automatic hibernate schema update script is computed after the above > steps to check that it is empty, and that the DB schema is in sync with the > mapping file. Optionally we may allow this step to apply more schema > updates for advanced users (at their own risk) that do not provide the > liquibase change log required. (keeping the existing internal custom > mapping feature, as well as the logic of our dynamic mapping). > > The pros: > - New database will no more be updated blindly, and liquibase will provide > a complete history of changes (except advanced user changes and dynamic > mapping) > - Existing database will start to have a complete history of schema > changes starting from 5.x (and a somewhat partial history for 4.x versions) > - Migration test will only be needed from pre-5.x to 5.x version, future > migration will be more isolated, and can be tested more locally between the > release introducing the migration and the previous one > - We optionally keep existing functionalities (step E) > > The cons: > - More work to be done while changing mapping file > - Advanced user may need also to be more careful about their change in > .hbm file > - We keep using our versioning solution (we do not really use the > liquibase way), maybe a advantage as well > - We need some time to put that in place and we have already a slipping RC1 > > However, the only alternative I see to fix > http://jira.xwiki.org/browse/XWIKI-9041 would be to revert the schema > changes until the above, or a similar solution is available. > > The bad news now. Once again, I have not planned to work on this, and my > planning is already really busy. It will also require to > be thoroughly tested, and Sorin will not say the opposite, migration test > is time consuming. So, this will not improve our schedule to release 5.0 > and I will need serious support from you guys to put that in place. > > WDTH ? > > -- > Denis Gervalle > SOFTEC sa - CEO > eGuilde sarl - CTO > _______________________________________________ > devs mailing list > [email protected] > http://lists.xwiki.org/mailman/listinfo/devs
-- Thomas Mortagne _______________________________________________ devs mailing list [email protected] http://lists.xwiki.org/mailman/listinfo/devs
