We want to flatten CoverLetter and Patch into Submission for performance and simplicity reasons.
We also want to avoid some of the debacles of previous migrations where sites were offline for hours to perform the migration, or needed to drop the migration out of Python entirely and in to raw SQL. So this is a sketch or outline of an approach were we migrate diff and pull_url from Patch to Submission using live-ish migrations (not fully live, but with no data migrations in the schema changes) and then drop the CoverLetter model. The way this works is pretty straight forward and is documented in more detail in the patches - basically patches 1-3 are all minor fixups, then you bring down your instance, apply patch 4, do the schema migration, bring up your instance, perform the data migration in the background, then bring down your instance, apply patches 5 and 6, do those migrations, and then you're done. Total downtime should be very short (hopefully no more than a few minutes), depending on how long it takes to add columns to your database. Lots to do before this is a full approach - I still have to migrate all the rest of the fields, and then clean up everything that depends on a Patch model existing. But I feel like this at least demonstrates that we can do the really hard work of the migration - moving the data around - in a way that is friendly to sysadmins of large instances. Daniel Axtens (6): pyenv: also install requirements for python2 tests: make test suite pass with XML-RPC disabled parsearchive,mail: use repr() to get a human readable exception Begin live migration of diff and pull_url to Submission Migration part 2: drop old fields Burninate CoverLetter patchwork/admin.py | 3 - patchwork/api/cover.py | 16 +-- patchwork/api/embedded.py | 2 +- patchwork/api/filters.py | 13 ++- patchwork/api/patch.py | 10 +- .../commands/livemigrate-v22-v23.py | 35 ++++++ patchwork/management/commands/parsearchive.py | 6 +- patchwork/management/commands/parsemail.py | 2 +- .../0037_prepare_old_diff_pull_url.py | 35 ++++++ .../0038_submission_new_diff_pull_url.py | 25 ++++ patchwork/migrations/0039_sanity_check.py | 27 +++++ .../migrations/0040_drop_old_diff_pull_url.py | 23 ++++ patchwork/migrations/0041_drop_coverletter.py | 29 +++++ patchwork/models.py | 110 +++++++++++++++--- patchwork/parser.py | 9 +- patchwork/signals.py | 4 +- patchwork/templates/patchwork/submission.html | 10 +- patchwork/tests/api/test_cover.py | 5 +- patchwork/tests/test_about.py | 24 +++- patchwork/tests/test_series.py | 14 +-- patchwork/tests/test_xmlrpc.py | 3 + patchwork/tests/utils.py | 6 +- patchwork/views/cover.py | 25 ++-- patchwork/views/patch.py | 1 + patchwork/views/xmlrpc.py | 5 +- tools/docker/Dockerfile | 11 +- 26 files changed, 378 insertions(+), 75 deletions(-) create mode 100644 patchwork/management/commands/livemigrate-v22-v23.py create mode 100644 patchwork/migrations/0037_prepare_old_diff_pull_url.py create mode 100644 patchwork/migrations/0038_submission_new_diff_pull_url.py create mode 100644 patchwork/migrations/0039_sanity_check.py create mode 100644 patchwork/migrations/0040_drop_old_diff_pull_url.py create mode 100644 patchwork/migrations/0041_drop_coverletter.py -- 2.20.1 _______________________________________________ Patchwork mailing list Patchwork@lists.ozlabs.org https://lists.ozlabs.org/listinfo/patchwork
