#25530: Deferred foreign keys operations fail when the column is changed during
the
same migration
------------------------------+--------------------------------------
Reporter: simonphilips | Owner: nobody
Type: Bug | Status: new
Component: Migrations | Version: 1.8
Severity: Normal | Resolution:
Keywords: | Triage Stage: Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
------------------------------+--------------------------------------
Description changed by simonphilips:
Old description:
> Possibly related to #25521
>
> The following migration is the result of squashing 3 smaller migrations
> together:
> 1. the initial migration
> 2. some random RunPython migration
> 3. changing the foreign key's column migration
>
> The second step is only necessary to prevent squashmigrations from being
> smart which avoids the issue.
>
> The squashed migration (both optimized and unoptimized) looks as
> following:
> {{{#!python
> # -*- coding: utf-8 -*-
> from __future__ import unicode_literals
>
> from django.db import migrations, models
>
> # bug.migrations.0002_run_python
>
> def do_something(apps, schema_editor):
> pass # Do nothing, it's not important anyway.
>
> class Migration(migrations.Migration):
>
> replaces = [('bug', '0001_initial'), ('bug', '0002_run_python'),
> ('bug', '0003_change_db_column')]
>
> dependencies = [
> ]
>
> operations = [
> migrations.CreateModel(
> name='Child',
> fields=[
> ('id', models.AutoField(auto_created=True,
> serialize=False, verbose_name='ID', primary_key=True)),
> ('data', models.IntegerField(default=0)),
> ],
> ),
> migrations.CreateModel(
> name='Parent',
> fields=[
> ('id', models.AutoField(auto_created=True,
> serialize=False, verbose_name='ID', primary_key=True)),
> ('data', models.IntegerField(default=0)),
> ],
> ),
> migrations.AddField(
> model_name='child',
> name='parent',
> field=models.ForeignKey(db_column='my_parent',
> to='bug.Parent'),
> ),
> migrations.RunPython(
> code=do_something,
> ),
> migrations.AlterField(
> model_name='child',
> name='parent',
> field=models.ForeignKey(to='bug.Parent'),
> ),
> ]
> }}}
>
> The models are:
> {{{#!python
> class Parent(models.Model):
> data = models.IntegerField(default=0)
>
> class Child(models.Model):
> # Original db_column:
> # parent = models.ForeignKey(Parent, db_column="my_parent")
>
> # New db_column:
> parent = models.ForeignKey(Parent)
> data = models.IntegerField(default=0)
> }}}
>
> This will result in the following PostgreSQL error:
> {{{
> 2015-10-08 15:00:27 CEST ERROR: column "my_parent" does not exist
> 2015-10-08 15:00:27 CEST STATEMENT: CREATE INDEX "bug_child_1e28668d" ON
> "bug_child" ("my_parent")
> }}}
>
> The reason seems to be that any foreign key operations are added in SQL
> format to the `schema_editor.deferred_sql` list and are then executed
> after all other SQL commands. However, the third operation already
> renamed the column for that index to 'parent_id'.
>
> The deferred statements should follow any changes made during further
> operations. Or, if that's not possible, perhaps some sort of 'insert
> deferred statements now' operation could be added after each
> 'submigration' in the squashed migration?
>
> Note that in this sample app it already crashes at the `CREATE INDEX`
> statement, but the statement after that is `ALTER TABLE "bug_child" ADD
> CONSTRAINT "bug_child_my_parent_3b5bc08e1603165f_fk_bug_parent_id"
> FOREIGN KEY ("my_parent") REFERENCES "bug_parent" ("id") DEFERRABLE
> INITIALLY DEFERRED`. I think the same issue might occur if
> `bug_parent(id)` would be renamed or removed.
New description:
Possibly related to #25521
The following migration is the result of squashing 3 smaller migrations
together:
1. the initial migration
2. some random RunPython migration
3. changing the foreign key's column migration
The second step is only necessary to prevent squashmigrations from being
smart which avoids the issue.
The squashed migration (both optimized and unoptimized) looks as
following:
{{{#!python
# -*- coding: utf-8 -*-
from __future__ import unicode_literals
from django.db import migrations, models
# bug.migrations.0002_run_python
def do_something(apps, schema_editor):
pass # Do nothing, it's not important anyway.
class Migration(migrations.Migration):
replaces = [('bug', '0001_initial'), ('bug', '0002_run_python'),
('bug', '0003_change_db_column')]
dependencies = [
]
operations = [
migrations.CreateModel(
name='Child',
fields=[
('id', models.AutoField(auto_created=True,
serialize=False, verbose_name='ID', primary_key=True)),
('data', models.IntegerField(default=0)),
],
),
migrations.CreateModel(
name='Parent',
fields=[
('id', models.AutoField(auto_created=True,
serialize=False, verbose_name='ID', primary_key=True)),
('data', models.IntegerField(default=0)),
],
),
migrations.AddField(
model_name='child',
name='parent',
field=models.ForeignKey(db_column='my_parent',
to='bug.Parent'),
),
migrations.RunPython(
code=do_something,
),
migrations.AlterField(
model_name='child',
name='parent',
field=models.ForeignKey(to='bug.Parent'),
),
]
}}}
The models are:
{{{#!python
class Parent(models.Model):
data = models.IntegerField(default=0)
class Child(models.Model):
# Original db_column:
# parent = models.ForeignKey(Parent, db_column="my_parent")
# New db_column:
parent = models.ForeignKey(Parent)
data = models.IntegerField(default=0)
}}}
This will result in the following PostgreSQL error:
{{{
2015-10-08 15:00:27 CEST ERROR: column "my_parent" does not exist
2015-10-08 15:00:27 CEST STATEMENT: CREATE INDEX "bug_child_1e28668d" ON
"bug_child" ("my_parent")
}}}
The reason seems to be that any foreign key operations are added in SQL
format to the `schema_editor.deferred_sql` list and are then executed
after all other SQL commands. However, the third operation already renamed
the column for that index to 'parent_id'.
The deferred statements should follow any changes made during further
operations. Or, if that's not possible, perhaps some sort of 'insert
deferred statements now' operation could be added after each
'submigration' in the squashed migration? In fact, that may even be
desired, since the RunPython submigration may depend on the foreign key's
behaviour/index for speed.
Note that in this sample app it already crashes at the `CREATE INDEX`
statement, but the statement after that is `ALTER TABLE "bug_child" ADD
CONSTRAINT "bug_child_my_parent_3b5bc08e1603165f_fk_bug_parent_id" FOREIGN
KEY ("my_parent") REFERENCES "bug_parent" ("id") DEFERRABLE INITIALLY
DEFERRED`. I think the same issue might occur if `bug_parent(id)` would be
renamed or removed.
--
--
Ticket URL: <https://code.djangoproject.com/ticket/25530#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/070.781bb405d80b20238726158aee140ce8%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
