Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by bcail):

 * needs_better_patch:  1 => 0

Comment:

 I reverted the "IF NOT EXISTS" changes, and split out the create-new-index
 and recreate-deleted-index conditions into separate methods.
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018da881d6ba-6c7884c4-ca83-455b-a691-ab2174f71594-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak):

 * needs_better_patch:  0 => 1

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018da65e5856-475b3d58-4c02-495b-bd30-982d8d8655fa-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by bcail):

 * needs_better_patch:  1 => 0

Comment:

 Hi Mariusz, I added that test. I also updated the code to use "CREATE
 INDEX IF NOT EXISTS"... what do you think of going that direction?
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018da443ce23-a3e5726d-89fb-4819-9125-c3ef0ef1ff12-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Comment (by Mariusz Felisiak):

 #35180 was a duplicate. We should add a test for altering
 `CharField(db_index=True, ...)` to the `TextField()`.
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018d9effc541-031fe8e4-9402-4e2e-a7f4-115e587a6554-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak):

 * needs_better_patch:  0 => 1

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018d86fc1e5a-358ee85c-1d24-4f2d-902e-b0b125f2b648-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by bcail):

 * needs_better_patch:  1 => 0

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018d83b13e83-ca5f1307-2e39-40f5-b6ab-225e3c66357b-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Mariusz Felisiak):

 * needs_better_patch:  0 => 1

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018d7e081be2-6f850fce-de89-4d80-9e37-6143d1381fec-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  bcail
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by bcail):

 * owner:  (none) => bcail
 * status:  new => assigned

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018d6693caab-66eaf0bf-7878-4ae4-84f3-0a766c172acb-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by bcail):

 * cc: bcail (added)
 * needs_better_patch:  1 => 0

Comment:

 [https://github.com/django/django/pull/17807 PR]
-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/0107018d6692d30c-d8934161-4189-4966-8bdc-f4b9fecb58f0-00%40eu-central-1.amazonses.com.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index,#djangocph|
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Carlton Gibson):

 * keywords:  postgresql,migration,index =>
 postgresql,migration,index,#djangocph


Comment:

 I'm going to mark this for #djangocph for the sprint in Copenhagen.

 Whilst it's right in the heart of the migration framework, I think it
 should be an easy fix.

 Here's the GitHub permalink to the problem `if` check:
 
https://github.com/django/django/blob/fb8fd535c0f47cffb4da0c5900f3f66e1ec8d432/django/db/backends/postgresql/schema.py#L124-L126

 If you apply the original suggested fix you get a small number of failures
 (3 I think). These relate to needing to add an index due to a  `unique`
 flag being added. That's the last `or` in the problem `if`.

 The test from https://github.com/django/django/pull/9438 checks the new
 problem behaviour. (Can it live with the tests that fail if you apply the
 suggested fix?)

 That new test fails because the `unique` property is essentially `_unique
 or primary_key`, which is too wide. As I said above, using
 `new_field._unique` was enough to make the test pass.

 The task here is to go through that and make sure it's correct. Make sure
 the test is in the right place. Add a comment in the code (if it's
 needed). Maybe a release note etc.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.471d33eb2d48d82ca3ff163defc1d99c%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index |
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  1
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Carlton Gibson):

 * needs_better_patch:  0 => 1


Comment:

 Comments on PR: we have an error in the boolean logic, not correctly
 distinguishing between the `_unique` and `primary_key` cases.

 (The original suggestion leads to just 3 failures a fix should be simple
 enough...)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.8edd22b675062033426ba5cbe30d69e0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  (none)
 Type:  Bug  |   Status:  new
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index |
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tomer Chachamu):

 * status:  assigned => new
 * owner:  Tomer Chachamu => (none)
 * has_patch:  0 => 1


Comment:

 I have added a reasonable PR. https://github.com/django/django/pull/9438

 The root cause is that {{{schema_editor.create_model}}} defers the
 creation of indexes, so at any time you can either have indexes on the
 database (can be found using {{{schema_editor._constraint_names}}}),
 deferred or not at all.
 
https://github.com/django/django/blob/master/django/db/backends/base/schema.py#L300

 According to the comment they are deferred for SQLite, so one solution
 would be letting schema editors override the behaviour - removing it for
 non-sqlite, or just for postgres. I think it will become more difficult to
 reason about the schema editor in that case, and if done properly the
 deferred sql can also be an optimiser, removing redundant index creations
 and deletions.

 Every place that an index is added ought to check whether the index
 creation is already deferred, and remove the deferred one if so (in favour
 of an immediate one). Every place that an index is removed needs to check
 both deferred indexes and actual indexes.

 We can probably find more bugs by parameterising the schema and migration
 tests to try using separate schema_editors (which flushes deferred SQL to
 the database every step) or sharing schema_editors.

 Not all the lines added have a test backing them up but I think it's ready
 for somebody to have a look at and decide whether the approach is good.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.7ed3a21f1c13a91b4f99377d3b5b9bbb%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  Tomer
 |  Chachamu
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tomer Chachamu):

 The test case given is incorrect, as Django always uses a fresh schema
 editor for each migration step:

 
https://github.com/django/django/blob/master/django/db/migrations/executor.py#L225

 This passes and is similar to other cases in
 {{{migrations/test_operations.py}}}:

 {{{

 def test_change_primary_key(self):
 # Create a model with two fields
 operation1 = migrations.CreateModel(
 'SimpleModel',
 [
 ("field1", models.SlugField(max_length=20,
 primary_key=True)),
 ("field2", models.SlugField(max_length=20)),
 ],
 )
 # Drop field1 primary key constraint - this doesn't fail
 operation2 = migrations.AlterField(
 "SimpleModel",
 "field1",
 models.SlugField(max_length=20, primary_key=False),
 )
 # Add a primary key constraint to field2 - this fails
 operation3 = migrations.AlterField(
 "SimpleModel",
 "field2",
 models.SlugField(max_length=20, primary_key=True),
 )

 project_state = ProjectState()
 new_state = project_state.clone()
 operation1.state_forwards("migrtest", new_state)
 with connection.schema_editor() as editor:
 operation1.database_forwards("migrtest", editor,
 project_state, new_state)
 project_state, new_state = new_state, new_state.clone()
 operation2.state_forwards("migrtest", new_state)
 with connection.schema_editor() as editor:
 operation2.database_forwards("migrtest", editor,
 project_state, new_state)
 project_state, new_state = new_state, new_state.clone()
 operation3.state_forwards("migrtest", new_state)
 with connection.schema_editor() as editor:
 operation3.database_forwards("migrtest", editor,
 project_state, new_state)
 }}}

 I'm going to try working off the original bug description to reproduce the
 bug.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.e98a67a770864ae00821caeab06039f0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL)

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  Tomer
 |  Chachamu
 Type:  Bug  |   Status:  assigned
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tomer Chachamu):

 * status:  new => assigned
 * cc: Tomer Chachamu (added)
 * owner:  nobody => Tomer Chachamu


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.40a7f6daa22cbc323fe23a6781e099f1%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28646: Migration calls "CREATE INDEX" when one already exists when 'unique' field attribute is added (PostgreSQL) (was: Migration calls "CREATE INDEX" when one already exists when 'uniqu

[Django]

#28646: Migration calls "CREATE INDEX" when one already exists when 'unique' 
field
attribute is added (PostgreSQL)
-+-
 Reporter:  Hari - 何瑞理|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Migrations   |  Version:  1.11
 Severity:  Normal   |   Resolution:
 Keywords:   | Triage Stage:  Accepted
  postgresql,migration,index |
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * component:  Database layer (models, ORM) => Migrations
 * stage:  Unreviewed => Accepted


Comment:

 I can reproduce as long as the three operations are in the same migration.
 The crash doesn't happen if you put the `AlterField` operations in a
 separate migration.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/066.42aafaeae5a61983fad4b062bc10173e%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.