Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
 Reporter:  akaihola |Owner:  aaugustin
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:  fixed
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by Aymeric Augustin ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"3a4276ffc35326f20e95890b50a67aebeabc9ad0"]:
 {{{
 #!CommitTicketReference repository=""
 revision="3a4276ffc35326f20e95890b50a67aebeabc9ad0"
 Tested that get_or_create raises IntegrityError.

 It used to raise "DatabaseError: no such savepoint" with the old
 transaction management. Closes #15117.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
 Reporter:  akaihola |Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by aaugustin):

 Indeed, this specific bug is fixed in master. I'm going to commit a
 variant of the test case attached by d0ugal.

 The code provided by the reporter now raises an !IntegrityError when the
 transaction is committed (immediately if no transaction is in progress) on
 databases that enforce integrity. It inserts bad data silently on other
 databases.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
 Reporter:  akaihola |Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by aaugustin):

 I just introduced `django.db.transaction.atomic` that precisely designed
 for this use case.

 It uses "release savepoint" instead of "commit", which allows rolling back
 to the savepoint if necessary.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
 Reporter:  akaihola |Owner:  aaugustin
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  master
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-
Changes (by aaugustin):

 * status:  new => assigned
 * owner:  nobody => aaugustin


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
 Reporter:  akaihola |Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  SVN
  (models, ORM)  |   Resolution:
 Severity:  Normal   | Triage Stage:  Accepted
 Keywords:   |  Needs documentation:  0
Has patch:  0|  Patch needs improvement:  0
  Needs tests:  0|UI/UX:  0
Easy pickings:  0|
-+-

Comment (by anonymous):

 What to do when in this situation where all get_or_create fail ?

 TYIA :)

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone:  1.4|  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:
Needs documentation:  0  |  Has patch:  0
Patch needs improvement:  0  |Needs tests:  0
  UI/UX:  0  |  Easy pickings:  0
-+-

Comment (by andrewgodwin):

 So, I worked with Dougal on this at the Europython sprints, and the
 problem is that the code tries to COMMIT the previous transaction, which
 fails, and then tries to ROLLBACK TO SAVEPOINT after the commit has
 failed.

 As far as I can tell, the failing COMMIT implicitly clears the savepoints
 in the previous transaction. The postgres client libraries reflect this by
 sending a BEGIN along with the ROLLBACK TO SAVEPOINT, as they assume the
 failing COMMIT cancels the previous transaction. This then causes the
 ROLLBACK TO SAVEPOINT to fail, as it's a new transaction without that
 savepoint.

 We changed the backend to not send the BEGIN after the COMMIT, but then
 Postgres complained that this was invalid, and so I suspect it's not valid
 to send ROLLBACK TO SAVEPOINT _after_ COMMIT (all the examples in the
 postgres docs have it before a COMMIT).

 It seems, then, that the "clean" fix would be to make this code just start
 a new transaction if its COMMIT fails, but that leads to potential nesting
 errors depending if the outer code has already started a transaction or
 not. I can't see a nice way to roll back to the savepoint at the moment,
 but my brain hurts after too much poking around with transactions, and I
 can't find the part of the SQL spec that Postgres is apparently adhering
 to in this regard (I think it's in SQL 1999, and I can't find where to
 download it).

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone:  1.4|  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:
Needs documentation:  0  |  Has patch:  0
Patch needs improvement:  0  |Needs tests:  0
  UI/UX:  0  |  Easy pickings:  0
-+-
Changes (by d0ugal):

 * ui_ux:   => 0


Comment:

 I turned on the log in postgres to see what was happening in the SQL.

 {{{
 2011-06-25 03:03:12 PDT LOG:  statement: SELECT
 "get_or_create_integrityerrortest"."country_id",
 "get_or_create_integrityerrortest"."info" FROM
 "get_or_create_integrityerrortest" WHERE
 "get_or_create_integrityerrortest"."country_id" = 1
 2011-06-25 03:03:12 PDT LOG:  statement: SAVEPOINT s1217009984_x1
 2011-06-25 03:03:12 PDT LOG:  statement: INSERT INTO
 "get_or_create_integrityerrortest" ("country_id", "info") VALUES (1, E'')
 2011-06-25 03:03:12 PDT LOG:  statement: COMMIT
 2011-06-25 03:03:12 PDT ERROR:  insert or update on table
 "get_or_create_integrityerrortest" violates foreign key constraint
 "get_or_create_integrityerrortest_country_id_fkey"
 2011-06-25 03:03:12 PDT DETAIL:  Key (country_id)=(1) is not present in
 table "get_or_create_person".
 2011-06-25 03:03:12 PDT STATEMENT:  COMMIT
 2011-06-25 03:03:12 PDT LOG:  statement: BEGIN; SET TRANSACTION ISOLATION
 LEVEL READ COMMITTED
 2011-06-25 03:03:12 PDT LOG:  statement: ROLLBACK TO SAVEPOINT
 s1217009984_x1
 2011-06-25 03:03:12 PDT ERROR:  no such savepoint
 2011-06-25 03:03:12 PDT STATEMENT:  ROLLBACK TO SAVEPOINT s1217009984_x1
 2011-06-25 03:03:12 PDT LOG:  statement: ROLLBACK
 }}}

 It appears that postgres has gotten rid of the savepoint.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone:  1.4|  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:
Needs documentation:  0  |  Has patch:  0
Patch needs improvement:  0  |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by d0ugal):

 * milestone:   => 1.4


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:
Needs documentation:  0  |  Has patch:  0
Patch needs improvement:  0  |Needs tests:  0
 |  Easy pickings:  0
-+-

Comment (by d0ugal):

 Another observation while I worked on this ticket: get_or_create has the
 following lines.

 {{{
 except IntegrityError, e:
 transaction.savepoint_rollback(sid, using=self.db)
 try:
 return self.get(**lookup), False
 except self.model.DoesNotExist:
 raise e
 }}}

 Would it not be preferable to do the following to preserve the full
 original traceback?

 {{{
 except IntegrityError, e:
 transaction.savepoint_rollback(sid, using=self.db)
 try:
 return self.get(**lookup), False
 except self.model.DoesNotExist:
 pass
 raise
 }}}

 This is not directly related to the tickets issue, but to the more general
 problem of improving get_or_creates error reporting.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:
Needs documentation:  0  |  Has patch:  0
Patch needs improvement:  0  |Needs tests:  0
 |  Easy pickings:  0
-+-

Comment (by d0ugal):

 I've attached a patch with a failing test case to show the problem.

 The problem here is only seen (from what I can tell and test) in ORM
 backends that support savepoints. Basically, when the integrity error is
 raised a rollback is attempted to the save point, however, it doesn't
 exist thus raising the "DatabaseError: no such savepoint". I can't figure
 out why the save point is lost, would it be rolled back automatically when
 the integrity error happens?

 The following lines then in get_or_create actually re-raise the Integrity
 error and I think this should be the expected behaviour.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:
Needs documentation:  0  |  Has patch:  0
Patch needs improvement:  0  |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by sirmmo@…):

 * cc: sirmmo@… (added)


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |  Owner:  nobody
   Type:  Bug| Status:  new
  Milestone: |  Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: |   Severity:  Normal
   Triage Stage:  Accepted   |   Keywords:
Needs documentation:  0  |  Has patch:  0
Patch needs improvement:  0  |Needs tests:  0
 |  Easy pickings:  0
-+-
Changes (by d0ugal):

 * cc: dougal85@… (added)
 * easy:   => 0


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
-+-
   Reporter:  akaihola   |Owner:  nobody
   Type:  Bug|   Status:  new
  Milestone: |Component:  Database layer
Version:  SVN|  (models, ORM)
 Resolution: | Severity:  Normal
   Triage Stage:  Accepted   | Keywords:
Needs documentation:  0  |Has patch:  0
Patch needs improvement:  0  |  Needs tests:  0
-+-
Changes (by jaddison):

 * type:   => Bug
 * severity:   => Normal


-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

Re: [Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
---+
  Reporter:  akaihola  | Owner:  nobody
Status:  new   | Milestone:
 Component:  Database layer (models, ORM)  |   Version:  SVN   
Resolution:|  Keywords:
 Stage:  Accepted  | Has_patch:  0 
Needs_docs:  0 |   Needs_tests:  0 
Needs_better_patch:  0 |  
---+
Changes (by russellm):

  * needs_better_patch:  => 0
  * stage:  Unreviewed => Accepted
  * needs_tests:  => 0
  * needs_docs:  => 0

Comment:

 Agreed; a savepoint error isn't the right response here. I'm not
 completely convinced that IntegrityError is right, either; get_or_create
 should be able to... well... get, or create. However, I can see that there
 are issues with that in practice.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.

[Django] #15117: get_or_create() raises "DatabaseError: no such savepoint" instead of IntegrityError (PostgreSQL)

[Django]

#15117: get_or_create() raises "DatabaseError: no such savepoint" instead of
IntegrityError (PostgreSQL)
--+-
 Reporter:  akaihola  |   Owner:  nobody
   Status:  new   |   Milestone:
Component:  Database layer (models, ORM)  | Version:  SVN   
 Keywords:|   Stage:  Unreviewed
Has_patch:  0 |  
--+-
 Given the following models in a `world` app:

 {{{
 class Country(models.Model):
 name = models.CharField(max_length=50)

 class CountryExtraInfo(models.Model):
 country = models.ForeignKey(Country, primary_key=True)
 info = models.TextField(blank=True, default='')
 }}}

 and assuming no Countries have been created yet, the following happens:

 {{{
 >>> from world.models import CountryExtraInfo
 >>> CountryExtraInfo.objects.get_or_create(pk=1)
 Traceback (most recent call last):
   File "", line 1, in 
   File "django/db/models/manager.py", line 135, in get_or_create
 return self.get_query_set().get_or_create(**kwargs)
   File "django/db/models/query.py", line 389, in get_or_create
 transaction.savepoint_rollback(sid, using=self.db)
   File "django/db/transaction.py", line 235, in savepoint_rollback
 connection._savepoint_rollback(sid)
   File "django/db/backends/__init__.py", line 70, in _savepoint_rollback
 self.cursor().execute(self.ops.savepoint_rollback_sql(sid))
   File "django/db/backends/postgresql_psycopg2/base.py", line 44, in
 execute
 return self.cursor.execute(query, args)
 DatabaseError: no such savepoint
 }}}

 Also:

 {{{
 >>> from world.models import Country, CountryExtraInfo
 >>> c = Country()
 >>> c.id = 1
 >>> CountryExtraInfo.objects.get_or_create(country=c)
 Traceback (most recent call last):
   File "", line 1, in 
   File "django/db/models/manager.py", line 135, in get_or_create
 return self.get_query_set().get_or_create(**kwargs)
   File "django/db/models/query.py", line 389, in get_or_create
 transaction.savepoint_rollback(sid, using=self.db)
   File "django/db/transaction.py", line 235, in savepoint_rollback
 connection._savepoint_rollback(sid)
   File "django/db/backends/__init__.py", line 70, in _savepoint_rollback
 self.cursor().execute(self.ops.savepoint_rollback_sql(sid))
   File "django/db/backends/postgresql_psycopg2/base.py", line 44, in
 execute
 return self.cursor.execute(query, args)
 DatabaseError: no such savepoint
 }}}

 In these situations, it would make more sense to get an !IntegrityError
 instead.

 Test environment:
  * Python 2.6.6
  * Django r15242
  * PostgreSQL 8.4.6
  * psycopg2 2.2.1

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To post to this group, send email to django-updates@googlegroups.com.
To unsubscribe from this group, send email to 
django-updates+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-updates?hl=en.