#29760: Cursors are being closed explicitly in autocommit mode
-------------------------------------+-------------------------------------
Reporter: dhargan | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Normal | Resolution:
Keywords: cursor, sql, | Triage Stage:
database, autocommit, | Unreviewed
transactions, postrgresql, |
psycopg |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by dhargan):
* version: 2.1 => 1.11
Old description:
> Recently we started to get this exception time to time:
>
> {{{
> OperationalError: cursor “_django_curs_<id>” does not exist
> }}}
>
> especially when our traffic is higher than usual. Each time the error is
> in a different line of the code so it is not easy to follow but
> appearantly it is a synchronisation issue; the cursor is being closed
> before the transaction ended.
>
> In django.db.models.sql.compiler.SQLAggregateCompiler:
>
> {{{
> def cursor_iter(cursor, sentinel, col_count, itersize):
> """
> Yield blocks of rows from a cursor and ensure the cursor is closed
> when
> done.
> """
> try:
> for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
> yield rows if col_count is None else [r[:col_count] for r in
> rows]
> finally:
> cursor.close()
> }}}
>
> According to [http://initd.org/psycopg/docs/usage.html#server-side-
> cursors psycopg documentation], server-side cursors should not be closed
> explicitly if it is in autocommit mode.
>
> According to
> [https://docs.djangoproject.com/en/2.1/topics/db/transactions/#autocommit
> Django documentation], Django sets autocommit mode true in default
> settings.
>
> Of course it is possible to set withhold setting to false. In this case,
> cursor.close() must be called at the end but otherwise, according to the
> documentation, it should not be called at all. So if I am not mistaken
> there should be a conditional statement before calling cursor.close().
New description:
Recently we started to get this exception time to time:
{{{
OperationalError: cursor “_django_curs_<id>” does not exist
}}}
especially when our traffic is higher than usual. Each time the error is
in a different line of the code so it is not easy to follow but
appearantly it is a synchronisation issue; the cursor is being closed
before the transaction ended.
In django.db.models.sql.compiler.SQLAggregateCompiler:
{{{
def cursor_iter(cursor, sentinel, col_count, itersize):
"""
Yield blocks of rows from a cursor and ensure the cursor is closed
when
done.
"""
try:
for rows in iter((lambda: cursor.fetchmany(itersize)), sentinel):
yield rows if col_count is None else [r[:col_count] for r in
rows]
finally:
cursor.close()
}}}
According to [http://initd.org/psycopg/docs/usage.html#server-side-cursors
psycopg documentation], server-side cursors should not be closed
explicitly if it is in autocommit mode.
According to
[https://docs.djangoproject.com/en/1.11/topics/db/transactions/#autocommit
Django documentation], Django sets autocommit mode true in default
settings.
Of course it is possible to set withhold setting to false. In this case,
cursor.close() must be called at the end but otherwise, according to the
documentation, it should not be called at all. So if I am not mistaken
there should be a conditional statement before calling cursor.close().
--
--
Ticket URL: <https://code.djangoproject.com/ticket/29760#comment:3>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/065.e97b99ce00ba55e802e125976572058a%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
