#28062: Using iterators behind pgBouncer leads to non-existent cursor erros
-------------------------------------+-------------------------------------
Reporter: Sergey Fursov | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 1.11
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Josh Smeaton):
* cc: josh.smeaton@… (added)
* severity: Release blocker => Normal
Comment:
I don't think documentation is a good enough solution for this. Lots of
people use pgbouncer, and any of them using .iterator() are about to start
experiencing crashes with what was perfectly working code. It's not fair
to ask people to change code or configuration for what should have been a
transparent performance (memory) improvement.
I think our options are limited, but I'd be keen to hear from others how
we should approach this particular bug.
- We could revert the change, and revisit for Django 2.0.
- We could also add an argument to .iterator() that allows users to opt-in
to the new server side cursors. We'd need to document the limitations, and
suggest ideal scenarios (use transaction.atomic, or
.using('non_pgbouncer_transaction_connection')).
For your particular setup, I'd maybe recommend running a second pgbouncer
in session mode (or not use pgbouncer), and for queries that use
.iterator(), using that specific connection. As long as iterator() isn't
used in the majority of your requests, the number of connections shouldn't
spike too high.
--
Ticket URL: <https://code.djangoproject.com/ticket/28062#comment:2>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To post to this group, send email to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/064.8f740bfb9f6e99659f689132a1dce6e3%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.
