Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by jdlourenco):

 I am running into what seems to be the same problem, however I'm not using
 ''pgBouncer''.

 I'm connecting directly to a **redshift** database and I get this error
 using either the **django.db.backends.postgresql_psycopg2** or the
 **django_redshift_backend** engines.

 {{{
 'redshift': {
 'NAME': 'dbname',
 ...
 'ENGINE': 'django.db.backends.postgresql_psycopg2',
 'HOST': 'xx.redshift.amazonaws.com',
 'PORT': 5439,
  }
 }}}


 I've enabled DEBUG logging and I can see Django creating a server side
 cursor by using a ''DECLARE WITH HOLD'' statement. It then issues two
 queries and the second fails with a ''cursor "_django_curs_XXX" does
 not exist'' error message. Setting **DISABLE_SERVER_SIDE_CURSORS=true**
 seems to solve the issue.

 According the Django documentation, this happens due to transaction
 pooling resulting in Django using a different connection for which the
 named cursor does not exist the second time it issues the query. However I
 can't understand why this is happening since afaik I'm not using
 ''pgBouncer'' nor any other middleware implementing transaction pooling.

 Can you help me understand why this is happening?

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.ce70941f99dbaa9d0029727584f3db49%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"2e7b1e32976127bbe71320c848ce89b6e4b7e1a3" 2e7b1e32]:
 {{{
 #!CommitTicketReference repository=""
 revision="2e7b1e32976127bbe71320c848ce89b6e4b7e1a3"
 [2.0.x] Refs #28062 -- Doc'd PostgreSQL server-side cursors as a backwards
 incompatible change.

 Backport of 2919a08c20d5ae48e381d6bd251d3b0d400d47d9 from master
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.3919fb0b67ef78b6e59312ccbf5c1e3b%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"0037cd1fa0f45c9a8545dcc16bdee7f71f94c773" 0037cd1f]:
 {{{
 #!CommitTicketReference repository=""
 revision="0037cd1fa0f45c9a8545dcc16bdee7f71f94c773"
 [1.11.x] Refs #28062 -- Doc'd PostgreSQL server-side cursors as a
 backwards incompatible change.

 Backport of 2919a08c20d5ae48e381d6bd251d3b0d400d47d9 from master
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.2033a944dea922e141766031ef3781ce%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by GitHub ):

 In [changeset:"2919a08c20d5ae48e381d6bd251d3b0d400d47d9" 2919a08c]:
 {{{
 #!CommitTicketReference repository=""
 revision="2919a08c20d5ae48e381d6bd251d3b0d400d47d9"
 Refs #28062 -- Doc'd PostgreSQL server-side cursors as a backwards
 incompatible change.
 }}}

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.28032d2088d51394dab9c57650b37e9d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham):

 [https://github.com/django/django/pull/9863 PR] for an addition to the
 1.11 release notes.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.af743e74d5e5b77173af4a69142bd13d%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Luke Plant):

 There really needs to be a note about this in the 1.11 release notes, as
 it's a breaking change that affects a pretty common set up. There is a
 note only in the 1.11.1 notes, which is not where people look.

-- 
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.a7ccc1cefc3e947f6f377992c0ac80b4%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Tim Graham ):

 In [changeset:"6a2624286b0501942522726bfdcb18537a86ce7c" 6a26242]:
 {{{
 #!CommitTicketReference repository=""
 revision="6a2624286b0501942522726bfdcb18537a86ce7c"
 [1.11.x] Fixed #28062 -- Added a setting to disable server-side cursors on
 PostgreSQL.

 When a connection pooler is set up in transaction pooling mode, queries
 relying on server-side cursors fail. The DISABLE_SERVER_SIDE_CURSORS
 setting in DATABASES disables server-side cursors for this use case.

 Backport of 88336fdbb5e101fa25825b737169c0d6af2faa93 from master
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.5ac27bfa894abe9f415d3fbc493badcc%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  closed
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:  fixed
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham ):

 * status:  assigned => closed
 * resolution:   => fixed


Comment:

 In [changeset:"88336fdbb5e101fa25825b737169c0d6af2faa93" 88336fdb]:
 {{{
 #!CommitTicketReference repository=""
 revision="88336fdbb5e101fa25825b737169c0d6af2faa93"
 Fixed #28062 -- Added a setting to disable server-side cursors on
 PostgreSQL.

 When a connection pooler is set up in transaction pooling mode, queries
 relying on server-side cursors fail. The DISABLE_SERVER_SIDE_CURSORS
 setting in DATABASES disables server-side cursors for this use case.
 }}}

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.e6ac7f36d641f5bb3e0f68e4bcc39b40%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Ready for
 |  checkin
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * stage:  Accepted => Ready for checkin


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.63b3b64a6918836ac72af704948309b0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Lachlan Cannon):

 * cc: lcannon@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.fcc4649197160227cdcce7434be7a414%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  1|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by François Freitag):

 * has_patch:  0 => 1


Comment:

 [https://github.com/django/django/pull/8430 PR]

 Working on this issue, I came to the conclusions that {{{None}}} is not
 necessary and I prefer the simpler scheme where:

 * the use of server-side cursors is disabled for a connection if the
 setting is {{{False}}}.
 * when the setting is {{{True}}}, server-side cursors are used by default
 and users and library code can opt-out specifying
 {{{chunked_fetch=False}}}.

 I believe auto opt-in server-side cursors is acceptable, since it's a
 memory improvement in most case. The issue here is that server-side
 cursors cannot be used with transaction pooling, which is global to a
 connection. Setting {{{ENABLE_SERVER_SIDE_CURSORS}}} to {{{False}}} for
 the whole connection makes sense and should disable the use of server-side
 cursors regardless of the {{{chunked_fetch}}} keyword argument.

 For use cases where server-side cursors are not desirable, the user /
 library-code can now turn them off explicitly using
 {{{chunked_fetch=False}}}.

 I think this solution is simpler and covers the use cases I can think of.
 Am I missing something?

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.3941028ae322fcdb5ea6bbd1ad4d65d0%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  François
 |  Freitag
 Type:  Bug  |   Status:  assigned
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by François Freitag):

 * owner:  nobody => François Freitag
 * status:  new => assigned


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.8ac16139b95b3b0f1b6b12b531305612%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by holvianssi):

 Opt-in is a good solution for end user code, but then library code can't
 opt-in, as library code can't know if it's running in pgBouncer setup or
 not. A database setting would get rid of this problem, but if there are
 cases where you want to use cursors for some cases, but not all of the
 cases, then you are screwed. The complex solution is to add both the opt-
 in flag and database settings. The default for the opt-in flag is None,
 meaning use the default from database settings. True forces this feature
 on, and False forces the feature off. Now, this covers all cases, but
 might create a situation where it's not easy for users to understand when
 exactly cursors are used.

 I'm slightly favouring the opt-in + database setting approach, next would
 be opt-in without database setting, and last database setting without opt-
 in. All of those solutions are fine for me, but I don't like the idea of
 removing the feature.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.fdaa93e523bf1f36e51b570bc98b6289%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Florian Apolloner):

 Replying to [comment:2 Josh Smeaton]:
 > - We could also add an argument to .iterator() that allows users to opt-
 in to the new server side cursors. We'd need to document the limitations,
 and suggest ideal scenarios (use transaction.atomic, or
 .using('non_pgbouncer_transaction_connection')).

 Opting in seems like a reasonable thing to do, though given the fact that
 it will generally not work for anyone using pgbouncer in a specific mode,
 it might make more sense if we'd also provide a switch to turn this off/on
 globally (database options).

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.53824f0f5ef9ff16204049243d25e8f3%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by François Freitag):

 * cc: mail@… (added)


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.468408454721798f1dd84c1f5d98fe43%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-

Comment (by Sergey Fursov):

 Hi Josh,

 > For your particular setup, I'd maybe recommend running a second
 pgbouncer in session mode (or not use pgbouncer), and for queries that use
 .iterator(), using that specific connection. As long as iterator() isn't
 used in the majority of your requests, the number of connections shouldn't
 spike too high.

 This is reasonable suggestion, thanks

 > We could also add an argument to .iterator() that allows users to opt-in
 to the new server side cursors. We'd need to document the limitations, and
 suggest ideal scenarios (use transaction.atomic, or
 .using('non_pgbouncer_transaction_connection')).

 This sound like the best option for me (at least compared to reverting
 changes)

 Another my point was the fact, that django use iterator internally in
 ModelChoiceField for optimizing iteration over queryset and in several
 serializer methods.
 For my particular case it would be enough to add additional
 ModelChoiceIterator to override default ModelChoiceField.iterator field
 (we don't use DB serialization much, so i don't know, does serialization
 part have big impact on production using). But i think this point should
 be considered while working on this issue, too, because it might break
 apps in production.

--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.f67638e0fd572ac3e116d977a1c87694%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors (was: Using iterators behind pgBouncer leads to non-existent cursor erros)

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Josh Smeaton):

 * severity:  Normal => Release blocker


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.36288cc3352dc226cb934d03bc19e63f%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.

Re: [Django] #28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor errors (was: Using iterators behind pgBouncer leads to non-existent cursor erros)

[Django]

#28062: Using QuerySet.iterator() with pgBouncer leads to nonexistent cursor 
errors
-+-
 Reporter:  Sergey Fursov|Owner:  nobody
 Type:  Bug  |   Status:  new
Component:  Database layer   |  Version:  1.11
  (models, ORM)  |
 Severity:  Release blocker  |   Resolution:
 Keywords:   | Triage Stage:  Accepted
Has patch:  0|  Needs documentation:  0
  Needs tests:  0|  Patch needs improvement:  0
Easy pickings:  0|UI/UX:  0
-+-
Changes (by Tim Graham):

 * type:  Uncategorized => Bug
 * severity:  Normal => Release blocker
 * stage:  Unreviewed => Accepted


--
Ticket URL: 
Django 
The Web framework for perfectionists with deadlines.

-- 
You received this message because you are subscribed to the Google Groups 
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-updates+unsubscr...@googlegroups.com.
To post to this group, send email to django-updates@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-updates/064.a8366928224f24428d4ec9f6b3b997f9%40djangoproject.com.
For more options, visit https://groups.google.com/d/optout.