Yes we should be documenting edge cases and unexpected results. We have a page that discusses some issues with server side cursors: https://docs.djangoproject.com/en/dev/ref/databases/#transaction-pooling-server-side-cursors
Is there anyway we could make SSC work without a transaction? We'd prefer to fix than document if possible. On Thursday, 3 August 2017 07:22:23 UTC+10, Evan Heidtmann wrote: > > Hey all, > > The docs for qs.iterator() say that, in Django 1.11 on Postgres, a > server-side cursor is used. > > Oracle and PostgreSQL >> <https://docs.djangoproject.com/en/1.11/ref/databases/#postgresql-server-side-cursors> >> use >> server-side cursors to stream results from the database without loading the >> entire result set into memory. > > > https://docs.djangoproject.com/en/1.11/ref/models/querysets/#iterator > > I discovered that this is only true if the query is run inside a > transaction. Outside a transaction, it appears that Django falls back to a > regular SELECT query, which could be extremely expensive and is certainly > unexpected. > > I don't know what happens if you call .iterator() inside a transaction > block and then exit that block. Undefined behavior? > > Therefore I suggest the following docs edit (changes in italic): > > Oracle and PostgreSQL >> <https://docs.djangoproject.com/en/1.11/ref/databases/#postgresql-server-side-cursors> >> use >> server-side cursors to stream results from the database without loading the >> entire result set into memory. *On PostgreSQL, server-side cursors are >> used only if .iterator() is called within a transaction.* > > > And thanks for this new feature -- it's transformative for my workload. > > -Evan > -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at https://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/8ca09d7d-4b53-4bc3-ad56-fa53fb36baa0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
