On 12/5/18 8:30 PM, Saleem Jaffer wrote:
Hi all,

The default paginator that comes with Django is inefficient when dealing with large tables. This is because the final query for fetching pages uses "OFFSET" which is basically a linear scan till the last index of the current page. Does it make sense to have a better paginator which does not use "OFFSET".

If this sounds like a good idea, I have some ideas on how to do it and with some help from you guys I can implement it.

There are a number of alternatives to this, as well as low-cost solutions to improve OFFSET / LIMIT pagination.

By adding an index to the sorting field(s), it can drastically improve the "simple" case.

Beyond that, you start getting into cases with more significant trade-offs. I know Matthew Schinckel was recently working on a drop-in replacement paginator that used "keyset pagination" https://bitbucket.org/schinckel/django-keyset-pagination

There's a lot of published work on this topic, and I'd be very interested to see, at least, a library implementing some of these alternatives.

At the very least, we could want to ensure the documentation recommends indexing on the ORDERing fields, where possible.

--
Curtis

--
You received this message because you are subscribed to the Google Groups "Django 
developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/f9c52aee-e28c-7ae0-306d-3ae2fbd828ee%40tinbrain.net.
For more options, visit https://groups.google.com/d/optout.
  • A... Saleem Jaffer
    • ... ludovic coues
      • ... Jason Johns
        • ... Adam Johnson
    • ... Curtis Maloney
      • ... Josh Smeaton
    • ... Kye Russell
      • ... Dan Davis
        • ... Adam Johnson
    • ... 'Ivan Anishchuk' via Django developers (Contributions to Django itself)
    • ... Cristiano Coelho
      • ... Tim Allen
        • ... Tom Forbes

Reply via email to