After some testing I have found out that even when using values_list to prevent massive object creation when fetching data, it is from 2 to 3 times slower tan using directly a cursor.execute query through the django cursor.
The issue started here http://stackoverflow.com/questions/33726004/django-queryset-vs-raw-query-performance when trying to improve some queries that were looking very slow on apache benchemark testing. It seems like compiler.results_iter, called from ValuesListQuerySet is very, very slow, due to all the for loops in Python code compared to when using a raw query through a C connector (like mysqlclient), there's just too much boilerplate that might be posible to be removed. As an example, a very ugly work around to those critical queries, was defining a function like this, which would convery a queryset into something usable by a cursor, which performs very, very efficiently, at least with mysqlclient connector. q = qs.query.get_compiler(qs.db).as_sql() with connection.cursor() as c: c.execute(q[0], q[1]) for r in c: yield r Would it be possible to have something similar to values_list, but that executes directly through a cursor, improving performance? I'm sure it will be less flexible than values_list, but the extra performance will be nice. -- You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscr...@googlegroups.com. To post to this group, send email to django-developers@googlegroups.com. Visit this group at http://groups.google.com/group/django-developers. To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/56493786.422b810a.59cd4.374d%40mx.google.com. For more options, visit https://groups.google.com/d/optout.
