Hi all, I'm working on a project that's using Django and Graphene which is seeing extremely poor performance (specifically time-to-first-byte) when the GQL object graph is nested more than a few levels deep and thousands of nodes are included in the response body.
We've optimized the SQL queries and they run on the order of MS, while the API response time is on the order of S (6s in our degenerate test case; local env) so the problem appears to lie within the Django/Graphene layer. We've used py-spy to attempt to pinpoint the bottleneck and it seems like the majority of the time is spent within the graphql library's execute_fields method. What's not clear is how we might improve the performance of these internals. My first thought is to use async dataloaders (still WIP in Graphene, IIRC) but this degenerate case is a top-level node which uses prefetching to query for all its constituent data, so I'm not sure there's a win there unless there's a way to concurrently construct/render the lower level nodes (i.e. parent->children(n-"threads") or something. Also, here's the py-spy profile: https://www.dropbox.com/scl/fi/61eo71pyb2hvvbqsttol4/profile-gunicorn-with-signed-url.svg?rlkey=j7ggax4k3f4r6clxodr8kwt0r&dl=0 Our issue is almost identical to this one (https://stackoverflow.com/questions/63964764/graphene-python-performance-issues-for-large-data-sets) which is concerning but also gives me hope that others have found ways around this problem. So, in summary, has anyone here encountered and addressed a similar issue? Are there more insights to be gained from deeper or better profiling? Is this performance overhead just the reality of trying to marshall 10k+ objects out of SQL and into GQL on a single thread? Thanks, Pete P.S. Congrats on the release of Django 5! -- You received this message because you are subscribed to the Google Groups "Django users" group. To unsubscribe from this group and stop receiving emails from it, send an email to django-users+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/cdf9f476-1119-43a1-aac1-d1b6892c6929n%40googlegroups.com.
