We are experiencing very slow performance from one view in the API that
returns a queryset of a filter. We expected 0.5 - 1.0 sec, but the request
is taking up to 4 seconds. Turning off pagination decreases the wait to 2
sec.
>From the Django shell, the query is extremely fast, < 0.001 sec.
>>> from django.db.models import F
>>> from app.models import Weather
>>> import numpy as np
>>> R_EARTH = 6371000.0 # [m] earth's radius
>>> search_radius = 160934
>>> latitude, longitude = 38.1, -122.1
>>> dtheta = np.rad2deg(search_radius / R_EARTH) # [degrees]
>>> queryset = Weather.objects.filter(
... latitude__range=(latitude - dtheta, latitude + dtheta),
... longitude__range=(longitude - dtheta, longitude + dtheta),
... data_type__in=Weather.ALL_DATA_TYPES
... ).annotate(distance_d=(
... (F('latitude') - latitude) ** 2 + (F('longitude') - longitude) ** 2)
... ).order_by('distance_d').exclude(distance_d__gt=(dtheta ** 2))
# 1000 loops, best of 3: 737 µs per loop - using %timeit
I tested serializing the queryset and rendering the response, and It is
still very fast, < 1 sec
# I know there is an easier/better way to do this, sorry :(
>>> from django.http import HttpRequest
>>> from rest_framework.request import Request
>>> from app.serializers import WeatherSearchSerializer
>>> request = Request(HttpRequest())
>>> request.META['SERVER_NAME'] = u'localhost'
>>> request.META['SERVER_PORT'] = 8000
>>> w = WeatherSearchSerializer(queryset, context={'request': request},
many=True)
# 10000 loops, best of 3: 56.5 µs per loop - using %timeit
>>> jsondata = JSONRenderer().render(w.data)
# 1 loops, best of 3: 677 ms per loop - using %timeit
But if I use the browseable API or send a request, it takes 3-4 seconds.
>>> import requests
>>> PARAMS = {'latitude': 38.2, 'longitude': -122.1}
>>> requests.get(url=URL, auth=AUTH, params=dict(PARAMS, limit=1000))
# 1 loop, best of 3: 4.21 s per loop - using %timeit
requirements:
Django>=1.8.5
djangorestframework>=3.3.3
psycopg2>=2.6.2
django-storages>=1.4
boto>=2.38.0
django-crispy-forms>=1.6.0
django-filter>=0.14.0
platform: AWS
EC2: c3.4xlarge instance based on Oracle Linux 7.2
RDS: Postgres on a db.m3.medium instance with 1TB of storage
S3
software: Apache httpd with mod_wsgi and python-2.7
models and serializers: totally vanilla, nothing custom
* serializer is `serializers.HyperlinkedModelSerializer`
views: totaly vanilla, nothing custom
* view uses `viewsets.ModelViewSet`
Any ideas where the extra seconds are coming from? I'm not sure where to
profile. Any assistance would be greatly appreciated.
Other relevant posts:
* https://groups.google.com/d/msg/django-rest-framework/AkLKdfxCrcU/qyOB1k37pJAJ
* that's all I could find search terms: slow queryset
--
You received this message because you are subscribed to the Google Groups
"Django REST framework" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
