Hi there ;).
Yesterday i noticed some strange behaviour of one of my sites - view [1]
serving json serialized data (one model, no references, 130 objects) was
executed about 20 secs[3] in production environment [2], and less than
1s via development server. Strange thing. What I've found - the view
served in production makes lots of writes(), and due to that - lots of
context switching. Every single json syntax character was put in
subsequent write. Serialized response was returned to client in portions
of n*1024B during this 20 sesc. Not too roboust.
Next thing to mention is that my production env uses tcp socket to
communication between frontend and django app - as [3] shows - that's
the worst scenario with view [1]. I've changed my view to write
serialized data to buffer before returning the response [4] - execution
time decreases dramatically, about 40 times. It's not due tu QoS - test
were made locally.
Further investigation i've made shows, that using tcp socket to
communication between nginx and django app improoves execution time as
well, but not so dramatically as using buffer - nevermind if with tcp or
unix socket.
I've experienced this behaviour as well with flup or with native django
runffcgi.
Writing directly to response [1] while using tcp socket is less
expensive (but not optimal) probably because of OS network buffers, am I
right?
What's the cause? Maybe ducktyping used somewhere in django code? Had no
time to find this issue, or maybe feature ;>. Anyway - mayby it's worth
to mention about that in django docs - these ar awsome for me in general
;). Is the [4] method optimal? Any ideas/hints?
My virtual environment is described in [5].
Any feedback will be appreciated ;)
[1]
def incidents_json(request):
i=Incident.objects.filter(public=True)
json_serializer = serializers.get_serializer("json")()
response=HttpResponse(mimetype='application/json')
json_serializer.serialize(i, ensure_ascii=False, stream=response,
fields=('date', 'slug', 'created', 'location','title'))
return response
[2] Production environment: django 1.2.5, python 2.7, virtualenv, flup,
nginx, supervosord
[3] execution times:
[bociek@webserver ~]$ time lynx --dump
http://www.mysite.tld/incydenty/json/ > /dev/null # serializing to
buffer, fcgi over unix socket
real 0m0.485s
^^^^^^^^
user 0m0.017s
sys 0m0.120s
[bociek@webserver ~]$ time lynx --dump
http://www.mysite.tld/incydenty/json-dt/ > /dev/null # serializing to
response, fcgi over unix socket
real 0m22.292s
^^^^^^^^^
user 0m0.037s
sys 0m0.100s
[bociek@webserver ~]$ time lynx --dump
http://www.fcgi.mysite.tld/incydenty/json/ > /dev/null # serializing to
buffer, fcgi over tcp socket
real 0m0.513s
^^^^^^^^
user 0m0.023s
sys 0m0.110s
[bociek@webserver ~]$ time lynx --dump
http://www.fcgi.mysite.tld/incydenty/json-dt/ > /dev/null # serializing
to response, fcgi over tcp socket
real 0m1.224s
^^^^^^^^
user 0m0.033s
sys 0m0.113s
[4]
def incidents_json2(request):
i=Incident.objects.filter(public=True)
from django.core import serializers
data = serializers.serialize("json",i, ensure_ascii=False,
fields=('date', 'slug', 'created', 'location','title'))
return HttpResponse(content=data, mimetype='application/json')
[5]
BeautifulSoup==3.2.0
Django==1.2.5
Markdown==2.0.3
PIL==1.1.7
Whoosh==1.7.1
django-authopenid==1.0.1
django-extensions==0.6
django-haystack==1.1.0
django-photologue==2.3
django-registration==0.7
django-tinymce==1.5.1a1
facebook-python-sdk==0.1
## FIXME: could not find svn URL in dependency_links for this package:
flup==1.0.3.dev-20110111
httplib2==0.6.0
oauth2==1.5.165
psycopg2==2.4-beta2
pyenchant==1.6.5
pyproj==1.8.8
python-openid==2.2.5
simplejson==2.1.3
wsgiref==0.1.2
xmpppy==0.5.0rc1
[5.1] communicating over tcp socket:
location / {
fastcgi_pass 127.0.0.1:9002;
include /etc/nginx/fcgi_django.params;
}
[5.2] and over unix socket:
location / {
fastcgi_pass unix:/tmp/sq.sock;
include /etc/nginx/fcgi_django.params;
}
fcgi_django.params:
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_param REMOTE_ADDR $remote_addr;
fastcgi_param REMOTE_PORT $remote_port;
fastcgi_param SERVER_ADDR $server_addr;
fastcgi_param SERVER_PORT $server_port;
fastcgi_param SERVER_NAME $http_host;
fastcgi_param SERVER_PROTOCOL $server_protocol;
fastcgi_pass_header Authorization;
fastcgi_intercept_errors off;
--
Cheers
Bartek
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
