Re: File upload failing occasionally
On Jan 6, 10:31 pm, ppdowrote: > To Karen: yes, I meant TCP RST, sorry. > > Update: > > Using mod_wsgi made the problem go away for Firefox. > > Limiting my research to an interaction problem between Apache and > Safari, I stumbled upon this bug report for Apache > https://bugs.webkit.org/show_bug.cgi?id=5760 that describes something > very similar to what is happening and it is apparently still open. For which the problem was identified as being in MacOSX Foundation framework. Supposedly fixed in 10.5.5, but some people still having problems after updating to that revision of OS. Thanks for finding that page in webkit issue tracker, will be good one to keep in mind id others see similar problem. I'll have to see if disabling keep alive solves my ab performance tests. Graham > Reading this gave me the idea to try and disable the keepalive and, > though I need to test it more extensively, it seems the problem is > gone. > > A simple: > > BrowserMatch "Safari" nokeepalive > > in the Apache configuration did the trick. As on the production server > this application will not have to stand much traffic anyway, it should > be ok also without keepalive. > > I'll update this thread if I have more findings. > > Thank you both for your ideas! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: File upload failing occasionally
To Karen: yes, I meant TCP RST, sorry. Update: Using mod_wsgi made the problem go away for Firefox. Limiting my research to an interaction problem between Apache and Safari, I stumbled upon this bug report for Apache https://bugs.webkit.org/show_bug.cgi?id=5760 that describes something very similar to what is happening and it is apparently still open. Reading this gave me the idea to try and disable the keepalive and, though I need to test it more extensively, it seems the problem is gone. A simple: BrowserMatch "Safari" nokeepalive in the Apache configuration did the trick. As on the production server this application will not have to stand much traffic anyway, it should be ok also without keepalive. I'll update this thread if I have more findings. Thank you both for your ideas! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: File upload failing occasionally
On Mon, Jan 5, 2009 at 5:30 PM, ppdowrote: > > I tried to monitor the interface using WireShark and the communication > just hangs during a data transfer: Apache is just acknowledging the > receipt of some packets and then just seems to hang. The suspicious > bit is a fairly long series of http RST about 20ms before the server > hangs. Though I'm not knowledgeable enough on http, to me this clearly > points in the direction of an Apache problem, though I'm not sure if > and how it can be solved. > I don't know what an http RST is. Do you mean TCP RST? You see a long series of TCP resets? That would indicate a problem. Which machine sends them? If you put a formatted trace somewhere I could see it I might be able to do some diagnosis, but it sounds like you may have a pretty low-level network problem going on. Karen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: File upload failing occasionally
On Jan 6, 9:30 am, ppdowrote: > I tried to monitor the interface using WireShark and the communication > just hangs during a data transfer: Apache is just acknowledging the > receipt of some packets and then just seems to hang. The suspicious > bit is a fairly long series of http RST about 20ms before the server > hangs. Though I'm not knowledgeable enough on http, to me this clearly > points in the direction of an Apache problem, though I'm not sure if > and how it can be solved. > > I will try the following: > 1. Check whether there is a known problem in my version of Apache > (it's the Leopard standard version, so it might be old enough) > 2. Reconfigure/recompile the server, to make it work in 32-bit mode > > Any idea additional is welcome, of course! AFAIK Apache itself wouldn't be generating those packets directly. It just uses a socket and the standard operating system network library socket functions. If there is an issue with packets at the TCP/IP level, it would be an operating system level issue. FWIW, when I am doing benchmarking on MacOSX I often see 'ab' stall when trying to do even moderate size test runs of 1 requests. I have never bothered to go into it enough detail to work out the cause, although for that case also don't believe it was Apache. If anything was blaming 'ab', but wouldn't be surprised if it was a network level issue, even though I was doing it against server on same box. Graham --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: File upload failing occasionally
I tried to monitor the interface using WireShark and the communication just hangs during a data transfer: Apache is just acknowledging the receipt of some packets and then just seems to hang. The suspicious bit is a fairly long series of http RST about 20ms before the server hangs. Though I'm not knowledgeable enough on http, to me this clearly points in the direction of an Apache problem, though I'm not sure if and how it can be solved. I will try the following: 1. Check whether there is a known problem in my version of Apache (it's the Leopard standard version, so it might be old enough) 2. Reconfigure/recompile the server, to make it work in 32-bit mode Any idea additional is welcome, of course! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: File upload failing occasionally
On Jan 5, 8:33 pm, ppdowrote: > The failure does not depend on the file size: I manage to upload files > bigger than 60 MB and fail on 30 kB files. > > I tried with Safari and Firefox and I got the same results, though I > got the *impression* (but given the randomness of the problem I am not > sure this is a real observation) that it fails more often if I use the > client on the same machine as the server and less if the client runs > from a different machine. Both client and server were all OSX/Leopard. > > In the Apache configuration there is no explicit timeout directive. > However, when the problem occurs the client hangs for a very long time > before reporting the error: from the server logs it takes about 5 > minutes. When it succeeds the operation completes within a few > seconds, depending on the size of the file. The default for Timeout if not set is 300 seconds (5 minutes). >From Apache documentation: """The TimeOut directive currently defines the amount of time Apache will wait for three things: The total amount of time it takes to receive a GET request. The amount of time between receipt of TCP packets on a POST or PUT request. The amount of time between ACKs on transmissions of TCP packets in responses.""" The timeout is significant here as the error in logs for mod_wsgi (which is more informative than mod_python and why I asked about it) was: [Mon Jan 05 08:13:06 2009] [error] [client 127.0.0.1] (70007)The timeout specified has expired: mod_wsgi (pid=46687): Unable to get bucket brigade for request., referer: http://127.0.0.1/myapp/upload/2009-01-03/1 So, what would appear to be happening is that at the client socket itself where Apache is applying this timeout, one of those three events is occurring. The first actually probably applies to POST request as well. What it actually means is how long it takes to read the headers of HTTP request. If the client is very slow providing the full headers it would timeout. It can't be that in this case though as the higher mod_wsgi code would never have been called in the first place if the full request headers hadn't been read. The others are more serious lower level socket network issues. I can't think of any other reason that this timeout would be triggered within Apache at a higher level than the client socket. All I can probably suggest at this point is use tcpdump to monitor the traffic to port 80 for some of these requests and see if you can capture what is occurring at network layer for socket when it hangs. >From that might at least confirm if data is getting to Apache. If you know enough about networking, maybe you can tell if all the correct protocol level messages for TCP are there. Graham --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: File upload failing occasionally
The failure does not depend on the file size: I manage to upload files bigger than 60 MB and fail on 30 kB files. I tried with Safari and Firefox and I got the same results, though I got the *impression* (but given the randomness of the problem I am not sure this is a real observation) that it fails more often if I use the client on the same machine as the server and less if the client runs from a different machine. Both client and server were all OSX/Leopard. In the Apache configuration there is no explicit timeout directive. However, when the problem occurs the client hangs for a very long time before reporting the error: from the server logs it takes about 5 minutes. When it succeeds the operation completes within a few seconds, depending on the size of the file. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~--~~~~--~~--~--~---
Re: File upload failing occasionally
Sorry, some more questions. How big are the files you are trying to upload? What browser/client are you using to do the uploads? What do you have Timeout directive set to in Apache configuration? Graham On Jan 5, 6:24 pm, ppdowrote: > Hi, > > Thanks for looking into this. I used embedded mode and the error is > the following (different message, but same place in > multipartparser.py): > > IOError at /myapp/upload/2009-01-03/1 > request data read error > Request Method: POST > Request URL: http://127.0.0.1/myapp/upload/2009-01-03/1 > Exception Type: IOError > Exception Value: > request data read error > Exception Location: /Library/Frameworks/Python64.framework/Versions/ > 2.6/lib/python2.6/site-packages/django/http/multipartparser.py in > read, line 406 > Python Executable: /usr/sbin/httpd > Python Version: 2.6.1 > Python Path: ['/djangoapps/', '/Library/Frameworks/Python64.framework/ > Versions/2.6/lib/python26.zip', '/Library/Frameworks/ > Python64.framework/Versions/2.6/lib/python2.6', '/Library/Frameworks/ > Python64.framework/Versions/2.6/lib/python2.6/plat-darwin', '/Library/ > Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat-mac', '/ > Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat- > mac/lib-scriptpackages', '/Library/Frameworks/Python64.framework/ > Versions/2.6/lib/python2.6/lib-tk', '/Library/Frameworks/ > Python64.framework/Versions/2.6/lib/python2.6/lib-old', '/Library/ > Frameworks/Python64.framework/Versions/2.6/lib/python2.6/lib-dynload', > '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/ > site-packages'] > Server time: Mon, 5 Jan 2009 08:20:20 +0100 > > Environment: > > Request Method: POST > Request URL:http://127.0.0.1/myapp/upload/2009-01-03/1 > Django Version: 1.0.2 final > Python Version: 2.6.1 > Installed Applications: > ['django.contrib.auth', > 'django.contrib.contenttypes', > 'django.contrib.sessions', > 'django.contrib.sites', > 'django.contrib.admin', > 'systemic.peergroup'] > Installed Middleware: > ('django.middleware.common.CommonMiddleware', > 'django.contrib.sessions.middleware.SessionMiddleware', > 'django.contrib.auth.middleware.AuthenticationMiddleware') > > Traceback: > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/core/handlers/base.py" in get_response > 86. response = callback(request, *callback_args, > **callback_kwargs) > File "/djangoapps/myapp/../myapp/application1/views.py" in upload_file > 137. form = UploadFileForm(request.POST, request.FILES) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/core/handlers/wsgi.py" in _get_post > 169. self._load_post_and_files() > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/core/handlers/wsgi.py" in > _load_post_and_files > 135. self._post, self._files = > self.parse_file_upload(self.META, self.environ['wsgi.input']) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/__init__.py" in parse_file_upload > 124. return parser.parse() > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in parse > 134. for item_type, meta_data, field_stream in Parser > (stream, self._boundary): > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in __iter__ > 607. for sub_stream in boundarystream: > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in next > 421. return LazyStream(BoundaryIter(self._stream, > self._boundary)) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in __init__ > 447. unused_char = self._stream.read(1) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in read > 300. out = ''.join(parts()) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in parts > 293. chunk = self.next() > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in next > 315. output = self._producer.next() > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in next > 376. data = self.flo.read(self.chunk_size) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in read > 406. return self._file.read(num_bytes) > > Exception Type: IOError at
Re: File upload failing occasionally
Hi, Thanks for looking into this. I used embedded mode and the error is the following (different message, but same place in multipartparser.py): IOError at /myapp/upload/2009-01-03/1 request data read error Request Method: POST Request URL:http://127.0.0.1/myapp/upload/2009-01-03/1 Exception Type: IOError Exception Value: request data read error Exception Location: /Library/Frameworks/Python64.framework/Versions/ 2.6/lib/python2.6/site-packages/django/http/multipartparser.py in read, line 406 Python Executable: /usr/sbin/httpd Python Version: 2.6.1 Python Path:['/djangoapps/', '/Library/Frameworks/Python64.framework/ Versions/2.6/lib/python26.zip', '/Library/Frameworks/ Python64.framework/Versions/2.6/lib/python2.6', '/Library/Frameworks/ Python64.framework/Versions/2.6/lib/python2.6/plat-darwin', '/Library/ Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat-mac', '/ Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat- mac/lib-scriptpackages', '/Library/Frameworks/Python64.framework/ Versions/2.6/lib/python2.6/lib-tk', '/Library/Frameworks/ Python64.framework/Versions/2.6/lib/python2.6/lib-old', '/Library/ Frameworks/Python64.framework/Versions/2.6/lib/python2.6/lib-dynload', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/ site-packages'] Server time:Mon, 5 Jan 2009 08:20:20 +0100 Environment: Request Method: POST Request URL: http://127.0.0.1/myapp/upload/2009-01-03/1 Django Version: 1.0.2 final Python Version: 2.6.1 Installed Applications: ['django.contrib.auth', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.sites', 'django.contrib.admin', 'systemic.peergroup'] Installed Middleware: ('django.middleware.common.CommonMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware') Traceback: File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/core/handlers/base.py" in get_response 86. response = callback(request, *callback_args, **callback_kwargs) File "/djangoapps/myapp/../myapp/application1/views.py" in upload_file 137. form = UploadFileForm(request.POST, request.FILES) File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/core/handlers/wsgi.py" in _get_post 169. self._load_post_and_files() File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/core/handlers/wsgi.py" in _load_post_and_files 135. self._post, self._files = self.parse_file_upload(self.META, self.environ['wsgi.input']) File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/__init__.py" in parse_file_upload 124. return parser.parse() File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in parse 134. for item_type, meta_data, field_stream in Parser (stream, self._boundary): File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in __iter__ 607. for sub_stream in boundarystream: File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in next 421. return LazyStream(BoundaryIter(self._stream, self._boundary)) File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in __init__ 447. unused_char = self._stream.read(1) File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in read 300. out = ''.join(parts()) File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in parts 293. chunk = self.next() File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in next 315. output = self._producer.next() File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in next 376. data = self.flo.read(self.chunk_size) File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ python2.6/site-packages/django/http/multipartparser.py" in read 406. return self._file.read(num_bytes) Exception Type: IOError at /myapp/upload/2009-01-03/1 Exception Value: request data read error I tried to switch to daemon mode but no behavioral differences: The configuration I use in apache (basically coming from http://code.djangoproject.com/wiki/django_apache_and_mod_wsgi) is the following: Alias /djadmin/ "/adminmedia/" Order allow,deny Options Indexes Allow from all IndexOptions FancyIndexing WSGIDaemonProcess site-1 user=auser group=agroup threads=25
Re: File upload failing occasionally
On Jan 5, 10:24 am, ppdowrote: > Hi all, > > I am trying to port my first Django 1.0.2 application to run on OSX/ > Leopard with Apache + mod_python 3.3.1 + python 2.6.1 (all running in > 64-bit mode) and I am experiencing an occasional error when uploading > a file that was not present when testing with the Django development > server. > > The code for the upload is similar to what described in the Django > documentation: > > class UploadFileForm(forms.Form): > file = forms.FileField() > description = forms.CharField(max_length=100) > notifygroup = forms.BooleanField(label='Notify Group?', > required=False) > > def upload_file(request, date, meetingid ): > print date, meetingid > if request.method == 'POST': > print 'before reloading the form...' > form = UploadFileForm(request.POST, request.FILES) > print 'after reloading the form' > if form.is_valid(): > try: > handle_uploaded_file(request.FILES['file'], > request.REQUEST['date'], request.REQUEST['description'], > form.cleaned_data['notifygroup'], meetingid ) > except: > return render_to_response('uploaded.html', > { 'message': 'Error! File not uploaded!' }) > return HttpResponseRedirect('/myapp/uploaded/') > else: > form = UploadFileForm() > return render_to_response('upload.html', {'form': form, > 'date':date, 'meetingid':meetingid}) > > This code normally works correctly, but sometimes (say, once every 10 > uploads) and after a fairly long waiting time, it fails with the > following error: > > IOError at /myapp/upload/2009-01-03/1 > Client read error (Timeout?) > Request Method: POST > Request URL: http://192.168.0.164/myapp/upload/2009-01-03/1 > Exception Type: IOError > Exception Value: > Client read error (Timeout?) > Exception Location: /Library/Frameworks/Python64.framework/Versions/ > 2.6/lib/python2.6/site-packages/django/http/multipartparser.py in > read, line 406 > Python Executable: /usr/sbin/httpd > Python Version: 2.6.1 > Python Path: ['/djangoapps/myapp/', '/djangoapps/', '/Library/ > Frameworks/Python64.framework/Versions/2.6/lib/python26.zip', '/ > Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6', '/ > Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/plat- > darwin', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/plat-mac', '/Library/Frameworks/Python64.framework/Versions/ > 2.6/lib/python2.6/plat-mac/lib-scriptpackages', '/Library/Frameworks/ > Python64.framework/Versions/2.6/lib/python2.6/lib-tk', '/Library/ > Frameworks/Python64.framework/Versions/2.6/lib/python2.6/lib-old', '/ > Library/Frameworks/Python64.framework/Versions/2.6/lib/python2.6/lib- > dynload', '/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages'] > Server time: Sun, 4 Jan 2009 22:42:04 +0100 > > Environment: > > Request Method: POST > Request URL:http://192.168.0.164/myapp/upload/2009-01-03/1 > Django Version: 1.0.2 final > Python Version: 2.6.1 > Installed Applications: > ['django.contrib.auth', > 'django.contrib.contenttypes', > 'django.contrib.sessions', > 'django.contrib.sites', > 'django.contrib.admin', > 'myapp.application1'] > Installed Middleware: > ('django.middleware.common.CommonMiddleware', > 'django.contrib.sessions.middleware.SessionMiddleware', > 'django.contrib.auth.middleware.AuthenticationMiddleware') > > Traceback: > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/core/handlers/base.py" in get_response > 86. response = callback(request, *callback_args, > **callback_kwargs) > File "/djangoapps/myapp/../myapp/application1/views.py" in upload_file > 137. form = UploadFileForm(request.POST, request.FILES) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/core/handlers/modpython.py" in > _get_post > 113. self._load_post_and_files() > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/core/handlers/modpython.py" in > _load_post_and_files > 87. self._post, self._files = self.parse_file_upload > (self.META, self._req) > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/__init__.py" in parse_file_upload > 124. return parser.parse() > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in parse > 134. for item_type, meta_data, field_stream in Parser > (stream, self._boundary): > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ > python2.6/site-packages/django/http/multipartparser.py" in __iter__ > 607. for sub_stream in boundarystream: > File "/Library/Frameworks/Python64.framework/Versions/2.6/lib/ >