[issue8771] Socket freezing under load issue on Mac.
Charles-Francois Natali neolo...@free.fr added the comment:
As explained by Jean-Paul, it's due to the fact that the closed TCP sockets
spend some time in TIME-WAIT state before being deallocated.
On Linux, this issue can be more or less worked-around using sysctl
(net.ipv4.tcp_tw_{reuse,recycle}). There might be something similar on OS-X.
It's definitely an OS-level tuning issue.
Suggesting to close.
--
nosy: +neologix
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8771
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
Alice Bevan-McGregor al...@gothcandy.com added the comment: Agreed; I'm now certain it's a local tuning issue. My first attempt to alter the file descriptor limits for local testing resulted in catastrophic system failure, though, so I have no clue as to the correct method to alter the TIME_WAIT time. I will continue to investigate, thank you for the lead. -- resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8771 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
Changes by Terry J. Reedy tjre...@udel.edu: -- nosy: -terry.reedy ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8771 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
Terry J. Reedy tjre...@udel.edu added the comment: This needs to be re-verified on a current version. -- nosy: +terry.reedy versions: +Python 2.7, Python 3.2 -Python 2.5, Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8771 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
Jean-Paul Calderone exar...@twistedmatrix.com added the comment: Have you looked at the number of TIME_WAIT sockets you have on the system when your benchmark gets to the 16000 request mark? This looks exactly like a regular TCP limitation to me. You'll find the limit on any platform, not just OS X. -- nosy: +exarkun ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8771 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
New submission from Alice Bevan-McGregor al...@gothcandy.com:
Using the wsgiref simple HTTP server or any other capable of 2000
requests/sec. demonstrates an issue with Macintosh sockets.
Mac OS X Version: 10.6.3 (Build 10D573)
Python Version: 2.6.1 (32-bit)
The minimal application needed to demonstrate the problem is:
import sys, cStringIO
from wsgiref.simple_server import make_server
sys.stderr = cStringIO.StringIO() # disable request logging
def app(environ, start_response):
start_response(200 OK, [('Content-Type', 'text/plain')])
return ['Hello world!\n']
httpd = make_server('', 8080, app)
httpd.serve_forever()
Then hammer the server using Apache Bench:
ab -n 2 -c 5 http://127.0.0.1:8080/
At almost exactly the 16000 request mark socket connections begin to time out.
Sockets are then freed up at the rate of about 40/second (on my box). Killing
the ab run when it freezes then immediately re-trying (and cancelling after a
few seconds) will show this rate. Time must pass for some connection 'pool' to
free the connections before you can do another 16000 requests.
This problem does not appear on the following setup:
Operating System: Gentoo Linux
Python Version: 2.6.4 (32-bit)
--
components: Library (Lib)
messages: 106117
nosy: amcgregor
priority: normal
severity: normal
status: open
title: Socket freezing under load issue on Mac.
type: behavior
versions: Python 2.6
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue8771
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
Alice Bevan-McGregor al...@gothcandy.com added the comment: I can confirm this issue also effecting 2.5.4 on my Mac. -- versions: +Python 2.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8771 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
R. David Murray rdmur...@bitdance.com added the comment: Why do you think this is a bug in Python? I'm not yet saying it isn't, but we'll need additional information to show that it is. Given that it works on Linux, it seems like the most likely case would be that it is an issue with the OS (perhaps requiring adjusting OS tuning parameters), since the Python socket library is a fairly thin wrapper around the OS socket library. -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8771 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8771] Socket freezing under load issue on Mac.
Alice Bevan-McGregor al...@gothcandy.com added the comment: Unfortunately, unless I can get instructions on how to properly diagnose socket libraries, I've exhausted my ability to debug this. I used to be a C programmer, but that was 12 years ago. I'm hoping to a) confirm the problem exists on Mac (not just my computer), and b) get someone familiar with Python's socket implementation and socket programming in general to figure out what's actually going on here. I also don't have access to Apple's radar bug tracker to check there. :( -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8771 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
