[Python-Dev] 2.7.4
It's been almost a year since 2.7.3, so it's time for another 2.7 bugfix release. 2013-02-02 - 2.7.4 release branch created; rc released 2013-02-16 - 2.7.4 released Does this work for you, Martin and Ned? -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7.4
In article capzv6o8msw1dtff7r3qvmw-to_aa4wob8_4gc_va7wjdvge...@mail.gmail.com, Benjamin Peterson benja...@python.org wrote: It's been almost a year since 2.7.3, so it's time for another 2.7 bugfix release. 2013-02-02 - 2.7.4 release branch created; rc released 2013-02-16 - 2.7.4 released Does this work for you, Martin and Ned? That works for me. There are also several pending issues that I want to get into 2.7.4 (and 3.2.4). That should be enough time to finish them. -- Ned Deily, n...@acm.org ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7.4
On Sat, Jan 19, 2013 at 9:30 PM, Benjamin Peterson benja...@python.org wrote: It's been almost a year since 2.7.3, so it's time for another 2.7 bugfix release. 2013-02-02 - 2.7.4 release branch created; rc released 2013-02-16 - 2.7.4 released The Cape Town Python User Group is having a Python Bug Day next weekend (26th 27th January) [1]. If there any particular 2.7 bugs that need looking at please shout, otherwise we'll just work from the top of the list of 2.7 issues on the bug tracker. [1] http://ctpug.org.za/wiki/Sprint20130126 Schiavo Simon ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.7.4
On Sat, 2013-01-19 at 14:30 -0500, Benjamin Peterson wrote: It's been almost a year since 2.7.3, so it's time for another 2.7 bugfix release. 2013-02-02 - 2.7.4 release branch created; rc released 2013-02-16 - 2.7.4 released Does this work for you, Martin and Ned? I have a pet issue that has a patch that requires application to the 2.7 branch, if anyone would be kind enough to do it: http://bugs.python.org/issue15881 It has already been applied to various 3.X branches. - C ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Modules/socketmodule.c: avoiding second fcntl() call worth the effort?
Hello folks, I noticed while stracing a process that sock.setblocking() calls always result in pairs of fcntl() calls on Linux. Checking 2.6.8, 2.7.3, and 3.3.0 Modules/socketmodule.c, the code seems to use the following (unless I have missed something): delay_flag = fcntl(s-sock_fd, F_GETFL, 0); if (block) delay_flag = (~O_NONBLOCK); else delay_flag |= O_NONBLOCK; fcntl(s-sock_fd, F_SETFL, delay_flag); Perhaps a check to see the flags changed might be worth making? int orig_delay_flag = fcntl(s-sock_fd, F_GETFL, 0); if (block) delay_flag = orig_delay_flag (~O_NONBLOCK); else delay_flag = orig_delay_flag | O_NONBLOCK; if (delay_flag != orig_delay_flag) fcntl(s-sock_fd, F_SETFL, delay_flag); OpenStack Swift using the Eventlet module, which sets the accepted socket non-blocking, resulting in twice the number of fcntl() calls. Not a killer on performance, but it seems simple enough to save a system call here. Thanks for your consideration, -peter ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Modules/socketmodule.c: avoiding second fcntl() call worth the effort?
2013/1/19 Peter Portante peter.a.porta...@gmail.com: Hello folks, I noticed while stracing a process that sock.setblocking() calls always result in pairs of fcntl() calls on Linux. Checking 2.6.8, 2.7.3, and 3.3.0 Modules/socketmodule.c, the code seems to use the following (unless I have missed something): delay_flag = fcntl(s-sock_fd, F_GETFL, 0); if (block) delay_flag = (~O_NONBLOCK); else delay_flag |= O_NONBLOCK; fcntl(s-sock_fd, F_SETFL, delay_flag); Perhaps a check to see the flags changed might be worth making? Considering most sockets are only set to blocking once, this doesn't seem very useful. -- Regards, Benjamin ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Modules/socketmodule.c: avoiding second fcntl() call worth the effort?
On Sat, Jan 19, 2013 at 8:49 PM, Peter Portante peter.a.porta...@gmail.com wrote: I noticed while stracing a process that sock.setblocking() calls always result in pairs of fcntl() calls on Linux. Checking 2.6.8, 2.7.3, and 3.3.0 Modules/socketmodule.c, the code seems to use the following (unless I have missed something): delay_flag = fcntl(s-sock_fd, F_GETFL, 0); if (block) delay_flag = (~O_NONBLOCK); else delay_flag |= O_NONBLOCK; fcntl(s-sock_fd, F_SETFL, delay_flag); Perhaps a check to see the flags changed might be worth making? int orig_delay_flag = fcntl(s-sock_fd, F_GETFL, 0); if (block) delay_flag = orig_delay_flag (~O_NONBLOCK); else delay_flag = orig_delay_flag | O_NONBLOCK; if (delay_flag != orig_delay_flag) fcntl(s-sock_fd, F_SETFL, delay_flag); OpenStack Swift using the Eventlet module, which sets the accepted socket non-blocking, resulting in twice the number of fcntl() calls. Not a killer on performance, but it seems simple enough to save a system call here. This would seem to be a simple enough fix, but it seems you are only fixing it if a *redundant* call to setblocking() is made (i.e. one that attempts to set the flag to the value it already has). Why would this be a common pattern? Even if it was, is the cost of one extra fcntl() call really worth making the code more complex? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Modules/socketmodule.c: avoiding second fcntl() call worth the effort?
I don't have a concrete case where a socket object's setblocking() method
is called with a value in one module, handed off to another module (which
does not know what the first did with it) which in turn also calls
setblocking() with the same value. It certainly seems that that not is a
common pattern, but perhaps one could argue a valid pattern, since the
state of blocking/nonblocking is maintained in the kernel behind the
fcntl() system calls.
Here is what I am seeing concretely.
This is the syscall pattern from eventlet/wsgi.py + eventlet/greenio.py
(after removing redundants call to set_nonblocking (see
https://bitbucket.org/portante/eventlet/commits/cc27508f4bbaaea566aecb51cf6c8b4629b083bd)).
First, these are the call stacks for the three calls to the
set_nonblocking() method, made in one HTTP request; the
greenio.py:set_nonblocking() method wraps the socketmodule.c:setblocking()
method:
pid 1385
File /usr/bin/swift-object-server, line 22, in module
run_wsgi(conf_file, 'object-server', default_port=6000, **options)
File /usr/lib/python2.6/site-packages/swift/common/wsgi.py, line 194,
in run_wsgi
run_server(max_clients)
File /usr/lib/python2.6/site-packages/swift/common/wsgi.py, line 158,
in run_server
wsgi.server(sock, app, NullLogger(), custom_pool=pool)
File /usr/lib/python2.6/site-packages/eventlet/wsgi.py, line 598, in
server
*client_socket = sock.accept()*
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 163, in
accept
return type(self)(client), addr
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 133, in
__init__
*set_nonblocking(fd)*
pid 1385
File /usr/lib/python2.6/site-packages/eventlet/greenpool.py, line 80,
in _spawn_n_impl
func(*args, **kwargs)
File /usr/lib/python2.6/site-packages/eventlet/wsgi.py, line 516, in
process_request
proto = self.protocol(socket, address, self)
File /usr/lib64/python2.6/SocketServer.py, line 616, in __init__
self.setup()
File /usr/lib/python2.6/site-packages/eventlet/wsgi.py, line 174, in
setup
*self.rfile = conn.makefile('rb', self.rbufsize)*
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 219, in
makefile
return _fileobject(self.dup(), *args, **kw)
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 214, in
dup
newsock = type(self)(sock)
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 133, in
__init__
*set_nonblocking(fd)*
pid 1385
File /usr/lib/python2.6/site-packages/eventlet/greenpool.py, line 80,
in _spawn_n_impl
func(*args, **kwargs)
File /usr/lib/python2.6/site-packages/eventlet/wsgi.py, line 516, in
process_request
proto = self.protocol(socket, address, self)
File /usr/lib64/python2.6/SocketServer.py, line 616, in __init__
self.setup()
File /usr/lib/python2.6/site-packages/eventlet/wsgi.py, line 175, in
setup
*self.wfile = conn.makefile('wb', self.wbufsize)*
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 219, in
makefile
return _fileobject(self.dup(), *args, **kw)
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 214, in
dup
newsock = type(self)(sock)
File /usr/lib/python2.6/site-packages/eventlet/greenio.py, line 133, in
__init__
*set_nonblocking(fd)*
The first one above is expected, the next two unexpectedly result in
fcntl() calls on the same fd. The strace looks like:
accept(8, {sa_family=AF_INET, sin_port=htons(54375),
sin_addr=inet_addr(127.0.0.1)}, [16]) = 14
fcntl(14, F_GETFL) = 0x2 (flags O_RDWR)
fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0
# *self.rfile = conn.makefile('rb', self.rbufsize)*
fcntl(14, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0
# *self.wfile = conn.makefile('wb', self.wbufsize)*
fcntl(14, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
fcntl(14, F_SETFL, O_RDWR|O_NONBLOCK) = 0
recvfrom(14, GET /sdb1/234456/AUTH_del0/gprfc..., 8192, 0, NULL, NULL) =
353
getsockname(14, {sa_family=AF_INET, sin_port=htons(6010),
sin_addr=inet_addr(127.0.0.1)}, [16]) = 0
...
It appears that conn.makefile() is attempting to dup() the fd, but rfile
and wfile end up with objects that share the same fd contained in conn.
For eventlet/wsgi.py based webservers, OpenStack Swift is the one I am
working with right now, handles millions of requests a day on our customer
systems. Seems like these suggested code changes are trivial compared to
the number of system calls that can be saved.
Thanks for indulging on this topic,
-peter
On Sun, Jan 20, 2013 at 12:38 AM, Guido van Rossum gu...@python.org wrote:
On Sat, Jan 19, 2013 at 8:49 PM, Peter Portante
peter.a.porta...@gmail.com wrote:
I noticed while stracing a process that sock.setblocking() calls always
result in pairs of fcntl() calls on Linux. Checking 2.6.8, 2.7.3, and
3.3.0
Modules/socketmodule.c, the code seems to use the following (unless I
have
missed
Re: [Python-Dev] 2.7.4
Am 19.01.2013 20:30, schrieb Benjamin Peterson: It's been almost a year since 2.7.3, so it's time for another 2.7 bugfix release. 2013-02-02 - 2.7.4 release branch created; rc released 2013-02-16 - 2.7.4 released Does this work for you, Martin and Ned? I would propose to sync this with 3.2.4 if there's no argument against it. Georg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
