[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Benjamin Ash]

Benjamin Ash added the comment:

Ok, thanks for quick followup. I didn't realize that the patch for Python-3, 
sorry about that.

The issue  I had was due to never calling self.listen() in the constructor of 
my server (asyncore.dispatcher). If this is not done the CPU spikes to 100%.

Thanks again.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Charles-François Natali]

Charles-François Natali added the comment:

> Using handle_accept() in my code and remembering to call listen() in my
> asyncore.dispatcher server's constructor did the trick.
>
> I am not sure if we still have a bug here though, since if the subclass
> doesn't define  a proper handle_accept() we get into the select() loop and
> 100% CPU utilization after the initial client connection.

No, it's not a bug.
The attached test case was for Python 3: Python 2 doesn't have
handle_accepted(), and since the default implementation of
handle_accept() doesn't nothing, the handler is called in a loop,
because the socket is effectively always ready for accept.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Benjamin Ash]

Benjamin Ash added the comment:

After doing a bit more testing, I was able to prevent the problem from 
occurring in asyncore_test.py with the following patch:

--- /proc/self/fd/112013-02-04 11:24:41.298347199 -0500
+++ asyncore_test.py2013-02-04 11:24:40.393318513 -0500
@@ -19,10 +19,18 @@
 self.bind(sock)
 self.listen(5)
 
-def handle_accepted(self, sock, addr):
-print('Incoming connection from %s' % repr(addr))
-handler = EchoHandler(sock)
+def handle_accept(self):
+pair = self.accept()
+if pair is not None:
+(sock, addr) = pair
+print('Incoming connection from %s' % repr(addr))
+handler = EchoHandler(sock)

Using handle_accept() in my code and remembering to call listen() in my 
asyncore.dispatcher server's constructor did the trick.

I am not sure if we still have a bug here though, since if the subclass doesn't 
define  a proper handle_accept() we get into the select() loop and 100% CPU 
utilization after the initial client connection.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Benjamin Ash]

Benjamin Ash added the comment:

Hi Charles-François,

I am using a recent version of Python-2.7 that does in fact contains this patch 
http://hg.python.org/cpython/rev/16bc59d37866:
  python-2.7.3-4.fc16.x86_64 (Fedora 16)

The CPU usage spikes after I make the initial client connection to the socket.

I am able to reproduce the issue using the attached script 'asyncore_test.py', 
and any of the following:
 - simple asyncore.dispatcher_with_send client.
 - simple asyncore.dispatcher client.
 - a direct socket.socket() connection.

Not sure what might be going wrong here. Any help would be most appreciated.

Thanks,

-ben

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Charles-François Natali]

Charles-François Natali added the comment:

I'm unable to reproduce it.
Are you using the attached script?
Did you apply the patch by hand, or are you using a recent Python version?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Benjamin Ash]

Benjamin Ash added the comment:

Hi,

I am still seeing this issue even with the patches applied. I see 100% CPU 
utilization, and strace shows the process is in same kind of select() loop as 
msg139899.

Any help would be greatly appreciated.

Thanks,

-ben

strace output:
"""
select(4, [3], [3], [3], {30, 0})   = 1 (out [3], left {29, 98})
select(4, [3], [3], [3], {30, 0})   = 1 (out [3], left {29, 98})
select(4, [3], [3], [3], {30, 0})   = 1 (out [3], left {29, 98})
select(4, [3], [3], [3], {30, 0})   = 1 (out [3], left {29, 98})
"""

--
nosy: +Benjamin.Ash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[STINNER Victor]

STINNER Victor  added the comment:

Woops, I don't know how, but I removed a neologix's message:

"Patch committed, closing.
Alexey, thanks for reporting this.
(I'll open a separate issue to increase the tests coverage)."

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[STINNER Victor]

Changes by STINNER Victor :


--
Removed message: http://bugs.python.org/msg140388

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Giampaolo Rodola']

Changes by Giampaolo Rodola' :


--
versions:  -Python 3.1

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Giampaolo Rodola']

Changes by Giampaolo Rodola' :


--
versions: +Python 2.7, Python 3.1, Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Charles-François Natali]

Charles-François Natali  added the comment:

Patch committed, closing.
Alexey, thanks for reporting this.
(I'll open a separate issue to increase the tests coverage).

--
resolution:  -> fixed
stage:  -> committed/rejected
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Roundup Robot]

Roundup Robot  added the comment:

New changeset ca077f2672e3 by Charles-François Natali in branch 'default':
Merge - Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/ca077f2672e3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Roundup Robot]

Roundup Robot  added the comment:

New changeset ed90c1c8ee62 by Charles-François Natali in branch '3.2':
Merge - Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/ed90c1c8ee62

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 42ec507815d2 by Charles-François Natali in branch '3.1':
Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/42ec507815d2

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Roundup Robot]

Roundup Robot  added the comment:

New changeset 16bc59d37866 by Charles-François Natali in branch '2.7':
Issue #12502: asyncore: fix polling loop with AF_UNIX sockets.
http://hg.python.org/cpython/rev/16bc59d37866

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Giampaolo Rodola']

Giampaolo Rodola'  added the comment:

Sorry for delay. Yes, patch looks ok to me and some tests for AF_UNIX sockets 
would definitively be welcome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Charles-François Natali]

Charles-François Natali  added the comment:

> Looks good, the patch seems to fix the problem.

Alright.
Giampaolo, do you agree?
Also, I noticed that test_asyncore currently only tests AF_INET sockets.
I'm working a patch to add AF_UNIX (and maybe AF_INET6) tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Ross Lagerwall]

Ross Lagerwall  added the comment:

Looks good, the patch seems to fix the problem.
This section of code indicates that the accepting socket shouldn't be in the 
write set...

def handle_write_event(self):
if self.accepting:
# Accepting sockets shouldn't get a write event.
# We will pretend it didn't happen.
return

--
nosy: +rosslagerwall

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Charles-François Natali]

Charles-François Natali  added the comment:

It's looping in Lib/asyncore.py:poll

select(4, [3], [3], [3], {30, 0})   = 1 (out [3], left {29, 94})
select(4, [3], [3], [3], {30, 0})   = 1 (out [3], left {29, 94})
select(4, [3], [3], [3], {30, 0})   = 1 (out [3], left {29, 94})

loop sets the Unix domain socket in the writable set, and contrarily to 
AF_INET/AF_INET6 sockets, bound AF_UNIX SOCK_STREAM sockets are reported as 
writable before any client connects to them, which triggers the loop.

I've attached a patch which just doesn't add the socket to the writable set if 
it's in the accepting state.
It fixes the loop, and doesn't seem to cause any regression in test_asyncore, 
but since it's the first time I'm looking at asyncore's code, I might very well 
have missed something :-)

--
keywords: +patch
nosy: +neologix
Added file: http://bugs.python.org/file22586/asyncore_unix_socket.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Antoine Pitrou]

Changes by Antoine Pitrou :


--
nosy: +giampaolo.rodola

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12502] 100% cpu usage when using asyncore with UNIX socket

[Алексей Агапитов]

New submission from Алексей Агапитов :

When using asyncore server with UNIX socket, I got 100% CPU usage.
I run modified code example from asyncore doc page.
This code was tested on two systems:

Ubuntu 10.04 2.6.32-32-generic #62-Ubuntu SMP

with two versions of Python:
Python 3.2 (r32:88445, Mar 29 2011, 08:55:36)

Python 3.2.1rc2 (default, Jul  5 2011, 20:33:19) Built from sources

and
Gentoo 2.6.36-hardened-r9 #6 SMP

with Python 3.1.3 (r313:86834, Mar 12 2011, 20:06:24)

I'm not sure, maybe it's because of the characteristics of UNIX socket?

--
components: Library (Lib)
files: asyncore_test.py
messages: 139898
nosy: Alexey.Agapitov
priority: normal
severity: normal
status: open
title: 100% cpu usage when using asyncore with UNIX socket
type: resource usage
versions: Python 3.2
Added file: http://bugs.python.org/file22585/asyncore_test.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com