Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-19 Thread Luke Marsden
Hi Juan, You could try: print reactor.__class__ This should allow you to check that the twistd command-line argument is working. Example: luke@pow:~$ twistd -noy foo.py |head -n 1 luke@pow:~$ twistd -r poll -noy foo.py |head -n 1

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-19 Thread Reza Lotun
Can you not do a: from twisted.python import log from twisted.internet import reactor log.msg('My reactor is: %r' % repr(reactor)) somewhere in your code to determine it? Are you running your application with twistd? You can pass a reactor implementation via the '-r' option. Reza 2011/5/19 Jua

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-19 Thread Juan Antonio Ibañez Santorum
Is there any way to check if it is using the the poll/epoll reactor? Regards 2011/4/26 Luke Marsden > On Tue, 2011-04-26 at 17:42 +0200, Juan Antonio Ibañez Santorum wrote: > > Hello! > >I have a Linux box serving my twisted app. All works ok except > > after some days running (where I can

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-10 Thread Juan Antonio Ibañez Santorum
2011/5/10 Itamar Turner-Trauring > > Still having problems... > > > > I can see that twistd process has a higher number than 1024 looking at > > /proc/pid/limits but whe 1024 descriptors number gets reached the system > > becomes unstable. It also has been launche using '-r epoll' option. Any > >

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-10 Thread Itamar Turner-Trauring
> Still having problems... > > I can see that twistd process has a higher number than 1024 looking at > /proc/pid/limits but whe 1024 descriptors number gets reached the system > becomes unstable. It also has been launche using '-r epoll' option. Any > other idea? 0. What does "unstable" mean? 1.

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-10 Thread Reza Lotun
Hmm, are you running your process as root? If so, you may need to add the following two lines in /etc/security/limits.conf: root       soft    nofile      99 root       hard    nofile      99 (yes, root, even though you previously supplied '*'). I'd also reboot just to be sure. If you

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-10 Thread Juan Antonio Ibañez Santorum
Still having problems... I can see that twistd process has a higher number than 1024 looking at /proc/pid/limits but whe 1024 descriptors number gets reached the system becomes unstable. It also has been launche using '-r epoll' option. Any other idea? 2011/5/5 Reza Lotun > 2011/5/5 Juan Anto

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-09 Thread Juan Antonio Ibañez Santorum
+1 Perfect explanation. Thank you very much! 2011/5/8 Reza Lotun > Hi Juan, > > 2011/5/8 Juan Antonio Ibañez Santorum > >> MUST I apply that change or may be I SHOULD do it? Why is it necessary? > > > Each network connection on your system is represented by a 'file > descriptor', in Unix termi

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-08 Thread Reza Lotun
Hi Juan, 2011/5/8 Juan Antonio Ibañez Santorum > MUST I apply that change or may be I SHOULD do it? Why is it necessary? Each network connection on your system is represented by a 'file descriptor', in Unix terminology. By default, there is a limit on the number of open file descriptors that c

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-08 Thread Juan Antonio Ibañez Santorum
MUST I apply that change or may be I SHOULD do it? Why is it necessary? Regards 2011/5/6 Reza Lotun > Small addition - I forgot to say to open /etc/security/limits.conf and then > place the following two lines in step 3 in: > > 3. Place the following two lines: >> * softnofile 99

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-05 Thread Reza Lotun
Small addition - I forgot to say to open /etc/security/limits.conf and then place the following two lines in step 3 in: 3. Place the following two lines: > * softnofile 99 > * hardnofile 99 > Either reboot, or also type in `ulimit -n 99` > Also, I'd be ha

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-05 Thread Juan Antonio Ibañez Santorum
Thank you Reza! 2011/5/5 Reza Lotun > 2011/5/5 Juan Antonio Ibañez Santorum > >> I had got the same problem using '-r epoll' options when starting the app >> via twistd... When 1024 descriptors are opened, I can see 100% CPU. >> >> Any ideas? >> >> > 1. Edit /etc/sysctl.conf and place the follo

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-05 Thread Glyph Lefkowitz
On May 5, 2011, at 10:52 AM, Reza Lotun wrote: > 2011/5/5 Juan Antonio Ibañez Santorum > I had got the same problem using '-r epoll' options when starting the app via > twistd... When 1024 descriptors are opened, I can see 100% CPU. > > Any ideas? > > > 1. Edit /etc/sysctl.conf and place th

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-05 Thread Reza Lotun
2011/5/5 Juan Antonio Ibañez Santorum > I had got the same problem using '-r epoll' options when starting the app > via twistd... When 1024 descriptors are opened, I can see 100% CPU. > > Any ideas? > > 1. Edit /etc/sysctl.conf and place the following line there: fs.file-max = 99 2. Ty

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-05-05 Thread Juan Antonio Ibañez Santorum
I had got the same problem using '-r epoll' options when starting the app via twistd... When 1024 descriptors are opened, I can see 100% CPU. Any ideas? 2011/4/26 Luke Marsden > On Tue, 2011-04-26 at 17:42 +0200, Juan Antonio Ibañez Santorum wrote: > > Hello! > >I have a Linux box serving m

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-27 Thread Ray Cote
No. It's recommended to fix bugs in Twisted that affect you :). Well, that too. :} ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-27 Thread Juan Antonio Ibañez Santorum
Thank you very much for your tip Patrick! 2011/4/27 Patrick Mylund Nielsen > That is sufficient, yes. > > You might even want to make that 'epoll' since it scales better than > poll (assuming you're running on Linux 2.6+ only) > > Best, > Patrick > > 2011/4/27 Juan Antonio Ibañez Santorum : > >

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-27 Thread Patrick Mylund Nielsen
That is sufficient, yes. You might even want to make that 'epoll' since it scales better than poll (assuming you're running on Linux 2.6+ only) Best, Patrick 2011/4/27 Juan Antonio Ibañez Santorum : > Must I make any change appart from start the app with twistd plus '-r poll'? > > 2011/4/27 Luke

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Juan Antonio Ibañez Santorum
Must I make any change appart from start the app with twistd plus '-r poll'? 2011/4/27 Luke Marsden > >> 2011/4/26 Luke Marsden > >> You can also switch to using the "poll" reactor > > On Wed, 2011-04-27 at 08:01 +0200, Juan Antonio Ibañez Santorum wrote: > > Is that the unique way to get mor t

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Luke Marsden
>> 2011/4/26 Luke Marsden >> You can also switch to using the "poll" reactor On Wed, 2011-04-27 at 08:01 +0200, Juan Antonio Ibañez Santorum wrote: > Is that the unique way to get mor than 1024 simultaneous connections? > > > Regads > On Linux and BSD, yes. See http://twistedmatrix.com/docume

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Juan Antonio Ibañez Santorum
Is that the unique way to get mor than 1024 simultaneous connections? Regads 2011/4/26 Luke Marsden > On Tue, 2011-04-26 at 17:42 +0200, Juan Antonio Ibañez Santorum wrote: > > Hello! > >I have a Linux box serving my twisted app. All works ok except > > after some days running (where I can

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Glyph Lefkowitz
On Apr 26, 2011, at 4:34 PM, Ray Cote wrote: > - Original Message - >> From: "Luke Marsden" >> To: "Twisted general discussion" >> Sent: Tuesday, April 26, 2011 2:08:40 PM >> Subject: Re: [Twisted-Python] 100% CPU on high opened descrip

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Ray Cote
- Original Message - > From: "Luke Marsden" > To: "Twisted general discussion" > Sent: Tuesday, April 26, 2011 2:08:40 PM > Subject: Re: [Twisted-Python] 100% CPU on high opened descriptors > On Tue, 2011-04-26 at 17:42 +0200, Juan Antonio Ibañez Santo

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Luke Marsden
On Tue, 2011-04-26 at 17:42 +0200, Juan Antonio Ibañez Santorum wrote: > Hello! >I have a Linux box serving my twisted app. All works ok except > after some days running (where I can see my CPU going to 100%). I saw > that CPU goes 100% usage when the app reaches to 1024 opened > descriptors (s

Re: [Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Glyph Lefkowitz
On Apr 26, 2011, at 11:42 AM, Juan Antonio Ibañez Santorum wrote: >I have a Linux box serving my twisted app. All works ok except after some > days running (where I can see my CPU going to 100%). I saw that CPU goes 100% > usage when the app reaches to 1024 opened descriptors (sockets). It s

[Twisted-Python] 100% CPU on high opened descriptors

2011-04-26 Thread Juan Antonio Ibañez Santorum
Hello! I have a Linux box serving my twisted app. All works ok except after some days running (where I can see my CPU going to 100%). I saw that CPU goes 100% usage when the app reaches to 1024 opened descriptors (sockets). It seems TM isn't closing some connections when clients loose connectio