Re: [Twisted-Python] IMAP4 fetch optimization
Please file a ticket in the issue tracker and attach a patch there. Patches sent to the list will just get forgotten about. Thanks. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] connectionLost never reached after calling loseConnection: stuck in CLOSE_WAIT forever
Hello Jean-Paul, thanks for looking into this. What did make a difference was to comment this line, the problem never happens without it: to.transport.registerProducer(_from.transport, True) This suggests that your problem is that you don't unregister the produce. The connection can never be closed as long as a producer is registered. Does your code ever unregister this producer? I am 100% sure that I unregister the producer from the transport right before I call loseConnection on that transport (see code snippet in my original post). Even using reactor.callLater for calling loseConnection asynchronously after the producer has been unregistered didn't help. Moreover, if it was just a problem of my code not unregistering the producer then this wouldn't explain why registering the producer as non-streaming fixes the issue. It also wouldn't explain why does this happen only in particular conditions and not always (even if I managed to reproduce it reliably, I still have to debug the exact external conditions that trigger it). As a side note, for once what you wrote is not 200% exact ;) because the connection is always closed properly if it's the peer closing it first, even if the producer is still registered in that case. (Maybe Twisted unregisters the producer by itself when this happens?) Anyway, I ran Twisted tests on my installation after the patch I mentioned in my previous mail and I got the same results as before applying it so at least it seems it doesn't break any obvious stuff. thank you, ciao ste ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] connectionLost never reached after calling loseConnection: stuck in CLOSE_WAIT forever
On 03:21 pm, s...@demaledetti.net wrote: Hello Jean-Paul, thanks for looking into this. What did make a difference was to comment this line, the problem never happens without it: to.transport.registerProducer(_from.transport, True) This suggests that your problem is that you don't unregister the produce. The connection can never be closed as long as a producer is registered. Does your code ever unregister this producer? I am 100% sure that I unregister the producer from the transport right before I call loseConnection on that transport (see code snippet in my original post). Even using reactor.callLater for calling loseConnection asynchronously after the producer has been unregistered didn't help. Okay. Then we're back to Glyph's idea, which is that there really is a bug to fix. :) Moreover, if it was just a problem of my code not unregistering the producer then this wouldn't explain why registering the producer as non-streaming fixes the issue. It also wouldn't explain why does this happen only in particular conditions and not always (even if I managed to reproduce it reliably, I still have to debug the exact external conditions that trigger it). Hopefully this will turn into a simple example which can reliably reproduce the problem for everyone. :) As a side note, for once what you wrote is not 200% exact ;) because the connection is always closed properly if it's the peer closing it first, even if the producer is still registered in that case. (Maybe Twisted unregisters the producer by itself when this happens?) Thanks for that clarification. Jean-Paul ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
[Twisted-Python] Getting a list of services and/or plugins are currently running
Hi all, I'm looking for a way to get a list of services and/or plugins that currently running on a machine. Does anyone have any suggestions on how to do that? Thanks. Liz ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Getting a list of services and/or plugins are currently running
On Oct 18, 2010, at 1:41 PM, Elizabeth Liao wrote: Hi all, I'm looking for a way to get a list of services and/or plugins that currently running on a machine. Does anyone have any suggestions on how to do that? Thanks. Liz This question is a bit too vague to be answered with any confidence. What do you mean by service? What do you mean by plugin? And what do you mean by running? These things have generic meanings and then more specific jargon-y meanings with respect to Twisted. Do you mean what twistd plugins are installed in my system Python, or what processes that use Twisted are currently running, or what Service objects are currently .running == True in my current process, or something else? ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Getting a list of services and/or plugins are currently running
Hi Elizabeth, What kind of services/plugins are you referring to? And is your platform you are referring to is an OS (windows/linux/mac) ? Elizabeth Liao wrote: Hi all, I'm looking for a way to get a list of services and/or plugins that currently running on a machine. Does anyone have any suggestions on how to do that? Thanks. Liz ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Getting a list of services and/or plugins are currently running
Sorry about that. I just started using twisted so I'll try to be more specific. We have 2 different twisted plugins that uses shared code. Each of these plugins is run as a TCPClient on separate machines. What would like to do is to somehow identify what twisted services/plugins are running because different things happen depending on what plugin is currently in use. We are running on Linux wanted to see if there was an easy way of doing that using the twisted libraries. I suppose one solution is to get a list of processes running and parse that but I would rather not do that. Thanks. Liz From: twisted-python-boun...@twistedmatrix.com [twisted-python-boun...@twistedmatrix.com] on behalf of Glyph Lefkowitz [gl...@twistedmatrix.com] Sent: Monday, October 18, 2010 1:44 PM To: Twisted general discussion Subject: Re: [Twisted-Python] Getting a list of services and/or plugins are currently running On Oct 18, 2010, at 1:41 PM, Elizabeth Liao wrote: Hi all, I'm looking for a way to get a list of services and/or plugins that currently running on a machine. Does anyone have any suggestions on how to do that? Thanks. Liz This question is a bit too vague to be answered with any confidence. What do you mean by service? What do you mean by plugin? And what do you mean by running? These things have generic meanings and then more specific jargon-y meanings with respect to Twisted. Do you mean what twistd plugins are installed in my system Python, or what processes that use Twisted are currently running, or what Service objects are currently .running == True in my current process, or something else? ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Getting a list of services and/or plugins are currently running
On Oct 18, 2010, at 3:10 PM, Elizabeth Liao wrote: Sorry about that. I just started using twisted so I'll try to be more specific. We have 2 different twisted plugins that uses shared code. Each of these plugins is run as a TCPClient on separate machines. What would like to do is to somehow identify what twisted services/plugins are running because different things happen depending on what plugin is currently in use. We are running on Linux wanted to see if there was an easy way of doing that using the twisted libraries. I suppose one solution is to get a list of processes running and parse that but I would rather not do that. Thanks. Liz Sorry, but there is no easy API to do this using Twisted. It's not really clear to me how such an API could be meaningful, either, given that any Twisted plugin could easily invoke any arbitrary python code that it wants to, including other Twisted plugins. But if you have a detailed specification for how it might work, please feel free to file a ticket. One way you could deal with this would be to have a dedicated UNIX socket at a well-known path for your particular application (e.g. /tmp/.your-app/global-socket), and connect to that socket to identify the current system's role. This requires a little more work, but has the advantage that if you go with an extensible protocol such as AMP, you can extend this communication mechanism to allow arbitrarily detailed interrogation depending on your needs in the future. If you wanted to go with the 'listing processes' route (it is definitely simpler), you don't have to parse it yourself. Something like this with the 'psi' module (http://pypi.python.org/pypi/PSI) might do what you want: from psi.process import ProcessTable pt = ProcessTable() for p in pt.values(): ... if 'twistd your-app' in p.command: ... print p.pid ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Getting a list of services and/or plugins are currently running
On 10/18/2010 09:10 PM, Elizabeth Liao wrote: [...] What would like to do is to somehow identify what twisted services/plugins are running because different things happen depending on what plugin is currently in use. We are running on Linux wanted to see if there was an easy way of doing that using the twisted libraries. I suppose one solution is to get a list of processes running and parse that but I would rather not do that. Hi Elizabeth, I don't think (and meanwhile know, because Glyph already answered) there's a ready-made solution for your requirements (inside twisted, at least), but it should be relatively straightforward to implement one using twisted. If I understand you correctly, you need an interface to query which plugins/services (aka t.a.s.Services) in which twisted processes are in which state on a given machine, or even all machines involved in your setup. Since you can have several machines running several twisted processes, where each process can have several plugins installed each in different states, and each plugin can offer several Services, which also can be in different states, (and even install/run other plugins and services ) twisted itself per default obviously cannot provide an interface to query all this information, if only because a given process just knows about itself. If I wanted to implement this, I'd start with a central service, that offers an interface for (twisted) apps/clients/servers to register themselves on startup. All participating plugins and services would of course have to know about that service, but depending on your requirements it might suffice to use a common subclass of twisted.application.service.Service, that takes care of performing the necessary steps to inform your central state-monitoring service about their existence and state. Johann ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Re: [Twisted-Python] Getting a list of services and/or plugins are currently running
On Mon, 2010-10-18 at 22:28 +0200, Johann Borck wrote: If I wanted to implement this, I'd start with a central service, that offers an interface for (twisted) apps/clients/servers to register themselves on startup. All participating plugins and services would of course have to know about that service, but depending on your requirements it might suffice to use a common subclass of twisted.application.service.Service, that takes care of performing the necessary steps to inform your central state-monitoring service about their existence and state. Something based on zeroconf mDNS (aka Rendevezous on Macs) would do this for you... and most modern linux versions include an implementation. ___ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
