On Wed, Aug 12, 2009 at 12:01 PM, Roman Mindalev<[email protected]> wrote: >> >> Hi, >> I'm investigating about the reason of some huge memory leaks I've with >> the aim and icq transport and I've found that they both include their >> own oscar implementation instead of using twisted.words. The interface >> seems to be equal, besides that fact the one included in the >> transports uses a thread based scheduler for sending packets, with the >> scary comment "there is no way to shut down all threads when a user >> logs out". Is there any reason for this? >> > > Hi, Fabio. > I can't said exactly about these reasons but can suggest them. > 1. OSCAR implementations in PyICQt and PyAIMt derived from Twisted > Words's one in 2005 and tuned for specific usage.
Ok, so basically it is branch with its own life (I was trying to see if it was possible to remove any "legacy" stuff and use twisted only), let's it live ;) > 2. Scheduler creates threads for every user and removes them after. Not > after logout but since some time. It can be seen in process status - > transport keep only 2 threads when idle. Besides the fact threads should be avoided in twisted (here perhaps a DeferredQueue or something similar would work as fine without the issues derived form threads), my experience while running gateways tells me the contrary. Also without heavy usage memory continues growing up until I have to kill the gateway (same behavior with both aim and icq) because it takes more than 1GB of memory. I've tried to investigate and I've found that the number of Scheduler and and QueueThread objects continues increasing whatever it happens. Since a new Scheduler and each thread is created for each SNACBased connection, I've hooked a stop method in the connectionLost method and here are the results, after few days of testing: - patched gw (aim): 27MB of memory (stable after the first day) - non patched gq (icq): 89MB of memory (steadily increasing) If you like I'll be happy to share patches -- Fabio Forno, Ph.D. Bluendo srl http://www.bluendo.com jabber id: [email protected] --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "py-transports" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/py-transports?hl=en -~----------~----~----~----~------~----~------~--~---
