Hi Mike, On Wed, Feb 22, 2012 at 7:22 AM, Mike Wesner <[email protected]> wrote: > I would love to learn more from google on python memory handling. As > Robert mentioned, we have observed that memory is not released/garbage > collected on python 2.5 instances. It seems to just hold on to it. > This works because the instances don't live forever and eventually get > shutdown and new ones take their place.
We haven't changed Python's built-in memory management in either of our Python runtimes. Note that, even excluding cyclic garbage collection, Python and the C allocator do not always release memory eagerly - they maintain their own memory pools for performance reasons. So, for example, you can't assume that an instance consuming 100MB of memory only has 28MB remaining to handle requests. Cheers, Brian The memory management used in the App Engine Python 2.5 and 2.7 runti > > -Mike > > On Feb 21, 10:33 am, alex <[email protected]> wrote: >> > One of the biggest issues is that the Python runtime leaks memory like mad >> >> could you elaborate on this? or give some references that support your >> claims (provided an app code is written according to best practices, etc) >> >> >> >> >> >> >> >> On Tuesday, February 21, 2012 5:10:16 PM UTC+1, Robert Kluin wrote: >> >> > To be clear, the scheduler can still dispatch multiple requests for >> > that url to the same instance. Only one request will execute at a >> > time though. One of the biggest issues is that the Python runtime >> > leaks memory like mad, so in this case you may wind up with 1) >> > increased latency and 2) still blowing mem limits fast. >> >> > Robert >> >> > On Tue, Feb 21, 2012 at 09:36, Jeff Schnitzer wrote: >> > > On Mon, Feb 20, 2012 at 5:33 AM, Johan Euphrosine wrote: >> >> > >> On Mon, Feb 20, 2012 at 11:25 AM, Andrin von Rechenberg >> > >> wrote: >> >> > >>> I guess that's the same solution as just deploying two different >> > >>> versions. A threadsafe one and a non threadsafe one. Or did >> > >>> I misunderstand you? >> >> > >> appcfg.py provide commands to help you manage your backends deployment >> > and >> > >> configuration, with `backends.yaml` and `appcfg backends update`. >> >> > >> That make the solution more convenient than using multiple versions on >> > >> frontend instances. >> >> > > This adds the significant downside of no auto-scaling. I don't see how >> > > that's more convenient :-) >> >> > > Andrin: In javaland I would simply synchronize the people-search >> > function >> > > so that at most one thread can execute that routine in a single instance >> > at >> > > once. It means all people-search requests in that instance will queue >> > up in >> > > serial, which could cause undesirable waits if the function takes >> > > significant time, but if the requests are spread out among enough >> > instances >> > > it probably won't be an issue. >> >> > > I don't know what the python equivalent of 'synchronized' is. >> >> > > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
