On May 24, 9:22 pm, Antoine Pitrou <[email protected]> wrote:
> Hello,
>
> Launch a Python interactive session. Don't do anything special, just
> launch it. Look at the memory usage of this process (e.g. `ps aux |
> grep python`). The process takes about 4 MB.
>
> Then, just type :
>
> >>> import tg
>
> Just that. Nothing else. No application modules or data. Look at the
> process size again. Here it is 28 MB /just for having imported
> TurboGears/. (your mileage may vary)
>
> Now consider that under e.g. a mod_wsgi setup, every worker thread
> will have a separate interpreter (so that there is true isolation). It
> means that those 28 MB won't be shared, because they are comprised of
> objects (dicts, strings, tuples, functions, etc.) which are created
> dynamically when importing the modules (yes, it is different from what
> e.g. C shared libraries work).
> If your mod_wsgi setup has instantiated 8 worker threads, it will
> already take more than 220 MB wthout any application-specific stuff
> thrown into the mix. (the default value for the `threads` parameter of
> a mod_wsgi daemon process is 15... I'll let you do the math)
You misunderstand how Apache/mod_wsgi works. Each worker thread does
NOT have a separate interpreter. See:
http://blog.dscpl.com.au/2009/03/python-interpreter-is-not-created-for.html
This was about mod_python but the same holds for mod_wsgi.
Now, if on the other hand your are using Apache prefork MPM and
running your application in embedded mode, then each process will take
up base memory of 28MB. Because prefork MPM can result in a lot of
processes being created, then that can be a problem. But then that is
partly your fault for using prefork in the first place as it isn't
suitable for fat Python web applications. See:
http://blog.dscpl.com.au/2009/03/load-spikes-and-excessive-memory-usage.html
Graham
> This shows some of the problems associated with the TurboGears
> approach of "re-use every third-party module under the sun". If you
> suddenly find out that your mod_wsgi daemon process bubbles to 300MB+
> values, no, there isn't necessarily a memory leak in your Web
> application. You are just using TurboGears :-)
>
> Regards
>
> Antoine.
>
> PS : those numbers are taken on a 64-bit setup. A 32-bit build of
> Python would show significantly smaller numbers, due to the smaller
> pointer (and C long) size.
>
> PPS : the cost of a single garbage collection then becomes quite funny
> to watch, too:
> - before tg is imported:
> python -m timeit -s "import gc" "gc.collect()"
> 100 loops, best of 3: 3.05 msec per loop
> - after tg is imported:
> python -m timeit -s "import tg, gc" "gc.collect()"
> 10 loops, best of 3: 61.5 msec per loop
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"TurboGears" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/turbogears?hl=en
-~----------~----~----~----~------~----~------~--~---
