On May 25, 10:53 am, Jorge Vargas <[email protected]> wrote:
> On Sun, May 24, 2009 at 7:22 AM, Antoine Pitrou
>
> <[email protected]> wrote:
>
> > Hello,
>
> > Launch a Python interactive session. Don't do anything special, just
> > launch it. Look at the memory usage of this process (e.g. `ps aux |
> > grep python`). The process takes about 4 MB.
>
> > Then, just type :
> >>>> import tg
>
> > Just that. Nothing else. No application modules or data. Look at the
> > process size again. Here it is 28 MB /just for having imported
> > TurboGears/. (your mileage may vary)
>
> > This shows some of the problems associated with the TurboGears
> > approach of "re-use every third-party module under the sun". If you
> > suddenly find out that your mod_wsgi daemon process bubbles to 300MB+
> > values, no, there isn't necessarily a memory leak in your Web
> > application. You are just using TurboGears :-)
>
> This is also a misinterpretation, it fact it almost falls into the FUD
> argument. At runtime X lines of code are X lines of code it doesn't
> maters from where they come from.
>
> Just for comparison, I know from a very good source than Django
> running under mod_python in a WebFaction server will take 20-30MB of
> RAM, per process (with 3 workers). go ahead and try your same
> experiment with that package.
>
> Now here is the fun fact. I'm sure TG+ all it's dependencies is many
> more LOC than django, someone should count those. Which simply means
> that your argument is completely reversed and using third-party
> dependencies is actually better as their memory footprint is smaller.
>
> > If your mod_wsgi setup has instantiated 8 worker threads,
>
> That is a ton of wasted RAM, you should be able to reduce our workers
> and increase their maxrequests and get the same results with much
> better ran.
Maybe not. It depends on the language of what was meant by 'worker'.
In his original email he talked about 'worker threads'. In a latter
email he said he was using daemon mode. If it was the case they were
using single process and multiple threads, then reducing 'worker'
threads wouldn't make much difference to resident memory. It may cause
virtual memory to drop because of the stupid default for thread stack
memory in Linux. That though is virtual memory and wouldn't normally
come to pass that it would be come resident memory. If one is worried
about how much virtual memory is set aside, because for example VPS
limit calculation makes reference to it, then one can through
configuration change the per thread stack size. For details on that
see:
http://code.google.com/p/modwsgi/wiki/ApplicationIssues#Memory_Constrained_VPS_Systems
Now, if when he said 'worker' he meant processes, then yes, dropping
the number of processes would be a good idea. But then, one can only
take that so far unless your application is also thread safe and you
use a multithreaded process.
Graham
> > Regards
>
> > Antoine.
>
> > PS : those numbers are taken on a 64-bit setup. A 32-bit build of
> > Python would show significantly smaller numbers, due to the smaller
> > pointer (and C long) size.
>
> > PPS : the cost of a single garbage collection then becomes quite funny
> > to watch, too:
> > - before tg is imported:
> > python -m timeit -s "import gc" "gc.collect()"
> > 100 loops, best of 3: 3.05 msec per loop
> > - after tg is imported:
> > python -m timeit -s "import tg, gc" "gc.collect()"
> > 10 loops, best of 3: 61.5 msec per loop
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"TurboGears" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/turbogears?hl=en
-~----------~----~----~----~------~----~------~--~---
