On 5/8/07, Andrew Wilson <[EMAIL PROTECTED]> wrote: > > Lets keep this on the list.. > > Yeah, I realized I messed that up; I did forward my response to the list a > couple minutes after I sent it to you. > > > > > are these all running as part of the same process, or are you just > > > > concerned > > > > about having two versions of CherryPy installed? > > > > > > > > > They aren't, at present, running at all; I'm trying to figure out how to > > > deploy them. > > > > Ok, so are you ultimately trying to make them all run as part of a single > process? > > It would be nice to have most of the serving done from one process. In case > you care, here's why: When running Python under Virtuzzo (with a Red Hat or > CentOS kernel), instantiating the smallest thread takes up 10mb of our > memory allocation. On a virtual machine running with 256mb of assigned > memory, this means that a 10-thread Python process burns up almost half our > total memory. We are trying to figure out why this happens, and how we can > avoid it,
I've heard of this happening before. The trick is to tell your hosting company not to double count shared libraries. > but so far: no luck. So we want to keep as much of the web > application / serving in one process as possible. > > What I'd like to clarify is how Python Paste works. > > My understanding was that it essentially imported all of my applications > into one Python interpreter, and created a "wrapper" WSGI application. I > think that it has to do this, because it's passing a single WSGI environ > object through the application stack. This "wrapper" WSGI application can > then be served using your WSGI server of choice, which is usually Paste's > scripter or Flup or similar. > > Upshot: All of your web applications are living in the same Python process. > Which is good for resource management (they all share the same thread pool, > threads get allocated to whichever web app is getting the most hits, etc.), > but is bad for modularization (because it's virtually impossible to have two > versions of the same library loaded into the same Python process, unless > that library was designed from the ground-up to allow for this, which most > are not). > > Am I wrong so far? > > So, I guess my question was, is there any way to take a WSGI application, > and "wrap" it so that it gets the full WSGI environment, etc., from a Paste > application stack, but so that the application is isolated into its own > Python process? I do believe it's possible to have multiple Python interpreters in the same process, and I believe some part of this whole WSGI standard mentions this. However, I'm not all that sure that has a large advantage over separate processes in terms of memory usage. > PS - I realize that a simpler solution would be to simply have each > application running in a seperate server process, and to proxy the > collection behind lighttpd (I don't believe in apache). (humor) Oh, believe it! I've seen Apache with my own two eyes! It exists! ;) > But that means that > each process must have its own separate thread pool; 5 applications at 5 > threads apiece consumes our entire memory pool. I think you're stuck with one of: * Using multiple processes. * Figuring out how to get multiple interpreters in the same process. It's possible, but I've never done it. * Telling your hosting company to stop counting memory the wrong way. I think Ben Bangert had to do this with his hosting company. Happy Hacking! -jj -- http://jjinux.blogspot.com/ _______________________________________________ Paste-users mailing list [email protected] http://webwareforpython.org/cgi-bin/mailman/listinfo/paste-users
