On Apr 25, 4:59 am, "Devin Torres" <[EMAIL PROTECTED]> wrote:
> > Use Apache and mod_wsgi and you have all that you want except playing
> > with 'processor affinity'. This is because Apache is multi process by
> > design and thus can properly make use of multiple CPUs. A lot of what
> > goes on in Apache is also not implemented in Python and thus not
> > subject to GIL issues.
>
> > You might also have a read of the following:
>
> > http://blog.dscpl.com.au/2007/09/parallel-python-discussion-and-modws...
> > http://blog.dscpl.com.au/2007/07/web-hosting-landscape-and-modwsgi.html
>
> Read.
>
> Apache is starting to look attractive now. So I assume I'm not looking
> for embedded mode, right? You say it's more performance, but at the
> cost of what? Using the worker MPM and, say, daemon-mode, using, say,
> 4 processes and 16 threads each, would my processes be dying as soon
> as they're not needed? My application takes awhile to load because I
> autoload my database using SQLAlchemy. Is it that easy to configure
> apache to start 4 by default and load balance between all of them?
If you are running a web site that requires the absolute best
performance possible, you would dedicate the Apache instance to
running just the one Python web application. That Apache instance
would be setup to use prefork MPM and you would use mod_wsgi embedded
mode. You would turn off keep alive for the Apache instance. You would
throw as much memory as possible into the system and you would use a
dedicated machine and not a VPS.
At the same time, all static media would be served from a distinct
nginx or lighttpd instance or via a content delivery provider. The
static media server would still use keep alive.
A typical default Apache prefork configuration is:
<IfModule mpm_prefork_module>
StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 0
</IfModule>
That is, initially create 5 child processes for serving requests. To
support maximum 150 clients at a time, because each child process is
single threaded, it can theoretically create up to 150 child processes
to handle requests, if demand so requires. As demand drops off and
process become unused, it will start to kill of additional child
processes, when this occurs it will keep arise between 5 and 10 of
these additional servers as spares for future bursts in traffic.
Apart from where it creates additional process to meet demand and then
kills them off when no longer required, the child process will be kept
around for ever. This is because max requests per child is set to 0.
If you had a problem with memory creep in an application, you could
set max requests per child to some non zero number and child processes
would be recycled after than number of requests.
Now, depending on how expensive loading the application is initially
and what you expected traffic volume is, you would customise these
values to keep as many persistent child processes around as possible
to meet average demand, plus some measure of bursts in traffic. What
the values would be you would have to experiment with.
Anyway, that is the extreme end where performance is the most
important thing. In this case you would use prefork MPM and mod_wsgi
embedded mode.
The other extreme end is a memory constrained system, in which case
you would use worker MPM, with small number of initial Apache child
processes, plus use mod_wsgi daemon mode with single daemon process
with limited number of threads. Static media would be served on same
Apache instance.
The limited number of threads would be to minimise possibility of
memory blowing out due to multiple concurrent requests allocating a
lot of transient memory at the same time. To temper this one would set
maximum number of requests for a process and set inactivity timeouts
so that daemon processes recycled if not doing anything, thus bring
memory back to minimal levels.
Apache, through which MPM you use and how you configure it, plus
mod_wsgi and whether you use embedded mode or daemon mode, plus how
you configure daemon mode, provide a great deal of flexibility in
creating a setup anywhere between these extremes. What configuration
is going to be best really depends on a lot of different issues, many
of which you don't expand on, such as how important is performance,
how much memory is available, how much memory your applications
require etc etc etc.
Even when you think you have a good idea of what sort of configuration
will work, you need to then properly test it, as well as compare that
performance to alternate configurations.
Personally I'd probably just suggest you start out with mod_wsgi
embedded mode with either prefork or worker MPM and just see how it
goes and get a feel for how Apache works, especially with respect to
it use of multiple processes to handle requests.
For most peoples web site applications, the configuration doesn't
generally matter that much as there application never has high enough
demands on it to become an issue. The only problematic case which
affects a lot of people is trying to do too much in a memory
constrained VPS system because of them not wanting to put up the cash
to get a better system with more memory. :-)
BTW, it is assumed you are using UNIX type systems here. Apache on
Windows is not multiprocess and mod_wsgi on Windows doesn't support
daemon mode.
Graham
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"pylons-discuss" group.
To post to this group, send email to pylons-discuss@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/pylons-discuss?hl=en
-~----------~----~----~----~------~----~------~--~---
