On 30-11-2008, Graham Dumpleton wrote:
>
> 2008/11/29 William Dode <[EMAIL PROTECTED]>:
>>
>> On 28-11-2008, Graham Dumpleton wrote:
>>>
>>> 2008/11/29 William Dode <[EMAIL PROTECTED]>:
>>>>
>>>> Hi,
>>>>
>>>> i've a little virtual server with few websites with very low traffics
>>>> and one website with more traffic (100000 hits/day). Most of the time it
>>>> works very very well, very fast, without using too much memory...
>>>>
>>>> But sometimes the load goes very high, i don't know why, i imagine that
>>>> it's because of somes web spiders or attack or a bug in my application.
>>>>
>>>> I would like somes tips to configure apache to can handle this problem.
>>>>
>>>> I use apache-worker and mod_wsgi in deamon mode.
>>>>
>>>> For the mod_wsgi app i use threads=1 (my app is not thread safe) and max
>>>> request=1000 (for memory leak)
>>>
>>> Can you post all the WSGI related directives you have in your Apache
>>> configuration so can see exactly what you have an whether it is
>>> correct.
>>
>> WSGIDaemonProcess seps.flibuste.net user=seps group=www-user
>> stack-size=524288 python-path=/home/web/seps/pynclude:/home/web/seps/seps.
>> flibuste.net/pynclude home=/home/web/seps threads=1 inactivity-timeout=600
>> maximum-requests=1000 display-name=wsgi-seps.flibuste.net
>> WSGIProcessGroup seps.flibuste.net
>> AddHandler wsgi-script .wsgi
>>
>>>
>>>> for apache i did this : BUT DON'T THINK IT'S CORRECT
>>>
>>> For your size site (approx 1-2 request/sec average) you could have
>>> just left default Apache worker configuration as is and the site would
>>> probably have worked fine. I wish people would stop trying to
>>> prematurely optimise performance when it isn't necessary, especially
>>> when they don't really understand Apache configuration and are likely
>>> to do more harm than good. :-(
>>
>> I would not touch the configuration if i had not the problem before of
>> course !
>>
>>>
>>> The thing that is more likely to kill you is your application not
>>> being thread safe, although depends on how you have setup WSGI
>>> directives, which you didn't supply. So, post the WSGI bits of the
>>> configuration and would then be able to comment more.
>>
>>
>>>
>>> Also indicate what the WSGI application is written in. Ie., which framework.
>>
>> My own framework, very light and fast, i use now webob, postgresql with
>> no orm, caching a lot...
>> This application was working fine for years, the problem came after an
>> upgrade to lenny (apache2-modwsgi), before i used
>> apache1+mod_proxy+twistedmatrix. I measure each page on the app side,
>> the average is 0.04s it's why i didn't bother to make it thread safe.
>
> What proportion of your overall requests are handled dynamically by
> your WSGI applications as opposed to how many are static file
> requests?
Maybe half
>
> The problem is that if a large portion of them are dynamic requests,
> or the URLs are such that search engines or SPAM bots might readily
> trigger them, because you have a single daemon process with only a
> single thread, it can only handle one request at a time. Even if it
> was a surge in traffic, you should have been able to tell that from
> the Apache access logs unless you have disabled them doing logging.
>
> Anyway, it doesn't matter how many processes/threads you configure for
> main Apache child worker processes, any dynamic requests will stall if
> a dynamic request before them takes a long time. In other words, you
> have created a potential bottleneck in the way you have configured the
> daemon process group.
Does this bottleneck can make the load so high to freeze the system ?
Because most of the time every think is fine i just would like that when
there is an overload apache answer a 503 service unavailable.
>
> At the minimum, you should create multiple daemon processes to handle
> requests if they are going to be single threaded. Thus:
I use a lot of cache inside my app that i will loose if i use multi
processes, it's also why it will take me a litle time to make it thread
safe. But i'll do it, just that i wanted to take the time since most of
the time it works very well.
>
> WSGIDaemonProcess seps.flibuste.net ... processes=5 threads=1 ...
>
> A further issue is, since you are using AddHandler instead of
> WSGIScriptAlias, how many different .wsgi scripts do you have.
I use addhandler but only one script. I do this because i have a generic
way to configure my virtualhosts.
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule (.*) seps.wsgi/$1 [QSA,PT,L]
I imagine that -f will take a little more cpu, but i use this to can
create somes statics pages dynamicaly.
> If you
> have numerous scripts, you need to be aware that each will be run in
> the context of its own sub interpreter within the daemon process.
Is it possible that even if i have one script it will run differents sub
interpreters ?
> The
> sub interpreters persist between requests for the same WSGI script,
> but by having a sub interpreter per WSGI script in this case, if you
> have a lot of scripts, you could be using up a lot more memory than
> you need, and if the startup costs are high because of what code has
> to be loaded, causing yourself slow downs while processes handle
> initial requests after a process restart due to having to load same
> code modules multiple times.
>
> If you have multiple WSGI scripts in your sort of situation and they
> can actually coexist in the same single sub interpreter, you would be
> better off running them in one interpreter rather than many. To force
> them all to run in same interpreter set:
>
> WSGIApplication %{GLOBAL}
>
> In this case we are actually running them in main interpreter.
>
> As to the load of box going up, did you actually identify using 'ps'
> that it was the Apache child worker processes or mod_wsgi daemon
> processes that was consuming the CPU? If you could clearly see this,
> then likely it is your application code itself if a mod_wsgi daemon
> process. As to Apache child worker processes having high load, that
> wouldn't be something I would expect unless running other no static
> handlers in Apache for handling requests.
My problem is here, i cannot identify clearly what is going on.
I'm admin of two servers since years, and never seen this : completly
nothing is visible in the differents logs or ps.
The *only things* make me looking at apache is that when there is the
high load netstat show me a lot (25) of line with wsgi socket.
unix 4 [ ] STREAM CONNECTING 0
/var/run/apache2/wsgi.1911.1.1.sock
unix 4 [ ] STREAM CONNECTED 886488
unix 4 [ ] STREAM CONNECTING 0
/var/run/apache2/wsgi.1911.1.1.sock
unix 4 [ ] STREAM CONNECTED 886487
ps aux doesn't show me any cpu or memory problem. I could also see that
the app was not restarted before or during the load (same pid).
One possibility is that it is because of the high load (i don't know
why) apache and my app lag. But there is nearly nothing else on this
server...
It's why i search for a configuration where i can move away this
possibility of a load because of apache. Like we could do in a shared
hosting.
>
> The only real situation I could see where something related to
> mod_wsgi could be cause of load suddenly going up high, is if your
> application ran code which caused the process to crash. This would
> cause mod_wsgi to immediately start another process in its place. If
> something was continually firing requests at same URL in very rapid
> fashion, and it crashed every time, you might see some load increase,
> but even then wouldn't expect it to be that noticeable unless your
> startup cost of application is very high.
>
> Only time would expect to see it be really noticeable is you were
> preloading code into mod_wsgi daemon processes and they were crashing
> at startup, in which case it would go into tight loop restarting
> process until Apache shutdown. This would be more problematic because
> not dependent on a subsequent request to come in to load code again.
>
> If processes were crashing and causing either of these scenarios,
> there should be some information in the Apache error logs about it. To
> ensure there is you should use:
>
> LogLevel debug
>
> in main Apache configuration, and if necessary any VirtualHost where
> LogLevel may also be set.
I should begin with this...
>
> Overall, I can't see a lot in the information you have provided to
> base any decision to change the Apache configuration. Seems you are
> more just trying things rather than actually knowing what the true
> trigger of the problems may have been.
Exactly... I even did a script to stress my app, it simulate players who
plays against the server very quickly, the result is complely different.
I've an other server with 17 applications under mod_wsgi, with exactly
the same configuration, the same framework (my), and it works like
a charm... So, this disturb me a lot !
Hmmm. I have an idea, i'll try to deliberately slow down my app and even
make it crash to see if it will reproduce the same problem...
thks
>
> I wouldn't see something like KeepAlive making much difference if
> using worker MPM, because even if sockets were all held, for worker
> MPM the maximum number of Apache child worker processes is usually
> small. This is in contrast to prefork where it can be a 100. In the
> case of prefork, if running out of sockets and having to create lots
> of Apache child worker processes, would still only really be an issue
> if those Apache child worker processes were having to perform
> something significant at startup or no first request. For example, if
> WSGI applications weren't actually running in mod_wsgi daemon process
> but in embedded mode and so every Apache child worker processes was
> having to load up your application as well, if request hitting that
> new process targets it.
>
> If you have some better information on the actual cause from
> information in access/error logs etc, or from what 'ps' was showing it
> would help as far as working out real problem and thus what
> configuration changes you might need, but otherwise can only suggest
> the above as far as mod_wsgi goes and how to it at least improve
> performance on that side of things.
>
> Graham
>
--
William Dodé - http://flibuste.net
Informaticien Indépendant
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"modwsgi" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/modwsgi?hl=en
-~----------~----~----~----~------~----~------~--~---
