On 25 March 2011 04:02, Jeremy Dunck <[email protected]> wrote: > I'd like to be able to monitor how many of the daemon processes are > handling requests vs. idle. > > Is there a tool for this already? Or is it something that Apache > provides directly? > > I'm concerned that our pool of processes are all busy, causing > requests to queue (or similar problems).
I started working on adding stuff to make this easier to monitor some time back. The problem was that although code could be added to try and capture metrics, there were no tools for displaying or making sense of the data in an easy way. With no target consumer application for the data it was therefore hard to know how to go about producing the data. As a consequence I pulled all the code out. Time has moved on and I have changed jobs. The good news in this area is that I now work in a company called New Relic. This company provides a SaaS solution for production web application monitoring. They don't support Python yet, but that is what I am there working on. The New Relic software is useful because one of the features it provides is the tracking of queuing time. That is, it is able to graphically show what percentage of overall request time in the application web server is spent in waiting for the application to pick up and handle the request. That is, the time between when Apache accepts the request and when the WSGI application running in the mod_wsgi daemon process starts to handle the request. In the case of where there are insufficient mod_wsgi daemon processes and requests can back up at times you will therefore see a greater proportion of queueing time. That is one indicator. The other thing that is useful to see is what percentage of request handler threads in the pool are being used at any one time. This is a bit harder because you have the threads in Apache child processes themselves and also those in the daemon process group. Also, a single WSGI application could technically be delegated to run across multiple daemon process groups where each has a different number of processes/threads. So, a single measure to show utilisation can be tricky. The New Relic software has for other languages a way of showing what notional overall thread pool utilisation is, but I haven't dived down yet into how I could make use of that if could come up with simple measure. Anyway, the New Relic software will provide some interesting opportunities. For the core stuff no changes will even be required to mod_wsgi. For the utilisation measure then may be necessary to add some metrics generation into mod_wsgi still, but not trying to tackle that just yet. BTW, before anyone thinks 'but New Relic costs money', the display of queue time is available in their free lite version. So it should be something that anyone can access with the costs of their higher subscription levels. I'll obviously have more to say about New Relic software down the track and you can be sure I will be finding ways to use it to assist in this problem of knowing how to best configure Apache/mod_wsgi as well as its core purpose of monitoring your web application performance. If you have never heard of New Relic, I really suggest you go have a look at it. In the production area there is nothing really like it for Python and so when available should open up a world of opportunity for better Python web application monitoring. Graham -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
