I'll address just the following first. On 20 January 2011 16:59, Aljoša Mohorović <[email protected]> wrote: > configuration of additional logs per wsgi application to store "TIME > CPU-USAGE MEMORY-USAGE DISK/IO-USAGE THREADS ..." every <n> seconds. > based on application usage (calculated trending?) and additional > configuration options increase/decrease resources wsgi application can > use.
If you look, I'd suggest that system monitoring tools which can capture that information are already available. I don't see a need for this to specifically be embedded within mod_wsgi. If you really wanted it to be done by the process itself, which would itself be odd in a multi process system because of duplicate logging of same information, then you could write it as a Python module and simply import the functionality into your application. > if server hosts wsgi applications that are active only in a specific > period of the day (or something similar) and since there is some basic > trending calculated (or configured based on trending/logs) wsgi > application should be pushed to swap/disk and use no memory. Swapping parts of memory not accessed by a mostly inactive process is what UNIX itself does. Trying to even trigger this from within an application process in a generic way would be a nightmare anyway as user process code isn't going to know what parts of the application process could be swapped out. This is why the operating system does it. > also, if wsgi app is at max resources configured and system has > resources (cpu, memory, ...) available it should enable wsgi app to > use additional resources, maybe with threads=auto or something more > appropriate. Auto creation of additional threads isn't usually going to help much unless the requests are long lived and will be in a blocked state most of the time. The limitations of the GIL mean additional processes is generally better where need to scale up for processor intensive activities. For just threads, in mod_wsgi 3.X it uses a scheme in daemon processes whereby it reuses the most recently used threads. This means that if you over allocate the number of threads you need and they are never required, although they exist in the thread pool they are never activated within the Python interpreter so extra memory overhead from that happening doesn't occur at least. If traffic bursts such that all threads were eventually used and then threads became idle later, the memory used up per thread still isn't that much anyway, so not sure what you are hoping to save by having a varying thread pool size. Anyway, that all said, with the way that threads are managed in daemon mode for mod_wsgi 3.X, it is perhaps at least easier to implement a dynamic number of threads than it was in older mod_wsgi versions. You have to remember though that to make use of such a feature you still have to setup Apache MPM settings for the Apache child processes which proxy to the daemon process so as to have enough capacity to handle the concurrent requests you expect. There is no point having a capacity in the daemon process to scale up to 500 threads if your Apache MPM settings are going to be a bottleneck and only let through 100 at a time. Overall it is generally better just to configure things to what is expected to be the maximum you can handle with resources available. What is your justification for wanting to do this? All I can see is that you are trying to fit more distinct WSGI applications on a system than the hardware can perhaps accommodate. :-) > i'm also interested in something like this: > WsgiConfigurationScript my_config_script.py > WsgiConfigurationScriptInterval 300 # seconds > which would result in calling some function in my_config_script.py > every 5 minutes and based on script output reconfigure settings at > runtime. > code i would personally put inside would do something like: > - if it's 1 am set threads=1, push to swap/disk or something similar > - if it's 7 am reset all settings to default/static apache configuration Reconfiguring daemon process groups on the fly like that cannot be done as that is handled by the Apache parent process, which runs as root, and for which it isn't possible to run background tasks. Right now what you could is predefine various daemon process groups with different configurations and use WSGIDispatchScript directive to reference Python script containing functions which dynamically for each request makes a decision as to which daemon process group a request for an application is delegated. Thus you could technically shift a WSGI application to a set of processes which uses less resources and then signal the old daemon process group to at least recycle themselves back to base memory level as if no WSGI application had been loaded. You can't however shut the processes down. With the dynamic daemon process creation I originally mentioned as a possible feature then technically you would then be able to set it up so the processes will shutdown after some period of inactivity and only startup again if new requests come in for them. Thus you could do what you want, but you would not actually be changing the defined daemon process configurations but shifting the WSGI applications between pre defined daemon process groups, where the creation of a new instance of the daemon process group for a particular WSGI application being delayed until first request arrives for it. > please note that by "new features" i'm also considering an in-depth > document(s) on how to setup something, probably mostly because > currently it's not obvious how to do something in an easy way. Some things aren't documented because it is a pain and so not keen on recommending those ways. The introduction of daemon process groups that can create new process on demand will solve many of those problems and then easier to provide recommendations. > also, is there something new in httpd 2.3 that will affect mod_wsgi in any > way? No. As far as I know it works the same under Apache httpd 2.3. I haven't compiled against it recently though so don't know if they have made further changes which will require me to change code in mod_wsgi yet again. They have already made a few changes along the way which have caused me to need to change things. Graham -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
