My answers are below, but before you peek, Graham, note that you and I have been through this memory discussion before & I've read the vertical partitioning article and use inactivity-timeout, "WSGIRestrictEmbedded On", considered maximum-requests, etc.
After years of this, I'm resigned to the fact that python is memory hungry, especially built on many of these web-stack and database libraries, etc. I'm Ok with that. I'm fine with a high-water RAM mark imposed by running under Apache, mostly. But, dang, it sure would be great if the 1 or 2% of requests that really (and legitimately) hog a ton of RAM, like, say 500MB extra, didn't keep it when done. I may revisit vertical partitioning again, but last time I did I think I found that the 1 or 2% in my case generally won't be divisible by url. In most cases I wouldn't know whether the particular request is going to need lots of RAM until *after *the database queries return (which is far too late for vertical partitioning to be useful). So I was mostly just curious about the status of nginx running wsgi, which doesn't solve python's memory piggishness, but would at least relinquish the extra RAM once python garbage collected. (Have you considered a max-memory parameter to mod_wsgi that would gracefully stop taking requests and shutdown after the threshold is reached for platforms that would support it? I recall -- maybe incorrectly -- you saying on Windows or certain platforms you wouldn't be able to support that. What about the platforms that *could *support it? It seems to me to be the very best way mod_wsgi could approach this Apache RAM nuance, so seems like it would be tremendously useful for the platforms that could support it.) Here ( http://blog.dscpl.com.au/2009/05/blocking-requests-and-nginx-version-of.html) you discuss nginx's tendency to block requests that may otherwise be executing in a different process, depending on timing, etc. Is this issue still the same (I thought I read a hint somewhere that there may be a workaround for that), so I ask. And so I wanted your opinion on nginx... ==== Here is what you asked for if it can still be useful. I'm on mod_wsgi-4.4.6 and the particular server that prompted me this time is running Apache 2.4 (prefork), though some of our clients use 2.2 (prefork). Our typical wsgi conf setting is something like this, though threads and processes varies depending on server size: LoadModule wsgi_module modules/mod_wsgi.so WSGIPythonHome /home/rarch/tg2env # see http://code.google.com/p/modwsgi/issues/detail?id=196#c10 concerning timeouts WSGIDaemonProcess rarch processes=20 threads=14 inactivity-timeout=1800 display-name=%{GROUP} graceful-timeout=5 python-eggs=/home/rarch/tg2env/lib/python-egg-cache WSGIProcessGroup rarch WSGISocketPrefix run/wsgi WSGIRestrictStdout Off WSGIRestrictStdin On # Memory tweak. http://blog.dscpl.com.au/2009/11/save-on-memory-with-modwsgi-30.html WSGIRestrictEmbedded On WSGIPassAuthorization On # we'll make the /tg/ directory resolve as the wsgi script WSGIScriptAlias /tg /home/rarch/trunk/src/appserver/wsgi-config/wsgi-deployment.py process-group=rarch application-group=%{GLOBAL} WSGIScriptAlias /debug/tg /home/rarch/trunk/src/appserver/wsgi-config/wsgi-deployment.py process-group=rarch application-group=%{GLOBAL} MaxRequestsPerChild 0 <IfModule prefork.c> MaxClients 308 ServerLimit 308 </IfModule> <IfModule worker.c> ThreadsPerChild 25 MaxClients 400 ServerLimit 16 </IfModule> Thanks for all your help and for excellent software! Kent On Wed, Mar 16, 2016 at 7:27 PM, Graham Dumpleton < [email protected]> wrote: > On the question of whether nginx will solve this problem, I can’t see how. > > When one talks about nginx and Python web applications, it is only as a > proxy for HTTP requests to some backend WSGI server. The Python web > application doesn’t run in nginx itself. So memory issues and how to deal > with them are the provence of the WSGI server used, whatever that is and > not nginx. > > Anyway, answer the questions below and can start with that. > > You really want to be using recent mod_wsgi version and not Apache 2.2. > > Apache 2.2 design has various issues and bad configuration defaults which > means it can gobble up more memory than you want. Recent mod_wsgi versions > have workarounds for Apache 2.2 issues and are much better at eliminating > those Apache 2.2 issues. Recent mod_wsgi versions also have fixes for > memory usage problems in some corner cases. As far as what I mean by > recent, I recommend 4.4.12 or later. The most recent version is 4.4.21. If > you are stuck with 3.4 or 3.5 from your Linux distro that is not good and > that may increase problems. > > So long as got recent mod_wsgi version then can look at using vertical > partitioning to farm out memory hungry request handlers to their own daemon > process group and better configure those to handle that and recycle > processes based on activity or, memory usage. A blog post related to that > is: > > * http://blog.dscpl.com.au/2014/02/vertically-partitioning-python-web.html > > Graham > > On 17 Mar 2016, at 7:15 AM, Graham Dumpleton <[email protected]> > wrote: > > What version of mod_wsgi and Apache are you using? > > Are you stuck with old versions of both? > > For memory tracking there are API calls mod_wsgi provides in recent > versions for getting memory usage which can be used as part of scheme to > trigger a process restart. You can’t use sys.exit(), but can use signals to > trigger a clean shutdown of a process. Again better to have recent mod_wsgi > versions as can then also set up some graceful timeout options for signal > induced restart. > > Also, what is your mod_wsgi configuration so can make sure doing all the > typical things one would do to limit memory usage, or quarantine particular > handlers which are memory hungry? > > Graham > > On 17 Mar 2016, at 4:29 AM, Kent Bower <[email protected]> wrote: > > Interesting idea.. yes, we are using multiple threads and also other > stack frameworks, so that's not straightforward, but worth thinking > about... not sure how to approach that with the other threads. Thank you > Bill. > > On Wed, Mar 16, 2016 at 1:11 PM, Bill Freeman <[email protected]> wrote: > >> I don't know about nginx, but one possibility, if the large memory >> requests are infrequent, is to detect when you have completed one and >> trigger the exit/reload of the daemon process (calling sys.exit() is not >> the way, since there could be other threads in the middle of something, >> unless you run one thread per process). >> >> On Wed, Mar 16, 2016 at 7:50 AM, Kent <[email protected]> wrote: >> >>> I'm looking for a very brief high-level pros vs. cons of wsgi under *apache >>> *vs. under *nginx *and then to be pointed to more details I can study >>> myself (or at least the latter). >>> >>> Our application occasionally allows requests that consume a large amount >>> of RAM (no obvious way around that, they are valid requests) and >>> occasionally this causes problems since we can't reclaim the RAM readily >>> from apache. (We already have tweaked with and do use >>> "inactivity-timeout". This helps, but still now and then we hit problems >>> where we run into swapping to disk.) >>> >>> I'm wondering if nginx may solve this problem. I've read much of what >>> you (Graham) have had to say about the memory strategies with apache and >>> mod_wsgi, but wonder what your opinion of nginx is and where you've already >>> discussed this. I've read articles I could find you've written on nginx, >>> such as "Blocking requests and nginx version of mod_wsgi," but wonder if >>> the same weaknesses are still applicable today, 7 years later? >>> >>> >>> Thank you very much in advance! >>> Kent >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "modwsgi" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/modwsgi. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "modwsgi" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/modwsgi/wyo2bJP0Cfc/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/modwsgi. >> For more options, visit https://groups.google.com/d/optout. >> > > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. > > > > -- > You received this message because you are subscribed to a topic in the > Google Groups "modwsgi" group. > To unsubscribe from this topic, visit > https://groups.google.com/d/topic/modwsgi/wyo2bJP0Cfc/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/modwsgi. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/modwsgi. For more options, visit https://groups.google.com/d/optout.
