That is actually for something totally unrelated as far as I can see. I will get to the daemon pooling, but have been busy as had to work to out why Python 3.2 was broken before they released rc1 and get mod_wsgi updated to work with subtle changes they made to how threading APIs work.
Graham On 17 January 2011 02:25, grassoalvaro <[email protected]> wrote: > Ok, i found the timeouts problem: > > /* we now proceed to read from the client until we get EOF, or until > * MAX_SECS_TO_LINGER has passed. the reasons for doing this are > * documented in a draft: > * > * http://www.ics.uci.edu/pub/ietf/http/draft-ietf-http-connection-00.txt > * > * in a nutshell -- if we don't make this effort we risk causing > * TCP RST packets to be sent which can tear down a connection before > * all the response data has been sent to the client. > */ > #define SECONDS_TO_LINGER 2 > > so this is not so easy that i thought ;-/ > > Can you tell more about "are other ways of avoiding the restarts"? > > On 13 Sty, 19:41, grassoalvaro <[email protected]> wrote: >> > FWIW, there are other ways of avoiding the restarts by preallocating >> > daemon process groups in advance and not using VirtualHost but other >> > means of virtual hosting. >> >> Hmm, this is interesting, any example (docs)? >> >> On 13 Sty, 12:28, Graham Dumpleton <[email protected]> wrote: >> >> > On 13 January 2011 22:14, grassoalvaro <[email protected]> wrote: >> >> > > Thanks. I'll try to modify Apache source code to change that timeout >> > > and then make some tests. >> >> > Have fun trying to work out where to change it. It seems to differ >> > between MPMs and I never quite got my head around it completely, or at >> > least I have forgotten exactly how it works. Overall It is not >> > something I would recommend and increasing the timeout will likely >> > only cause other problems. That is, rather than have requests be >> > killed, you will cause new requests to stall for a lot longer and >> > sites will simply appear to not be responding. >> >> > > Event if mod_wsgi wan't create for shared hosting - many people use it >> > > for this. And it works well (except Apache reloading ;-)) Solution to >> > > use one Apache for one client is good but takes too many memory. >> >> > FWIW, there are other ways of avoiding the restarts by preallocating >> > daemon process groups in advance and not using VirtualHost but other >> > means of virtual hosting. >> >> > Graham >> >> > > Anywany, thanks for information. >> >> > > On 11 Sty, 23:30, Graham Dumpleton <[email protected]> wrote: >> > >> On 11 January 2011 23:06, grassoalvaro <[email protected]> wrote: >> >> > >> > Thanks for your very useful description. >> > >> > Do you know Is there any good reason why apache is killing process >> > >> > after 3 seconds? Why no 15 for example? >> >> > >> It is a hard wired timeout period it applies to Apache child processes >> > >> when a 'restart' is applied. When it does a graceful restart, it >> > >> handles its own child process differently and lets them run to >> > >> completion, but in that graceful restart mode it still kills off >> > >> processes after fixed 3 seconds if they are what it regards as other >> > >> processes, which is what it regards mod_wsgi daemon processes. >> >> > >> Note that mod_wsgi was originally intended for self hosting of your >> > >> own web applications with daemon mode not actually a part of the >> > >> original goals as far as features. The model of how it is implemented >> > >> isn't necessarily suited to doing shared hosting, especially where >> > >> changes are going to be made often to Apache configuration. >> >> > >> Graham >> >> > >> > On 11 Sty, 10:17, Graham Dumpleton <[email protected]> wrote: >> > >> >> On 11 January 2011 10:27, grassoalvaro <[email protected]> >> > >> >> wrote: >> >> > >> >> > Ok, short description is not enought. >> >> > >> >> > My app is an ordering app for shared hosting accounts (for Python >> > >> >> > mod_wsgi) and it's based on mod_wsgi also. >> > >> >> > App is calling some API method (server-app) which is restarting >> > >> >> > apache >> > >> >> > after httpd modifications. >> > >> >> > The problem is that I can of course use sleep && httpd graceful in >> > >> >> > background, but this will kill other (customer apps) connections >> > >> >> > after >> > >> >> > sleep. >> > >> >> > So, is there any method do reload config without killing any >> > >> >> > mod_wsgi >> > >> >> > connections? >> >> > >> >> Connections are transient and not permanent. For a well written site >> > >> >> connections would be maintained for less than a second. >> >> > >> >> The way the process restart works for daemon processes, regardless of >> > >> >> whether you do an Apache restart or graceful restart, is that when >> > >> >> daemon process is signalled to restart, it will stop accepting any >> > >> >> more connections. If there are no active connections for current >> > >> >> requests it will shutdown process immediately and then restart. Any >> > >> >> connections that come during that time effectively queue up with >> > >> >> Apache child processes and/or within port 80 socket listener queue. >> > >> >> When processes restart, the held over requests will be handled. >> >> > >> >> In the case that the daemon process still had active requests, then >> > >> >> rather than daemon process shutting down and restarting immediately >> > >> >> it >> > >> >> waits, giving the current requests an opportunity to finish first at >> > >> >> what point the process is then shutdown and restarted. The way Apache >> > >> >> works however, is that it gives a maximum of three seconds grace to >> > >> >> the mod_wsgi daemon processes and if they have not shutdown by then, >> > >> >> because of the requests not finishing, Apache will forcibly kill them >> > >> >> off and they will be restarted. >> >> > >> >> The problem you saw was because your request handler was likely >> > >> >> blocked in some way on waiting for the Apache process to restart and >> > >> >> that very act meant the request didn't complete within the three >> > >> >> seconds allowing the process to shutdown cleanly. >> >> > >> >> Now, as I said, for a well written site, requests should be sub >> > >> >> second, only long requests like large file uploads might be affected. >> > >> >> So, for typical case, the restart wouldn't usually be noticed and >> > >> >> wouldn't affect any connections because outstanding requests would >> > >> >> finish within the three seconds dictated by Apache. >> >> > >> >> Of course, if you are constantly restarting Apache because of >> > >> >> configuration changes, then you increase the risk that you will hit a >> > >> >> requests that doesn't finish within the three seconds and exit in >> > >> >> time >> > >> >> to allow process to restart cleanly. >> >> > >> >> Whether you can avoid doing the restarts I cannot comment because >> > >> >> have >> > >> >> no idea what changes you are enacting to the Apache configuration >> > >> >> which triggers you wanting to restart it. If the changes relate to >> > >> >> mod_wsgi daemon process provisioning then for some cases you may be >> > >> >> able to avoid it, but it means setting up things in a particular way. >> > >> >> Overall, you are often better off having a nginx front end and simply >> > >> >> giving each customer their own Apache instance behind that. That way >> > >> >> only customer to whom the changes apply are affected and no one else. >> > >> >> This is how WebFaction and some other quality Python hosting >> > >> >> companies >> > >> >> work. >> >> > >> >> > Regards. >> >> > >> >> > On 11 Sty, 00:15, Graham Dumpleton <[email protected]> >> > >> >> > wrote: >> > >> >> >> On 11 January 2011 07:53, grassoalvaro <[email protected]> >> > >> >> >> wrote: >> >> > >> >> >> > Hi. >> >> > >> >> >> > I have application deployed on apache + mod_wsgi which is >> > >> >> >> > changing >> > >> >> >> > httpd.conf and then reloading apache (httpd graceful). >> > >> >> >> > And here is the problem: application is making request changing >> > >> >> >> > httpd >> > >> >> >> > config and reloading him and while request is in progress the >> > >> >> >> > process >> > >> >> >> > is killing: >> >> > >> >> >> > Premature end of script headers: start.wsgi >> > >> >> >> > (2)No such file or directory: mod_wsgi (pid=31624): Unable to >> > >> >> >> > connect >> > >> >> >> > to WSGI daemon process 'pylons_app' on >> > >> >> >> > '/var/logs/wsgi.31409.0.1.sock' >> > >> >> >> > after multiple attempts., >> >> > >> >> >> > So it is possible to reload _only_ apache config without >> > >> >> >> > reloading >> > >> >> >> > whole mod_wsgi? >> >> > >> >> >> The short answer is no. >> >> > >> >> >> > Or any other idea how to deal with this problem? >> >> > >> >> >> How are you triggering the Apache restart? The daemon processes >> > >> >> >> themselves will not have sufficient privileges to do it itself, >> > >> >> >> so you >> > >> >> >> must be using a separate suid script or invoking a remote API for >> > >> >> >> some >> > >> >> >> service that does it on your behalf. >> >> > >> >> >> If it is a command line script, then invoke it as a background >> > >> >> >> command >> > >> >> >> with a time delay before it. Ie. >> >> > >> >> >> os.system('(sleep 1; /some/path/restart-apache)&') >> >> > >> >> >> If it is via a remote API using XML-RPC or something, instead of >> > >> >> >> doing >> > >> >> >> it from web application, make it a script executed as above. >> >> > >> >> >> A further way is to create a background thread which calls the >> > >> >> >> API. >> >> > >> >> >> In all ways, you are attempting to avoid the problem of the >> > >> >> >> request >> > >> >> >> handler blocking on waiting for the restart command to complete. >> >> > >> >> >> Anyway, will help if you explain how you are triggering the >> > >> >> >> Apache restart. >> >> > >> >> >> Graham >> >> > >> >> > -- >> > >> >> > You received this message because you are subscribed to the Google >> > >> >> > Groups "modwsgi" group. >> > >> >> > To post to this group, send email to [email protected]. >> > >> >> > To unsubscribe from this group, send email to >> > >> >> > [email protected]. >> > >> >> > For more options, visit this group >> > >> >> > athttp://groups.google.com/group/modwsgi?hl=en. >> >> > >> > -- >> > >> > You received this message because you are subscribed to the Google >> > >> > Groups "modwsgi" group. >> > >> > To post to this group, send email to [email protected]. >> > >> > To unsubscribe from this group, send email to >> > >> > [email protected]. >> > >> > For more options, visit this group >> > >> > athttp://groups.google.com/group/modwsgi?hl=en. >> >> > > -- >> > > You received this message because you are subscribed to the Google >> > > Groups "modwsgi" group. >> > > To post to this group, send email to [email protected]. >> > > To unsubscribe from this group, send email to >> > > [email protected]. >> > > For more options, visit this group >> > > athttp://groups.google.com/group/modwsgi?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "modwsgi" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/modwsgi?hl=en. > > -- You received this message because you are subscribed to the Google Groups "modwsgi" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/modwsgi?hl=en.
