Re: mod_fcgid concurrency bottleneck, issue#53693
Hi, all I prefer the following solution, can anyone review it? procmgr_post_spawn_cmd() will be blocked until process manager create a new fcgid process, the worst case is someone else take the new created process before I do, and I have to post another spawn command to PM again. The extreme case is loop FCGID_APPLY_TRY_COUNT but get no process slot. Index: fcgid_bridge.c === --- fcgid_bridge.c (revision 1373226) +++ fcgid_bridge.c (working copy) @@ -30,7 +30,7 @@ #include "fcgid_spawn_ctl.h" #include "fcgid_protocol.h" #include "fcgid_bucket.h" -#define FCGID_APPLY_TRY_COUNT 2 +#define FCGID_APPLY_TRY_COUNT 4 #define FCGID_REQUEST_COUNT 32 #define FCGID_BRIGADE_CLEAN_STEP 32 @@ -447,19 +447,13 @@ if (bucket_ctx->procnode) break; -/* Avoid sleeping the very first time through if there are no - busy processes; the problem is just that we haven't spawned - anything yet, so waiting is pointless */ -if (i > 0 || j > 0 || count_busy_processes(r, &fcgi_request)) { -apr_sleep(apr_time_from_sec(1)); - -bucket_ctx->procnode = apply_free_procnode(r, &fcgi_request); -if (bucket_ctx->procnode) -break; -} - /* Send a spawn request if I can't get a process slot */ procmgr_post_spawn_cmd(&fcgi_request, r); + +/* Try again */ +bucket_ctx->procnode = apply_free_procnode(r, &fcgi_request); +if (bucket_ctx->procnode) +break; } /* Connect to the fastcgi server */ 2012-08-15 pqf 发件人:Mike M 发送时间:2012-08-15 04:29 主题:mod_fcgid concurrency bottleneck, issue#53693 收件人:"dev" 抄送: Ahoy! With mod_fcgid I've observed a performance bottleneck in which concurrent requests to a virtualhost where the number of concurrent requests exceed the number of available fcgid-spawned processes result in a significant delay in responding to requests. Details of this situation, how to reproduce, various scenarios tried, etc are detailed in the issue submission [1]. In file modules/fcgid/fcgid_bridge.c, there is this section of code: if (i > 0 || j > 0 || count_busy_processes(r, &fcgi_request)) { apr_sleep(apr_time_from_sec(1)); If I change the sleep time from 1 second to 0 seconds (or, comment out this section of code entirely), the bottleneck goes away. It seems like the more appropriate action here would be to turn the sleep time into a configurable value, with the current 1s value as the default. I presume this code is meant to be a way to help defer spawn requests so that a server is not over-whelmed with spin-up related I/O. My questions are: - Is this intended to be a throttling mechanism and if so, is there a more efficient way to handle this throttling? - If I outright reduce that sleep delay to 0 (or comment out this code), what potential or expected problems am I introducing to mod_fcgid? [1] https://issues.apache.org/bugzilla/show_bug.cgi?id=53693 --- Mike M You can't learn in school what the world is going to do next year.
Re: svn commit: r1373169 - /httpd/httpd/branches/2.4.x/STATUS
On 15.08.2012 02:00, traw...@apache.org wrote: Author: trawick Date: Wed Aug 15 00:00:42 2012 New Revision: 1373169 URL: http://svn.apache.org/viewvc?rev=1373169&view=rev Log: clarify :) Modified: httpd/httpd/branches/2.4.x/STATUS Modified: httpd/httpd/branches/2.4.x/STATUS URL: http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1373169&r1=1373168&r2=1373169&view=diff == --- httpd/httpd/branches/2.4.x/STATUS (original) +++ httpd/httpd/branches/2.4.x/STATUS Wed Aug 15 00:00:42 2012 @@ -100,7 +100,8 @@ PATCHES ACCEPTED TO BACKPORT FROM TRUNK: when some unconfigured run-time file starts respecting their DefaultRuntimeDir directive after an upgrade. +1: trawick, jim, rjung - rjung: backport candidates are: + rjung: applicable trunk revisions WITHOUT the compatibility tweaks + described above: heartbeat r1364695 scoreboard r1369477 core/pid file r1369808 Sorry I didn't catch the incompatibility but also didn't want to indicate a real vote on those revs. Just wanted to provide a sneak preview, which uses of the runtime directory you prepared in trunk. Good that you fixed the misunderstanding. Regards, Rainer
problem with ap_md5 in custom module
Dear all i'm having a problem with ap_md5, i just want to write a custom module that compute che md5 checksum of the requested url and give it back to the user. This is my code: * * static int kcache_handler(request_rec* r) { if (!r->handler || strcmp(r->handler, "kcache")) return DECLINED; if (r->method_number != M_GET) return HTTP_METHOD_NOT_ALLOWED; char* kcache_md5; kcache_md5 = (char *)ap_md5(r->pool,r->unparsed_uri); ap_set_content_type(r, "text/html;charset=ascii"); ap_rputs("", r); ap_rputs("K-Hello World!", r); ap_rprintf(r,"K-Hello World!%s=%s", r->unparsed_uri,kcache_md5); return OK; } * * i've got a warning during compilation: src/mod_kcache.c:18:15: warning: cast to pointer from integer of different size [-Wint-to-pointer-cast] Is quite strange to me that ap_md5 returns an int, as in the documentation it is reported to return a char * http://ci.apache.org/projects/httpd/trunk/doxygen/group__APACHE__CORE__MD5.html By the way, if i try to run it i get a segfault, if i comment the line that prints kcache_md5 with ap_rprintf the module doesn't segfault. So, where i'm wrong? Thanks to all in advance -- /*/ nik600
Re: Proposed T&R for this Friday
On 14.08.2012 23:17, Jim Jagielski wrote: With my Mom's hospitalization and stuff, this kinda slipped... I hope she is doing better now. I propose this Friday: Aug 17th. Any conflicts?? You might want to take a look at http://svn.apache.org/viewvc?view=revision&revision=r1241953 Since for 2.4.2 there was a deps tarball (and for 2.4.3 there will be again?) this might benefit from another rephrasing (plus sync with trunk where it was never applied). Regards, Rainer
Re: How to align shm in an neat way?
On 14.08.2012 23:20, Jim Jagielski wrote: slotmem handles this well, afaict ;) The fix applied in r1373270 which Joe also had a look at is very similar to the handling in slotmem. I added some macros as proposed by Joe. Fortunately the code in shmcb already did all the address calculations using macros so I only had to add the alignment there. Regards, Rainer
Re: mod_fcgid concurrency bottleneck, issue#53693
2012/8/15 pqf : > Hi, all > I prefer the following solution, can anyone review it? > procmgr_post_spawn_cmd() will be blocked until process manager create a new > fcgid process, the worst case is someone else take the new created process > before I do, and I have to post another spawn command to PM again. The > extreme case is loop FCGID_APPLY_TRY_COUNT but get no process slot. > > > Index: fcgid_bridge.c > === > --- fcgid_bridge.c (revision 1373226) > +++ fcgid_bridge.c (working copy) > @@ -30,7 +30,7 @@ > #include "fcgid_spawn_ctl.h" > #include "fcgid_protocol.h" > #include "fcgid_bucket.h" > -#define FCGID_APPLY_TRY_COUNT 2 > +#define FCGID_APPLY_TRY_COUNT 4 > #define FCGID_REQUEST_COUNT 32 > #define FCGID_BRIGADE_CLEAN_STEP 32 > > @@ -447,19 +447,13 @@ > if (bucket_ctx->procnode) > break; > > -/* Avoid sleeping the very first time through if there are no > - busy processes; the problem is just that we haven't spawned > - anything yet, so waiting is pointless */ > -if (i > 0 || j > 0 || count_busy_processes(r, &fcgi_request)) { > -apr_sleep(apr_time_from_sec(1)); > - > -bucket_ctx->procnode = apply_free_procnode(r, > &fcgi_request); > -if (bucket_ctx->procnode) > -break; > -} > - > /* Send a spawn request if I can't get a process slot */ > procmgr_post_spawn_cmd(&fcgi_request, r); > + > +/* Try again */ > +bucket_ctx->procnode = apply_free_procnode(r, &fcgi_request); > +if (bucket_ctx->procnode) > +break; > } > > /* Connect to the fastcgi server */ if You get rid of sleep apache will not wait for free process if all of them are busy, this will lead to 503 errors currently mod_fcgid waits FCGID_APPLY_TRY_COUNT * FCGID_REQUEST_COUNT * 1 second and this is usually 64 seconds, this means if You have an overloaded vhost with low FcgidMaxProcessesPerClass it can bring whole server down, each thread waits 64 seconds so it doesn't take long before all threads are occupied. In my setup we lowerd the wait time and FCGID_REQUEST_COUNT to lower the impact of overloaded class but I think the best solution will be to add availability to each class. Total wait time will be related to it (by changing sleep time, and FCGID_APPLY_TRY_COUNT). If request is unsuccessful availability will be halved so next time the wait time will be shorter. This way congested class will get 0% availability, and new connections will instantly get 503 it there are no free slots. A successful wait will increase availability. Regards, Michal Grzedzicki
Re: Re: mod_fcgid concurrency bottleneck, issue#53693
Hi, Michal My solution do "add availability to each class", which is the procmgr_post_spawn_cmd() call in each loop do. The sleep() call is intrudused for a stress test without warm up time, in this case, mod_fcgid will create more processes than a slow start one(each process handler can't apply a free slot on the very begining, so send a request to process manager to create one, it's easy to reach the max # of process limit while httpd startup, but the idle process will be killed later), the sleep() call is a little like a "server side warm up delay". But since someone said remove this sleep(), the server work fine without bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove the sleep() is a good idea. But reduce the time of sleep() is fine to me too. Any other idea? Regards, Ryan Pan 2012-08-16 pqf 发件人:Lazy 发送时间:2012-08-15 21:48 主题:Re: mod_fcgid concurrency bottleneck, issue#53693 收件人:"dev" 抄送: 2012/8/15 pqf : > Hi, all > I prefer the following solution, can anyone review it? > procmgr_post_spawn_cmd() will be blocked until process manager create a new > fcgid process, the worst case is someone else take the new created process > before I do, and I have to post another spawn command to PM again. The > extreme case is loop FCGID_APPLY_TRY_COUNT but get no process slot. > > > Index: fcgid_bridge.c > === > --- fcgid_bridge.c (revision 1373226) > +++ fcgid_bridge.c (working copy) > @@ -30,7 +30,7 @@ > #include "fcgid_spawn_ctl.h" > #include "fcgid_protocol.h" > #include "fcgid_bucket.h" > -#define FCGID_APPLY_TRY_COUNT 2 > +#define FCGID_APPLY_TRY_COUNT 4 > #define FCGID_REQUEST_COUNT 32 > #define FCGID_BRIGADE_CLEAN_STEP 32 > > @@ -447,19 +447,13 @@ > if (bucket_ctx->procnode) > break; > > -/* Avoid sleeping the very first time through if there are no > - busy processes; the problem is just that we haven't spawned > - anything yet, so waiting is pointless */ > -if (i > 0 || j > 0 || count_busy_processes(r, &fcgi_request)) { > -apr_sleep(apr_time_from_sec(1)); > - > -bucket_ctx->procnode = apply_free_procnode(r, > &fcgi_request); > -if (bucket_ctx->procnode) > -break; > -} > - > /* Send a spawn request if I can't get a process slot */ > procmgr_post_spawn_cmd(&fcgi_request, r); > + > +/* Try again */ > +bucket_ctx->procnode = apply_free_procnode(r, &fcgi_request); > +if (bucket_ctx->procnode) > +break; > } > > /* Connect to the fastcgi server */ if You get rid of sleep apache will not wait for free process if all of them are busy, this will lead to 503 errors currently mod_fcgid waits FCGID_APPLY_TRY_COUNT * FCGID_REQUEST_COUNT * 1 second and this is usually 64 seconds, this means if You have an overloaded vhost with low FcgidMaxProcessesPerClass it can bring whole server down, each thread waits 64 seconds so it doesn't take long before all threads are occupied. In my setup we lowerd the wait time and FCGID_REQUEST_COUNT to lower the impact of overloaded class but I think the best solution will be to add availability to each class. Total wait time will be related to it (by changing sleep time, and FCGID_APPLY_TRY_COUNT). If request is unsuccessful availability will be halved so next time the wait time will be shorter. This way congested class will get 0% availability, and new connections will instantly get 503 it there are no free slots. A successful wait will increase availability. Regards, Michal Grzedzicki