Re: problem with ap_md5 in custom module

2012-08-15 Thread Ben Noordhuis 
On Wed, Aug 15, 2012 at 5:13 PM, nik600 nik...@gmail.com wrote:
 Dear all

 i'm having a problem with ap_md5, i just want to write a custom module
 that compute che md5 checksum of the requested url and give it back to
 the user.

 This is my code:
 *
 *
 static int kcache_handler(request_rec* r)
 {
 if (!r-handler || strcmp(r-handler, kcache))
 return DECLINED;

 if (r-method_number != M_GET)
 return HTTP_METHOD_NOT_ALLOWED;

 char* kcache_md5;
 kcache_md5 = (char *)ap_md5(r-pool,r-unparsed_uri);

 ap_set_content_type(r, text/html;charset=ascii);
 ap_rputs(!DOCTYPE HTML PUBLIC \-//W3C//DTD HTML 4.01//EN\, r);
 ap_rputs(htmlheadtitleK-Hello World!/title/head, r);
 ap_rprintf(r,bodyh1K-Hello
 World!/h1p%s=%s/p/body/html, r-unparsed_uri,kcache_md5);
 return OK;
 }
 *
 *

 i've got a warning during compilation:

 src/mod_kcache.c:18:15: warning: cast to pointer from integer of
 different size [-Wint-to-pointer-cast]

 Is quite strange to me that ap_md5 returns an int, as in the
 documentation it is reported to return a char *

 http://ci.apache.org/projects/httpd/trunk/doxygen/group__APACHE__CORE__MD5.html

 By the way, if i try to run it i get a segfault, if i comment the line
 that prints kcache_md5 with   ap_rprintf the module doesn't segfault.

 So, where i'm wrong?

It seems you don't include util_md5.h so the compiler defaults the
function prototype to `int ap_md5()`. Compile with -Wall -Wextra and
you'll get warnings about things like that.

Re: mod_fcgid concurrency bottleneck, issue#53693

2012-08-15 Thread pqf 
Hi, all
I prefer the following solution, can anyone review it? 
procmgr_post_spawn_cmd() will be blocked until process manager create a new 
fcgid process, the worst case is someone else take the new created process 
before I do, and I have to post another spawn command to PM again. The extreme 
case is loop FCGID_APPLY_TRY_COUNT but get no process slot.


Index: fcgid_bridge.c
===
--- fcgid_bridge.c  (revision 1373226)
+++ fcgid_bridge.c  (working copy)
@@ -30,7 +30,7 @@
 #include fcgid_spawn_ctl.h
 #include fcgid_protocol.h
 #include fcgid_bucket.h
-#define FCGID_APPLY_TRY_COUNT 2
+#define FCGID_APPLY_TRY_COUNT 4
 #define FCGID_REQUEST_COUNT 32
 #define FCGID_BRIGADE_CLEAN_STEP 32
 
@@ -447,19 +447,13 @@
 if (bucket_ctx-procnode)
 break;
 
-/* Avoid sleeping the very first time through if there are no
-   busy processes; the problem is just that we haven't spawned
-   anything yet, so waiting is pointless */
-if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) {
-apr_sleep(apr_time_from_sec(1));
-
-bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);
-if (bucket_ctx-procnode)
-break;
-}
-
 /* Send a spawn request if I can't get a process slot */
 procmgr_post_spawn_cmd(fcgi_request, r);
+
+/* Try again */
+bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);
+if (bucket_ctx-procnode)
+break;
 }
 
 /* Connect to the fastcgi server */
2012-08-15



pqf



发件人：Mike M
发送时间：2012-08-15 04:29
主题：mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

Ahoy! 

With mod_fcgid I've observed a performance bottleneck in which concurrent 
requests to a virtualhost where the number of concurrent requests exceed 
the number of available fcgid-spawned processes result in a significant 
delay in responding to requests. 

Details of this situation, how to reproduce, various scenarios tried, etc 
are detailed in the issue submission [1]. 

In file  modules/fcgid/fcgid_bridge.c, there is this section of code: 

if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) { 
apr_sleep(apr_time_from_sec(1)); 

If I change the sleep time from 1 second to 0 seconds (or, comment out 
this section of code entirely), the bottleneck goes away.  It seems like 
the more appropriate action here would be to turn the sleep time into a 
configurable value, with the current 1s value as the default. 

I presume this code is meant to be a way to help defer spawn requests so 
that a server is not over-whelmed with spin-up related I/O. 

My questions are: 
- Is this intended to be a throttling mechanism and if so, is there a more 
efficient way to handle this throttling? 
- If I outright reduce that sleep delay to 0 (or comment out this code), 
what potential or expected problems am I introducing to mod_fcgid? 


[1] https://issues.apache.org/bugzilla/show_bug.cgi?id=53693 


--- 
Mike M 

You can't learn in school what the world is going to do next year.

Re: svn commit: r1373169 - /httpd/httpd/branches/2.4.x/STATUS

2012-08-15 Thread Rainer Jung 

On 15.08.2012 02:00, traw...@apache.org wrote:

Author: trawick
Date: Wed Aug 15 00:00:42 2012
New Revision: 1373169

URL: http://svn.apache.org/viewvc?rev=1373169view=rev
Log:
clarify :)

Modified:
 httpd/httpd/branches/2.4.x/STATUS

Modified: httpd/httpd/branches/2.4.x/STATUS
URL: 
http://svn.apache.org/viewvc/httpd/httpd/branches/2.4.x/STATUS?rev=1373169r1=1373168r2=1373169view=diff
==
--- httpd/httpd/branches/2.4.x/STATUS (original)
+++ httpd/httpd/branches/2.4.x/STATUS Wed Aug 15 00:00:42 2012
@@ -100,7 +100,8 @@ PATCHES ACCEPTED TO BACKPORT FROM TRUNK:
 when some unconfigured run-time file starts respecting their
 DefaultRuntimeDir directive after an upgrade.
 +1: trawick, jim, rjung
-   rjung: backport candidates are:
+   rjung: applicable trunk revisions WITHOUT the compatibility tweaks
+  described above:
heartbeat  r1364695
scoreboard r1369477
core/pid file  r1369808


Sorry I didn't catch the incompatibility but also didn't want to 
indicate a real vote on those revs. Just wanted to provide a sneak 
preview, which uses of the runtime directory you prepared in trunk. Good 
that you fixed the misunderstanding.


Regards,

Rainer

problem with ap_md5 in custom module

2012-08-15 Thread nik600 
Dear all

i'm having a problem with ap_md5, i just want to write a custom module
that compute che md5 checksum of the requested url and give it back to
the user.

This is my code:
*
*
static int kcache_handler(request_rec* r)
{
if (!r-handler || strcmp(r-handler, kcache))
return DECLINED;

if (r-method_number != M_GET)
return HTTP_METHOD_NOT_ALLOWED;

char* kcache_md5;
kcache_md5 = (char *)ap_md5(r-pool,r-unparsed_uri);

ap_set_content_type(r, text/html;charset=ascii);
ap_rputs(!DOCTYPE HTML PUBLIC \-//W3C//DTD HTML 4.01//EN\, r);
ap_rputs(htmlheadtitleK-Hello World!/title/head, r);
ap_rprintf(r,bodyh1K-Hello
World!/h1p%s=%s/p/body/html, r-unparsed_uri,kcache_md5);
return OK;
}
*
*

i've got a warning during compilation:

src/mod_kcache.c:18:15: warning: cast to pointer from integer of
different size [-Wint-to-pointer-cast]

Is quite strange to me that ap_md5 returns an int, as in the
documentation it is reported to return a char *

http://ci.apache.org/projects/httpd/trunk/doxygen/group__APACHE__CORE__MD5.html

By the way, if i try to run it i get a segfault, if i comment the line
that prints kcache_md5 with   ap_rprintf the module doesn't segfault.

So, where i'm wrong?

Thanks to all in advance

-- 
/*/
nik600

Re: Proposed TR for this Friday

2012-08-15 Thread Rainer Jung 

On 14.08.2012 23:17, Jim Jagielski wrote:

With my Mom's hospitalization and stuff, this kinda slipped...


I hope she is doing better now.


I propose this Friday: Aug 17th.

Any conflicts??


You might want to take a look at

http://svn.apache.org/viewvc?view=revisionrevision=r1241953

Since for 2.4.2 there was a deps tarball (and for 2.4.3 there will be 
again?) this might benefit from another rephrasing (plus sync with trunk 
where it was never applied).


Regards,

Rainer

Re: How to align shm in an neat way?

2012-08-15 Thread Rainer Jung 

On 14.08.2012 23:20, Jim Jagielski wrote:

slotmem handles this well, afaict ;)


The fix applied in r1373270 which Joe also had a look at is very similar 
to the handling in slotmem. I added some macros as proposed by Joe. 
Fortunately the code in shmcb already did all the address calculations 
using macros so I only had to add the alignment there.


Regards,

Rainer

Re: mod_fcgid concurrency bottleneck, issue#53693

2012-08-15 Thread Lazy 
2012/8/15 pqf p...@mailtech.cn:
 Hi, all
 I prefer the following solution, can anyone review it?
 procmgr_post_spawn_cmd() will be blocked until process manager create a new
 fcgid process, the worst case is someone else take the new created process
 before I do, and I have to post another spawn command to PM again. The
 extreme case is loop FCGID_APPLY_TRY_COUNT but get no process slot.


 Index: fcgid_bridge.c
 ===
 --- fcgid_bridge.c  (revision 1373226)
 +++ fcgid_bridge.c  (working copy)
 @@ -30,7 +30,7 @@
  #include fcgid_spawn_ctl.h
  #include fcgid_protocol.h
  #include fcgid_bucket.h
 -#define FCGID_APPLY_TRY_COUNT 2
 +#define FCGID_APPLY_TRY_COUNT 4
  #define FCGID_REQUEST_COUNT 32
  #define FCGID_BRIGADE_CLEAN_STEP 32

 @@ -447,19 +447,13 @@
  if (bucket_ctx-procnode)
  break;

 -/* Avoid sleeping the very first time through if there are no
 -   busy processes; the problem is just that we haven't spawned
 -   anything yet, so waiting is pointless */
 -if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) {
 -apr_sleep(apr_time_from_sec(1));
 -
 -bucket_ctx-procnode = apply_free_procnode(r,
 fcgi_request);
 -if (bucket_ctx-procnode)
 -break;
 -}
 -
  /* Send a spawn request if I can't get a process slot */
  procmgr_post_spawn_cmd(fcgi_request, r);
 +
 +/* Try again */
 +bucket_ctx-procnode = apply_free_procnode(r, fcgi_request);
 +if (bucket_ctx-procnode)
 +break;
  }

  /* Connect to the fastcgi server */


if You get rid of sleep apache will not wait for free process if all
of them are busy, this will lead to 503 errors

currently mod_fcgid waits FCGID_APPLY_TRY_COUNT * FCGID_REQUEST_COUNT
* 1 second and this is usually 64 seconds, this means if You have an
overloaded vhost with low FcgidMaxProcessesPerClass it can bring whole
server down, each thread waits 64 seconds so it doesn't take long
before all threads are occupied.

In my setup we lowerd the wait time and FCGID_REQUEST_COUNT to lower
the impact of overloaded class but I think
the best solution will be to add availability to each class. Total
wait time will be related to it (by changing sleep time, and
FCGID_APPLY_TRY_COUNT). If request is unsuccessful availability will
be halved so next time the wait time will be shorter. This way
congested class will get 0% availability, and new connections will
instantly get 503 it there are no free slots. A successful wait will
increase availability.


Regards,

Michal Grzedzicki

Re: Re: mod_fcgid concurrency bottleneck, issue#53693

2012-08-15 Thread pqf 
Hi, Michal
My solution do add availability to each class, which is the 
procmgr_post_spawn_cmd() call in each loop do.
The sleep() call is intrudused for a stress test without warm up time, in this 
case, mod_fcgid will create more processes than a slow start one(each process 
handler can't apply a free slot on the very begining, so send a request to 
process manager to create one, it's easy to reach the max # of process limit 
while httpd startup, but the idle process will be killed later), the sleep() 
call is a little like a server side warm up delay.
But since someone said remove this sleep(), the server work fine without 
bottleneck(Maybe he didn't notise the warm up issue?), so I thought remove the 
sleep() is a good idea. But reduce the time of sleep() is fine to me too.
Any other idea?

Regards,

Ryan Pan

2012-08-16



pqf



发件人：Lazy
发送时间：2012-08-15 21:48
主题：Re: mod_fcgid concurrency bottleneck, issue#53693
收件人：devdev@httpd.apache.org
抄送：

2012/8/15 pqf p...@mailtech.cn: 
 Hi, all 
 I prefer the following solution, can anyone review it? 
 procmgr_post_spawn_cmd() will be blocked until process manager create a new 
 fcgid process, the worst case is someone else take the new created process 
 before I do, and I have to post another spawn command to PM again. The 
 extreme case is loop FCGID_APPLY_TRY_COUNT but get no process slot. 
 
 
 Index: fcgid_bridge.c 
 === 
 --- fcgid_bridge.c  (revision 1373226) 
 +++ fcgid_bridge.c  (working copy) 
 @@ -30,7 +30,7 @@ 
  #include fcgid_spawn_ctl.h 
  #include fcgid_protocol.h 
  #include fcgid_bucket.h 
 -#define FCGID_APPLY_TRY_COUNT 2 
 +#define FCGID_APPLY_TRY_COUNT 4 
  #define FCGID_REQUEST_COUNT 32 
  #define FCGID_BRIGADE_CLEAN_STEP 32 
 
 @@ -447,19 +447,13 @@ 
  if (bucket_ctx-procnode) 
  break; 
 
 -/* Avoid sleeping the very first time through if there are no 
 -   busy processes; the problem is just that we haven't spawned 
 -   anything yet, so waiting is pointless */ 
 -if (i  0 || j  0 || count_busy_processes(r, fcgi_request)) { 
 -apr_sleep(apr_time_from_sec(1)); 
 - 
 -bucket_ctx-procnode = apply_free_procnode(r, 
 fcgi_request); 
 -if (bucket_ctx-procnode) 
 -break; 
 -} 
 - 
  /* Send a spawn request if I can't get a process slot */ 
  procmgr_post_spawn_cmd(fcgi_request, r); 
 + 
 +/* Try again */ 
 +bucket_ctx-procnode = apply_free_procnode(r, fcgi_request); 
 +if (bucket_ctx-procnode) 
 +break; 
  } 
 
  /* Connect to the fastcgi server */ 


if You get rid of sleep apache will not wait for free process if all 
of them are busy, this will lead to 503 errors 

currently mod_fcgid waits FCGID_APPLY_TRY_COUNT * FCGID_REQUEST_COUNT 
* 1 second and this is usually 64 seconds, this means if You have an 
overloaded vhost with low FcgidMaxProcessesPerClass it can bring whole 
server down, each thread waits 64 seconds so it doesn't take long 
before all threads are occupied. 

In my setup we lowerd the wait time and FCGID_REQUEST_COUNT to lower 
the impact of overloaded class but I think 
the best solution will be to add availability to each class. Total 
wait time will be related to it (by changing sleep time, and 
FCGID_APPLY_TRY_COUNT). If request is unsuccessful availability will 
be halved so next time the wait time will be shorter. This way 
congested class will get 0% availability, and new connections will 
instantly get 503 it there are no free slots. A successful wait will 
increase availability. 


Regards, 

Michal Grzedzicki