Problem with long http request generation time - process restarting

2011-12-14 Thread Andrew Merton
Hi --

I am getting the following message recurring in my log:

Parent: child process exited with status 255 -- Restarting.

followed by the process restart sequence.

I have tracked it down to a call to $r->content which doesn't seem to
return properly.

It would appear that

1. The POST request has Encoding-type: chunked (message total length is
~5kb);

2. The client sends the HTTP headers, then pauses 3-4 seconds while it
generates the body of the message (confirmed with WireShark);

3. So when I call $r-content, when it succeeds I see a 3-4 second pause
before it completes;

4. If another request arrives during the call to $r->content, the process
aborts and restarts - I assume it is interfering with the read() call in
$r->content.

Since the pause is so long, it is very likely to be interrupted even though
I only have 5 clients currently :(

Given the the chunked implemenation in mod_perl 2.0 is apparently
incomplete, have I struck one area where this affects the result?

Can anyone make suggestions as to how I can work around this problem?  I
wondered about maybe moving my handler to the log phase (currently it's a
Response handler), by which time I assume the entire message must be
available, but it seems like a major kludge and presumably would lead to
other issues anyway.

Or is there some multi-threading thing I should have done - currently I
have not really done anything specific...

I am using Apache2.2 on Win7 (no flames please :)), with ActiveState Perl
"(v5.12.2) built for MSWin32-x86-multi-thread".

Andrew


Adding Post Data to a SubRequest

2011-12-14 Thread Jordan Michaels
I've been working to find a solution to adding Post data to a sub 
request for a few days now (since last week when I posted about 
retrieving the POST data in the first place).


Adding the POST data to the $subr object using write doesn't work, 
because it tries to add it to the output response and not the input request.


I then found a message on google that indicated that you could modify 
the incoming request POST data via a filter, so I've written something 
similar to the following:


--
use Apache2::SubRequest  ();
use Apache2::ServerUtil  ();
use Apache2::RequestUtil ();
use Apache2::RequestRec  ();
use Apache2::RequestIO   ();
use Apache2::Filter  ();
use Apache2::Log ();
use APR::Table   ();
use APR::Brigade ();
use APR::Bucket  ();

sub handler {
my $r   = shift;
our @ProxyInputContent  = ();

sub proxy_input_filter {
my ( $f, $bb ) = @_;
my $ba = $bb->bucket_alloc();
my $newBucket = '';
# remove the existing buckets
$bb->cleanup;
# create buckets using the orig request POST data
foreach ( @ProxyInputContent ) {
# the current array element will be represented as $_
# create a new bucket
$newBucket = APR::Bucket->new($ba, $_);
# put the new bucket in the subr brigade
$bb->insert_head($newBucket);
}
# the subrequest should now contain the post data
return Apache2::Const::OK;
}

# handle POST requests
if ( $r->method() eq "POST" ) {

# set the subrequest method to also be a post
$subr->method('POST');

# create a post data buffer
my $PostBuffer = '';

# loop over each line of data
while($r->read($PostBuffer, 1024)) {

# add the content to the ProxyInputContent Array
push(@ProxyInputContent, $PostBuffer);
}

		# make the subrequest go through the input filter so we can add the 
post data to it.

$subr->add_input_filter( \&proxy_input_filter );

}

# perform the proxy request
$subr->run;
--

However, this doesn't work. I still don't see the POST data on the 
Tomcat side.


And, being the noob I am, I haven't yet figured out how to log the data 
from the filter to I can make sure I'm doing it right.


So, my questions are simple:

1) Is there an ideal way to add POST data to a subrequest that I'm 
missing here?


2) Is it possible to log data from a filter so I can debug it?

Thank you *so much* for your help and patience with this. I truly 
appreciate your time.


--
Warm Regards,
Jordan Michaels


Re: Problem with long http request generation time - process restarting

2011-12-14 Thread Randolf Richardson
> Hi --
> 
> I am getting the following message recurring in my log:
> 
> Parent: child process exited with status 255 -- Restarting.
> 
> followed by the process restart sequence.
> 
> I have tracked it down to a call to $r->content which doesn't seem to
> return properly.
> 
> It would appear that
> 
> 1. The POST request has Encoding-type: chunked (message total length is
> ~5kb);
> 
> 2. The client sends the HTTP headers, then pauses 3-4 seconds while it
> generates the body of the message (confirmed with WireShark);
> 
> 3. So when I call $r-content, when it succeeds I see a 3-4 second pause
> before it completes;
> 
> 4. If another request arrives during the call to $r->content, the process
> aborts and restarts - I assume it is interfering with the read() call in
> $r->content.
> 
> Since the pause is so long, it is very likely to be interrupted even though
> I only have 5 clients currently :(
> 
> Given the the chunked implemenation in mod_perl 2.0 is apparently
> incomplete, have I struck one area where this affects the result?
> 
> Can anyone make suggestions as to how I can work around this problem?  I
> wondered about maybe moving my handler to the log phase (currently it's a
> Response handler), by which time I assume the entire message must be
> available, but it seems like a major kludge and presumably would lead to
> other issues anyway.

I find it curious that you observe this behaviour within just under 
5 seconds.  The KeepAliveTimeout setting, which defaults to 5 
seconds, might be of interest to you -- I would certainly try setting 
it to 20 seconds temporarily in the hopes of at least ruling it out 
as a possible cause:

http://httpd.apache.org/docs/2.2/mod/core.html#keepalivetimeout

> Or is there some multi-threading thing I should have done - currently I
> have not really done anything specific...
> 
> I am using Apache2.2 on Win7 (no flames please :)), with ActiveState Perl
> "(v5.12.2) built for MSWin32-x86-multi-thread".

I have noticed that socket I/O under Windows behaves strangely at 
times.  I eventually moved off to running Apache HTTPd 2.2 with 
ModPerl 2 on NetBSD under VirtualBox, which basically seemed to cure 
all the unpredictable problems I encountered (not just with ModPerl 2 
but with other technologies, such as ImageMagick's PerlMagick 
libraries, even without ModPerl 2 in the mix, issues with compiling 
code downloaded from CPAN, etc.) as well as yielded better 
performance with faster response times.

I found ActiveState Perl to be a great implementation, but one of 
the problems I had was that it sometimes didn't have a pre-compiled 
module that I needed, and moving to Unix was a lot easier.

I mention those points in the hopes that it will be helpful to you 
(switching to Unix was helpful to a few of my friends who love PHP, 
and one who does a lot of xBase {Clipper} programming), and I hope 
you don't view this as a flame.

Randolf Richardson - rand...@inter-corporate.com
Inter-Corporate Computer & Network Services, Inc.
Beautiful British Columbia, Canada
http://www.inter-corporate.com/




Re: Problem with long http request generation time - process restarting

2011-12-14 Thread Issac Goldstand
That stinks of a segfault.  The admission to using Windows at the end
makes me suspect it even more, as Windows has an unfortunate habit, due
to the MPM implementation, of a thread segfault taking the whole server
down with it, causing a several second delay while it cleans up the old
process and replaces it with a new one (with mod_perl doing all its init
stuff again)

I used to hate having to deal with that :)

Having said that, I don't think it's a chunking issue, as httpd core
handles that, not mod_perl.  Can you supply more info?  Not sure what,
but my gut says there's a missing piece here.

  Issac

On 14/12/2011 23:44, Andrew Merton wrote:
> Hi --
>
> I am getting the following message recurring in my log:
>
> Parent: child process exited with status 255 -- Restarting.
>
> followed by the process restart sequence.
>
> I have tracked it down to a call to $r->content which doesn't seem to
> return properly.
>
> It would appear that
>
> 1. The POST request has Encoding-type: chunked (message total length
> is ~5kb);
>
> 2. The client sends the HTTP headers, then pauses 3-4 seconds while it
> generates the body of the message (confirmed with WireShark);
>
> 3. So when I call $r-content, when it succeeds I see a 3-4 second
> pause before it completes;
>
> 4. If another request arrives during the call to $r->content, the
> process aborts and restarts - I assume it is interfering with the
> read() call in $r->content.
>
> Since the pause is so long, it is very likely to be interrupted even
> though I only have 5 clients currently :(
>
> Given the the chunked implemenation in mod_perl 2.0 is apparently
> incomplete, have I struck one area where this affects the result?
>
> Can anyone make suggestions as to how I can work around this problem?
>  I wondered about maybe moving my handler to the log phase (currently
> it's a Response handler), by which time I assume the entire message
> must be available, but it seems like a major kludge and presumably
> would lead to other issues anyway.
>
> Or is there some multi-threading thing I should have done - currently
> I have not really done anything specific...
>
> I am using Apache2.2 on Win7 (no flames please :)), with ActiveState
> Perl "(v5.12.2) built for MSWin32-x86-multi-thread".
>
> Andrew 
>
>