Re: requests possibly not reaching the log phase
Many thanks for all the answers on this one folks - in summary the suggestions were packet tracing, mod_security, RewriteLog, and mod_log_forensic. I went for packet tracing using tshark with a capture condition to minimise the performance impact, which worked nicely. The requests in question were completely missing, so the next possibility is an occasional DNS lookup failure. cheers John Paul Querna wrote: mod_log_forensic was created for exactly this purpose: http://httpd.apache.org/docs/2.2/mod/mod_log_forensic.html hope that helps, Paul
requests possibly not reaching the log phase
(apologies if this is a dupe, I originally sent from the wrong address) Hi there, I have an unusual problem - a large e-commerce site integrated with Authorize.net for card payments which appears to be failing to log some requests. The Authorize.net system makes HTTP POST requests to our server, and about 1 in every 500 transactions, the Authorize.net system reports a timeout and there's no trace of the request in our logs. Authorize.net won't investigate in any detail because their system is reporting that the request simply timed out. I'm using mod_perl but not hooking into the logging phase, and using a mod_log CustomLog directive which outputs the usual stuff, plus request time, connection state, and PID. So for now, I'm assuming a request is being sent but the log handler phase isn't running. The only way I can make this happen in a test environment is by opening a TCP connection and then closing it without sending any data. Are there any other reasons the log phase wouldn't be run? My next steps are to add simple request logging during the Trans phase, and failing that, packet sniffing, but this is a live high-traffic server so I'm trying to avoid that if possible. cheers John
Re: requests possibly not reaching the log phase
Graham Leggett wrote: If httpd isn't logging anything, the most likely explanation is that the request isn't reaching httpd at all. How reliable is your network between their system and yours? Are there any load balancing devices or other network magic in the way that could potentially be misconfigured? That gives me a few new places to hunt down the issue, thanks. Packet sniffing will answer the question was there any evidence of a request, and would probably be the least invasive way of measuring this. It's always useful to see what the network is actually doing, rather than what you think it's doing. Definitely - I've decided to bite the bullet and run tshark in the background, hopefully that will turn up some new information. Debugging an error which happens at random a few times a month is an interesting challenge! cheers John
libapreq2 on CPAN
Hi folks, Is there a problem with libapreq2 on CPAN? cpan make libapreq2 Warning: Cannot install libapreq2, don't know what it is. cpan make Apache2::Request Warning: Cannot make Apache2::Request, don't know what it is. thanks in advance, John
MaxClients inside Location?
Hi there, I couldn't find the answer to this in various user docs/forums/etc, apologies if it's a simple thing. I have a particularly busy URI on a particular VirtualHost and I'd like to limit how many clients can request that URL at a time. I've been looking for things like being able to use MaxClients inside a Location. I don't know where in the server/request life cycle the connection gets handed off to a child process, but I guess it'll be before the request is read in - if so we're probably not going to be able to do this. The requests are large POST requests, so we'd need to read the request headers before allocating a child process, we can't just rewrite the request. My 'plan B' is to run another httpd on a different port to handle these requests - I have control of the clients so can change the URI. That's actually sounding like a much simpler solution now! thanks John
Re: ProxyIOBufferSize
[re-sent from correct address, apologies for any dupes] Jim Jagielski wrote: On Oct 22, 2008, at 11:29 AM, Mladen Turk wrote: The upper limit in case of ajp protocol is 65536, so either silently enforcing that (like now), breaking, or logging as warning. Except we don't enforce an upper limit at all... That's the point :) A range of 512-65536 seems reasonable as well... Just to throw in an end-user's view here... I'm using mod_proxy as a reverse proxy, and have this setting such that 80% of objects can be handled in a single buffer. There must be many different applications of mod_proxy all with wildly different average object sizes. With that in mind, would it be sensible to look at what impact (or lack thereof) large values would have in some contrived applications? That way you could justify either having, or not having, a hard limit. If you don't have a hard limit, some suggested values could be added to the docs. cheers John
Re: ProxyIOBufferSize
Jim Jagielski wrote: On Oct 22, 2008, at 11:29 AM, Mladen Turk wrote: The upper limit in case of ajp protocol is 65536, so either silently enforcing that (like now), breaking, or logging as warning. Except we don't enforce an upper limit at all... That's the point :) A range of 512-65536 seems reasonable as well... Just to throw in an end-user's view here... I'm using mod_proxy as a reverse proxy, and have this setting such that 80% of objects can be handled in a single buffer. There must be many different applications of mod_proxy all with wildly different average object sizes. With that in mind, would it be sensible to look at what impact (or lack thereof) large values would have in some contrived applications? That way you could justify either having, or not having, a hard limit. If you don't have a hard limit, some suggested values could be added to the docs. cheers John
Re: bugs/inappropriate coding practice discovered by interproceduralcode analysis for version 2.2.8 of Apache
Henrik Nordstrom wrote: On tor, 2008-05-15 at 21:00 +0200, Ruediger Pluem wrote: \apache\src\log.c(682):apr_file_puts(errstr, logf); I see nothing reasonable that we can do in this situation but ignoring the error. syslog? Please excuse me barging in here but I think it's appropriate to look at typical use for this one. This error would typically happen on a busy site with a full log partition. Writing to syslog is likely to fail for the same reason, and in a typical setup a full /var partition is going to cause all sorts of problems. Good log files are very important on busy and/or distributed sites, and sometimes business critical. As a sysadmin I would prefer apache to die rather than end up with corrupt logs, but I think the default should remain as it is, so perhaps this could be a config file option? LogFaultAction or something. cheers John
flood patches/mods
Hi folks, Who's currently maintaining flood? I'm hoping to put in some time on improvements in the next month or two but might need to discuss it with someone to avoid submitting silly patches. cheers John
Re: flood reg ex matching problems
Guy Ferraiolo wrote: My experience is similar. It seems that some simple matching works but not in all cases. Any advice from anyone? I figured it out in the end! Flood isn't retrieving the full response by default for some reason, so can't match what's in the body. After poking around in the code I found there's a different response handler: Just addrecv_respgeneric_fullresp_recv_resp/recv_respat the end of your profile entry and the reg ex's work perfectly. Re your other email, I'm seriously considering putting some time (or one my minions time more likely!) into developing flood, so I'll keep the list posted on that. cheers John
flood reg ex matching problems
Hi folks, I grabbed flood from svn and built it according to the instructions at http://httpd.apache.org/test/flood/ I've been getting 'Regular expression match failed' most of the time, and even tried running one of the examples: -- [EMAIL PROTECTED]:/home/staff/john/flood# /usr/local/flood/bin/flood examples/round-robin-dynamic.xml Regular expression match failed (a href=([^]*)FAQ/a) postprocessing failed (http://httpd.apache.org/). Error running farmer 'Joe': Internal error. -- I even tried a ridiculously simple one and it still failed: url responsetemplate=(DOCTYPE) responsename=foohttp://httpd.apache.org/url I've even tried updating my pcre version (v3 was installed on this Ubuntu system) to no avail - it looks as though theres a version compiled in statically anyway (I'm using --disable-shared as per the doc, not sure if it has any effect on pcre). I've had some simple regex's working - eg. if I try responsetemplate=href=quot;([^quot;]+)quot; it sometimes works. Any ideas? thanks in advance, John O'Rourke
APR::Request::Param charset and UTF-8 flag
Hi folks,
I'm not sure if this is a bug and I'm new to the list so please forgive
any bad form.
I'm trying to use APR::Request::Param-charset(8) but it doesn't seem to
be setting perl's UTF-8 flag as the documentation says it should.
Here's the code:
sub handler { my $r=shift;
$query = Apache2::Request-new($r);
$param_table = $query-param();
$param_table-param_class( 'APR::Request::Param' );
$_-charset(8) foreach values %$param_table;
my $val = $param_table-{my_param}; # gives me a
APR::Request::Param, OK so far
print $val-charset(); # gives me '8', OK so far...
print utf8::is_utf8( $val-value() ); # false!
print utf8::is_utf8( $val ); # also false!
}
If this is correct behaviour then I can't see any purpose for the
charset method - can someone enlighten me?
cheers
John
Re: MP2 easy cookie interface available
Thanks for the feedback Philip. I believe it does subclass APR::Request::Cookie but I just left in all the 'use' statements from Apache2::Cookie. I figured the APR:: namespace would be the wrong place for it too. It need to override so many methods because they need to get the RequestRec object so the get_secret method can read the secret from dir_config - I couldn't think of an easier way but it does seem a bit much. cheers John Philip M. Gollucci wrote: Fowarding to apreq-dev (at) apache.org Did you really need to override all those methods? Granted I haven't looked at it that closely. The bigest thing I would say is you should subclass perhaps APR::Request::Cookie instead as that is the API we recommend these days. John ORourke wrote: http://www.versatilia.com/downloads/Validated.pm
Re: Apache2::Cookie odd return values
Right, I've solved my problem with a hack and some RTFSing, but I think
I've found a bug either in the docs or Apache2::Cookie...
Basically if I read cookies like this:
$cookie_hash_ref = APR::Request::Apache2-handle($r)-jar();
it works, I can treat the return as a hash of cookies.
However, if I (following the Apache2::Cookie man page) go:
$jar=Apache2::Cookie-new($r);
foreach $name ($jar-cookies()){ blah blah }
then the names are actually the full baked header value (eg. name=value)
I have to crack on with my project but I'll leave that one with you.
Phil, thanks for the pointers and the cookie recipes in the FAQ!
John
