Re: ApacheCon report

[Michael Blakeley]

>  From: "Perrin Harkins" <[EMAIL PROTECTED]>
>  To: "Ask Bjoern Hansen" <[EMAIL PROTECTED]>
>  Cc: <[EMAIL PROTECTED]>
>  Sent: Tuesday, October 31, 2000 8:47 PM
>  Subject: Re: ApacheCon report
>
>  > > Mr. Llima must do something I don't, because with real world
>  > > requests I see a 15-20 to 1 ratio of mod_proxy/mod_perl processes at
>  > > "my" site. And that is serving <500byte stuff.
>  >
>  > I'm not following.  Everyone agrees that we don't want to have big
>  > mod_perl processes waiting on slow clients.  The question is whether
>  > tuning your socket buffer can provide the same benefits as a proxy server
>  > and the conclusion so far is that it can't because of the lingering close
>  > problem.  Are you saying something different?
>
>  A tcp close is supposed to require an acknowledgement from the
>  other end or a fairly long timeout.  I don't see how a socket buffer
>  alone can change this.Likewise for any of the load balancer
>  front ends that work on the tcp connection level (but I'd like to
>  be proven wrong about this).

Solaris lets a user-level application close() a socket immediately 
and go on to do other work. The sockets layer (the TCP/IP stack) will 
continue to keep that socket open while it delivers any buffered 
sends - but the user application doesn't need to know this (and 
naturally won't be able to read any incoming data if it arrives). 
When the tcp send buffer is empty, the socket will truly close, with 
all the usual FIN et. al. dialogue.

Anyway, since the socket is closed from the mod_perl point of view, 
the heavyweight mod_perl process is no longer tied up. I don't know 
if this holds true for Linux as well, but if it doesn't, there's 
always the source code.

The socket buffers on most Unix and Unix-like OSes tend to be 32kB to 
64 kB. Some OSes allow these to be tuned (ndd on Solaris).

-- Mike
-- 
Michael Blakeley   [EMAIL PROTECTED] <http://www.blakeley.com/>
 Performance Analysis for Internet Technologies

Re: further adventures with taint

[Michael Blakeley]

At 11:52 AM -0700 9/27/2000, Doug MacEachern wrote:
>On Mon, 4 Sep 2000, Michael Blakeley wrote:
>
>>  I've been running with AP616 and Taint On for three days now, and it
>>  seems to have fixed my problems. I hope so. I really hope so.
>
>still looking good?  would be good to know if this isn't a problem on the
>mod_perl side :)

Well... there are still taint errors, but they're much less frequent. 
A grep over the past 30 days shows 5 errors:

[Fri Sep 22 05:24:55 2000] [error] Insecure dependency in require 
while running with -T switch at 
/usr/local/lib/perl5/site_perl/5.6.0/MIME/Lite.pm line 2145.
[Mon Sep 18 13:46:21 2000] [error] Insecure dependency in require 
while running with -T switch at (eval 258) line 3.
[Sat Sep 16 11:14:13 2000] [error] Insecure dependency in require 
while running with -T switch at 
/usr/local/lib/perl5/site_perl/5.6.0/MIME/Lite.pm line 2145.
[Fri Sep  1 13:05:50 2000] [error] Insecure dependency in require 
while running with -T switch at 
/usr/local/lib/perl5/site_perl/5.6.0/MIME/Lite.pm line 2145.
[Wed Aug 30 11:07:47 2000] [error] Insecure dependency in require 
while running with -T switch at 
/usr/local/lib/perl5/site_perl/5.6.0/MIME/Lite.pm line 2145.

The access logs show that we've called that routine 92 times during 
that period. An error rate of 5.4% isn't thrilling, but it's better 
than the 99% errors that I saw before applying AP616. It seems that 
before AP616, the server would run ok for a while, then all queries 
of this type would err. Now it seems to be more subtle - perhaps the 
patch causes Perl to clean up its error, so I only get one failure at 
a time.

I don't understand why it reports a line number in some cases, and 
the eval in others. The access log shows that all these taint errors 
accessed the same URI with similar inputs. Of course, pointer errors 
tend to exhibit this kind of unpredictable behavior, and AFAICT 
"taint" is just a flipped bit inside perl.

The line referenced above, BTW, is unexceptional to my eye:
require Net::SMTP;

So there may still be a lurking post-AP616 bug or two. I'll certainly 
keep an eye on perl.com and try 5.6.1 when it's released. But I don't 
really suspect mod_perl at this point.

-- Mike

Re: further adventures with taint

[Michael Blakeley]

At 3:58 PM -0700 7/25/2000, Michael Blakeley wrote:
>Solaris 2.6, Perl 5.6, apache 1.3.9, mod_perl 1.24.
>
>I see intermittent Apache error_log entries like:
>
>[Mon Jul 24 04:08:02 2000] [error] Insecure dependency in require 
>while running with -T switch at (eval 85) line 3.
>
>OK, I (normally) run with PerlTaintCheck on, and I've read perlsec. 
>My PATH is ''. How can I find out where this insecure dependency 
>_is_, so I can turn taint back on? The timestamp tells me which 
>script failed, and it uses a whole slew of modules.
>   use strict;
>   use Apache::Constants qw/:http/;
>   use LWP;
>   use tuner;
>   use MIME::Lite;
>   use HTML::Parse;
>
>I suspect MIME::Lite, but the code won't work if I remove it. How 
>can I be sure if it's to blame, os I can attempt to patch it (or its 
>dependencies)? Is there anything like a Taint::Carp, that'd give me 
>a full back-trace from the insecure code (and works under mod_perl)?
>
>Running with PERL5OPT=-d is no good; the problem never shows up. 
>perl -cwT shows nothing (which confirms that it's a runtime error, I 
>suppose).
>
>The Guide, perlrun, perlsec, and the perl books I've read are... a 
>little weak on Taint debugging techniques. If I can ever work 
>through the problems I'm having, I'll be delighted to contribute 
>some docs on this.

I think I've worked through the problem. Alas, I haven't uncovered 
any cool new debugging techniques to document for the list. Instead, 
I lucked into a fix.

http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2000-07/msg00599.html
http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2000-08/msg00126.html

Basically it seems that AP616 (ActiveState's mostly-Win32 patch to 
date) is a pretty good 5.6.1 "release candidate" for Solaris and 
Linux, too. It fixes several known bugs in 5.6.0 (none of them are 
directly Taint-related, but...).

I did have to rebuild perl, mod_perl, and apache - but I'm used to 
that. Also, AP616 added a test for Sys::Syslog, which failed on my 
Solaris system. 5.6.0 didn't have this test, and I don't use 
Sys::Syslog, so I just removed the test. Also, perl -V now returns 
some ActiveState info (which I suppose you could remove).

I've been running with AP616 and Taint On for three days now, and it 
seems to have fixed my problems. I hope so. I really hope so.

-- Mike

Re: param quirk

[Michael Blakeley]

At 3:37 PM -0700 8/16/2000, Michael Blakeley wrote:

>  But the problem is that
>   [EMAIL PROTECTED]
>also comes through as
>   email=mike [EMAIL PROTECTED]
>instead of
>   [EMAIL PROTECTED]

Never mind; my test wasn't correct. Apparently wget decodes the input 
URL before performing the fetch (it's lost some credibility with me). 
Just for the record, the better test is to telnet directly to the 
server port:

$ telnet localhost 80

GET /test?email=foo%2bbar HTTP/1.0

HTTP/1.1 200 OK
Server: Apache/1.3.9 (Unix) mod_perl/1.24

email=foo+bar

$ telnet localhost 80

GET /test?email=foo+bar HTTP/1.0

HTTP/1.1 200 OK
Server: Apache/1.3.9 (Unix) mod_perl/1.24

email=foo bar

The "test" handler is pretty obvious, but here's the heart of it:

my $tab = $r->{apr}->parms;
for (keys %$tab) {
 $r->print("$_=" . $tab->{$_} . "\n");
}

Anyway, I wanted to close the loop in case anyone was wondering about 
my sanity (it comes and goes).

-- Mike

param quirk

[Michael Blakeley]

It seems that Apache::Request::param returns inputs like
[EMAIL PROTECTED]
as
email=mike [EMAIL PROTECTED]

I can live with that, since + is a reserved character and lots of 
apps expect '+' to read as ' '. But the problem is that
[EMAIL PROTECTED]
also comes through as
email=mike [EMAIL PROTECTED]
instead of
[EMAIL PROTECTED]

Now, this seems like double-unescaping to me. I've been looking 
through the source to see if I can spot the bug, but maybe someone 
out there already knows? where to look?

thanks,
-- Mike

Re: further adventures with taint

[Michael Blakeley]

At 11:03 PM + 7/26/2000, Jeremy Howard wrote:
>  > I see intermittent Apache error_log entries like:
>>
>>  [Mon Jul 24 04:08:02 2000] [error] Insecure dependency in require
>>  while running with -T switch at (eval 85) line 3.
>>  ...
>>  I suspect MIME::Lite, but the code won't work if I remove it
>
>Yes, MIME::Lite needs special treatment to be taint-safe. Get the latest
>version from CPAN, then do a 'perldoc taint'. I submitted a patch to the
>author which he has documented, that enables a taint-safe mode.

cpan> i MIME::Lite 
Module id = MIME::Lite
 DESCRIPTION  Single module for composing simple MIME msgs
 CPAN_USERID  ERYQ (Eryq <[EMAIL PROTECTED]>)
 CPAN_VERSION 2.101
 CPAN_FILEE/ER/ERYQ/MIME-Lite-2.101.tar.gz
 DSLI_STATUS  RdpO (released,developer,perl,object-oriented)
 MANPAGE  MIME::Lite - low-calorie MIME generator
 INST_FILE/usr/local/lib/perl5/site_perl/5.6.0/MIME/Lite.pm
 INST_VERSION 2.101

$ perldoc taint
No documentation found for "taint".

$ perldoc MIME::Lite
  Version 1.142
  Added new, taint-safe invocation of "sendmail", one
  which also sets up the `-f' option.  Unfortunately, I
  couldn't make this automatic:  the change could have
  broken a lot of code out there which used
  send_by_sendmail() with unusual "sendmail" variants. So
  you'll have to configure "send" to use the new
  mechanism:

  MIME::Lite->send('sendmail');   ### no args!

  Thanks to Jeremy Howard for suggesting these features.

But I'm using MIME::Lite->send("smtp", "localhost"), which the author 
describes as taint-safe. It certainly should be taint-safe, since the 
message is sent out via socket communication in Net::SMTP and 
friends. I'm using it primarily so that I can munge headers freely - 
postfix understandably doesn't allow complete freedom from the 
command line.

>Apache always prints the line number and module the error occurs on when
>I've had taint issues. I don't know why it doesn't for you... maybe you
>just need to add:
>use Apache qw(warn);
>at the start of your modules?

I'll try that, but Apache::warn is supposed to override CORE::warn. If I put
warn "this is a warning";
in my code, it works as documented.

>I suggest rereading perldoc perlsec a few more times--there's a lot of
>info in there and it took me a while for it to sink in. Setting your
>ENV{...} safely isn't enough--any unsafe function (such as open() used by
>MIME::Lite) must have its input data cleaned unless it's input data is a
>constant.

I'm only passing scalar text to MIME::Lite - no file attachments, so 
it shouldn't open(). Anyway, if it were, why don't I see "Insecure 
dependency in open" instead of "...in require"?

Thanks for the ideas, though.

-- Mike

further adventures with taint

[Michael Blakeley]

Solaris 2.6, Perl 5.6, apache 1.3.9, mod_perl 1.24.

I see intermittent Apache error_log entries like:

[Mon Jul 24 04:08:02 2000] [error] Insecure dependency in require 
while running with -T switch at (eval 85) line 3.

Here's what perldiag says:
  Insecure dependency in %s
  (F) You tried to do something that the tainting
  mechanism didn't like.  The tainting mechanism is turned
  on when you're running setuid or setgid, or when you
  specify -T to turn it on explicitly.  The tainting
  mechanism labels all data that's derived directly or
  indirectly from the user, who is considered to be
  unworthy of your trust.  If any such data is used in a
  "dangerous" operation, you get this error.  See the
  perlsec manpage for more information.

OK, I (normally) run with PerlTaintCheck on, and I've read perlsec. 
My PATH is ''. How can I find out where this insecure dependency 
_is_, so I can turn taint back on? The timestamp tells me which 
script failed, and it uses a whole slew of modules.
use strict;
use Apache::Constants qw/:http/;
use LWP;
use tuner;
use MIME::Lite;
use HTML::Parse;

I suspect MIME::Lite, but the code won't work if I remove it. How can 
I be sure if it's to blame, os I can attempt to patch it (or its 
dependencies)? Is there anything like a Taint::Carp, that'd give me a 
full back-trace from the insecure code (and works under mod_perl)?

Running with PERL5OPT=-d is no good; the problem never shows up. perl 
-cwT shows nothing (which confirms that it's a runtime error, I 
suppose).

The Guide, perlrun, perlsec, and the perl books I've read are... a 
little weak on Taint debugging techniques. If I can ever work through 
the problems I'm having, I'll be delighted to contribute some docs on 
this.

I'd really like to run with PerlTaintCheck on - any ideas?

thanks,
-- Mike

[ jobs ] Sonicbox mod_perl coders

[Michael Blakeley]

Sonicbox, Inc.  is hiring mod_perl 
programmers. Experience with HTTP, HTML, and the major mod_perl tools 
is assumed. Experience with SQL, PostgresQL, Solaris, and CVS is 
helpful, but not essential.

The marketeers tell me that "Sonicbox makes listening to high quality 
Internet radio and audio from around the world as easy as tuning your 
FM dial." The corporate culture is relaxed, with a mix of technical 
folks and music industry folks. We're well-funded, pre-IPO, and not 
pointy-haired at all. Sonicbox is based in Mountain View, CA, with 
employees scattered around the USA.

If interested, please contact [EMAIL PROTECTED] .

thanks,
-- Mike

Re: bogus taint error?

[Michael Blakeley]

At 11:48 AM -0400 7/3/2000, Vivek Khera wrote:
>  >>>>> "MB" == Michael Blakeley <[EMAIL PROTECTED]> writes:
>
>MB> At 3:37 PM -0700 7/2/2000, Michael Blakeley wrote:
>>>  At 2:00 PM -0700 7/2/2000, Michael Blakeley wrote:
>>>>  With perl 5.6.0, Solaris 2.6, apache 1.3.9, and mod_perl 1.24, I'm
>>>>  seeing intermittent taint errors like
>>>>  [Sat Jul  1 18:50:13 2000] [error] PerlRun: `Insecure dependency in
>>>>  require while running with -T switch at /foo.pl line 5.
>
>MB> This is just plain weird. My last resort was to turn taint off:
>
>I've had to do that on a production server as well.  The development
>box still works fine with taint on, curiously enough.  The production
>box *used* to work fine with taint checks, and then suddenly one day
>decided to stop working.

What platform and version of perl do you use? Perhaps there's a correlation.

Oddly, this is the first real difficulty I've had since moving to 
5.6.0. There are some known bugs in 5.6.0 now, but I don't know if 
any of them are relevant to this problem

thanks,
-- Mike

Re: bogus taint error?

[Michael Blakeley]

At 10:49 AM -0500 7/3/2000, Matt Carothers wrote:
>On Sun, 2 Jul 2000, Michael Blakeley wrote:
>
>>  This is just plain weird. My last resort was to turn taint off:
>>
>>  $ ls -l logs/httpd.pid
>>  -rw-rw-rw-   1 root other  6 Jul  2 19:23 logs/httpd.pid
>>  $ grep -i taint conf/httpd.conf
>  > #PerlTaintCheck On
>>  $ ls -l conf/httpd.conf
>>  -rw-r--r--   1 root other   7437 Jul  2 17:22 conf/httpd.conf
>>
>>  I'm not including any sort of startup.pl file. So taint is off, right?
>
>Is the script running setuid or setgid?  If the script's real and effective
>uids or gids don't match, perl enables taint mode automatically.  You might
>add some debugging code to verify that $< == $> and $( == $).

There's no sticky bit on the file, and the uid and gid of the file 
are the same as the apache user.

I think I may have an answer for taint's on-when-off behavior: I'd 
been syncing /usr/local/lib/perl5 all along, but it seems that some 
of the perl binaries may have drifted apart. Since syncing them up,
#PerlTaintCheck On
seems to disable taint, as it should. I'm not sure what drifted out 
of sync - I haven't rebuilt perl, apache, or mod_perl in months.

But I still have to turn PerlTaintCheck off in production, even 
though the same script, on the same OS, with the same perl binaries 
and libraries, and the same mod_perl and apache binaries, works fine 
for dev. I'd love to understand why - a race condition triggered by 
high concurrency, maybe? So far I haven't been able to trigger it 
with ab -c 5 -n 1000 against the dev server.

-- Mike

Re: bogus taint error?

[Michael Blakeley]

At 3:37 PM -0700 7/2/2000, Michael Blakeley wrote:
>At 2:00 PM -0700 7/2/2000, Michael Blakeley wrote:
>>With perl 5.6.0, Solaris 2.6, apache 1.3.9, and mod_perl 1.24, I'm 
>>seeing intermittent taint errors like
>>[Sat Jul  1 18:50:13 2000] [error] PerlRun: `Insecure dependency in 
>>require while running with -T switch at /foo.pl line 5.

This is just plain weird. My last resort was to turn taint off:

$ ls -l logs/httpd.pid
-rw-rw-rw-   1 root other  6 Jul  2 19:23 logs/httpd.pid
$ grep -i taint conf/httpd.conf
#PerlTaintCheck On
$ ls -l conf/httpd.conf
-rw-r--r--   1 root other   7437 Jul  2 17:22 conf/httpd.conf

I'm not including any sort of startup.pl file. So taint is off, right?

error log:
[Sun Jul  2 21:20:20 2000] [error] Insecure dependency in require 
while running with -T switch at (eval 756) line 3.

This system used to be stable, and I can't find anything that's 
changed. Sunspots?

Thanks for any ideas.

-- Mike

Re: bogus taint error?

[Michael Blakeley]

At 2:00 PM -0700 7/2/2000, Michael Blakeley wrote:
>With perl 5.6.0, Solaris 2.6, apache 1.3.9, and mod_perl 1.24, I'm 
>seeing intermittent taint errors like
>[Sat Jul  1 18:50:13 2000] [error] PerlRun: `Insecure dependency in 
>require while running with -T switch at /foo.pl line 5.
>
>head -6 foo.pl shows:
>
>#!perl
>
>use Apache::Constants qw/:http/;
>use LWP;
>use MIME::Lite;
>use strict;
>
>Seeing http://forum.swarthmore.edu/epigone/modperl/luthesmex I 
>thought that moving 'use strict' to the top might help:
>
>#!perl
>
>use strict;
>use Apache::Constants qw/:http/;
>use LWP;
>use MIME::Lite;
>
>It didn't help.

Scratch the rest of that. The line number did move with MIME::Lite 
after all. But... I'm still hitting the taint errors. The weird thing 
is that I can do

perl -Tw
use strict;
use Apache::Constants qw/:http/;
use LWP;
use MIME::Lite;


all I like, and never see the taint error. Ideas?

thanks,
-- Mike

bogus taint error?

[Michael Blakeley]

With perl 5.6.0, Solaris 2.6, apache 1.3.9, and mod_perl 1.24, I'm 
seeing intermittent taint errors like
[Sat Jul  1 18:50:13 2000] [error] PerlRun: `Insecure dependency in 
require while running with -T switch at /foo.pl line 5.

head -6 foo.pl shows:

#!perl

use Apache::Constants qw/:http/;
use LWP;
use MIME::Lite;
use strict;

Seeing http://forum.swarthmore.edu/epigone/modperl/luthesmex I 
thought that moving 'use strict' to the top might help:

#!perl

use strict;
use Apache::Constants qw/:http/;
use LWP;
use MIME::Lite;

It didn't help. But the really odd thing is that the error message 
didn't change at all:
[Sun Jul  2 13:16:25 2000] [error] PerlRun: `Insecure dependency in 
require while running with -T switch at /foo.pl line 5.

So... perl was objecting to MIME::Lite before, and now it's objecting 
to LWP? This smells wrong to me. I've never been able to reproduce 
this problem under httpd -X; and it usually comes up only after the 
server's been running for a while.

Has anyone seen this before? Any suggestions for debugging?

thanks,
-- Mike

Re: Apache::Registry RFE

[Michael Blakeley]

At 12:40 PM -0700 6/1/2000, Doug MacEachern wrote:
>On Tue, 30 May 2000, Michael Blakeley wrote:
>
>  > $ diff -w Registry.pm.orig Registry.pm
>>  26a27,28
>>  >  $Apache::Registry::AutoLog ||= 32768;
>>  >
>>  155c157,160
>>  <   $r->log_error($errsv);
>>  ---
>>  >{
>>  >   local $^W = 0;
>>  >   $r->log_error($errsv) if 
>>$Apache::Registry::AutoLog > 0+$errsv
>>  >}
>>
>>  This way, setting $Apache::Registry::AutoLog = 300 gives me the
>>  behavior I want (no extra log lines for 302, 304, 500) while still
>>  letting "true" eval errors through. The default value should come
>>  pretty close to the original behavior.
>
>thanks for the patch, but we've stopped adding features to
>Apache::Registry.  you can implement this with a subclass of
>Apache::RegistryNG, something like so:

Thanks - but RegistryNG causes problems elsewhere in my code. 
Apache::File::mtime() doesn't seem to work with RegistryNG. The code:

$mtime = 954291795;
 if (defined $mtime) {
 $r->update_mtime($mtime);
 warn "mtime=".join(',', $mtime, $r->mtime);
 $r->set_last_modified;
 }

works fine in Apache::Registry, but seems to set mtime to 0 in 
Apache::RegistryNG. Since I can't seem to get a handle on this 
problem, I've gone back to Apache::Registry.

Solaris 2.6 + patches, Apache 1.3.9, mod_perl 1.24, perl -V says

Summary of my perl5 (revision 5.0 version 6 subversion 0) configuration:
   Platform:
 osname=solaris, osvers=2.6, archname=sun4-solaris
 uname='sunos mail 5.6 generic_105181-17 sun4u sparc sunw,ultra-5_10 '
 config_args='-des -Dcc=gcc -Dprefix=/usr/local -Doptimize=-g -O 
-Duseshrplib'
 hint=recommended, useposix=true, d_sigaction=define
 usethreads=undef use5005threads=undef useithreads=undef 
usemultiplicity=undef
 useperlio=undef d_sfio=undef uselargefiles=define
 use64bitint=undef use64bitall=undef uselongdouble=undef usesocks=undef
   Compiler:
 cc='gcc', optimize='-g -O', gccversion=2.95.1 19990816 (release)
 cppflags='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include'
 ccflags ='-DDEBUGGING -fno-strict-aliasing -I/usr/local/include 
-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64'
 stdchar='unsigned char', d_stdstdio=define, usevfork=false
 intsize=4, longsize=4, ptrsize=4, doublesize=8
 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=16
 ivtype='long', ivsize=4, nvtype='double', nvsize=8, 
Off_t='off_t', lseeksize=8
 alignbytes=8, usemymalloc=y, prototype=define
   Linker and Libraries:
 ld='gcc', ldflags =' -L/usr/local/lib '
 libpth=/usr/local/lib /lib /usr/lib /usr/ccs/lib
 libs=-lsocket -lnsl -ldl -lm -lc -lcrypt -lsec
 libc=/lib/libc.so, so=so, useshrplib=true, libperl=libperl.so
   Dynamic Linking:
 dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags='  -R 
/usr/local/lib/perl5/5.6.0/sun4-solaris/CORE'
 cccdlflags='-fPIC', lddlflags='-G -L/usr/local/lib'


Characteristics of this binary (from libperl):
   Compile-time options: DEBUGGING USE_LARGE_FILES
   Built under solaris
   Compiled at Apr  2 2000 14:37:00
   @INC:
 /usr/local/lib/perl5/5.6.0/sun4-solaris
 /usr/local/lib/perl5/5.6.0
 /usr/local/lib/perl5/site_perl/5.6.0/sun4-solaris
 /usr/local/lib/perl5/site_perl/5.6.0
 /usr/local/lib/perl5/site_perl/5.005/sun4-solaris
 /usr/local/lib/perl5/site_perl/5.005
 /usr/local/lib/perl5/site_perl
 .

-- Mike

Re: Apache::Registry RFE

[Michael Blakeley]

Of course there was an error...

At 9:54 PM -0700 5/30/2000, Michael Blakeley wrote:

>>$r->log_error($errsv) if $Apache::RegistryYA::AutoLog;

s/RegistryYA/Registry

I'm also having second thoughts about the binary nature of this 
thing... I'd forgotten that $errsv might well be a text message like:
Can't locate object method foo via package "Apache" at...
which I'd quite like to see in the log.

So maybe...

$ diff -w Registry.pm.orig Registry.pm
26a27,28
>  $Apache::Registry::AutoLog ||= 32768;
>
155c157,160
<   $r->log_error($errsv);
---
>{
>   local $^W = 0;
>   $r->log_error($errsv) if $Apache::Registry::AutoLog > 0+$errsv
>}

This way, setting $Apache::Registry::AutoLog = 300 gives me the 
behavior I want (no extra log lines for 302, 304, 500) while still 
letting "true" eval errors through. The default value should come 
pretty close to the original behavior.

-- Mike

Apache::Registry RFE

[Michael Blakeley]

I finally decided to do away with an Apache::Registry quirk that's 
bugged me for months. The problem, briefly, is that Apache::Registry 
writes to the error log whenever its eval returns non-zero. So if you 
want your registry script to return a custom error, you can write a 
log message and call Apache::exit(SERVER_ERROR), but you'll get two 
lines in error_log (yours, and Apache::Registry's). Same for 302, 
304, etc.

Apache::RedirectLogFix.pm is one solution - but only for redirect. 
One could also do all the logging in a module, but why duplicate 
Apache's existing code?

Thus the patch... Basically, logging is on by default, and can be 
turned off with a package variable. I think that the "right way" 
would be to sense the difference between a code error and a 
deliberate return value... but I suspect that would require 
modifications to Apache::exit(), with which I'm not inclined to muck.

$ diff -w Registry.pm.orig Registry.pm
26a27,28
>  $Apache::Registry::AutoLog ||= 1;
>
155c157
<   $r->log_error($errsv);
---
>$r->log_error($errsv) if $Apache::RegistryYA::AutoLog;

Anyway, hopefully this will be useful to someone else too. If there's 
a better way, I'm sure someone will tell me RSN :-).

-- Mike

Re: patch for Apache::Session::Store::Postgres

[Michael Blakeley]

At 9:10 AM -0700 5/29/2000, Jeffrey W. Baker wrote:
>On Mon, 29 May 2000, Michael Blakeley wrote:
>
>>  Patch for Apache::Session::Store::Postgres, from
>>  Apache-Session-1.51.tar.gz, to resolve problems with
>>
>>  >prepare_cached(SELECT a_session FROM sessions WHERE id = ? FOR
>>  >UPDATE) statement handle DBI::st=HASH(0x369a2c) is still active
>>
>>  after a transient error. The solution is to call sth->finish()
>>  whether the SELECT was successful or not.
>>
>>  diff Store/Postgres.pm.orig Store/Postgres.pm
>>  78a79,81
>>  ># success or failure, don't leave FOR UPDATE lying around
>>  >  $self->{materialize_sth}->finish;
>>  >
>>  83,84d85
>>  < $self->{materialize_sth}->finish;
>>  <
>
>Hrmm, I'm not really an expert here.  If I do a SELECT ... FOR UPDATE on a
>row that doesn't exist, shouldn't that just do nothing?

I'm no expert either - but I believe that the problem is the 
"unfinished" sth hanging around in DBI or DBD::Pg, rather than any 
results or lack thereof. Note that the error comes from 
DBI::prepare_cached(), not from the postgres backend. One _could_ 
regard this as a DBD::Pg bug, if it's not present in other DBD 
modules... but what's wrong with the fix above?

According to perldoc DBD::Pg, the prepare/execute thing is fairly 
pointless in Postgres anyway. PG doesn't know prepared statements 
from /dev/null, so you might as well just selectall_arrayref every 
time. I've done some benchmarks that show a very slight improvement 
in some cases (possibly due to state maintained by DBD::Pg), but it's 
less than 10%.

>To squash this warning, we could just as easily use the allow_active flag
>in the prepare method.

True, but the above seems cleaner to me. Isn't dying and leaving 
unfinished statement handles a bit untidy, like leaving unclosed 
database handles, or unclosed file handles? It's a potential memory 
leak as well, isn't it?

It's not a warning, BTW - for my Apache::Session application, it's 
fatal. I've had to patch all previous versions of Apache::Session in 
a similar way (somewhere you may have an older email from me on this 
subject). The fix originally came from someone else, but I can't find 
my notes at the moment to give proper credit.

-- Mike

patch for Apache::Session::Store::Postgres

[Michael Blakeley]

Patch for Apache::Session::Store::Postgres, from 
Apache-Session-1.51.tar.gz, to resolve problems with

>prepare_cached(SELECT a_session FROM sessions WHERE id = ? FOR 
>UPDATE) statement handle DBI::st=HASH(0x369a2c) is still active

after a transient error. The solution is to call sth->finish() 
whether the SELECT was successful or not.

diff Store/Postgres.pm.orig Store/Postgres.pm
78a79,81
># success or failure, don't leave FOR UPDATE lying around
>  $self->{materialize_sth}->finish;
>
83,84d85
< $self->{materialize_sth}->finish;
<

-- Mike

Apache::RegistryBB patch for 'use strict'

[Michael Blakeley]

Apache::RegistryBB from mod_perl-1.24.tar.gz needs a small patch for 
'use strict'.

diff Apache/RegistryBB.pm.orig Apache/RegistryBB.pm
5c5
< use Apache::Constants qw(NOT_FOUND FORBIDDEN OK);
---
>  use Apache::Constants qw(NOT_FOUND FORBIDDEN OK DECLINED);

-- Mike

CGI::Delete for Apache::Request

[Michael Blakeley]

I've been migrating some code off of the ENV-dependent methods in 
CGI.pm, so I can turn of PerlSetupEnv. Anyway, I couldn't find an 
Apache::Request method that had the functionality of CGI::Delete. 
Since I use Delete extensively, I coded a replacement based on the 
param() prototype in Apache::Request. This may be useful to other 
programmer porting CGI.pm code.

# for compatibility with CGI::Delete - call with $apr->Delete("foo")
sub Delete {
 my $self = shift;
 my $tab = $self->parms;
 for (@_) {
 $tab->unset($_);
 }
} # end sub Delete

Trivial, once you know how. But maybe worth including in a future rev 
of Apache::Request? Or at least documenting? I spent some time just 
finding "unset".

It'd be even better if unset() would take a list argument, of course

-- Mike