Re: lame load balancer, mod_proxy, and sticky sessions

[Perrin Harkins]

[EMAIL PROTECTED] wrote:
> The idea to modify mod_proxy.c is probaly the most 
> convenient solution. Instead of configure backend 
> machine from the ProxyPass setting, you may specifically 
> assign it to the one in the cookie, which takes only a 
> few lines of code to change --- well, plus extra steps 
> to handle the cookie.

But you're not accounting for the possibility of server failure on the 
backend.  A proper load-balancer (including the open source ones and 
mod_backhand) would detect dead servers and handle the failover to 
another server.  Building this yourself is probably not worth it, with 
so many open source solutions already available.

- Perrin

Re: mod_perl statistics on securityspace.com

[Perrin Harkins]

Mark Coffman wrote:
> I can't imagine that mod_perl will ever be the major "scripting" language
> since it, by nature, is unrestrictive.  On a multi-user/multi-host server, I
> think I'd rather PHP be run than mod_perl, simply because I don't want sites
> stepping on each other's toes and have to worry about restarting httpd.

Isn't PHP just as dangerous as mod_perl when run in the server process 
(as opposed to CGI mode) on a multi-user virtual host server?

- Perrin

Re: Morning bug w/out using Apache::DBI?

[Perrin Harkins]

Your problem doesn't sound like something that Apache::DBI would cause. 
  Deadlock problems are caused by conflicting updates, which could only 
be coming from your code.

> I'm not positive but maybe it seemed like there were 
> stale handlers lying around that weren't being closed

If you put database handles in globals or in closures, or in some data 
structure (like an object) that gets put into a global, it will stay 
around just like it does with Apache::DBI.  Is it possible you did 
something like partially read a set of results and leave the rest 
dangling?  You could try putting some $sth->finish() statements in your 
code to deal with this.

> Regardless, I was wondering what a solution would be for this situation. 
> Currently, I do not do a ping if a handler goes stale and attempt a 
> reconnect.  Should I implement this feature?

No, you would just be rewriting Apache::DBI.

- Perrin

Re: performance regarding mod_perl vs mod_c with embedded perl

[Perrin Harkins]

Pierre Laplante wrote:
> If I compiled a c module that embed a perl interpreter and
> I benchmark this again the same module in mod_perl
> 
> I got a big difference in favor of mod_c.

It will be hard for anyone to give you a good answer unless you post the 
code that you benchmarked.  At a guess, I would say that mod_perl is 
probably doing more work in order to give you access to apache 
internals.  You also may not have configured mod_perl for best 
performance.  For example, if you have it configured to emulate CGI by 
setting up environment variables that will take more time.

- Perrin

Re: performance regarding mod_perl vs mod_c with embedded perl

[Perrin Harkins]

Pierre Laplante wrote:
> I do not use mod_perl with CGI emulation.

Actually PerlSetupEnv is on by default.  Put PerlSetupEnv Off in your 
httpd.conf.

> Here is my mod_perl code:

You are not running the same Perl code in both situations.  Under 
mod_perl, you are using Apache::File and various methods of the perl 
version of the request record ($r).  In your mod_c version, you do those 
things in C.  That will make a difference.  You seem to be using 
Error.pm to trap errors in the mod_perl version and normal eval/die 
constructs (called from C) in the mod_c version.  Error.pm will slow 
things down a little.  Also, if you are not handling any phases of the 
request other than the content handler, you can improve the speed of 
mod_perl by compiling it without the hooks for the other phases.

- Perrin

Re: Error messages in Apache::Registry

[Perrin Harkins]

William McKee wrote:
>>The way that most people would recommend in A::R or anything else is to
>>trap errors with eval blocks and then handle them appropriately.
> 
> I thought I was doing that at a global level by throwing an error with 
> die() that was caught by CGI::Carp. I'm realizing that it isn't so simple.

Matt wrote some stuff for the guide explaining why the die handler is 
not the best approach.  Basically, it means all errors will be treated 
the same.

> Oh, so if I handle the error myself, I should unset $@?

No, I think I was wrong about that.  You shouldn't need to unset it 
yourself.

> One of the the members of the CGI::Application mailing list suggested 
> using Graham Barr's Error module and a wrap around the instantiation 
> script. It's an interesting approach which I'm going to take a look at 
> today.

I don't recommend that.  There's no reason to use Error instead of just 
an eval block.  It can be this simple:

eval {
   ... your code here ...
};
if ($@) {
   ... print nice error page with $@ message ...
}

This will only work if you turn off all the DIE handlers.

> I still am having no luck. I've walked through the execution with the 
> debugger and am finding that Carp::longmess_heavy is catching my die() 
> calls despite the fact that I have set `local $SIG{__DIE__} = \&mydie;`.

As you discovered, this seems to be interference from the debugger.  I'm 
not sure how to turn that off, although you could probably look at 
perl5db.pl and edit it.

- Perrin

Re: mod_perl 2.x vs. mod_perl 1.x benchmarks

[Perrin Harkins]

Josh Chamas wrote:
> I just did a benchmarks to compare mod_perl + apache versions 1 & 2.

Cool.

Any idea why bytes/hit is lower on apache 2?  Are some headers being 
omitted?

- Perrin

Re: top for apache? [OT]

[Perrin Harkins]

Nigel Hamilton wrote:
>   It would be great to have a similar tool for mod_perl/apache.

The closest thing available is a combination of mod_status and 
Apache::Status.  If you haven't tried these yet, give them a shot.  They 
provide a good deal of information.

- Perrin

Re: Apache::Session and user sessions

[Perrin Harkins]

Todd W wrote:
> Im looking at Apache::Session and trying to figure out what it does.

It provides shared storage of a hash of data, and gives you a unique ID 
that you can tie to a user.

> From what I
> can tell, Apache::Session will only give generic sessions, of which I know
> nothing about the user untill they give me information during that 
> particular session.

It's just a storage mechanism.  Typically the procedure is that one a 
user identified herself with some kind of login process, you put her 
user ID (a primary key to a database table) into the session, and keep 
it as a key for accessing that data.

> I have a table with some basic user information (first name, last name, 
> address,
> phone number, etc...).

That's permanent data, not session data.  Session data is transient.

> What i did was created the two columns, and hoped it would work without 
> the id column being the primary key.

It won't.  All of the Apache::Session data is in a blob in the a_session 
column.  It has no access to the other columns.

> So now Trying to decide what to do, in a perlHeaderParserHandler Ill 
> just get an
> id from Sys::UniqueID, send it to the browser each request in a cookie or
> whatever, then use DBI::Tie to reinstate the session for each request.
> (Thinking about it, that sounds easier than Apache::Session anyways)

Isn't your user table referenced by a user ID?  You have to connect the 
user ID to a browser somewhere.  The normal way to do this is give the 
browser an ID (the session ID) and then store the relationship with 
Apache::Session.  If you have no other transient data besides the user 
ID, you can skip Apache::Session and just send a user ID cookie.  Make 
sure you have security in place to prevent people from simply entering 
another user ID in their cookie and gaining access to another person's 
information.

By the way Tie::DBI is slow.  Writing some kind of module for accessing 
your specific user table would be faster.

- Perrin

Re: same module with different pragmas

[Perrin Harkins]

[EMAIL PROTECTED] wrote:
> In the sets of applications that runs under mod_perl on our webserver
> we need the same modules twice, but with different pragmas.
> 
> app1:  use module qw(standard pragma1);
> app2:  use module qw(standard pragma2);
> 
> now, of course - whichever application is needed first it loads the
> module with the mentioned pragma. Later - when the second app wants to
> load the module, mod_perl uses the already module with the "wrong"
> pragma.

The problem is that CGI.pm is a bunch of nasty old code that uses global 
variables like they were going out of style (which they are).  The 
simplest solution might be to stop using CGI.pm and switch to something 
faster and cleaner.  However, failing that you may have to hack the 
CGI.pm code a little to make a way to turn the XHTML header on and off. 
  (Although, can't you just use -no_xhtml all the time?)

> Is there any trick around this problem or do I need to copy the module
> and alter its name ?

That won't help.  It's about the module namespace, not the file name.

- Perrin

Re: modules and pragmas - part II

[Perrin Harkins]

[EMAIL PROTECTED] wrote:
> At the first request each instance prints out the no_xhtml-header, but
> at the second call the no_xhtml-pragma is forgotten and the
> xhtml-header is printed out.

Are you setting $CGI::XHTML to 0 somewhere?

> btw and OT : in the previous thread there have been rumours about
> better things than the CGI-module. What could you recommend instead ?
> (compatible modules prefered cause I really use the sticky- and
> nostickyfeatures :)

There are many.  The most compatible is CGI::Simple.  If you need CGI 
stuff but not the HTML generation, you can use Apache::Request or CGI_Lite.

- Perrin

Re: [Q][LONG] using IPC::Shareable to cache data, Apache doesnt start

[Perrin Harkins]

Juan Natera wrote:
> The worst of all is that Apache simply doesnt start, and I get no error 
> message at all.

The error might be on the console, or you could try capturing it and 
writing it to a file.  However, I suggest you ditch IPC::Shareable since 
it's dog slow.  Use MLDBM::Sync, Cache::FileCache, IPC::MM, or Cache::Mmap.

- Perrin

Re: perl script not reloading

[Perrin Harkins]

Michael Grant wrote:
> It seems that as I work on my script, Apache doesn't reload it when
> the script changes on disk.  

Are you using Apache::Registry?  It only reloads the main file, not any 
modules you might be using.  For that, you need to use Apache::Reload or 
Apache::StatINC (as Nigel pointed out).

> (if so, it should be documented in the mod_perl_traps man page).

It's all documented here:
http://perl.apache.org/docs/1.0/guide/

- Perrin

Re: Apache::SharedMem

[Perrin Harkins]

[EMAIL PROTECTED] wrote:
> We are using IPC::MM and it works great.

IPC::MM is the fastest game in town.  It's only drawback is that the 
data is not persistent, so if you want your cache to persist across 
restarts it won't work for you.

Apache::SharedMem and all the other modules based IPC::ShareLite or 
IPC::Shareable are slow.  If IPC::MM won't work for you, I'd suggest 
MLDBM::Sync, Cache::FileCache, or Cache::Mmap.

- Perrin

Re: identifying a unique browser session

[Perrin Harkins]

Martin Moss wrote:
> How would I go about identifying if a user logs in from 2 different
> browsers?

Can you be more specific?  Do you mean two browser windows, or two 
different browser programs on the same machine, or two different machines?

> I Have a Session object, but I want to hold data within that session object
> that identifies which browser a user is using. So I can confirm that a user
> who provides a session key in their cookie, is checked and that that session
> key matches that browser.

If you issue each user a unique cookie, then each separate browser will 
have a separate identifying ID.  I can't tell what the problem is from 
your description.  Maybe you're looking for something like 
Apache::AuthTicket?

- Perrin

Re: memory usage problem!

[Perrin Harkins]

Also, try to find an alternative to loading all that data into memory. 
You could put it in a dbm file or use Cache::FileCache.  If you really 
have to have it in memory, load it during startup.pl so that it will be 
shared between processes.

- Perrin

Anthony E. wrote:
> look into Apache::Resource or Apache::SizeLimit which
> will allow you to set a maximum size for the apache
> process.
> 
> both can be added to your startup.pl
> 
> --- Plamen Stojanov <[EMAIL PROTECTED]> wrote:
> 
>>Hi all,
>>I have a ig mem usage problem with perl. 
>>I load 2Mb data from a database in perl hash and
>>perl takes 15Mb memory. As I 
>>use this under mod_perl - perl never returns this
>>memory to the OS. I must 
>>set a little number for MaxRequestsPerChild in order
>>to restart perl 
>>interpreter not to eat a lot of memory. 
>>Is there any solution to avoid such memory usage?

Re: memory usage problem!

[Perrin Harkins]

Eric wrote:
> What about in the case of a big query result? That is where it seems 
> like you can get killed.

Riding a bike without a helmet will get you killed; big query results 
are no problem.  All you have to do is write your program so that it 
pages through results rather than loading them all into memory at once. 
  The only time this becomes difficult is when you need to load a large 
BLOB fram the database.

- Perrin

Re: memory usage problem!

[Perrin Harkins]

Rodney Broom wrote:
> From: Eric <[EMAIL PROTECTED]>
> 
> 
>>What about in the case of a big query result?
> 
> 
> I may have come into this thread a bit late, but can't you just undefine the storage 
>when you're done with it?
> 
>   $data = $sth->fetchall_arrayref;
>   #... do some stuff;
>   $data = undef;

That will release the memory from that lexical so that perl can use it 
elsewhere, but it will not release it back to the OS.  It's a good idea 
though, in any situation where you occasionally load large chunks of 
data into a scalar.  (Not a good idea if you load large chunks of data 
into that scalar on most requests.)

- Perrin

Re: figures/resources on content via apache SSI vs. database-driven(perl DBI)

[Perrin Harkins]

grant stevens wrote:

> I think all I'm asking about is a performance comparison for a  site 
> comprised of  95% static content between Apache SSI and a mod_perl 
> db/template system.


Well, mod_include (SSI) is the best choice if it meets your needs.  A 
modern OS will cache your include files in RAM and the whole thing will 
be very fast.  Yes, some perl templating systems cache content in memory 
and might manage to squeeze out some pages slightly faster, but you will 
pay for that with hugely increased memory consumption which reduces the 
number of requests you can handle in parallel.  In the end, throughput 
from a server running mod_include will probably be significantly higher.

Apache::SSI is about the same speed as mod_include (on apache 1.x) but 
uses much more memory.  If you are just serving static files, stick with 
mod_include.

The real reason people use templating systems beyond SSI is that they 
require more power and flexibility.  At the point where you start 
needing to do things with dynamic content, you should probably kiss SSI 
goodbye.

By the way, SSI (or something very close to it) is pretty much the only 
way to go if you have thousands of unique static files.  A system like 
Mason or Template Toolkit that caches the files in memory would quickly 
blow up in that situation, and you would end up needing to work around 
it by keeping most of the pages on disk and loading them at request 
time, just like SSI.

- Perrin

Re: Forking process

[Perrin Harkins]

Aaron Johnson wrote:
> So in a nutshell
> 
> - Create a temp (flag) file
> - Generate and send the HTML to the browser which includes a meta
> refresh
> - Add the temp file to their session data (or something similar)
> - Exec a process and send the temp file name to it
> - On page refresh look to see if the temp (flag) file still exists and
> if so create a new waiting page ( meta refresh ), otherwise deliver end
> results.

Randal has a description of this process -- with code -- here:
http://www.stonehenge.com/merlyn/WebTechniques/col20.html

It's for CGI, but all the principles are the same.

- Perrin

Re: [OT] migrating from Apache to iPlanet; any mod_perl counterpart?

[Perrin Harkins]

Paul wrote:
> I know there are servlets, but I was led to believe that I would almost
> be able to drop my mod_perl modules into the iPlanet server, as if it
> has some equivelent functionality. If so, I can't find any evidence of
> it, and I'm rather skeptical.

I think your only hope is FastCGI, or PerlEx if you're running it on 
Windows.

- Perrin

Re: [OT] migrating from Apache to iPlanet; any mod_perl counterpart?

[Perrin Harkins]

Perrin Harkins wrote:
> Paul wrote:
> 
>> I know there are servlets, but I was led to believe that I would almost
>> be able to drop my mod_perl modules into the iPlanet server, as if it
>> has some equivelent functionality. If so, I can't find any evidence of
>> it, and I'm rather skeptical.
> 
> I think your only hope is FastCGI, or PerlEx if you're running it on 
> Windows.

Wait, there is also PersistentPerl (formerly SpeedyCGI), and Matt's 
PPerl.  Both on CPAN.

- Perrin

Re: [OT] migrating from Apache to iPlanet; any mod_perl counterpart?

[Perrin Harkins]

Paul wrote:
> The problem isn't so much the registry as the API.

Any use of the Apache API would have to be rewritten.  There is no way 
around that.

> I don't know how I'm going to do all that with iPlanet/LDAP without a
> lot of recoding, probably in Java. :(

There's nothing you've mentioned so far that requires Java.  It would be 
much faster to port it to FastCGI or the like.

- Perrin

Re: [OT] migrating from Apache to iPlanet; any mod_perl counterpart?

[Perrin Harkins]

On Thu, 2002-10-10 at 14:43, Paul wrote:
> > There's nothing you've mentioned so far that requires Java.  It would
> > be much faster to port it to FastCGI or the like.
> 
> I just meant that iPlanet's internal API was probably going to require
> Java or C, and not Perl.

FastCGI and PersistentPerl are both working equivalents to mod_perl with
similar capabilities and performance.  They should work just fine with
iPlanet.

> I can redo most of it as CGI if necessary, but
> some of that will require slieght-of-code like embedding data in
> cookies or hidden form elements

I don't see why that would be any different from what you currently
have.  Any state mechanism requires cookies, URLs, or hidden form values
for maintaining at least a unique ID on the client side.  There is no
other way to do it.

> I just prefer having all the data from the engine available
> from the request object, and I'm gonna miss that. 

Again, I don't understand what you think you're going to lose.  All the
basic data available under mod_perl is available under everything else
as well, albeit with a slightly different API.

- Perrin

Re: [OT] migrating from Apache to iPlanet; any mod_perl counterpart?

[Perrin Harkins]

Paul wrote:
> What I mean is that before I had a custom access
> handler installed to use MySQL without resorting to state management
> other than the http NCSA Basic Authentication header

You should be able to do that with FastCGI.  Not sure about 
PersistentPerl.  You'd have to ask the author.

- Perrin

Re: [OT] migrating from Apache to iPlanet; any mod_perl counterpart?

[Perrin Harkins]

Paul wrote:
> Looks like the FastCGI binaries are only available for Windows
> versions. We'll be on Sun Solaris, but I can probably recompile the
> source, if that doesn't cause the open-source police to come get me.

I'm afraid it's not as obvious how to do it as it is with mod_perl.  You 
may need to ask on the mailing list.

> I still think we're miscommunicating, though. We weren't even using
> Apache::Registry; the CGI speed improvement of mod_perl is only of
> incidental interest to us. The API hooks were what we were using.
> FastCGI (as I understand it, e.g., from
> http://www.fastcgi.com/devkit/doc/fastcgi-whitepaper/fastcgi.htm )
> doesn't offer any API

Everything offers an API.  CGI is an API.

If I understand correctly, you just want to write a custom access 
control thingy.  FastCGI calls that an "authorizer" and it is documented 
in that whitepaper.

- Perrin

Re: Apache::DBI and CGI::Application with lots of modules.

[Perrin Harkins]

I'm just going to point out a few problems.  These are not all related 
to your questions.

>package Holds;
>

The case of "Holds" doesn't match the example sub you posted above.  I'm 
assuming that was a typo.

>use strict;
>use Carp;
>use warnings;
>use QueryPrint;
>use vars qw($dbh $processed_hnd $status_hnd);
>use gentimeid; # generate time id based
>
>
>sub new { 
>my $invocant = shift;
>my $class = ref($invocant) || $invocant;
>

That looks like voodoo code copied from a man page.  If you call this as 
Holds->new(), you don't need that junk about ref.  (And most people 
recommend against the "new Holds" syntax.)

>my $self  = { @_ };
>bless ($self, $class);
>$dbh = db_connect();
>

You don't seem to need this.  You aren't using the database handle for 
anything in this sub and you aren't gaining anything by calling it here.

>sub GetProcessed {
>
>my $self = shift;
>
> This has a bug, somtimes the cached query doesn't stick around.
>

If you lose your database connection, Apache::DBI will reconnect.  Any 
prepared queries will be lost.  You *must* prepare every time, but see 
below...

>sub db_connect {
>
>require DBI;
>

You don't need that.  You should have already loaded it in startup.pl.

>my $dbname = 'CS';
>my ($dbuser, $dbpasswd) = ('myuser', 'mypass');
>

Probably should be in a config file, rather than buried in here.

>my $dbh = DBI->connect("DBI:mysql:$dbname", $dbuser, $dbpasswd)
>   or die "can't connect: $DBI::errstr\n";
>   
>   # we need these waiting for queries, so we are going to prepare them ahead of
> time, and yes
>   # horror of horror they will be global. Sorry Mom I tried :( 
>   $processed_hnd = $dbh->prepare_cached("select ord_tpak_processed from orders
>where ord_num=?") or confess("can't get tpak processed");
>   $status_hnd = $dbh->prepare_cached("select is_hold_set,holdstate from
>holds where ord_num=?") or confess("can't get hold status");
>   #DBI->trace(2,"/usr/local/apache/htdocs/out.log");
>   return $dbh;
>

Don't put those in globals.  The prepare_cached call already stores them 
for the life of your database connection.  Apache::DBI will keep that 
connection alive (in a global hash) as long as it can and reconnect if 
the connection is lost.  If the connection does get lost, the statement 
handles in these globals will stop working.  You do recreate them every 
time since you call this sub every time, but you could lose the 
connection between the time this sub is called and the time you use 
these handles.

> 2. Every once in a while I get an out of memory error.
>

You can control process growth over time in a number of ways.  They are 
documented in the mod_perl guide.

>3. My main search result page is getting cached, the closure type of
>problem.
>

Are you using any modules that have subs with sub ref prototypes, like 
Error.pm?  That can do it.

>All I have read says that because I am using oop
>modules and use strict along with use vars that should not happen.
>

It's actually really easy to create closures.  Here is a closure:

my $holdtype = $q->param('holdstate');
display_holdtype();

sub display_holdtype {
print "holdtype: $holdtype in process $$\n";
}

This will always print whatever the value was the first time, no matter 
what you change it to later.  (The first time for that process, that 
is.)  Watch out for things like that.  You should always pass params 
explicitly.

>4. I know the way I have done these db connects is sloppy. But I can't seem
>to find a better way. Could I make one db_connect sub,and inherite it all
>though my modules? 
>

Make one sub that returns a database handle and use it from everywhere. 
 Doesn't need to be inherited, you can just stick it in a module that 
all the other modules call.

Hope some of that was helpful,
Perrin

Re: Apache::DBI and CGI::Application with lots of modules.

[Perrin Harkins]

Eric Frazier wrote:
> Here is the kind of thing that is driving me nuts. Please see: 
> http://perl.apache.org/docs/general/perl_reference/perl_reference.html#Remed
> ies_for_Inner_Subroutines
> 
> If what this says is true, then either I don't have a closure type problem,
> or else what is says isn't true.

That documentation refers to one particular problem involving nested 
subs.  You don't need to have nested subs to have closures, and closures 
may not even be the problem.

You need to do some debugging.  Narrow things down by verifying your 
assumptions one by one.  Is CGI.pm really giving you the values you 
expect?  (Some people have experienced issues with params not being 
reset when using CGI.pm in certain ways.)  Is your SQL query being built 
correctly each time?  Is the data that you're passing to the template 
correct?  Throw in some warn statements.  Run it in the debugger if you 
need to.

- Perrin

Re: Apache::DBI and CGI::Application with lots of modules.

[Perrin Harkins]

Eric Frazier wrote:
> I wanted the DBH to be global since just about every sub in Holds does a
> query of some sort.

Three options:
1) Pass it to every sub
2) Make a utility sub that returns a dbh and call it from each sub. 
(Sounds like you already made one of these.)
3) Stuff it in $r->pnotes(), where it will get cleaned up after each 
request.

> What is the problem with the my $holdcheck = new Holds() type of syntax?
> I never read anything about that either way.

It's been discussed quite a bit in various places.  It is now documented 
in the perlobj man page: 
http://perldoc.com/perl5.8.0/pod/perlobj.html#Indirect-Object-Syntax

- Perrin

Re: Apache Hello World Benchmarks Updated

[Perrin Harkins]

Josh Chamas wrote:

>   Set MaxRequestsPerChild to 100 for applications that seem to leak
>   memory which include Embperl 2.0, HTML::Mason, and Template Toolkit.
>   This is a more typical setting in a mod_perl type application that
>   leaks memory, so should be fairly representative benchmark setting.


Maybe I'm more careful about memory growth than some people, but 100 
sounds a bit on the low side to me.  I use Apache::SizeLimit instead of 
MaxRequestsPerlChild, but I'm pretty sure every child serves more 
requests than that.  Did it seem to affect the performance numbers much?

Incidentally, I hate to call this stuff "memory leaks", since it's 
usually just normal growth as memory gets used.  Calling it "leaks" 
implies that memory is being wasted as a result of mistakes with 
pointers or some such.  When people hear "memory leaks" they go running 
for stuff like Apache::Leak, thinking they can find some problem and fix 
it, which is usually not the case.

- Perrin

Re: Can I parse content that has been returned to user by simplecgi script?

[Perrin Harkins]

Ruslan U. Zakirov wrote:

>I want to upgrade my project with implementing some feature.
>Project was writen for mod_cgi, but I would like to parse content that
>was generated by my scripts to implement something like SSI or
>Apache::UCase and etc.
>

You can't do that.  However, you can run your CGI scripts under 
Apache::PerlRun and then use Apache::Filter on them to do things like 
SSI.  This will also make them much faster.

- Perrin

Re: Apache::DBI for persistent connections

[Perrin Harkins]

Paul Simon wrote:
> I was under the impression that Apache::DBI isn't compatible for this 
> set up:
> 
> win2000 + apache2.0.42 + perl5.8 + mod_perl1.99
> 
> For a multi threaded environment, isn't Apache::DBIPool necessary?  I'd 
> rather use Apache::DBI.

Apache::DBIPool doesn't actually exist yet.  Apache::DBI will work 
correctly in a threaded environment if your DBD is thread-safe. 
Remember, no variables are shared between Perl threads unless you 
explicitly share them.

- Perrin

Re: current state of conf/code, feedback?

[Perrin Harkins]

Paul Simon wrote:
> I currently have CGI pages caching on the client side, which 
> is helping some, but I'm also going to experiment with CGI::Cache. 

There are some mod_perl specific version of this too, like the one from 
the mod_perl Developer's Cookbook (Apache::CacheContent), but these are 
mod_perl 1.x and probably require work to get them running on mod_perl 
2.  Traditionally this has mostly been done with a caching reverse 
proxy, and that's another angle you could try.

> What kind of trouble is 
> there for having use DBI and use HTML::Template in both startup.pl and 
> Snoopy.pm (besides being redundant)?

None.  That is the right thing to do, because it preloads the modules 
(by having them in startup.pl) and documents the dependencies (by 
putting in Snoopy.pm).

> Since Snoopy.pm is the meat and 
> pototoes, should it also be in startup.pl?

Yes.

> Does declaring Class 
> variables, such as the DBI handle, offer any benefits (or pain) of a 
> shared connection? 

You're getting yourself into major trouble there.  You should use 
Apache::DBI instead.  The way you have it now (with the connection code 
in the "main" section of Snoopy.pm), that database connection will be 
created when you use the module in startup.pl.  Apache will then fork 
and multiple children will try to share the same $DBH.  Disaster.

Also, your solution will not try to reconnect if it loses its 
connection.  Apache::DBI will.  And you are creating a closure here with 
your $DBH which might come back to bite you later if your module gets 
automatically reloaded when you change it.

- Perrin

Re: Is there an easy way to trace / debug Apache::DBI (Postgres)leaks?

[Perrin Harkins]

Kirk Bowe wrote:
> Unfortunately after a couple of hours of moderate use Postgres reaches its
> max_clients level (which is set below max httpds in httpd.conf)

This is usually caused by mistakes in your connection calls where they 
have slightly different connect info resulting in multiple connections.

> At this point there's no
> work being done, so presumably the handles aren't being freed.

Database handles are not supposed to be freed.  That's the point of 
Apache::DBI.  However, you should only have one per httpd process.

> Does anyone have any tips on how to, at this point, go about detecting
> where the handle leak might be (which I'm presuming is the problem)?

http://perl.apache.org/docs/1.0/guide/databases.html#Debugging_Apache__DBI

> should Template::Toolkit be avoided altogether when using mod_perl and
> database connectivity?

TT is not related to this.  It works great with mod_perl.

- Perrin

Re: current state of conf/code, feedback?

[Perrin Harkins]

Paul Simon wrote:
> I was cruising with Apache::DBI, definitely better than the way I had 
> it, but now suddenly I'm getting this error:
> 
> DBD::ODBC::dr FETCH failed: handle 1 is owned by thread 1e90bdc not 
> current thread b0f18c (handles can't be shared between threads and your 
> driver may need a CLONE method added) at C:/Perl/site/lib/Apache/DBI.pm 
> line 64.
> 
> What's going on?

You'd be better off asking on the DBI list, but it looks to me like your 
DBD is not thread-safe, or else you opened a connection before the fork 
(in startup.pl).

- Perrin

Re: [newbie] How do I send a custom HTTP::Response?

[Perrin Harkins]

Chris Pizzo wrote:

The documentation tells me how to create a new response object but how do I
reply to a request using my custom response?


HTTP::Response?  That's an LWP thing, not a mod_perl thing.  Maybe 
you're a little confused here?  Tell us what you're trying to do.

- Perrin

Re: [newbie] How do I send a custom HTTP::Response?

[Perrin Harkins]

Chris Pizzo wrote:

OK,  I am getting a request from a server and I need to respond with an XML
doc.


So your mod_perl handler is getting an HTTP request and you want to send 
back an XML document?  No problem, just send it.  Set the content-type 
if you need to.  There's no trick to it.  See the mailing list archives 
for more on this.

- Perrin

Re: code evaluation in regexp failing intermittantly

[Perrin Harkins]

Rodney Hampton wrote:

Let me preface it 
by stating that I'm building a very simple templating application.
[...]

Not satisfied, I wanted to make it possible to do something like:
<% code_ref Util::Test_Util::test_expand %>
and have it swap in the text output from the sub into the template. That 
way, unlike other templating applications, I could get away from writing 
my own mini language and use the full power of perl in a sub called by a 
simple tag in the template.

Do you realize that almost every templating system on CPAN already does 
this?  If you are doing this because you think you've hit on a new 
feature, stop now and download one of those.  If you're doing it because 
hacking perl is fun and you're in no rush, then go ahead and have a good 
time.

- Perrin

Re: Do all apache childs create persistant mysql connections?

[Perrin Harkins]

John Cameron wrote:


Thankyou! We are using connect_on_init, so this may explain our problem.
What happens if I turn off connect_on_init? Do I need to change our code in
any way? Or will the connection be made automatically?



The connection will be made when you first do a DBI connect in that 
process.  When you use connect_on_init, it is made when the child 
process spawns.

You would still be advised to move the static files to another server.

- Perrin

Re: Do all apache childs create persistant mysql connections?

[Perrin Harkins]

John Cameron wrote:


2) Does Apache create a new mysql connection/process for EVERY child 
apache process that is spawned?


It creates one in each process that uses the database.


I assume some apache processes are spawned to handle simple 
non-database actions such as retrieving a graphic or static html file. 
Because we're using Apache::DBI, does this mean that even these little 
processes are creating a big mysql process?


No, if they never use the database they will never create a connection 
(unless you use connect_on_init).  Of course you have no way of keeping 
the static file requests to a single process, so it's likely that each 
process is serving some static files and some dynamic requests that 
require database handles.

 This is bringing us to our knees. Any help or comments, no matter how 
obscure, would be greatly appriciated!


Use a reverse proxy as described in the mod_perl documentation to keep 
static requests from tying up database connections.  Use the MaxClients 
setting to liit the total number of apache children, and thus the total 
number of MySQL connections.  Make sure you are always using the exact 
same connection info, or you could be opening multiple database 
connections per child.

- Perrin

Re: Apache::AuthcookieDBI issue - NS broswers display login form incorrectly?

[Perrin Harkins]

George Valpak wrote:

The browser is getting what appears to be the right html for the login form, but it thinks the content-type is text/plain for some reason.


This sounds like a known bug in IE: if it doesn't get a content-type 
header it will guess based on the file extension.  Netscape does not 
have this bug.

I tried adding a meta tag in the html template to no avail:




You can't do that, it needs to be a real header.


I also tries setting the header directly in the response in my login.pl file immediately before sending the header:

$r->header_out(content_type=> "text/html");
$r->send_http_header;


Try $r->content_type("text/html") instead of header_out.

- Perrin

Re: [OT] Perl vs. PHP..... but where is mod_perl?

[Perrin Harkins]

[EMAIL PROTECTED] wrote:

This is a bug introduced by having to insert workarounds for segfaults 
caused by Apache::Cooke/mod_perl. I've been asking for help with this 
issue for off and on for months now.

I suggest you stop using Apache::Cookie and see if the segfaults go 
away.  There are pure Perl modules that handle cookies well, and it's 
not an expensive operation.  Apache::Cookie is probably overkill in most 
situations.

- Perrin

Re: repost: [mp1.0] recurring segfaults on mod_perl-1.27/apache-1.3.26

[Perrin Harkins]

Ed wrote:

Could be bad hardware.  Search google for Signal 11.


That's actually pretty rare.  Segfaults are usually just a result of 
memory-handling bugs in C programs.

- Perrin

Re: [OT] Perl vs. PHP..... but where is mod_perl?

[Perrin Harkins]

[EMAIL PROTECTED] wrote:


Btw when I mean escalate, i mean that the odds of any browser getting 
a segfaulting page were increased, not that they are random - a 
particular request - URI,User-Agent,Accept,Cookie, etc combo - 
consistently segfaults, at least for a few days.


Then it's probably fixable, but the people who could fix it are mostly 
off working on mod_perl 2.

While trying to debug this we replaced Apache::Cookie (i'm not certain 
if every instance of which, but I think we did) with regexes against 
$r->header_in("Cookie"), to no avail.

At this point we are using Apache::Cookie and not overriding 
Apache::Subrequest::run(), and this is working without the segfaults.
But, we just recently tried to add an additional call to 
Apache::Cookie for our ad system and they all came right back.


Then, again, I would stop using Apache::Cookie.  You don't need it, and 
using it seems to cause problems..

- Perrin

Re: [OT] Perl vs. PHP..... but where is mod_perl?

[Perrin Harkins]

Chris Winters wrote:


On Fri, 2002-10-18 at 17:46, Tobyn Baugher wrote:
 

As someone fairly new to mod_perl could you make a suggestion of a good
alternative to Apache::Cookie? I was using it just because, like
Apache::Request, it was *there*.
   


The pure-perl CGI::Cookie works fine.



That's a good one, and so is CGI::Simple::Cookie.

- Perrin

Re: Handler Access to vars created by other modules. (modperl 2.0)

[Perrin Harkins]

Erich Oliphant wrote:

I am having difficulty accessing this variable (via the ENV 
hash).  I decided to make it a PerlLogHandler and register it 
REALLY_LAST, thinking that mod_unique_id would've exported UNIQUE_ID 
prior to that.  However, that does not seem to be the case :)

Something is wrong with your mod_unique_id setup.  The mod_unique_id 
variable should in %ENV from the beginning.

- Perrin

Re: AuthCookie questions

[Perrin Harkins]

Christian Gilmore wrote:

Hi, Michael. Let me try again with more specifics. I'm required to mash my
service into another organization's authentication scheme, ditching my own
secure methods for their cross-domain unencrypted, unsigned cookie.

[...]

On a side note, if anyone finds the proposed design lacking for security or
anything else, please let me know.


It sounds like you are already aware that it lacks security.  The 
important thing to remember about cookies is that unless you use some 
kind of cryptographic signature to verify them you have absolutely no 
idea if the cookie came from your site or not.  People can very easilly 
put whatever they want in a cookie to send to your site usingone of the 
thousands of HTTP testing programs and libraries, and if you use that 
cookie as a key to a data structure they may be able to gain access to 
other people's data.

Even if you use a crypto signature they can still sniff someone else's 
legit cookie off the wire, but at least you can prevent them from 
tampering with the contents of the cookie.

"Never trust the client."

- Perrin

Re: Making a module-It can't be this hard.

[Perrin Harkins]

Robert Covell wrote:

I simply want to make a module so I can reuse a common header instead of
manually changing each page.  Under mod-perl how do you simply create a
module that I can use/require/include that I can call a subroutine/function
to generate some html based on the page you are on.

It works 5 out of 10 times.  When it fails I get:

[Fri Oct 25 14:24:05 2002] [error] Undefined subroutine
&Apache::ROOTvirusmailcenter_2erolet_2ecom::index_2epl::DisplaySection
called at /mnt/data1/www/htdocsM/mailcenter/index.pl line 410.


It sounds like you are using perl4-style libs that don't declare a 
package name.  You can find a description of the problem and some 
solutions here:
http://perl.apache.org/docs/1.0/guide/porting.html#Name_collisions_with_Modules_and_libs

The basic fix is to give your modules unique package names.  See the 
perlmod man page for more on package names.

- Perrin

Re: code evaluation in regexp failing intermittantly

[Perrin Harkins]

Rodney Hampton wrote:


I really only need 3 tags: one to link the templates together, one 
to bring in images, and one to call code that can by dynamically 
inserted.



Like an eval, or a subroutine call?  Either way, this is all covered by 
most of the other systems.  Even Apache::SSI can do this.

  Every item in my application's database (templates, images, and 
other files) has user and group permissions that need to be checked 
on access. Thus, any templating system that did caching is pretty 
much out of the running because as soon as I revoke a user's 
privileges or change permissions on the item, I want access to 
change immediately.



The only one that has built-in caching of results is Mason, and it won't 
do it unless you tell it to.  All other references to caching mean 
caching the template parsing.  That wouldn't be an issue for this.

I'm guessing from your previous post that these things are accessed by 
calling a subroutine from the templates, and the subroutine does the 
access checking.  That should be no problem.

  Also, when an advertiser makes a change to one of his/her 
templates, it needs to propogate immediately.



All of them handle that.  They check for changes, usually by stat-ing 
the template files.

Additionally, I need to track users so I had to be able to establish 
and maintain a session so this was a definate requirement.



Session tracking has nothing to do with templates.  Systems like Embperl 
and Apache::ASP (and now Mason) integrate it, but that's because they 
are much more than just template systems.  With a typical pipeline 
arhcitecture, you would handle the session in your code and then pass 
the data (including whatever session data you need) off to the template.

  Also, I needed to be able to selectively protect certain areas of 
the application with login screens (not necessarily the same one 
depending on where you navigate).



Access control also has nothing to do with templates.  You can use 
something like Apache::AuthCookie for this, or you can handle it 
directly in your perl code.

I've done some benchmarking on what I've built already and am 
satisfied with the results.



It's easy to write a fast templating system.  What's hard is writing a 
fast templating system that has no bugs, and can adapt to future needs. 
It doesn't take long before you find you need more features: whitespace 
control, a more powerful if/else construct, a way to format dates, etc. 
That's why the modules on CPAN seem big.  They have been extended to 
handle most of the real-world problems that people need to solve with 
templating systems.

Anyway, I'm not trying to say that what I'm doing is better or worse 
than what has come before.  I just happen to have different 
requirements and I didn't think that the other systems out there 
were going to be both lightweight and allow me to meet my need for 
access control.



I'm pretty sure that there are modules on CPAN that do just what you 
need.  If you already have a working system, then go with it.  If you 
find yourself having to revisit the code because you need to add things 
or are finding bugs or performance problems, consider benefitting from 
someone else's hard work.

- Perrin

Re: Apache::Clean, Apache::Compress, mod_gzip/deflate, cross sitescripting and more.

[Perrin Harkins]

Richard Clarke wrote:

Before I embark on a day exploring the pros and cons of today's
cleaning/compression tools, I wondered if any of you could give me some
feedback about your own experiences within the context of medium/large scale
web sites/applications (E-Toys etc).


We didn't use compression at eToys.  There were two main reasons for 
this.  First, it was still a fairly experimental idea at the time, and I 
had seen browser compatibility problems when hitting some sites that 
used it.  (I believe these have been fixed since then.)  Second, more 
than half of our content was served from the mod_proxy cache, and I'm 
not aware of a compression tool that works with that.

Is there any advantage to using modules like HTML::Tree or HTML::Parser to
remove information (i.e. removing autoexecuting javascript tags and any
other CSS issues) from user submitted information.


No, the best approach is to not attempt to filter user input at all but 
escape everything when you display it.  Then if a user enters HTML 
characters they will just show up literally in the output.  The only 
time you can't do it this way is if there are certain HTML characters 
you want to accept.

- Perrin

Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Mithun Bhattacharya wrote:


http://public.yahoo.com/~radwin/talks/yahoo-phpcon2002.htm

If nothing else this should be atleast generate some thoughts ??



It does: hooray!  Yahoo is moving from a proprietary server-side 
scripting tool to an open source one.  Great news for all of us, since 
they will help legitimize open source ("the largest site in the world 
runs on it") and maybe feed good ideas back to the open source world. 
(Of course Yahoo has used other open source tools forever.)

If you look at the performance graphs they made, mod_perl comes in neck 
and neck with PHP.  We always see mod_perl go faster than PHP in 
Joshua's tests, but Yahoo was testing with this Ioncube cache that keeps 
PHP code in memory (like mod_perl).  The performance tests were so close 
that a little optimization on the mod_perl side might actually turn it 
around.

They didn't make their decision on performance though.  They seem to 
have been most influenced by the idea that perl allows too much 
flexibility in coding style, although I can't see how PHP is going to 
help with that.

They also say they plan to continue using lots of perl in all the places 
they use it now: off-line processing, filling in the includes and dbm 
files that the server-side script assembles.  Perl is not being removed.

- Perrin

Re: [OT] Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Tom Servo wrote:


Check out their online map site, they do use Python for that.



I'm actually surprised they didn't go with Python, because the people I 
know there love it.  If their backend data processing ever gets moved 
from Perl to something else, it would probably be moved to Python.

- Perrin

Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Mithun Bhattacharya wrote:


No it is not being removed but this could have been a very big thing
for mod_perl. Can someone find out more details as to why PHP was
preferred over mod_perl it cant be just on a whim.



Think about what they are using it for.  Yahoo is the most extreme 
example of a performance-driven situation.  They have so much volume 
that they are willing to write things in C++ just to get the speed that 
is required.  They pre-process and cache things as much as they possibly 
can.  The server-side scripting language they use is basically a 
glorified include processor.  They pull in pre-built include files and 
read things out of dbms with it.  That's all.  The real application 
stuff is built in other languages.  (At least this is the impression I 
get from the paper and from talking to people there.)

Given that, PHP is a reasonable fit.  It's fast, has a less flexible 
syntax, and uses less memory than mod_perl.  They also have more of a 
need than most people to integrate with C/C++, and I've been told that 
it's easier to hack those into PHP.

Think about the implications of the memory graph.  If they can run 5 
more apache children at once on PHP because of its lower memory 
consumption, 5 * 4500 = 22500 more users they can handle at any given 
moment!

Also how does one go about justifying the fact that Ioncube cache is
doing a better cacheing than any mod_perl based solution ??



Sorry, I read that wrong.  It was actually mod_perl that had the best 
performance.  The cache they use works just the same as Perl's normal 
operation, i.e. once some code is compiled it stays in memory and 
doesn't need to be compiled again.  You don't have to do anything to get 
that in Perl.

- Perrin

Re: Same $dbh under different pids?

[Perrin Harkins]

harm wrote:


On Wed, Oct 30, 2002 at 06:05:51PM +0800, Philippe M. Chiasson wrote:
 

For the same reason that running this:
$> perl -e'fork; { $foo = {}; print "$$:$foo\n"}'
1984:HASH(0x804c00c)
1987:HASH(0x804c00c)

produces this for me, every single time I run this program

You are assuming that if (0x804c00c) is equal in different processes,
they must be pointers(or references, or handles) to the same thing. And
it is not the case ;-)



Wait, ins't it the case?  That number is supposed to be the location in 
memory.  It seems like these are all pointing to the same hash.  I can't 
explain how that would happen though, based on the code shown here.

- Perrin

Re: Same $dbh under different pids?

[Perrin Harkins]

Mrs. Brisby wrote:


$ perl -e '$foo = {}; fork; print "$$:$foo\n";'
18161:HASH(0x80fd254)
18162:HASH(0x80fd254)
$ perl -e 'fork; $foo = {}; print "$$:$foo\n";'
18163:HASH(0x80fd254)
18164:HASH(0x80fd254)



I expected the first.  I didn't expect the second.  Thanks for the 
explanation.

- Perrin

Re: DBD::Oracle/Windows2000 OK from prompt, not mod_perl?

[Perrin Harkins]

Larry Leszczynski wrote:


I'm having a problem on Windows 2000 where DBD::Oracle works fine from
perl on the command prompt but not from inside mod_perl.  I think it is a
problem loading DLLs but I can't figure out what's different running under
mod_perl.  The machine is running:
  ActiveState perl 5.6.1 build 633
  Apache 1.3.26 (prebuilt)
  mod_perl 1.27_01-dev (Randy Kobe's build)
  DBD-Oracle8 (ActiveState ppm)



You can't just mix and match like that.  Modules that you want to use 
under mod_perl have to be built against the same perl that mod_perl was 
built with.  If there is a DBD::Oracle with Randy's distro, use it. 
Otherwise, you need to build DBD::Oracle with the same perl he used, or 
build mod_perl with ActiveState 633.

- Perrin

Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Cristóvão Dalla Costa wrote:


Perrin Harkins wrote:


They also have more of a
need than most people to integrate with C/C++, and I've been told that
it's easier to hack those into PHP.



What a joke.



Have  you written C extensions for both Perl and PHP and think Perl is 
easier?  Most people complain about XS being challenging.  The picture 
might be different when SWIG and Inline::C are taken into account, but 
I've never used them so I couldn't say.  In general, it makes sense that 
a simple language would be simple to extend with C.  That's why people 
like TCL.

- Perrin

Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Iain 'Spoon' Truskett wrote:


In general, it makes sense that a simple language would be simple to
extend with C. That's why people like TCL.
   


They do? =)



Sure.  That's why Vignette used TCL: adding your own C commands to the 
language is easy.  Probably the same story for AOLServer.

- Perrin

Re: hangs on $ENV{'QUERY_STRING'}

[Perrin Harkins]

Michael Forbes wrote:


Well, I've managed to make some significant progress:  I got past that
hang (apparently modperl was having problems with the URL b/c some of
the values were being passed empty... i.e.,
"stage=4&eqtype=All&cont=&restrictflags=12"... notice that cont has no
value.



That sounds pretty bad.  Maybe you should go back to mod_perl 1.3, which 
definitely does not have problems with that.

My next problem, which took four hours to track down, is that it turns
out that mod_perl is choking on null values being retrieved from MySQL. 
I'm using DBI for the interface, and the actual fetch_row is performing
exactly the way it should... but when it tries to do anything with my
scalar once it has a null in it, it just chokes & sits there until it
times out.


Are you using the pre-fork MPM?  If not, do you know if your DBD driver 
is threadsafe?

- Perrin

Re: [O] Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Franck PORCHER wrote:


But  is a lot easier to read and debug, IMHO   Is there a
significant performance difference in using map instead?
   


My experience is that in most cases, the  construct is used
to apply the same treatment to all the elements of an array,
whence the  construct. In those cases, the readibility
is definitely altered by the index that get
in the way.



I think you're forgetting that "for" is just a synonym for "foreach". 
You don't need an index.

Of course, this is a quick analysis not taking into account other
factors like readibility versus efficiency.

In fact, regarding the efficiency of the  construct,
I often wondered whether Perl detects  being ran in a void context, so as
to give it an iterative interpretation, avoiding to build the output
list.



It's really not a matter of efficiency, but rather of correctness. 
Using map in a void context is a mistake.  There is a time and place 
for map: when you want to do something to each element in an array and 
return the array on the other side.  Otherwise, use for, like this:

some_function($_) for @array;.

Even when map is not incorrect, I usually avoid it because it has a 
tendency to take your code from nice, readable perl to evil h@x0r PERL 
in 3 characters.  This is the kind of thing that Yahoo was probably 
worried about when the decided not to use Perl.

- Perrin

Re: [O] Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Tony Bowden wrote:


It sounds like you're saying that you should only use a subset of Perl
as some programmers may not understand the other parts of it?



That is what I'm saying.  I'm aware that this is a controversial opinion 
in the Perl world.  However, I think it's reasonable to know your 
audience and write for them.  That's the good part of More Than One Way. 
The New York Times writes their articles for a certain reading level, 
and I do the same with my code.

Note that different audiences have different levels.  You can expect 
someone who opens up the code in a module on CPAN to be pretty fearless 
about advanced Perl ideas.  On the other hand,  the people in my current 
office are mostly novices so I try to take it easy on them when writing 
code for work.  I will still use a Schwartzian Transform if I need one, 
but I'll put in a comment saying "this is a Schwartzian Transform, see 
such and such URL for an explanation."  Same deal with things like 
hash-slices, and I try to avoid using $_ when it isn't necessary.

Given:
 sub square { $_[0] ** 2 }
 my @nums = (1 .. 10);

Which would say is more readable and maintainable?

1)
 my @squares;
 push @squares, square($_) for @nums;

2)
 my @squares = @nums;
 $_ = square($_) for @squares;

3)
 my @squares = map square($_), @nums;

I'd go for (3) every time.



So would I, in that situation.  It's not that map is so evil, but rather 
that I have often seen people overuse it (especially after a 
first-reading of "Effective Perl"), and write confusing code with it by 
jamming too much into the "map { some stuff } @list" form.

- Perrin

Re: MOD_Perl Converter

[Perrin Harkins]

Ged Haywood wrote:


On Fri, 1 Nov 2002, Frank Naude (FJ) wrote:
 

but, is there any mod_cgi to mod_perl converter available?
   


Have a look at Apache::Registry.



And to address the specific problem you mentioned about initializaing 
variables, look at Apache::PerlRun.

- Perrin

Re: [O] Re: Yahoo is moving to PHP ??

[Perrin Harkins]

Tony Bowden wrote:


... but I think that there should be a certain level of ability that
should be assumed when coding commercially ...



My current situation is somewhat unusual because Perl is not the 
language that the people I am coding with were hired to write.  They are 
mostly Java programmers, and I'm just doing some useful admin and 
database scripts in Perl because it's so much quicker than doing them in 
Java.  Someone else will eventually have to maintain them, so I write in 
a way that a novice with a copy of Learning Perl has a hope of 
understanding.

When I work in an environment that is more Perl-centric, I expand the 
vocabulary of my code more.

things like map,
hash slices, and even $_ aren't what I could class as advanced Perl ideas.



Perl is a large language, and not all of it needs to be used all the 
time.  I wouldn't call $_ advanced, but I know I hate reading code where 
people use it more than they have to.  Hash slices and map are uncommon 
enough that I had been writing fairly challenging OO Perl for quite a 
while before I ever saw one.  I don't use them when something more basic 
will do just as well.

Correct Perl style is probably not something that any two people will 
ever agree on.  This used to bother me more before I realized that all 
languages have this problem, even though they try to deny it.  I've seen 
some hideous Java code at this place that really takes the wind out of 
the "Java is more maintainable" argument.

- Perrin

Re: When using Apache::DBI...

[Perrin Harkins]

Clinton Gormley wrote:


Am I correct in this:

Apache::DBI can only really do its stuff when you perform a
DBI->connect, so by calling $dbh = DBI->connect(..) during PerlChildInit
and then never trying to reconnect, you are defeating the purpose of
using Apache::DBI. 


That's right.


To expand on this, when Apache::DBI intercepts a connection request:
* it returns a stored live handle which matches the connection
credentials
* checks that the handle is indeed still live, and if it isn't,
reconnects automatically (potentially destroying any $sth's that you
have prepared and retained)



It also adds a cleanup handler to issue a rollback of any uncommitted 
work on that database handle after the request completes.

So would this be the right balance between efficiency and safety:
1) use Apache::DBI
2) Do "my $dbh = DBI->connect()" at the beginning of each request (ie as
early as required during each request) 
3) use "my $sth = $dbh->prepare_cached()" once as early as required
during each request


Yes.  The call to prepare_cached won't do anything if that statement is 
already cached.

The way I like to do it is to have a utility class that implements a 
get_dbh() method.  Then I just call that when I want one.  Inside of 
that it can do the DBI->connect business and anything else I need done 
(maybe resetting the isolation level on that connection if I messed with 
that anywhere).  It can also cache the dbh for the length of the request 
inside of $r->pnotes if I want to avoid multiple calls to Apache::DBI.

- Perrin

Re: use http-equiv to refresh the page

[Perrin Harkins]

Wei Gao wrote:


In my perl program executing in Apache web server, I have the 
following code:
 
use CGI ;
 
$query = new CGI ;
$url = http://www.mycite.com ;  #The url to refresh.
 
 print $query->header(-status=>'200 Ok', -type=>'text/html');
 print "";


Uh, that's not a redirect; that's an ugly proprietary hack.  You should 
be using standard HTTP redirects.  See 
http://search.cpan.org/author/JHI/perl-5.8.0/lib/CGI.pm#GENERATING_A_REDIRECTION_HEADER 
for more.

- Perrin

Re: use http-equiv to refresh the page

[Perrin Harkins]

Wei Gao wrote:


I have tried "print 
$query->redirect('http://somewhere.else/in/movie/land') ;" before, 
which works fine as to redirect the user to the web page. However, if 
the user then tries to refresh this page, the CGI script is called 
again without any params, which result in "Internal Server Error".


You lost me.  If you redirect the user to http://mycite.com/, and then 
the user reloads, he should be reloading http://mycite.com/.  I don't 
see any reason why that wouldn't work.  Are you saying that reload in IE 
goes back to the URL that issued the redirect and reloads that?  Even if 
it does, it should still be submitting the query string or POST data, 
although the user may get a pop-up asking if he wants to submit POST 
data again.

Is using  tag a "bad" approach?



Yes.  It's something that Netscape added to their browser, which others 
may or may not add to their browsers.  It's not part of any HTTP spec 
and isn't guaranteed to work, even on totally correct web browsers.

- Perrin

Re: use http-equiv to refresh the page

[Perrin Harkins]

Chris Shiflett wrote:


A meta tag is not something unique to Netscape



I said it was added by Netscape, and I'm pretty sure it was, back in 1.1 
or 2.0.

As with any other HTML tag, the meta tag does not need to be part of an HTTP specification in order to be valid. Also, it is guaranteed to work on any compliant Web browser. HTML has its own specification, and the latest version describes the meta tag here:



http://www.w3.org/TR/html4/struct/global.html#h-7.4.4.2



Look a little further down that page:

"/*Note.* Some user agents support the use of META 
 to refresh the 
current page after a specified number of seconds, with the option of 
replacing it by a different URI. Authors should *not* use this technique 
to forward users to different pages, as this makes the page inaccessible 
to some users. Instead, automatic page forwarding should be done using 
server-side redirects."/

I might be overzealous about this, but I dislike seeing HTTP-EQUIV meta 
tags used when actual HTTP headers are available to do the same thing. 
It's fine if there's a reason for it, but usually people do it because 
they don't realize they can just send a real header instead..

- Perrin

Re: use http-equiv to refresh the page

[Perrin Harkins]

Wei Gao wrote:


Thanks for the reminder.  I think the reason that "print 
$query->redirect(-uri=>'http://www.mysite.com', -nph=>1);" is not 
working, is because my program doesn't seem to know how to handle 
"nph". I am using Apach1.3.26 and Perl 5.6.1. I have
"use CGI qw(:standard -nph) ;" at the beginning of the script. 
However, when I tried to use nph, the server complains about "Bad 
Header".
 
Is there any known issues that the versions I use don't support nph? 
Am I missing something?


I don't think NPH is related to the problem you're having, but Apache 
determines if a script is NPH by looking at the prefix of the file.  Try 
naming your script "nph-something.cgi" and it should support NPH.  This 
is not very well documented, unfortunately.

You don't need to use an NPH script to make redirection work.  It's also 
still not clear to me what isn't working for you.  When I get redirected 
in IE and then reload the page, it reloads the page I was redirected to.

Since this is all getting pretty far off topic for the mod_perl list, 
you might want to try asking on a CGI-specific list or on 
http://perlmonks.org/.

- Perrin

Re: use http-equiv to refresh the page

[Perrin Harkins]

Chris Shiflett wrote:


I just wanted to mention that the meta tag as well as its http-equiv 
attribute are both official parts of the HTML standard and have been 
for quite some time. Netscape also introduced things like cookies and 
SSL, but that should in no way discredit the technology.


I'm just bitter about Netscape because I worked at a company that made 
me use frames and JavaScript when the 2.0 version came out.

It is also the only option for the "pause, then redirect" behavior the 
original poster desired that I can think of..


It is the only way I know to do that, but I didn't think that's what he 
was trying to do.  He had a wait time of 0 in his example.

- Perrin

Re: use http-equiv to refresh the page

[Perrin Harkins]

[EMAIL PROTECTED] wrote:


Also, NPH is only implemented in the NS browsers, and was a way for a webserver
to send multiple documents "inline" down to a browser, and was an ancient way
to write status pages and such that "automagically" refreshed themselves.



No, that's "server push" you're thinking of.  NPH (non-parsed header) 
scripts are CGI scripts that talk directly to the client without the 
server parsing headers and adding others (like the one that says it's 
Apache). Normally, mod_cgi adds the response line and certain other 
headers, so it parses your output.  This is the same as using mod_perl 
with the PerlSendHeader option on.  NPH script behave like mod_perl with 
PerlSendHeader off.

- Perrin

Re: Help mod_perl 1.27 and DB

[Perrin Harkins]

Tony Simbine wrote:


wenn i reload it, then sometimes i get the document or an error.



Well, what's the error?  Look at your log file.  Then look at 
http://perl.apache.org/docs/1.0/guide/troubleshooting.html and see if 
the error is listed there.

- Perrin

Re: Can't locate object method "new" via package "Apache::Request"(via Mason)...SOLVED

[Perrin Harkins]

DeAngelo Lampkin wrote:


And of course the other reason is that if the solution to the problem 
was so obvious from the error message, somebody would have posted a 
solution before I figured it out (with help from you guys).


There is documentation related to this problem in the troubleshooting 
section, and if you'd like to generalize it a little so it will be 
easier for others to find when looking at this type of message, your 
help would be welcome.  The current documentation is here:
http://perl.apache.org/docs/1.0/guide/troubleshooting.html#install_driver_Oracle__failed__Can_t_load__DBD_Oracle_Oracle_so__for_module_DBD__Oracle

- Perrin

Re: Using Perl END{} with Apache::Registry

[Perrin Harkins]

Justin Luster wrote:


I have an included file that I’m requiring:

require “test.pl”;

Without the END { } block if the script cannot find test.pl I get a
Server error 500 and an appropriate error message in the log file.  When
I include the END{ } block I get no Server Error and no message in the
log file.  It is almost as if the END{ } is overwriting the
ModPerlRegistry error system.



Does it make any difference if you change what's in the END block?

- Perrin

Re: Using Perl END{} with Apache::Registry

[Perrin Harkins]

Justin Luster wrote:


After doing some additional testing it appears that this problem only
occurs on my Windows machine with Apache 2.0.  I tried it on my Linux
box Apache 1.3 and things worked fine.



That's a key distinction.  Please make sure you say mod_perl 2 when 
that's what you are using.  There is actually no module called 
Apache::Registry in mod_perl 2, so I assumed you meant the 1.x series.

- Perrin

Re: problem with session ids

[Perrin Harkins]

Minas wrote:


Recently I installed the Apache::Session module on my server in order
to give a kind of identity to my e-shop visitors, seems to work but
generates different session ids when I reload the bellow test cgi.
What can I do in order to have my visitor the same session id, up to
close his web browser.



You're going to have to do some debugging to find out why your script 
isn't working.  You use of Apache::Session::File looks okay, except that 
you don't handle possible errors.  Check if the cookie is being set, if 
the files are being created in /tmp, etc.

- Perrin

Re: RFC: Template::YetAnother

[Perrin Harkins]

Thomas Klausner wrote:

Hi!

On Sat, Nov 16, 2002 at 10:43:44AM +0100, Marcin Kasperski wrote:



One note: while talking about templating systems for generating HTML,
it can be inspiring to take a look at the Zope Page Templates (yes, I
know, this is the python/zope world, not perl...). They found the nice
idea of injecting loops/ifs etc into HTML in the way which does not
spoil HTML markup. 


There is a simmilar Perl implementation of this called HTML_Tree
http://homepage.mac.com/pauljlucas/software/html_tree/

A module based on HTML_Tree is on CPAN under the name of
HTML-Seamstress:
http://search.cpan.org/author/TBONE/HTML-Seamstress-1.17/Seamstress.pm


There is also an implementation that copies the Zope style pretty 
closely called Petal:
http://search.cpan.org/author/JHIVER/Petal-0.77/lib/Petal.pm

- Perrin

Re: Executing Apache::Registry cached script from normal mod_perl

[Perrin Harkins]

Matthew Hodgson wrote:

On Fri, 15 Nov 2002, Josh Chamas wrote:



Matthew Hodgson wrote:


Hi,

I have a script which executes under Apache::Registry and resides in
/perl/blah.pl.  However, it only generates a fragment of a page, and is
normally embedded into an html template using SSI with something like:



I am very aware that this would be better implemented as a raw perl module
used from apache's startup.pl (and then embedded using something like
) - but practically this isn't an option right
now.



How about just executing this file like this:

  do "$path_to/perl/blah.pl";




I guess that just as long as the script is never actually executed under
Apache::Registry, and if the only place it's ever do'd from is the plain
library, then this is the best solution. :)  Presumably if I needed to
execute blah.pl from many different libraries as well as actually embed it
using , this wouldn't be so great as
each instance of it would be compiled separately as part of that
particular block of code, thus bloating the apache process horribly.


The other problem is that "do" will compile the script every time, even 
if it has been run before in that same process.  Slow.

You might want to look at how Apache::SSI does it.  You can copy the 
include handling code from there.  Otherwise, I would suggest copying 
some code from Apache::RegistryNG to compile it into a subroutine.

- Perrin

Re: RFC: Template::YetAnother

[Perrin Harkins]

Thomas Klausner wrote:

Yes, this is a RFC for Yet Another Templating System. I know that
there are a lot of those around. But this own might be
different/interesting (at least I think/hope so...)

I also posted this on perlmonks:
http://www.perlmonks.org/index.pl?node_id=213300


Ovid on Perlmonks already said some of the things I would have said 
about needing a presentation language.  For a templating system to be 
useful, it has to be able to deal with loops and conditionals.  Your 
system pushes those into the Perl code, splitting the presentation 
control into multiple places.  This fails one of my major goals: the 
template coders should be able to change the presentation without help 
from a Perl coder.  What if they decide they don't want to loop through 
the links, but rather just print a message saying there are some?  What 
if they only want to show the first two?  What if they want to display a 
list with commas between the elements where they didn't use a separator 
before?  A mature templating system can handle all of these issues 
without changing the main Perl module code.

Because IMO, the main reason for using templates is to seperate code
from markup.


It sounds to me like you want one of the HTML attribute ones, like Petal 
or HTML::Seamstress.  What was wrong with those?

There is one module, CGI::FastTemplate, that does seperate code from
markup completly. But the way different templates are strung together
seems rather comlicated to me.


It's not complicated, but it is somewhat confusing the first time you 
look at it.  I don't find your approach much easier though.

  # generate a new template handler
  my $th=Template::YetAnother->new
({
  namespace=>'acme',
  template_dir=>'/projects/acme/templates/',
 });
 
  # build up a data structure
  $data={
 title=>$th->title('this is the title'),
 breadcrumb=>[
  $th->link({url=>'/index.html',text=>'Home'}),
  $th->separator_breadcrumb,
  $th->link({url=>'/acme/index.html',text=>'Acme'}),
  $th->separator_breadcrumb,
  $th->link({url=>'/acme/bleach.html',text=>'Bleach'}),
 ],
 content=>[
   $th->element({heading=>'SYNOPSIS',value=>'blabla'}),
   $th->element({heading=>'DESCRIPTION',value=>'foo bar'}),
  ],
 lastmod=>scalar localtime,
 
};

   # fill template & print
   print $th->fill({data=>$data});

Isn't your fill method just doing a sort of multi-level join here?  All 
of the data is passed in already, so there is no reason to delay 
evaluation of the templates.

More importantly, your use of AUTOLOAD to treat method calls as file 
names looks neat, but is not a good approach.  It's limiting (all 
templates in one directory!) and has potential security issues with 
namespace clashes.  These are all just the same method call with a 
single argument changed, so why not write them that way?

Here's the same thing written with Template Toolkit.  You could use 
HTML::Template, CGI::FastTemplate, Text::Template, etc. here with minor 
changes:

my $th = Template->new({INCLUDE_PATH => '/projects/acme/templates/');
$th->process->('title.tmpl', {title => 'this is the title'});
$th->process->('link.tmpl', {url => '/index.html', text => 'Home'});
$th->process->('separator.tmpl');
$th->process->('link.tmpl', {url => '/index.html', text => 'Home'});
$th->process->('separator.tmpl');
$th->process->('link.tmpl', {url => '/acme/index.html', text => 'Acme'});
$th->process->('separator.tmpl');

... And so on.

Text::Template allows Perl code to be put in the template, and 
HTML::Template/TT allow loops and conditionals, but no one is going to 
force you to use them.  They are flexible modules.  I think you should 
look at them more closely before you go off on your own.

- Perrin

Re: Newbie: how to use PerlSetVar

[Perrin Harkins]

[EMAIL PROTECTED] wrote:


>What about namespaces, i.e. how
>do I avoid stepping on someone else's variable?


Prepend some custom string in front of the VarName:

 PerlSetVar MyApp_Var1 value1
 PerlSetVar MyApp_Var2 value2



Also, note that these are scoped within the  block where they 
are defined.  This means you can actually have different values for 
different locations, different virtual hosts, etc.

- Perrin

Re: [SOT] Strange browser behavior

[Perrin Harkins]

Nick Challoner wrote:


>My questions are, is there some IE anomaly or some anomalous way
>of configuring or using IE that can cause it to all of a sudden
>do a GET instead of a POST (anything short of manually entering
>the url)?


User bookmarking the page (well, "adding to favourites" considering we're
talking about IE) and then returning to it via the bookmark?



It may also no longer send POST data or give the option of no longer 
sending it if you go back to a page resulting from a POST after a 
certain amount of time.

- Perrin

Re: mod_Perl script and SSI need advice

[Perrin Harkins]

Coexec wrote:


Hi all, I have a question about how to pass form data
with mod_perl and SSI.

I have an HTML page with a mod_perl script included.
The script creates a form and takes its input and then
prints output based on the input (pretty basic).  I
have the form action set to the script
(action="/perl/test.cgi"), the problem is that when
the form is submitted, the only thing that gets
printed to the screen is the cgi output.

I could have the mod_perl script process the
information and then generate a new URL and appended a
Query_string and then redirect($url1?$foo=$bar).  I am
sure that there is a better way to do this, I just
don't know what it is.



I'm afraid I don't quite understand your description of the problem.  Is 
it that the SSI is not working, or that the CGI isn't getting the right 
 input, or what?  Can you show us some of the code from the HTML page 
and the CGI script?  How are you running the CGI?  Apache::Registry?

- Perrin

Re: General interest question: PDF contents handling in PostgreSQL.

[Perrin Harkins]

Fabián R. Breschi wrote:

   I wonder if using ModPerl and PostgreSQL there's any possibility to 
resemble what in Oracle is called 'Intermedia', in this particular case 
parsing/indexing content of PDF files inside PostgreSQL as  a LOB or 
alternatively as a flat OS file with metadata parsed/indexed from it 
into the RDBMS.

You can easilly add this to DBIx::FullTextSearch.  All you need to do is 
write a simple frontend that uses a PDF reading module to extract the 
text.  However, it uses MySQL rather than PostgreSQL.

- Perrin

Re: Resetting cache Apache::Registry

[Perrin Harkins]

Justin Luster wrote:


I know that when you “require” or “use” helper files in a Perl Script,
and you are using Apache::Registry, when changes are made to the helper
files they are not recognized until you restart Apache.  In the
documentation it says that you can change the Apache configuration file
to do this for you.



That's a reference to Apache::Reload or Apache::StatINC.  You can do 
that in an htaccess file if your host allows it.

  What I want to know is if there is a way to clear
out the files or code in the Apache::Registry cache via a Perl Script. 


Yes, but understand that we're not talking about the Apache::Registry 
cache here.  A::R just caches your CGI script itself, not the modules. 
The modules are cached as a normal function of Perl, i.e. once it has 
compiled a module that module stays in memory until the Perl process 
shuts down.

To make Perl reload a module, you can do this:

delete $INC{'Your/Module.pm'};
require Your::Module;

But that glosses over a lot of details about imports and such, so you're 
better off using Apache::Reload if you can.

- Perrin

Re: Apache::Session and user sessions

[Perrin Harkins]

md wrote:

My question is with regards to whether I need or
should put the submitted data into the session as the
user navigates the forms (to create an account). The
user will be taken through three forms to create an
account. So for instance, form one will ask the user
to create a username, password, and provide an email
address. Before moving on to form two (billing info),
should I put this data in the session, or just go
ahead and dump it in the database (after making any
nec. checks), since I won't need the info until they
actually login? Or should I collect all the info from
all three screens by putting it in the session as the
user traverses the forms and then put it all in the
database at once? I'm currently using the first
option. BTW, it is possible for a user to create a
free account by hitting form one only, so no harm
would come if something happened after form one.


This is really a question of requirements.  In systems where all 
information needs to be collected before a valid account can be created, 
you have to wait until the end to put it in the permanent tables.  I 
usually don't store form input in the session because it leads to 
strange results if the user has multiple browser windows open on the 
site, but that may not be an issue for your application.

Another question, while not mod_perl related (sorry:),
is how to taint check input data like usernames,
address fields and email addresses. All info is just
put in the database, no unsafe system calls are run.
I'm curious as to what characters to limit for
usernames in particular.


If you're using bind variables with DBI, there is no technical reason to 
restrict the characters at all.  Just make sure you HTML-escape them 
whenever you display them on a page.

- Perrin

Re: Apache::Session and user sessions

[Perrin Harkins]

Rafiq Ismail (ADMIN) wrote:

I'm not sure how often a user will attempt to complete one form through
multiple browsers.  To be honest I'm not sure that he/she should.


There are all kind of forms.  An obvious example would be a search. 
Users often open up multiple windows when browsing a site and do 
searches in them.  If you store search-related data in the session, the 
multiple windows will interfere with each other.  These should be stored 
in the HTML instead.

- Perrin

Re: how to really bang on a script?

[Perrin Harkins]

On Sat, 28 Oct 2000, Matthew Byng-Maddick wrote:

> On Sat, 28 Oct 2000, Matt Sergeant wrote:
> > exactly the same thing (changing server logs into a benchmark tool) at
> > ApacheCon, only I can't for the life of me remember who it was.
> 
> Theo, during the mod_backhand talk, or at lunch just before, I can't
> remember.

It was during the talk.  The tool is called Daquiri, and he said it was
available in the mod_backhand CVS tree.

I have also found httperf and http_load pretty useful for this stuff,
although they don't support logfile playback.

- Perrin

Re: Proxy Rewrite

[Perrin Harkins]

On Mon, 30 Oct 2000, Kermit Tensmeyer wrote:
>  ProxyPassReverse works as described and modifies
>  the Location header. Is there something else that
>  will filter/translate included URL's?
> 
>  For example
>ProxyPass  /product/itemA/  
>http://modperl.internal.com/root/service/path/mason/
>ProxyPass  /product/itemB/ 
> http://websphere.internal.com/content/product/information/
> 
>ProxyPass  /customer/Service/  
> http://iis.internal.com/inetpub//ASP/customer/
> 
>  Will translate on the way in  and the equiv ProxyPassReverse 
> will modify the Location for the proxied pages going out. What 
> methods of others used on the Light Server to modify the urls
> generated by pages on the backend servers to resemble 
> http://external.server.com/product/itemB/faq.shtml ?

You need to either use relative URLs or make the back-end server generate
correct fully-qualified URLs that point to the front-end.  If the back-end
server is not under your control, you will have to use some kind of
search/replace on the front-end.  On a heavy traffic site this would most
likely need to be done as a hack to mod_proxy.  (Apache 2 will solve this
sort of problem using filter modules.)

- Perrin

Re: Apache::DB and core dump

[Perrin Harkins]

On Mon, 30 Oct 2000, Marek W wrote:

> Do you possibly know  what could have caused this error while trying to run
> this module. I use Linux RH 6.2. and mod_perl 1.23

I've had problems with Apache::DB when using Apache::Request.  I have not
attempted to solve them yet.
- Perrin

Re: ApacheCon report

[Perrin Harkins]

On Mon, 30 Oct 2000, Tim Sweetman wrote:

> Matt Sergeant wrote:
> > 
> > On Fri, 27 Oct 2000, Tim Sweetman wrote:
> > 
> > > In no particular order, and splitting hairs some of the time...
> > >
> > > Sounded like mod_backhand was best used NOT in the same Apache as a phat
> > > application server (eg. mod_perl), because you don't want memory-heavy
> > > processes sitting waiting for responses. You'd be better off with a
> > > separate switching machine - or serve your static content from
> > > machine(s) that know to backhand dynamic requests to a phat machine. I
> > > think that's what Theo reckoned...
> > 
> > Yes, but the backend mod_perl servers are running backhand. So you have:
> > 
> > B  B  B  B
> >  \ |  | /
> >   \ \/ /
> >\|/
> > F
> > 
> > Where all the servers are running mod_backhand, but only F is publically
> > accessible. There may also be >1 F. Its in his slides, and is prettier
> > than the above :-)
> 
> Yeah. I know how it was set up in Theo's demo (like that) but I got the
> impression that this wouldn't be optimal for a mod_Perl setup (or other
> big-footprinted configuration). You _can_ run mod_backhand on your
> content servers. You don't _have_ to.

Here's what I recall Theo saying (relative to mod_perl):

- Don't use a proxy server for doling out bytes to slow clients; just set
the buffer on your sockets high enough to allow the server to dump the
page and move on.  This has been discussed here before, notably in this
post:

[EMAIL PROTECTED]">http://forum.swarthmore.edu/epigone/modperl/grerdbrerdwul/[EMAIL PROTECTED]

The conclusion was that you could end up paying dearly for the lingeirng
close on the socket.

- If you use apache+mod_proxy as a load balancer in front of your back-end
servers (as opposed to a commercial solution like big ip), use
mod_backhand instead and your front-end server will be able to handle
requests itself when it's not too busy.

- He has created a way for proxied requests to use keep-alive without
enabling keep-alive for the whole server.  The obvious problem - that each
server will soon use up every other server's available clients - is
somehow avoided by sharing open sockets to the other servers in some
external daemon.  This sounded cool, but fishy.

Ultimately, I don't see any way around the fact that proxying from one
server to another ties up two processes for that time rather than one, so
if your bottleneck is the number of processes you can run before running
out of RAM, this is not a good approach.  If your bottleneck is CPU or
disk access, then it might be useful.  I guess that means this is not so
hot for the folks who are mostly bottlenecked by an RDBMS, but might be
good for XML'ers running CPU hungry transformations.  (Yes, I saw Matt's
talk on AxKit's cache...)

> One thing I have my eye on (which doesn't mean I'll necessarily get it
> done :) ) is some sort of data-holding-class that sits between an
> application and a template in a transparent way (eg. it could hold the
> method names & args that you were passing to a templating system, like a
> "command" design pattern IIRC).

In Perl, we call that a "variable".  (Sorry, couldn't resist...)  
Templating systems in Java jump through hoops to make generic data
structures, but Perl doesn't have to.  Just freeze it with Storable if you
need to save it.

> This would potentially allow:
> + switching between different templating systems ...?

Nearly all of them use standard perl hash refs.

> + checking out template tweaks without rerunning the app - good for
> "interactive" systems which
>   keep chucking form data at users

This is easy to do with most systems.  I once wrote something that could
turn arbitrary form inputs into a data structure suitable for feeding to
Template Toolkit or similar.  Then I could create a form for entering
different kinds of test data, and save the test data using Storable.  It
was used for building templates for a system which ran the templating as a
batch process and so had a terrible turnaround time for testing changes.

> + (fairly transparent) conversion to XML/XSL/etc at an appropriate time,
> as/when/if a site/project
>   grows to an appropriate size

There are some modules out there that serialize perl data structures as
XML.  Or you can just write a template for it.

- Perrin

Re: ApacheCon report

[Perrin Harkins]

On Tue, 31 Oct 2000, Les Mikesell wrote:
> > Ultimately, I don't see any way around the fact that proxying from one
> > server to another ties up two processes for that time rather than one, so
> > if your bottleneck is the number of processes you can run before running
> > out of RAM, this is not a good approach.
> 
> The point is you only tie up the back end for the time it takes to deliver
> to the proxy, then it moves on to another request while the proxy
> dribbles the content back to the client.   Plus, of course, it doesn't
> have to be on the same machine.

I was actually talking about doing this with no front-end proxy, just
mod_perl servers.  That's what Theo was suggesting.

I use a mod_proxy front-end myself and it works very well.

- Perrin

Re: ApacheCon report

[Perrin Harkins]

On Tue, 31 Oct 2000, Gunther Birznieks wrote:
> As a bonus, if you write your app smart with cache directive 
> headers, some of the dynamic content can truly be cached by the front-end 
> server.

We're using this technique now and it really rocks.  Great performance.

- Perrin

Re: hashes and mod_perl

[Perrin Harkins]

On Tue, 31 Oct 2000, Scott Alexander wrote:
> Is it in anyway possible to tie the hash once to a dbm file?
> 
> At the moment it is tied every time a script is called. 

There's a whole chapter on this: http://perl.apache.org/guide/dbm.html.

The short answer is that you have to tie the dbm file once in every child,
but only once.

- Perrin

Re: ApacheCon report

[Perrin Harkins]

On Tue, 31 Oct 2000, Ask Bjoern Hansen wrote:

> On Mon, 30 Oct 2000, Perrin Harkins wrote:
> 
> [...]
> > - Don't use a proxy server for doling out bytes to slow clients; just set
> > the buffer on your sockets high enough to allow the server to dump the
> > page and move on.  This has been discussed here before, notably in this
> > post:
> > 
> > 
>[EMAIL PROTECTED]">http://forum.swarthmore.edu/epigone/modperl/grerdbrerdwul/[EMAIL PROTECTED]
> > 
> > The conclusion was that you could end up paying dearly for the lingeirng
> > close on the socket.
> 
> Mr. Llima must do something I don't, because with real world
> requests I see a 15-20 to 1 ratio of mod_proxy/mod_perl processes at
> "my" site. And that is serving <500byte stuff.

I'm not following.  Everyone agrees that we don't want to have big
mod_perl processes waiting on slow clients.  The question is whether
tuning your socket buffer can provide the same benefits as a proxy server
and the conclusion so far is that it can't because of the lingering close
problem.  Are you saying something different?

- Perrin

Re: Putting together the TPC mod_perl track

[Perrin Harkins]

On Wed, 1 Nov 2000, Nathan Torkington wrote:
>  * Case studies showing how big companies use mod_perl
> 
> This latter is an important part of the Perl conference.  Many
> companies who would never 'fess up to using Perl seem quite happy
> to send employees to speak at conferences.  Their talks end up as
> a big advertisement for Perl, and lets us name-drop the company as
> a Perl user.  I see no reason why the same shouldn't happen with
> mod_perl.

I may be able to offer something on how we use mod_perl at eToys.  We
recently rewrote our codebase to take better advantage of mod_perl and are
using some fun OO stuff, as well as a bunch of scalability tricks.

I was also thinking about presenting a comparison of templating methods
and modules.

- Perrin

Re: return from nested functions

[Perrin Harkins]

On Wed, 1 Nov 2000, Matt Sergeant wrote:
> Definitely use exceptions. I prefer Error.pm for this (sorry, Dave!),
> which allows your handler to simply be:
> 
> sub handler {
>   return try {
>   ...
>   } catch Exception::RetCode with {
>   my $E = shift;
>   return $E->{code};
>   } 
>   ;
> }

Be very careful with nested try/catch blocks.  Error.pm uses prototypes to
make these into anonymous subs, which is usually fine, but this leaks
memory like mad:

my $foo;
try {
something();
try {
$foo++;
};
} otherwise {
handle_error();
};

It's not Graham Barr's fault; that's just a byproduct of Perl's support
for closures (I think).

- Perrin

Re: ApacheCon report

[Perrin Harkins]

On Wed, 1 Nov 2000, Leslie Mikesell wrote:
> I still like the idea of having mod_rewrite in a lightweight
> front end, and if the request turns out to be static at that
> point there isn't much point in dealing with proxying.

Or if the request is in the proxy cache...

> Has anyone tried putting software load balancing behind the front end
> proxy with something like eddieware, balance or ultra monkey?  In that
> scheme the front ends might use an IP takeover failover and/or DNS
> load balancing and would proxy to what they think is a single back end
> server - then this would hit a tcp level balancer instead.

We use that setup with a hardware load balancer.  It works very well.

- Perrin

Re: Connection Pooling / TP Monitor

[Perrin Harkins]

Tim Bunce wrote:
> You could have a set of apache servers that are 'pure' DBI proxy servers.
> That is, they POST requests containing SQL (for prepare_cached) plus
> bind parameter values and return responses containing the results.
> 
> Basically I'm proposing that apache be used as an alternative framework for
> DBI::ProxyServer. Almost all the marshaling code and higher level logic
> is already in DBI::ProxyServer and DBD::Proxy. Shouldn't be too hard to do
> and you'd gain in all sorts of ways.

I think this is a really good idea.  The thing is, any effort put into
this kind of thing right now feels like a throw away, because mod_perl
2.0 will solve the problem in the right way with real pooling of
database handles (and other objects) between threads.

Maybe it's time for DBD:: authors to start checking their code for
thread safety?

- Perrin

Re: HTTP Mod_Perl mini-server

[Perrin Harkins]

On 3 Nov 2000, David Hodgkinson wrote:
> Dare I add that Squid has plenty of low-latency cacheing features you
> could use? 

In my tests, a modern version of mod_proxy (serving from cache) was faster
than Squid on Linux.

- Perrin

Re: HTTP Mod_Perl mini-server

[Perrin Harkins]

On Fri, 3 Nov 2000, Vivek Khera wrote:
> Lately I've been getting very interested in using solid-state disks
> for high-performance issues.  They're expensive, but if you need that
> much speed, they're worth it.

Are they?  I tried one once, and it wasn't any faster than my normal disk
because I had so much RAM it was all getting buffered already.  If you
don't have enough RAM, it might help, but I suspect these are more
expensive than equivalent amounts of RAM.

- Perrin

Re: HTTP Mod_Perl mini-server

[Perrin Harkins]

On 3 Nov 2000, David Hodgkinson wrote:

> > In my tests, a modern version of mod_proxy (serving from cache) was faster
> > than Squid on Linux.
> 
> Really? Cool. What about taking memory usage into account?

Well, Squid is kind of a memory hog and mod_proxy has been extremely light
and well-behaved, even under heavy load.  I would encourage others to try
benchmarking it for themselves, but I currently see no reason to use Squid
over mod_proxy.
- Perrin