Re: mod_proxy caching documentation
Dan McCormick wrote:
Hi,
After struggling with trying to figure out mod_proxy's caching algorithm
and noting from the list archive's that others had, too -- and due to
the dearth of existing documentation on the subject -- I came up with
some documentation below by sifting through the source code. Most of it
isn't explicitly mod_perl-related, but I hope those trying to set it up
Thanks for the read. Very enlightening. I'm guessing
the dir levels matters because it lets the files be
spread over that many more directories, so there isn't a
large directory hashing penalty on a HUGE number of files.
5 is probably a bit much though if it really creates 4-5
directories for each file it stores, and if you are using
this only for a proxy in reverse mode for mod_perl, its likely
you could get away with 2-3 levels.
I think it would be interesting if you chronicled the capacity
improvements to your site using the mod_proxy server like this.
I don't know how well mod_proxy does this caching from a performance
perspective, and it might be nice to see some numbers that
one could later compare with some of the commercial caching
products.
--Joshua
will find it useful. Included at the end is a Perl script to determine
the filename that mod_proxy uses to cache files, which is helpful in
manually cleaning up the cache. If anyone has comments or
suggestions, please let me know.
Thanks,
Dan
Setting up Apache with mod_proxy to cache content from a mod_perl server
The documentation for mod_proxy can be found at
http://httpd.apache.org/docs/mod/mod_proxy.html. Unfortunately, aside
from the configuration parameters, not much detail is provided on how to
set up mod_proxy to cache pages from a downstream server. This
explanation hopes to fill that void. Most of its content was derived by
going through the mod_proxy.c, proxy_cache.c, and proxy_util.c source
files and comments in the src/modules/proxy directory of the Apache
1.3.12 distribution.
* The Short Story
In short, mod_proxy will cache all requests that contain a Last-Modified
header and an Expires header. You can insert this into your mod_perl
scripts with something like this:
use Apache::File ();
use HTTP::Date;
$r-set_last_modified((stat $r-finfo)[9]); # see Eagle book p. 493 for
explanation
$r-header_out('Expires', HTTP::Date::time2str(time + 24*60*60)); #
expires in one day
The page will live in the cache until the current time passes the time
defined by the Expires header or the time since the file was cached
exceeds the CacheMaxExpire parameter as set in the server config file.
* The Long Story
To understand how the caching proxy server works, let's trace the flow
of two simple HTTP exchanges for the same file, from the browser request
to the returned page.
- The browser makes a request to the proxy server like this:
GET /index.html HTTP/1.0
- The proxy server takes the URL and converts it to a filename on your
filesystem. This filename has no resemblance to the actual URL.
Instead, it is an MD5 hash of the fully qualified URL (e.g.
http://www.myserver.com:80/mypage.html) to the document and is broken up
in a number of directory levels, as defined by the CacheDirLevels
parameter in the config file. (WHY DOES IT MATTER HOW MANY DIR LEVELS
ARE IN THE CACHE?) Each of these directories will have a certain number
of characters in its name, as defined by the CacheDirLength parameter in
the config file. The directories will live under CacheRoot, also
defined in the config file. For example, /index.html might be converted
to /proxy_cache/m/EYRopVKBHMrHd2VF6WXOQ (with CacheDirLevels and
CacheDirLength set to 1 and CacheRoot set to /proxy_cache).
- For this example, we'll assume that at this point the cached file does
not exist. The proxy server then consequently forwards the request to
the mod_perl server and gets a response back. The response will then be
cached UNLESS any of the following conditions are true
(ap_proxy_cache_update):
- The HTTP status returned by the mod_perl server is not one of OK,
HTTP_MOVED_PERMANENTLY, or HTTP_NOT_MODIFIED
- The response does not contain an Expires header
- The response contains an Expires header that Apache can't parse
- The HTTP status is OK but there's not a Last-Modified header
- The mod_perl server sent only an HTTP header
- The mod_perl server sent an Authorization field in the header
(Furthermore, if any of the above conditions are met, any existing
cached file will be deleted.)
- If the server decides to cache the file, it will store the file
exactly as it was received from the mod_perl server, with the addition
of a one-line header at the start of the file. This header contains the
following information in the following format:
current time last modified time expiration time "version"
content length
All times are stored as hex seconds since 1970 and
Re: Microperl
On Thu, 16 Nov 2000, Robin Berjon wrote: At 00:12 16/11/2000 -0600, Les Mikesell wrote: Nothing against mod_rewrite -- I was just wondering if a small perl could be embedded with out bloating the server too much. I don't think 'small' and 'perl' belong in the same sentence... I know what you mean but this is MicroPerl, not Perl. I don't know how much difference it makes, but it's certainly smaller. I'm afraid I can't help with embedding it though. I like the idea, not just for rewrites (I'm quite happy with mod_rewrite most of the time) but for all the conf stuff. It doesn't have dynaloader, but you could still do things like read conf from a flat file db or that sort of thing. It doesn't have anything that isn't *entirely* portable and *entirely* guaranteeable, which means a large number of syscalls, and anything which isn't in KR2. MBM -- Science is built up of facts, as a house with stones. But a collection of facts is no more a science than a heap of stones is a house. -- Poincare
RE: [RFC] Apache::Expires
On Wed, 15 Nov 2000, Geoffrey Young wrote: -Original Message- From: Robin Berjon [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 15, 2000 10:43 AM To: Geoffrey Young Cc: '[EMAIL PROTECTED]' Subject: Re: [RFC] Apache::Expires At 10:26 15/11/2000 -0500, Geoffrey Young wrote: anyone else see an interest in an Apache::Expires (or whatever)? Is there something out there already? I was thinking of working on it if there is nothing out there, but it would require lots of RFC reading and wrestling with stuff like entity tags... It would definitely be interesting to provide an easy to use API to expiration and caching (at least one that would cover common needs), it's easy to get mixed up in the details. I think Andreas wrote something about all those issues last year, but I can't track it down right now. I think he wrote the section in the guide on headers - that will definitely help sort things out... It's kinda a chapter, not a section: http://perl.apache.org/guide/correct_headers.html the more I think about it, thoug, I think it will be quite difficult. say you want to cache a dynamic document for a week from creation - how do you define when the week started? maybe some compromises will be needed, like generically saying "months start at 1" and "weeks start at sunday", etc... just thinking out loud... --Geoff -- robin b. Always remember you're unique just like everyone else. _ Stas Bekman JAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide http://perl.apache.org/guide mailto:[EMAIL PROTECTED] http://apachetoday.com http://jazzvalley.com http://singlesheaven.com http://perlmonth.com perl.org apache.org
Uri modification at translation phase ...
Hi Everybody. I'm making a module that modifies the uri at the translation phase, but I have a doubt. The way I do it is modifying the uri and returning DECLINED as I read in the book "Writing Apache Modules with Perl And C". But working like this, the environment variable QUERY_STRING is well modified but the uri at the browser is not changed. I have test return REDIRECT but then POST calls don't work. Any suggestion? Thanks.
Environment configuration problems
OK, I've tried everything I can think of and I'm still having environment configuration problems. I have a series of scripts that access an Oracle database. At a minimum, two environment variables need to be set up for this; ORACLE_HOME and LD_LIBRARY_PATH (must include $ORACLE_HOME/lib) I set both of these in the httpd.conf file for both the CGI and mod_perl environment using using SetEnv and PerlSetEnv respectively. Looking at the environment using /perl-status shows them set properly. Now, when I try to kick hit one of these via mod_perl, I consistently get a'failed compilation' error when the script attempts to hook up to the database. THE EXACT SAME SCRIPT will run fine as a plain old CGI. If I shutdown the web server, set ORACLE_HOME and LD_LIBRARY_PATH in my environment (root's environment actually) and restart Apache, the mod_perl references start working fine. Is there some strangeness in my current version of Apache that only allows environment variables to be set for mod_perl *if* they are set in the Apache startupenvironment first? I'm currently running Apache 1.3.12 on RedHat 6.0 (2.2.14)with mod_perl 1.24 build on Perl 5.00503. Apache::DBI 0.87 is loaded along with DBI 1.14. Obviously the workaround for the time being is to start up the server manually, but this DOES NOT help matters if the server has to reboot for some reason. One other thing, I have an almost identical configuration running on two other servers, and it seems to run fine. The differences being the Linux version (Mandrake 7.0 vice RedHat) on one and Apache 1.3.9 on the other. I've 'diffed' everything I can think of and just can't find the problem I'm certain that I have just missed *one little line* in a config script, but try as I might, I can't find it. Thanks for the help! - Ralph Hughes [EMAIL PROTECTED]Sr. Systems Engineer 334-260-3200Anteon Corp. DSN 596-5631Montgomery, AL
RE: [RFC] Apache::Expires
-Original Message-
From: Perrin Harkins [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, November 15, 2000 6:19 PM
To: Geoffrey Young
Cc: '[EMAIL PROTECTED]'
Subject: Re: [RFC] Apache::Expires
On Wed, 15 Nov 2000, Geoffrey Young wrote:
I was wondering if anyone has some experience with expire
headers for
dynamic documents - kinda like mod_expires but for dynamic stuff.
We do this, and let mod_proxy use our headers to control its cache and
handle If-Modified requests.
I guess I was thinking about sans proxy...
Um, it's pretty easy:
my $last_mod = time;
my $expires = time + 360; # expires in one hour
$r-header_out('Expires' = Apache::Util::ht_time($expires));
$r-header_out('Last-Modified' =
Apache::Util::ht_time($last_modified));
Or did you have something different in mind?
maybe... I might not have my head in the right place, though...
it's the lack of a 304 that's bothering me (today :)
If I just put your lines into a handler I get this from netscape:
If-Modified-Since = Thu, 16 Nov 2000 12:48:04 GMT; length=1150
but, since there is no modification time for the 'document' to compare to
locally,
my headers out are:
Expires = Thu, 16 Nov 2000 12:56:45 GMT
Last-Modified = Thu, 16 Nov 2000 12:50:45 GMT
with status 200 since the document was (again) generated (with new headers).
whereas for static documents, I see:
If-Modified-Since = Wed, 15 Nov 2000 14:45:13 GMT; length=2057
and a 304 out that contains:
Last-Modified = Wed, 15 Nov 2000 14:45:13 GMT
ETag = "20f73-809-3a12a179"
I guess what I am really after is intercepting the If-Modified-Since tag and
return a 304 prior to content generation - maybe in a fixup handler... I
think it is the entity tag stuff that is starting to throw me, too...
make sense? It's a new day (but before my coffee) so who knows...
thanks for the input
--Geoff
- Perrin
RE: Environment configuration problems
try using a startup.pl to pre-load DBD::Oracle (and everything else while you're at it) and set these environment variables from within a BEGIN block there. I've found that this, along with PerlSetEnv, works fine on Linux. if that fails, try a BEGIN block in each script as well... HTH --Geoff -Original Message-From: Hughes, Ralph [mailto:[EMAIL PROTECTED]]Sent: Thursday, November 16, 2000 8:00 AMTo: ModPerl (E-mail)Subject: Environment configuration problems OK, I've tried everything I can think of and I'm still having environment configuration problems. I have a series of scripts that access an Oracle database. At a minimum, two environment variables need to be set up for this; ORACLE_HOME and LD_LIBRARY_PATH (must include $ORACLE_HOME/lib) I set both of these in the httpd.conf file for both the CGI and mod_perl environment using using SetEnv and PerlSetEnv respectively. Looking at the environment using /perl-status shows them set properly. Now, when I try to kick hit one of these via mod_perl, I consistently get a'failed compilation' error when the script attempts to hook up to the database. THE EXACT SAME SCRIPT will run fine as a plain old CGI. If I shutdown the web server, set ORACLE_HOME and LD_LIBRARY_PATH in my environment (root's environment actually) and restart Apache, the mod_perl references start working fine. Is there some strangeness in my current version of Apache that only allows environment variables to be set for mod_perl *if* they are set in the Apache startupenvironment first? I'm currently running Apache 1.3.12 on RedHat 6.0 (2.2.14)with mod_perl 1.24 build on Perl 5.00503. Apache::DBI 0.87 is loaded along with DBI 1.14. Obviously the workaround for the time being is to start up the server manually, but this DOES NOT help matters if the server has to reboot for some reason. One other thing, I have an almost identical configuration running on two other servers, and it seems to run fine. The differences being the Linux version (Mandrake 7.0 vice RedHat) on one and Apache 1.3.9 on the other. I've 'diffed' everything I can think of and just can't find the problem I'm certain that I have just missed *one little line* in a config script, but try as I might, I can't find it. Thanks for the help! - Ralph Hughes [EMAIL PROTECTED]Sr. Systems Engineer 334-260-3200Anteon Corp. DSN 596-5631Montgomery, AL
RE: AuthCookie solution
-Original Message- From: Michael [mailto:[EMAIL PROTECTED]] Sent: Wednesday, November 15, 2000 10:20 PM To: [EMAIL PROTECTED] Subject: Re: AuthCookie solution a little off the subject, but close. it the pointer for login is a complete URL ie http://foo.com/login.pl then the $r-prev-args; seem to get lost. There is real application for this sort of thing and I would like to figure out how to make it work. Any ideas why it does not? $r-prev() only knows about the previous internal redirect. $r-header_out(Location = 'http://foo.com/login.pl'); results in a full redirect (ie a new request) as an aside, I've found it safer to when using $r-prev to code like my $query = $r-prev ? $r-prev-args : $r-args; because $r-prev is undefined on $r-is_initial_req() which leads to runtime errors (beware of my $x if 0 coding, too :) depends on your situation, though - YMMV... --Geoff Michael [EMAIL PROTECTED]
Re: mod_proxy caching documentation
On Thu, Nov 16, 2000 at 01:37:41AM -0500, Dan McCormick wrote: I came up with some documentation below by sifting through the source code. Excellent, thanks! If a malformed Expires: prevents mod_proxy from caching a response ( The response will then be cached UNLESS any of the following conditions are true (ap_proxy_cache_update): [snip] - The response contains an Expires header that Apache can't parse ), why do they go to some lengths to make up for a malformed one ( if the Expires time cannot be parsed and a Last Modified time exists from the previous step, then the Expires time is set to "now + min((date - lastmod) * factor, maxexpire)" (as noted in the source code comments) )? I'm assuming that it can because that's a bit of extra logic that wouldn't need to be there otherwise. Or maybe it's leftover code that never fires. I thought (not that I remember why) that it didn't need an Expires: header and that it would make up a value of it's own based on the .conf settings. - Barrie
RE: database access
-Original Message- From: Gunther Birznieks [mailto:[EMAIL PROTECTED]] Sent: Thursday, November 16, 2000 4:04 AM To: Les Mikesell; [EMAIL PROTECTED] Cc: Tim Bunce; Aaron; [EMAIL PROTECTED] Subject: Re: database access 3. As a plug, I had also suggested a couple years ago that I would like the feature to be able to specifically not having the ping() method by Apache::DBI if the database had already been pinged within a set period of time. Apache::DBI-setPingTimeOut($data_source, $timeout) no? I similarly pointed out last last year that there is a (small) bug in Apache::DBI that when you want it to ping _every_ time it won't if two pings are in the same 'second' due to the granularity of time() (ie, a content handler and log handler that access the db within one second of eachother, but in separate phases). --Geoff
RE: database access
Uh, I was just seeing if anyone actually read my emails. :) Actually, I am quite happy this was added. Thanks! At 09:02 AM 11/16/00 -0500, Geoffrey Young wrote: -Original Message- From: Gunther Birznieks [mailto:[EMAIL PROTECTED]] Sent: Thursday, November 16, 2000 4:04 AM To: Les Mikesell; [EMAIL PROTECTED] Cc: Tim Bunce; Aaron; [EMAIL PROTECTED] Subject: Re: database access 3. As a plug, I had also suggested a couple years ago that I would like the feature to be able to specifically not having the ping() method by Apache::DBI if the database had already been pinged within a set period of time. Apache::DBI-setPingTimeOut($data_source, $timeout) no? I similarly pointed out last last year that there is a (small) bug in Apache::DBI that when you want it to ping _every_ time it won't if two pings are in the same 'second' due to the granularity of time() (ie, a content handler and log handler that access the db within one second of eachother, but in separate phases). --Geoff __ Gunther Birznieks ([EMAIL PROTECTED]) eXtropia - The Web Technology Company http://www.extropia.com/
Re: security suggestion
Following up on the security suggestion (I'm actually responding to private mail, so I'll just quote the person who wrote to me without giving a name) - Of course you can do this in an .htaccess file, too: Perl arbitrary perl code... /Perl I'd argue that people shouldn't be able to do that - unless, of course, they can ExecCGI. The general idea is this: Regular users without ExecCGI may have good reason to say things like, PerlAuthenHandler Apache::SomeNewAuthModule E.g., suppose you want them to be able to use some local secure cookie- based auth handler. Or they may want to use a locally written response handler that converts HTML to XHTML on the fly - or innumerable other modules. Although we want those users to have access to these modules (and to the added functionality they bring), we may not want them to be able to execute arbitrary Perl code. And if we DO want them to execute arbitrary Perl code, we should have to explicitly grant them permission to do it. Permission shouldn't be granted implicitly by doing something else (e.g., by enabling another module like mod_perl). My sense is that the existing ExecCGI mechanism is a perfectly reasonable way of handling the permissions here, since being able to execute arbi- trary Perl code is basically the same as ExecCGI privileges (it's a bit more powerful, but why introduce more options if we don't really need to?). -- Richard Goerwitz[EMAIL PROTECTED]
RE: security suggestion
-Original Message-
From: Richard L. Goerwitz [mailto:[EMAIL PROTECTED]]
Sent: Thursday, November 16, 2000 10:11 AM
To: [EMAIL PROTECTED]
Cc: Geoffrey Young
Subject: Re: security suggestion
Following up on the security suggestion (I'm actually responding to
private mail, so I'll just quote the person who wrote to me without
giving a name) -
Of course you can do this in an .htaccess file, too:
Perl
arbitrary perl code...
/Perl
I'd argue that people shouldn't be able to do that - unless,
of course,
they can ExecCGI.
The general idea is this: Regular users without ExecCGI may have good
reason to say things like,
PerlAuthenHandler Apache::SomeNewAuthModule
but what about
PerlSetEnv PERL5LIB '/my/lib/'
PerlAuthenHandler My::Handler
there's nothing that says an AuthenHandler has to authenticate :)
maybe it would be possible to limit
PerlAuthenHandler 'sub {do something desctructive};'
via a directive, but this is mod_perl - I can't see how you would be able to
allow good activity without there being some way around it for destructive
types...
at least not in a generic sense - you could write an Init handler that
pushes Your::Validator to the front of each phase which runs all configured
handlers through various checks (of course one that seemed valid could push
another bad handler to the stack on the fly)
see what I mean - protection is not guaranteed...
Although we want those users to have access to these modules
(and to the
added functionality they bring), we may not want them to be able to
execute arbitrary Perl code.
PERL_SECTIONS=0 at build time can deactivate this across the board
--Geoff
RE: security suggestion
Perhaps you ought to gfind a way to use Safe; then?
-Original Message-
From: Richard L. Goerwitz [mailto:[EMAIL PROTECTED]]
Sent: Thursday, November 16, 2000 9:09 AM
To: mod_perl list
Subject: security suggestion
At Doug's suggestion I'm moving a brief conversation we've had
in private
to this list so others can get involved.
I've been following out the security implications of mod_perl here at
Brown University, and from what I can see, enabling it gives people the
same basic access privileges that ExecCGI does. How does it do this?
Well, for one thing, you can inject arbitrary Perl code at any stage in
the Apache request cycle by using a literal
PerlWhateverHandler 'sub { ... }'
in an htaccess file. You can also specify
PerlWhateverHandler
Some::Arbitrary::Path::To::An::Arbitrary::Module.pm
I've been thinking that, since giving users the ability to do these
things is essentially like giving them ExecCGI privileges, that these
things should only be allowed where the ExecCGI option is turned on.
There's also the problem, if a person can ExecCGI, that they can then
use Apache (), and then access all dir_config, notes, and
other settings.
At times it may be useful for a systems administrator to create private
dir_config variables that run-of-the-mill CGI scripts simply can't see.
These are two separate issues, of course.
The more pressing one, I think, is the first, namely that we need to
tie the ability to use arbitrary Perl subroutines and arbitrary Perl
modules (i.e., modules in non-system paths) to the ExecCGI option.
I realize that forcing us all to turn ExecCGI on for directories where
we want to use arbitrary Perl subroutines would be a bit of a pain at
this point. So for the current 1.24_0x series I'd suggest adding some
new "paranoid" compile-time option that would deny arbitrary subs and
modules in non-system paths unless ExecCGI is enabled. Another idea
that I haven't really thought through would be to create some new
PerlLiteralSubs On|Off directive that could not be overridden once
turned on.
For the future, though, it might be useful to tie arbitrary subs and
the ability to use Perl modules to ExecCGI. ExecCGI-like privileges
something a sysadmin should always have to decide to do. My sense is
that they should never be turned on implicitly, e.g., by simply adding
and enabling a module.
Does this make sense to anyone else?
--
Richard Goerwitz
[EMAIL PROTECTED]
Re: Apache::Sandwich and CGI scripts
"HJE" == Hackett, Jonny E [EMAIL PROTECTED] writes: HJE I'm attempting to incorporate Apache::Sandwich into a site I'm working on HJE and I'm having some problems getting sandwich to work with cgi scripts. What shows up in your error logs? Did you read the section in the Apache::Sandwich docs about sandwiching mod_perl programs? If you are trying to sandwich plain old CGI's (which it doesn't seem like) then you can set SandwichHandler to cgi-handler and let it do the work. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= Vivek Khera, Ph.D.Khera Communications, Inc. Internet: [EMAIL PROTECTED] Rockville, MD +1-240-453-8497 AIM: vivekkhera Y!: vivek_khera http://www.khera.org/~vivek/
Re: Microperl
At 18:35 16/11/2000 +0100, Fabrice Scemama wrote: ok so what about miniperl, which is used when building perl? I think using whichever of the small perls that are used during build (or were thought about to use during build) could potentially be interesting. -- robin b. As a computer, I find your faith in technology amusing.
RE: security suggestion
Maybe it's just me, but it seems that the responses richard has gotten haven't really touched on the core of the problem. That mod_perl isn't exactly friendly to sysadmin's who want to run apache on a (i'm guessing), student accessed server, with user dir's and all that other stuff. I'm pretty sure (for no particular reason), that there aren't many people on this list that are doing that. If you have .htaccess stuff turned on right now, you can do all sorts of great things through apache that you wouldn't want untrusted accounts on the box being able to do. The servers that had apache on them for users when i was at school didn't even allow normal cgi, so i have no idea how one would approach doing something like this with mod_perl. If, on the other hand, i'm misinterpreting richards problems, feel free to ignore me. Adam
RE: security suggestion
The thing is, though, that as a web administrator I don't want those same developers (or at least all of them) to be able to create and in- stall _arbitrary_ handlers or arbitrary perl code. Sometimes the de- velopers just don't know enough. And sometimes I just don't trust them enough to allow it. And just putting your configuration for their portion of the site into the main configuration files would be insufficient? Christian
RE: Table Joins
Speaking from an Oracle perspective, it is generally better to use a join in preference to an "IN" type subquery since the execution plan is normally more efficient. However, some types of queries can only be solved with a subquery clause such is the case with correlated subqueries. This is a huge subject and the mod_perl list is not an appropriate forum for this type of discussion. Regards, -Original Message- From: ASHISH MUKHERJEE [mailto:[EMAIL PROTECTED]] Sent: Thursday, November 16, 2000 9:20 AM To: [EMAIL PROTECTED] Subject: Table Joins Hello everyone ! Can someone shed some light on something related to databases. I'd like to know is it more expensive to perform a Table Join to obtain a result or is it better to use a sub-query instead. Thanks Ashish --- cout"Hello World !"; Get FREE Email/Voicemail with 15MB at Lycos Communications at http://comm.lycos.com
Files .. in virtualhosts in perl sections
Anyone have any experience doing Files sections inside of perlsections virtualhosts. I can't see anything in the docs about this. Just wanted to ask before I start experimenting, as I don't have the first clue about how this syntax might work. Thanks, -- Tom Lancaster Red Hat, Inc. Web Engineer(415) 777-9810 x 228
Trouble building debug verson of Apache+mod_perl on Linux
I am getting segfaults from an Apache/mod_perl/Mason setup so I decided I needed to build both Apache and mod_perl from source using -g and without DSOs so I could easily debug it with gdb. Alas, that is turning out to be tougher than I expected. I'm probably doing something fundamentally wrong. Hopefully someone here can steer me in the right direction. My environment is Linux Mandrake 7.1. I'm trying to build and run Apache 1.3.14 and mod_perl 1.24_01 from their source distributions (I'm not using RPMs). I have two problems: 1. It appears that not all the modules I asked to be enabled at configure time are found, and 2. When I configure mod_perl it seems to override my desired set of apache builtin modules. I configured apache using these settings: ./configure --with-layout=RedHat --enable-module=proxy \ --enable-module=rewrite --enable-module=log_agent \ --enable-module=log_referer --enable-module=info \ --enable-module=auth_anon --enable-module=digest \ --enable-module=expires --enable-module=headers \ --enable-module=usertrack --enable-module=userdir After compiling (without any mod_perl fiddling), executing "src/httpd -l" yields Compiled-in modules: http_core.c mod_env.c mod_log_config.c mod_log_agent.c mod_log_referer.c mod_mime.c mod_negotiation.c mod_status.c mod_info.c mod_include.c mod_autoindex.c mod_dir.c mod_cgi.c mod_asis.c mod_imap.c mod_actions.c mod_userdir.c mod_alias.c mod_rewrite.c mod_access.c mod_auth.c mod_auth_anon.c mod_digest.c mod_proxy.c mod_expires.c mod_headers.c mod_usertrack.c mod_setenvif.c suexec: disabled; invalid wrapper /usr/sbin/suexec However, when I check the config file with "src/httpd -t" it complains: Syntax error on line 137 of /etc/httpd/conf/httpd.conf: Invalid command 'UserDir', perhaps mis-spelled or defined by a module not included in the server configuration Line 137 is the standard UserDir public_html Why is apache complaining? I configured mod_perl like so (after configuring and building apache as above): perl Makefile.PL APACHE_SRC=../apache_1.3.14/src \ DO_HTTPD=1 USE_APACI=1 EVERYTHING=1 After running "make" in the mod_perl directory, "src/httpd -l" yields an abbreviated set of modules: Compiled-in modules: http_core.c mod_env.c mod_log_config.c mod_mime.c mod_negotiation.c mod_status.c mod_include.c mod_autoindex.c mod_dir.c mod_cgi.c mod_asis.c mod_imap.c mod_actions.c mod_userdir.c mod_alias.c mod_access.c mod_auth.c mod_setenvif.c mod_perl.c suexec: disabled; invalid wrapper /usr/local/apache/bin/suexec There must be some way to build a debug version of apache+mod_perl with my desired set of modules statically bound into the httpd executable. Any suggestions would be much appreciated. Thx, -- Skip Montanaro ([EMAIL PROTECTED]) Support the Mojam.com Affiliates Program: http://www.mojam.com/affl/ (847)971-7098
Re: Files .. in virtualhosts in perl sections
Anyone have any experience doing Files sections inside of perlsections
virtualhosts.
I can't see anything in the docs about this.
Just wanted to ask before I start experimenting, as I don't have the first
clue about how this syntax might work.
Assuming I'm not missing something subtle about Files blocks vs. Location
blocks--I've done the latter, not the former--you can just assign your Files
key a hashref:
push @{ $VirtualHosts{$IP} }, {
Files = {
'/foo' = {
SetHandler = 'perl-script',
PerlHandler = 'Foo',
},
},
};
And so on. I use "push" instead of direct assignment because it allows for
multiple named virtual hosts.
bye,
Ben
Re: [RFC] Apache::Expires
I guess what I am really after is intercepting the If-Modified-Since tag and return a 304 prior to content generation - maybe in a fixup handler... This makes sense to me. The fixup handler has to decide if new content has to be delivered e.g. the db has changed or to return the 304 I think it is the entity tag stuff that is starting to throw me, too... As long as your browsers doesn't request with a If-Match or If-Non-Match header, you can simply forget about the Entity-Tag and as far as I know the entity tag isn't supported very widely by the bowsers. Gerald - Gerald Richterecos electronic communication services gmbh Internetconnect * Webserver/-design/-datenbanken * Consulting Post: Tulpenstrasse 5 D-55276 Dienheim b. Mainz E-Mail: [EMAIL PROTECTED] Voice:+49 6133 925151 WWW:http://www.ecos.de Fax: +49 6133 925152 -
Failure to find function in Apache::SSI using DBI
I installed Apache::SSI last night as I needed a quick way of including some
costing info in a page (from my dbase) that already used std SSI.
The install worked fine, my extension seems to be fine but when my code
runs, I get an unexpected error.
[Thu Nov 16 21:45:33 2000] [error] [Thu Nov 16 21:45:33 2000] null: Can't
locate object method "selectrow_array" via package "GodZoneSSI" at
/usr/local/godzone/modperl/GodZoneSSI.pm line 27, GEN0 chunk 1.
The DBI-connect would appear to work, but perl complains it cannot find the
$dbase-selectrow_array function in my code.
The code is pathetically small.
Startup script for apache
#! /usr/bin/perl
#
# Preload the packages used by the various .epl scripts.
#
# The main apache process will compile these into its name space and they
# will therefore be immediately available to the child apache processes.
#
BEGIN {
use Apache();
use lib ('/usr/local/godzone/modperl');
}
use strict;
# Check that the environment is appropriate
$ENV{ 'GATEWAY_INTERFACE' } =~ /^CGI-Perl/ ||
die("GATEWAY_INTERFACE is NOT Perl !!" );
# Common Packages
use Apache::Registry();
use Apache::Constants();
use CGI qw( -compile :all );
use CGI::Carp();
use DBI;
use Apache::DBI();
#
# That's all folks
#
1;
-
My SSI Module
-
#! /usr/bin/perl
#
#
# This module extends the base SSI functionality by adding GodZone
# specific directives.
#
#
# Modification History
# 16 Nov 2000 AGE First Written
#
package GodZoneSSI;
use DBI();
use Apache::SSI();
@ISA = qw( Apache::SSI );
#
# Resource Cost Lookup
#
# !--#ResCost zone=n code=nnn --
sub ssi_rescost {
my( $self, $oParm ) = @_;
my( $oDBase ) = DBI-connect( 'dbi:Pg:dbname=admin', '?', '??' );
my( $oDBase, $iResCode, $iZone ) = @_;
my( @iAmount )= $oDBase-selectrow_array( END_SQL );
SELECT rate_internal, rate_local, rate_national, rate_international
FROM resource
WHERE code = $oParm{ code }
END_SQL
my( $iAmt ) = $iAmount[ $oParm{ zone }];
my( $szAmt ) = $iAmt 0 ? sprintf( "(\$%.2f)", -( $amt/100 ))
: sprintf( "\$%.2f", ($amt/100));
1 while ( $szAmt =~ s/(\d)(\d\d\d)(?!\d)/$1,$2/);
$oDBase-disconnect();
return( $szAmt );
}
$ENV{ 'PATH' } = '/usr/bin:/bin';
#
# End of Module
#
--
Glen Eustace, Systems Engineer - Networking.
Information Technology Services, Turitea, Massey University, Palmerston
North, NZ.
Ph: +64 6 350 5799 x 2707, Fax: +64 6 350 5607
Failure to find function in Apache::SSI using DBI
I installed Apache::SSI last night as I needed a quick way of including some
costing info in a page (from my dbase) that already used std SSI.
The install worked fine, my extension seems to be fine but when my code
runs, I get an unexpected error.
[Thu Nov 16 21:45:33 2000] [error] [Thu Nov 16 21:45:33 2000] null: Can't
locate object method "selectrow_array" via package "GodZoneSSI" at
/usr/local/godzone/modperl/GodZoneSSI.pm line 27, GEN0 chunk 1.
The DBI-connect would appear to work, but perl complains it cannot find the
$dbase-selectrow_array function in my code.
The code is pathetically small.
Startup script for apache
#! /usr/bin/perl
#
# Preload the packages used by the various .epl scripts.
#
# The main apache process will compile these into its name space and they
# will therefore be immediately available to the child apache processes.
#
BEGIN {
use Apache();
use lib ('/usr/local/godzone/modperl');
}
use strict;
# Check that the environment is appropriate
$ENV{ 'GATEWAY_INTERFACE' } =~ /^CGI-Perl/ ||
die("GATEWAY_INTERFACE is NOT Perl !!" );
# Common Packages
use Apache::Registry();
use Apache::Constants();
use CGI qw( -compile :all );
use CGI::Carp();
use DBI;
use Apache::DBI();
#
# That's all folks
#
1;
-
My SSI Module
-
#! /usr/bin/perl
#
#
# This module extends the base SSI functionality by adding GodZone
# specific directives.
#
#
# Modification History
# 16 Nov 2000 AGE First Written
#
package GodZoneSSI;
use DBI();
use Apache::SSI();
@ISA = qw( Apache::SSI );
#
# Resource Cost Lookup
#
# !--#ResCost zone=n code=nnn --
sub ssi_rescost {
my( $self, $oParm ) = @_;
my( $oDBase ) = DBI-connect( 'dbi:Pg:dbname=admin', '?', '??' );
my( $oDBase, $iResCode, $iZone ) = @_;
my( @iAmount )= $oDBase-selectrow_array( END_SQL );
SELECT rate_internal, rate_local, rate_national, rate_international
FROM resource
WHERE code = $oParm{ code }
END_SQL
my( $iAmt ) = $iAmount[ $oParm{ zone }];
my( $szAmt ) = $iAmt 0 ? sprintf( "(\$%.2f)", -( $amt/100 ))
: sprintf( "\$%.2f", ($amt/100));
1 while ( $szAmt =~ s/(\d)(\d\d\d)(?!\d)/$1,$2/);
$oDBase-disconnect();
return( $szAmt );
}
$ENV{ 'PATH' } = '/usr/bin:/bin';
#
# End of Module
#
--
Glen Eustace, Systems Engineer - Networking.
Information Technology Services, Turitea, Massey University, Palmerston
North, NZ.
Ph: +64 6 350 5799 x 2707, Fax: +64 6 350 5607
Re: security suggestion
"Adam Prime" [EMAIL PROTECTED] wrote:
Maybe it's just me, but it seems that the responses richard has gotten
haven't really touched on the core of the problem. That mod_perl isn't
exactly friendly to sysadmin's who want to run apache on a (i'm guessing),
student accessed server, with user dir's and all that other stuff. I'm
pretty sure (for no particular reason), that there aren't many people on
this list that are doing that.
I assumed the question was related to the situation of an ISP (or more specifically
web hosting provider) who wants to offer some mod_perl functionality to users on a
shared server. the environment may be similar to the educational scenario, but the
difference is that a web presence provider that is "first" to offer mod_perl (at
prices competitive with oher shared hosting services) stands to differentiate itself
ffrom it's competition and see a signficant revenue reward for it's efforts, either
from being able to charge extra for the mod_perl add-on service, or simply as a swell
of users flock to signup for new service there.
It's an interesting problem in that context. offering mod_perl in such an environment
"safely" would be a big advantage to my hosting provider for instance (pair.com) who
*does* offer custom CGI, is extremely developer-friendly (supports perl, C, php and
python CGI) but has so many users sharing the same httpd that the mod_perl problems
are many and varied. we can't access .conf files or restart apache so we're limited
to .htaccess configuration.
There are so many active developers already that they run "reaper" processes on each
server to kill runaway scripts, and enforce resource limits on user processes.
If they could enforce these rlimits and automatically kill runaway code in users'
mod_perl processes, and *if* those user scripts could be isolated such that they can't
bring down the parent httpd process, mod_perl would be an awesome feature of the
service!
...If you have .htaccess stuff turned on right
now, you can do all sorts of great things through apache that you wouldn't
want untrusted accounts on the box being able to do.
right. it seems like raw perl in .htaccess files would almost *have* to be disabled.
or would it? bad/uncompilable code in .htaccess would just affect the requests that
invoked it, not the parent process...
as long as the server is running as an unpriveledged nobody-user or SetUID as the
~/user who is owns it, the security concerns should be minimal. if a mod_perl process
can attain root priveledges that would, uh, be a bad thing :-)
Of course, in such a restricted environment, many of mod_perl's advantages would be
nullified; scripts would not stay compiled in memory, and modules would be reloaded
on every request (since editing conf files or startup.pl or restarting apache isn't
possible). But hooking into the auth process alone would add a LOT of flexibility to
the many sites hosted on shared servers there.
plus there are endless support requests on "how to do this or that with mod_rewite
regexps"... (i agree with Randal here... how did that ever catch on? heheh).
rewriting and redirecting with mod_perl is *so* much simpler and more flexible.
The servers that had apache on them for users when i was at school didn't
even allow normal cgi, so i have no idea how one would approach doing
something like this with mod_perl.
right. many (lazy) commercial web hosting providers don't even (to this day) offer
perl CGI either. But most decent ones have realized that even the unwashed masses
need perl scripts to run, and have already successfully addressed ExecCGI security
issues in order to remain competitive.
pair has even recently begun to equip servers with mod_php... but i think mod_perl has
been deemed too powerful for use on shared servers, for now... {sigh}
-dave (secretly hoping that the mod_php scripters here crash the servers with endless
loops and stuff)
RE: Failure to find function in Apache::SSI using DBI
Derrr... Dumb error, never trust a cut and paste. I copied in one two many lines from the original program.
Re: Microperl
At 18:35 16/11/2000 +0100, Fabrice Scemama wrote: ok so what about miniperl, which is used when building perl? Is your question, what about embedding miniperl rather than microperl ? I don't know enough about the diffs to decide, and anyway I couldn't do the job myself. I was just supporting bill's idea as one that I think could be interesting, should anyone have tuits to spare on testing this out :) -- robin b. Don't panic.
Re: Re:coredump on Carps in startup.pl
PJ When I start up apache (apachectl startssl), I get a core dump. PJ Attached is an "strace httpd -X" for those of you who find this useful. FWIW, I have the same issue also. I noticed it when using CGI::Carp. My solution was not "use" CGI::Carp; in startup.pl (a real lame solution, I know ;-), but my goal was to deal with it later (I needed to get my development environment back on-line) You may find that this patch fixes things for you: http://forum.swarthmore.edu/epigone/modperl/sningvezerd/26091135.MAA1456 [EMAIL PROTECTED]
RE: [RFC] Apache::Expires
On Thu, 16 Nov 2000, Geoffrey Young wrote: it's the lack of a 304 that's bothering me (today :) If I just put your lines into a handler I get this from netscape: If-Modified-Since = Thu, 16 Nov 2000 12:48:04 GMT; length=1150 but, since there is no modification time for the 'document' to compare to locally, my headers out are: Expires = Thu, 16 Nov 2000 12:56:45 GMT Last-Modified = Thu, 16 Nov 2000 12:50:45 GMT with status 200 since the document was (again) generated (with new headers). Okay, so what you want is not to implement the headers but rather the If-Modified handler. My best advice is stay true to the laziness that led us to Perl in the first place and just set up mod_proxy. It will handle this for you quite nicely. If you really don't want to use a proxy server, the first step is to handle the If-Modified-Since part, and your proposal for intercepting those sounds fine. You'd need to keep a database mapping URLs to last-mod times and expiration times. You'd look up the incoming URL, send back the appropriate response it it's not modified and hasn't expired, and let it pass through in all other cases. Of course this is sort of lame because if it's really good until the time in your Expires header you should be caching it to hand out to other clients without re-generating. To do a real caching scheme with good performance, I would propose this: - Use URLs that map to real file names. - Create your content generation handler as a 404-handler which gets invoked when one of those files is not on disk. - When you generate a page, send it to the client and then write it to disk. - Keep a little database of expiration times, which a cron job can use to periodically delete expired pages. This is the best possible performance you can get from a caching scheme, since when things are cached they are served as static files and Apache worries about the If-Modified stuff for you. If you're paranoid about serving a page that has been expired for even a second, you'll have to do something more complex with a TransHandler that decides whether or not you can use the static file. This will be a bit slower. I think AxKit does something along these lines, and you could probably steal some code for it from there. - Perrin
Re: Microperl
I don't think another choice could hurt and potentially is cool. However, at the same time... I do know that when I go to the Microsoft World I get really annoyed at the fact that VBScript syntax means different things to different apps including IIS. And that it's so different from VB, I can't believe they have the nerve to say VB Programmers will pick up VBScript quickly. Anyway, viva la choice. If you want to try it, go ahead. There are obviously people who really want to use it. In fact, I suspect mod_backhand would really like it so that Apache::Backhand could run in a smaller footprint for writing candidacy functions (just don't run them in Florida -- sorry that was bad, I couldn't resist). Later, Gunther At 10:49 PM 11/16/00 +0100, Robin Berjon wrote: At 18:35 16/11/2000 +0100, Fabrice Scemama wrote: ok so what about miniperl, which is used when building perl? Is your question, what about embedding miniperl rather than microperl ? I don't know enough about the diffs to decide, and anyway I couldn't do the job myself. I was just supporting bill's idea as one that I think could be interesting, should anyone have tuits to spare on testing this out :) -- robin b. Don't panic. __ Gunther Birznieks ([EMAIL PROTECTED]) eXtropia - The Web Technology Company http://www.extropia.com/
Re: Microperl
At 08:53 17/11/2000 +0800, Gunther Birznieks wrote: However, at the same time... I do know that when I go to the Microsoft World I get really annoyed at the fact that VBScript syntax means different things to different apps including IIS. And that it's so different from VB, I can't believe they have the nerve to say VB Programmers will pick up VBScript quickly. I don't think this is really an issue. The small Perls have the same syntax, the main difference is that some things are missing. It should be fairly easy then to come up with a list of what one can't do. so that Apache::Backhand could run in a smaller footprint for writing candidacy functions (just don't run them in Florida -- sorry that was bad, I couldn't resist). Lol :) -- robin b. "What I like about deadlines is the lovely whooshing sound they make as they rush past." --Douglas Adams
Re: compiling modperl on alpha
On Thu, 16 Nov 2000, Didier Godefroy wrote: I'm trying to compile mod_perl as a dso with apxs on Alpha/Tru64 unix and there is an error from the linker: [snip] Can anyone give a clue? Make sure the ld that is being called is the exact same one that was used to build perl itself, e.g. by altering your PATH environment variable to point to either the vendor default first (under /usr/bin) or perhaps to the GNU ld that might have gotten installed somewhere under /freeware/bin or /usr/local/bin. ld flags are usually set during the compilation of perl using the Configure script; you can run `perl -V` to show what the flags are on the perl binary earliest in your PATH. Changing the ld binary used or ld flags generally involves a manual recompilation of all things perl. :-/ Also note that mod_perl as a DSO isn't recommended by the mod_perl documentation... -- Jeremy Mates http://www.sial.org/
Re: Replacing mod_prewrite with a PerlTransHandler
On 15 Nov 2000, David Hodgkinson wrote: Geoffrey Young [EMAIL PROTECTED] writes: Perhaps that would make for a good talk ;) mod_rewrite recovery? Ok seriously then, we're proposing replacing a lite apache and mod_rewrite with a slightly heavier, but presumably highly shared mod_perled apache at the front end. Measurements anyone? Perhaps, as others have suggested, mod_backhand makes this moot. If not, then I can tell you that on Linux, I was able to make a light-weight front-end that was 1/3 the RSS of a mod_perl backend without any modules loaded. Adding modules would make the back-end even heavier. -- -- Tom Mornini -- InfoMania Printing and Prepress
Re: compiling modperl on alpha
on 11/16/00 8:50 PM, Jeremy A. Mates at [EMAIL PROTECTED] uttered the following: Make sure the ld that is being called is the exact same one that was used to build perl itself, e.g. by altering your PATH environment variable to point to either the vendor default first (under /usr/bin) or perhaps to the GNU ld that might have gotten installed somewhere under /freeware/bin or /usr/local/bin. GNU ld isn't on either system and the error is the exact same on both, they seem to all this in common: -Wl,-rpath,/usr/local/lib/perl5/5.6.0/alpha-dec_osf/CORE' Configure script; you can run `perl -V` to show what the flags are on the perl binary earliest in your PATH. Changing the ld binary used or ld here it is: perl -V Summary of my perl5 (revision 5.0 version 6 subversion 0) configuration: Platform: osname=dec_osf, osvers=4.0, archname=alpha-dec_osf uname='osf1 ulysium.ulysium.net v4.0 564 alpha ' config_args='' hint=recommended, useposix=true, d_sigaction=define usethreads=undef use5005threads=undef useithreads=undef usemultiplicity=undef useperlio=undef d_sfio=undef uselargefiles=define use64bitint=define use64bitall=define uselongdouble=define usesocks=undef Compiler: cc='cc', optimize='-O4', gccversion= cppflags='-std -ieee -D_INTRINSICS -I/usr/local/include -DLANGUAGE_C' ccflags ='-std -fprm d -ieee -D_INTRINSICS -I/usr/local/include -DLANGUAGE_C' stdchar='unsigned char', d_stdstdio=define, usevfork=false intsize=4, longsize=8, ptrsize=8, doublesize=8 d_longlong=define, longlongsize=8, d_longdbl=define, longdblsize=8 ivtype='long', ivsize=8, nvtype='long double', nvsize=8, Off_t='off_t', lseeksize=8 alignbytes=8, usemymalloc=n, prototype=define Linker and Libraries: ld='ld', ldflags =' -L/usr/local/lib' libpth=/usr/local/lib /usr/shlib /usr/ccs/lib /usr/lib/cmplrs/cc /usr/lib /var/shlib libs=-lbind -ldbm -ldb -lm -liconv libc=/usr/shlib/libc.so, so=so, useshrplib=true, libperl=libperl.so Dynamic Linking: dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -Wl,-rpath,/usr/local/lib/perl5/5.6.0/alpha-dec_osf/CORE' cccdlflags=' ', lddlflags='-shared -expect_unresolved "*" -O4 -msym -std -s -L/usr/local/lib' Characteristics of this binary (from libperl): Compile-time options: USE_64_BIT_INT USE_64_BIT_ALL USE_LARGE_FILES Built under dec_osf At first when I tried building mod_perl, it complained about the large file support no being in apache, so I recompiled apache for that. Also note that mod_perl as a DSO isn't recommended by the mod_perl documentation... What is the problem with doing that? I tried compiling it statically first, but I get other types of errors when running the tests, it always fails. The httpd is running but the tests won't, and the log doesn't say anything that helps to find out why. What other choices are there then? -- Didier Godefroy mailto:[EMAIL PROTECTED]
Tempfile and send_fd()
Hi,
Could somebody tell me why the following testcase doesn't work?
use Apache ();
use Apache::File ();
my $r = Apache-request();
$r-content_type('text/plain');
$r-send_http_header();
my $f = Apache::File-tmpfile();
print $f "test\ntest\n";
$r-send_fd($f);
$f-close;
All I get is an empty document. My understanding is that the data
written to the tmpfile should be available immediately through the
filehandle even if it hasn't been flushed.
This is running under Registry, on Linux.
Thanks,
Steve
Re: compiling modperl on alpha
On Thu, 16 Nov 2000, Didier Godefroy wrote: GNU ld isn't on either system and the error is the exact same on both, they seem to all this in common: -Wl,-rpath,/usr/local/lib/perl5/5.6.0/alpha-dec_osf/CORE' [snip] dlsrc=dl_dlopen.xs, dlext=so, d_dlsymun=undef, ccdlflags=' -Wl,-rpath,/usr/local/lib/perl5/5.6.0/alpha-dec_osf/CORE' cccdlflags=' ', lddlflags='-shared -expect_unresolved "*" -O4 -msym -std -s -L/usr/local/lib' Hmmm... the Digitals I have share similar ccdlflags statements. Did you install perl 5.6.0 from scratch or via a package of some kind? (If via package, the people who built the package may have used a different ld or had different environment settings from yours.) You might be able to pass the flag ld appeared to be complaining about with the following trick: $ LDFLAGS="-_SYSTYPE_SVR4" command_that_fails Which should work for Bourne-related shells. The ld man page didn't appear contain anything useful, so I'm guessing moreso than usual at this point. :) You could also try asking on the tru64-unix-managers list: http://www.ornl.gov/cts/archives/mailing-lists/ mailto:[EMAIL PROTECTED]?body=subscribe What is the problem with doing that? I tried compiling it statically first, but I get other types of errors when running the tests, it always fails. The httpd is running but the tests won't, and the log doesn't say anything that helps to find out why. What other choices are there then? If static fails, then you can probably get away with DSO if you keep the httpd process memory usage down with various directives. There's more on this just recently in this list's archives, as I recall... -- Jeremy Mates http://www.sial.org/
Re:Tempfile and send_fd()
"Steve" == Steve Smith [EMAIL PROTECTED] writes: Hi, Could somebody tell me why the following testcase doesn't work? snip Nevermind, I got it from the archives eventually : seek $f, 0, 0; $r-send_fd($f); Cheers, Steve
Re: Tempfile and send_fd()
On Fri, Nov 17, 2000 at 02:42:43PM +1100, Steve Smith wrote:
Hi,
All I get is an empty document. My understanding is that the data
written to the tmpfile should be available immediately through the
filehandle even if it hasn't been flushed.
I wouldn't bet on it flushing: mod_perl may not have written it out yet.
perl often doesn't flush writes to disk until close. Try adding a
select( (select( $fh ), $| = 1 )[0] ) ;
just after the open and see if that fixes things.
Also, Apache doesn't seek or rewind the FD (see below), so even if it's flushed,
your file position may no longer be at the beginning of the file, depending
on your C RTL. I seem to recall that some require you to seek() between
reads and writes to ensure synchronization.
That lack of a rewind can be nice when you're caching things: you can sysread
some header lines, then turn the rest over to Apache to send.
- Barrie
API_EXPORT(long) ap_send_fd_length(FILE *f, request_rec *r, long length)
{
char buf[IOBUFSIZE];
long total_bytes_sent = 0;
register int n, w, o, len;
if (length == 0)
return 0;
ap_soft_timeout("send body", r);
while (!r-connection-aborted) {
if ((length 0) (total_bytes_sent + IOBUFSIZE) length)
len = length - total_bytes_sent;
else
len = IOBUFSIZE;
while ((n = fread(buf, sizeof(char), len, f)) 1
ferror(f) errno == EINTR !r-connection-aborted)
continue;
Re: Tempfile and send_fd()
On Fri, Nov 17, 2000 at 03:51:35PM +1100, Steve Smith wrote:
"Steve" == Steve Smith [EMAIL PROTECTED] writes:
Hi, Could somebody tell me why the following testcase doesn't work?
snip
Nevermind, I got it from the archives eventually :
seek $f, 0, 0;
Had a look in Apache::File (below), and it sysopens, so you might want
to sysseek(...) instead.
That seek is flushing the filehandle for you though, here's without the
seek (on Linux, YMMV):
strace perl -we 'use Fcntl ; sysopen F, "foo", O_RDWR | O_CREAT or die $! ; print F
"hi1\n" ; sleep 5 ; print F "hi2\n" ; close F'
[snip]
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) =
0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x400180
00
_llseek(0x3, 0, 0, 0xb67c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) =
0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
time([974438839]) = 974438839
SYS_175(0, 0xb8d0, 0xb850, 0x8, 0) = 0
SYS_174(0x11, 0, 0xb654, 0x8, 0x11) = 0
SYS_175(0x2, 0xb850, 0, 0x8, 0x2) = 0
nanosleep(0xb7bc, 0xb7bc, 0x401791b4, 0xb7bc, 0xb8d0) = 0
time([974438844]) = 974438844
write(3, "hi1\nhi2\n", 8) = 8
close(3)= 0
and here's with the seek():
strace perl -we 'use Fcntl ; sysopen F, "foo", O_RDWR | O_CREAT or die $! ; print F
"hi1\n" ; seek F, 0, 0 ; sleep 5 ; print F "hi2\n" ; close F'
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40018000
_llseek(0x3, 0, 0, 0xb66c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
write(3, "hi1\n", 4)= 4
_llseek(0x3, 0, 0, 0xb788, 0) = 0
time([974438924]) = 974438924
SYS_175(0, 0xb8c0, 0xb840, 0x8, 0) = 0
SYS_174(0x11, 0, 0xb644, 0x8, 0x11) = 0
SYS_175(0x2, 0xb840, 0, 0x8, 0x2) = 0
nanosleep(0xb7ac, 0xb7ac, 0x401791b4, 0xb7ac, 0xb8c0) = 0
time([974438929]) = 974438929
write(3, "hi2\n", 4)= 4
close(3)= 0
Calling syswrite bypasses Perl's buffering, but doesn't address the seek
issue:
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40018000
_llseek(0x3, 0, 0, 0xb66c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
write(3, "hi1\n", 4)= 4
time([974439066]) = 974439066
SYS_175(0, 0xb8c0, 0xb840, 0x8, 0) = 0
SYS_174(0x11, 0, 0xb644, 0x8, 0x11) = 0
SYS_175(0x2, 0xb840, 0, 0x8, 0x2) = 0
nanosleep(0xb7ac, 0xb7ac, 0x401791b4, 0xb7ac, 0xb8c0) = 0
time([974439071]) = 974439071
write(3, "hi2\n", 4)= 4
close(3)= 0
- Barrie
sub tmpfile {
my $class = shift;
my $limit = 100;
my $r = Apache-request;
while($limit--) {
my $tmpfile = "$TMPDIR/${$}" . $TMPNAM++;
my $fh = $class-new;
sysopen($fh, $tmpfile, $Mode, $Perms);
$r-register_cleanup(sub { unlink $tmpfile }) if $r;
if($fh) {
return wantarray ? ($tmpfile,$fh) : $fh;
}
}
}
Re: Tempfile and send_fd()
On Fri, Nov 17, 2000 at 12:32:33AM -0500, barries wrote:
On Fri, Nov 17, 2000 at 03:51:35PM +1100, Steve Smith wrote:
seek $f, 0, 0;
Had a look in Apache::File (below), and it sysopens, so you might want
to sysseek(...) instead.
No, nevermind, don't: sysseek won't flush the buffer:
strace perl -we 'use Fcntl ; sysopen F, "foo", O_RDWR | O_CREAT or die $! ; print F
"hi1\n" ; sysseek F, 0, 0 ; sleep 5 ; print F "hi2\n" ; close F'
open("foo", O_RDWR|O_CREAT|0x8000, 0666) = 3
fcntl(3, F_GETFL) = 0x8002 (flags O_RDWR|0x8000)
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
mmap(0, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x40018000
_llseek(0x3, 0, 0, 0xb66c, 0x1) = 0
fstat(3, {st_mode=S_IFCHR|S_ISUID|S_ISGID|0463, st_rdev=makedev(79, 18), ...}) = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
_llseek(0x3, 0, 0, 0xb8f8, 0) = 0
time([974439311]) = 974439311
SYS_175(0, 0xb8c0, 0xb840, 0x8, 0) = 0
SYS_174(0x11, 0, 0xb644, 0x8, 0x11) = 0
SYS_175(0x2, 0xb840, 0, 0x8, 0x2) = 0
nanosleep(0xb7ac, 0xb7ac, 0x401791b4, 0xb7ac, 0xb8c0) = 0
time([974439316]) = 974439316
write(3, "hi1\nhi2\n", 8) = 8
close(3)= 0
Re: [RFC] Apache::ProxyRewrite
On Tue, 14 Nov 2000 [EMAIL PROTECTED] wrote:
Please see http://www.apache.org/~ask/junk.txt
- ask
I will soon need something similar to this myself.
In my case, it will be necessary to authenticate on a user by user basis.
It would be good to extend this module to cope with this eventuality, with
pluggable backends to retrieve the passwords (I use LDAP but an
abstraction layer would be most flexible).
It may also be necessary to handle cookie based authentication/session
handling from the backend.
OK - so I know there are lots of issues to overcome. That's one reason why
I haven't attempted it yet ;-)
But functionality like this will allow me to proxy my users onto external
services without them worrying about multiple passwords, as remembering
just one seems to be beyond some of them !, and without me having to send
the "master" password out to third parties in plain text.
Hopefully I will be able to contribute some code back if the problems have
not already been solved by the time I need to implement a solution.
HTH,
Simon.
From "Christian Gilmore" [EMAIL PROTECTED]
Date 14 November 2000
To
"Modperl Mailing List (E-mail)" Time 16:54
[EMAIL PROTECTED]
Copy to (bcc: Simon Wilcox/BASE/WilliamsLea)
Bcc Simon Wilcox/BASE/WilliamsLea
Fax to
Subject [RFC] Apache::ProxyRewrite
I've completed work on a proxying module we needed here at work. I intend
to release it to the community, but first I want to get comments on its
current name and design. Perhaps there is a direction for it to grow
before initial release?
The Problem I Needed to Solve:
We need to proxy our external web services, but secure and insecure, to
our internal personnel while also doing authentication on the personnel's
behalf behind the scenes. In order to minimize muddying of customer data,
only a single "group" userid exists. This userid is to be used for the
purpose of authenticating and authorizing internal personnel to certain
areas of our external site.
The Solution:
Apache::ProxyRewrite will proxy content, rewriting arbitrary URLs embedded
in the content (if HTML) per run-time configuration. A configuration
example for the host www-internal.tivoli.com:
Location /
SetHandler perl-script
PerlHandlerProxyRewrite
RProxyTo http://www.tivoli.com
RProxyAuthInfo "BASIC dG32cvVwcnQ6amF4MzhfYXS="
RProxyAuthRedirect On
RProxyRewrite https://www.tivoli.com/secure /secure
/Location
Location /secure
SetHandler perl-script
PerlHandlerProxyRewrite
RProxyTo https://www.tivoli.com/secure
RProxyAuthInfo "BASIC dG32cvVwcnQ6amF4MzhfYXS="
RProxyAuthRedirect On
RProxyRewrite http://www.tivoli.com//
RProxyRewrite http://foo.bar.com/ /secure/foo
/Location
Requests for "/" will first be proxied to http://www.tivoli.com. The
content at the URL will be parsed (quickly via a single pass through the
code, not with HTML::Parser and its variants). There will be an implicit
rule that references to relative path of the argument to RProxyTo ("/" in
this case) in the document will be rewritten to the relative URI in the
current Location (also "/" in this case). Further, references to
https://www.tivoli.com/secure on the backend will be rewritten to /secure.
The RProxyAuthInfo directive allows for automatic authentication and
authorization for a predetermined userid. The RProxyAuthRedirect directive
allows the server to receive backend 401 responses and redirect the client
directly to that backend URI. I don't anticipate this directive having
much value to the general community, but it was a requirement of our
installation.
Please send comments, questions, flames (hopefully none of these!) back to
the list. I attempted to contact the owner of the Apache::RewritingProxy
package to no avail. His package, though, seems designed to rewrite
content, not URIs, so I think there's room for both.
Thanks,
Christian
-
Christian Gilmore
Infrastructure Tools Team Lead
Web Multimedia Development
Tivoli Systems, Inc.
__
This email contains proprietary information some or all of which may be
legally privileged. It is for the intended
Re: [RFC] Apache::ProxyRewrite
On Thu, 16 Nov 2000, Ask Bjoern Hansen wrote: On Tue, 14 Nov 2000 [EMAIL PROTECTED] wrote: Please see http://www.apache.org/~ask/junk.txt ah, shoot. My low cluerate beats everything. I obviously only meant to send that to Simon. Sorry. - ask -- ask bjoern hansen - http://www.netcetera.dk/~ask/ more than 70M impressions per day, http://valueclick.com - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: database access
- Original Message - From: "Gunther Birznieks" [EMAIL PROTECTED] To: "Les Mikesell" [EMAIL PROTECTED]; [EMAIL PROTECTED] Cc: "Tim Bunce" [EMAIL PROTECTED]; "Aaron" [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Thursday, November 16, 2000 3:04 AM Subject: Re: database access 1. I don't see the scenario you are talking about (dynamic connection pooling) actually working too well in practice because with most web sites there is usually peak times and non-peak times. Your database still has to handle the peak times and keep the connection open, so why not just leave the connection open during non-peak. It doesn't seem like it would really matter. Do you have a real scenario that this is a concern in? This is not so much a problem where you have one or two databases on a production server that is used by most pages - you are right that you just have to be able to handle those connections. The problem I see is on an internal intranet server where there are lots of little special-purpose databases - separate calendar, forums, and assorted applications, each with its own database and usage pattern. If these all get used at once, the backend database server accumulates a lot of connections. At the moment I don't have run-time user/password based connections per user, but that would be an even bigger problem. 2. I do see adding a regex on the connect string to explicitly stop Apache::DBI from caching the connection being valuable. That would probably be enough - or the other way around to specify the one(s) you know should be cached. 3. As a plug, I had also suggested a couple years ago that I would like the feature to be able to specifically not having the ping() method by Apache::DBI if the database had already been pinged within a set period of time. That would fall into place if it timestamped the usage per handle and also did a real disconnect on any that hadn't been used recently. Les Mikesell [EMAIL PROTECTED]
Re: database access
1. I don't see the scenario you are talking about (dynamic connection pooling) actually working too well in practice because with most web sites there is usually peak times and non-peak times. Your database still has to handle the peak times and keep the connection open, so why not just leave the connection open during non-peak. It doesn't seem like it would really matter. Do you have a real scenario that this is a concern in? 2. I do see adding a regex on the connect string to explicitly stop Apache::DBI from caching the connection being valuable. 3. As a plug, I had also suggested a couple years ago that I would like the feature to be able to specifically not having the ping() method by Apache::DBI if the database had already been pinged within a set period of time. I think the conclusion to #3 was that it would be good to add, but that I should add it. Unfortunately, I don't run a huge site on mod_perl so I didn't really care about eeking out performance this way (I think it was off the thread that the ping() method was very heavyweight on DBD::Sybase). Thus, I never added it and I don't know if anyone else ever was motivated to either. Later, Gunther At 07:53 PM 11/15/2000 -0600, Les Mikesell wrote: - Original Message - From: [EMAIL PROTECTED] To: "Les Mikesell" [EMAIL PROTECTED] Cc: "Tim Bunce" [EMAIL PROTECTED]; "Aaron" [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Wednesday, November 15, 2000 2:21 AM Subject: Re: database access On Tue, 14 Nov 2000, Les Mikesell wrote: I wonder if Apache::DBI could figure out what connections are worth holding open by itself? It could have some hints in the config file like regexps to match plus a setting that would tell it not to make a connection persist unless it had seen x instances of that connect string in y seconds. That really, really sucks, but Apache is selecting on the HTTP socket, and nothing matters beyond that, except signals of course. What you are implying is that DBI will be aware of the connections being closed or SIGALRM coming thru to the apache and on its lap, but it won't happen. No, I realize there is nothing to do after the fact - what I meant was that Apache::DBI might allow disconnect to really happen the first few times it sees a connect string after a child startup. If it saved the string with a timestamp and waited until it had seen the same string several times within a short interval it would be fairly likely that it would be worth staying connected. You'd trade some slower hits off against not accumulating a huge number of little-used database connections. Les Mikesell [EMAIL PROTECTED] __ Gunther Birznieks ([EMAIL PROTECTED]) eXtropia - The Web Technology Company http://www.extropia.com/
cvs commit: modperl-site/embperl Changes.pod.1.html Changes.pod.10.html Changes.pod.11.html Changes.pod.12.html Changes.pod.13.html Changes.pod.14.html Changes.pod.15.html Changes.pod.16.html Changes.pod.17.html Changes.pod.18.html Changes.pod.19.html Changes.pod.2.html Changes.pod.20.html Changes.pod.21.html Changes.pod.22.html Changes.pod.23.html Changes.pod.24.html Changes.pod.25.html Changes.pod.26.html Changes.pod.27.html Changes.pod.28.html Changes.pod.29.html Changes.pod.3.html Changes.pod.30.html Changes.pod.31.html Changes.pod.32.html Changes.pod.33.html Changes.pod.34.html Changes.pod.35.html Changes.pod.36.html Changes.pod.37.html Changes.pod.38.html Changes.pod.39.html Changes.pod.4.html Changes.pod.40.html Changes.pod.41.html Changes.pod.42.html Changes.pod.43.html Changes.pod.5.html Changes.pod.6.html Changes.pod.7.html Changes.pod.8.html Changes.pod.9.html Changes.pod.cont.html
richter 00/11/16 21:31:19 Modified:embperl Changes.pod.1.html Changes.pod.10.html Changes.pod.11.html Changes.pod.12.html Changes.pod.13.html Changes.pod.14.html Changes.pod.15.html Changes.pod.16.html Changes.pod.17.html Changes.pod.18.html Changes.pod.19.html Changes.pod.2.html Changes.pod.20.html Changes.pod.21.html Changes.pod.22.html Changes.pod.23.html Changes.pod.24.html Changes.pod.25.html Changes.pod.26.html Changes.pod.27.html Changes.pod.28.html Changes.pod.29.html Changes.pod.3.html Changes.pod.30.html Changes.pod.31.html Changes.pod.32.html Changes.pod.33.html Changes.pod.34.html Changes.pod.35.html Changes.pod.36.html Changes.pod.37.html Changes.pod.38.html Changes.pod.39.html Changes.pod.4.html Changes.pod.40.html Changes.pod.41.html Changes.pod.42.html Changes.pod.43.html Changes.pod.5.html Changes.pod.6.html Changes.pod.7.html Changes.pod.8.html Changes.pod.9.html Changes.pod.cont.html Log: Embperl Webpages - Changes Revision ChangesPath 1.187 +13 -41modperl-site/embperl/Changes.pod.1.html Index: Changes.pod.1.html === RCS file: /home/cvs/modperl-site/embperl/Changes.pod.1.html,v retrieving revision 1.186 retrieving revision 1.187 diff -u -r1.186 -r1.187 --- Changes.pod.1.html2000/11/15 08:50:58 1.186 +++ Changes.pod.1.html2000/11/17 05:31:16 1.187 @@ -1,6 +1,6 @@ HTML HEAD -TITLE1.3b7 15. Nov 2000/TITLE +TITLE1.3b8_dev -- That's what currently under developement/TITLE LINK REV="made" HREF="mailto:[EMAIL PROTECTED]" /HEAD @@ -11,7 +11,7 @@ tr td valign=bottom align=center font size=6strong -A NAME="1_3b7_BETA_15_Nov_2000"1.3b7 (BETA) 15. Nov 2000/a/strong/font +A NAME="1_3b8_dev_That_s_what_current"1.3b8_dev -- That's what currently under developement/a/strong/font /tdtd rowspan=2 align=right/td /trtrtd valign=top align=center @@ -19,47 +19,19 @@ img src="line.jpg" alt="" WIDTH="732" HEIGHT="35" nbsp;nbsp;nbsp;nbsp;/td/tr/table -[a href="" HOME/a]nbsp;nbsp; [a href="Changes.pod.cont.html"CONTENT/a]nbsp;nbsp; [a href="Changes.pod.cont.html"PREV (Revision History - Content)/a]nbsp;nbsp; [a href="Changes.pod.2.html"NEXT (1.3b6 (BETA) 18. Oct 2000)/a]nbsp;nbsp; brhr +[a href="" HOME/a]nbsp;nbsp; [a href="Changes.pod.cont.html"CONTENT/a]nbsp;nbsp; [a href="Changes.pod.cont.html"PREV (Revision History - Content)/a]nbsp;nbsp; [a href="Changes.pod.2.html"NEXT (1.3b7 (BETA) 15. Nov 2000)/a]nbsp;nbsp; brhr P -PRE - Fixed problem that call to close triggers error in Session.pm when using Store::File. - Spotted by Alvar Freude. - - Fixed problem that GetSession doesn't return a value for new session. - Spotted by Angus Lees. - - Added new function SetSessionCookie, which can be used to set the session cookie - header in case you use Embperl session management, but don't use a - Embperl page in that request. Suggested by Alex Schmelkin. - - Fixed problem that DeleteSession does not work outside the base template, when - used with EmbperlObject. Spotted by Angus Lees. - - Fixed problem when use multiple nested ../foo.htm inside EmbperlObject. - Spotted by Neil Gunton. - - Included patch from Angus Lees to ease building on debian. - - make test now works again with BEN-SSL httpsd. With help from Neil Gunton. - - Included grammaticaly updated Embperl.pod. Many thanks to Ilia Lobsanov - for doing the proof reading. - - removed dbgDisableCache to avoid problems with this quot;featurequot; that isn't working - anymore for a long time. - - Added validaten for session cookie. If Apache::Session 1.53+ is installed - it uses the validate method from Apache::Session::Generate::xxx. If a invalid - session id is found a new one is generated. Spotted by Angus Lees. - - If a not existing session id is received, Embperl generates now a new one. - - Enhancements of Cookie resending logic. To make sure cookies are send when - neccessary, but not more often. (Handles now write to session data, after - a delete in the same request correctly). - - Added more tests for Sessionhandling. - - Fixed a bug that Content-Lenght was 2 to much, when escmode was set to zero. - Spotted by Michael Smith. - - Added patches from Randy Korbes for ActiveState Support.
