Re: Breaks in mod_perl, works in Perl
Mark Hazen wrote: Hmm, then create a ramdisk and read from the file virtually stored in the RAM. Stas, This is an elegant solution that I had not thought of. My problem is that I can't get ramdisks to work on my Red Hat 6.2 with 2.4.9 machine. But that's really my problem, and you've all been a big help. Have a look at TMPFS. It creates a RAM-based filesystem that is more flexible than a RAM disk. Add to your /etc/fstab: none /tmpfs tmpfs defaults,noatime,size=200M 0 0 ...or something similar, and away you go!
Re: [OT] email attachments; was modperl growth
Jonathan M. Hollin wrote: Er, that's not strictly true. Outlook handles encrypted and/or signed email as well as any other client. Outlook displays the signed email with a unique icon to identify it as such. The attachment contains the actual PGP info (in case you want to see it). I think that's fair enough isn't it? I don't know about Outlook Express though (which is a completely different mailer). Outlook Express handles S/MIME, but OpenPGP multipart/signed messages confuse it.
Re: mod_deflate problem with chunked encoding
Geoffrey Young wrote: Philip Mak wrote: The following webpage on a mod_deflate enabled server is not working correctly in some browsers: ... a few of us have been wondering where all the traffic on these modules has been coming from, is all - I thought it might genuinely be some mis-documentation or something... I originally introduced these modules to this list in an earlier post titled Report on mod_accel and mod_deflate in which I promoted these modules as an excellent tool for creating a front-end accelerator to mod_perl web applications. It included brief documentation for installing and configuring the modules--previously there had only been Russian documentation. There's been no other location for English language discussion of these modules. So the discussion has ended up here. I take full responsibility for any OT pollution as a result. ;-) Given that these modules are new, don't have an active discussion home in English yet, and are very relevent to mod_perl programmers, I'd like to see discussion remain here, or at least have Igor post the occassional update on new releases etc. If that bothers anyone then I'm happy to set up a mailing list elsewhere, of course.
mod_accel/mod_deflate update
mod_accel and mod_deflate are the modules I discussed a couple of weeks ago to allow you to create an efficient HTTP accelerator front-end to your mod_perl servers. I incorrectly reported 2 days ago that mod_deflate does not work with Mozilla-based browsers. It turns out that the problem is nothing to do with mod_deflate, but is actually the combination of mod_accel and mod_ssl and how Mozilla handles keep-alives. Igor Sysoev has done a marvellous job of diagnosing this problem, and has identified the workaround that in your SSL virtual host section you need to add: SetEnvIf User-Agent .*Gecko.* \ nokeepalive ssl-unclean-shutdown \ downgrade-1.0 force-response-1.0 This is not necessary with mod_proxy HTTP accelerators because mod_proxy always uses HTTP/1.0.
mod_deflate warning
I posted a couple of weeks ago about how well the mod_deflate/mod_accel combination works on a front-end HTTP accelerator. I've just discovered a problem that I thought I'd mention here for anyone who's trying this out. It appears that Mozilla-based browsers fail when using gzip encoding with POST requests. I've added DeflateDisableRange MSIE 4. Mozilla to my httpd.conf as a workaround for this bug. The bug appears intermittently, and I haven't yet worked out exactly how to replicate it. However, it's been reported by many of my users and I have experienced it myself, so I'd suggest being wary of this combination.
Re: mod_accel reverse proxying?
Philip Mak wrote: On Fri, 28 Dec 2001, Igor Sysoev wrote: I think it should have reverse syntax: AccelReverse http://127.0.0.1:8001/ / Or not ? Of course it complicates porting from mod_proxy to mod_accel but I think it's clearer then ProxyPassReverse syntax. I don't think either order is more clearer than the other, but since ProxyPassReverse has it like / http://127.0.0.1:8001/, my personal opinion is that AccelPassReverse should have it in the same order too to avoid confusion. On the other hand, I find mod_proxy's syntax weird and support Igor's proposal. Still, either one would be very handy!
Re: Report on mod_accel and mod_deflate
Philip Mak wrote: In ftp://ftp.lexa.ru/pub/apache-rus/contrib/ (where I have been told to download mod_accel/mod_deflate from before), I see another file called mod_realip-1.0.tar.gz just released one week ago. From looking at the keywords in the documentation, it looks like a module to be installed on the backend httpd that will parse these X-* headers to retrieve the original IP address. Correct. This is a simple module that fills out $R-connection-ip with the X- header added by mod_deflate.
Report on mod_accel and mod_deflate
Igor Sysoev mentioned recently on this list that he has written a module
called 'mod_accel' that provides a caching HTTP accelerator, as well as a
mod_gzip replacement called 'mod_deflate'. These modules are both used on
Kaspersky labs' busy sites, as well as at the popular portal
http://www.rambler.ru. mod_deflate is used at around 700 sites.
So why haven't we heard more of them? Because the docs are only in Russian,
and almost all implementations have occured on Russian sites. Well, the good
news is that I've now implemented a mod_accel/mod_deflate front-end on
http://www.fastmail.fm, allowing me to announce a substantial speedup to our
users:
http://www.emailaddresses.com/forum/showthread.php?threadid=1603
You may notice in the linked announcement that I mention I've been able to
remove the restriction that attachments have to be downloaded within an
hour. We previously had this restriction because even with mod_proxy as a
front-end, uploads larger than the TCP buffer get fed through bit by bit,
keeping the mod_perl handler open the whole time... So we had code like:
|| my $oldalarm=alarm(0);
|| alarm 3600;
|| my $Status = $R-parse;
|| alarm $oldalarm;
Without that, the backend app server can hang around for ever serving one
client. We had similar code for downloads, as well as uploads. BTW, Joe
Schaefer has posted a mod_proxy patch to this list that fixes this for POST
uploads.
Well, mod_accel gets around this problem by automatically buffering large
uploads and downloads onto the filesystem! :-) In fact, mod_accel reads the
backend response and sends it to client asynchronously, so the client sees
the first part of the response even before the backend has completed the
request. But the backend is never blocked by the client, since it is
buffered to the file-system as required.
mod_deflate works really well with mod_accel so that both dynamic forwarded
requests and static requests are compressed. mod_deflate also has lots of
intelligence to avoid compressing content to buggy browsers that can't
handle it.
OK, enough evangelism. On to practical matters. Many of the following
snippets and comments were provided to me by Igor in private email, so thank
him, not me. Firstly, here's how to compile everything (including mod_ssl on
the front-end) and install it into /usr/local/apache_accel:
tar -zxvf mod_accel-1.0.7.tar.gz
tar -zxvf mod_deflate-1.0.9.tar.gz
tar -zxvf mm-1.1.3.tar.gz
cd mm-1.1.3
./configure --disable-shared
make
cd ../mod_ssl-2.8.5-1.3.22
./configure --with-apache=../apache_1.3.22_accel/ \
--with-crt=/INSERT_APPROPRIATE_PATH.crt \
--with-key=/INSERT_APPROPRIATE_PATH.key
cd ../mod_accel-1.0.7
./configure --with-apache=../apache_1.3.22_accel/ \
--with-eapi=../mod_ssl-2.8.5-1.3.22/pkg.eapi/
make
cd ../mod_deflate-1.0.9
./configure --with-apache=../apache_1.3.22_accel/
make
cd ../apache_1.3.22_accel/
EAPI_MM=../mm-1.1.3 SSLBASE=SYSTEM ./configure --enable-rule=EAPI \
--prefix=/usr/local/apache_accel --disable-rule=EXPAT \
--enable-module=most \
--enable-module=rewrite \
--activate-module=src/modules/accel/libaccel.a \
--activate-module=src/modules/extra/mod_deflate.o
make
make install
mkdir /var/accelcache
chown nobody:nobody /var/accelcache/
/usr/local/apache_accel/bin/apachectl startssl
Now, in /usr/local/apache_accel/conf/httpd.conf, append the following:
AccelCacheRoot /var/accelcache 1
AccelNoCacheon
AccelPass/mail/http://127.0.0.1:8080/mail/
AccelWaitBeforeBodyRead 100
AccelUnlinkNoCached off
AccelSetXHost on
AccelSetXRealIP on
AccelSetXURL on
This creates an HTTP accelerator without any caching. I don't know much
about the caching directives and haven't tried them out, so I can't provide
any guidance there.
The AccelCacheRoot directive sets the number of levels of subdirectories in
the cache. 16 first level subdirectories named 0 .. f and 256 second level
subdirectories named 00 .. ff are used by default (i.e. AccelCacheRoot
path 1 2). Since I'm not caching, my config above does not specify any
directory hashing (i.e. AccelCacheRoot path 1).
AccelNoCache simply turns off caching.
AccelPass specifies that I want everything under /mail/ forwarded to the
backend. You can also specify '/' for AccelPass, and then use something like
AccelNoPass /download /images ~*\.jpg$ to create exceptions (uses
standard regex syntax).
Note that mod_accel can also be called by utilising the mod_rewrite [P]
directive, just like with mod_proxy.
AccelWaitBeforeBodyRead sets the time in milliseconds to wait between
receiving the response header and the first part of the body. It is useful
over Ethernet where packets are small. If mod_accel waits a little it can
read most of the body or even the whole body with one just one
select()/read() pair of syscalls. You can see the first read size and the
number of reads in the log with %{accel}x in a CustomLog directive. Igor
usually uses 100-200
Re: Comparison of different caching schemes
Rob Mueller (fastmail) wrote: And ++ on Paul's comments about Devel::DProf and other profilers. Ditto again. I've been using Apache::DProf recently and it's been great at tracking down exactly where time is spent in my program. One place that Rob and I still haven't found a good solution for profiling is trying to work out whether we should be focussing on optimising our mod_perl code, or our IMAP config, or our MySQL DB, or our SMTP setup, or our daemons' code, or... Can anyone suggest a way (under Linux 2.4, if it's OS dependent) to get a log of CPU (and also IO preferably) usage by process name over some period of time? Sorry to move slightly off-topic here, but in optimising our mod_perl app we have to work out which bit of the whole system to prioritise.
Re: Comparison of different caching schemes
Perrin Harkins wrote: Also, I'd like to see MLDBM + BerkeleyDB (not DB_File) with BerkeleyDB doing automatic locking, and IPC::MM, and IPC::Shareable, and IPC::ShareLite (though it doesn't serialize complex data by itself), and MySQL with standard tables. Of course I could just do them myself, since you were kind enough to provide code. IPC::ShareLite freezes/thaws the whole data structure, rather than just the hash element being accessed, IIRC, so is probably going to have extremely poor scaling characteristics. Worth adding to check, of course. MLDBM+BDB without using MLDBM::Sync is a cool idea that I hadn't thought of. That will certainly be interesting. Another interesting option is mapping a MySQL table data structure directly to the data structure being stored. This can be faster because you don't have to serialise, and also don't have to update the whole structure, just the fields that change (which is common in session handling and similar). Unfortunately this is hard to include in this benchmark, because the benchmark creates random complex structures. Maybe we'll create a custom one for this option for the sake of reference, even although it won't be directly comparable. I'm not sure what a 'standard table' in MySQL is any more... Berkeley, MyISAM, ISAM... I guess we can try all these, but that's benchmarking the DB rather than the caching scheme, and we're not about to try every DB server we can find!
Re: Locating infinite loops
Matt Sergeant asked:
I have an error on my server that I think is caused by an infinite loop in
perl code [*]. Does anyone have a reliable way to detect where this is
happening on a server with lots of code?
$SIG{ALRM} = sub {
Carp::confess(Got Apache::Event ALRM);
};
alarm(300);
Replacing 300 with some amount which any process should finish in.
Not elegent, but it seems to work. For instance, this:
use Carp;
$SIG{ALRM} = sub {
Carp::confess(Got Apache::Event ALRM);
};
alarm(3);
temp();
sub temp {
sleep(5);
}
...gives this result:
Got Apache::Event ALRM at testal.pl line 4
main::__ANON__('ALRM') called at testal.pl line 9
main::temp() called at testal.pl line 7
...which even shows which line Perl was up to when the SIGALRM was called
(line 9 in this case).
Re: [OT] Excellent article on Apache/mod_perl at eToys
By the way, I noticed he documented still another gotcha due to a sneaky closure. Does anyone know off-hand whether the Perl 6 folks plan to change the closure syntax so they don't sneak into your code this way? Closures, references, et al are being thoroughly revised, such that there will be far fewer gotchas. However, these kinds of memory leaks will be less of an issue anyway because Perl 6 will be garbage collected. Also, proper exception handling will be built into the language. PS: I haven't seen many of the regulars here on the Perl 6 lists. There's both language design and internals streams currently happening, so everyone's interests and skills should be catered for. Please have a look at http://dev.perl.org and consider getting involved. It's interesting work and there's a similarly positive and intellectual spirit on the P6 lists to what we experience here.
Re: Catching user-presses-stop early
Stas Bekman wrote:
Gunther Birznieks wrote:
Would sending a null byte work with a reverse proxy method of mod_perl
if the reverse proxy caches and doesn't deliver the data right away? I
don't know if there is a way to control this or what the behavior is.
The technique in the guide doesn't work with apache as a reverse proxy,
only with direct connection. I didn't try with squid though.
Anyone know of a technique that does work through mod_proxy (since that's
the system that we're moving to).
As an aside, why not just send whitespace instead of a nullbyte? It's
supposed to be ignored anyway unless you are sending binary data.
At 06:35 AM 10/13/2001, Jeremy Howard wrote:
Our site is suddenly getting to the point where resource constraints
are
becoming an issue for the first time. So, apologies in advance if I
have
lots of optimization-related questions over the next couple of weeks...
One thing I would like to catch is the related problems of:
- Users pressing stop in the middle of a long process
- Users double-clicking rather than single-clicking links
(resulting in 2 processes handling the request)
I've read the 'user-pressed-stop' bit of the Guide (thanks Stas!) and
although I think I understand the issues now, I'm looking for practical
advice on approaches people use. I'm thinking that I should try writing
a
null byte to the client:
- At the beginning of the handler
- Every few iterations of potentially long loops
- Before creating large files to send to the client.
If I do this then Apache should receive the signal that the client has
gone
away, and should terminate the process happily (as long as I clean up
properly, of course).
So, does this sound like the right approach? Any way to simplify this
to
avoid explicitly adding print-a-null-byte lines to every loop? I'm
wondering
whether it's possible to add an alarm() and SIG{ALRM} handler or
something
like that, which sends a null byte every second or 2. Anyone using
something
like this successfully? Any potential problems from sending a null
byte to a
client?
__
Gunther Birznieks ([EMAIL PROTECTED])
eXtropia - The Open Web Technology Company
http://www.eXtropia.com/
--
_
Stas Bekman JAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide http://perl.apache.org/guide
mailto:[EMAIL PROTECTED] http://ticketmaster.com http://apacheweek.com
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/
Apache 1.3.22 is out
I just noticed that Apache 1.3.22 is out. Details here: http://www.apacheweek.com/issues/01-10-12 3 security patches included, plus enhancements to ab, mod_auth (now understands FS permissions), and the manual, plus numerous other bits and pieces.
Catching user-presses-stop early
Our site is suddenly getting to the point where resource constraints are
becoming an issue for the first time. So, apologies in advance if I have
lots of optimization-related questions over the next couple of weeks...
One thing I would like to catch is the related problems of:
- Users pressing stop in the middle of a long process
- Users double-clicking rather than single-clicking links
(resulting in 2 processes handling the request)
I've read the 'user-pressed-stop' bit of the Guide (thanks Stas!) and
although I think I understand the issues now, I'm looking for practical
advice on approaches people use. I'm thinking that I should try writing a
null byte to the client:
- At the beginning of the handler
- Every few iterations of potentially long loops
- Before creating large files to send to the client.
If I do this then Apache should receive the signal that the client has gone
away, and should terminate the process happily (as long as I clean up
properly, of course).
So, does this sound like the right approach? Any way to simplify this to
avoid explicitly adding print-a-null-byte lines to every loop? I'm wondering
whether it's possible to add an alarm() and SIG{ALRM} handler or something
like that, which sends a null byte every second or 2. Anyone using something
like this successfully? Any potential problems from sending a null byte to a
client?
Re: Apache::Hello
Ray and Lara Recendez wrote: I am trying to get Apache::Hello to work. ... #!/usr/local/bin/perl Turn PerlWarn On as described here: http://perl.apache.org/src/mod_perl.html#SWITCHES Also try running Apache in single-user mode: /usr/local/apache/bin/apachectl -X This helps you pick up problems in simple scripts.
Re: keeping perl in memory
Miroslav Madzarevic wrote: I don't wan't anything related to Apache (no Apache, not even Apache::Registry or anything else, I'm a mod perl, Mason programmer so I'm familiar with them). I just wanted to have perl binary persistent in memory (_like_ in mod_perl) so that perl doesn't have to be loaded #!/usr/local/bin/perl on each _shell_ request and there could be many of requests (once per email). http://sourceforge.net/projects/speedycgi/ If there is another list to wich I should direct questions like these please direct me. The newsgroup comp.lang.perl.misc is most appropriate forum for general Perl questions. Also, comp.lang.perl.modules is for discussing use of Perl modules. If you don't have access to a Usenet (NNTP) server you can use the web interface at http://groups.google.com
Re: Info required
Medi Montaseri wrote: Having said thatI recommend Writing Apache Modules with Perl and C known as the 'eagle book'. See Chapter 3, 'The Apache Module Architecture and API' and for a quick feeling, see Figure 3-3, page 60, The Request Loop. Of course I'm new to mod_perl, so if Pokymon Masters find me in fault, please train me as I want to be the best pokymon trainer...yes I got a four-year old... Well if you're new to mod_perl the Eagle is exactly the book to help you on your way to becoming a Poki-master. However, other Poki-masters have distilled their experience into a great tome that they make available to you as a great gift--the mod_perl Guide: http://perl.apache.org/guide You must read both of these ancient texts to complete your quest.
Re: AxKit.org/axkit.apache.org timing?
Ed Loehr wrote: I recently read that AxKit was in the process of becoming an ASF xml project. Does anyone have a sense of the timing for when this might happen and when axkit.org/axkit.apache.org will return/arrive? Also, does anyone know of a mirror site for axkit.org? You can see where Matt's up to at his diary; details here: http://use.perl.org/article.pl?sid=01/08/23/0040239
Re: mod_perl memory consumption
Paul DuBois wrote: mysql_store_result/mysql_use_result determine whether or not the server sends all the rows to the client at once, or one at a time as the client asks for them. mysql_store_result is therefore more memory intensive for the client (which must hold the entire result set). mysql_use_result is less memory intensive for the client, which needs memory only for one row at a time, but can cause other processes trying to use the server to block, because the tables from which you're retrieving rows are read-locked for the duration of the query (that is, until the client actually fetches the last row). For more information see Paul's great book 'MySQL' (New Riders), p257-259. If you're not lucky enough to have the book, there's a brief description here: http://perl.apache.org/guide/databases.html#mysql_use_result_vs_mysql_store
Re: using mod_perl
Hi! I am wondering if anyone could tell me how to actually run modperl on my webserver i have perl modules installed to run without modperl and dont know how to invoke the process to get them working can anyone help? jason The best place to learn mod_perl is the Guide: http://perl.apache.org/guide Sorry for the 'RTFM' reply, but you'll find that this document answers beginner questions better than anyone here can in a quick email, and it's kept very up to date thanks to the marvellous Stas Bekman.
Re: Subroutine xxxx redefined
Stas Bekman wrote: Something forces the reload of your files. I'd suggest putting something like: use Carp; cluck(I was loaded); in one of the files run the server in single server mode (httpd -X) and check the trace on the second reload. Nice idea. You might also try 'warn @INC' in one of your modules to see what the include path looks like. You might have a path appearing twice in @INC.
Re: mod_perl install for win32
Steven Sajous wrote: From: Jeremy Howard [mailto:[EMAIL PROTECTED]] Steven Sajous wrote: referring to nmake with VC++ It comes with it? I don't see it Yes it does. Run the batch file Robert mentioned and it'll work from the command line. You can also download it from: ftp://ftp.microsoft.com/softlib/mslfiles/nmake15.exe PS: Please add your message _after_ quoting the previous message, otherwise you get a confusing set of quotations as you see here. Thanks :-) I used nmake but after going for a little while I get there this error 'cl' is not recognized as an internal command operable program or batch file... cl.exe is the compiler that comes with VC++. It sounds like you haven't run the .bat file that Robert pointed you too yet. Running that batch script allows you to use VC++ tools from the command line. If you have VC++, you definitely have cl.exe, since otherwise you wouldn't be able to compile anything through the IDE!
Re: Looking for answers using mod_perl and windows nt.
Boyd, David wrote: I am using Windows NT, running apache with mod_perl. The problem that I am having is that I am dynamiclly filling in the options of a select box from a database. now when a user selects a value, I expect that value to be on the URL, but it is not. I am currently saving the selected value to a hidden field, which does show up on the URL. Anyone know the reason why. I'm not sure I understand your question, David. If you're saying that you would expect the result of pressing a submit button in a form to be that request is sent like 'http://some.host/file.pl?combo=value', then you need to ensure that the form in your HTML is set to GET rather than POST style encoding. But that's not really a mod_perl issue...
Re: file upload memory leak
Jack Cushman wrote: I am having a problem with very large file uploads (eg 100 MB). While the files load, the apache process stays at about 12000 K (the ps VSZ size). When the file finishes uploading, the thread suddenly jumps to over 20. My guess is that something is loading the whole file into ram, but I'm not sure what. I am running CGI.pm version 2.56, mod_perl version 1.25, and Apache version 1.3.19. Try the latest version of libapreq. perl -MCPAN -e 'install Apache::Request' should do it.
Re: note on wombat
brian moseley wrote: for those of you who were at the presentation this morning and wanted more info on wombat - here are the urls: http://libservlet.sourceforge.net http://wombat.sourceforge.net Unfortunately I wasn't at the preso, but I've looked at the slides you've got up on your site. But... I don't get it :-( Where does this fit--it's all very new to me? Is it another templating engine? ;-) Is it a session manager? Is it solving some other problem that I haven't come across yet? In the preso you mention an IMAP layer. What is this, and how would it work? Would it provide an IMAP server, or is it an interface to an existing IMAP server? Sorry for the dumb questions--from the web site it all sounds intriguing, but it's gone straight over my head!
Re: [OT] WebMacro (was: RoboWeb 1.0b, web application test suite generator)
Please be aware that WebMacro is a moderately popular Java templating tool hosted at http://www.webmacro.org Naming it WWW::WebMacro might be a bit confusing. It's your choice. I just wanted to make you aware of the other WebMacro. You're right--I just remembered that... Back to the drawing board :-( The saying is right--the hardest part of writing a Perl module is finding a suitable name...
[OT] WebMacro (was: RoboWeb 1.0b, web application test suite generator)
Gunther Birznieks wrote: If you guys end up finally collaborating, one very minor request I would have is that it goes into CPAN as something more standard like WWW:: namespace rather than a marketing name like RoboWeb. The current plan is for the 'scripting language' which provides a simple interface for scripting interactions will be called 'WWW::Automate'. The project that combines RoboWeb and WWW::Automate into a flexible recorder for web interactions might be 'WebMacro' which would appear in CPAN as 'WWW::WebMacro'. These are all preliminary suggestions and open to discussion. HTTP::WebTest already is in an appropriate place in the hierarchy, and is descriptively named. HTTP::WebTest will probably be able to use WWW::Automate scripts (either as a plugin in just through its generic Perl API) but this doesn't require any new project or name. If anyone is interested in contributing, note that RoboWeb is already on SourceForge, and WWW::Automate will be up in the next week or two. Contact me privately if you'd like more information on getting involved.
mod_dev + mod_perl very nasty conflict
A number of people have come across a problem where mod_dav shows the contents of the filesystem root: http://dav.lyra.org/pipermail/dav-dev/2001-June/002487.html I have also come across this problem, and narrowed it down (at least for me) to a conflict with mod_perl. I have no other non-core modules installed, no DSO install, and the out-of-the-box httpd.conf. If I just install mod_dav and add the mimimal httpd.conf lines for mod_dav, everything works perfectly: DAVLockDB /usr/local/apache/DAVLock/DAVLock DAVMinTimeout 600 DAVDepthInfinity On Directory /usr/local/apache/htdocs/DAVdocs DAV On /Directory Now I delete everything and untar it all again, and repeat the exact same process but install mod_perl as well after mod_dav with: cd ../mod_perl-1.25 perl Makefile.PL DO_HTTPD=1 EVERYTHING=1 APACHE_SRC=../apache_1.3.20 USE_APACI=1 PREP_HTTPD=1 make test make install cd ../apache_1.3.20 SSLBASE=../openssl-0.9.6a ./configure --prefix=/usr/local/apache \ --enable-module=most --disable-module=auth_dbm --disable-module=digest --ena ble-module=auth_digest \ --activate-module=src/modules/perl/libperl.a \ --activate-module=src/modules/dav/libdav.a make make install Now I get the problem of seeing the filesystem root when using DAV. This problem occurs with both a cadaver 'ls' and with MS Web Folders. The httpd.conf file is identical to when it worked fine (ie without mod_perl), and it contains no IfModule mod_perl.c sections. Any suggestions/workarounds/patches? Anyone using mod_perl + mod_dav successfully? What versions?
Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory
Perrin Harkins wrote: What I was saying is that it doesn't make sense for one to need fewer interpreters than the other to handle the same concurrency. If you have 10 requests at the same time, you need 10 interpreters. There's no way speedycgi can do it with fewer, unless it actually makes some of them wait. That could be happening, due to the fork-on-demand model, although your warmup round (priming the pump) should take care of that. I don't know if Speedy fixes this, but one problem with mod_perl v1 is that if, for instance, a large POST request is being uploaded, this takes a whole perl interpreter while the transaction is occurring. This is at least one place where a Perl interpreter should not be needed. Of course, this could be overcome if an HTTP Accelerator is used that takes the whole request before passing it to a local httpd, but I don't know of any proxies that work this way (AFAIK they all pass the packets as they arrive).
Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory
Joe Schaefer wrote: "Jeremy Howard" [EMAIL PROTECTED] writes: I don't know if Speedy fixes this, but one problem with mod_perl v1 is that if, for instance, a large POST request is being uploaded, this takes a whole perl interpreter while the transaction is occurring. This is at least one place where a Perl interpreter should not be needed. Of course, this could be overcome if an HTTP Accelerator is used that takes the whole request before passing it to a local httpd, but I don't know of any proxies that work this way (AFAIK they all pass the packets as they arrive). I posted a patch to modproxy a few months ago that specifically addresses this issue. It has a ProxyPostMax directive that changes it's behavior to a store-and-forward proxy for POST data (it also enabled keepalives on the browser-side connection if they were enabled on the frontend server.) FYI, this patch is at: http://www.mail-archive.com/modperl@apache.org/msg11072.html
Re: load average: 24.07, 14.76, 9.20
Perrin Harkins wrote:
Buddy Lee Haystack wrote:
How much of a performance penalty does using Apache::SizeLimit have?
Not enough that you'll notice it.
It really depends on 2 things:
- What OS you're on
- How complex your scripts are.
Here's the code that does the size check, which varies depending on your OS:
# return process size (in KB)
sub linux_size_check {
my $size = 0;
local(*FH);
if (open(FH, "/proc/self/status")) {
while (FH) { last if (($size) = (/^VmRSS:\s+(\d+)/)) }
close(FH);
} else {
error_log("Fatal Error: couldn't access /proc/self/status");
}
return($size);
}
sub solaris_2_6_size_check {
my $size = -s "/proc/self/as" or
error_log("Fatal Error: /proc/self/as doesn't exist or is empty");
$size = int($size/1024); # to get it into kb
return($size);
}
sub bsd_size_check {
return( (BSD::Resource::getrusage())[2] );
}
As you can see, your mod_perl handler would have to be _extremely_ simple
for this code to take a non-trivial proportion of its run-time.
Re: greetings and questions
Stas Bekman wrote: Come'n, are you sure you have ever looked at perl.apache.org? http://perl.apache.org/#docs Books and Documentation: Writing Apache Modules with Perl and C book by Lincoln Stein and Doug MacEachern. == Start learning mod_perl with the mod_perl Guide by Stas Bekman. ... I remember when I started with mod_perl last year, I didn't notice the guide until I saw it mentioned on the list. perl.apache.org is a long page, and it's not obvious what order to read the various links in... The FAQ linked to on this page includes 'What documentation should I read?' and doesn't include the Guide in the answer... The link to the guide is the 32nd of 97 links on this page... It is one of 11 documentation links... So I guess what I'm trying to say is that it's not that obvious ;-) Maybe the mod_perl site could start with: New to mod_perl? First, a href="http://perl.apache.org/dist/"download/a it and then a href="http://perl.apache.org/guide/install.html"learn/a to install it. Now read a href="http://perl.apache.org/guide/""The Guide"/a and become and mod_perl guru! ...or something like that.
Re: fork inherits socket connection
Stas Bekman wrote: But you don't need to call setsid() when you fork. Why looking for complicated workaround when you can do it properly without any workaround. Have you ever seen an example of fork that uses setsid? But your Guide says: A much better approach would be to spawn a sub-process, hand it the information it needs to do the task, and have it detach (close STDIN, STDOUT and STDERR + execute setsid()).
[OT] Re: is morning bug still relevant?
Stas Bekman wrote: Apologies Tim, somehow I always thought that 'infamous' is better than just 'famous'... I meant the 'famous'... sorry about that. I've corrected it in the guide. Thanks, Tom for clearing this out for me ;) Yeah, this is one of the more confusing words in English... Check out the film 'The Three Amigos'--the premise of the entire film is based around the lead characters thinking that 'infamous' means 'extra famous'!
Re: Segmentation fault
Per Moeller wrote: A couple of days ago I asked the question below, but nobody seemed to be able to answer it. Well, in case somebody else runs into this problem I can tell that my solution to the problem was to use perl5.005 and not install perl5.6.0 until after i completed the installation of mod_perl / Apache and the modules i needed for it. I suggested applying this patch: http://www.geocrawler.com/archives/3/182/2000/6/0/3863601/ Did this not work? After applying the patch did you recompile both Perl _and_ mod_perl?
Re: Trouble with AxKit at runtime - XML parsing stage
Dave Rolsky wrote: On Thu, 14 Dec 2000, Khachaturov, Vassilii wrote: http://bugs.perl.org/perlbug.cgi?req=querybody=Bizarre+copy seems to yield only one bug relevant to my particular error message case - http://bugs.perl.org/perlbug.cgi?req=bidbid=20001207.005range=6012format= H which is still open. No patch/patchid seen there. Did you mean this bug, or another? Is there some informal patch around? I'm pretty sure that's the bug. Sarathy said it was patched ages ago. I don't know why the bug's still open (you might want to let P5P know about that). The patch is here: http://www.geocrawler.com/archives/3/182/2000/6/0/3863601/
Re: greetings and questions
Darren Duncan wrote: ... Now some questions that I hope some of you can help me with. Hopefully each has short answers. ... Hi Darren, I think that all of your questions are answered in the mod_perl guide, which is at: http://perl.apache.org/guide/ I don't have time right now to post a link to the answer to each individual question, but after you've had a good look through the guide feel free to ask again if you can't find the answer to a particular question, or if you need clarification on some point. PS: I hope you're not offended by this 'RTFM' reply--but the Guide will give much better answers than I can since it's been revised over some time by the whole community...
Re: Mod_perl tutorials
Randal L. Schwartz wrote: META COMMENT... Maybe it's time we spun off a mailing list for this discussion, unless it's still interesting to the rest of the onlookers. Anyone care to host it or take that on? I would have thought that most people here are interested in how to teach mod_perl--a lot of us bring on staff who need to learn mod_perl. Personally, I'd be happy to see the discussion stay here...
Re: Email (mod_perl) Apache module?
George Sanderson wrote: I think it would be cool to have an email Apache module. I was thinking that if the URL was something like: http://www.site.com/user/mail it would activate the module. The module would allow the users to read and send email. Kind of like how I did Apache::FileMan (an Apache web site file manager). This would allow sites to get email interfaces up and running very quickly and provide a common user interface. Now that would be advocacy;-) IMHO, the best open source WebMail servers are PHP based: http://twig.screwdriver.net/about.php3 http://www.horde.org/imp There is a Perl based WebMail server at: http://www.webbasedemail.com/ although I haven't tried it.
[OT] Re: Which is good mailing list for perl developers.
John Michael Soileau wrote: Excuse me if I make an error. This is my first post to this list and not sure how it all fits together exactly. I just joined the news perl list you selected to have a look at it and now wonder why you don't put this mailing list in the same type of setup. There may be a way to do it that I don't know about and if there is, info greatly appreciated. Anyway, It is very tough keeping up with all of the emails and subjects. I now see that I should have joined with an email address created just for this list so that I could filter it to a folder somewhere. Anyway, why doesn't someone put this list on a news feeder. It would be much easier to maintain from a users point of view. Setting up a Usenet group is quite a lot of work, and the mod_perl list may not be high enough volume to justify it. Using Procmail or Sieve makes it easy to work with mailing lists. Create a folder for each list, and then create a rule where any message with to/cc containing '[EMAIL PROTECTED]' is moved to that folder.
Re: Apache::Session benchmarks
Perrin Harkins wrote: Apache::Session::File - Dual-PIII-600/512MB/Linux 2.2.14SMP: Ran 4 times. First time: ~2.2s. Second time: ~5.0s. Third time: ~8.4s. Fourth time: ~12.2s. Is there any reason not to use a file tree approach (splitting first and second characters of filenames into separate directories) like mod_proxy does? This seems like a pretty easy solution to the problem of degraded performance on larger sets of data. If you use File::Cache for storage, then it does this for you, to a user-defined depth.
Re: Linux Hello World: TT Optimized...
Joshua Chamas wrote: If you are using CGI.pm object methods, I would worry about calling all those methods to build your HTML and if you are performance minded, I would use them frugally. IIRC, CGI.pm is actually slower to run the functional syntax than the object syntax. This is because accessing CGI's functions end up getting dispatched through a complex autoload(ish) mechanism. I haven't benchmarked this though, so it's only theory!
Re: persistent connections for handlers
[EMAIL PROTECTED] wrote: I'm also interested in this problem. I've been piping my mails to sendmail and I'm told that this is not a good idea with mod_perl. I find that talking to SMTP server is noticeably slower although I don't know whether the slowness is just in the initial connection. I am using the local sendmail daemon as the SMTP server. On Sat, Dec 09, 2000 at 11:05:50AM -0500, John Saylor wrote: I am writing a mod_perl handler that takes a HTTP POST and sends mail. What I'd like to do is to open up a connection to the SMTP server [a Net::SMTP object] and a log file [filehandle] that exist outside of the handler. So that when the handler is run, it has access to these items, and can send data to them, but that they don't get opened up and brought down each time. Have a look at the Apache::DBI source, which keeps a pool of connections open like you need. Use a PerlChildInitHandler to set up a connection for each process (if you do it in startup.pl you have to share one between processes, which isn't ideal). There's nothing wrong with piping to sendmail though. If you fork() under a decent operating system, most memory is shared between processes. So just fork() and open a pipe to sendmail. For logging you might prefer to use Unix::Syslog rather than opening a file directly. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
[OT] Re: [ANNOUNCE] Take23
On Wed, 6 Dec 2000, Matt Sergeant wrote: OK, so this isn't really much of an announcement, since its still the same web site. Anyway, I'm renaming modperl.sergeant.org to take23.org. DNS should propogate shortly. Until then its available at modperl.sergeant.org (and will continue to be available there). Looks like DNS isn't working at all... I'm a complete DNS moron - is there anyone out there willing to offer me very a simple DNS setup on their primary and secondary servers? Sure, I can look after that. Shoot me a private email with the details. Through your registry, set your primary NS to fastmail.fm, and secondary to ns1.telstra.net. PS: You can also use http://www.granitecanyon.com for free DNS. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: debuggers
martin langhoff wrote: I wonder how do those hardcore guys that develop using handlers debug. Mhhh. They must write 'perlfect' code, I guess, and/or understand those cryptic debuggers ... Actually, debugging handlers is pretty easy. Just run httpd with the -X flag to make it single process, and use Apache::DB to get mod_perl to run under the debugger. Get Apache::DB from CPAN perl -MCPAN -e 'install Apache::DB' and read its docs perldoc Apache::DB When Apache hits your Perl code, you are dropped into the standard Perl debugger. Although typing '?' at the debug prompt shows a bewildering array of options, you can do a lot by knowing: n - Step over s - Step into x expr - Print the value of expr w - Show a few lines either side of the current one b sub - break at sub c - continue (stop debugging) c line# - continue, and break at line# HTH, Jeremy - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: shared mem [was: mod_perl advocacy project resurrection]
Perrin Harkins wrote: On Wed, 6 Dec 2000, Paul wrote: I was pointed to IPC::Sharable, IPC::Sharelite. I'll look at those. Take a look at IPC::MM for a shared memory hash implemented in C. Also, File::Cache is sometimes faster than the IPC modules. I don't think any of these solve problems like sharing sockets and file handles though. Is there _any_ solution to sharing sockets? My WebMail server keeps a pool of IMAP connections open (one per user) but it currently needs to create a connection for every user for every process that they use. This is awfully inefficient. I'm planning on creating a little Unix Socket daemon that keeps a single pool of connections, and make all IMAP connections go through the daemon. This seems like it should be unnecessary though--isn't there some way of sharing sockets amongst processes? PS: I second your suggestion to look at File::Cache--I use it for my session store and find it excellent. Speed depends on your OS's bufferring, however. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: [ libapreq ] desesperatly need of .32 or .33
But before anyone bites off more than they can chew, perhaps some discussion of the current bugs and future needs for libapreq should be aired out. My own problems with libapreq revolved around the multipart buffer code, and since I patched it a while back, I haven't bumped into any other snags. I agree--the multipart buffer memory issues are the key problem with the current 'official' release. There are at least two multipart buffer fixes around though, so it would be nice to merge one of these into the CPAN distribution. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Re:coredump on Carps in startup.pl
PJ When I start up apache (apachectl startssl), I get a core dump. PJ Attached is an "strace httpd -X" for those of you who find this useful. FWIW, I have the same issue also. I noticed it when using CGI::Carp. My solution was not "use" CGI::Carp; in startup.pl (a real lame solution, I know ;-), but my goal was to deal with it later (I needed to get my development environment back on-line) You may find that this patch fixes things for you: http://forum.swarthmore.edu/epigone/modperl/sningvezerd/26091135.MAA1 456 http://forum.swarthmore.edu/epigone/modperl/sningvezerd/26091135.MAA 1456 urk: no such article archive = 'modperl' thread = 'sningvezerd' article = '26091135.MAA1456' Make sure it's all on one line: http://forum.swarthmore.edu/epigone/modperl/sningvezerd/26091135.MAA145 [EMAIL PROTECTED] You'll probably need to copy and paste the line into your text editor first, since I suspect that my mailer will put a new-line in it by default. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Re:coredump on Carps in startup.pl
PJ When I start up apache (apachectl startssl), I get a core dump. PJ Attached is an "strace httpd -X" for those of you who find this useful. FWIW, I have the same issue also. I noticed it when using CGI::Carp. My solution was not "use" CGI::Carp; in startup.pl (a real lame solution, I know ;-), but my goal was to deal with it later (I needed to get my development environment back on-line) You may find that this patch fixes things for you: http://forum.swarthmore.edu/epigone/modperl/sningvezerd/26091135.MAA1456 [EMAIL PROTECTED]
Re: Querying MS Access DB from ASP files through ODBC
I'm not sure if this is the right place, but here goes my problem: I'm running Apache::ASP on apache 1.3.14 with mod_perl 1.24_01 on a i686 with RH 6.2. I've got a database query system written in ASP that at some point uses an ODBC connection to an MS Acess database to query it. The unusual thing is that both ODBC components need to run on the linux box and that therefore there can be no NT box involved. I'm basically trying to migrate the whole thing from a Windows NT 4 box to my linux box. Is there a free / GPL app that can help me accomplish this and where could I find a detailed manual on it? Or are there any appropriate resources online where I should look? Presumably your ASP system is currently written in VBScript or JavaScript. Apache::ASP requires that you write your script in Perl, so you'll have to do some rewriting. You'll also have to deal with migration of the Access database, which isn't really mod_perl related, so you might prefer to email me directly if you need more help beyond what follows... Basically, I'd suggest moving to a light-weight Linux database rather than trying to use an Access DB through ODBC, if you have to get NT out of the equation. MySQL (www.mysql.org) has a similar feature set to Access, although it is a server-based rather than file-based database system. It does not support transactions, updatable joins, or subqueries--if you need these you could try PostgreSQL (www.postgresql.org). To transfer your database you can use MyODBC to create a connection to your MySQL database from Windows, and then create Make Table queries in Access setting the destination database to your MySQL database in View-Properties. You'll have to create indexes yourself if you take this route. I've got a VBA script that creates MySQL scripts from an Access DB that I can send you if you prefer, although it's a bit flakey.
libapreq (Apache::Request) unhappiness
Jim Winstead back on June 26 finished debugging a patch to libapreq (that's Apache::Request) that avoids the problem of the mod_perl process eating up all the memory required for an upload (and then some!) and not giving it back. http://forum.swarthmore.edu/epigone/modperl/thahwhahyu/2626150219.A9842@ trainedmonkey.com Joe Schaefer then rewrote this to avoid some unnecessary malloc()s, http://forum.swarthmore.edu/epigone/modperl/rorprilswe/m3ya3m513y.fsf@mumonk an.sunstarsys.com which David Mitchell then patched to make more portable. http://forum.swarthmore.edu/epigone/modperl/rorprilswe/27051203.NAA20171 @tiree.fdgroup.co.uk Why this history lesson? Well, Joe and David's version, I've just discovered, fails to correctly deal with large POSTs that are not file uploads. In particular, I just wrote a lengthy (3 hours!) email in my webmail system, and when it got to the recipiant only the first and last bits were there :-( I've replicated this behaviour with a number of large POSTs, and the problem always occurs (under Linux 2.2+latest stable mod_perl and Apache). The good news is that Jim's version doesn't have this problem. Are there any volunteers to fix Joe and David's more efficient version? (Sorry, I'm no C guru...) -- Jeremy Howard [EMAIL PROTECTED]
Re: ignore header_only()?
Ken Fox said: Sorry to be a pain, but I still haven't heard anything back about my proposal: Ken Fox wrote: I was using $request-sent_header and found out that it's not terribly useful if PerlSendHeader is off -- sent_header always returns 1. I changed mod_perl_sent_header so that it returns 0, 1 or 2: 0 - headers have not been sent (PerlSendHeader on) 1 - headers have not been sent (PerlSendHeader off) 2 - headers have already been sent Is this the right list for change proposals? Should I re-submit my patch? You could try: The dev list is for discussions about the development of the core mod_perl modules. Subscription information. a.. subscribe to the list: [EMAIL PROTECTED] b.. unsubscribe from the list: [EMAIL PROTECTED] c.. get help with the list: [EMAIL PROTECTED] Doug popped his head up a few weeks ago to say he's in read-only mode at the moment. I don't know why--maybe he's busy with v2 at the moment. If you don't have any luck on the dev list you could try mailing him directly, I guess. -- Jeremy Howard [EMAIL PROTECTED]
Re: Prototype mismatches?
...redefined constant sub warning...
This warning also appears with any scalar constant created with
constant.pm -- pretty darn annoying for cgi error_logs. Is there
a more 'official' fix or suggestion why mod_perl triggers this?
It's not 'official', but I patched constant.pm so that at the end of the
import sub, it checks that the sub isn't already defined. Just wrap this
around the last block in import():
if (!(defined {"${pkg}::$name"})) {
import the const sub
}
I don't have this problem anymore with 5.6 though...
--
Jeremy Howard
[EMAIL PROTECTED]
FastMail--Sign up at http://fastmail.fm
Re: [OT] Error reads: Do you need to predeclare new?
From: "Kenneth Lee" [EMAIL PROTECTED] I wrote a module CGI::CommonObject which is a subclass of CGI. If I preload them both in conf/startup.pl, error occurs when I try to instantiate CGI::CommonObject. ... my $q = new CGI::CommonObject; Kenneth, _never_ use that style of calling class methods. To really understand why, read Damien Conway's excellent 'Object Oriented Perl', in which in gives an excellent overview of the many problems this causes with inherited classes. Instead, use: my $q = CGI::CommonObject-new(); It's less pretty, perhaps, but _much_ more reliable.
Re: further adventures with taint
I see intermittent Apache error_log entries like:
[Mon Jul 24 04:08:02 2000] [error] Insecure dependency in require
while running with -T switch at (eval 85) line 3.
...
I suspect MIME::Lite, but the code won't work if I remove it
Yes, MIME::Lite needs special treatment to be taint-safe. Get the latest
version from CPAN, then do a 'perldoc taint'. I submitted a patch to the
author which he has documented, that enables a taint-safe mode.
Apache always prints the line number and module the error occurs on when
I've had taint issues. I don't know why it doesn't for you... maybe you
just need to add:
use Apache qw(warn);
at the start of your modules?
I suggest rereading perldoc perlsec a few more times--there's a lot of
info in there and it took me a while for it to sink in. Setting your
ENV{...} safely isn't enough--any unsafe function (such as open() used by
MIME::Lite) must have its input data cleaned unless it's input data is a
constant. To clean the input data you can use (off the top of my head):
$dirtyData =~ /^([^|]*)$}/;
if ($1) {
my $cleanData = $1;
# do stuff with $cleanData in unsafe functions
} else {
# handle error nicely
}
The regex here detects obvious problem characters in Unix. A more robust
version would specifically check for safe chars, rather than removing
unsafe chars.
--
Jeremy Howard
[EMAIL PROTECTED]
FastMail--Sign up at http://fastmail.fm
Re: further adventures with taint
I'm only passing scalar text to MIME::Lite - no file attachments, so
it shouldn't open(). Anyway, if it were, why don't I see "Insecure
dependency in open" instead of "...in require"?
Yes, good point. The open() is only used for sendmail, not for Net::SMTP
anyway.
Perhaps you need to add to your startup.pl (before any 'use' statements):
delete @ENV{qw(PATH IFS CDPATH ENV BASH_ENV)};
--
Jeremy Howard
[EMAIL PROTECTED]
FastMail--Sign up at http://fastmail.fm
Re: SIGNAL 11 - Help anyone?
...segv trace...
I also encountered occasionally the "Bizarre copy of ARRAY in aassign at
/usr/lib/perl5/5.6.0/Carp/Heavy.pm line 79" error discussed some time before
on the list.
Both the segv and the 'bizarre copy' are fixed by this Perl 5.6 patch:
//depot/perl/cop.h#49 (text)
Index: perl/cop.h
--- perl/cop.h.~1~ Thu Jun 8 06:58:03 2000
+++ perl/cop.h Thu Jun 8 06:58:03 2000
@@ -106,13 +106,9 @@
} STMT_END
#endif /* USE_THREADS */
-#ifdef USE_ITHREADS
- /* junk in @_ spells trouble when cloning CVs, so don't leave any */
-# define CLEAR_ARGARRAY() av_clear(cx-blk_sub.argarray)
-#else
-# define CLEAR_ARGARRAY() NOOP
-#endif /* USE_ITHREADS */
-
+/* junk in @_ spells trouble when cloning CVs and in pp_caller(), so
don't
+ * leave any */
+#define CLEAR_ARGARRAY() av_clear(cx-blk_sub.argarray)
#define POPSUB(cx,sv) \
STMT_START { \
//depot/perl/t/op/runlevel.t#16 (xtext)
Index: perl/t/op/runlevel.t
--- perl/t/op/runlevel.t.~1~Thu Jun 8 06:58:03 2000
+++ perl/t/op/runlevel.tThu Jun 8 06:58:03 2000
@@ -349,3 +349,18 @@
bar
B 2
bar
+
+sub n { 0 }
+sub f { my $x = shift; d(); }
+f(n());
+f();
+
+sub d {
+my $i = 0; my @a;
+while (do { { package DB; @a = caller($i++) } } ) {
+@a = @DB::args;
+for (@a) { print "$_\n"; $_ = '' }
+}
+}
+EXPECT
+0
End of Patch.
--
Jeremy Howard
[EMAIL PROTECTED]
File upload gateway (was Re: Forking off lengthy processes)
I said: The other place I'd like to avoid holding up my mod_perl processes is in waiting for file uploads to finish (which is a common situation as people upload attachments for their emails). Is there a any way to do this... perhaps by using some kind of 'gateway' server? Barrie Slaymaker wrote: The most often wished-for approach is for a http cache to accept the whole request before passing it through to the main httpd, but I'm not aware of any that do that. Exactly! If anyone is doing something similar to this could you direct me to the code you use? The beauty of it would be that the upload (and other CGI params) could be passed from the cache to the mod_perl server over the fast internal network link (so the mod_perl process doesn't wait long), while the slow upload from a user's modem only holds up a small httpd process. I guess this really should be a module written in C, since that would be the smallest way of doing it memory-wise (I'd rule out a Perl CGI since it has to be recompiled each time, and this gateway would presumably be high volume). How do proxy servers like Squid work? Do they just pass the packets on, or do they accept the whole request and _then_ pass it on? -- Jeremy Howard [EMAIL PROTECTED]
Forking off lengthy processes
I've just read the Guide's advice on fork and system calls, but it doesn't really seem to offer a resolution to the problems it raises. I'm trying to set up an SMS gateway for my webmail users. The gateway just uses LWP to contact a web-SMS provider and sends a POST request (following some redirects in the process). This can be a lengthy process, so rather than have my mod_perl process waiting, I'd like to pass on the request to another process. Here are the potential solutions I can think of and their problems: * fork(): Memory hog, since it copies the entire Apache process (is this right--does it share the memory for all the modules etc...?) * system(): Slow--would have to start up a new Perl process * Create a simple preforking server and have mod_perl just pass a request to it through a socket: Seems like overkill. The most hopeful possibility seems to be fork(), but the Guide says to avoid it. Isn't almost all the memory shared though? The other place I'd like to avoid holding up my mod_perl processes is in waiting for file uploads to finish (which is a common situation as people upload attachments for their emails). Is there a any way to do this... perhaps by using some kind of 'gateway' server? -- Jeremy Howard [EMAIL PROTECTED] FastMail--Sign up at http://fastmail.fm
Re: Apache::Request and memory
...Problem with patch to fix memory blow-out with file uploads... Okay, I think I tracked this down to a one-byte buffer overflow. Try the attached patch to see if that fixes it (it fixes things in my testing). Thanks--certainly an improvement. I tried a 25k file, which worked fine. However a 1.8MB file still caused a segfault. If you can't replicate the problem, let me know what diagnostics you'd like me to run. I'm not familiar with gdb, so I'm not sure where to start. Unfortunately, the overflow seemed to sneak through with no problems on FreeBSD, and on Linux if you compile with -g. I'm running latest Apache, mod_perl, and Perl, under Linux (2.2 kernel). PS: Can anyone point me to a tutorial on debugging XS modules? From time to time Doug provides some magic gdb commands which are really helpful--I'd love to be able to do this kind of stuff myself... -- Jeremy Howard [EMAIL PROTECTED]
Re: Apache::Request and memory
Okay, I think I tracked this down to a one-byte buffer overflow. Try the attached patch to see if that fixes it (it fixes things in my testing). Oops. Please ignore my last message. Your fix works just fine... I had some code to automatically kill my process after it got an upload 1MB, in order to work around the libapreq bug that your patch fixed! -- Jeremy Howard [EMAIL PROTECTED]
Re: Apache::Request and memory
Apache::Request eats up memory during a file upload Attached is a patch to libapreq that addresses this problem. ... Thanks for this, Jim. Unfortunately, I'm having some problems with the patch. When I try and upload a file greater than a couple of k, I get a segfault in httpd. Could you possibly email me your current working libapreq sources?--perhaps there's some other differences hiding somewhere... Has anyone applied this patch and got it working with large uploads? -- Jeremy Howard [EMAIL PROTECTED]
Perl 5.6+mod_perl--nearly there now!
I've been testing the patch posted to this list yesterday to fix the
problem with Perl 5.6 'over-optimising' by leaving junk in @_, that
causes problems with CGI::Carp and others.
The patch fixes up all the problems I had, except for using CGI::Carp
under Apache::DB, when importing CGI::Carp with
use CGI::Carp ();
and then using it with
CGI::Carp::croak('some message');
To get this to work, I had to change the EXPORT_OK line in CGI/Carp.pm
to:
@EXPORT_OK = qw(croak carp confess carpout fatalsToBrowser wrap
set_message cluck);
I don't know why this is necessary, but otherwise I get a segfault using
Apache::DB.
--
Jeremy Howard
[EMAIL PROTECTED]
Fwd: Re: [ID 20000527.001] Carp::Heavy - Bizarre copy of HASH in aassign
You might remember the 'Bizarre copy of HASH in aassign' when using CGI::Carp I posted a workaround for... Here's an update from Perl 5 Porters--it's being fixed in the next release: - Original message - From: Gurusamy Sarathy [EMAIL PROTECTED] To: Jeremy Howard [EMAIL PROTECTED] Date: 2000-06-07 11:04:22 Subject: Re: [ID 2527.001] Carp::Heavy - Bizarre copy of HASH in aassign On Wed, 07 Jun 2000 16:00:57 +0200, Richard Foley wrote: The following causes a 'Bizarre copy of HASH in aassign' error perl -MCGI::Carp -d -e 'print croak(1)' Then press 'c' to continue in the debugger It only occurs in debug mode. [...] I don't know if there's any reason the DB package was used in Carp::Heavy... If so, this fix may break something. Yes, that will break things. The real problem should be fixed in 5.6.1 (coming sometime before mid-July, I hope). Sarathy [EMAIL PROTECTED] -- Jeremy Howard [EMAIL PROTECTED]
Re: Memory usage for libareq uploads
Jie Gao said (talking about avoiding memory problems with
libareq::parms()):
What I do is kill the child process after uploading is finished:
$r-child_terminate();
Not an elegant way, but it works for the time being.
Good idea. I've implemented something which uses this approach if the
file upload size is 'too big'. Here's the code I use--I hope someone else
finds it useful:
use constant MAX_SIZE_KILLPROC = 200;
my $TotalSize = 0;
foreach my $ThisAtt ($R-upload()) {
# Check that filehandle for uploaded file is valid
if ($ThisAtt-fh) {
SaveAtt ($MsgId, $ThisAtt)
### Replace next line with your own error handler
|| return $Self-SignalError(ERR_UPLOAD_FAILED);
$TotalSize += $ThisAtt-size();
} # If valid fh
# Work around bug in libareq-parms, that uses up too much memory
$R-post_connection(sub {$R-child_terminate()})
if $TotalSize MAX_SIZE_KILLPROC;
}
--
Jeremy Howard
[EMAIL PROTECTED]
--
Jeremy Howard
[EMAIL PROTECTED]
Memory usage for libareq uploads
Way back in the annals of history (well, err, 24 Feb 2000, anyway) there was some discussion about uploads chewing up memory in libareq... Jim Winstead said: quote Is it just me, or is it pretty dodgy code? It allocates memory like its going out of style (it doesn't appear to reuse any of the buffers it allocates from the pool), resulting in server processes that balloon up to five times the uploaded file's size. Its also appears to be flat-out slow because of all the extra work it is doing. Just an observation. I (or a coworker if I'm lucky :) will be rolling up my shirt-sleeves and diving in tomorrow. This is just my found-the-leak-now-I-get-to-sleep-on-it-and-hope-it-was-fixed- already-or-I'm-being-dumb mail. /quote to which Doug replied: quote it's pretty dodgy code :) the multipart buffer code in libapreq is actually just a translation from CGI.pm's Perl code to C. notice the comment: /*XXX: we can optimize this loop*/ meaning, I'm aware it's not reusing the buffer, but it's using a subpool which is cleared at the end of multipart_buffer_fill, each call to that should be no more than 4k or so. I have a recent mail (not to the list) from somebody who experienced the balloon, looked into it, and thinks that ap_clear_pool() is not working. anyhow, my plan is to get 1.22 out, then revisit libapreq, then onto modperl-2.0 /quote Well, I'm facing this problem now. The parms() method (or parse() or anything else that calls parms()) chews up enormous amounts of memory when uploading large files. I like to give my webmail users freedom to upload pretty much whatever size attachments they like, but that means my Apache processes grow very large, and never shrink again. What can I do to avoid this? I know I can use one of the various directives that kills off large processes, but that seems a bit aggressive... Has anyone got any sample code that spins off file upload activity to an external CGI and then returns to a mod_perl handler when the dirty work is done? ...Or is there some new code out there (or coming Real Soon Now) that avoids this whole issue? TIA, Jeremy -- Jeremy Howard [EMAIL PROTECTED]
Re: Bugs 5.6.0 modperl use?
[Wed May 24 19:58:28 2000] [error] PerlRun: `Bizarre copy of HASH in aassign
at /usr5/perl/lib/5.6.0/Carp/Heavy.pm line 79.
...
Do the following:
In Carp::Heavy, change 'DB' to 'DB_tmp':
Line 39: while (do { { package DB_tmp; @a = caller($i++) } } ) {
Line 79: @a = @{DB_tmp::args}; # must get local copy of args
In CGI::Carp, change Line 257:
FROM: my $message = shift;
TO: my ($message) = @_;
...
do you have small test case to reproduce this bug?
Well, that turned out to be a challenging request! Here's a module that
always segfaults, if you run with
httpd -X -Dperldb -Dperlsi
#!/usr/bin/perl -w
package MailApp::temp;
use CGI::Carp ;
use Apache ();
if (Apache-server-dir_config('InDB')) {
1;
}
use Email::Valid ();
$a = Email::Valid-mx(1);
1;
And here's my perl.conf (which defines the perldb and perlsi symbols):
IfDefine perldb
PerlSetVar InDB 1
PerlFreshRestart On
/IfDefine
PerlRequire conf/startup.pl
IfDefine perlsi
PerlModule Apache::StatINC
PerlInitHandler Apache::StatINC
PerlSetVar StatINCDebug On
/IfDefine
IfDefine !perldb
PerlTaintCheck On
/IfDefine
Location /test
IfDefine perldb
PerlFixupHandler Apache::DB
/IfDefine
SetHandler perl-script
PerlHandler MailApp::temp
Order deny,allow
Allow from all
/Location
Note that there's no 'use Apache::DB'--httpd segfaults with or without
it... So it doesn't seem to be a debugger bug.
If I remove _any_ of the lines from the test module, I don't get the
error. That's why it's so hard to debug and replicate.
--
Jeremy Howard
[EMAIL PROTECTED]
Re: Bugs 5.6.0 modperl use?
Well, that turned out to be a challenging request! Here's a module that always segfaults, if you run with httpd -X -Dperldb -Dperlsi Sorry, I forgot to mention... the segfault only occurs if you _don't_ have Net::DNS installed. -- Jeremy Howard [EMAIL PROTECTED]
Perl 5.6+mod_perl bugs (was Re: Win32 mod_perl binary [update])
Randy Kobes said:
- the patch posted to this list a few days ago by
Jeremy Howard for an error about a "Bizarre copy of HASH ...":
In Carp::Heavy, change 'DB' to 'DB_tmp':
Line 39: while (do { { package DB_tmp; @a = caller($i++) } } ) {
Line 79: @a = @{DB_tmp::args}; # must get local copy of args
In CGI::Carp, change Line 257:
FROM: my $message = shift;
TO: my ($message) = @_;
%%%
also fixes a problem with both Apache-ASP and HTML-Mason
under Perl-5.6.0 of the (0.6) mod_perl binary. The minimal
prerequisites for both ASP and Mason are included in this binary.
Although these changes get around the immediate segfaults and errors when
using Perl 5.6, there are a few things to be aware of if you are looking
at using mod_perl + Perl 5.6.
Basically, when problems in mod_perl come up after moving to Perl 5.6,
I've found that they tend to appear as segfaults in httpd, which I never
used to get under 5.005_03. Problems I'm aware of:
* I'm still working with Lincoln to properly fix CGI::Carp (the patch I
submitted, and his original code, ignores all but the first argument to
die()). Unfortunately any attempt to access beyond the 1st argument
causes a segfault under certain circumstances
* Many problems appear running under Apache::DB. In particular,
Apache::Util::escape_uri causes a segfault under Apache::DB in some
circumstances, as does Apache::Request::register_cleanup (when called as
a class rather than an object method)
* Slight changes to code can cause a segfault for no apparent reason.
For instance, under Apache::DB, the following causes a crash (from
Email::Valid)...
$Resolver = new Net::DNS::Resolver unless $@;
...while the following works fine...
if (!$@) {
$Resolver = new Net::DNS::Resolver;
}
You may well not hit any of these problems, since many only come up with
certain combinations of modules loaded. Having said that, I'd recommend
avoiding Perl 5.6 for the time being until these issues are sorted out,
since if you do hit one of these issues they are a real pain to debug.
--
Jeremy Howard
[EMAIL PROTECTED]
Re: Bugs 5.6.0 modperl use?
CGI::Carp preloaded.
DBI preloaded.
[Wed May 24 19:58:28 2000] [error] PerlRun: `Bizarre copy of HASH in aassign
at /usr5/perl/lib/5.6.0/Carp/Heavy.pm line 79.
...
Anyone else seen these?
Oh yes, I've been battling that tonight...
Do the following:
In Carp::Heavy, change 'DB' to 'DB_tmp':
Line 39: while (do { { package DB_tmp; @a = caller($i++) } } ) {
Line 79: @a = @{DB_tmp::args}; # must get local copy of args
In CGI::Carp, change Line 257:
FROM: my $message = shift;
TO: my ($message) = @_;
Yes, this really does matter--I carefully reproduced a problem that only
occured when 'shift' was used!
I'll contact the relevant authors tomorrow. I don't know if this is a
Perl 5.6 bug, or a module bug that 5.6 has surfaced.
--
Jeremy Howard
[EMAIL PROTECTED]
Re: XHTML_DTD redefined?
I'm getting this on every startup: Constant subroutine XHTML_DTD redefined at /usr/lib/perl5/5.00503/constant.pm line 175. what's this about? This isn't actually a mandatory warning any more (as others have suggested), but of course being a good boy you've got -w on, so you get the warning anyway ;-) I'm a good boy too, so rather than just turn -w off, I've patched my constants.pm so that in the import sub, it first checks whether the symbol is defined in the calling package (and if so, it does nothing). I'd attach the patch, but since I upgraded to 5.6 this problem has fixed itself, so I haven't got the patched version anymore (I'm a bigger masochist than you, Randal--I upgraded Perl, mod_perl, and Apache all at the same time!) -- Jeremy Howard [EMAIL PROTECTED]
Re: Prb: Apache::DB plus Perl 5.6 doesn't break
i am able to set breakpoints no problem with 5.6.0 (perl-current, actually). i would suggest stripping back your Perl config to something simple (i tested with b Apache::Status::handler) and make sure require Apache::DB/Apache::DB-init is the first piece of Perl code to run. Thanks heaps, Doug--moving require Apache::DB/Apache::DB-init to the top fixed it! Previously I had 'use Apache' 1st, which worked fine under the "old" version... It's funny the things that change between versions, isn't it? In fact, Apache::DB is now working better than before--I used to have some subs outside of modules that wouldn't break properly... now they work fine! Is anyone aware of any specific reason not to switch to 5.6.0 with mod_perl for production machines (other than just 'in case')? -- Jeremy Howard [EMAIL PROTECTED]
Re: [preview] Search engine for the Guide
Stas Bekman [EMAIL PROTECTED] wrote: Ok, We have a preview ready for you. Randy Kobes worked hard to prepare this one. So your comments are very welcome. If you like it we'll put this into production. Please keep either the list CC'ed or if you reply to me in person, make sure you keep Randy CC'ed -- all the kudos should go his way :) When I search for 'dbi' or 'DBI', it finds nothing, and the search box shows 'dby'! It looks like it's try to helpfully change my search pattern... -- Jeremy Howard [EMAIL PROTECTED]
Prb: Apache::DB plus Perl 5.6 doesn't break
Well, that subject line looks a bit odd, doesn't it? What I mean is that with Perl 5.6.0, Apache::DB is broken, because it doesn't break... Er, that is, you can add breakpoints, but Apache::DB never actually breaks on them. Is anyone else having this problem? Is there a new version in the works? -- Jeremy Howard [EMAIL PROTECTED]
Re: Prb: Apache::DB plus Perl 5.6 doesn't break
Well, that subject line looks a bit odd, doesn't it? What I mean is that with Perl 5.6.0, Apache::DB is broken, because it doesn't break... Er, that is, you can add breakpoints, but Apache::DB never actually breaks on them. Is anyone else having this problem? Is there a new version in the works? I've got a bit more info on this now. Nothing has changed on my PC (Linux RH 6.2) since this morning, other than updating Perl from 5.005_03 to 5.6.0. Apache::DB _was_ working fine. When I add a breakpoint, tab completion works fine to enter a sub name, and no errors are given by the debugger. However, doing an 'L' doesn't show any breakpoints, and no breaks occur. I've tried replacing perl5db.pl in 5.6.0 with the version from 5.005_03, but the behaviour is the same. Anyone else having this trouble? Anyone successfully used Apache::DB in 5.6.0? -- Jeremy Howard [EMAIL PROTECTED]
Guide search engine (was Re: multiple copies of a module)
Stas Bekman wrote: Hold on, at this very moment a few mod_perl fellas are working on having a good search engine for the guide. Just give it some more time, I'm trying to bring the best so it'll take a while... I'm glad you brought this up again. Since I mentioned I'd be happy to host such a thing, and asked for suggestions, I've got a total of one (from Stas--thanks!). That suggestion was to use ht://dig http://www.htdig.org/. Has anyone got a search engine up and running that they're happy with? Stas has made the good point that it needs to be able to hilight found words, since the pages are quite large. If anyone has a chance to do a bit of research about (free) search engines, I'd really appreciate it if you could let me know what you find out. It'd be nice publicity if it was mod_perl based, I guess, but it doesn't really matter. My only concern is that it seems a little odd to keep this just to the Guide. Wouldn't it be useful for the rest of perl.apache.org? I wouldn't have thought it's much extra work to add a drop-down box to search specific areas of the sight (the Guide being one)... If there's a good reason to have the Guide's search engine separate to the rest of perl.apache.org, should it have a separate domain (modperlguide.org?, guide.perl.apache.org?)? -- Jeremy Howard [EMAIL PROTECTED]
Re: Guide search engine (was Re: multiple copies of a module)
BTW: Your email client is broken and not wrapping words. I know--sorry. I'm fixing that this week. I'm just going through the RFCs to see exactly how to implement this right... (The email client is a web-based thing I've written in mod_perl--of course ;-) I just wrote a very simple SQL based engine - so I would say I'm happy with that. It's fast and it's all in perl. I could very simply rip out the search parts of the code for someone to play with if they wanted to. Sounds good. Personally, I'd rather a simple engine we can fiddle with ourselves than a big system written in C. Does your engine generate a database from flat files? Is there some basic parameterisation (a 'stop list' for common words, definable 'keyword' characters, ...)? I think word highlighting is overrated. It's only necessary in this case because the guide is so damn huge now. The size problem could be eliminated by making the guide split itself up into smaller sections. My proposal would be to do that by converting the guide to docbookXML and use AxKit to display the resulting docbook pages. The AxKit docbook stylesheets are nice and friendly, and written in Perl, not some obscure XML stylesheet language. And after all that, it would make converting the guide to a format O'Reilly likes to publish (i.e. docbook), trivial. Your word highlighting statement is, I suspect, controversial. On the other hand, converting to docbook is unlikely to meet much resistance from users--as long as Stas doesn't mind maintaining it!... To get the best of both worlds, why not simply chain the search engine result through a filter that does the highlighting. I bet someone's written such a filter already--anyone? My only concern is that it seems a little odd to keep this just to the Guide. Wouldn't it be useful for the rest of perl.apache.org? I wouldn't have thought it's much extra work to add a drop-down box to search specific areas of the sight (the Guide being one)... perl.apache.org already has a search engine. So I've heard, but: * Where is it? (doing a Find on the front page doesn't show it) * Does it do highlighting? * Can you select a subset of the site? (e.g. just the Guide) If there's a good reason to have the Guide's search engine separate to the rest of perl.apache.org, should it have a separate domain (modperlguide.org?, guide.perl.apache.org?)? guide.modperl.org ? Looks like modperl.org is taken: Domain Name: MODPERL.ORG Registrar: NETWORK SOLUTIONS, INC. Whois Server: whois.networksolutions.com Referral URL: www.networksolutions.com Name Server: DNS2.BASCOM.COM Name Server: DNS.THAKKAR.NET Updated Date: 24-nov-1999 They're not using it though--maybe they would transfer? Probably better to stick in the perl.apache.org domain though. BTW, thanks to everyone who's already responded privately to my renewed request. Keep it up! -- Jeremy Howard [EMAIL PROTECTED]
Re: Guide search engine (was Re: multiple copies of a module)
...the perl.apache.org search facility * Where is it? (doing a Find on the front page doesn't show it) At the bottom of all guide pages. How funny--I'd never even noticed it! I see that it's using 'Swish-E' http://sunsite.berkeley.edu/SWISH-E/. Stas--did you get that up and running? Can we tailor it for our needs? Here's an attempt at listing what I think we've decided we should aim for: - Allow restriction of search to just the guide - Allow searching of other documents through a popup selection (probably make the guide the default?) - Highlight found words - Try and index in a way that suits programmers, not English writers. e.g. include @, %, $, ::, in indexed words. Have I missed anything? (I'm ignoring the docbook issue for the moment since it's not directly related, and I guess it's really Stas' call anyhow.) I would have thought the best bet would be to put it on the footer of every perl.apache.org page. A popup which allows selecting a subset of the site might default to either 'whole site' or 'mod_perl Guide', or maybe it changes it's default to whatever part of the site is currently being viewed... The outstanding issues, I believe, are: - Who looks after the perl.apache.org search facility? Are they happy to expand its functionality as described? - What tool? Potential options so far are Swish-e, htdig, or custom Perl (perhaps based on Matt's engine). Any of these could be piped through a word-hilighting filter - What's the best 1st step? i.e. How can we get a simple search going quickly, while providing the foundation for a more complete system down the track? - Who's going to do the actual work? As I've mentioned, if a machine is required, I'm happy to provide it. However, I don't have the experience in this area to lead the work--although of course I'll contribute where I can! It would be nice to get a private mailing list going to avoid filling up this list too much more. Anyone who's interesting in getting involved, email me, and I'll ensure that I add your name to the list. You don't have to be a programming guru, of course... there's always plenty of ways to get involved in these things. -- Jeremy Howard [EMAIL PROTECTED]
Re: speed up/load balancing of session-based sites
Murali said: As I understand from this discussion we have 2 methods involving creating a session-server which will store all session data. a) NFS mount a server which will store all session data b) Have a DB in this server which stores this data. Through a network connect to the DB and retrieve the info. I don't think you have to take either approach to store all session specific data. When a session is started, simply use a redirect a go to a URL that includes the IP address of the machine that will serve this session. From then on, the session will always go to the same machine, and therefore the session data doesn't have to be centralised or distributed. The downside of this is that bookmarked pages will not contain your normal internet address, but instead the address of the specific server that is serving the current session. Although a little unattractive, it doesn't really matter. Just make sure that when this bookmark is used, you deal with it properly. That is, notice that this session id is no longer active on this machine, and create a new session just like the 'front-end' server would. The 2nd downside is that the first connection in a new session may have one additional redirect. The front-end server needs access to a central data store to do authentication (since it needs to know about _all_ users), but that only generally needs a username/password/userid, so it's unlikely to cause very much network traffic. -- Jeremy Howard [EMAIL PROTECTED]
Re: [OT] Great book!
another great one ive yet to purchase is object oriented perl there is some sample chapters online somewhere See http://www.manning.com/Conway/index.html. It really is a tremendous book. I thought I knew OO Perl pretty well, but after reading Conway's book I realised I'd hardly begun!... Some of the useful techniques described: * Using closures to create static class data * Encapsulation with Pseudo-hashes and 'use fields' * Secure encapsulation with SecureHash and the flyweight pattern * Auto generating classes with Method::Maker * Implementing persistance * Inheriting class data * Blessing regexps, subs, and globs I could keep going, but there's a taste... There's a good forum at the attached web-site that Conway visits regularly. -- Jeremy Howard [EMAIL PROTECTED]
Re: search engine for the Guide
At 01:28 PM 5/4/00 +0300, Stas Bekman wrote:
Two things:
1) I'd better concentrate on improving the content and structure of the
Guide and will leave this search engine task to someone who needs to use
the Guide but find it unusable without the proper search engine.
2) perl.apache.org doesn't have mod_perl installed, so it's better to use
some other site. I don't have any.
I'd be happy to host a search engine on my site. I'm not prepared to write one from
scratch though, so if anyone has any suggestions of what the best 'off-the-shelf'
solutions are I'd love to hear.
One option is to use Google. Have a look at this link
http://www.google.com/search?q=cache:perl.apache.org/guide/performance.html+%22mod_perl+guide%22hl=en
(put it on one line, of course). It highlights the searched terms ('mod_perl' 'guide'
in this case). Google doesn't allow searches within a site. However, if the same
unique string were placed on each page of the guide, adding that string to the search
query would only return hits from the guide. I think the best way to do this would be
to create a custom search page that links to Google, and automatically includes the
unique string in the request.
Of course, if there are any free search tools that provide this functionality and come
with source, that would be even better!
Re: [request] live server updates + ideas for guide/control.html
On 5/2/00 2:19 PM, Stas Bekman wrote: 2. The other request is about the "Controlling and Monitoring the Server" chapter itself. Do you think that in addition to the existing items (see below) you/we want to see other things related to this chapter. I'd be interested in seeing information on safe production updates in multi-server situations. Another interesting angle here would be how to get CVS to help with the process as much as possible. I'd also like to see ideas on how to run maintenance on data without taking down the server. For example, I've got a number of 'clean-up daemons' (basically big SQL update statements that check the consistency in a MySQL database) that I'd like to run regularly, but some of which require an exclusive lock on the tables. Finally, in the 'An Intentional Disabling of Live Scripts' section, I think a neater solution is to have an IfDefine in your .conf file, like: IfDefine maint PerlHandler Construction /IfDefine IfDefine !maint PerlHandler MyApp /IfDefine An then create aliases which add -Dmaint to calls to httpd/apachectl during the maintenance process.
Security in displaying arbitrary HTML
I'm interested in providing 'HTML email' support for my users (like HotMail, Outlook Express, Eudora 4.0, etc provide), but I'm very nervous about security. Essentially, providing HTML email involves letting any arbitrary HTML get displayed by Apache... Has anyone done this, or can anyone provide any tips on what the minimum amount of HTML laundering I need to do to avoid security holes? I say 'minimum', because I would like to maximise the amount of working HTML users can receive. I assume I don't have to worry about PHP/EmbPerl/etc tags, since by the time mod_perl is finished with it, it's too late for other handlers to step in (unless I specifically chain them). Is that right? The only potential holes I can think of are 'javascript:' URLs, which I could just filter out, and cross-site scripting URLs (does anyone have any code that recognises hrefs with potential cross-site scripting problems?) TIA, Jeremy
Re: Avoiding redefining constant subs
I've been getting frustrated by the mandatory warning created by 'use
constant' under mod_perl, caused by constant subroutines being redefined
when imported. In particular, with Archive::Zip this was adding hundreds
of
lines to my log every restart.
what version of Perl are you using?
the mandatory constant-sub-redefined warning went away in 5.004_66:
Sorry--you're quite right... it's not a mandatory warning at all any more. I
run all my scripts when developing under -w, so I still get the warning.
I think it would be useful to specifically check that the sub is not already
defined in the caller's namespace:
if (!(defined {"${pkg}::$name"})) {
import the const sub
}
This works fine for me... without it, it's very hard to notice many warnings
which provide useful diagnostic information. Worse still, some people may
just turn off -w altogether to avoid their logs filling up.
