cvs commit: modperl-2.0/t/response/TestError - New directory
stas2003/01/13 22:41:49 modperl-2.0/t/response/TestError - New directory
cvs commit: modperl-2.0/t/error - New directory
stas2003/01/13 22:41:49 modperl-2.0/t/error - New directory
cvs commit: modperl-2.0/t/response/TestError push_handlers.pm
stas2003/01/13 22:42:44
Modified:src/modules/perl modperl_callback.c
.Changes
Added: t/error push_handlers.t
t/response/TestError push_handlers.pm
Log:
prevent a segfault when push_handlers are used to push a handler into
the currently phase and switching the handler (perl-script/modperl) +
tests
Revision ChangesPath
1.52 +8 -4 modperl-2.0/src/modules/perl/modperl_callback.c
Index: modperl_callback.c
===
RCS file: /home/cvs/modperl-2.0/src/modules/perl/modperl_callback.c,v
retrieving revision 1.51
retrieving revision 1.52
diff -u -r1.51 -r1.52
--- modperl_callback.c1 Jan 2003 03:45:54 - 1.51
+++ modperl_callback.c14 Jan 2003 06:42:43 - 1.52
@@ -93,7 +93,7 @@
modperl_handler_t **handlers;
apr_pool_t *p = NULL;
MpAV *av, **avp;
-int i, status = OK;
+int i, nelts, status = OK;
const char *desc = NULL;
AV *av_args = Nullav;
@@ -173,11 +173,15 @@
};
/* XXX: deal with {push,set}_handler of the phase we're currently in */
-MP_TRACE_h(MP_FUNC, running %d %s handlers\n,
- av-nelts, desc);
+/* for now avoid the segfault by not letting av-nelts grow if
+ * somebody push_handlers to the phase we are currently in, but
+ * different handler e.g. jumping from 'modperl' to 'perl-script',
+ * before calling push_handler */
+nelts = av-nelts;
+MP_TRACE_h(MP_FUNC, running %d %s handlers\n, nelts, desc);
handlers = (modperl_handler_t **)av-elts;
-for (i=0; iav-nelts; i++) {
+for (i=0; inelts; i++) {
if ((status = modperl_callback(aTHX_ handlers[i], p, r, s, av_args)) != OK)
{
status = modperl_errsv(aTHX_ status, r, s);
}
1.100 +4 -0 modperl-2.0/Changes
Index: Changes
===
RCS file: /home/cvs/modperl-2.0/Changes,v
retrieving revision 1.99
retrieving revision 1.100
diff -u -r1.99 -r1.100
--- Changes 12 Jan 2003 04:05:48 - 1.99
+++ Changes 14 Jan 2003 06:42:44 - 1.100
@@ -10,6 +10,10 @@
=item 1.99_09-dev
+prevent a segfault when push_handlers are used to push a handler into
+the currently phase and switching the handler (perl-script/modperl)
+[Stas]
+
Add $filter-seen_eos to the streaming filter api to know when eos has
been seen, so special signatures can be passed and any data stored in
the context flushed + tests. [Stas]
1.1 modperl-2.0/t/error/push_handlers.t
Index: push_handlers.t
===
use strict;
use warnings FATAL = 'all';
use Apache::Test;
use Apache::TestUtil;
use Apache::TestRequest;
plan tests = 1;
my $location = /TestError::push_handlers;
my $expected = ok;
my $received = GET_BODY $location;
ok t_cmp($expected, $received);
1.1 modperl-2.0/t/response/TestError/push_handlers.pm
Index: push_handlers.pm
===
package TestError::push_handlers;
# This test verifies that we don't segfault when push_handlers are
# used incorrectly. Here the handler() is running under
# SetHandler modperl
# and it modifies its handler to be 'perl-script', plus pushes another
# handler to run. The result is that the first time handler() is run
# under the 'modperl' handler it returns declined, therefore Apache
# runs the registered 'perl-script' handler (which handler() has
# pushed in plus itself. So the handler() is executed again, followed
# by real_response(). Notice that it pushes yet another real_response
# callback onto the list of handlers.
#
# suprisingly the response eventually works, but this is a wrong way
# to accomplish that thing. And one of the earlier stages should be
# used to push handlers.
#
# Don't modify the handler (modperl|perl-script) during the response
# handler run-time, because if OK is not returned, the handler will be
# executed again.
use strict;
use warnings;# FATAL = 'all';
use Apache::RequestRec ();
use Apache::RequestIO ();
use Apache::RequestUtil ();
use Apache::Const -compile = qw(OK DECLINED);
sub handler {
my $r = shift;
#warn handler called\n;
$r-handler(perl-script);
$r-push_handlers(PerlResponseHandler = \real_response);
return Apache::DECLINED;
}
sub real_response {
my $r = shift;
#warn real_response called\n;
$r-content_type('text/plain');
$r-print('ok');
return Apache::OK;
}
1;
__END__
array through pages
hello everybody thank you for al your request and suggests. I have a problem with array in perl cgi .I want to pass an array trhough two pages , just like we do with a simple varaiable ; but i can't find the way to do it . Please can you give me sme ideas about it Thank you koudjo MSN Search, le moteur de recherche qui pense comme vous ! Cliquez-ici
Re: array through pages
Hi koudjo, easy: use Storable qw(freeze thaw); use MIME::Base64 qw(encode_base64 decode_base64); # to encode use: $pass_throw_var = encode_base64(freeze(\@your_array)); ... # to decode use: $your_array_ref = thaw(decode_base64($pass_throw_var)); Regards Sven. Am Mon, 2003-01-13 um 10.26 schrieb koudjo ametepe: hello everybody thank you for al your request and suggests. I have a problem with array in perl cgi .I want to pass an array trhough two pages , just like we do with a simple varaiable ; but i can't find the way to do it . Please can you give me sme ideas about it Thank you koudjo _ MSN Search, le moteur de recherche qui pense comme vous ! http://search.msn.fr/worldwide.asp
unsubscribe modperl
unsubscribe modperl -- _ Senatsverwaltung für Stadtentwicklung Informationssystem Stadt und Umwelt IXB2/Br Ulrich Brochhagen Webmaster - Internetserver Brückenstr. 6, 10179 Berlin, Raum 4.007 Tel.+49 - 30 - 9025 2125 Fax +49 - 30 - 9025 2520 e-mail: [EMAIL PROTECTED] http://www.stadtentwicklung.berlin.de/
unsubscribe modperl
unsubscribe modperl -- mail2web - Check your email from the web at http://mail2web.com/ .
Redirect or Subprocess - Problems
Hi everybody, I have a problem with Mod_Perl 1.27 on Apache 1.3.12. Wehave an Perl Transaction Handler which takes all requests. The Transaction Handler is only for redirecting to different pages. One of this pages is an embedded Perl page. Our problem is, that one request will be proceed more then 3 times from the transaction handler. After that the results will be displayed.So we will have awful performance problems. Our archivtecture is like this : --Request- Perl Transaction Handler (4 or more times)--- Content Display By Embedded Perl-- Response Has someone an idea to solve this problem ??? Greentings from Germany Marc Ian BrewerFraunhofer-Institut für Software- und SystemtechnikEmil-Figge-Str. 9144227 Dortmund Tel. 0231/9 76 77 - 419eMail: [EMAIL PROTECTED]
Re: pre-spawning database connections[newbie]
Hi Ben, Do you use Apache::DBI? I mean yes because you're using connect_on_init. Apache::DBI do not really close your DBI connection. You will get the same connection with the same connection parameters, when you call DBI-connect. All connections are cached by Apache::DBI. Yes, you should call DBI-connect. Each sub process will get it's own connections. Regards Sven. Am Mon, 2003-01-13 um 13.08 schrieb Ben Wrigley: Hi All, I'm a mod_perl newbie and just trying to understand a little more about the startup.pl files and prespawning databases. I am using the connect_on_init routine in the startup.pl which is fine. What I'm not sure is then how to use this most economically in my scripts. It seems that in the scripts you should call the DBI-connect again, but that seems to be bypassing the connection I made at startup or am I misunderstanding completely. Thanks for your help Ben
Re: Redirect or Subprocess - Problems
On Mon, 13 Jan 2003, Marc Ian Brewer wrote:
Hi everybody,
I have a problem with Mod_Perl 1.27 on Apache 1.3.12. We have an Perl
Transaction Handler which takes all requests. The Transaction Handler
is only for redirecting to different pages. One of this pages is an
embedded Perl page.
Our problem is, that one request will be proceed more then 3 times
from the transaction handler. After that the results will be
displayed. So we will have awful performance problems.
Our archivtecture is like this :
--Request- Perl Transaction Handler (4 or more
times)--- Content Display By Embedded Perl
-- Response
Has someone an idea to solve this problem ???
You do not make your problem very clear. What is performance
problems? If you mean that the httpd hangs around for a long time waiting
for the other processes to end, and thus your web server gets bogged down,
than you should look at setting up a thin front-end apache with the
mod_perl server behind. That's the subject of a good deal of info in the
mod_perl Guide and in the archives of this list.
- nick
Nick Tonkin {|8^)
Load balancers
(This may seem off topic for this list, but I'll try to bring it around before the end of the message :) We've been struggling with load balancers for a while now. My requirements are pretty simple. I have a handful of plain and mod_perl apache servers, some of which are identical and a few of which are the only ones that can perform unique functions. We want a load balancer that: * distributes requests to the server(s) that can handle them. I need to route to specific servers based on requested URL and perhaps headers and/or protocol type (SSL). * detects when a server is down and does not send request to that server. This process should be as transparent to the user as possible. * doesn't crash or lock up. We started with mod_backhand, but it frequently seemed to get confused, failing to send requests to some hosts that were perfectly able to service requests, and then locking up entirely when a back-end server became overloaded. I also found the diagnostics to be inaccurate and misleading (or at least poorly documented), and the documentation in general to be lacking. Next, we tried pound, which looked promising due to its simplicity. Unfortunately, it too seems to freak out at least once a day, filling its log with errors like this: @40003e1ef2461015ce34 warning: error read from 123.123.123.123: Success @40003e1ef24621631c8c warning: cannot read headers. got [] @40003e1ef2462163d424 warning: error read from 123.123.123.123: Success @40003e1ef2462902488c warning: cannot read headers. got [] @40003e1ef246290332ec warning: error read from 123.123.123.123: Success @40003e1ef246312225dc warning: cannot read headers. got [] @40003e1ef2463122be34 warning: error read from 123.123.123.123: Success @40003e1ef24706782a84 warning: cannot read headers. got [] @40003e1ef2470678bef4 warning: error read from 123.123.123.123: Success @40003e1ef247176094bc warning: cannot read headers. got [] We've been in contact with the author and have tried digging around in and/or patching the source code with no improvement so far. But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) This brings me to the mod_perl angle. Has anyone ever tried using a slimmed down mod_perl server as a load balancer? Is this feasible? Making routing decisions is obviously the easy part when using mod_perl, but would those mod_perl apache process just be too big and too slow to proxy requests efficiently? And how would they deal with detecting back-end servers that have failed? (We're running various versions of Linux on our servers, if that makes any difference.) So...suggestions? How are other people handling load balancing? -John
Re: Memory Usage
Hi!
On Mon, Jan 13, 2003 at 10:15:58AM -0500, Chris Faust wrote:
Sometimes it will run for days without an issue and other times the machine
will die, normally because of out of memory problems.
..
Any help or direction would be appreciated.
There is quite a lot of documentation about memory issues at
http://perl.apache.org/docs/1.0/guide/performance.html
or in the mod_perl Site in general.
http://perl.apache.org
Did you RTFM?
--
#!/usr/bin/perl http://domm.zsi.at
for(ref bless{},just'another'perl'hacker){s-:+-$-gprint$_.$/}
Re: Load balancers
On 1/13/03 11:12 AM, Ask Bjoern Hansen wrote: On Mon, 13 Jan 2003, John Siracusa wrote: So...suggestions? How are other people handling load balancing? With hardware load balancers. :-) Sure, rub it in ;) You forgot to include the information about number of servers, requests per second at peak times, reponse sizes, etc etc. * four servers * peak of 30 hits/sec (fluctuates though) * response sizes vary: most are less than 20K, but a significant minority are 500K-5MB (i.e. file downloads) But really, stability is out #1 concern right now. Slowdown under load is much more desirable that error under load or lock-up under load :) -John
Re: Load balancers
John Siracusa wrote: But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) I've always used hardware ones. I believe big/ip does everything you need. However, if I were going to use a software solution I would be looking at Linux Virtual Server, probably starting with the Red Hat offering based on it. This brings me to the mod_perl angle. Has anyone ever tried using a slimmed down mod_perl server as a load balancer? Is this feasible? Not a good idea. It would eat all your memory. You'd be much better off with the mod_rewrite solution mentioned in this thread. - Perrin
Re: Load balancers
On Mon, 13 Jan 2003, John Siracusa wrote:
But meanwhile, we're still open to alternatives. Surprisingly, there don't
seem to be many (software) options. (A hardware load balancer is not an
option at his time, but I'll also take any suggestions in that area :)
My experience in this area led quickly to the conclusion that there is no
viable software solution, at least when you get a bit of traffic.
We quickly moved to hardware load balancing and did very well with the
Coyote Point Equalizer, which was affordable and powerful. Also the
developers (Bill Kish is/was the lead developer) were very responsive and
worked with us to tweak as necessary.
After a while we got one of those loser CFOs who was golf pals with the
new COO or something, and he bought a bunch of Foundry Systems Server
Irons that worked alright but were orders of magnitude more expensive.
- nick
Nick Tonkin {|8^)
Re: Load balancers
On Monday 13 January 2003 08:03 am, John Siracusa wrote: (We're running various versions of Linux on our servers, if that makes any difference.) So...suggestions? How are other people handling load balancing? -John Hey John, Take a look at the LVS project, www.foundrynetworks.com and www.f5networks.com. I've used Foundry and F5 gear for load-balancing with session persistence and had really good results - cost might be an issue with both of these units. I have been putting together an LVS (DR) solution for some work that's similar to what your doing but haven't finished it yet. Steve
Re: Load balancers
John Siracusa wrote: [snip] But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) [snip] So...suggestions? How are other people handling load balancing? Squid in accelerator mode for ordinary http and the linux virtual server stuff for https. Works nicely for us (about a couple of million requests a day, mostly http). Andy -- Andy Osborne Vertical B2B Communities Senior Internet Engineer Sift Group100 Victoria Street, Bristol BS1 6HZ tel:+44 117 915 9600 fax:+44 117 915 9630 http://www.sift.co.uk
Re: Load balancers
The Load Balancing section of this doc might help: http://httpd.apache.org/docs/misc/rewriteguide.html Just straight apache + mod_rewrite could be the simple solution you seek. The Proxy Throughput Round-Robin example shows how to add a script to do mapping as well - could work for your availability detection requirement. We've also used mod_rewrite in a lightweight reverse proxy as a stopgap when load balancing hardware was not available. One option for detecting live backend servers is to use the prg rewrite map like Paolo says, where a script does some sort of availability check. Another option which we have used is the rnd map where a server is selected randomly from a flat file, where that file can be kept updated by an external (non-Apache) process like a cron job that checks backend server availability (mod_rewrite notices on-the-fly when that file has changed). Larry Leszczynski [EMAIL PROTECTED]
RE: Load balancers
Hi John -- But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) Why is hardware not an option? Cost? If so, I'd take a look at the system Nick recommended -- the Coyote Point Equalizer: http://www.coyotepoint.com/equalizer.htm A few years ago I bought their E-250. It cost about $3,200, which was nothing compared to the other solutions out there at the time. This load balancing system (LBS) was for a corporate intranet with a user base of 50k to 60k users, with the expectation that it would be capable of handling a third of the total user base at any time. The CPE-250 handled the job with ease. We initially rolled out with four separate Web Application Server machines (LAMP architecture), and a procedure to allow us to add additional machines in four hours, if need be. The system has been in production for about two years. In that time we have had individual servers crash, but the intranet has had less than 15 minutes of downtime since we implemented the CPE. Regarding software solutions, mod_perl or otherwise: I initially looked at this, and ultimately rejected it as an option. No robust systems had emerged at the time. Based on a cost/performance comparison against the CP product I calculated it would cost far more than $3,200 to roll our own system which would provide the configurability, ease of use and reliability of the available hardware solution. FWIW, the Equalizer is implemented as a rack-mountable BSD machine running custom software. The software implements load-balancing via dynamic port-forwarding (*NOT* an HTTP proxy), so it is capable of forwarding any TCP service (SMTP, database, etc.). The GUI is web-based, very easy to use, and includes graphical reporting of activity. Load balancing can be configured to be purely round-robin, but the CPE features a very effective dynamic adaptive load balancing system based on server response time which is very effective. It has been a couple years since I talked to Coyote Point about their product. I would check to see if it is still cost-competitive. HTH, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: Load balancers
Hi John, I use wackamole (http://www.backhand.org/wackamole/) on my (2) front end servers with 6 IP addresses doing Round Robin (RR) DNS. This is suffichent for loadbalencing the light (HTML / IMAGE only) front end. I had a issue with this once, it got the IP address allocation confused, but otherwise it's been running fine for a year. The front end machines proxy pass to a backend App (mod_perl) server, they do this by name e.g. backend-server, we then have a script that monitors the backend-server and alters the /etc/resolv.conf file if server 1 goes down and alters the ip address to our second machine. Our second machine is actually the DB machine so this is just an emergency fall back until the apps server can be brought up again. Thankfully I've never had to test this in anger! Leo more info: front ends run debian on standard compaq rack mount servers backend 2 Sun Solaris boxes, 1 with mod_perl other with MySQL traffic: 4 / mill pages a month, 50 / 60 hits a second peak Lots and lots of cacheing on the app and indexes on the DB.
Re: Load balancers
Perrin Harkins [EMAIL PROTECTED] wrote: John Siracusa wrote: But meanwhile, we're still open to alternatives. Surprisingly, there don't seem to be many (software) options. (A hardware load balancer is not an option at his time, but I'll also take any suggestions in that area :) I've always used hardware ones. I believe big/ip does everything you need. However, if I were going to use a software solution I would be looking at Linux Virtual Server, probably starting with the Red Hat offering based on it. We're currently using a couple of big/ip switches, but don't have web servers behind them yet (using them for smtp and such at the moment). We're looking at using them or one of the switches from NetScaler (netscaler.com) which looked quite impressive. NetScalar is really built for web servers (or so it seems from our meetings with them) while big/ip is a more generic solution. Both big/ip and netscalar allow sessions to be bound to a backend server, iirc, which can be a nice optimization (which we haven't had to take advantage of yet). -- James Smith [EMAIL PROTECTED], 979-862-3725 Texas AM CIS Operating Systems Group, Unix
Re: Load balancers
John Siracusa wrote: We've been struggling with load balancers for a while now. My requirements are pretty simple. I have a handful of plain and mod_perl apache servers, So...suggestions? How are other people handling load balancing? I have tested pen. It's easy to set up and works fine so far. The server has only 70k hits/day. I don't know how will it work under heavy load. The purpose was to keep working if one server was down. Pen is a load balancer for simple TCP-based protocols such as HTTP or SMTP. It allows several servers to appear as one to the outside. It automatically detects servers that are down and distributes clients among the available servers. This gives high availability and scalable performance. http://siag.nu/pen/
Inserting a handler in stack of handlers.
Hello All! Short synopsis: How to push handler just after handler that working now? More about the problem. I've got main handler, that forms stack of handlers from query string by calling push_handlers(). Then each module doing his job. Some handlers needs to put another hook just after they end thier job. I do it with direct call to SomeModule::handler($r), it works for me, but it's rude back(as i think). I've tried to do the same with push_handlers, but this function push handlers only at the end of handler's list and content appears at the bottom of page :( Any suggestion? Best regards, Ruslan.
RE: OSCON ideas - missing proceedings
Mark Schoonover writes: Are there plans to do the University again?? Every year or two I try again to revive it. Your message started me again this year. No promises, but we're looking into it. Thanks Nat for the work you did down here!! Thanks for your kind words. I love every minute of being at a conference, so it's hard to describe it as work.[*] Nat [*] Just don't ask me about the minutes organizing the conference before it all happens :-)
Re: Load balancers
That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. We're investigating LVS right now. It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) We chose pound over pen, but we may revisit pen again. I suspect we will have similar problems with our expected load, however. Whackamole, fun name aside, does not seem to be what we need. We don't need the caching part of Squid, and I wasn't aware it did load balancing too. I'll check it out. Thanks for all the info, and please feel free to send me more, especially if there's some gem of a software load balancer out there somewhere... :) -John
Re: Load balancers
John Siracusa wrote: The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. ... It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) The mod_rewrite option is a Perl solution. You would write some Perl code to manage the availability checks and optionally to provide a new load-balancing algorithm. It's very similar to using mod_perl for it, except you get to skip writing lots of annoying proxy code. - Perrin
Re: Load balancers
On 1/13/03 1:04 PM, Perrin Harkins wrote: John Siracusa wrote: The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. ... It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) The mod_rewrite option is a Perl solution. You would write some Perl code to manage the availability checks and optionally to provide a new load-balancing algorithm. It's very similar to using mod_perl for it, except you get to skip writing lots of annoying proxy code. But in a full-fledged mod_perl solution, I could back out gracefully and retry another server if I happened to initially choose a dead server before my dead server detection code caught it. With the rewrite solution, I can't think of a reasonable way to keep users from ever getting passed to dead server. (Obviously I can't check the server I plan to send to on every request :) -John
Re: Inserting a handler in stack of handlers.
Ruslan U. Zakirov wrote: Hello All! Short synopsis: How to push handler just after handler that working now? More about the problem. I've got main handler, that forms stack of handlers from query string by calling push_handlers(). Then each module doing his job. Some handlers needs to put another hook just after they end thier job. I do it with direct call to SomeModule::handler($r), it works for me, but it's rude back(as i think). I've tried to do the same with push_handlers, but this function push handlers only at the end of handler's list and content appears at the bottom of page :( Any suggestion? Best regards, Ruslan. you can't really do this now I don't think. generally, the way would be to use get_handlers() to get the current chain, then use set_handlers() to set it to what you want it to be (inserting logic to splice the added handler where you want it). unfortunately, you can't call set_handlers() for the current phase, so adding another handler right after the current one runs probably isn't possible. HTH --Geoff
RE: Load balancers
Hi John -- That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. Actually, $3200 was a STEAL! Cisco's Local Director was in the mid five-figures at the time, IIRC. :-) It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) Ah yes... Fun, for sure. I would have loved to make one, but that was just not the correct thing to do at the time. TTYL, -Jesse- -- Jesse Erlbaum The Erlbaum Group [EMAIL PROTECTED] Phone: 212-684-6161 Fax: 212-684-6226
Re: Load balancers
John Siracusa wrote: But in a full-fledged mod_perl solution, I could back out gracefully and retry another server if I happened to initially choose a dead server before my dead server detection code caught it. That sounds cool, but how important is it really? I'm not sure any of these solutions (including the commercial ones) do that level of seamless failover effectively. - Perrin
Re: Load balancers
On 1/13/03 1:28 PM, Perrin Harkins wrote: John Siracusa wrote: But in a full-fledged mod_perl solution, I could back out gracefully and retry another server if I happened to initially choose a dead server before my dead server detection code caught it. That sounds cool, but how important is it really? I'm not sure any of these solutions (including the commercial ones) do that level of seamless failover effectively. The five-figure ones better! ;) (they should also make me lunch) -John
RE: Load balancers
On Mon, 13 Jan 2003, Jesse Erlbaum wrote: That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. Actually, $3200 was a STEAL! Cisco's Local Director was in the mid five-figures at the time, IIRC. :-) No, the local directors were that never that much, maybe low five-figures, like $12-15K or so. Boston.com was running on a pair of (saturated) low end LD's up until around Y2K, when we revamped the load balancing architecture and switched over to ArrowPoint (which then got gobbled up by Cisco) load distributing switches. At that time, a past-its-prime LD could be had for $3K or so, and a state-of-the-art Arrowpoint CS-150 ran around $12K (but we installed the higher capacity CS-800, which ran around $30K each -- just don't ask them to support HTTP 100% correctly... ;^) I've been out of touch with hardware load balancing equipment over the last couple of years, but if the decline in hardware costs of these things has continued, I'd guess a reasonably chunky solution can be had in the $5-7K range at this point. (Of course, you'd need to multiply that cost by two for complete redundancy...) [ Love to hear from someone with current knowledge if this is the case though...] -- Steve Reppucci [EMAIL PROTECTED] | Logical Choice Software http://logsoft.com/ | =-=-=-=-=-=-=-=-=-=- My God! What have I done? -=-=-=-=-=-=-=-=-=-=
Re: Load balancers
I've developed an embedded (Linux) load balancer solution. It's small form factor, runs on 8MB flash minimum, no hard drive, no fan so no wories about hard drive failure. It is LVS NAT (L4) based with configurable monitoring service. It load balances any port you want. I've deployed it to load balance web servers, mail server, etc. One of the busier sites is using it to load balance 6 web servers with 5M average hits a day. They've had twice that traffic and the load balancer seems to be fine. LVS says it can balance up to 24 servers. Cost $500. John Siracusa wrote: That's for all the info so far. To answer some questions, hardware is a cost issue right now. It's somewhat scary that $3,200 was a reasonable price several years ago, but I suppose it could be worse. We will investigate further. The mod_rewrite solutions lack dead server detection, and that's something I'd rather not try to roll on my own, especially after seeing how well (or not, actually) existing software solutions do. But I've added it to the list. We're investigating LVS right now. It's kind of disappointing to hear that the mod_perl solution it probably not feasible. Perl solutions are always more fun to implement ;) We chose pound over pen, but we may revisit pen again. I suspect we will have similar problems with our expected load, however. Whackamole, fun name aside, does not seem to be what we need. We don't need the caching part of Squid, and I wasn't aware it did load balancing too. I'll check it out. Thanks for all the info, and please feel free to send me more, especially if there's some gem of a software load balancer out there somewhere... :) -John
Writing to stdin of subprocess under modperl 1.0 fails :/
Hello all,
I'm having trouble with following script under mod_perl
1.26 (perl 5.6.1).
use strict ();
use IO::Handle ();
use IPC::Open2 ();
###
#
# Check $username $password against YP database
sub check_password {
my ($username, $passwd) = @_;
my($rh, $wh);
# open bi-dir pipe
my $pid = IPC::Open2::open2($rh, $wh,
/usr/local/bin/ypmatch_by_ilmo, $username);
$write-printflush($passwd\n);
my $ret = $read;
close($write);
close($read);
waitpid $pid, 0;
return $ret;
}
This snippet works as expected when run from command line. When run under
mod_perl the program spawned by open2 receives nothing but EOF from its
stdin, but is still able to provide its output to caller process.
What I'm doing wrong? Is there something similar to
Apache::SubProcess::spawn_proc_prog in MP1.x?
--
Antti Haapala
RE: Load balancers
On Mon, 13 Jan 2003, Jesse Erlbaum wrote:
Hi John --
That's for all the info so far. To answer some questions,
hardware is a cost issue right now. It's somewhat scary that
$3,200 was a reasonable price several years ago, but I
suppose it could be worse. We will investigate further.
Actually, $3200 was a STEAL! Cisco's Local Director was in the mid
five-figures at the time, IIRC. :-)
So it was that range, not $15K as I thought. Yep, I remember that we got
both of them for far far less than a Cisco, and much better support. We
had the chief developer's cell phone number, and he would help us out all
the time. At the time we were growing so fast we were almost a test case
for them ... we took them up to at least 70 million dynamic requests a
day, with stickiness required, using the adaptive balancing, with no real
problems, before moving to the Foundry Systems hardware.
- nick
Nick Tonkin {|8^)
1.3.27 DSO hassles
All, Having been successfully using modperl for the last 2 years statically linked with Apache, I have been trying again to make modperl work with 1.3.27 when the Apache core modules are loaded as DSOs. There has been some traffic in the past on this subject and I checked the archives and followed through on some of the suggestions - Recompiled perl 5.6 with the --Ubincompat5005 option for specific use with modperl - Setup modperl using the perl compiled with --Ubincompat5005 - I use the following configure options for the APACI for Apache 1.3.27 ./configure --prefix=/opt/apache-so \ --enable-rule=SHARED_CORE \ --enable-module=most \ --enable-shared=max \ --activate-module=src/modules/perl/libperl.a \ --disable-shared=perl Apache is compiled and statically links modperl without any problems (Solaris 2.6). When Apache is executed, I receive this error: fatal: relocation error: file /opt/apache-so/libexec/mod_negotiation.so: symbol __floatdisf: referenced symbol not found I have tried the recommendations, specifically the issue with perl's malloc on Solaris which can be corrected with the --Ubincompat5005 option. Any ideas Thanks Alan
Re: Unregister streamed output filters
Esteban Fernandez Stafford wrote: On Sat, 11 Jan 2003, Stas Bekman wrote: Esteban Fernandez Stafford wrote: Hello all, is there a way to unregister a streamed filter? I have seen this in many (all?) apache (C programmed) filters; they are able of declining the filtering of a certain stream, for example, when they do not know how to handle a certain content type. In the apache api this is done with ap_remove_output_filter(f). Is there something similar in mp2? Not at this moment, but hopefully it'll be supported soon. Since you need this feature, telling us in what situation you'd like to remove a filter will help us to build a better test case and provide a good real-world example for documentation. The easiest example that comes to mind is a filter for text/html that performs some sort of transformation. This filter should unregister for any content type that is not text/html. Browsing through some apache code I have found two ways of doing this. One involves the ap_remove_output_filter function (modules/filters/mod_deflate.c) and the other returns a DECLINED at a cetrtain point (modules/filters/mod_include.c). I am not sure about the internals of each approach but I thought it might help. Yup, I was reading mod_deflate just yesterday and saw it too. Thanks! It occurs to me just now that it maybe also be possible to do this staticaly in httpd.conf. Something like: PerlOutputFilterHandler MyApache::MyHtmlFilter text/html Good idea, eventually there should be TIMTOWTDI ;) __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: mod_perl mod_jk
MATHIHALLI,MADHUSUDAN (HP-Cupertino,ex1) wrote: I think I found the problem - it was because of a LoadModule directive that was defined in 1.99-07, but renamed to PerlLoadModule in 1.99-08. The reason why I ran into the problem of unable to load mod_jk, was because I was trying to load mod_jk after the mod_perl is loaded. What was happening is that the mod_perl directives' gets registered with Apache as soon as mod_perl is loaded. With the LoadModule defined by mod_perl also, apache's ap_find_command_in_modules command returns a pointer to mod_perl's loadmodule function (modperl_cmd_load_module), instead of apache's load_module function (because mod_perl is the first in the modp list). And then, to complicate the matters, the cmd-req_override is not set to EXEC_ON_READ - thus mod_jk is never loaded.. Question: Is it okay if I rename LoadModule to PerlLoadModule in 1.99-07, and continue with it ?. Does it need more modifications than just renaming it ?.. Should be just that (though some tests will fail, but ignore that) Index: src/modules/perl/mod_perl.c === RCS file: /home/cvs/modperl-2.0/src/modules/perl/mod_perl.c,v retrieving revision 1.141 retrieving revision 1.142 diff -u -r1.141 -r1.142 --- src/modules/perl/mod_perl.c 17 Sep 2002 02:05:21 - 1.141 +++ src/modules/perl/mod_perl.c 7 Oct 2002 02:05:43 - 1.142 @@ -636,7 +636,7 @@ MP_CMD_DIR_RAW_ARGS_ON_READ(=cut, pod_cut, End of POD), MP_CMD_DIR_RAW_ARGS_ON_READ(__END__, END, Stop reading config), -MP_CMD_SRV_RAW_ARGS(LoadModule, load_module, A Perl module), +MP_CMD_SRV_RAW_ARGS(PerlLoadModule, load_module, A Perl module), #ifdef MP_TRACE MP_CMD_SRV_TAKE1(PerlTrace, trace, Trace level), #endif Thanks for the debugging pointers, You are welcome ;) __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: unsubscribe modperl
[EMAIL PROTECTED] wrote: unsubscribe modperl The information to unsubscribe is in the header of the email: list-help: mailto:[EMAIL PROTECTED] list-unsubscribe: mailto:[EMAIL PROTECTED] list-post: mailto:[EMAIL PROTECTED] You have to unsubscribe yourself, I can't help you, so please don't email me. ;) __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: pre-spawning database connections[newbie]
Sven Geisler wrote: Hi Ben, Do you use Apache::DBI? I mean yes because you're using connect_on_init. Apache::DBI do not really close your DBI connection. You will get the same connection with the same connection parameters, when you call DBI-connect. All connections are cached by Apache::DBI. Yes, you should call DBI-connect. Each sub process will get it's own connections. What Sven said, plus it helps to RTFM before asking the list. Please read: http://perl.apache.org/docs/1.0/guide/databases.html#Apache__DBI___Initiate_a_persistent_database_connection Am Mon, 2003-01-13 um 13.08 schrieb Ben Wrigley: Hi All, I'm a mod_perl newbie and just trying to understand a little more about the startup.pl files and prespawning databases. I am using the connect_on_init routine in the startup.pl which is fine. What I'm not sure is then how to use this most economically in my scripts. It seems that in the scripts you should call the DBI-connect again, but that seems to be bypassing the connection I made at startup or am I misunderstanding completely. Thanks for your help Ben -- __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: Load balancers
John Siracusa wrote: (This may seem off topic for this list, but I'll try to bring it around before the end of the message :) We've been struggling with load balancers for a while now. It seems that most experts hang at [EMAIL PROTECTED] It's also preferrable that the hw solutions will be discussed there. [...] This brings me to the mod_perl angle. Has anyone ever tried using a slimmed down mod_perl server as a load balancer? Is this feasible? Making routing decisions is obviously the easy part when using mod_perl, but would those mod_perl apache process just be too big and too slow to proxy requests efficiently? And how would they deal with detecting back-end servers that have failed? As someone has mentioned, squid is doing that (See the guide). The good thing is that it spawns the process and never quits it, so you don't have an overhead of perl startup for each request. Indeed it'll use a lot of memory. But may be toying with mod_perl 2.0 / threaded mpm will prove to be more memory efficient. Also PPerl comes to a mind. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: Writing to stdin of subprocess under modperl 1.0 fails :/
Antti Haapala wrote: Hello all, I'm having trouble with following script under mod_perl 1.26 (perl 5.6.1). use strict (); use IO::Handle (); use IPC::Open2 (); [...] Use IPC::Run instead of the IPC::Open* family, it surely works and a much more flexible tool! Probably need to add an item to the guide's troubleshooting. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: 1.3.27 DSO hassles
Sinclair, Alan (CORP, GEAccess) wrote: All, Having been successfully using modperl for the last 2 years statically linked with Apache, I have been trying again to make modperl work with 1.3.27 when the Apache core modules are loaded as DSOs. There has been some traffic in the past on this subject and I checked the archives and followed through on some of the suggestions - Recompiled perl 5.6 with the --Ubincompat5005 option for specific use with modperl - Setup modperl using the perl compiled with --Ubincompat5005 - I use the following configure options for the APACI for Apache 1.3.27 ./configure --prefix=/opt/apache-so \ --enable-rule=SHARED_CORE \ --enable-module=most \ --enable-shared=max \ --activate-module=src/modules/perl/libperl.a \ --disable-shared=perl Apache is compiled and statically links modperl without any problems (Solaris 2.6). When Apache is executed, I receive this error: fatal: relocation error: file /opt/apache-so/libexec/mod_negotiation.so: symbol __floatdisf: referenced symbol not found I have tried the recommendations, specifically the issue with perl's malloc on Solaris which can be corrected with the --Ubincompat5005 option. There were some suggestions offered in this thread (large files CFLAGS?) http://marc.theaimsgroup.com/?t=10168427183r=1w=2 Though I didn't see a success report. If somebody on Solaris 2.6 were able to get it to work, please chime in. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
Re: 1.3.27 DSO hassles
In message [EMAIL PROTECTED], Stas Bekman writes: There were some suggestions offered in this thread (large files CFLAGS?) http://marc.theaimsgroup.com/?t=10168427183r=1w=2 Though I didn't see a success report. If somebody on Solaris 2.6 were able to get it to work, please chime in. Apache/modperl works fine for me as DSO under Solaris 7,8,9. 2.6 is EOL, but I suspect it will work with latest recommended patches. Using gcc2.95.3 Using perl5.8, but also worked under 5.6.1 and 5.00503 Build openssl ./config --prefix=/opt/openssl \ --openssldir=/opt/openssl shared env LD_RUN_PATH=/opt/openssl/lib make Build Apache/mod_ssl ./configure --with-apache=../apache_1.3.27 cd ../apache_1.3.27 env SSL_BASE=/opt/openssl ./configure --prefix=/opt/apache \ --enable-module=most --enable-shared=max # Add the var CFLAGS=-D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 to the env command if you want largefiles or built your perl that way # Edit src/modules/ssl/Makefile and add -R$(SSL_LIBDIR) to SSL_LDFLAGS # If using gcc2, edit src/modules/proxy/Makefile and add -L/path/to/gcc-lib/$host_info/$gcc_version/ -lgcc to LIBS_SHLIB Build modperl perl Makefile.PL USE_APXS=1 WITH_APXS=/opt/apache/bin/apxs \ EVERYTHING=1 PERL_TRACE=1 John [EMAIL PROTECTED]
Re: mod_perl 2.0 and print/send_http_header method SEGFAULT
Jérôme Augé wrote:
Hi,
I'm beginning with mod_perl (mod_perl-1.99_05 + Apache 2.0.40 from
RedHat 8.0) and I want to write a module for rewriting the documents
that passes through the Apache proxy. So, I looked at the
Apache::AdBlocker
(http://perl.apache.org/docs/tutorials/tips/mod_perl_tricks/mod_perl_tricks.html#A_Banner_Ad_Blocker)
module and I'm facing some problems for writing the content of the
documents back to the client.
My main problem is that I get a SEGFAULT when calling the $r-print()
or $r-send_http_header() method.
I get the request, copy the headers from headers_in, make my own
request with LWP, copy the headers to headers_out, then it SEGFAULT
when writing the document ... Are this methods deprecated/not fully
implemented ? what is the correct way to write data to the client ?
The other problem is that if I use the $r-push_handlers(PerlHandler =
\proxy_handler) mechanism, my proxy_handler() function is never
called, so I do the work directly into the handler sub, is this ok ?
I attached my test module below (I register it with a PerlTransHandler
Apache::Plop statement in httpd.conf)
After making your example work, I don't see any segfaults. Please try
again with modperl-1.99_08 which was released a few days ago.
I've attached Plop.pm that apparently works. Hope that this is what you
wanted to accomplish. I've used the following config:
Location /plop/
SetHandler perl-script
PerlHeaderParserHandler Apache::Plop
/Location
Now to your code:
1. You can't push_handlers when you are inside a response handler.
Use PerlHeaderParserHandler instead
2. $r-headers_in-do() expects a return value and will abort on 0; see
the attached code
also it should be:
$request-header( $_[0] = $_[1] );
instead of:
$request-header( {$_[0]} = $_[1] );
have you looked at error_log? You'd have seen that error reported.
3. This is not good:
my $request = HTTP::Request-new( $r-method, $r-uri);
since you don't the whole url. Use this instead:
my $request = HTTP::Request-new( $r-method, $r-construct_url);
this requires 'use Apache::URI'
4. Finally I've used a special header: (which can be anything)
$request-header( GetReal = 1 );
to know that now I'm inside the real request.
Hope that this helps.
Also you might want to use a sub-request rather than a heavy weighted LWP
to accomplish what you do.
__
Stas BekmanJAm_pH -- Just Another mod_perl Hacker
http://stason.org/ mod_perl Guide --- http://perl.apache.org
mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com
http://modperlbook.org http://apache.org http://ticketmaster.com
package Apache::Plop;
use strict;
use Apache::RequestRec;
use Apache::RequestIO;
use Apache::RequestUtil;
use Apache::Const;
use Apache::ServerUtil;
use Apache::Response;
use Apache::URI;
use APR::Table;
use LWP::UserAgent;
my $ua = LWP::UserAgent-new();
sub handler {
my $r = shift;
if( $r-proxyreq ) {
return Apache::DECLINED;
}
print STDERR Good, this is a proxyreq ...\n;
$r-handler(perl-script); #ok, let's do it
$r-push_handlers(PerlResponseHandler = \proxy_handler);
return Apache::OK;
}
sub proxy_handler {
my $r = shift;
if( $r-method ne GET ) {
return Apache::DECLINED;
}
print STDERR Good, this is a GET method ...\n;
if ( ($r-headers_in-get('GetReal')||0) == 1) {
$r-content_type('text/plain');
print hey;
return Apache::OK;
}
# prepare the real request
my $request = HTTP::Request-new( $r-method, $r-construct_url);
# copy headers from client request
my %headers_in;
print STDERR -- client headers --\n;
$r-headers_in()-do(
sub {
warn $_[0]: $_[1]\n;
$headers_in{ $_[0] } = $_[1];
$request-header( $_[0] = $_[1] );
return 1;
}
);
print STDERR -- end --\n;
# make the real request myself
$ua-agent( $headers_in{ 'User-Agent' } );
$request-header( GetReal = 1 );
warn $request-as_string;
my $response = $ua-request( $request );
if ( ! $response-is_success() ) {
print STDERR == ERROR ==\n;
return Apache::DECLINED;
}
print STDERR -- server headers --\n;
my %headers_out;
$response-headers()-scan(
sub {
print STDERR $_[0]: $_[1]\n;
$headers_out{$_[0]} = $_[1];
}
);
print STDERR -- end --\n;
# simply override the content
my $content = $response-content;
$content = htmlbodyplop/body/html;
# adjust the headers for the new content
$headers_out{ 'Content-length' } = length( $content );
$headers_out{ 'Content-type' } = 'text/html';
# copy the modified response headers back to Apache
foreach (keys %headers_out) {
$r-headers_out-{$_} = $headers_out{$_};
}
Re: mod_perl 2.0 and print/send_http_header method SEGFAULT
Stas Bekman wrote: Jérôme Augé wrote: [...] After making your example work, I don't see any segfaults. Please try again with modperl-1.99_08 which was released a few days ago. [...] 1. You can't push_handlers when you are inside a response handler. Use PerlHeaderParserHandler instead I've played some more with your original code and did find the segfault you were talking about. Though it happens when you use push_handlers in the same phase to which you push to. Hence I didn't see it in the code that I've fixed. So most likely your _05 version will work just fine with the version that I've posted in my original reply. Meanwhile I'm looking at how it's the best to prevent from the segfault to happen and push_handlers() be used in a wrong place. __ Stas BekmanJAm_pH -- Just Another mod_perl Hacker http://stason.org/ mod_perl Guide --- http://perl.apache.org mailto:[EMAIL PROTECTED] http://use.perl.org http://apacheweek.com http://modperlbook.org http://apache.org http://ticketmaster.com
