Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Perrin Harkins]

On Wed, 17 Jan 2001, Sam Horrocks wrote:
> If in both the MRU/LRU case there were exactly 10 interpreters busy at
> all times, then you're right it wouldn't matter.  But don't confuse
> the issues - 10 concurrent requests do *not* necessarily require 10
> concurrent interpreters.  The MRU has an affect on the way a stream of 10
> concurrent requests are handled, and MRU results in those same requests
> being handled by fewer interpreters.

On a side note, I'm curious about is how Apache decides that child
processes are unused and can be killed off.  The spawning of new processes
is pretty agressive on a busy server, but if the server reaches a steady
state and some processes aren't needed they should be killed off.  Maybe
no one has bothered to make that part very efficient since in normal
circusmtances most users would prefer to have extra processes waiting
around than not have enough to handle a surge and have to spawn a whole
bunch.

- Perrin

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Joe Schaefer]

Roger Espel Llima <[EMAIL PROTECTED]> writes:

> "Jeremy Howard" <[EMAIL PROTECTED]> wrote:

I'm pretty sure I'm the person whose words you're quoting here,
not Jeremy's.

> > A backend server can realistically handle multiple frontend requests, since
> > the frontend server must stick around until the data has been delivered
> > to the client (at least that's my understanding of the lingering-close
> > issue that was recently discussed at length here). 
> 
> I won't enter the {Fast,Speedy}-CGI debates, having never played
> with these, but the picture you're painting about delivering data to
> the clients is just a little bit too bleak.

It's a "hypothetical", and I obviously exaggerated the numbers to show
the advantage of a front/back end architecture for "comparative benchmarks" 
like these.  As you well know, the relevant issue is the percentage of time 
spent generating the content relative to the entire time spent servicing 
the request.  If you don't like seconds, rescale it to your favorite 
time window.

> With a frontend/backend mod_perl setup, the frontend server sticks
> around for a second or two as part of the lingering_close routine,
> but it doesn't have to wait for the client to finish reading all the
> data.  Fortunately enough, spoonfeeding data to slow clients is
> handled by the OS kernel.

Right- relative to the time it takes the backend to actually 
create and deliver the content to the frontend, a second or
two can be an eternity.  

Best.
-- 
Joe Schaefer

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Roger Espel Llima]

"Jeremy Howard" <[EMAIL PROTECTED]> wrote:
> A backend server can realistically handle multiple frontend requests, since
> the frontend server must stick around until the data has been delivered
> to the client (at least that's my understanding of the lingering-close
> issue that was recently discussed at length here). 

I won't enter the {Fast,Speedy}-CGI debates, having never played
with these, but the picture you're painting about delivering data to
the clients is just a little bit too bleak.

With a frontend/backend mod_perl setup, the frontend server sticks
around for a second or two as part of the lingering_close routine,
but it doesn't have to wait for the client to finish reading all the
data.  Fortunately enough, spoonfeeding data to slow clients is
handled by the OS kernel.

-- 
Roger Espel Llima, [EMAIL PROTECTED]
http://www.iagora.com/~espel/index.html

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Sam Horrocks]

This is planned for a future release of speedycgi, though there will
probably be an option to set a maximum number of bytes that can be
bufferred before the frontend contacts a perl interpreter and starts
passing over the bytes.

Currently you can do this sort of acceleration with script output if you
use the "speedy" binary (not mod_speedycgi), and you set the BufsizGet option
high enough so that it's able to buffer all the output from your script.
The perl interpreter will then be able to detach and go handle other
requests while the frontend process waits for the output to drain.

 > Perrin Harkins wrote:
 > > What I was saying is that it doesn't make sense for one to need fewer
 > > interpreters than the other to handle the same concurrency.  If you have
 > > 10 requests at the same time, you need 10 interpreters.  There's no way
 > > speedycgi can do it with fewer, unless it actually makes some of them
 > > wait.  That could be happening, due to the fork-on-demand model, although
 > > your warmup round (priming the pump) should take care of that.
 > >
 > I don't know if Speedy fixes this, but one problem with mod_perl v1 is that
 > if, for instance, a large POST request is being uploaded, this takes a whole
 > perl interpreter while the transaction is occurring. This is at least one
 > place where a Perl interpreter should not be needed.
 > 
 > Of course, this could be overcome if an HTTP Accelerator is used that takes
 > the whole request before passing it to a local httpd, but I don't know of
 > any proxies that work this way (AFAIK they all pass the packets as they
 > arrive).

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Ask Bjoern Hansen]

On Thu, 21 Dec 2000, Sam Horrocks wrote:

>  > Folks, your discussion is not short of wrong statements that can be easily
>  > proved, but I don't find it useful.
> 
>  I don't follow.  Are you saying that my conclusions are wrong, but
>  you don't want to bother explaining why?
>  
>  Would you agree with the following statement?
> 
> Under apache-1, speedycgi scales better than mod_perl with
> scripts that contain un-shared memory 

Maybe; but for one thing the feature set seems to be very different
as others have pointed out. Secondly then the test that was
originally quoted didn't have much to do with reality and showed
that whoever made it didn't have much experience with setting up
real-world high traffic systems with mod_perl.


  - ask

-- 
ask bjoern hansen - 
more than 70M impressions per day, 

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Jeremy Howard]

Joe Schaefer wrote:
> "Jeremy Howard" <[EMAIL PROTECTED]> writes:
> > I don't know if Speedy fixes this, but one problem with mod_perl v1 is
that
> > if, for instance, a large POST request is being uploaded, this takes a
whole
> > perl interpreter while the transaction is occurring. This is at least
one
> > place where a Perl interpreter should not be needed.
> >
> > Of course, this could be overcome if an HTTP Accelerator is used that
takes
> > the whole request before passing it to a local httpd, but I don't know
of
> > any proxies that work this way (AFAIK they all pass the packets as they
> > arrive).
>
> I posted a patch to modproxy a few months ago that specifically
> addresses this issue.  It has a ProxyPostMax directive that changes
> it's behavior to a store-and-forward proxy for POST data (it also enabled
> keepalives on the browser-side connection if they were enabled on the
> frontend server.)
>
FYI, this patch is at:

  http://www.mail-archive.com/modperl@apache.org/msg11072.html

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Gunther Birznieks]

At 10:17 PM 12/22/2000 -0500, Joe Schaefer wrote:
>"Jeremy Howard" <[EMAIL PROTECTED]> writes:
>
>[snipped]
>I posted a patch to modproxy a few months ago that specifically
>addresses this issue.  It has a ProxyPostMax directive that changes
>it's behavior to a store-and-forward proxy for POST data (it also enabled
>keepalives on the browser-side connection if they were enabled on the
>frontend server.)
>
>It does this by buffering the data to a temp file on the proxy before
>opening the backend socket.  It's straightforward to make it buffer to
>a portion of RAM instead- if you're interested I can post another patch
>that does this also, but it's pretty much untested.
Cool! Are these patches now incorporated in the core mod_proxy if we 
download it off the web? Or do we troll through the mailing list to find 
the patch?

(Similar question about the forwarding of remote user patch someone posted 
last year).

Thanks,
 Gunther

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Joe Schaefer]

"Jeremy Howard" <[EMAIL PROTECTED]> writes:

> Perrin Harkins wrote:
> > What I was saying is that it doesn't make sense for one to need fewer
> > interpreters than the other to handle the same concurrency.  If you have
> > 10 requests at the same time, you need 10 interpreters.  There's no way
> > speedycgi can do it with fewer, unless it actually makes some of them
> > wait.  That could be happening, due to the fork-on-demand model, although
> > your warmup round (priming the pump) should take care of that.

A backend server can realistically handle multiple frontend requests, since
the frontend server must stick around until the data has been delivered
to the client (at least that's my understanding of the lingering-close
issue that was recently discussed at length here). Hypothetically speaking,
if a "FastCGI-like"[1] backend can deliver it's content faster than the 
apache (front-end) server can "proxy" it to the client, you won't need as 
many to handle the same (front-end) traffic load.

As an extreme hypothetical example, say that over a 5 second period you
are barraged with 100 modem requests that typically would take 5s each to 
service.  This means (sans lingerd :) that at the end of your 5 second 
period, you have 100 active apache children around.

But if new requests during that 5 second interval were only received at 
20/second, and your "FastCGI-like" server could deliver the content to
apache in one second, you might only have forked 50-60 "FastCGI-like" new 
processes to handle all 100 requests (forks take a little time :).

Moreover, an MRU design allows the transient effects of a short burst 
of abnormally heavy traffic to dissipate quickly, and IMHO that's its 
chief advantage over LRU.  To return to this hypothetical, suppose 
that immediately following this short burst, we maintain a sustained 
traffic of 20 new requests per second. Since it takes 5 seconds to 
deliver the content, that amounts to a sustained concurrency level 
of 100. The "Fast-CGI like" backend may have initially reacted by forking 
50-60 processes, but with MRU only 20-30 processes will actually be 
handling the load, and this reduction would happen almost immediately 
in this hyothetical.  This means that the remaining transient 20-30 
processes could be quickly killed off or _moved to swap_ without adversely 
affecting server performance.

Again, this is all purely hypothetical - I don't have benchmarks to
back it up ;)

> I don't know if Speedy fixes this, but one problem with mod_perl v1 is that
> if, for instance, a large POST request is being uploaded, this takes a whole
> perl interpreter while the transaction is occurring. This is at least one
> place where a Perl interpreter should not be needed.
> 
> Of course, this could be overcome if an HTTP Accelerator is used that takes
> the whole request before passing it to a local httpd, but I don't know of
> any proxies that work this way (AFAIK they all pass the packets as they
> arrive).

I posted a patch to modproxy a few months ago that specifically 
addresses this issue.  It has a ProxyPostMax directive that changes 
it's behavior to a store-and-forward proxy for POST data (it also enabled 
keepalives on the browser-side connection if they were enabled on the 
frontend server.)

It does this by buffering the data to a temp file on the proxy before 
opening the backend socket.  It's straightforward to make it buffer to 
a portion of RAM instead- if you're interested I can post another patch 
that does this also, but it's pretty much untested.


[1] I've never used SpeedyCGI, so I've refrained from specifically discussing 
it. Also, a mod_perl backend server using Apache::Registry can be viewed as 
"FastCGI-like" for the purpose of my argument.

-- 
Joe Schaefer

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Keith G. Murphy]

Perrin Harkins wrote:
> 
> 
> Keith Murphy pointed out that I was seeing the result of persistent HTTP
> connections from my browser.  Duh.
> 
I must mention that, having seen your postings here over a long period,
anytime I can make you say "duh", my week is made.  Maybe the whole
month.

That issue can be confusing.  It was especially so for me when IE did
it, and Netscape did not...

Let's make everyone switch to IE, and mod_perl looks good again!  :-b

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Matt Sergeant]

On Thu, 21 Dec 2000, Sam Horrocks wrote:

>  > Folks, your discussion is not short of wrong statements that can be easily
>  > proved, but I don't find it useful.
>
>  I don't follow.  Are you saying that my conclusions are wrong, but
>  you don't want to bother explaining why?
>
>  Would you agree with the following statement?
>
> Under apache-1, speedycgi scales better than mod_perl with
> scripts that contain un-shared memory

NO!

When you can write a trans handler or an auth handler with speedy, then I
might agree with you. Until then I must insist you add "mod_perl
Apache::Registry scripts" or something to that affect.

-- 


/||** Director and CTO **
   //||**  AxKit.com Ltd   **  ** XML Application Serving **
  // ||** http://axkit.org **  ** XSLT, XPathScript, XSP  **
 // \\| // ** Personal Web Site: http://sergeant.org/ **
 \\//
 //\\
//  \\

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Jeremy Howard]

Perrin Harkins wrote:
> What I was saying is that it doesn't make sense for one to need fewer
> interpreters than the other to handle the same concurrency.  If you have
> 10 requests at the same time, you need 10 interpreters.  There's no way
> speedycgi can do it with fewer, unless it actually makes some of them
> wait.  That could be happening, due to the fork-on-demand model, although
> your warmup round (priming the pump) should take care of that.
>
I don't know if Speedy fixes this, but one problem with mod_perl v1 is that
if, for instance, a large POST request is being uploaded, this takes a whole
perl interpreter while the transaction is occurring. This is at least one
place where a Perl interpreter should not be needed.

Of course, this could be overcome if an HTTP Accelerator is used that takes
the whole request before passing it to a local httpd, but I don't know of
any proxies that work this way (AFAIK they all pass the packets as they
arrive).

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Perrin Harkins]

On Thu, 21 Dec 2000, Ken Williams wrote:
> So in a sense, I think you're both correct.  If "concurrency" means
> the number of requests that can be handled at once, both systems are
> necessarily (and trivially) equivalent.  This isn't a very useful
> measurement, though; a more useful one is how many children (or
> perhaps how much memory) will be necessary to handle a given number of
> incoming requests per second, and with this metric the two systems
> could perform differently.

Yes, well put.  And that actually brings me back around to my original
hypothesis, which is that once you reach the maximum number of
interprerters that can be run on the box before swapping, it no longer
makes a difference if you're using LRU or MRU.  That's because all
interpreters are busy all the time, and the RAM for lexicals has already
been allocated in all of them.  At that point, it's a question of which
system can fit more interpreters in RAM at once, and I still think
mod_perl would come out on top there because of the shared memory.  Of
course most people don't run their servers at full throttle, and at less
than total saturation I would expect speedycgi to use less RAM and
possibly be faster.

So I guess I'm saying exactly the opposite of the original assertion:
mod_perl is more scalable if you define "scalable" as maximum requests per
second on a given machine, but speedycgi uses fewer resources at less than
peak loads which would make it more attractive for ISPs and other people
who use their servers for multiple tasks.

This is all hypothetical and I don't have time to experiment with it until
after the holidays, but I think the logic is correct.

- Perrin

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Perrin Harkins]

Hi Sam,

>  Processes 1, 2, 3 are running.  1 finishes and requests the mutex, then
>  2 finishes and requests the mutex, then 3 finishes and requests the mutex.
>  So when the next three requests come in, they are handled in the same order:
>  1, then 2, then 3 - this is FIFO or LRU.  This is bad for performance.

Thanks for the explanation; that makes sense now.  So, I was right that
it's OS dependent, but most OSes use a FIFO approach which leads to LRU
selection in the mutex.

Unfortunately, I don't see that being fixed very simply, since it's not
really Apache doing the choosing.  Maybe it will be possible to do
something cool with the wake-one stuff in Linux 2.4 when that comes out.

By the way, how are you doing it?  Do you use a mutex routine that works
in LIFO fashion?

>  > In my experience running
>  > development servers on Linux it always seemed as if the the requests
>  > would continue going to the same process until a request came in when
>  > that process was already busy.
> 
>  No, they don't.  They go round-robin (or LRU as I say it).

Keith Murphy pointed out that I was seeing the result of persistent HTTP
connections from my browser.  Duh.

>  But a point I'm making is that with mod_perl you have to go to great
>  lengths to write your code so as to avoid unshared memory.  My claim is that
>  with mod_speedycgi you don't have to concern yourself as much with this.
>  You can concentrate more on the application and less on performance tuning.

I think you're overstating the case a bit here.  It's really easy to take
advantage of shared memory with mod_perl - I just add a 'use Foo' to my
startup.pl!  It can be hard for newbies to understand, but there's nothing
difficult about implementing it.  I often get 50% or more of my
application shared in this way.  That's a huge savings.

>  I don't assume that each approach is equally fast under all loads.  They
>  were about the same with concurrency level-1, but higher concurrency levels
>  they weren't.

Well, certainly not when mod_perl started swapping...

Actually, there is a reason why MRU could lead to better performance (as
opposed to just saving memory): caching of allocated memory.  The first
time Perl sees lexicals it has to allocate memory for them, so if you
re-use the same interpreter you get to skip this step and that should give
some kind of performance benefit.

>  I am saying that since SpeedyCGI uses MRU to allocate requests to perl
>  interpreters, it winds up using a lot fewer interpreters to handle the
>  same number of requests.

What I was saying is that it doesn't make sense for one to need fewer
interpreters than the other to handle the same concurrency.  If you have
10 requests at the same time, you need 10 interpreters.  There's no way
speedycgi can do it with fewer, unless it actually makes some of them
wait.  That could be happening, due to the fork-on-demand model, although
your warmup round (priming the pump) should take care of that.

>  I don't think it's pre-forking.  When I ran my tests I would always run
>  them twice, and take the results from the second run.  The first run
>  was just to "prime the pump".

That seems like it should do it, but I still think you could only have
more processes handling the same concurrency on mod_perl if some of the
mod_perl processes are idle or some of the speedycgi requests are waiting.

>  > This is probably all a moot point on a server with a properly set
>  > MaxClients and Apache::SizeLimit that will not go into swap.
> 
>  Please let me know what you think I should change.  So far my
>  benchmarks only show one trend, but if you can tell me specifically
>  what I'm doing wrong (and it's something reasonable), I'll try it.

Try setting MinSpareServers as low as possible and setting MaxClients to a
value that will prevent swapping.  Then set ab for a concurrency equal to
your MaxClients setting.

>  I believe that with speedycgi you don't have to lower the MaxClients
>  setting, because it's able to handle a larger number of clients, at
>  least in this test.

Maybe what you're seeing is an ability to handle a larger number of
requests (as opposed to clients) because of the performance benefit I
mentioned above.  I don't know how hard ab tries to make sure you really
have n simultaneous clients at any given time.

>  In other words, if with mod_perl you had to turn
>  away requests, but with mod_speedycgi you did not, that would just
>  prove that speedycgi is more scalable.

Are the speedycgi+Apache processes smaller than the mod_perl
processes?  If not, the maximum number of concurrent requests you can
handle on a given box is going to be the same.

>  Maybe.  There must a benchmark somewhere that would show off of
>  mod_perl's advantages in shared memory.  Maybe a 100,000 line perl
>  program or something like that - it would have to be something where
>  mod_perl is using *lots* of shared memory, because keep in mind that
>  there are still going to be a whole lot

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Stas Bekman]

On Thu, 21 Dec 2000, Sam Horrocks wrote:

>  > Folks, your discussion is not short of wrong statements that can be easily
>  > proved, but I don't find it useful.
> 
>  I don't follow.  Are you saying that my conclusions are wrong, but
>  you don't want to bother explaining why?
>  
>  Would you agree with the following statement?
> 
> Under apache-1, speedycgi scales better than mod_perl with
> scripts that contain un-shared memory 

I don't know. It's easy to give a simple example and claim being better.
So far whoever tried to show by benchmarks that he is better, most often
was proved wrong, since the technologies in question have so many
features, that I believe no benchmark will prove any of them absolutely
superior or inferior. Therefore I said that trying to tell that your grass
is greener is doomed to fail if someone has time on his hands to prove you
wrong. Well, we don't have this time.

Therefore I'm not trying to prove you wrong or right. Gunther's point of
the original forward was to show things that mod_perl may need to adopt to
make it better. Doug already explained in his paper that the MRU approach
has been already implemented in mod_perl-2.0. You could read it in the
link that I've attached and the quote that I've quoted.

So your conclusions about MRU are correct and we have it implemented
already (well very soon now :). I apologize if my original reply was
misleading.

I'm not telling that benchmarks are bad. What I'm telling is that it's
very hard to benchmark things which are different. You benefit the most
from the benchmarking when you take the initial code/product, benchmark
it, then you try to improve the code and benchmark again to see whether it
gave you any improvement. That's the area when the benchmarks rule and
their are fair because you test the same thing. Well you could read more
of my rambling about benchmarks in the guide.

So if you find some cool features in other technologies that mod_perl
might adopt and benefit from, don't hesitate to tell the rest of the gang.



Something that I'd like to comment on:

I find it a bad practice to quote one sentence from person's post and
follow up on it. Someone from the list has sent me this email (SB> == me):

SB> I don't find it useful

and follow up. Why not to use a single letter:

SB> I

and follow up? It's so much easier to flame on things taken out of their
context.

it has been no once that people did this to each other here on the list, I
think I did too. So please be more careful when taking things out of
context. Thanks a lot, folks!

Cheers...

_
Stas Bekman  JAm_pH --   Just Another mod_perl Hacker
http://stason.org/   mod_perl Guide  http://perl.apache.org/guide 
mailto:[EMAIL PROTECTED]   http://apachetoday.com http://logilune.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/  

Re: Fwd: [speedycgi] Speedycgi scales better than mod_perl withscriptsthat contain un-shared memory

[Stas Bekman]

Folks, your discussion is not short of wrong statements that can be easily
proved, but I don't find it useful. Instead please read:

http://perl.apache.org/~dougm/modperl_2.0.html#new

Too quote the most relevant part:

"With 2.0, mod_perl has much better control over which PerlInterpreters
are used for incoming requests. The intepreters are stored in two linked
lists, one for available interpreters one for busy. When needed to handle
a request, one is taken from the head of the available list and put back
into the head of the list when done. This means if you have, say, 10
interpreters configured to be cloned at startup time, but no more than 5
are ever used concurrently, those 5 continue to reuse Perls allocations,
while the other 5 remain much smaller, but ready to go if the need
arises."

Of course you should read the rest.

So the moment mod_perl 2.0 hits the shelves, this possible benefit
of speedycgi over mod_perl becomes irrelevant. I think this more or less
summarizes this thread.

And Gunther, nobody tries to shut people expressing their opinions here,
it's just that different people express their feelings in different ways,
that's the way the open list goes... :) so please keep on forwarding
things that you find interesting. I don't think anybody here has a relief
when you are busy and not posting as you happen to say -- I believe that
your posts are very interesting and you shouldn't discourage yourself from
keeping on doing that. Those who don't like your posts don't have to read
them.

Hope you are all having fun and getting ready for the holidays :) I'm
going to buy my ski equipment soonish!

_
Stas Bekman  JAm_pH --   Just Another mod_perl Hacker
http://stason.org/   mod_perl Guide  http://perl.apache.org/guide 
mailto:[EMAIL PROTECTED]   http://apachetoday.com http://logilune.com/
http://singlesheaven.com http://perl.apache.org http://perlmonth.com/