Modperlers...,
I'd like to start a discussion about the deficiences in Apache/modperl
and get you feedback with regard to this issue. The problem as I see
it is that the process model that Apache uses is very hard on modperl.
It is very memory inneffecient basically. Each process of apache has
it's registry which holds the compiled perl scripts in..., a copy of
each for each process. This has become an issue for one of the
companies that I work for, and I noted from monitoring the list that
some people have apache processes that are upwards of 25Megs, which is
frankly ridiculous.
This is not meant to be a flame, and I'd really like to get down to
the nitty gritty on how we can solve this problem. Zach Brown wrote
phhttpd which is a threaded server which can handle a lot more load
than apache, but the problem is it doesn't have the features that
Apache has, and it's not going to catch up any time soon so I think
it's not going to be the cure-all. I wrote a very small perl engine
for phhttpd that worked within it's threaded paradigm that sucked up a
neglibible amount of memory which used a very basic version of
Apache's registry. Once again though it didn't have the feature set
that Apache/modperl has. Now to address the issue: I think we have a lot of
code in Modperl that is basically awesome. Not to mention the fact
that Apache itself has a lot of modules and other things which are
quite usefull. However I think if it were possible to divorce the
actually perl engine from the Apache process we could solve this memory
usage problem.
Basically heres what I'm thinking might be possible, but if it's not
just let me know. (Well, I know it's possible, but I mean how much
work would it take to institute, and has someone else worked on this,
or taken a look at how much work we'd be talking about) What I'm
thinking is essentially we take the perl engine which has the apache
registry and all the perl symbols etc., and seperate it into it's own
process which would could be multithreaded (via pthreads) for multiple
processor boxes. (above 2 this would be beneficial probably) On the
front side the apache module API would just connect into this other
process via shared memory pages (shmget et. al), or Unix pipes or
something like that. The mod_perl process would have a work queue
that the Apache processes could add work to via our front end API.
The work threads inside of that mod_perl process would take work
"orders" out of the work queue and process them and send the result
back to the waiting apache process. (Maybe just something as simple
as a blocking read on a pipe coming out of the mod_perl process...
this would keep down context switching issues and other nasty bits)
One of my concerns is that maybe the apache module API is simply too
complex to pull something like this off. I don't know, but it seems
like it should be able to handle something like this.
Does anyone know of any program which has been developed like this?
Basically we'd be turning the "module of apache" portion of mod_perl
into a front end to the "application server" portion of mod_perl that
would do the actual processing. It seems quite logical that something
like this would have been developed, but possibly not. The seperation
of the two components seems like it should be done, but there must be
a reason why no one has done it yet... I'm afraid this reason would be
the apache module API doesn't lend itself to this.
Well, thanks to everyone in advance for their thoughts/comments...
Shane Nay.
