Dan Melomedman writes:
> I could be wrong here, but running a process per connection is wasteful on
> very busy servers in any case. In other words, correctly designed
> multi-threaded (not necessarily on Linux, and not necessarily with
> pthreads) servers conserve memory and require less kernel scheduling
> overhead if any.
Only on platforms where processes are expensive. I really don't see much of
a difference between a process and a thread. The only difference is whether
they use the same or different address spaces.
> In threaded programs memory is shared, whereas in multi-process forking
> servers it's allocated per process. This means higher memory overhead.
No. Immediately after a fork(), you have processes that run in the same
memory space. All that fork() does on modern systems is duplicate a process
structure, and the associated resources, and goes through each data page and
marks it copy-on-write. If one of the processes writes to a data page, it
will fault, and the kernel will make a copy of the data page.
For processes that are mostly code, with very little data, fork()s introduce
very little overhead at all.
> Also, why does apache prefork a number of servers for instance?
Because it can. That's the direct answer.
>
> Also, here's an interesting unrelated blurb about copy-on-write:
> http://www.netbsd.org/Documentation/kernel/vfork.html
>
>
> Forking is multitudes more expensive than threading, memory and
> kernel-wise.
Only on broken platforms. Forking itself does very little. Only when you
begin to modify the data pages of one of the forked processes will you run
into overhead.
>> From what I can tell the state-threads project aren't really threads. It
>> looks like an event-driven library.
>
> Exactly. And a very cool one indeed since it allowes good system and load
> scalability, while at the same time allowing to avoid thread safety
> concerns, and mutexes are not usually needed.
Event-driven execution models are very cumbersome ones to work with for
anything other than GUI applications.
A very good example of a application platform that foists an event-driven
model on everyone is Windows. The end result are applications that are
utter piles of crap, from a technical viewpoint. No longer can you go ahead
and create a logical technical design for your application. Instead, you
will need to design a steaming pile of spaghetti code, twisted into a single
tight knot in its middle, whose purpose is to respond to pseudo-random I/O
events.
>> It seems like most of your concerns have more to do with the web server
>> than sqwebmail. Justus
>
> Not really concerns. I was just dreaming about a stand-alone server that
> was designed to scale from scratch. This could be useful for large ISPs.
Only if the ISP can deal with a single point of failure. ISPs that have
some amount of technical experience and know how prefer to use multiple,
redundant servers, where is no single point of failure.
In order to provide fault-tolerant storage you do not see Netapp plonk a
single fat disk into their boxes. Same basic concept.
--
Sam
