1) What do I need to know in order to develop a web server, generally?
A good starting point would be taking a look at current web servers and
seeing what they do well etc. Apache httpd is the most widely used one
which would be a good starting point, particularly feature wise, but
it's also worth taking a look at others such as lighttpd, nginx and
cherokee. Note that the main selling points for the latter are speed
rather than functionality (although I believe this is changing).
I haven't looked at it too deeply, but Mongrel2 looks to be very
interesting. IIRC, it's written in pure C and supports several
different protocols. I believe it leans heavily on the performance
side. However, it's not finished yet.
Since D's concurrency is going to use message passing, you may want to
check out Yaws which is written in Erlang to see how they handle
connections and what not. However, do keep in mind that creating
threads in D is much more expensive than creating processes in Erlang.
Casey
