Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
Scott McEachern wrote: ... I ended up doing this: - one OpenBSD box, with multiple IP address aliases - one OpenBSD firewall, which rdr's external IPs to the appropriate webserver IP - 5 chrooted OpenBSD default (1.3.29) Apache's (at this time, I have no need for Apache 2, but hey, it's in ports.) - 5 custom httpd.conf files for each - 5 custom php.ini files for each (plus other related config file friends) - 5 different httpd daemons for each (httpd0-4), just in case - virtual aliases with Apache is not a solution because the sites use https/ssl - all the sites have all the php-*, pear-*, mod_* stuff at their disposal I have the same setup running. Each apache instance runs chrooted under their own user id and home directory. The setup I had before that was more interesting as it only needed one IP. A main httpd instance was setup to do proxy for the individual httpd instances of each site. The main instance ran on port 80 with the real IP. The site instances ran on localhost with each their own port number and weren't accessible from outside of the machine. Logging, SSL and maintenance is a pain though.
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
On Thu, 11 Mar 2010 16:47:54 -0600, Claus cnie...@gmx.net wrote: I have the same setup running. Each apache instance runs chrooted under their own user id and home directory. That's a lot of apache instances running... and how much functionality are you really getting out of them? Lighttpd or NginX with FastCGI works very well. I'm running php-fastcgi once per domain, chrooted to its virtual host directory; I've also got non-PHP FastCGI applications running in unrelated chroots. One process (lighttpd) handles SSL and most logging (each PHP instance logs in its chroot, but that separates different users' PHP logs too). Maintenance is still a pain, though, as I have to copy all relevant binaries, PHP modules, and dependent shared libraries into each chroot every upgrade. I keep meaning to write a script to maintain that: copy new binaries (e.g., php-fastcgi) over, determine what shared objects they link to, copy those over, and delete old versions. -- Matthew Weigel hacker unique idempot . ent
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
Claus wrote:
I have the same setup running. Each apache instance runs chrooted
under their own user id and home directory.
I realized after I sent that message that I left out a couple of
details, like each instance also having its own user (www0-4). I leave
the default www user and /var/www stuff pretty much untouched in case I
need to look at something 'untainted' by my fingers. The normal install
of the modules modifies those bits of course, which are later copied to
the individual httpd homedirs as needed. I don't recall exactly what
does and doesn't need copying, I have it all _very_ throughly documented
kinda script-like so I can reproduce it quickly if need be, with my
notes and copy/paste-able mass link / copy / etc commands.
The setup I had before that was more interesting as it only needed one
IP. A main httpd instance was setup to do proxy for the individual
httpd instances of each site. The main instance ran on port 80 with
the real IP. The site instances ran on localhost with each their own
port number and weren't accessible from outside of the machine.
Logging, SSL and maintenance is a pain though.
I never tried the proxy method simply because I wanted all daemons to be
autonomous. If something died, so be it (I should note it's never
happened yet). Not to mention, I use a couple of the sites for
development, so sometimes I have to kill an individual httpd{x} instance
when I monkey with the config.
I have briefly considering moving from Apache to nginx, but haven't for
a few reasons:
1) ATM, I don't need the performance of nginx vs. Apache, not by a long shot
2) I love the track record of OpenBSD's Apache. It's been fine for me
for years.
3) just when I was peeking into nginx (stable) a security vuln popped
up. I'm sure it's excellent, but *to me* it could mature,
security-wise. (no flames please)
4) time to play with it all and get everything nicely together
5) simple philosophy: if it ain't broke, don't fix it.
When I have time, I need to figure out some automated solution to deal
with the logs. I use cronolog for rotation with custom log file
formats, and have plans to do some things with webalizer-type apps, but
that's still on the back burner.
My interest is in using relayd to filter bad requests (again, back
burner for now.) I have *not* done my homework on this yet, but when I
farted around with it briefly a few days back, I ran into problems with
the relayd config for SSL acceleration. Again, when I have time I'll
look into it, but I was stumped and figured I'll make sure my RTFM-fu is
strong before I post here about it. (Besides, isn't it somehow more
satisfying to finally go *aha I fixed my mistake* without asking for help?)
I knew I wasn't the only one that realized (for many circumstances, I'm
not saying _all_) that VM'ing a lot of services is just silly, but it's
nice to hear from others also doing the multiple httpd thing with OpenBSD.
For Matthew Weigel:
Yes, there are a lot of httpd instances. I'm not entirely sure of how
shared memory applies in this case (probably not), but on my web server
my memory use is 129M/316M, and that includes a bunch of other daemons
(eg. databases), when pretty much idle. It has plenty of room to grow,
but if memory becomes an issue, I'll look harder into nginx. (I'd like
to do it just for the knowledge, but again, time constraints.)
For the installation of everything into the chroot, I can't comment on
non-Apache setups, but with Apache it loads that stuff before chrooting
so only one installed version needs to be done, which makes life
easier. The links (etc) still have to be done. It could easily be
scripted, but I prefer to have my notes (with my big don't forget
warnings) where I can just paste the commands. If your documentation
(notes) are solid, you'll be fine, and I just played musical tables with
the servers (new drives for both) using carp and another box a few
months back with no probs. As long as your notes are thorough enough
that a blind drunk moron could do it.. :)
Hope this isn't noise on the list.
--
-RSM
http://www.erratic.ca
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
OMG!! running multiple daemons??? Wow why didn't I think of that?? I *love* OS overhead on misbehaving hardware emulation because it is what the industry prescribes. Don't forget the 50% hit on I/O speed because that is what every enterprise needs. And lets not forget the windows only license servers combined with management tools that also run only on windows and IE. Virtualization is so awesome. On Mon, Mar 08, 2010 at 02:00:22AM -0500, Scott McEachern wrote: bofh wrote: Is there *ANY* good virtualization software out there? I don't care what OS it needs to host it (preferably not windows :)) - my needs are simple (home use): This doesn't answer your question or help you in any way, but I thought I'd mention it for the list archives (with a nicely searchable subject). A while back I was considering using some type of virtualization for running 5 web servers on the same box. I ended up tossing the idea of virtualization for a couple of reasons: 1) I couldn't really find any VM software I liked that ran nicely on OpenBSD. I was not aware of qemu at the time, so no flames please. (I didn't look all that hard, apparently.) 2) The performance hit you'll inevitably take. (Why I didn't look too hard.) There are probably many (better) ways to go about this, and I'd love to hear them, but I ended up doing this: - one OpenBSD box, with multiple IP address aliases - one OpenBSD firewall, which rdr's external IPs to the appropriate web server IP - 5 chrooted OpenBSD default (1.3.29) Apache's (at this time, I have no need for Apache 2, but hey, it's in ports.) - 5 custom httpd.conf files for each - 5 custom php.ini files for each (plus other related config file friends) - 5 different httpd daemons for each (httpd0-4), just in case - virtual aliases with Apache is not a solution because the sites use https/ssl - all the sites have all the php-*, pear-*, mod_* stuff at their disposal I did have to change /etc/rc (I know, I'm a sinner) so it did it's normal things, but slightly adjusted for each of the 5 servers. I run a single instance each of chrooted MySQL and PostgreSQL servers, which the various sites can access by IP as their own restricted database users. I considered using FreeBSD's jail functionality, but the drawbacks were thus: 1) for the time it would take to learn about configuring FreeBSD's jails, I could do the stuff above many times over. 2) I wouldn't get the OpenBSD version of httpd that has much love from the team (tx henning@ and others). I'm probably forgetting details, since it's been a while since I did it. The end result works just fine for *my* needs, and best of all it's still on my OS of choice so well within my comfort zone. I haven't a clue how this would scale for a web hosting provider, but then again, that's not my problem. :) Sorry for the noise. I once searched for this a long time ago and didn't find anything, so for future reference, yes, it's easily doable. PS: I'm dying for the day that relayd handles https too. :) -- -RSM http://www.erratic.ca
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
On Mon, Mar 08, 2010 at 09:40:30AM -0600, Marco Peereboom wrote: OMG!! running multiple daemons??? Wow why didn't I think of that?? I *love* OS overhead on misbehaving hardware emulation because it is what the industry prescribes. Don't forget the 50% hit on I/O speed because that is what every enterprise needs. And lets not forget the windows only license servers combined with management tools that also run only on windows and IE. Virtualization is so awesome. It's more that the current industry standard kind-of is apache, or enterprise shit based on jakarta AND multiple boxen. solutions to the web server issues, such as using fastci + nginx/lighthttpd, only start to become more or less well-spread. And of course, all the time investment of the so-called sys-admins who learnt how to configure big apache/jakarta installations would go down the drain. Can't have that. They need to protect their investment.
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
On Mon, Mar 8, 2010 at 9:37 AM, Marc Espie es...@nerim.net wrote: On Mon, Mar 08, 2010 at 09:40:30AM -0600, Marco Peereboom wrote: OMG!! running multiple daemons??? Wow why didn't I think of that?? I *love* OS overhead on misbehaving hardware emulation because it is what the industry prescribes. Don't forget the 50% hit on I/O speed because that is what every enterprise needs. And lets not forget the windows only license servers combined with management tools that also run only on windows and IE. Virtualization is so awesome. It's more that the current industry standard kind-of is apache, or enterprise shit based on jakarta AND multiple boxen. solutions to the web server issues, such as using fastci + nginx/lighthttpd, only start to become more or less well-spread. And of course, all the time investment of the so-called sys-admins who learnt how to configure big apache/jakarta installations would go down the drain. Can't have that. They need to protect their investment. I can think of one good reason I need a vm machine: So I can put OpenBSD on the Xserves, and run OSX in the vm for mac-only apps the client requires.
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
On Mon, 08 Mar 2010 10:18 -0700, Ted Roby ted.r...@gmail.com wrote: I can think of one good reason I need a vm machine: So I can put OpenBSD on the Xserves, and run OSX in the vm for mac-only apps the client requires. Another good reason: Reverting compromised Windows machines back to a point in time when they were probably clean (or at least not obviously infected). The malware served-up in some of the ads on the Intertubes is horrible. Even ads on main stream websites can cause severe infestations. Brad
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
Use adsuck and let your user whine when something doesn't work. On Mon, Mar 08, 2010 at 12:47:24PM -0500, Brad Tilley wrote: On Mon, 08 Mar 2010 10:18 -0700, Ted Roby ted.r...@gmail.com wrote: I can think of one good reason I need a vm machine: So I can put OpenBSD on the Xserves, and run OSX in the vm for mac-only apps the client requires. Another good reason: Reverting compromised Windows machines back to a point in time when they were probably clean (or at least not obviously infected). The malware served-up in some of the ads on the Intertubes is horrible. Even ads on main stream websites can cause severe infestations. Brad
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
On Mar 8, 2010, at 11:37 AM, Marc Espie wrote: On Mon, Mar 08, 2010 at 09:40:30AM -0600, Marco Peereboom wrote: OMG!! running multiple daemons??? Wow why didn't I think of that?? I *love* OS overhead on misbehaving hardware emulation because it is what the industry prescribes. Don't forget the 50% hit on I/O speed because that is what every enterprise needs. And lets not forget the windows only license servers combined with management tools that also run only on windows and IE. Virtualization is so awesome. It's more that the current industry standard kind-of is apache, or enterprise shit based on jakarta AND multiple boxen. solutions to the web server issues, such as using fastci + nginx/ lighthttpd, only start to become more or less well-spread. And of course, all the time investment of the so-called sys-admins who learnt how to configure big apache/jakarta installations would go down the drain. Can't have that. They need to protect their investment. Like many things these days, the term enterprise has been co-opted by those with an ulterior--and often opposite--motive. Enterprise should mean reliable, scalable and simple (otherwise known as manageable). It has become the opposite. VMware makes a great toy on my macbook: I can build custom RPMs for linuxy stuff, make release when I don't have a physical machine available. In my enterprise, we have some Dell 1850s and a 1950 and an xserve. I evaluated ESXi thinking I'd be able to build VMs on my macbook and then deploy them on the xserve or the dells. I decided not to screw around with converting VMs from fusion to esxi and back. The final straw was how to intelligently manage exsi without windows/ internet exploder. We're currently running about 15 rails, php and coldfusion apps with the number growing almost weekly. As much as possible, each app gets its own VM (or two) and is proxied to an outward facing web server. I use running xen on centos. Not my first choice, but it is OK behind pf. With a little scripting, I can create a VM and deploy an app in under 5 minutes. We are a small non-profit and that necessarily rules out Enterprise solutions.
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
On Mon, Mar 08, 2010 at 03:17:29PM -0500, daniel wrote: We're currently running about 15 rails, php and coldfusion apps with the number growing almost weekly. As much as possible, each app gets its own VM (or two) and is proxied to an outward facing web server. I use running xen on centos. Not my first choice, but it is OK behind pf. With a little scripting, I can create a VM and deploy an app in under 5 minutes. Learn about fastcgi and nginx, ditch the vm. your performance will go UP, and you will be about as secure. (and you won't have to figure out what memory to allocate for each vm). With fastcgi and nginx, it's really easy to run each app as a separate user. Unless you fuck up really badly, a compromise in your app WON'T compromise the whole machine. And heck, you could probably ldap your userbase or something, and have the convenience of having the same accounts work on each of those apps.
OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
bofh wrote: Is there *ANY* good virtualization software out there? I don't care what OS it needs to host it (preferably not windows :)) - my needs are simple (home use): This doesn't answer your question or help you in any way, but I thought I'd mention it for the list archives (with a nicely searchable subject). A while back I was considering using some type of virtualization for running 5 web servers on the same box. I ended up tossing the idea of virtualization for a couple of reasons: 1) I couldn't really find any VM software I liked that ran nicely on OpenBSD. I was not aware of qemu at the time, so no flames please. (I didn't look all that hard, apparently.) 2) The performance hit you'll inevitably take. (Why I didn't look too hard.) There are probably many (better) ways to go about this, and I'd love to hear them, but I ended up doing this: - one OpenBSD box, with multiple IP address aliases - one OpenBSD firewall, which rdr's external IPs to the appropriate web server IP - 5 chrooted OpenBSD default (1.3.29) Apache's (at this time, I have no need for Apache 2, but hey, it's in ports.) - 5 custom httpd.conf files for each - 5 custom php.ini files for each (plus other related config file friends) - 5 different httpd daemons for each (httpd0-4), just in case - virtual aliases with Apache is not a solution because the sites use https/ssl - all the sites have all the php-*, pear-*, mod_* stuff at their disposal I did have to change /etc/rc (I know, I'm a sinner) so it did it's normal things, but slightly adjusted for each of the 5 servers. I run a single instance each of chrooted MySQL and PostgreSQL servers, which the various sites can access by IP as their own restricted database users. I considered using FreeBSD's jail functionality, but the drawbacks were thus: 1) for the time it would take to learn about configuring FreeBSD's jails, I could do the stuff above many times over. 2) I wouldn't get the OpenBSD version of httpd that has much love from the team (tx henning@ and others). I'm probably forgetting details, since it's been a while since I did it. The end result works just fine for *my* needs, and best of all it's still on my OS of choice so well within my comfort zone. I haven't a clue how this would scale for a web hosting provider, but then again, that's not my problem. :) Sorry for the noise. I once searched for this a long time ago and didn't find anything, so for future reference, yes, it's easily doable. PS: I'm dying for the day that relayd handles https too. :) -- -RSM http://www.erratic.ca
Re: OT: multiple web servers on OpenBSD (WAS: OT: vmware blah blah)
Scott McEachern wrote: PS: I'm dying for the day that relayd handles https too. :) Many thanks to Todd T. Fries for pointing out relayd does SSL/https. Dunno if it changed, or if I misread at the time, but I could have sworn it only did layer 7. My bad. -- -RSM http://www.erratic.ca
