Re: jails or chroot?
On 5/9/06, Chad Leigh -- Shire.Net LLC [EMAIL PROTECTED] wrote: On May 9, 2006, at 5:53 AM, Michael Grant wrote: When it comes time to upgrade, how does one upgrade 100 different jails? This will be a nightmare! Actually, not. You only need 1 master jail and a bunch of nullfs read only mounts plus some exclusive space for each jail.I run 44 jails at the moment this way. Upgrading is relatively easy as I only have to upgrade one master jail (and unfortunately lots of jail etc if such happens but a few scripts can automate much of that). I basically set up /local/jails/master and install according to man jail into this place. I never start this jail. I happen to use disk backed md devices as the root for each jail. I mount each on on /local/jail/jailname Then I do /sbin/mount_nullfs -o ro /local/jails/master/bin /local/jails/adcmw/bin /sbin/mount_nullfs -o ro /local/jails/master/lib /local/jails/adcmw/lib /sbin/mount_nullfs -o ro /local/jails/master/libexec /local/jails/ adcmw/libexec /sbin/mount_nullfs -o ro /local/jails/master/sbin /local/jails/adcmw/ sbin /sbin/mount_nullfs -o ro /local/jails/master/usr /local/jails/adcmw/usr /sbin/mount -t procfs proc /local/jails/adcmw/proc devfs_domount /local/jails/adcmw/dev devfsrules_jail devfs_set_ruleset devfsrules_jail /local/jails/adcmw/dev /sbin/devfs -m /local/jails/adcmw/dev rule -s 4 applyset In my master jail I have some symlinks so that each jail has its own / usr/local/ that is writable. All the jails run out of one installed jail and they also have the side benefit of the main system directories being read only so exploits in one jail cannot affect all the running jails. Wow, I really like the setup you have make.. One question.How do you update the system(and the jail) ? ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: jails or chroot?
On May 10, 2006, at 2:33 AM, Iantcho Vassilev wrote: On 5/9/06, Chad Leigh -- Shire.Net LLC [EMAIL PROTECTED] wrote: On May 9, 2006, at 5:53 AM, Michael Grant wrote: When it comes time to upgrade, how does one upgrade 100 different jails? This will be a nightmare! Actually, not. You only need 1 master jail and a bunch of nullfs read only mounts plus some exclusive space for each jail.I run 44 jails at the moment this way. Upgrading is relatively easy as I only have to upgrade one master jail (and unfortunately lots of jail etc if such happens but a few scripts can automate much of that). snipppage All the jails run out of one installed jail and they also have the side benefit of the main system directories being read only so exploits in one jail cannot affect all the running jails. Wow, I really like the setup you have make.. One question.How do you update the system(and the jail) ? I shut all the jails down, and update the system. Then I boot without starting the jails and rebuild the master jail according to man jail. Then I start a special main jail that was used to install ports used, if any, into a common area and do any updates necessary -- this last one from 5.4 to 6.0 I just made sure I had the 5x compatibility stuff installed and all was fine for now so I have more time to redo individual ports or SW built frmo scratch. When that is done I restart all the jails. I had about 40 jails active when I went from 5.4 to 6.0 on this particular machine (some earlier ones I did from 5.4 to 6.0 had maybe 1 or 2 jails so they were not the definitive test case). Had no problems once I made sure all the jails were accessing the compat 5x stuff (which I did by editing in each jail /etc -- you could use a script but I am lousy at writing more than simple scripts -- the rc.conf and making sure that ldconfig_paths= was set appropriately to the master jail wide compat5x library location... Done, finis Chad --- Chad Leigh -- Shire.Net LLC Your Web App and Email hosting provider chad at shire.net ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
jails or chroot?
I host a bunch of websites on my box. Recently I had some problems with file access problems with php which caused me to look into putting each of my clients into their own jail or chroot. I have roughly 100 different domains I'd need to split. Has anyone done this for more than a handfull of clients? Using apache and their mass virtual hosting, 100 domains is a breeze. But with a jail or chroot, I need a separate apache process for each domain. This is going to mean hundreds of apache processes. This seems unreasonable. When it comes time to upgrade, how does one upgrade 100 different jails? This will be a nightmare! What do you folks do who run lots of domains on freebsd? Michael Grant ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: jails or chroot?
On 5/9/06, Michael Grant [EMAIL PROTECTED] wrote: I host a bunch of websites on my box. Recently I had some problems with file access problems with php which caused me to look into putting each of my clients into their own jail or chroot. I have roughly 100 different domains I'd need to split. I won't be doing this even if someone pays me twice for doing it. This is going to create a HELL lot of problems later on, especially during upgrades. BTW can you tell us your exact requirements? Thanks and Best Regards Subhro -- Subhro Kar Security Engineer iViZ Techno Solutions Pvt. Ltd. eRevMax House, 1st Floor Plot XI-16, Sector V Salt Lake City 700091 India ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: jails or chroot?
I'll try to be more explicit on my requirements. I'm not worried about mail. I'm mostly worried about web. Each client has a web site with one or more domains. I currently offer them apache+php+mysql+mod_perl+mod_ssl. One of them needs java server pages, tomcat I think. Everyone gets access to their own logs and to geolizer (webalizer). Some clients would like shell access. Most clients write their web site using ftp. Certain ones need also the MS Front Page Extensions. Some clients want an ftp upload area. Ssl poses a special problem in that I need to allocate an ip address for those who have their own ssl certificate. It's pretty much all standard stuff. But yes, I totally agree with you, it is an administration nightmare to set up separate jails and keep track of which has which version of what and so on. There must be an easier way to do this. Some of you folks who run hosting sites, how do you manage large numbers of clients? Michael Grant On 5/9/06, Subhro [EMAIL PROTECTED] wrote: On 5/9/06, Michael Grant [EMAIL PROTECTED] wrote: I host a bunch of websites on my box. Recently I had some problems with file access problems with php which caused me to look into putting each of my clients into their own jail or chroot. I have roughly 100 different domains I'd need to split. I won't be doing this even if someone pays me twice for doing it. This is going to create a HELL lot of problems later on, especially during upgrades. BTW can you tell us your exact requirements? Thanks and Best Regards Subhro -- Subhro Kar Security Engineer iViZ Techno Solutions Pvt. Ltd. eRevMax House, 1st Floor Plot XI-16, Sector V Salt Lake City 700091 India ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: jails or chroot?
On 5/9/06, Michael Grant [EMAIL PROTECTED] wrote: I host a bunch of websites on my box. Recently I had some problems with file access problems with php which caused me to look into putting each of my clients into their own jail or chroot. I have roughly 100 different domains I'd need to split. Has anyone done this for more than a handfull of clients? Using apache and their mass virtual hosting, 100 domains is a breeze. But with a jail or chroot, I need a separate apache process for each domain. This is going to mean hundreds of apache processes. This seems unreasonable. Agreed that creation hundreds of chroots or jails would be an administrative nightmare. File access can be solved with suexec (compile apache with suexec enabled), this means that for each virtual host entry in your apache config you add User and Group (check http://httpd.apache.org/docs/2.2/suexec.html or your apache version doc set). This will make each apache process run as the user specified in virtual host entry (not www) allowing you to restrict their access to files with filesystem ACL's and even ugidfw, you could also then setup process/memory restrictions in /etc/login.conf It will also make updating pretty much as standard as it is now. Give it a burl if it sounds like what you need. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
RE: jails or chroot?
Hi, Sure, jails require more work regarding administration. Ports are not the biggest problem I think, it's the easy part. The problem is when you have to update the world. But even here, with a good script, it's not such a nightmare. Maybe all you need is Michael's solution. But take into account that with jails, you have a great flexibility regarding the application you install for a particular client. And all the security that a jail system can offer, plus a fantastic way of managing your backups. I personally run a jail based VPS server, based on FreeBSD 6.0, with 13 jails at the moment. It's a dual xeon, with 4GB RAM, and RAID 5 SCSI HDs. I have 355 MB RAM active, 1525 inactive and 1679 MB RAM are free. I intend to run a maximum of 50 jails on this server. And until now, nothing seems to oppose to my plans. Beware of one thing with jails, though: a bug in FreeBSD does not permit a clean shutdown of jails. But tust me: you never need to! Hope this helps, and keep us informed of your choice. Philippe Lang -Message d'origine- De : [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] De la part de Jahilliya Envoyé : mardi, 9. mai 2006 14:48 À : Michael Grant Cc : freebsd-questions@freebsd.org Objet : Re: jails or chroot? On 5/9/06, Michael Grant [EMAIL PROTECTED] wrote: I host a bunch of websites on my box. Recently I had some problems with file access problems with php which caused me to look into putting each of my clients into their own jail or chroot. I have roughly 100 different domains I'd need to split. Has anyone done this for more than a handfull of clients? Using apache and their mass virtual hosting, 100 domains is a breeze. But with a jail or chroot, I need a separate apache process for each domain. This is going to mean hundreds of apache processes. This seems unreasonable. Agreed that creation hundreds of chroots or jails would be an administrative nightmare. File access can be solved with suexec (compile apache with suexec enabled), this means that for each virtual host entry in your apache config you add User and Group (check http://httpd.apache.org/docs/2.2/suexec.html or your apache version doc set). This will make each apache process run as the user specified in virtual host entry (not www) allowing you to restrict their access to files with filesystem ACL's and even ugidfw, you could also then setup process/memory restrictions in /etc/login.conf It will also make updating pretty much as standard as it is now. Give it a burl if it sounds like what you need. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] smime.p7s Description: S/MIME cryptographic signature
Re: jails or chroot?
On Tuesday 09 May 2006 08:24, Michael Grant wrote: I'll try to be more explicit on my requirements. I'm not worried about mail. I'm mostly worried about web. Each client has a web site with one or more domains. I currently offer them apache+php+mysql+mod_perl+mod_ssl. One of them needs java server pages, tomcat I think. Everyone gets access to their own logs and to geolizer (webalizer). Some clients would like shell access. Most clients write their web site using ftp. Certain ones need also the MS Front Page Extensions. Some clients want an ftp upload area. Ssl poses a special problem in that I need to allocate an ip address for those who have their own ssl certificate. It's pretty much all standard stuff. I use suphp with apache in a mass hosting configuration for about 50 websites to take care of the php access issues. You'll need to setup the ACLs correctly so there is no snooping. I then use scponly to allow chrooted sftp access to their web directories. Webalizer logs are automatically generated an placed in their chrooted directory for download. As for shell access I don't allow it. If people want easy command line access I just tell use sshfs on FreeBSD or Linux. The Windows and Mac users don't care about shell access. For the Tomcat, Frontpage, and SSL users just setup jails for them. With the inclusion of mergemaster -u subsequent base system upgrades are much less painful. Using null mounts for the common areas should lessen the version sync issues. Once unionfs is stable again, you could just use one jail as a base image and allow the others to be cloned off of that. Hopefully some of the above helps you in your situation. But yes, I totally agree with you, it is an administration nightmare to set up separate jails and keep track of which has which version of what and so on. There must be an easier way to do this. Some of you folks who run hosting sites, how do you manage large numbers of clients? Michael Grant On 5/9/06, Subhro [EMAIL PROTECTED] wrote: On 5/9/06, Michael Grant [EMAIL PROTECTED] wrote: I host a bunch of websites on my box. Recently I had some problems with file access problems with php which caused me to look into putting each of my clients into their own jail or chroot. I have roughly 100 different domains I'd need to split. I won't be doing this even if someone pays me twice for doing it. This is going to create a HELL lot of problems later on, especially during upgrades. BTW can you tell us your exact requirements? Thanks and Best Regards Subhro -- Subhro Kar Security Engineer iViZ Techno Solutions Pvt. Ltd. eRevMax House, 1st Floor Plot XI-16, Sector V Salt Lake City 700091 India ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED] -- Anish Mistry pgpQunBnXtHJk.pgp Description: PGP signature
Re: jails or chroot?
With the inclusion of mergemaster -u subsequent base system upgrades are much less painful. Using null mounts for the common areas should lessen the version sync issues. Once unionfs is stable again, you could just use one jail as a base image and allow the others to be cloned off of that. ezjail might come in handy as well... http://erdgeist.org/arts/software/ezjail/ ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: jails or chroot?
On May 9, 2006, at 5:53 AM, Michael Grant wrote: When it comes time to upgrade, how does one upgrade 100 different jails? This will be a nightmare! Actually, not. You only need 1 master jail and a bunch of nullfs read only mounts plus some exclusive space for each jail.I run 44 jails at the moment this way. Upgrading is relatively easy as I only have to upgrade one master jail (and unfortunately lots of jail etc if such happens but a few scripts can automate much of that). I basically set up /local/jails/master and install according to man jail into this place. I never start this jail. I happen to use disk backed md devices as the root for each jail. I mount each on on /local/jail/jailname Then I do /sbin/mount_nullfs -o ro /local/jails/master/bin /local/jails/adcmw/bin /sbin/mount_nullfs -o ro /local/jails/master/lib /local/jails/adcmw/lib /sbin/mount_nullfs -o ro /local/jails/master/libexec /local/jails/ adcmw/libexec /sbin/mount_nullfs -o ro /local/jails/master/sbin /local/jails/adcmw/ sbin /sbin/mount_nullfs -o ro /local/jails/master/usr /local/jails/adcmw/usr /sbin/mount -t procfs proc /local/jails/adcmw/proc devfs_domount /local/jails/adcmw/dev devfsrules_jail devfs_set_ruleset devfsrules_jail /local/jails/adcmw/dev /sbin/devfs -m /local/jails/adcmw/dev rule -s 4 applyset In my master jail I have some symlinks so that each jail has its own / usr/local/ that is writable. All the jails run out of one installed jail and they also have the side benefit of the main system directories being read only so exploits in one jail cannot affect all the running jails. Chad --- Chad Leigh -- Shire.Net LLC Your Web App and Email hosting provider chad at shire.net ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
