Re: Xen Dom0, are we making progress?
Matthew Dillon wrote: In anycase, as usual I rattle on. If FreeBSD is interested I recommend simply looking at the cool features I added to DragonFly's kernel to make virtual kernels possible. It's really just three major items: Signal mailboxes, a new MAP_VPAGETABLE for mmap, and the new vmspace_*() system calls for managing VM spaces. Once those features were in place it didn't take long for me to create a 'vkernel' platform that linked against libc and used the new system calls. Virtual kernels are a cool idea, but I (and I believe practically anyone using FreeBSD for non-development work) would much rather see a Xen-like functionality (to be precise: ability to run foreign kernels and Windows; qemu is too slow) than just a variation of the native kernel. signature.asc Description: OpenPGP digital signature
Re: Xen Dom0, are we making progress?
: It seems very similar to User Mode Linux, rather than a true VM environment. http://user-mode-linux.sourceforge.net/ Each DragonFlyBSD vkernel runs as a process. I don't know why this is even interesting, for anything but kernel developers. Improving BSD jails to the same level as Solaris Containers (Solaris Containers are Solaris Zones with resource control), would widely useful for many BSD users. : : In VM environment, like Xen, each VM has its own kernel and possibly different OS. Xen has managed to get a lot of people interested in their VM environment, so there are a lot of OSes that support the Xen architecture. And for those that don't there is early support for booting them by using virtual features in newer CPUs (ex. Windows). Microsoft has joined the Xen bandwagon, even though the core is all open source, as they are threatened in the enterprise space by the VMWare juggernaut, and their Virtual Server/Virtual PC product is so bland, no one cares. : : UML has been available for longer than Xen, but Xen already outperforms it. I don't see a lot of future in the virtual kernel concept. : :Tom Well, judging by the history of how UML is used, the biggest uses appear to be (A) Kernel development, (B) Machine virtualization for sale to third parties (virtual servers), and (C) Security separation. You can't really compare BSD jails to a virtual kernel. From a security standpoint, its night and day. Jails require a ton of hooks all over the kernel and even with those hooks they have no real ability to compartmentalize resource use, nor is security assurable with any real level of confidence. You are still running directly under the real kernel and it shows. Virtual kernels are far more secure, even more so once we give them a new syscall table map that disables all real-kernel system calls other then read, write, vmspace_*() calls, and a few other things required for operation once the vkernel has initialized. They can be made extremely secure in ways that jails cannot. Regarding Xen, there is not much of a difference between a virtual kernel implementation like UML or DragonFly's vkernel and something like Xen. Both use the same concepts and have approximately the same overhead, so its mainly an issue of algorithms and coding. I do believe that Xen and vkernel environments are easier to optimize then complete machine virtualization (vmware-like) environments in the long term, simply because the kernels running under Xen or as virtual kernels *know* they are operating virtually and can be heavily optimized for that fact. For example, it would be possible to truely free pages marked 'free' in the VM page queues. As with many linux-centric projects, a great deal of effort is made by certain individuals to optimize performance for patricular types of applications, with varying results and varying implications to maintainability. It is not a direction I really care to go. Xen suffers from this myopia to a degree, though probably not as bad as VMWare suffers from it. My primary reason for doing it in DragonFly is as a kernel development aid. Testing kernel code in a virtual kernel environment reduces the engineering cycle time from ~7-10 minutes to about 20 seconds. It's really amazing. But there are already a number of subsystems that I think I may move into a virtual kernel for security reasons. Our wiki is a good example. I just don't trust all the myrid applications we have to run to support the site. -- The two biggest issues in machine virtualized environments are (1) system calls and (2) page table faults. At the moment (and without any real effort on my part to optimize it), system calls are about 10 times as expensive: vkernel# /tmp/sc1 timing standard getuid() syscall getuid() 0.978s 302100 loops = 3.237uS/loop test28# /tmp/sc1 timing standard getuid() syscall getuid() 0.940s 3178900 loops = 0.296uS/loop Page table faults are somewhat less expensive, but still not cheap. It depends on the type of fault. Read faults are highly optimizable but the 'dirty' bit in the virtualized page table has to be emulated so writable VM maps have to be mapped read-only for a read rather then read-write for a read in order to be able to take a write fault and set the dirty bit in the virtualized page table. With the vmspace_*() system calls, the page faults are still handled by the real kernel so it isn't as bad as one might imagine. So, e.g. compiles are still fairly reasonable. I haven't done a full buildworld test but compile overhead seems to be only about 30% more. Long running services whos main interaction with the system is with fairly optimal network and file I/O calls seem to do the best. Virtual
Re: Xen Dom0, are we making progress?
:Virtual kernels are a cool idea, but I (and I believe practically anyone :using FreeBSD for non-development work) would much rather see a Xen-like :functionality (to be precise: ability to run foreign kernels and :Windows; qemu is too slow) than just a variation of the native kernel. There is certainly a functionality there that people will find useful, but you also have to realize that Xen involves two or more distinct operating systems which will multiply the number of bugs you have to deal with and create major compatibility issues with the underlying hardware, making it less then reliable. Really only the disk and network I/O can be made reliably compatible in a Xen installation. Making sound cards, video capture cards, encryption cards, graphics engines, and many other hardware features work well with the guest operating system will not only be difficult, but it will also be virtually unmaintainable in that environment over the long term. Good luck getting anything more then basic application functionality out of it. For example, you would have no problem running pure network applications such as web and mail servers on the guest operating system, but the moment you delve outside of that box and into sound and high quality (or high performance) video, things won't be so rosy. I don't see much of an advantage in having multi-OS hardware virtualization for any serious deployment. It would be interesting and useful on a personal desktop, at least within the scope of the limited hardware compatibility, but at the same time it will also lock you into software and OS combinations that aren't likely to extend into the future, and which will be a complete an utter nightmare to maintain. Any failure at all could lead to a completely unrecoverable system. -Matt Matthew Dillon [EMAIL PROTECTED] ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
On 3/13/07, Kip Macy [EMAIL PROTECTED] wrote: I know you were working on Xen support in FreeBSD, but web about it (http://www.fsmware.com/xenofreebsd/7.0/STATUS) has one year old info (support planned in FreeBSD 6.1). So is there any progress, or Xen will not be in any near future release? Basically Xen did not mature in the fashion that I anticipated. As far as I can tell it is really only good for server consolidation for large Linux distro vendors. You need to have what amounts to a private branch. The xen developers don't appear to understand the importance of interface versioning. They broke ABI compatibility going from 3.0.2 - 3.0.3 (trivial to fix, but that is not the point). When last I worked on it, they had one branch that was in constant flux and another branch that only received minor bug fixes and was 18 months behind from a functionality standpoint (think 5.x / 4.x). There are numerous other logging / supportability issues that I think are only addressed by the major distros. As it stood 6 months ago, unless you understood the internals of various bits of the code, there was no way of diagnosing failures due to a misconfiguration. This is not to say that it isn't cool technology, but rather that isn't going to be useful for the things I wanted to use it for so my time is being directed elsewhere. If I ever have a need for EC2 I may look at it again. One of the guys who ported FreeBSD to the xbox has expressed interest - so something may yet come of it. I'm happy to provide technical support to an individual who is largely self-sustaining in working on the code. What about implementing something like DragonFly BSD virtual kernels? Matthew Dillon talks about it in is bsdtalk interview: http://cisx1.uma.maine.edu/~wbackman/bsdtalk/bsdtalk098.mp3 I suppose it's sorta like linux compat / windows on windows / coLinux... Towards the end of the interview he talk about it being extremely easy to implement: 1. signal mailboxes. 2. memory map / virtual page table support. 3. vm space management. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
- Nikolas Britton [EMAIL PROTECTED] wrote: What about implementing something like DragonFly BSD virtual kernels? Matthew Dillon talks about it in is bsdtalk interview: http://cisx1.uma.maine.edu/~wbackman/bsdtalk/bsdtalk098.mp3 It seems very similar to User Mode Linux, rather than a true VM environment. http://user-mode-linux.sourceforge.net/ Each DragonFlyBSD vkernel runs as a process. I don't know why this is even interesting, for anything but kernel developers. Improving BSD jails to the same level as Solaris Containers (Solaris Containers are Solaris Zones with resource control), would widely useful for many BSD users. In VM environment, like Xen, each VM has its own kernel and possibly different OS. Xen has managed to get a lot of people interested in their VM environment, so there are a lot of OSes that support the Xen architecture. And for those that don't there is early support for booting them by using virtual features in newer CPUs (ex. Windows). Microsoft has joined the Xen bandwagon, even though the core is all open source, as they are threatened in the enterprise space by the VMWare juggernaut, and their Virtual Server/Virtual PC product is so bland, no one cares. UML has been available for longer than Xen, but Xen already outperforms it. I don't see a lot of future in the virtual kernel concept. Tom ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
Nikolas Britton wrote: On 3/12/07, Andras Gót [EMAIL PROTECTED] wrote: Nikolas Britton wrote: On 3/12/07, Ronald Klop [EMAIL PROTECTED] wrote: On Mon, 12 Mar 2007 20:16:32 +0100, Nikolas Britton [EMAIL PROTECTED] wrote: Is FreeBSD making any progress in Xen Dom0 / Intel VT support? I'd really like to consolidate some underutilized FreeBSD servers. Are their any alternative solutions that will enable me to do this kind of stuff with FreeBSD, or would it be better to go with Solaris Dom0 + FreeBSD DomU? http://docs.freebsd.org/44doc/papers/jail/jail.html google: jail freebsd Yes I'd like to know more about jails, is there a high level / executive summary type document that I can read somewhere? From what I remember jails are mostly designed to partition stuff... for security reasons. What I'd really love to do is split up each service (httpd, postgres, samba/nfs, ldap/nis, asterisk, etc.) into discrete virtual machines. It's too much work trying to make them all play nice on one system, especially during upgrades. As it is right now I don't upgrade any services once a system is in production use. Hi, For first read man jail. :) Apache, bind, mysql and postfix run fine in a jail. For postgres you've to turn on the jail.ipc. This is basicly not so bad, but definitely reduces security. For samba/nfs/ldap/nis and asterisk I don't have the experience, but if they not need ipc, they'll run fine out of the box. In jails I suggest that you mount your ports tree with some nullfs mount. With this you'll save some hd capacity. (The installed port list is in /var, not in /usr/ports.) In jails you can't do resource control, so keep that in mind. Is their anyway to transfer jails on the fly between systems... For example, say I wanted to transfer the http service to a more powerful box because load was too high, can you do stuff like this? You could export the jail file system via nfs, or use some other form of shared storage to share the file system. I have seen systems that put the jail IP address onto the loop back interface and then use OSPF to advertise the service to your border routers. If your storage subsystem supports if (NFS will) you can have both jails up and running at the same time and then just change the routing advertisements to move the service. Just an idea, I have never tried it, but I did see some fail over project that used the methods above. The project advertised the fact that not only can you move services between hosts but also you can move them between physical sites if your routers all run ospf. Tom ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
Kip Macy wrote: On 3/12/07, Ivan Voras [EMAIL PROTECTED] wrote: Nikolas Britton wrote: Free Solaris DVD software kits (Free shipping too): http://www.sun.com/solaris/freemedia Or NetBSD: http://www.netbsd.org/Ports/xen/ Heh, you're still confused about what list your on. I know you were working on Xen support in FreeBSD, but web about it (http://www.fsmware.com/xenofreebsd/7.0/STATUS) has one year old info (support planned in FreeBSD 6.1). So is there any progress, or Xen will not be in any near future release? Miroslav Lachman ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
I know you were working on Xen support in FreeBSD, but web about it (http://www.fsmware.com/xenofreebsd/7.0/STATUS) has one year old info (support planned in FreeBSD 6.1). So is there any progress, or Xen will not be in any near future release? Basically Xen did not mature in the fashion that I anticipated. As far as I can tell it is really only good for server consolidation for large Linux distro vendors. You need to have what amounts to a private branch. The xen developers don't appear to understand the importance of interface versioning. They broke ABI compatibility going from 3.0.2 - 3.0.3 (trivial to fix, but that is not the point). When last I worked on it, they had one branch that was in constant flux and another branch that only received minor bug fixes and was 18 months behind from a functionality standpoint (think 5.x / 4.x). There are numerous other logging / supportability issues that I think are only addressed by the major distros. As it stood 6 months ago, unless you understood the internals of various bits of the code, there was no way of diagnosing failures due to a misconfiguration. This is not to say that it isn't cool technology, but rather that isn't going to be useful for the things I wanted to use it for so my time is being directed elsewhere. If I ever have a need for EC2 I may look at it again. One of the guys who ported FreeBSD to the xbox has expressed interest - so something may yet come of it. I'm happy to provide technical support to an individual who is largely self-sustaining in working on the code. -Kip ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
Free Solaris DVD software kits (Free shipping too): http://www.sun.com/solaris/freemedia Sweet! On 3/12/07, Nikolas Britton [EMAIL PROTECTED] wrote: Is FreeBSD making any progress in Xen Dom0 / Intel VT support? I'd really like to consolidate some underutilized FreeBSD servers. Are their any alternative solutions that will enable me to do this kind of stuff with FreeBSD, or would it be better to go with Solaris Dom0 + FreeBSD DomU? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
On Mon, 12 Mar 2007 20:16:32 +0100, Nikolas Britton [EMAIL PROTECTED] wrote: Is FreeBSD making any progress in Xen Dom0 / Intel VT support? I'd really like to consolidate some underutilized FreeBSD servers. Are their any alternative solutions that will enable me to do this kind of stuff with FreeBSD, or would it be better to go with Solaris Dom0 + FreeBSD DomU? http://docs.freebsd.org/44doc/papers/jail/jail.html google: jail freebsd -- Ronald Klop Amsterdam, The Netherlands ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
Nikolas Britton wrote: Free Solaris DVD software kits (Free shipping too): http://www.sun.com/solaris/freemedia Or NetBSD: http://www.netbsd.org/Ports/xen/ signature.asc Description: OpenPGP digital signature
Re: Xen Dom0, are we making progress?
On 3/12/07, Ivan Voras [EMAIL PROTECTED] wrote: Nikolas Britton wrote: Free Solaris DVD software kits (Free shipping too): http://www.sun.com/solaris/freemedia Or NetBSD: http://www.netbsd.org/Ports/xen/ Yes that was my next choice after FreeBSD but they don't support the Areca RAID controllers we use and IIRC they have no plans to port the FreeBSD arcmsr driver. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
On 3/12/07, Ivan Voras [EMAIL PROTECTED] wrote: Nikolas Britton wrote: Free Solaris DVD software kits (Free shipping too): http://www.sun.com/solaris/freemedia Or NetBSD: http://www.netbsd.org/Ports/xen/ Heh, you're still confused about what list your on. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
On Mar 12, 2007, at 4:34 PM, Nikolas Britton wrote: What I'd really love to do is split up each service (httpd, postgres, samba/nfs, ldap/nis, asterisk, etc.) into discrete virtual machines. It's too much work trying to make them all play nice on one system, This is the purpose for which we (ab)use jails for too. Works great, less filling. Check out ezjail in the ports; it makes very lightweight jails by using nullfs to share the full install tree across multiple jails. This may not be what you want, but it is what I wanted. What you don't get is a private kernel per jail, and some services are not really virtualized like network, and SYSV IPC. And you can only assign one IP per jail. For what I need, Xen is overkill. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
Nikolas Britton wrote: On 3/12/07, Ronald Klop [EMAIL PROTECTED] wrote: On Mon, 12 Mar 2007 20:16:32 +0100, Nikolas Britton [EMAIL PROTECTED] wrote: Is FreeBSD making any progress in Xen Dom0 / Intel VT support? I'd really like to consolidate some underutilized FreeBSD servers. Are their any alternative solutions that will enable me to do this kind of stuff with FreeBSD, or would it be better to go with Solaris Dom0 + FreeBSD DomU? http://docs.freebsd.org/44doc/papers/jail/jail.html google: jail freebsd Yes I'd like to know more about jails, is there a high level / executive summary type document that I can read somewhere? From what I remember jails are mostly designed to partition stuff... for security reasons. What I'd really love to do is split up each service (httpd, postgres, samba/nfs, ldap/nis, asterisk, etc.) into discrete virtual machines. It's too much work trying to make them all play nice on one system, especially during upgrades. As it is right now I don't upgrade any services once a system is in production use. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] Hi, For first read man jail. :) Apache, bind, mysql and postfix run fine in a jail. For postgres you've to turn on the jail.ipc. This is basicly not so bad, but definitely reduces security. For samba/nfs/ldap/nis and asterisk I don't have the experience, but if they not need ipc, they'll run fine out of the box. In jails I suggest that you mount your ports tree with some nullfs mount. With this you'll save some hd capacity. (The installed port list is in /var, not in /usr/ports.) In jails you can't do resource control, so keep that in mind. Regards, Andras ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
On 3/12/07, Ronald Klop [EMAIL PROTECTED] wrote: On Mon, 12 Mar 2007 20:16:32 +0100, Nikolas Britton [EMAIL PROTECTED] wrote: Is FreeBSD making any progress in Xen Dom0 / Intel VT support? I'd really like to consolidate some underutilized FreeBSD servers. Are their any alternative solutions that will enable me to do this kind of stuff with FreeBSD, or would it be better to go with Solaris Dom0 + FreeBSD DomU? http://docs.freebsd.org/44doc/papers/jail/jail.html google: jail freebsd Yes I'd like to know more about jails, is there a high level / executive summary type document that I can read somewhere? From what I remember jails are mostly designed to partition stuff... for security reasons. What I'd really love to do is split up each service (httpd, postgres, samba/nfs, ldap/nis, asterisk, etc.) into discrete virtual machines. It's too much work trying to make them all play nice on one system, especially during upgrades. As it is right now I don't upgrade any services once a system is in production use. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Xen Dom0, are we making progress?
On 3/12/07, Andras Gót [EMAIL PROTECTED] wrote: Nikolas Britton wrote: On 3/12/07, Ronald Klop [EMAIL PROTECTED] wrote: On Mon, 12 Mar 2007 20:16:32 +0100, Nikolas Britton [EMAIL PROTECTED] wrote: Is FreeBSD making any progress in Xen Dom0 / Intel VT support? I'd really like to consolidate some underutilized FreeBSD servers. Are their any alternative solutions that will enable me to do this kind of stuff with FreeBSD, or would it be better to go with Solaris Dom0 + FreeBSD DomU? http://docs.freebsd.org/44doc/papers/jail/jail.html google: jail freebsd Yes I'd like to know more about jails, is there a high level / executive summary type document that I can read somewhere? From what I remember jails are mostly designed to partition stuff... for security reasons. What I'd really love to do is split up each service (httpd, postgres, samba/nfs, ldap/nis, asterisk, etc.) into discrete virtual machines. It's too much work trying to make them all play nice on one system, especially during upgrades. As it is right now I don't upgrade any services once a system is in production use. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED] Hi, For first read man jail. :) Apache, bind, mysql and postfix run fine in a jail. For postgres you've to turn on the jail.ipc. This is basicly not so bad, but definitely reduces security. For samba/nfs/ldap/nis and asterisk I don't have the experience, but if they not need ipc, they'll run fine out of the box. In jails I suggest that you mount your ports tree with some nullfs mount. With this you'll save some hd capacity. (The installed port list is in /var, not in /usr/ports.) In jails you can't do resource control, so keep that in mind. Is their anyway to transfer jails on the fly between systems... For example, say I wanted to transfer the http service to a more powerful box because load was too high, can you do stuff like this? ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
