Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 2013-09-04 2:49 AM, Marc Stürmer m...@marc-stuermer.de wrote: Well in my point of view it boils down to that: someone wants to use ZFS on Linux. Fine. This means you've got to be a good citizen and obey its license, of course. It is for those legal reasons that ZFS is not included into the Linux kernel mainline source tree. It is also for those reasons you got to compile it as a module. One of the points made was that this is FUD, and that there is NO logel reason that it cannot be included. There is also the fact that it *could* be included, as long as it wasn't provided directly in the kernel sources, but as an overlay/patch type process, which could still be provided by the gentoo source repositories.
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Am 02.09.2013 10:47, schrieb Joerg Schilling: Solaris is dynamic from the beginning: Well in my point of view it boils down to that: someone wants to use ZFS on Linux. Fine. This means you've got to be a good citizen and obey its license, of course. It is for those legal reasons that ZFS is not included into the Linux kernel mainline source tree. It is also for those reasons you got to compile it as a module. So somebody wants it being static into his kernel, modules being disabled on his machine because of security concerns. Unless he is going to do that stuff himself this is unlikely to ever happen. So it boils down to those possible solutions: a) writing that stuff himself (unlikely to happen), b) just using the module and going to be happy (also unlikely to happen as it seems), c) choosing another, native file system like Btrfs (which is still yet not production ready as a fast moving target) or going with something like XFS or Ext4 (and LVM), or the most natural choice then, which is d) choosing an operating system, which supports ZFS out of the box like FreeBSD and forget about all the rest of the problems. I would go for d and forget about all of the rest of the problems. FreeBSD has been around long enough, and is stable and mature enough for most anything you can throw at and it is a nice, clean, well structured system anyway. There's also Gentoo/FreeBSD around, but personally I would use the native ports system instead.
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Walter Dnes waltd...@waltdnes.org wrote: Grub works this way: 1) It loads /platform/i86pc/kernel/$ISADIR/unix Question... how does it read that file off a ZFS partition? OK, so ZFS code has to be installed statically into GRUB instead of statically into the kernel. Please stop the shell game. Grub was enhanced by Sun to understand ZFS. You need such an enhanced grub if you like to boot off ZFS. Note also that this is a Gentoo *LINUX* mailing list. We're more concerned about how Linux works. Linux does not contain code to boot AFAIK Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Mon, Sep 2, 2013 at 4:47 PM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Mark David Dumlao madum...@gmail.com wrote: the disk... OOPS. This is a classic chicken and egg situation. On Solaris no problem with loadable modules - everything is dynamically loaded. ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***. I'm confused as to what this means. Grub reads a filesystem, loads a kernel with options, and may give it an initrd. What happens from then on is none of grub's business. The filesystem it reads from and the one the kernel uses may be completely unrelated - this is why we have /boot filesystems. At what point does grub present a zfs interface for the kernel to use? After it booted the kernel You may not know dynamic kernels as Linux is a static kernel that just may load additional modules _after_ it mounted the root fs. Solaris is dynamic from the beginning: Ah I see. But I think by default when we talk about the kernel on this mailing list, it's assumed that we're talking about Linux. And in the Linux case, Grub does not do anything like provide a filesystem interface to Linux. It just loads the kernel into memory, and passes it any arguments, like the initrd. So your grub needs to be able to read the filesystem containing the kernel and that's it. If the filesystem containing the kernel is also a zfs filesystem, then your grub needs a driver that can read that filesystem. Well sys-boot/grub-2.00 provides one. See /boot/grub/zfs.mod -- This email is:[ ] actionable [x] fyi[ ] social Response needed: [ ] yes [ ] up to you [x] no Time-sensitive: [ ] immediate[ ] soon [x] none
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Mark David Dumlao madum...@gmail.com wrote: containing the kernel is also a zfs filesystem, then your grub needs a driver that can read that filesystem. Well sys-boot/grub-2.00 provides one. See /boot/grub/zfs.mod You don't need grub2, a capable older grub does it also, see: http://hg.berlios.de/repos/schillix-on for a related source. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: Integrated ZFS for Gentoo - WAS Re:[gentoo-user] Optional /usr merge in Gentoo
The 04/09/13, Joerg Schilling wrote: Linux does not contain code to boot AFAIK Sure, it does. You can boot on the kernel directly without a boot manager. -- Nicolas Sebrecht
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Mon, Sep 02, 2013 at 10:47:35AM +0200, Joerg Schilling wrote Mark David Dumlao madum...@gmail.com wrote: At what point does grub present a zfs interface for the kernel to use? After it booted the kernel You may not know dynamic kernels as Linux is a static kernel that just may load additional modules _after_ it mounted the root fs. Solaris is dynamic from the beginning: - no static loading at all - no predefined data sizes - everything is allocated - no predefined major device numbers - numbers are assigned at first load Grub works this way: 1)It loads /platform/i86pc/kernel/$ISADIR/unix Question... how does it read that file off a ZFS partition? OK, so ZFS code has to be installed statically into GRUB instead of statically into the kernel. Please stop the shell game. Note also that this is a Gentoo *LINUX* mailing list. We're more concerned about how Linux works. -- Walter Dnes waltd...@waltdnes.org I don't run desktop environments; I run useful applications
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Mark David Dumlao madum...@gmail.com wrote: the disk... OOPS. This is a classic chicken and egg situation. On Solaris no problem with loadable modules - everything is dynamically loaded. ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***. I'm confused as to what this means. Grub reads a filesystem, loads a kernel with options, and may give it an initrd. What happens from then on is none of grub's business. The filesystem it reads from and the one the kernel uses may be completely unrelated - this is why we have /boot filesystems. At what point does grub present a zfs interface for the kernel to use? After it booted the kernel You may not know dynamic kernels as Linux is a static kernel that just may load additional modules _after_ it mounted the root fs. Solaris is dynamic from the beginning: - no static loading at all - no predefined data sizes - everything is allocated - no predefined major device numbers - numbers are assigned at first load Grub works this way: 1) It loads /platform/i86pc/kernel/$ISADIR/unix 2) It checks the file unix and sees ELF dependencies. It loads the ELF dependencies (genunix and dtracestubs) listed in the ELF headers from unix. 3) It loads /platform/i86pc/$ISADIR/boot_archive The Kernel then uses the filesystem callbacks in grub to load modules from the filesystem in the boot archive. After the kernel did mount the root filesystem, it switches to the normal kernel drivers just loaded and frees the memory space used by grub before. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Walter Dnes waltd...@waltdnes.org wrote: You can get away with most stuff as modules; ***BUT NOT THE ROOT FILESYSTEM***. Think about it for a minute. Gentoo reads modules off the disk. If the code for the root filesystem is a module, Gentoo would have to read the module off the disk to enable it to read the module off the disk... OOPS. This is a classic chicken and egg situation. On Solaris no problem with loadable modules - everything is dynamically loaded. You need a grub that understands ZFS and that gives a ZFS interface to the kernel to use before ZFS was loaded. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 2013-08-31 7:29 AM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Tanstaafltansta...@libertytrek.org wrote: You must have missed the point that this is for*servers*, that most people*disable modules* on. I*know* that it is available as a module. Why, for security reasons? Because if you don't need something, why enable it? If modules are totally disabled, then there is no worry about any security issue involving modules at all.
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 2013-08-31 11:55 PM, Walter Dnes waltd...@waltdnes.org wrote: Also, I really wonder what the point is in having to use initramfs on a system where /usr is part of /. You don't, it is only *required* if you have a separate /usr... in fact that is what the whole argument was about. At least that is my understanding of the situation now... please don't tell me I'm wrong and there was another vote and it is now required just to be able to use gentoo?
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 2013-08-31 7:32 AM, Alon Bar-Lev alo...@gentoo.org wrote: If this is not mainline, and it is not trivial gentoo kernels maintainer patch, and you must have this as static, you can just put the patch within/etc/portage/patches/sys-kernel/gentoo-sources/, so it will patch your kernel every time you emerge new one. Interesting, but this would require manually updating the patch every time, right? Or could the 'patch' be configured to automatically pull the right version (compatible with the kernel being installed) every time? That would not be such a bad thing... but if not... well... Computers excel at automating things. People excel at breaking things, and I'd like this to be automated as much as possible. That said, I've never applied patches in this manner, so, is there an up to date how-to on how to do this? It might be something I can get comfortable with unless/until an automated process is implemented. On 2013-08-31 8:19 AM, Joerg Schilling wrote: So there seems to be no real need to create a static linux kernel with ZFS inside. sigh There is for those who *do not want modules enabled on their servers*. Why is it so hard for some people to just not get that their way is not the only way. Again, Joerg... please *stop arguing* about this point, it has *nothing* to do with the thread. On 2013-08-31 2:44 PM, Mark David Dumlao madum...@gmail.com wrote: You must have missed the point that this is for *servers*, that most people *disable modules* on. I*know* that it is available as a module. Ok, I was just asking. But as for what most people do on their servers, speak for yourself. Ok, I left out two words: '... I know ... ' - and the fact is, most everyone I know (over a dozen) who runs linux servers (not just gentoo) runs them with modules disabled, and I've seen countless others say the same thing over the years... The fact is, *many* people do this, and if it trivial to implement it in gentoo (which appears it is), then why not do so?
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sun, Sep 01, 2013 at 09:49:23AM +0200, Joerg Schilling wrote Walter Dnes waltd...@waltdnes.org wrote: You can get away with most stuff as modules; ***BUT NOT THE ROOT FILESYSTEM***. Think about it for a minute. Gentoo reads modules off the disk. If the code for the root filesystem is a module, Gentoo would have to read the module off the disk to enable it to read the module off the disk... OOPS. This is a classic chicken and egg situation. On Solaris no problem with loadable modules - everything is dynamically loaded. ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***. So instead of needing ZFS built into the kernel, you need ZFS built into GRUB... ***AND*** you need a ZFS module for the main system... ***AND*** you need to keep both versions in sync. I'm not impressed. -- Walter Dnes waltd...@waltdnes.org I don't run desktop environments; I run useful applications
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sun, Sep 01, 2013 at 10:11:01AM -0400, Tanstaafl wrote You don't, it is only *required* if you have a separate /usr... in fact that is what the whole argument was about. At least that is my understanding of the situation now... please don't tell me I'm wrong and there was another vote and it is now required just to be able to use gentoo? This is for the people who want *EVERYTHING INCLUDING THE ROOT FILE SYSTEM CODE* built as a module. Note that the Gentoo (AMD64) docs at http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7say... Don't compile the file system you use for the root filesystem as module, otherwise your Gentoo system will not be able to mount your partition. Using an initramfs allows you to ignore that warning. -- Walter Dnes waltd...@waltdnes.org I don't run desktop environments; I run useful applications
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sep 2, 2013 5:21 AM, Walter Dnes waltd...@waltdnes.org wrote: On Sun, Sep 01, 2013 at 09:49:23AM +0200, Joerg Schilling wrote Walter Dnes waltd...@waltdnes.org wrote: You can get away with most stuff as modules; ***BUT NOT THE ROOT FILESYSTEM***. Think about it for a minute. Gentoo reads modules off the disk. If the code for the root filesystem is a module, Gentoo would have to read the module off the disk to enable it to read the module off the disk... OOPS. This is a classic chicken and egg situation. On Solaris no problem with loadable modules - everything is dynamically loaded. ***YOU NEED A GRUB THAT UNDERSTANDS ZFS AND THAT GIVES A ZFS INTERFACE TO THE KERNEL TO USE BEFORE ZFS WAS LOADED***. I'm confused as to what this means. Grub reads a filesystem, loads a kernel with options, and may give it an initrd. What happens from then on is none of grub's business. The filesystem it reads from and the one the kernel uses may be completely unrelated - this is why we have /boot filesystems. At what point does grub present a zfs interface for the kernel to use?
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sun, Sep 01, 2013 at 01:41:30PM +0800, Mark David Dumlao wrote Case in point - do you enable all the ext4 options, like acls and whatnot? Let's say no. What if you suddenly have to mount an external hard disk to recover some system on your server and the hard disk uses those ext4 options? If ext4 is hard built into your kernel, your recompile will have to basically redo the whole thing, whereas if ext4 was a module you would only recompile ext4 itself. Have you ever actually done this? I'd be very leery of pulling such a stunt. The clean way of switching module versions is to... * unload the old module, and * load the new module You obviously can't do this in your setup, because unloading the old module would mean you could no longer access the file system to read in the new module... OOPS. You could run a script that creates /dev/shm/lib/3.1.4.1.5.9-gentoo/ (easy as pieG) and copies the new module to that dir. Then unload the old module and load the new one, using modprobe with -d /dev/shm/. That still looks impossible. The problem is that you generally have a whole bunch of files open at any time. E.g. try... lsof -d txt | grep -v /proc/ | less ...and look at the output. Shutting down all those open files would be disastrous. But that's not what you're saying. You seem to imply that file system code can be overwritten *IN PLACE, WHILE IN USE*, without any problems. Colour me skeptical about that one. -- Walter Dnes waltd...@waltdnes.org I don't run desktop environments; I run useful applications
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Friday 30 Aug 2013 21:21:10 Alan McKinnon wrote: Ahem, Mr Bothwick! Our friend with the thing about free lunches needs you to demonstrate your penmanship, considering you have some proven results in this area. ...and I'd happily act as editor... :-) ;-) -- Regards, Peter
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sat, Aug 31, 2013 at 12:10 PM, Mark David Dumlao madum...@gmail.com wrote: On Sat, Aug 31, 2013 at 4:16 AM, Mick michaelkintz...@gmail.com wrote: On Friday 30 Aug 2013 15:44:35 Tanstaafl wrote: On 2013-08-30 10:34 AM, Alan McKinnon alan.mckin...@gmail.com wrote: On 30/08/2013 16:29, Tanstaafl wrote: Why would there be a problem if someone decided to create a 3rd party overlay *not* part of the official gentoo portage tree that contained *only* the zfs stuff, and when this overlay was installed combined with a zfs keyword for the kernel, portage would then pull in the required files, and automagically build a kernel with an up to date version of zfs properly and fully integrated? Would this not work, *and* have no problems with licensing? there is no problem with licensing in that case. The ebuild could even go in the portage tree, as Gentoo is not redistributing sources when it publishes an ebuild. Thanks Alan! Just the answer I wanted. Ok, so... how hard would this be then? What would the chances be that this could actually happen? I'll happily go open a bug for it if you think the work would be minimal... It seems to me that I can't be the only one who would like to see this happen? Nope! I will vote for you. ;-) -- Regards, Mick Sounds like an awful lot of trouble for a problem that's already solved by installing sys-kernel/module-rebuild and running module-rebuild rebuild after every kernel update, which is how nvidia, broadcom, and other kernel modules are dealt painlessly with anyways... Well, if you follow Tanstaafl in the other thread, you'll see that he wants ZFS to be integrated into the kernel, not existing as a kernel module. Rgds, -- FdS Pandu E Poluan ~ IT Optimizer ~ • LOPSA Member #15248 • Blog : http://pepoluan.tumblr.com • Linked-In : http://id.linkedin.com/in/pepoluan
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Pandu Poluan pa...@poluan.info wrote: Well, if you follow Tanstaafl in the other thread, you'll see that he wants ZFS to be integrated into the kernel, not existing as a kernel module. But why does someone want things to be inside a static kernel? Since 1991/1992, Solaris does not have anything in the static kernel than the startup code, the basic scheduler code and the pager daemon. You need a bootloader that knows about ELF dependencies, but grub has been enhanced for that feature. Everything is dynamic, you would however put a lot of effort into the linux kernel to get to that state...e.g. automated major device numbering. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 2013-08-31 1:10 AM, Mark David Dumlao madum...@gmail.com wrote: Sounds like an awful lot of trouble for a problem that's already solved by installing sys-kernel/module-rebuild and running module-rebuild rebuild after every kernel update, which is how nvidia, broadcom, and other kernel modules are dealt painlessly with anyways... You must have missed the point that this is for *servers*, that most people *disable modules* on. I *know* that it is available as a module.
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 2013-08-31 7:04 AM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Everything is dynamic, you would however put a lot of effort into the linux kernel to get to that state...e.g. automated major device numbering. ??? I've been running my servers without modules since... I started running servers. Servers are not like desktops - constantly changing devices. They - in most cases - *are* static, and most people *want* them that way. Regardless, please do *not* distract this thread with arguments about it. If you don't want or see the benefit, fine, just ignore this thread.
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Tanstaafl tansta...@libertytrek.org wrote: On 2013-08-31 1:10 AM, Mark David Dumlao madum...@gmail.com wrote: Sounds like an awful lot of trouble for a problem that's already solved by installing sys-kernel/module-rebuild and running module-rebuild rebuild after every kernel update, which is how nvidia, broadcom, and other kernel modules are dealt painlessly with anyways... You must have missed the point that this is for *servers*, that most people *disable modules* on. I *know* that it is available as a module. Why, for security reasons? On Solaris, you can disable loading unsigned modules, is this not supported by Linux? Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sat, Aug 31, 2013 at 2:28 PM, Tanstaafl tansta...@libertytrek.org wrote: On 2013-08-31 7:04 AM, Joerg Schilling joerg.schill...@fokus.fraunhofer.de wrote: Everything is dynamic, you would however put a lot of effort into the linux kernel to get to that state...e.g. automated major device numbering. ??? I've been running my servers without modules since... I started running servers. Servers are not like desktops - constantly changing devices. They - in most cases - *are* static, and most people *want* them that way. Regardless, please do *not* distract this thread with arguments about it. If you don't want or see the benefit, fine, just ignore this thread. I do not understand this thread. If this is not mainline, and it is not trivial gentoo kernels maintainer patch, and you must have this as static, you can just put the patch within /etc/portage/patches/sys-kernel/gentoo-sources/, so it will patch your kernel every time you emerge new one. Regards, Alon
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
In linux.gentoo.user, Mr Schilling wrote: On Solaris, you can disable loading unsigned modules, is this not supported by Linux? CONFIG_MODULE_SIG -- Regards, Gregory.
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
Gregory Shearman zek...@gmail.com wrote: In linux.gentoo.user, Mr Schilling wrote: On Solaris, you can disable loading unsigned modules, is this not supported by Linux? CONFIG_MODULE_SIG So there seems to be no real need to create a static linux kernel with ZFS inside. Jörg -- EMail:jo...@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin j...@cs.tu-berlin.de(uni) joerg.schill...@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sat, Aug 31, 2013 at 7:25 PM, Tanstaafl tansta...@libertytrek.org wrote: On 2013-08-31 1:10 AM, Mark David Dumlao madum...@gmail.com wrote: Sounds like an awful lot of trouble for a problem that's already solved by installing sys-kernel/module-rebuild and running module-rebuild rebuild after every kernel update, which is how nvidia, broadcom, and other kernel modules are dealt painlessly with anyways... You must have missed the point that this is for *servers*, that most people *disable modules* on. I *know* that it is available as a module. Ok, I was just asking. But as for what most people do on their servers, speak for yourself. -- This email is:[ ] actionable [ ] fyi[x] social Response needed: [ ] yes [ ] up to you [x] no Time-sensitive: [ ] immediate[ ] soon [x] none
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote So there seems to be no real need to create a static linux kernel with ZFS inside. See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7 Now go to File Systems and select support for the filesystems you use. Don't compile the file system you use for the root filesystem as module, otherwise your Gentoo system will not be able to mount your partition. You can get away with most stuff as modules; ***BUT NOT THE ROOT FILESYSTEM***. Think about it for a minute. Gentoo reads modules off the disk. If the code for the root filesystem is a module, Gentoo would have to read the module off the disk to enable it to read the module off the disk... OOPS. This is a classic chicken and egg situation. -- Walter Dnes waltd...@waltdnes.org I don't run desktop environments; I run useful applications
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sat, Aug 31, 2013 at 7:13 PM, Walter Dnes waltd...@waltdnes.org wrote: On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote So there seems to be no real need to create a static linux kernel with ZFS inside. See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7 Now go to File Systems and select support for the filesystems you use. Don't compile the file system you use for the root filesystem as module, otherwise your Gentoo system will not be able to mount your partition. You can get away with most stuff as modules; ***BUT NOT THE ROOT FILESYSTEM***. Think about it for a minute. Gentoo reads modules off the disk. If the code for the root filesystem is a module, Gentoo would have to read the module off the disk to enable it to read the module off the disk... OOPS. This is a classic chicken and egg situation. I usally use ext4 as filesystem. # lsmod|grep ext ext3 100768 0 jbd39586 1 ext3 ext2 49572 0 ext4 263621 1 crc16 1255 2 ext4,bluetooth mbcache 4450 3 ext2,ext3,ext4 jbd2 48679 1 ext4 Isn't great what an initramfs can do? Regards. -- Canek Peláez Valdés Posgrado en Ciencia e Ingeniería de la Computación Universidad Nacional Autónoma de México
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sun, Sep 1, 2013 at 8:13 AM, Walter Dnes waltd...@waltdnes.org wrote: On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote So there seems to be no real need to create a static linux kernel with ZFS inside. See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7 Now go to File Systems and select support for the filesystems you use. Don't compile the file system you use for the root filesystem as module, otherwise your Gentoo system will not be able to mount your partition. You can get away with most stuff as modules; ***BUT NOT THE ROOT FILESYSTEM***. Think about it for a minute. Gentoo reads modules off the disk. If the code for the root filesystem is a module, Gentoo would have to read the module off the disk to enable it to read the module off the disk... OOPS. This is a classic chicken and egg situation. And this is why the initrd was actually invented. http://en.wikipedia.org/wiki/Initrd It's a means of loading kernel modules so that the root filesystem can be mounted as a module. -- This email is:[ ] actionable [x] fyi[ ] social Response needed: [ ] yes [ ] up to you [x] no Time-sensitive: [ ] immediate[ ] soon [x] none
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sep 1, 2013 7:51 AM, Mark David Dumlao madum...@gmail.com wrote: On Sun, Sep 1, 2013 at 8:13 AM, Walter Dnes waltd...@waltdnes.org wrote: On Sat, Aug 31, 2013 at 02:19:56PM +0200, Joerg Schilling wrote So there seems to be no real need to create a static linux kernel with ZFS inside. See http://www.gentoo.org/doc/en/handbook/handbook-amd64.xml?full=1#book_part1_chap7 Now go to File Systems and select support for the filesystems you use. Don't compile the file system you use for the root filesystem as module, otherwise your Gentoo system will not be able to mount your partition. You can get away with most stuff as modules; ***BUT NOT THE ROOT FILESYSTEM***. Think about it for a minute. Gentoo reads modules off the disk. If the code for the root filesystem is a module, Gentoo would have to read the module off the disk to enable it to read the module off the disk... OOPS. This is a classic chicken and egg situation. And this is why the initrd was actually invented. http://en.wikipedia.org/wiki/Initrd It's a means of loading kernel modules so that the root filesystem can be mounted as a module. Not everyone is willing to use an initr* thingy. It's another potential breaking point. I have no problem with /usr being 'merged' with /, in fact I have been doing that for a couple of years now. But I will keep myself a mile away from an initr* thingy. Rgds, --
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
I usally use ext4 as filesystem. # lsmod|grep ext ext3 100768 0 jbd39586 1 ext3 ext2 49572 0 ext4 263621 1 crc16 1255 2 ext4,bluetooth mbcache 4450 3 ext2,ext3,ext4 jbd2 48679 1 ext4 Isn't great what an initramfs can do? In this case, initramfs is your root filesystem, from which you load another fs and then transfer (pivot root?) to it. You have to build initramfs support into the kernel, to boot an initramfs. So my argument still stands, regardless of whether your *INITIAL* filesystem is ext4fs, or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the kernel. Also, I really wonder what the point is in having to use initramfs on a system where /usr is part of /. -- Walter Dnes waltd...@waltdnes.org I don't run desktop environments; I run useful applications
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sat, Aug 31, 2013 at 10:55 PM, Walter Dnes waltd...@waltdnes.org wrote: I usally use ext4 as filesystem. # lsmod|grep ext ext3 100768 0 jbd39586 1 ext3 ext2 49572 0 ext4 263621 1 crc16 1255 2 ext4,bluetooth mbcache 4450 3 ext2,ext3,ext4 jbd2 48679 1 ext4 Isn't great what an initramfs can do? In this case, initramfs is your root filesystem, from which you load another fs and then transfer (pivot root?) to it. You have to build initramfs support into the kernel, to boot an initramfs. So my argument still stands, regardless of whether your *INITIAL* filesystem is ext4fs, or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the kernel. Interesting perspective. Of course, support for an initramfs is not actually a file system (it's not even in the File systems section of the kernel configuration, is in General setup); it's not possible to have initramfs as a module (that would make no sense at all); and it's code that is several orders of magnitude more simpler than the one used by ext4 (or any other journal file system). But you are right that for booting with an initramfs, you need initramfs support. Also, I really wonder what the point is in having to use initramfs on a system where /usr is part of /. Well, since some months ago I've been running as a module almost everything that can be compiled as a module. This allows me to run a *truly* minimal kernel, and only the necessary modules autoload automatically (one big exception: binfmt_script, I compiled that into the kernel because it was not loading automatically). I can also unload some modules when not in use anymore (and this is great to debug sometimes). This also lets me to add a lot of stuff in the kernel, as long as I add them as modules, without me worrying about bloating my kernel. Only when they are needed they are loaded. I have USB speakers, but I almost never use them; no problem, they (like almost everything else) live as modules, and only are loaded (automagically, thanks to udev) when needed. And again, I can unload them when not in use. And also, it turns out that by using dracut+systemd you could boot faster than without initramfs (although I can't find the link anymore). Finally, using only modules and dracut liberates me from thinking what should it be compiled in and what not; I just put *everything* as a module, and the kernel, udev and dracut take care of loading what's necessary. Thus, my kernel (the one running in memory) is as minimal as it can be, all the time. Oh, and one more thing; by having everything as a module, if suddenly I need support for new hardware, usually I can do a quick make menuconfig; make modules_install, and the new module can be modprobe'd into the kernel without needing a reboot. That's convenient. Regards. -- Canek Peláez Valdés Posgrado en Ciencia e Ingeniería de la Computación Universidad Nacional Autónoma de México
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sun, Sep 1, 2013 at 11:55 AM, Walter Dnes waltd...@waltdnes.org wrote: I usally use ext4 as filesystem. # lsmod|grep ext ext3 100768 0 jbd39586 1 ext3 ext2 49572 0 ext4 263621 1 crc16 1255 2 ext4,bluetooth mbcache 4450 3 ext2,ext3,ext4 jbd2 48679 1 ext4 Isn't great what an initramfs can do? In this case, initramfs is your root filesystem, from which you load another fs and then transfer (pivot root?) to it. You have to build initramfs support into the kernel, to boot an initramfs. So my argument still stands, regardless of whether your *INITIAL* filesystem is ext4fs, or ZFS, or initramfs, that *INITIAL* filesystem has to be built into the kernel. Also, I really wonder what the point is in having to use initramfs on a system where /usr is part of /. It allows you to keep some kernel bits in modules. If ever you change your mind on whether to include / exclude / reconfigure those kernel bits in the future, your kernel recompile will take a lot, lot, shorter. Case in point - do you enable all the ext4 options, like acls and whatnot? Let's say no. What if you suddenly have to mount an external hard disk to recover some system on your server and the hard disk uses those ext4 options? If ext4 is hard built into your kernel, your recompile will have to basically redo the whole thing, whereas if ext4 was a module you would only recompile ext4 itself. -- This email is:[ ] actionable [x] fyi[ ] social Response needed: [ ] yes [ ] up to you [x] no Time-sensitive: [ ] immediate[ ] soon [x] none
Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 2013-08-30 10:34 AM, Alan McKinnon alan.mckin...@gmail.com wrote: On 30/08/2013 16:29, Tanstaafl wrote: Why would there be a problem if someone decided to create a 3rd party overlay *not* part of the official gentoo portage tree that contained *only* the zfs stuff, and when this overlay was installed combined with a zfs keyword for the kernel, portage would then pull in the required files, and automagically build a kernel with an up to date version of zfs properly and fully integrated? Would this not work, *and* have no problems with licensing? there is no problem with licensing in that case. The ebuild could even go in the portage tree, as Gentoo is not redistributing sources when it publishes an ebuild. Thanks Alan! Just the answer I wanted. Ok, so... how hard would this be then? What would the chances be that this could actually happen? I'll happily go open a bug for it if you think the work would be minimal... It seems to me that I can't be the only one who would like to see this happen?
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On 30/08/2013 16:44, Tanstaafl wrote: On 2013-08-30 10:34 AM, Alan McKinnon alan.mckin...@gmail.com wrote: On 30/08/2013 16:29, Tanstaafl wrote: Why would there be a problem if someone decided to create a 3rd party overlay *not* part of the official gentoo portage tree that contained *only* the zfs stuff, and when this overlay was installed combined with a zfs keyword for the kernel, portage would then pull in the required files, and automagically build a kernel with an up to date version of zfs properly and fully integrated? Would this not work, *and* have no problems with licensing? there is no problem with licensing in that case. The ebuild could even go in the portage tree, as Gentoo is not redistributing sources when it publishes an ebuild. Thanks Alan! Just the answer I wanted. Ok, so... how hard would this be then? What would the chances be that this could actually happen? I'll happily go open a bug for it if you think the work would be minimal... It seems to me that I can't be the only one who would like to see this happen? Ahem, Mr Bothwick! Our friend with the thing about free lunches needs you to demonstrate your penmanship, considering you have some proven results in this area. :-) -- Alan McKinnon alan.mckin...@gmail.com
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Friday 30 Aug 2013 15:44:35 Tanstaafl wrote: On 2013-08-30 10:34 AM, Alan McKinnon alan.mckin...@gmail.com wrote: On 30/08/2013 16:29, Tanstaafl wrote: Why would there be a problem if someone decided to create a 3rd party overlay *not* part of the official gentoo portage tree that contained *only* the zfs stuff, and when this overlay was installed combined with a zfs keyword for the kernel, portage would then pull in the required files, and automagically build a kernel with an up to date version of zfs properly and fully integrated? Would this not work, *and* have no problems with licensing? there is no problem with licensing in that case. The ebuild could even go in the portage tree, as Gentoo is not redistributing sources when it publishes an ebuild. Thanks Alan! Just the answer I wanted. Ok, so... how hard would this be then? What would the chances be that this could actually happen? I'll happily go open a bug for it if you think the work would be minimal... It seems to me that I can't be the only one who would like to see this happen? Nope! I will vote for you. ;-) -- Regards, Mick signature.asc Description: This is a digitally signed message part.
Re: Integrated ZFS for Gentoo - WAS Re: [gentoo-user] Optional /usr merge in Gentoo
On Sat, Aug 31, 2013 at 4:16 AM, Mick michaelkintz...@gmail.com wrote: On Friday 30 Aug 2013 15:44:35 Tanstaafl wrote: On 2013-08-30 10:34 AM, Alan McKinnon alan.mckin...@gmail.com wrote: On 30/08/2013 16:29, Tanstaafl wrote: Why would there be a problem if someone decided to create a 3rd party overlay *not* part of the official gentoo portage tree that contained *only* the zfs stuff, and when this overlay was installed combined with a zfs keyword for the kernel, portage would then pull in the required files, and automagically build a kernel with an up to date version of zfs properly and fully integrated? Would this not work, *and* have no problems with licensing? there is no problem with licensing in that case. The ebuild could even go in the portage tree, as Gentoo is not redistributing sources when it publishes an ebuild. Thanks Alan! Just the answer I wanted. Ok, so... how hard would this be then? What would the chances be that this could actually happen? I'll happily go open a bug for it if you think the work would be minimal... It seems to me that I can't be the only one who would like to see this happen? Nope! I will vote for you. ;-) -- Regards, Mick Sounds like an awful lot of trouble for a problem that's already solved by installing sys-kernel/module-rebuild and running module-rebuild rebuild after every kernel update, which is how nvidia, broadcom, and other kernel modules are dealt painlessly with anyways... -- This email is:[ ] actionable [x] fyi[ ] social Response needed: [ ] yes [x] up to you [ ] no Time-sensitive: [ ] immediate[ ] soon [x] none
