Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Sat, Dec 29, 2012 at 3:00 PM, Paul Colquhoun paul...@andor.dropbear.id.au wrote: I'd certainly be happy fixing FHS to say that tools for mounting and recovering essential system partitions be located in /, and that these essential system partitions contain the tools for mounting and recovering non-essential partitions. The beef with the comment on /home being nonessential is besides the point, /usr, /var, or /opt could have been some special case FUSE filesystem, making it still impossible to predict which files _should_ be in /. The more relevant matter here is that plan FHS, in combination with FUSE, makes that difficult. -- This email is:[ ] actionable [ ] fyi[x] social Response needed: [ ] yes [x] up to you [ ] no Time-sensitive: [ ] immediate[ ] soon [x] none
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Sun, 30 Dec 2012 20:19:44 +0800 Mark David Dumlao madum...@gmail.com wrote: I'd certainly be happy fixing FHS to say that tools for mounting and recovering essential system partitions be located in /, and that these essential system partitions contain the tools for mounting and recovering non-essential partitions. The beef with the comment on /home being nonessential is besides the point, /usr, /var, or /opt could have been some special case FUSE filesystem, making it still impossible to predict which files _should_ be in /. The more relevant matter here is that plan FHS, in combination with FUSE, makes that difficult. That's not best practice though is it and I completely disagree with the rules you seem to believe the english language has too. It is not a difficult problem, just FUSE is not expected or intended for that, if that changes it is easily fixed immediately by the admin or by the packager preferably in concert with some root management body or project. Many/All of these issues that have come up are actually of 0 effect, we are not talking about preventing users from merging them as most Linux users do because they just hit ok ok ok in ubuntus installation but about a major degradation due to some devs whim and without I might add proper community involvement or commentry ALLOWED. One things for sure real problems will arise directly due to this merge if this merge becomes standard and possibly with won't fixes used leading to pointlessly breaking existing servers and linux becoming even more of an unorganised mess. On windows production machines I arrived at putting c: on it's own smaller partition and program files on a larger partition. It meant I could have many more c: backups and restore much more quickly too resulting in much higher uptime and reduced loss in the cases that registry restore wasn't good enough and system restore is crap. With windows 7 it's not so beneficial as windows 7 is huge but still useful as everything is getting huge on windows these days. You do get the occasional dumb program perhaps fixable with a drive link within c:. Windows 8 should be more reliable but I expect brings new issues in this area due to app restrictions and where sandboxing could have been used for security instead.
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
The latest FHS dates from 2004, the same year as the *earliest* FUSE release I can see on the FUSE web site. I'd say a good working hypothesis is that FHS was simply written *before* any user-space file systems were more than an experimental oddity. IF the system's /home directory is formatted as an OpenBSD partition, then yes, FHS demands that tools for mounting and recovering it be in /. I'd certainly be happy fixing FHS to say that tools for mounting and recovering essential system partitions be located in /, and that these essential system partitions contain the tools for mounting and recovering non-essential partitions. Which would include testdisk (As far as I know the only linux tool able to read an OpenBSD partition) in /usr. Of course the admin is free to move a copy of testdisk to /. No-one is saying the FHS is perfect, I know the BSD crowd would say far from it but we want it to move in the right not wrong direction. If you are wondering where I stand, I currently boot with an initramfs, since I have everything except /boot located on LVM devices. This includes / and a seperate /usr, done mostly from habit after 15 years of habit, and working where that was the corporate standard production practice. As to system recovery, nowdays I ususlly do that by booting from a live CD/DVD so I have access to all the tools when I need them. Which reminds me that I need to update my rescue DVD to the latest version... A rescue CD has the benefit of being on read only media and perhaps including tools and perhaps enabling permissions you don't want on the system or auditing without running anything from the system and as a fallback but in general single user is more appropriate than both cd and ramdisk and atleast is useful as it can be tailored to the system, is the system and is more likely familiar to the user, a system may not have a cd and maybe not usbs or be remote and as shown is less likely to be upto date and so secure and so useful online, especially if you need a host to upload the cd image. Note: This should highlight how wrong Gregs freedesktop.org links are. -- ___ 'Write programs that do one thing and do it well. Write programs to work together. Write programs to handle text streams, because that is a universal interface' (Doug McIlroy) ___
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
TLDR: FHS is unrealistic about its promises. if we move our binaries / libraries to /usr and work it to make sure /usr is mounted, we will better serve FHS goals and also happen to fix some systemic, but silent bugs. On Fri, Dec 28, 2012 at 12:20 PM, Michael Mol mike...@gmail.com wrote: On Thu, Dec 27, 2012 at 5:37 PM, Mark David Dumlao madum...@gmail.com wrote: Second, going back to problem solving in general - just because you can put down in words what you think the problem is, doesn't mean you've mapped out an accurate or even consistent statement of the problem. There really are cases where it's not enough to just give general airy abstractions and rules of thumb to map out a problem, where it isn't obvious that you're running into edge cases until you really look at it deeply, and yes, the / and /usr split is one of them. So let's look at it deeply, since nobody else will. I'm game. This is the most detailed technical discussion of the problem anyone's cared to actually have, as far as I've been able to observe. For the purposes of clarity I'm going to compare two competing standards, which I will be identifying as follows: s1) FHS, or plain FHS, based on FHS2.3, as identified in http://www.pathname.com/fhs/pub/fhs-2.3.html s2) merged FHS, or merged standard, based on FHS2.3 as above, but with the caveat that all binaries and libraries are placed in /usr instead of being split between /usr and /, as described by http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge It will be helpful to examine how each system reacts to strange cases that challenge FHS. I think some of the following considerations are helpful in determining which one works better. Whichever one is emphasized conspicuously depends on which systems you're interested in maintaining, how many people are using them, your personal taste, sense of justice, etc. Perhaps you could add some of your own. g1) Primary FHS purpose: software/users can predict location of installed files and directories g2) make distro maintainers' job easier g3) make sysads' job easier g4) it does not directly conflict with general practice It is my contention that in all goals, merged FHS is better than plain FHS. Secondly, it is also my contention that plain FHS with a separate /usr does not give enough information to reliably satisfy its own primary goal (g1). Back to the cases below. = === FUNDAMENTAL PROBLEM: / and /usr desync === = Thesis: FHS promise of /usr being sharable is not really deliverable unless it contains the libraries in /. And the we have a standard part is effectively not true anymore, on the matter of the / and /usr split. That is - what the specification says should happen is not happening, on a massive scale, because it turns out that it's not that trivial to determine which binaries go in / and which go in /usr. Give me an example, and I'll describe a reasonably detailed solution. It would be my pleasure. The most fundamental and relevant one for us Gentoo users is this: - how can /usr be sharable among different hosts if it depends on libraries in /? http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY Purpose /usr is the second major section of the filesystem. /usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere. Many distros place fundamental libraries that many programs in /usr depend on in /lib. Especially bad for Gentoo - libraries in /lib may be recompiled as same-version variants if you want to change the USE flags, resulting in clients that don't synchronously recompile their own libraries in /lib to both silently and noisily fail. In other words, many programs in /usr in practice are functionally inseparable from the libraries in /, conflicting with the notion that they were properly shared in the first place. There are certain implicit assumptions made in the spec that are important. First, it's assumed the binaries are compatible with all the hosts. It's assumed you're not sharing s360 binaries with x86 hosts, or sparc binaries with ppc hosts. From there, it's reasonable to assume that the authors of the spec assume the administrator to be smart enough to not do things like: * Mix compiler versions * Mix program compile options * Place a dependency on a binary that's going to be missing. The spec is very, very much a do what you want within these guidelines; don't shoot yourself in the foot thing, it's very much not a declarative bikeshedding of everything related to it. Unfortunately, FHS actually does explicitly specify the meaning of shareable. http://www.pathname.com/fhs/pub/fhs-2.3.html#THEFILESYSTEM Shareable files are those that can be stored on one host and used on others.
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Sat, Dec 29, 2012 at 01:16:34AM +0800, Mark David Dumlao wrote: TLDR: FHS is unrealistic about its promises. if we move our binaries / libraries to /usr and work it to make sure /usr is mounted, we will better serve FHS goals and also happen to fix some systemic, but silent bugs. On Fri, Dec 28, 2012 at 12:20 PM, Michael Mol mike...@gmail.com wrote: On Thu, Dec 27, 2012 at 5:37 PM, Mark David Dumlao madum...@gmail.com wrote: Second, going back to problem solving in general - just because you can put down in words what you think the problem is, doesn't mean you've mapped out an accurate or even consistent statement of the problem. There really are cases where it's not enough to just give general airy abstractions and rules of thumb to map out a problem, where it isn't obvious that you're running into edge cases until you really look at it deeply, and yes, the / and /usr split is one of them. So let's look at it deeply, since nobody else will. I'm game. This is the most detailed technical discussion of the problem anyone's cared to actually have, as far as I've been able to observe. For the purposes of clarity I'm going to compare two competing standards, which I will be identifying as follows: s1) FHS, or plain FHS, based on FHS2.3, as identified in http://www.pathname.com/fhs/pub/fhs-2.3.html s2) merged FHS, or merged standard, based on FHS2.3 as above, but with the caveat that all binaries and libraries are placed in /usr instead of being split between /usr and /, as described by http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge It will be helpful to examine how each system reacts to strange cases that challenge FHS. I think some of the following considerations are helpful in determining which one works better. Whichever one is emphasized conspicuously depends on which systems you're interested in maintaining, how many people are using them, your personal taste, sense of justice, etc. Perhaps you could add some of your own. g1) Primary FHS purpose: software/users can predict location of installed files and directories g2) make distro maintainers' job easier g3) make sysads' job easier g4) it does not directly conflict with general practice It is my contention that in all goals, merged FHS is better than plain FHS. Secondly, it is also my contention that plain FHS with a separate /usr does not give enough information to reliably satisfy its own primary goal (g1). Back to the cases below. = === FUNDAMENTAL PROBLEM: / and /usr desync === = Thesis: FHS promise of /usr being sharable is not really deliverable unless it contains the libraries in /. And the we have a standard part is effectively not true anymore, on the matter of the / and /usr split. That is - what the specification says should happen is not happening, on a massive scale, because it turns out that it's not that trivial to determine which binaries go in / and which go in /usr. Give me an example, and I'll describe a reasonably detailed solution. It would be my pleasure. The most fundamental and relevant one for us Gentoo users is this: - how can /usr be sharable among different hosts if it depends on libraries in /? http://www.pathname.com/fhs/pub/fhs-2.3.html#THEUSRHIERARCHY Purpose /usr is the second major section of the filesystem. /usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere. Many distros place fundamental libraries that many programs in /usr depend on in /lib. Especially bad for Gentoo - libraries in /lib may be recompiled as same-version variants if you want to change the USE flags, resulting in clients that don't synchronously recompile their own libraries in /lib to both silently and noisily fail. In other words, many programs in /usr in practice are functionally inseparable from the libraries in /, conflicting with the notion that they were properly shared in the first place. There are certain implicit assumptions made in the spec that are important. First, it's assumed the binaries are compatible with all the hosts. It's assumed you're not sharing s360 binaries with x86 hosts, or sparc binaries with ppc hosts. From there, it's reasonable to assume that the authors of the spec assume the administrator to be smart enough to not do things like: * Mix compiler versions * Mix program compile options * Place a dependency on a binary that's going to be missing. The spec is very, very much a do what you want within these guidelines; don't shoot yourself in the foot thing, it's very much not a declarative bikeshedding of everything related to it. Unfortunately, FHS actually does
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Sat, Dec 29, 2012 at 1:33 AM, Bruce Hill da...@happypenguincomputers.com wrote: Dang, I got an Excedrin® headache! Heh. Mike said he was game. -- This email is:[ ] actionable [ ] fyi[x] social Response needed: [ ] yes [x] up to you [ ] no Time-sensitive: [ ] immediate[ ] soon [x] none
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Fri, Dec 28, 2012 at 12:46 PM, Mark David Dumlao madum...@gmail.com wrote: On Sat, Dec 29, 2012 at 1:33 AM, Bruce Hill da...@happypenguincomputers.com wrote: Dang, I got an Excedrin® headache! Heh. Mike said he was game. It's going to have to wait a bit. I'm not going to be able to get to this this weekend, most likely; the level of detail involved is higher. :) But, thank you. Also, I recommend you give a full read of 2.3, as I'm going to be referencing both it and its substance. -- :wq
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Sat, 29 Dec 2012 01:16:34 +0800 Mark David Dumlao madum...@gmail.com wrote: whatever filesystem type it is. Following this, for any distro to correctly FHS, there needs to be a package manager switch to copy arbitrary packages (and dependent libraries) from /usr to /. As of yet not implemented. Not at all, FUSE is a userspace flesystem meant to be used after single user. The spec says you have to be able to mount other filesystems not all other filesystems. I'd like to see you mount an OpenBSD ffs partition. So no your point does not stand. As has already been said the cure is worse than the disease many of which have been demonstrated to amount to exactly nothing in all cases and likely why Greg refused to specify what was broken. You've completely ignored the part of FHS about the root filesystem and completely made up your own rules to justify Linux having management problems that some irresponsible devs chose to enforce upon all and now eudev is working to fix and bring the core of linux back into compliance and higher reliability. I'm not surprised Michael can't be bothered to reply. I would use your time more constructively than responding to this thread pollution in any comprehensive manner.
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Sat, Dec 29, 2012 at 2:53 AM, Kevin Chadwick ma1l1i...@yahoo.co.uk wrote: On Sat, 29 Dec 2012 01:16:34 +0800 Mark David Dumlao madum...@gmail.com wrote: whatever filesystem type it is. Following this, for any distro to correctly FHS, there needs to be a package manager switch to copy arbitrary packages (and dependent libraries) from /usr to /. As of yet not implemented. Not at all, FUSE is a userspace flesystem meant to be used after single user. The spec says you have to be able to mount other filesystems not all other filesystems. I'd like to see you mount an OpenBSD ffs partition. If other filesystems is not qualified (and it is not), normal English rules would have it mean all other filesystems which I take to mean all other filesystems on the system. Can you justify a better interpretation? IF the system's /home directory is formatted as an OpenBSD partition, then yes, FHS demands that tools for mounting and recovering it be in /. -- This email is:[ ] actionable [ ] fyi[x] social Response needed: [ ] yes [x] up to you [ ] no Time-sensitive: [ ] immediate[ ] soon [x] none
Re: Should /usr be merged with /? (Was: Re: [gentoo-user] Re: Anyone switched to eudev yet?)
On Sat, 29 Dec 2012 12:27:03 Mark David Dumlao wrote: On Sat, Dec 29, 2012 at 2:53 AM, Kevin Chadwick ma1l1i...@yahoo.co.uk wrote: On Sat, 29 Dec 2012 01:16:34 +0800 Mark David Dumlao madum...@gmail.com wrote: whatever filesystem type it is. Following this, for any distro to correctly FHS, there needs to be a package manager switch to copy arbitrary packages (and dependent libraries) from /usr to /. As of yet not implemented. Not at all, FUSE is a userspace flesystem meant to be used after single user. The spec says you have to be able to mount other filesystems not all other filesystems. I'd like to see you mount an OpenBSD ffs partition. If other filesystems is not qualified (and it is not), normal English rules would have it mean all other filesystems which I take to mean all other filesystems on the system. Can you justify a better interpretation? The latest FHS dates from 2004, the same year as the *earliest* FUSE release I can see on the FUSE web site. I'd say a good working hypothesis is that FHS was simply written *before* any user-space file systems were more than an experimental oddity. IF the system's /home directory is formatted as an OpenBSD partition, then yes, FHS demands that tools for mounting and recovering it be in /. I'd certainly be happy fixing FHS to say that tools for mounting and recovering essential system partitions be located in /, and that these essential system partitions contain the tools for mounting and recovering non-essential partitions. If you are wondering where I stand, I currently boot with an initramfs, since I have everything except /boot located on LVM devices. This includes / and a seperate /usr, done mostly from habit after 15 years of habit, and working where that was the corporate standard production practice. As to system recovery, nowdays I ususlly do that by booting from a live CD/DVD so I have access to all the tools when I need them. Which reminds me that I need to update my rescue DVD to the latest version... -- Reverend Paul Colquhoun, ULC.http://andor.dropbear.id.au/~paulcol Asking for technical help in newsgroups? Read this first: http://catb.org/~esr/faqs/smart-questions.html#intro
