Re: packages being essential but having stuff in /usr/?!
On Fri, Jul 16, 2010 at 07:30:17PM +0200, Christoph Anton Mitterer wrote: What about /etc? Well, this one is easy: /etc *can not* be on its own partition. It has to be on the root filesystem so it will be available. Regards, Patrick -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100719081649.ga23...@debian
Re: packages being essential but having stuff in /usr/?!
]] Patrick Schoenfeld | On Fri, Jul 16, 2010 at 07:30:17PM +0200, Christoph Anton Mitterer wrote: | What about /etc? | | Well, this one is easy: /etc *can not* be on its own partition. | It has to be on the root filesystem so it will be available. Nah, it just has to be mounted when init is started. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87zkxnqsmc@qurzaw.linpro.no
Re: packages being essential but having stuff in /usr/?!
On Thu, 15 Jul 2010, Russ Allbery wrote: Early system startup (before $remote_fs) is a weird and special environment, and most services should just depend on $remote_fs and not worry about it. Normally they have to anyway since the daemon being started is in /usr. Services that do not depend on $remote_fs are services that have to be prepared to run in a limited and special environment and will require special attention and thought. Which potentially includes anything hooked to udev. /usr is NOT guaranteed to be available to anything in /etc/udev/* and /lib/udev/*, unless some other bondary condition forces the kernel to never issue a certain event during startup. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100718123941.gd15...@khazad-dum.debian.net
Re: packages being essential but having stuff in /usr/?!
On Thu, 15 Jul 2010, Christoph Anton Mitterer wrote: Is there any policy document or that like,... which mandates: a) What is guaranteed to be available in initramfs images? Not much is guaranteed to be available in initramfs, unless you arranged for it to be, AFAIK. b) What is guaranteed to be available as soon as the root-fs is mounted Only whatever is in /dev, /lib, /etc, /bin, /sbin. Fail this, and your package is RC buggy, and not fit for release (in the grounds that it just plain doesn't work in a supported configuration). All policy would have to say in the matter is that we do support /usr outside of /, if that much. c) When (!) it is guaranteed that also /usr/ is there? Is it after $remote_fs? Or after mountall-nfs.sh? $remote_fs. If NFS /usr is being mounted after $remote_fs is available, it is a grave bug on the NFS scripts. Policy doesn't have to mandate this directly, the definition of $remote_fs already does. Only init scripts that do not depend on $remote_fs have to do this check. There are quite a lot... Anything running from udev that depends on /usr also has to gracefully handle /usr not being available. Usually, this means you skip the udev hook if /usr isn't there, and have an *indepondent* initscript set things up at the end of the boot process. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100718130356.ge15...@khazad-dum.debian.net
Re: packages being essential but having stuff in /usr/?!
On Fri, 16 Jul 2010, Christoph Anton Mitterer wrote: But I think: 1) the policy description of essential should be clarified then, as now Essential means that the *package* essential functionality (not all of its contents or functionality) has to be available at all times in the lifetime of the *package*, which is related to it being upgraded/replaced. It certainly has nothing to do with the lifetime of the *SYSTEM* runtime. What the essential functionality of a package is varies with the package, and it is far more related to the needs of dpkg and a subset of the maintainer scripts, than anything else. The people who have to deal with Essential packages are *required* to know in depth everything Essential really means and why. Debian Policy doesn't (and has never) gone into that level of detail, AFAIK. it really reads be available and usable on the system at all times. I guess we should at least exclude initramfs from that,... an perhaps also all or parts of the boot process. Unless you propose a *conservative* patch to the policy text through the proper channels, nobody will care. It is an old horse, really. It is not like you have any room to wiggle in these strictly technical matters, so there is little need for policy to steer things. If you don't get it exactly right, things break *badly*... and there is usually very few ways to get it exactly right. Why do I think this is important? Well,... one thing the policy implies on essential packages is, that you don't have to depend on them (in terms of package dependencies) I guess its logical to conclude that one also doesn't have to check for the core stuff like cp/cat/rm... this would really clutter many scripts. If you have ANYTHING hooked to udev or the initscripts/upstart subsystems, and not depending on $remote_fs, you are *REQUIRED* to know what the hell you're doing. It is that simple. initramfs is probably even more restrictive :) But right now one may think that _all_ coreutils packages are guaranteed to be always there. Then, that one is not only not ready yet to deal with early userspace or udev hooks, which would be the only situation where it would matter. 2) Personally, I'd prefer to put some of the current /usr/bin utilities from coreutils to /bin, especially [, test, printf ... but actually some more... I guess this makes /bin not much larger, but would be a nice benefit. You will have to state strong and specific requirements for every utility you want to move to /bin or to /sbin. It won't be done just in case. And it obviously means anything that utility uses (e.g. any libraries, and the dependencies of these libraries, etc) must also be moved to / (/lib, /sbin, /bin, /etc...) -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100718133733.gf15...@khazad-dum.debian.net
Re: packages being essential but having stuff in /usr/?!
On Fri, 16 Jul 2010, Christoph Anton Mitterer wrote: Having $remote_fs is a really nice way to secure that /usr/-stuff is there (and also other stuff like /var...) It is the *only* supported way, AFAIK... As far as I understand - please correct me if I'm wrong - the root-fs is just guaranteed, to have /bin/, /sbin and /lib, right? Neither /var nor /opt. What about /etc? Also, right after the init system starts, neither /proc, nor /dev, nor /sys are there, right? You can assume /etc is available along with the root filesystem (/etc must be part of /). However, it will be *read-only*. Read-write scrach space goes into /lib/init/rw during early userspace, and it does not even exist during very early userspace. /proc, /dev and /sys get mounted VERY early, before / is available in read-write mode. This does NOT mean everything you might want in /dev is already functional, or that every module has already been loaded (and therefore it is possible that something you might want in /proc or /sys might not be there yet, if ever). So, it is You Must Really Know What You're Doing land. Look at the code. We risk much breakage should docs and reality go out of sync. And it depends on whether an initramfs did some setup or not. And if you're doing *anything* that early, you are to subscribe to the relevant mailinglists for the initscript subsystems to keep in the loop. Would it be a good idea, to add new virtual facilities like $dev, $proc or so? Well, this is stuff that is too fragile. And stuff will only be on /proc or /dev AFTER the kernel support is active, which can happen rather later than you would expect (or much earlier!). It is best to come up with specific scenarios, and bring these up on the initscripts subsystem ML. That way init-scripts could try to even restrict their dependencies more, and don't need to find out the current init script (e.g. mountdevsubfs.sh) Only if it doesn't make things even more uncertain. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100718135610.gg15...@khazad-dum.debian.net
Re: packages being essential but having stuff in /usr/?!
On Sun, 18 Jul 2010, Henrique de Moraes Holschuh wrote: On Fri, 16 Jul 2010, Christoph Anton Mitterer wrote: Having $remote_fs is a really nice way to secure that /usr/-stuff is there (and also other stuff like /var...) It is the *only* supported way, AFAIK... Erk. Look at $local_fs as well. I didn't double check if /usr is in the set required to be local (but AFAIK it is not). -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100718140105.gh15...@khazad-dum.debian.net
Re: packages being essential but having stuff in /usr/?!
OoO Pendant le repas du vendredi 16 juillet 2010, vers 19:30, Christoph Anton Mitterer cales...@scientia.net disait : Also, right after the init system starts, neither /proc, nor /dev, nor /sys are there, right? /dev is always here. It may not be complete but it contains the strict minimum needed by any program (like /dev/null). /proc is mounted very very early. Except for the script which should mount it, all other scripts should safely assume that /proc is here. The same now applies to /sys. -- I WILL NOT DRIVE THE PRINCIPAL'S CAR I WILL NOT DRIVE THE PRINCIPAL'S CAR I WILL NOT DRIVE THE PRINCIPAL'S CAR -+- Bart Simpson on chalkboard in episode 7F06 pgpZJgLIc0Itv.pgp Description: PGP signature
Re: packages being essential but having stuff in /usr/?!
On 15.07.2010 21:34, Christoph Anton Mitterer wrote: On Thu, 2010-07-15 at 20:15 +0100, Julien Cristau wrote: No, and there doesn't need to be. Now can you stop beating this dead horse? It would like to rot in hell unharmed. Wow,... supposing that you speak for Debian,... this reaction is really ... interesting... Always thought that Debian would be an open project,... so - either I'm wrong with my assumptions on the policy and one could (being so open) explain the where and why... - or I'm right,.. than there's somewhere some problem, either in the policy,.. or the package... But just saying go away,... I don't wanna know anything... and ignoring that there are e.g. non-dash/bash shells which conform to the policy as a /bin/sh but still would break here... this is really Drepper-ism in it's purest form :) Thank your for that ... uhm... entertainment... I think it also your fault. ;-) We understand your concerns. BTW a lot of people tried to solve and simplify the booting process, but as you see, most of them failed. It is not a simple task, it is not a theoretical homework (maybe yes, but it will take a lt of time to find and describe the requirements for every supported debian configuration, nobody success on it, also because new hardwares and setusp). Allowing new shells is not so simple as you may things, it is not only about shell syntax, build-in programs, etc., but all libraries and support program needed by the shell] [Note: bash already fails on few setups, because of NIS+ IIRC] Also the booting system is a changing area We moved from sysv style to inserv, and maybe will go to upstart or systemd. So a new policy requirements should be enough generic also for transitions. IMHO requiring that at call of /bin/init (the first program called in the new root filesystem at boot) that the essential debian system is ready it is IMHO very impractical for many setups. Moving things to initramd (or to inittab) don't solve the problem you only more one/two layers earlier. And talking about stages is also impractical with the new inserv/upstart/systemd concept. ciao cate -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c40305d.7080...@debian.org
Re: packages being essential but having stuff in /usr/?!
On Fri, 2010-07-16 at 12:11 +0200, Giacomo A. Catenazzi wrote: Also the booting system is a changing area We moved from sysv style to inserv, Isn't that still sysv + just some auto-ordering and so? IMHO requiring that at call of /bin/init (the first program called in the new root filesystem at boot) that the essential debian system is ready it is IMHO very impractical for many setups. Don't get me wrong,... (in what I've written before)... I did not meant to request that everything essential must go into something that is guaranteed available in the root-fs. As some of you already pointed out this wouldn't make sense for e.g. *dpkg*... and also not for all binaries from coreutils (e.g. dircolors). But I think: 1) the policy description of essential should be clarified then, as now it really reads be available and usable on the system at all times. I guess we should at least exclude initramfs from that,... an perhaps also all or parts of the boot process. Why do I think this is important? Well,... one thing the policy implies on essential packages is, that you don't have to depend on them (in terms of package dependencies) I guess its logical to conclude that one also doesn't have to check for the core stuff like cp/cat/rm... this would really clutter many scripts. But right now one may think that _all_ coreutils packages are guaranteed to be always there. 2) Personally, I'd prefer to put some of the current /usr/bin utilities from coreutils to /bin, especially [, test, printf ... but actually some more... I guess this makes /bin not much larger, but would be a nice benefit. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Thu, 2010-07-15 at 13:08 -0700, Russ Allbery wrote: It's after $remote_fs. Please don't assume that all non-local file systems are NFS. Yeah sorry ;) Having $remote_fs is a really nice way to secure that /usr/-stuff is there (and also other stuff like /var...) As far as I understand - please correct me if I'm wrong - the root-fs is just guaranteed, to have /bin/, /sbin and /lib, right? Neither /var nor /opt. What about /etc? Also, right after the init system starts, neither /proc, nor /dev, nor /sys are there, right? Would it be a good idea, to add new virtual facilities like $dev, $proc or so? That way init-scripts could try to even restrict their dependencies more, and don't need to find out the current init script (e.g. mountdevsubfs.sh) Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Thu, 2010-07-15 at 16:31 -0500, Peter Samuelson wrote: What problem are you trying to solve? Did you actually try to use an init script that use printf and doesn't depend on $remote_fs, on a system where /bin/sh is neither bash nor dash? Or is this just a big gedanken-experiment? Admittedly the later ;) Personally I like dash so I stay with it :) It sounds a great deal like the latter, in which case I think you would waste a lot less time by simply joining forces with those people who are working toward being able to run other shells as /bin/sh. Let _them_ know that if they don't provide builtin test and printf commands, there will be problems before /usr is mounted. Well ;) ... I guess that's not the right way to go... no standard specifies that such things have to be built-in... and it's quite unpolite to request others to do it as we do now with bash/dash :) If they want to crusade about it, I think it would sound more credible than you doing so with no apparent concrete goals. Didn't want to start a crusade ;) ... just making things more conforming and trouble-secure. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
Christoph Anton Mitterer cales...@scientia.net writes: As far as I understand - please correct me if I'm wrong - the root-fs is just guaranteed, to have /bin/, /sbin and /lib, right? Neither /var nor /opt. What about /etc? I'm not sure, but I believe you can count on /etc always being available (although possibly read-only). For /var, you need to depend on $local_fs. For /opt, I would depend on $remote_fs. Also, right after the init system starts, neither /proc, nor /dev, nor /sys are there, right? No idea. Would it be a good idea, to add new virtual facilities like $dev, $proc or so? How many init scripts need them? In other words, is there a concrete problem that you're trying to solve? That way init-scripts could try to even restrict their dependencies more, and don't need to find out the current init script (e.g. mountdevsubfs.sh) Most init scripts should just depend on $remote_fs. Those that don't should almost always depend on $local_fs. Init scripts depending on neither are probably being maintained by the boot system maintainers, who know exactly what the sequencing is. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87eif3s4n4@windlord.stanford.edu
Re: packages being essential but having stuff in /usr/?!
Le vendredi 16 juillet 2010 à 19:30 +0200, Christoph Anton Mitterer a écrit : [...] As far as I understand - please correct me if I'm wrong - the root-fs is just guaranteed, to have /bin/, /sbin and /lib, right? Neither /var nor /opt. What about /etc? As long as init scripts are stored in /etc/init.d [0], I guess /etc is considered as being always available (even as read-only), am I right? Cheers, Julien [0] http://www.debian.org/doc/debian-policy/ch-opersys.html#s-/etc/init.d -- Julien Valroff jul...@kirya.net http://www.kirya.net GPG key: 4096R/290D20C5 092F 4CB5 5F19 E006 1CFD B489 D32B 8D66 290D 20C5 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1279303361.2728.3.ca...@gaia.kirya.net
Re: packages being essential but having stuff in /usr/?!
On Fri, Jul 16, 2010 at 10:44:31AM -0700, Russ Allbery wrote: Christoph Anton Mitterer cales...@scientia.net writes: [...] Would it be a good idea, to add new virtual facilities like $dev, $proc or so? How many init scripts need them? In other words, is there a concrete problem that you're trying to solve? [...] I would think almost all init scripts depend on /dev and /proc! Certainly start-stop-daemon uses them. But only init scripts in rcS.d need to explicitly depend on these, and they should presumably depend on $mountkernfs (if not on $local_fs). Ben. -- Ben Hutchings We get into the habit of living before acquiring the habit of thinking. - Albert Camus -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100716182349.gc3...@decadent.org.uk
Re: packages being essential but having stuff in /usr/?!
[Christoph Anton Mitterer] As far as I understand - please correct me if I'm wrong - the root-fs is just guaranteed, to have /bin/, /sbin and /lib, right? Neither /var nor /opt. What about /etc? The root file system need to have /etc/, yes, as well as the others. Do not have the complete list available, but those you mention need to be there. /var/ is not required to be on the root file system, but must be available after the $local_fs point during boot. Also, right after the init system starts, neither /proc, nor /dev, nor /sys are there, right? This depends on what the initrd did. Would it be a good idea, to add new virtual facilities like $dev, $proc or so? Probably not. The early boot is so special anyway, and consist of so few packages that there advantage of trying to generalize is probably outweighted by the effort to implenent it. The effort with init.d scripts should instead be on moving scripts out of rcS.d/ into rc[1-5].d/, to improve single user mode and increase concurrency during boot. Happy hacking, -- Petter Reinholdtsen -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100716181116.gb8...@login1.uio.no
Re: packages being essential but having stuff in /usr/?!
On Fri, 2010-07-16 at 20:02 +0200, Julien Valroff wrote: As long as init scripts are stored in /etc/init.d [0], I guess /etc is considered as being always available (even as read-only), am I right? Sir! I take a bow :) *bompf* That was too obvious for me to see it *G* Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Fri, 2010-07-16 at 19:23 +0100, Ben Hutchings wrote: I would think almost all init scripts depend on /dev and /proc! Certainly start-stop-daemon uses them. I guess Russ meant,... whether I have any examples for services/daemons,.. that need _just_ /proc or /dev,... but do not already depend on $local_fs/$remote_fs anyways. Admittedly I have no example,... except the real low level stuff (as mentioned by Petter). My motivation is more that I think it's always a good idea to build up such frameworks, even if there's not yet a concrete example who needs it. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Fri, 2010-07-16 at 20:11 +0200, Petter Reinholdtsen wrote: /var/ is not required to be on the root file system, but must be available after the $local_fs point during boot. Ah.. good to know,.. thought that could perhaps also be on remote-filesystems... Also, right after the init system starts, neither /proc, nor /dev, nor /sys are there, right? This depends on what the initrd did. Ok.. i see... but at least it's not guaranteed,.. as systems might not use an initramfs at all,... and then these are mounted by the respective init-script, right? Probably not. The early boot is so special anyway, and consist of so few packages that there advantage of trying to generalize is probably outweighted by the effort to implenent it. The effort with init.d scripts should instead be on moving scripts out of rcS.d/ into rc[1-5].d/, to improve single user mode and increase concurrency during boot. mhh,.. ok... I see :) Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
Le vendredi 16 juillet 2010 à 19:25 +0200, Christoph Anton Mitterer a écrit : But I think: 1) the policy description of essential should be clarified then 2) Personally, I'd prefer to put some of the current /usr/bin utilities from coreutils to /bin 3) I think you’re the only one to care. And since you have failed to explain the benefits, this is not going to change. -- .''`. Josselin Mouette : :' : `. `' “If you behave this way because you are blackmailed by someone, `-[…] I will see what I can do for you.” -- Jörg Schilling -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1279317859.2893.2.ca...@tomoe
Re: packages being essential but having stuff in /usr/?!
6, 2010 at 5:56 PM, Christoph Anton Mitterer cales...@scientia.net wrote: On Fri, 2010-07-16 at 20:11 +0200, Petter Reinholdtsen wrote: /var/ is not required to be on the root file system, but must be available after the $local_fs point during boot. Ah.. good to know,.. thought that could perhaps also be on remote-filesystems... What about services that start before $remote_fs is provided that place pidfiles in /var? Portmapper is an example of this. Also, right after the init system starts, neither /proc, nor /dev, nor /sys are there, right? This depends on what the initrd did. Ok.. i see... but at least it's not guaranteed,.. as systems might not use an initramfs at all,... and then these are mounted by the respective init-script, right? Yes, /etc/init.d/mountkernfs.sh Probably not. The early boot is so special anyway, and consist of so few packages that there advantage of trying to generalize is probably outweighted by the effort to implenent it. The effort with init.d scripts should instead be on moving scripts out of rcS.d/ into rc[1-5].d/, to improve single user mode and increase concurrency during boot. mhh,.. ok... I see :) Cheers, Chris. -- -Will Orr -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktinh3ykdncjbezq-5tbzlskiw9i1encmynx0f...@mail.gmail.com
Re: packages being essential but having stuff in /usr/?!
On Fri, 2010-07-16 at 18:13 -0400, Will wrote: Ah.. good to know,.. thought that could perhaps also be on remote-filesystems... What about services that start before $remote_fs is provided that place pidfiles in /var? Portmapper is an example of this. Ok... I should have said parts of /var... e.g. I've seen sites which put subdirs of /var/log or /var/backups on remote filesystems. But then these are usually not used by init-scripts :) Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On 14.07.2010 17:36, Christoph Anton Mitterer wrote: Hi. I wonder why this never came up before,.. or did it an I'm just blind? I've just read parts of POSIX, where echo is more or less deprecated in favour of printf (http://www.opengroup.org/onlinepubs/9699919799/utilities/echo.html#tag_20_37_16). Whether users will do this or not is a different question but I've seen that Debian/corutils ships echo in /bin, but printf in /usr/bin. The same for many other binaries part of coreutils. Now coreutils is marked as essential, right?! This means per policy section 3.8 (http://www.debian.org/doc/debian-policy/ch-binary.html#s3.8): Essential is defined as the minimal set of functionality that must be available and usable on the system at all times As far as I understand,.. it's fully ok, to have /usr on a separate (i.e. non-root-) filesystem. System initialisation and in general system script are outside POSIX scope, as well many common command executed by such scripts (administration tools are also outside POSIX). Additionally system initialisation is very complex and it should handle to many different setups (from no local disks to very complex local disks setup, etc.). So it is impossible to have POSIX compatible scripts (or at least at early stage) and on the other hand, over-specifying the booting process will reduce flexibility to build a better system and to correct existing and future bugs. Our boot people take care about init scripts, their requirements and thus what it should be moved from /usr to root. It is a case-by-case analysis. OTOH essential packages will provide a basic system for all debian packages, developers and users, thus it is important to have more than basic tools to boot in essential packages. And the contrary is also valid: a diskless system need some network packages, some of them surely not essential btw: Personally, I'd support to stop using echo,.. it's not really portable... not portable? Do you have some real/widely used examples of incompatible use? Most echo command don't start with -, thus fully portable, but nearly all echo support the -n option. BTW GNU/Linux is not fully POSIX compatible by design. It follow the LSB (an other standard) and there is a ISO groups to find and try to correct the differences. echo is one of required difference. Anyway the initialisation script are special and well distribution controlled, thus it would be easier to posixify, when/if needed. Anyway this was discussed already several times. ciao cate -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c3ede5e.6050...@debian.org
Re: packages being essential but having stuff in /usr/?!
On Wed, 2010-07-14 at 19:22 +0200, Petter Reinholdtsen wrote: I believe this is a misunderstanding. The quoted section do not mean that all files in a essential package need to be on the root partition, but that the package should always be installed. Well but what's the benefit then at all? If it's not guaranteed to be there...?! This is the first time I hear someone read the policy section the way you do it, and I believe it is not representative for the intention behind that part of the policy. At least to me it reads like that. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Thu, 2010-07-15 at 12:09 +0200, Giacomo A. Catenazzi wrote: System initialisation and in general system script are outside POSIX scope, as well many common command executed by such scripts (administration tools are also outside POSIX). Well yes,... nevertheless I guess that it's always a good idea to restrict to scripts to the bare minimum... of course as far as possible. Additionally system initialisation is very complex and it should handle to many different setups (from no local disks to very complex local disks setup, etc.). That's what I've meant... and with such complex setups, not having many coreutils available could be a problem. Our boot people take care about init scripts, their requirements and thus what it should be moved from /usr to root. It is a case-by-case analysis. Uhm... looking at coreutils, I find many programs which I guess can be used (or are actually) during system initialisation, e.g.: env base64 dirname [ test stat timeout id printf just to name a few. not portable? Do you have some real/widely used examples of incompatible use? Especially -n,... which is widely used,... but not portable. BTW GNU/Linux is not fully POSIX compatible by design. It follow the LSB (an other standard) and there is a ISO groups to find and try to correct the differences. echo is one of required difference. Yeah I know,... but it does not automatically mean that this were the right choice. I guess LSBCo. just made it because it was already so widely used, that you could never convince people to do different. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On 15.07.2010 14:31, Christoph Anton Mitterer wrote:
On Thu, 2010-07-15 at 12:09 +0200, Giacomo A. Catenazzi wrote:
System initialisation and in general system script are outside POSIX
scope, as well many common command executed by such scripts
(administration tools are also outside POSIX).
Well yes,... nevertheless I guess that it's always a good idea to
restrict to scripts to the bare minimum... of course as far as possible.
Yes, and usually it is so. In a short period the maintainer will
receive bug report about non working init.d script with some
configuration, which force people to minimize the init scripts.
Additionally system initialisation is very complex and it should handle
to many different setups (from no local disks to very complex local
disks setup, etc.).
That's what I've meant... and with such complex setups, not having many
coreutils available could be a problem.
usually no. see below.
Our boot people take care about init scripts, their requirements
and thus what it should be moved from /usr to root.
It is a case-by-case analysis.
Uhm... looking at coreutils, I find many programs which I guess can be
used (or are actually) during system initialisation, e.g.:
env
base64
dirname
[
test
stat
timeout
id
printf
just to name a few.
Early init script doesn't need a lot of complexity, and
they are must pretty stupid, so they usually don't need some
commands of coreutils.
e.g. 'stat' and 'id' are not so usefull at such stage,
the script could assume root and basic root files available.
'env' also not is very usefull: easy to work around with
standard shell. The other standard use of 'env' is '#!/usr/bin/env
so an already hard coded path.
'dirname', '[' and 'test' could cause some problem. Usually they are
build-in on shell, but it is not mandatory, and policy BTW mandate
some extended (from POSIX) syntax on built-in 'test', but I think
policy missed the case of 'test' not being built-in and not
being available (because it is in /usr/bin).
[this is IMHO a BUG in policy]
timout could be interesting, but when a init.d script will
need it, I think there will be no problem to more it in /bin/
not portable? Do you have some real/widely used examples of
incompatible use?
Especially -n,... which is widely used,... but not portable.
Yes, but it is very difficult (maybe impossible) to see a real
script where echo -n is intentionally intended to write -n (at
beginning of a line).
BTW GNU/Linux is not fully POSIX compatible by design. It follow the
LSB (an other standard) and there is a ISO groups to find and try
to correct the differences. echo is one of required difference.
Yeah I know,... but it does not automatically mean that this were the right
choice.
I guess LSBCo. just made it because it was already so widely used, that
you could never convince people to do different.
But I think now echo -n must be supported by all systems (not only on
LSB systems), because of wide usage.
POSIX successfully standardized a lot of things, but POSIX also failed
on few points ('echo -n' and 'pax'), and IMHO it is a lost campain.
I expect that in next posix the 'echo -n' and 'tar' will reach the
normative status.
ciao
cate
--
To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/4c3f0643.5080...@debian.org
Re: packages being essential but having stuff in /usr/?!
On Thu, Jul 15, 2010 at 02:18:53PM +0200, Christoph Anton Mitterer wrote: On Wed, 2010-07-14 at 19:22 +0200, Petter Reinholdtsen wrote: I believe this is a misunderstanding. The quoted section do not mean that all files in a essential package need to be on the root partition, but that the package should always be installed. Well but what's the benefit then at all? If it's not guaranteed to be there...?! AIUI, it's guaranteed to be there after system startup. Michael -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100715132839.ga32...@nighthawk.chemicalconnection.dyndns.org
Re: packages being essential but having stuff in /usr/?!
Giacomo A. Catenazzi c...@debian.org writes: 'dirname', '[' and 'test' could cause some problem. Usually they are build-in on shell, but it is not mandatory, and policy BTW mandate some extended (from POSIX) syntax on built-in 'test', but I think policy missed the case of 'test' not being built-in and not being available (because it is in /usr/bin). [this is IMHO a BUG in policy] Possibly, but I don't think Policy is really the place to try to rule out any unreasonable thing that someone might consider doing. I have a hard time imagining anyone trying to use a shell as /bin/sh which doesn't have test as a built-in, and even if they did, dealing with NFS-mounted /usr and the location of test would be the least of their problems. That said, yes, Policy does have a trap door clause to deal with test not being implemented as a built-in because people objected to the assumption that it was a built-in at the time. I think it's a marginal case, though, and I'm not inclined to worry too much about it. Especially -n,... which is widely used,... but not portable. Policy requires echo -n be portable to any Debian system. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/874og0veru@windlord.stanford.edu
Re: packages being essential but having stuff in /usr/?!
Michael Banck mba...@debian.org writes: On Thu, Jul 15, 2010 at 02:18:53PM +0200, Christoph Anton Mitterer wrote: On Wed, 2010-07-14 at 19:22 +0200, Petter Reinholdtsen wrote: I believe this is a misunderstanding. The quoted section do not mean that all files in a essential package need to be on the root partition, but that the package should always be installed. Well but what's the benefit then at all? If it's not guaranteed to be there...?! AIUI, it's guaranteed to be there after system startup. Right, the point is that other packages can assume those binaries are available during any normal package operations and during package installation and removal. Early system startup (before $remote_fs) is a weird and special environment, and most services should just depend on $remote_fs and not worry about it. Normally they have to anyway since the daemon being started is in /usr. Services that do not depend on $remote_fs are services that have to be prepared to run in a limited and special environment and will require special attention and thought. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87zkxsu04m@windlord.stanford.edu
Re: packages being essential but having stuff in /usr/?!
On Wed, 2010-07-14 at 19:26 +0200, Sven Joachim wrote: It's been reported as bug #428189 already, but without any followup. See also #532324. Well while 532324 is a perfect example how some developers obviously think they can ignore the policy just as they like (this issue is really unbelievable,... wonder why we have all that policy crap...) in order to save them work... I'd agree that at least section 10.4 of the policy does not mandate that the SUS3 Utilities are available,... as far as I understand it just talks about compliance of shells providing /bin/sh with the SUS3 Shell Command Language (which is not the Utilities),... and IIRC printf is not required to be a built-in... Nevertheless I still think that this stuff has to be there because of the definition of essential. - Either this on is not exactly formed (so essential means only always installed, but does not mean always available)... but this would make the whole concept rather stupid IMHO,.. because then we'd again have to check for every single essential thing we're using anywhere. - Or it does mean, that everything essential has to be available always,.. which would include at least the time when init takes over from the kernel or the initramfs image. But then we'd have a problem as many packages put their stuff in /usr/ This is indeed a problem if /bin/sh has no printf builtin, but it does not affect people who use dash or bash. Well but it's rather ugly to simply say dash/bash support it,.. = we're fine... So is dpkg, and it lives almost completely under /usr, except for start-stop-daemon. Well,... but dpkg is probably not needed during system startup... That however would mean, that even outsite initramfs images (which are probably a special case and do not count) many of corutils' binaries are not _always_ available. Before /usr is mounted, yes. Is there any policy document or that like,... which mandates: a) What is guaranteed to be available in initramfs images? b) What is guaranteed to be available as soon as the root-fs is mounted (I mean /etc/, /bin/, /sbin, /lib/, but not /usr c) When (!) it is guaranteed that also /usr/ is there? Is it after $remote_fs? Or after mountall-nfs.sh? Only init scripts that do not depend on $remote_fs have to do this check. There are quite a lot... Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Thu, Jul 15, 2010 at 21:07:11 +0200, Christoph Anton Mitterer wrote: Is there any policy document or that like,... which mandates: a) What is guaranteed to be available in initramfs images? b) What is guaranteed to be available as soon as the root-fs is mounted (I mean /etc/, /bin/, /sbin, /lib/, but not /usr c) When (!) it is guaranteed that also /usr/ is there? Is it after $remote_fs? Or after mountall-nfs.sh? No, and there doesn't need to be. Now can you stop beating this dead horse? It would like to rot in hell unharmed. Cheers, Julien signature.asc Description: Digital signature
Re: packages being essential but having stuff in /usr/?!
Christoph Anton Mitterer cales...@scientia.net writes: c) When (!) it is guaranteed that also /usr/ is there? Is it after $remote_fs? Or after mountall-nfs.sh? It's after $remote_fs. Please don't assume that all non-local file systems are NFS. Someday, I'd like to support /usr in AFS, for instance, since that's sometimes used at heavy AFS sites. -- Russ Allbery (r...@debian.org) http://www.eyrie.org/~eagle/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87iq4gse3h@windlord.stanford.edu
Re: packages being essential but having stuff in /usr/?!
On 07/15/2010 09:07 PM, Christoph Anton Mitterer wrote: On Wed, 2010-07-14 at 19:26 +0200, Sven Joachim wrote: It's been reported as bug #428189 already, but without any followup. See also #532324. Well while 532324 is a perfect example how some developers obviously think they can ignore the policy just as they like (this issue is really unbelievable,... wonder why we have all that policy crap...) in order to save them work... Full ack. If the policy does not fit reality, then it should be changed *or* (which is what I'd prefer), the package needs to be fixed. -- Bernd ZeimetzDebian GNU/Linux Developer http://bzed.dehttp://www.debian.org GPG Fingerprints: 06C8 C9A2 EAAD E37E 5B2C BE93 067A AD04 C93B FF79 ECA1 E3F2 8E11 2432 D485 DD95 EB36 171A 6FF9 435F -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4c3f6fbe.9030...@bzed.de
Re: packages being essential but having stuff in /usr/?!
On Thu, 2010-07-15 at 14:59 +0200, Giacomo A. Catenazzi wrote:
Yes, and usually it is so. In a short period the maintainer will
receive bug report about non working init.d script with some
configuration, which force people to minimize the init scripts.
Agreed.
Early init script doesn't need a lot of complexity, and
they are must pretty stupid, so they usually don't need some
commands of coreutils.
Aggreed with the exception that you may have,.. as I noted in my
email just before stuff in initramfs-images which do use such things.
But I'm fully ok with putting this under the responsibility of the
respective author :)
Nevertheless,... I'd like to see definite clarification on this
situation in the policy :)
'dirname', '[' and 'test' could cause some problem. Usually they are
build-in on shell, but it is not mandatory, and policy BTW mandate
some extended (from POSIX) syntax on built-in 'test', but I think
policy missed the case of 'test' not being built-in and not
being available (because it is in /usr/bin).
[this is IMHO a BUG in policy]
Yes I see also a problem here...
timout could be interesting, but when a init.d script will
need it, I think there will be no problem to more it in /bin/
Is it really that easy moving such things? I've seen many scripts
throughout debian which hardcode the absolute path (and do not (have to)
set a secure PATH for that reason)... all of them would fail after such
movings...
Yes, but it is very difficult (maybe impossible) to see a real
script where echo -n is intentionally intended to write -n (at
beginning of a line).
Admittedly,... I just noted this, because personally I also like other
non-Linux Unices... and we should not add incompatibilities if
avoidable :)
But I think now echo -n must be supported by all systems (not only on
LSB systems), because of wide usage.
POSIX successfully standardized a lot of things, but POSIX also failed
on few points ('echo -n' and 'pax'), and IMHO it is a lost campain.
I expect that in next posix the 'echo -n' and 'tar' will reach the
normative status.
Would be great!... Hopefully also local :D
Best wishes,
Chris.
smime.p7s
Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Thu, 2010-07-15 at 10:26 -0700, Russ Allbery wrote: Right, the point is that other packages can assume those binaries are available during any normal package operations and during package installation and removal. Ok... than perhaps one can add a note to the policy, that this means after the system has booted, or especially after all filesystems including /usr are mounted. And apart from that,... during initramfs only that what has been included (or is part of busybox, if used ist available), and berofre /usr... only that what the respective maintainers (e.g. coreutils) decided to put into non-/usr locations. Right? Early system startup (before $remote_fs) is a weird and special environment, and most services should just depend on $remote_fs and not worry about it. Normally they have to anyway since the daemon being started is in /usr. Well I came across this by writing a (hopefully) improved version of the current /etc/init.d/skeleton to ask for its inclusion,... and I did not want to restrict the notes I give there for just these normal kinds of daemons. (So much for my motivation.) Services that do not depend on $remote_fs are services that have to be prepared to run in a limited and special environment and will require special attention and thought. Well I wrote some keyscripts for cryptsetup, which happen to execute long before any /usr or so is there,... and I use e.g. printf and some others. I never noticed however that printf is not there, because of that built-in versions ;) Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
On Thu, 2010-07-15 at 20:15 +0100, Julien Cristau wrote: No, and there doesn't need to be. Now can you stop beating this dead horse? It would like to rot in hell unharmed. Wow,... supposing that you speak for Debian,... this reaction is really ... interesting... Always thought that Debian would be an open project,... so - either I'm wrong with my assumptions on the policy and one could (being so open) explain the where and why... - or I'm right,.. than there's somewhere some problem, either in the policy,.. or the package... But just saying go away,... I don't wanna know anything... and ignoring that there are e.g. non-dash/bash shells which conform to the policy as a /bin/sh but still would break here... this is really Drepper-ism in it's purest form :) Thank your for that ... uhm... entertainment... Best wishes, Chris. smime.p7s Description: S/MIME cryptographic signature
Re: packages being essential but having stuff in /usr/?!
[Christoph Anton Mitterer] This is indeed a problem if /bin/sh has no printf builtin, but it does not affect people who use dash or bash. Well but it's rather ugly to simply say dash/bash support it,.. = we're fine... What problem are you trying to solve? Did you actually try to use an init script that use printf and doesn't depend on $remote_fs, on a system where /bin/sh is neither bash nor dash? Or is this just a big gedanken-experiment? It sounds a great deal like the latter, in which case I think you would waste a lot less time by simply joining forces with those people who are working toward being able to run other shells as /bin/sh. Let _them_ know that if they don't provide builtin test and printf commands, there will be problems before /usr is mounted. If they want to crusade about it, I think it would sound more credible than you doing so with no apparent concrete goals. -- Peter Samuelson | org-tld!p12n!peter | http://p12n.org/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20100715213129.ge3...@p12n.org
Re: packages being essential but having stuff in /usr/?!
[Christoph Anton Mitterer] Now coreutils is marked as essential, right?! This means per policy section 3.8 (http://www.debian.org/doc/debian-policy/ch-binary.html#s3.8): Essential is defined as the minimal set of functionality that must be available and usable on the system at all times As far as I understand,.. it's fully ok, to have /usr on a separate (i.e. non-root-) filesystem. That however would mean, that even outsite initramfs images (which are probably a special case and do not count) many of corutils' binaries are not _always_ available. I believe this is a misunderstanding. The quoted section do not mean that all files in a essential package need to be on the root partition, but that the package should always be installed. This is the first time I hear someone read the policy section the way you do it, and I believe it is not representative for the intention behind that part of the policy. Happy hacking, -- Petter Reinholdtsen -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/2fl630i3rmu@login1.uio.no
Re: packages being essential but having stuff in /usr/?!
On 2010-07-14 17:36 +0200, Christoph Anton Mitterer wrote: I wonder why this never came up before,.. or did it an I'm just blind? It's been reported as bug #428189 already, but without any followup. See also #532324. I've just read parts of POSIX, where echo is more or less deprecated in favour of printf (http://www.opengroup.org/onlinepubs/9699919799/utilities/echo.html#tag_20_37_16). Whether users will do this or not is a different question but I've seen that Debian/corutils ships echo in /bin, but printf in /usr/bin. This is indeed a problem if /bin/sh has no printf builtin, but it does not affect people who use dash or bash. The same for many other binaries part of coreutils. Now coreutils is marked as essential, right?! So is dpkg, and it lives almost completely under /usr, except for start-stop-daemon. This means per policy section 3.8 (http://www.debian.org/doc/debian-policy/ch-binary.html#s3.8): Essential is defined as the minimal set of functionality that must be available and usable on the system at all times As far as I understand,.. it's fully ok, to have /usr on a separate (i.e. non-root-) filesystem. That however would mean, that even outsite initramfs images (which are probably a special case and do not count) many of corutils' binaries are not _always_ available. Before /usr is mounted, yes. People would again have to check for them in e.g. their initscripts, or basically everything before /usr is mounted, e.g. via NFS. Only init scripts that do not depend on $remote_fs have to do this check. The same might be a problem with other essential packages, too. I have /usr on a separate partition and did not have any problems in the last few years with that. Don't know how the situation is with /usr on NFS, though. Sven -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/8739vm3rh0@turtle.gmx.de
