Re: /usr/lib vs /usr/libexec
Josselin Mouette [EMAIL PROTECTED] writes: Le jeudi 12 mai 2005 à 18:32 -0700, Thomas Bushnell BSG a écrit : You said it: there is a cache. After the first access, the directory will be in the cache. Making all of this a purely imaginary problem. The whole directory is in the cache? I don't think so. Remember, that in between each lookup, a library gets searched, which probably flushes the entire cache. Currently, on my 256 MB machine, the kernel is using 60 MB for the cache. I think this is enough to include a few directory blocks, especially some as frequently used as /usr/lib. If it doesn't while loading an application then something is seriously wrong or you will be swapping. But the problem remains that you have to look at each dire entry in unhashed ext2/3, fat or minix. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Thu, May 19, 2005 at 11:47:31AM +0200, Goswin von Brederlow wrote: [snip] But the problem remains that you have to look at each dire entry in unhashed ext2/3, fat or minix. Ehrm, I don't think having /usr/lib on a fat FS is an option anyway, considering its lacking file ownership/permission support and its filename munging... And I somehow doubt that minix is a problem either, these days. Regards: David Weinehall -- /) David Weinehall [EMAIL PROTECTED] /) Rime on my window (\ // ~ // Diamond-white roses of fire // \) http://www.acc.umu.se/~tao/(/ Beautiful hoar-frost (/ -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
[David Weinehall] Ehrm, I don't think having /usr/lib on a fat FS is an option anyway, considering its lacking file ownership/permission support and its filename munging... I should think the lack of symlink support is the real problem. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Le jeudi 12 mai 2005 à 18:32 -0700, Thomas Bushnell BSG a écrit : You said it: there is a cache. After the first access, the directory will be in the cache. Making all of this a purely imaginary problem. The whole directory is in the cache? I don't think so. Remember, that in between each lookup, a library gets searched, which probably flushes the entire cache. Currently, on my 256 MB machine, the kernel is using 60 MB for the cache. I think this is enough to include a few directory blocks, especially some as frequently used as /usr/lib. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: /usr/lib vs /usr/libexec
Goswin von Brederlow [EMAIL PROTECTED] writes: Which doesn't? Minix maybe. Even ext2/3 has hashes for dir if you format it that way. Thomas Bushnell BSG wrote: Is this the Debian default for installation? Yes, it is. I just checked and every install I've done turned this on without my knowledge. :-) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Josselin Mouette [EMAIL PROTECTED] writes: Le mercredi 11 mai 2005 13:35 -0300, Humberto Massa a crit : Imagine that, to load Konqui, you have to go 200 times to the disk (ok, cache, but...), each of them reading the 1 entries I have in /usr/lib, some of them twice or three times, to follow the symlinks. This is a real inefficiency. You said it: there is a cache. After the first access, the directory will be in the cache. Making all of this a purely imaginary problem. The whole directory is in the cache? I don't think so. Remember, that in between each lookup, a library gets searched, which probably flushes the entire cache.
Re: /usr/lib vs /usr/libexec
On Tue, May 10, 2005 at 04:40:11PM -0700, Thomas Bushnell BSG wrote: What does the default Debian install do? Debian seems to use ext3 without directory indexing by default. Which is a sane choice as directory indexing on ext3 still seems to be not fully mature. And as mentioned in another thread it's not available on ext2 at all. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Wednesday 11 May 2005 05:50, Goswin von Brederlow
[EMAIL PROTECTED] wrote:
Russell Coker [EMAIL PROTECTED] writes:
On Wednesday 11 May 2005 01:28, Goswin von Brederlow
[EMAIL PROTECTED] wrote:
Why would it be desirable to have arch-os directories under libexec?
On fedora-devel Bill Nottingham suggested having /usr/lib vs /usr/lib64
for programs that care about such things and /usr/libexec for programs
that don't.
32bit mozilla with flash plugin and 64bit mozilla without. A lot of
people seem to want that.
Bill's idea seems to work in that case. Although as you would need
different names in /usr/bin it might make sense to just name the libexec
files with the same extension as the file in /usr/bin that launches them.
What about mips O32, N32, N64 abis?
/lib, /lib32 and /lib64?
If you currently have /lib, /lib32, and /lib64 on MIPS then with Bill's idea
those directories could be used for three different versions of Mozilla.
What about i386 knetbsd and linux?
What is required there?
The multiarch /arch-os/ path is already present in the toolchain for
many things including include files and libs and works for all cases
of multiarch in a clean way. The lib{,32,64} subdirs are different on
every arch, confusing and insuffient for the bsd case.
Surely for every case in which multiple versions of binaries are needed we
also need multiple versions of libs. So having multiple /usr/lib directories
should do.
--
http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages
http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark
http://www.coker.com.au/postal/Postal SMTP/POP benchmark
http://www.coker.com.au/~russell/ My home page
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes:
On Wednesday 11 May 2005 05:50, Goswin von Brederlow
[EMAIL PROTECTED] wrote:
Russell Coker [EMAIL PROTECTED] writes:
On Wednesday 11 May 2005 01:28, Goswin von Brederlow
[EMAIL PROTECTED] wrote:
Why would it be desirable to have arch-os directories under libexec?
On fedora-devel Bill Nottingham suggested having /usr/lib vs /usr/lib64
for programs that care about such things and /usr/libexec for programs
that don't.
32bit mozilla with flash plugin and 64bit mozilla without. A lot of
people seem to want that.
Bill's idea seems to work in that case. Although as you would need
different names in /usr/bin it might make sense to just name the libexec
files with the same extension as the file in /usr/bin that launches them.
What about mips O32, N32, N64 abis?
/lib, /lib32 and /lib64?
I just thought of something even worse: knetbsd-amd64.
- bsd 64 bit libs
- bsd 32 bit libs
- linux 64 bit libs
- linux 32 bit libs
Or does bsd amd64 not support all 4 of those?
If you currently have /lib, /lib32, and /lib64 on MIPS then with Bill's idea
those directories could be used for three different versions of Mozilla.
What about i386 knetbsd and linux?
What is required there?
/usr/lib/i386-linux/ and /usr/lib/i386-knetbsd/.
The multiarch /arch-os/ path is already present in the toolchain for
many things including include files and libs and works for all cases
of multiarch in a clean way. The lib{,32,64} subdirs are different on
every arch, confusing and insuffient for the bsd case.
Surely for every case in which multiple versions of binaries are needed we
also need multiple versions of libs. So having multiple /usr/lib directories
should do.
Note that multiarch does not imply multiple versions of the same
binary though. So only the normal bin dirs we already have.
But different binaries for different archs might use the same
libraries and then those must be available for each arch. Since the
library name does not contain an unique arch-os pair different dirs
are a must. Like you say.
The question is just what do we name them. :)
The /lib, /lib64, /lib32 dirs used currently are a bit haphazzard and
can't cover all cases.
MfG
Goswin
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG [EMAIL PROTECTED] writes: Goswin von Brederlow [EMAIL PROTECTED] writes: Thomas Bushnell BSG [EMAIL PROTECTED] writes: Goswin von Brederlow [EMAIL PROTECTED] writes: Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 02:03:01PM -0300, Humberto Massa wrote: These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. ext2 doesn't. Convert it to utilize directory hashing. The ability is there but iirc isn't used by default. What does the default Debian install do? Afaik the good, old, slow linear list. With that file open/stat is O(n) and ls also O(n) (cause you keep reading the dir instead of starting at the top each time). In which case, we do have that bug. Would you agree that that bug should be fixed (in Etch), irrespective of whether the FHS is also changed to split /usr/lib? Martin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG wrote: Humberto Massa [EMAIL PROTECTED] writes: with the possible exception of FAT and Minix. Q: are they used by a default? A: Last time I installed Debian (15 days ago), it asked me if I wanted my partition ext3, xfs, or reiserfs IIRC; I chose reiserfs, and I am pretty sure finding a file in a directory in reiserfs is O(log n) in the worse case. (Actually, I think that except for HUGE directories [far larger than /usr/lib] it accesses two or three blocks of disk in every case, hence being O(1)). How many directory entries do you think fit in a block? Is this a trick question? see... the average lib*.so.x.y in my disk is 12 characters long, a block has 8192 bytes, with an overhead of two dwords per dentry we have an average 409.6 directory entries in a block. YMMV. ls /usr/lib | wc -l brings me 9000, so it's very different to the disk thrice and twenty times just to give a ENOENT (ten times in the average to give success in load one lib) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 04:40:11PM -0700, Thomas Bushnell BSG wrote: What does the default Debian install do? Debian seems to use ext3 without directory indexing by default. Which is a sane choice as directory indexing on ext3 still seems to be not fully mature. And as mentioned in another thread it's not available on ext2 at all. So that means that, in fact, directory lookups on Debian are O(n), and we are, in fact, subject to problems from huge directories. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Martin Dickopp [EMAIL PROTECTED] writes: Would you agree that that bug should be fixed (in Etch), irrespective of whether the FHS is also changed to split /usr/lib? I'm not expert enough on the other factors that might be relevant to say. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG wrote: Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 04:40:11PM -0700, Thomas Bushnell BSG wrote: What does the default Debian install do? Debian seems to use ext3 without directory indexing by default. Which is a sane choice as directory indexing on ext3 still seems to be not fully mature. And as mentioned in another thread it's not available on ext2 at all. So that means that, in fact, directory lookups on Debian are O(n), and we are, in fact, subject to problems from huge directories. As I said before, as far as I recall, the Debian installer suggested me only filesystems that have O(1) [O(log n) worst case] directory lookup. I chose reiserfs, but the installer IIRC suggested ext3 and xfs as alternatives. I will probably re-install my laptop this weekend, and then I can give you more accurate info. HTH -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Humberto Massa [EMAIL PROTECTED] writes: Thomas Bushnell BSG wrote: Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 04:40:11PM -0700, Thomas Bushnell BSG wrote: What does the default Debian install do? Debian seems to use ext3 without directory indexing by default. Which is a sane choice as directory indexing on ext3 still seems to be not fully mature. And as mentioned in another thread it's not available on ext2 at all. So that means that, in fact, directory lookups on Debian are O(n), and we are, in fact, subject to problems from huge directories. As I said before, as far as I recall, the Debian installer suggested me only filesystems that have O(1) [O(log n) worst case] directory lookup. I chose reiserfs, but the installer IIRC suggested ext3 and xfs as alternatives. I will probably re-install my laptop this weekend, and then I can give you more accurate info. BUt according to Christoph Hellwig, the ext3 which is the default is used without directory indexing, which returns you to O(n). -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Wednesday 11 May 2005 17:21, Thomas Bushnell BSG wrote: BUt according to Christoph Hellwig, the ext3 which is the default is used without directory indexing, which returns you to O(n). You have yet to present any numbers which show there is a problem here. Can we please discuss real world problems? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Will Newton wrote: On Wednesday 11 May 2005 17:21, Thomas Bushnell BSG wrote: BUt according to Christoph Hellwig, the ext3 which is the default is used without directory indexing, which returns you to O(n). You have yet to present any numbers which show there is a problem here. Can we please discuss real world problems? This is not an imaginary problem, after all, in principle. Let's see, as I wrote before, my installation has *thousands* of files in /usr/lib and, in some filesystems, this can add up to a very large time (and ab-use of dentry cache memory) to link, say, Konqueror (which links to *hundreds* of shared objects). Imagine that, to load Konqui, you have to go 200 times to the disk (ok, cache, but...), each of them reading the 1 entries I have in /usr/lib, some of them twice or three times, to follow the symlinks. This is a real inefficiency. So, if you ask me for MHO, ext3 should be used by default *with* directory indexing. And maybe FHS should be pressed to provide something like /usr/libexec. -- HTH Massa -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Wednesday 11 May 2005 17:35, Humberto Massa wrote: This is not an imaginary problem, after all, in principle. Let's see, as I wrote before, my installation has *thousands* of files in /usr/lib and, in some filesystems, this can add up to a very large time (and ab-use of dentry cache memory) to link, say, Konqueror (which links to *hundreds* of shared objects). Imagine that, to load Konqui, you have to go 200 times to the disk (ok, cache, but...), each of them reading the 1 entries I have in /usr/lib, some of them twice or three times, to follow the symlinks. This is a real inefficiency. That is a possibility, it does sound sub-optimal. However, if you optimise before measuring there is no guarantee things will get any faster. Is reading the directory taking an appreciable amount of time compared to say, doing relocations? So, if you ask me for MHO, ext3 should be used by default *with* directory indexing. And maybe FHS should be pressed to provide something like /usr/libexec. How much stuff would go in libexec? I suspect it would mostly be stuff in currently in subdirectories of /usr/lib, which is less than 7% of my /usr/lib. So 7% performance improvement on something that is yet to be proven to be a bottleneck. On some filesystems. Without benchmarks it's a pointless discussion anyway. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
[Humberto Massa] As I said before, as far as I recall, the Debian installer suggested me only filesystems that have O(1) [O(log n) worst case] directory lookup. I chose reiserfs, but the installer IIRC suggested ext3 and xfs as alternatives. As Christoph (I think) said, Debian creates ext3 filesystems without the dir_index flag, by default. He even mentioned that ext3 dir_index may have a few problems remaining, so that this is a wise default. You may of course use 'tune2fs -O dir_index /dev/whatever' to change this, and then you'll have your O(1) lookups and opens. Requires remounting, and I think an fsck pass. HOWEVER This is a very silly thing to argue about without benchmarks. Those who care about this - yes, Thomas, I mean you - should get numbers. Here's how: (1) dynamicly link a hello world program to a dozen or so libraries (2) find or create a Debian system with a few thousand /usr/lib files (3) figure out execution time using your favorite loop technique (4) rename /usr/lib to /usr/libexec, create a new /usr/lib, and use 'ln' (not 'ln -s' of course - symlinks of course just add *more* lookups, in the original big directory) to repopulate /usr/lib with just a few hundred library files and symlinks. You may wish to give /lib similar treatment, but tread with care lest you find yourself unable to complete the procedure. (For one thing you'll want to do it in 'sash' or 'busybox'.) (5) repeat your favorite loop technique Getting a cold dentry cache before steps 3 and 5 is left as an exercise to the poor sap with so much dedication to the Gentoo premature optimisations R us ideals. Oh yeah, for bonus points: (6) put your original /usr/lib back where it was, then convert libfoo.so.N symlinks to hard links. That will cut your directory lookups in half. See if this makes any difference. If so, ponder a new proposal taking this optimisation into account as well. signature.asc Description: Digital signature
Re: /usr/lib vs /usr/libexec
Le mercredi 11 mai 2005 à 13:35 -0300, Humberto Massa a écrit : Imagine that, to load Konqui, you have to go 200 times to the disk (ok, cache, but...), each of them reading the 1 entries I have in /usr/lib, some of them twice or three times, to follow the symlinks. This is a real inefficiency. You said it: there is a cache. After the first access, the directory will be in the cache. Making all of this a purely imaginary problem. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: This is a digitally signed message part
Re: /usr/lib vs /usr/libexec
Peter Samuelson wrote: (...) HOWEVER This is a very silly thing to argue about without benchmarks. Those who care about this - yes, Thomas, I mean you - should get numbers. Here's how: (steps 1-6) You are 100% right and I stand corrected. -- HTH, Massa -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Tue, May 10, 2005 at 05:42:31AM +0200, Bernd Eckenfels wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. Why is that? Missing bootloader support. - a larger FS has more chance of failing so you risk having a fully broken system more often And two file systems have even more chance. One read only file system is pretty stable. Sure, that's why I have /usr mounted read-only. - /usr can be easily network (shared accross the same arch) mounted while / (due to /etc) can't - / needs functioning device nodes on it while usr can be mounted nodev I agreee, those arguments and the netboot stuff is an argment for a smaller root partition. However our root filesystem is too big anyway. That's true: $ df -h FilesystemSize Used Avail Use% Mounted on /dev/hda5 99M 75M 19M 80% / [...] $ du -sh /etc/gconf 26M /etc/gconf That's 1/3 of my root fs. It's damn too much. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Once upon a time GOMBAS Gabor said... $ df -h FilesystemSize Used Avail Use% Mounted on /dev/hda5 99M 75M 19M 80% / [...] $ du -sh /etc/gconf 26M /etc/gconf That's 1/3 of my root fs. It's damn too much. I discovered this a while ago and learned that most of it can be removed. I think they must be conf files so were never automatically removed, but you should find newer versions of all the files in the /etc/gconf directory in /usr/share/gconf. I've got only a couple of schemas left in /etc/gconf and if I check, those may be able to be removed by now as well. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
In article [EMAIL PROTECTED] you wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. Why is that? Missing bootloader support. the bootloader does not need to access the root filesystem. It only loads the kernel and the initrd from /boot. Greetings Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Mon, May 09, 2005 at 08:38:02AM -0700, Thomas Bushnell BSG wrote: Martin Waitz [EMAIL PROTECTED] writes: The BSDs use libexec but I don't really see a good reason why it exists. It reduces search times in libraries, which is important. We do not have that bug, so it's not important to us. -- .''`. ** Debian GNU/Linux ** | Andrew Suffield : :' : http://www.debian.org/ | `. `' | `- -- | signature.asc Description: Digital signature
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG wrote: Martin Dickopp [EMAIL PROTECTED] writes: Thomas Bushnell BSG [EMAIL PROTECTED] writes: If there is a reason to separate /usr from / (which so many people think there is, though I don't understand why, since it has no semantic significance at all), why separate /lib from /etc? I don't see a semantic difference between /bin and /usr/bin (or /lib and /usr/lib). IMHO, the only reason for /bin and /lib is that some programs and libraries need to be available before is /usr is mounted. That doesn't make sense. If you get rid of the /usr vs / distinction, then there is no before /usr is mounted. Have you considered that for some (eg embedded) applications, the block device containing / does not have enough space to contain everything that is under /usr? And that, as a result, it would have to mount it as a network drive, for instance? That wasn't my argument. My argument is that I don't consider shared libraries and internal executables different kinds of things. They are both binaries loaded and executed by a program. Sure, and documentation and libraries are both files opened by programs. The difference is that libraries are also generic things that are shared by many programs, and searched by the linker, whereas executables are not. In fact, a library is loaded and executed by only two programs (ld and ld.so) in the normal scheme of things. The difference between /usr/lib and /usr/shared is more clear: what is under /usr/lib depends on the arch; what is under /usr/share doess not. So, /usr/lib can be mounted in a network drive, shared by a lot of machines of the same arch in a network. And /usr/shared can be (hmpf)... shared... between all machines in a network, independently of arch. Thomas HTH, Massa -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG wrote: Goswin von Brederlow [EMAIL PROTECTED] writes: That doesn't make sense. If you get rid of the /usr vs / distinction, then there is no before /usr is mounted. But then you have a minimum 1-5GB /. That sucks. Why, exactly? I know people think it's obvious, but the lack of stated reasons worries me. I know the *original* reasons why / needed to be small, but they don't apply anymore. So I'll buy the it's obvious if you can state the original reasons and explain why you think they still apply. If not, then what is the current reason? Thomas What do you think are the original reasons / needed to be small? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Le mardi 10 mai 2005 à 10:21 +0200, GOMBAS Gabor a écrit : On Tue, May 10, 2005 at 05:42:31AM +0200, Bernd Eckenfels wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. Why is that? Missing bootloader support. Which bootloader doesn't support reiserfs? $ du -sh /etc/gconf 26M /etc/gconf That's 1/3 of my root fs. It's damn too much. Almost all the schemas were already moved out to /usr/share. We plan to move the defaults directory structure to /var/lib/gconf after the release - at least, the defaults brought by package; we have to keep a defaults structure in /etc for local modifications. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: /usr/lib vs /usr/libexec
On Tuesday 10 May 2005 02:18, Goswin von Brederlow [EMAIL PROTECTED] wrote: Russell Coker [EMAIL PROTECTED] writes: It seems to me that /usr/libexec is a better name for such things, and having the same directory names used across distributions provides real benefits (copying config files and binaries from other distributions when a bug stops a server working and it's REALLY important to get it fixed fast). Should we change some of these to /usr/libexec? That would be /usr/libexec/arch-os/postfix vs /usr/lib/arch-os/postfix then. If you consider any change then please include the multiarch changes at the same time. No point doing 2 transitions for etch. Why would it be desirable to have arch-os directories under libexec? By definition anything in libexec is a program that is a part of another program. In the case of Postfix there are many cooperating programs running with different privs (different UID and different SE Linux domain) that are used for different aspects of Postfix functionality. Why would you want one of those programs to be 32bit and another to be 64bit? Firstly there is little performance to be gained by running Postfix 64bit. Any performance issues that a Postfix server will experience will be related to disk IO (not a word size issue) or the speed of external programs such as virus scanners and Postgrey (which interoperate with Postfix via TCP sockets and have no inter-dependencies on word size etc). Secondly there is no benefit to using different word sizes for different parts of Postfix. I can't imagine any reason for using different word sizes. There is a possibility of having Postfix call external shared objects which may make some dependencies on word size. But if you have two shared objects one of which is 32bit and another 64bit then trying to get Postfix processes to use both will be a losing game. AFAIK there is no design documentation which precisely states which Postfix sub-process will use a given shared object. Finally no-one has tested Postfix for such things. I think that Postfix is very well written and should work even if you had parts of it being 64bit and parts being 32bit. But this is not tested or guaranteed. If there was a bug in this regard then it could have security implications, do you want to take the chance? -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Tuesday 10 May 2005 10:36, Goswin von Brederlow [EMAIL PROTECTED] wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. I believe that there are LILO patches for /boot on LVM. There's no reason why GRUB and other boot loaders couldn't be updated in the same manner. The most obvious solution is to have the LV containing /boot being marked in the LVM setup as contiguous. Then the boot loader merely has to use a different method of finding the first block. ReiserFS has been supported for /boot for ages. When I took over maintenance of LILO it was initially to get the ReiserFS patches included. RAID-0 could be supported by boot loaders. The only real complication is the need to correctly identify two boot disks (correctly identifying one is enough pain as anyone who has worked with boot loader code knows well). In fact I'm reasonably sure that the only reason for not having support for RAID-0 in boot loaders is the fact that it's so painful to identify two disks correctly. When you have N disks the difficulty (for programmer and for sys-admin) will be of order N*(N-1) instead of being of order N. But generally the best solution to such problems is to have a separate partition for /boot. Red Hat defaults to an LVM install (including LVM root) with a regular 100M partition for /boot, that works well. - a larger FS has more chance of failing so you risk having a fully broken system more often I doubt that. I think that a FS that is written less has less chance of failing, which is an extra reason for having /var and /home on different file systems to /. If you have /var and /home on separate partitions then / should not get many writes and should have little chance of corruption regardless of size. - /usr can be easily network (shared accross the same arch) mounted while / (due to /etc) can't Why is this desirable in the days of large disks? There is no machine for which I am responsible which has a disk space issue other than my laptop. None of my machines has /usr taking any significant portion of the disk space. - / needs functioning device nodes on it while usr can be mounted nodev The current Fedora setup of having an initrd create device nodes on a tmpfs is working quite well and could be copied for Debian. - a small / can be replicated across many disks to ensure the system always comes up and e.g. at least send an email to the admin. / can even be an initrd How big can an initrd be? The default is 4M which isn't going to do well for /. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Tue, May 10, 2005 at 11:16:54AM +0200, Bernd Eckenfels wrote: the bootloader does not need to access the root filesystem. It only loads the kernel and the initrd from /boot. (I assume that /boot is on /. If not, the following still applies to /boot.) Well, grub _does_ access the filesystem directly so it needs explicit support for every filesystem you want to boot from. I think reiserfs is OK nowadays (I do not use it so I'm not sure). xfs + lilo (from the XFS FAQ): No, for root partition installations because the XFS superblock is written at block zero, where LILO would be installed. This is to maintain compatibility with the IRIX on-disk format, and will not be changed. lvm, raid0, raid5: the boot loader must understand the lvm/raid descriptors to be able to determine where to load the kernel/initrd from (and in case of raid5, it has to be able to reconstruct the data using the parity disk if one of the non-parity chunks is inaccessible). AFAIK neither lilo nor Grub supports these. Gabor -- - MTA SZTAKI Computer and Automation Research Institute Hungarian Academy of Sciences - -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Andrew Suffield [EMAIL PROTECTED] writes: We do not have that bug, so it's not important to us. Still, nobody has said. What filesystems available on Debian have a better than linear search time for open, and are they used by a default Debian install? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Humberto Massa [EMAIL PROTECTED] writes: What do you think are the original reasons / needed to be small? I know what they are. PDP-11 boot loaders couldn't access long block addresses. This was copied into 32V on the Vax, where it entered 4BSD. Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Wednesday 11 May 2005 00:55, GOMBAS Gabor [EMAIL PROTECTED] wrote: On Tue, May 10, 2005 at 11:16:54AM +0200, Bernd Eckenfels wrote: the bootloader does not need to access the root filesystem. It only loads the kernel and the initrd from /boot. (I assume that /boot is on /. If not, the following still applies to /boot.) Of course /boot only really needs to be about 20M in size. It can be read-only mounted most of the time (or even entirely umounted), so it's in an entirely different category to the rest of the system. Well, grub _does_ access the filesystem directly so it needs explicit support for every filesystem you want to boot from. On the system I use to write this email (Fedora): /boot/grub/xfs_stage1_5 /boot/grub/reiserfs_stage1_5 So I guess that GRUB supports XFS and ReiserFS. Incidentally if you have a separate partition for /boot (as some people recommend doing for reliability issues) then XFS and ReiserFS are bad options due to the journals which have overly large minimum sizes. Ext2/3 really is the best option for a /boot file system. It's only when /boot is on the root file system that some other file system may be desired. I think reiserfs is OK nowadays (I do not use it so I'm not sure). xfs + lilo (from the XFS FAQ): No, for root partition installations because the XFS superblock is written at block zero, where LILO would be installed. This is to maintain compatibility with the IRIX on-disk format, and will not be changed. The simple solution to this is that you don't install LILO on the first block of the partition but instead you install it as the boot block of the hard disk (IE use /dev/hda instead of /dev/hda1 or whatever). If you wanted to use LILO with XFS and LVM then I guess that you would have to put the LILO boot block on /dev/hda anyway. The only reason for installing the LILO boot block onto /dev/hda1 instead of /dev/hda is for a software RAID-1 installation where you want to have the Debian-MBR on /dev/hda and /dev/hdc while /dev/hda1 and /dev/hdc1 have the LILO boot blocks. Apart from that I recommend putting the LILO boot loader in the partition table. lvm, raid0, raid5: the boot loader must understand the lvm/raid descriptors to be able to determine where to load the kernel/initrd from (and in case of raid5, it has to be able to reconstruct the data using the parity disk if one of the non-parity chunks is inaccessible). AFAIK neither lilo nor Grub supports these. Here is the relevant section of the lilo changelog regarding LVM: lilo (1:22.6.1-4) unstable; urgency=high * debian/patches/13_bad-partition-warn.dpatch: - Rewrote. Now it adds LVM partitions as a secure partition type to install LILO in, while leaving the warning for really dangerous partition types. It also removes the prompt when the user wants to install LILO on a Windows(R) partition. * debian/lilo.config: - Never set to 'false' lilo/runme template. (Closes: #284929) -- AndrC3A9s RoldC3A1n [EMAIL PROTECTED] Thu, 9 Dec 2004 20:22:14 + -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
GOMBAS Gabor [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 11:16:54AM +0200, Bernd Eckenfels wrote: the bootloader does not need to access the root filesystem. It only loads the kernel and the initrd from /boot. (I assume that /boot is on /. If not, the following still applies to You've missed the point. Split / and /boot, that makes sense if it's necessary. Splitting / and /usr does not make sense. Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Bernd Eckenfels [EMAIL PROTECTED] writes: In article [EMAIL PROTECTED] you wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. Why is that? Lvm has its backup data in /etc by default. If you ever need it you are screwed with / on lvm. Also snapshots and pvmove don't work (deadlock). raid0/5 don't have support in the bootloaders. reiserfs/xfs miss support in bootloaders or their tail usage feature breaks them. - a larger FS has more chance of failing so you risk having a fully broken system more often And two file systems have even more chance. One read only file system is pretty stable. Hardly anyone has it read-only nowadays. - /usr can be easily network (shared accross the same arch) mounted while / (due to /etc) can't - / needs functioning device nodes on it while usr can be mounted nodev I agreee, those arguments and the netboot stuff is an argment for a smaller root partition. However our root filesystem is too big anyway. Greetings Bernd Not all of those arguments work for everyone nor do they all work together. Every user case is different. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Goswin von Brederlow [EMAIL PROTECTED] writes: Lvm has its backup data in /etc by default. If you ever need it you are screwed with / on lvm. Also snapshots and pvmove don't work (deadlock). raid0/5 don't have support in the bootloaders. reiserfs/xfs miss support in bootloaders or their tail usage feature breaks them. None of these are arguments for separating /usr and /; they are arguments for separating /boot and /. Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Josselin Mouette [EMAIL PROTECTED] writes: Le mardi 10 mai 2005 à 10:21 +0200, GOMBAS Gabor a écrit : On Tue, May 10, 2005 at 05:42:31AM +0200, Bernd Eckenfels wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. Why is that? Missing bootloader support. Which bootloader doesn't support reiserfs? milo, alilo, silo, ... $ du -sh /etc/gconf 26M /etc/gconf That's 1/3 of my root fs. It's damn too much. 3.8M/etc/gconf Almost all the schemas were already moved out to /usr/share. We plan to move the defaults directory structure to /var/lib/gconf after the release - at least, the defaults brought by package; we have to keep a defaults structure in /etc for local modifications. Some improvement but still way to big for my taste. Maybe /usr/etc isn't such a bad idea. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes: On Tuesday 10 May 2005 02:18, Goswin von Brederlow [EMAIL PROTECTED] wrote: Russell Coker [EMAIL PROTECTED] writes: It seems to me that /usr/libexec is a better name for such things, and having the same directory names used across distributions provides real benefits (copying config files and binaries from other distributions when a bug stops a server working and it's REALLY important to get it fixed fast). Should we change some of these to /usr/libexec? That would be /usr/libexec/arch-os/postfix vs /usr/lib/arch-os/postfix then. If you consider any change then please include the multiarch changes at the same time. No point doing 2 transitions for etch. Why would it be desirable to have arch-os directories under libexec? 32bit mozilla with flash plugin and 64bit mozilla without. A lot of people seem to want that. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes: On Tuesday 10 May 2005 10:36, Goswin von Brederlow [EMAIL PROTECTED] wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. I believe that there are LILO patches for /boot on LVM. There's no reason why GRUB and other boot loaders couldn't be updated in the same manner. The most ... They don't now and there are way more bootloaders than just grub and lilo. But generally the best solution to such problems is to have a separate partition for /boot. Red Hat defaults to an LVM install (including LVM root) with a regular 100M partition for /boot, that works well. No lvm backup data available in case of superblock corruption. Bad idea. No booting with init=/bin/sh to patch things back together as / can't be mounted. Bad idea again. / on lvm is a major pain in case of error and if you already need a seperate / partition adding another for /boot is a bit stupid. The best solution is a regular 100-200Mb partition for / including /boot imho. - a larger FS has more chance of failing so you risk having a fully broken system more often I doubt that. I think that a FS that is written less has less chance of failing, which is an extra reason for having /var and /home on different file systems to /. If you have /var and /home on separate partitions then / should not get many writes and should have little chance of corruption regardless of size. - /usr can be easily network (shared accross the same arch) mounted while / (due to /etc) can't Why is this desirable in the days of large disks? There is no machine for which I am responsible which has a disk space issue other than my laptop. None of my machines has /usr taking any significant portion of the disk space. I have two recently bought systems without disk. Just a small flash to boot from. Those mips based wireless and dsl routers from linksys and similar are very popular as well as meshcubes. - / needs functioning device nodes on it while usr can be mounted nodev The current Fedora setup of having an initrd create device nodes on a tmpfs is working quite well and could be copied for Debian. - a small / can be replicated across many disks to ensure the system always comes up and e.g. at least send an email to the admin. / can even be an initrd How big can an initrd be? The default is 4M which isn't going to do well for /. The limit is either 1GB or slightly below 2GB for 32bit (+amd64) systems. Less for mips I guess due to the hardwired address space. Enough ram provided of course (4GB ram for a 2GB ramdisk as it gets copied around by the kernel). The problem is more how much ram do you want to spend on it than how much linux can use. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Le mardi 10 mai 2005 à 17:27 +0200, Goswin von Brederlow a écrit : Almost all the schemas were already moved out to /usr/share. We plan to move the defaults directory structure to /var/lib/gconf after the release - at least, the defaults brought by package; we have to keep a defaults structure in /etc for local modifications. Some improvement but still way to big for my taste. Maybe /usr/etc isn't such a bad idea. Whoa? Once this move is made, there will be nothing in /etc unless the system administrator explicitly does something about it. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom
Re: /usr/lib vs /usr/libexec
On Wednesday 11 May 2005 01:39, Goswin von Brederlow [EMAIL PROTECTED] wrote: Russell Coker [EMAIL PROTECTED] writes: On Tuesday 10 May 2005 10:36, Goswin von Brederlow [EMAIL PROTECTED] wrote: - / can't be on lvm, raid0, raid5, reiserfs, xfs without causing problems for /boot. I believe that there are LILO patches for /boot on LVM. There's no reason why GRUB and other boot loaders couldn't be updated in the same manner. The most ... They don't now and there are way more bootloaders than just grub and lilo. Yes there are way more boot loaders than GRUB and LILO. The task for the developers of such boot loaders is easier because they can grab some source from GRUB and LILO for some of these things. But generally the best solution to such problems is to have a separate partition for /boot. Red Hat defaults to an LVM install (including LVM root) with a regular 100M partition for /boot, that works well. No lvm backup data available in case of superblock corruption. Bad idea. No booting with init=/bin/sh to patch things back together as / can't be mounted. Bad idea again. If you are making backups of your machines (generally accepted to be a good idea) then LVM metadata will be included in such backups. If you don't have any backups then just reinstall the machine - it apparently doesn't have any data worth keeping. / on lvm is a major pain in case of error and if you already need a seperate / partition adding another for /boot is a bit stupid. / on LVM allows for snapshot backups which are the most convenient method of backup. - /usr can be easily network (shared accross the same arch) mounted while / (due to /etc) can't Why is this desirable in the days of large disks? There is no machine for which I am responsible which has a disk space issue other than my laptop. None of my machines has /usr taking any significant portion of the disk space. I have two recently bought systems without disk. Just a small flash to boot from. Those mips based wireless and dsl routers from linksys and similar are very popular as well as meshcubes. How much storage do the wireless and DSL routers have? Flash devices up to 1G in size are quite common nowadays. Distributions that can work with 32M of storage are still available (Familiar is one example). -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG wrote: Andrew Suffield [EMAIL PROTECTED] writes: We do not have that bug, so it's not important to us. Still, nobody has said. What filesystems available on Debian have a better than linear search time for open, and are they used by a default Debian install? These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. Q: are they used by a default? A: Last time I installed Debian (15 days ago), it asked me if I wanted my partition ext3, xfs, or reiserfs IIRC; I chose reiserfs, and I am pretty sure finding a file in a directory in reiserfs is O(log n) in the worse case. (Actually, I think that except for HUGE directories [far larger than /usr/lib] it accesses two or three blocks of disk in every case, hence being O(1)). HTH -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Wednesday 11 May 2005 01:28, Goswin von Brederlow [EMAIL PROTECTED] wrote: Why would it be desirable to have arch-os directories under libexec? On fedora-devel Bill Nottingham suggested having /usr/lib vs /usr/lib64 for programs that care about such things and /usr/libexec for programs that don't. 32bit mozilla with flash plugin and 64bit mozilla without. A lot of people seem to want that. Bill's idea seems to work in that case. Although as you would need different names in /usr/bin it might make sense to just name the libexec files with the same extension as the file in /usr/bin that launches them. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG wrote: You've missed the point. Split / and /boot, that makes sense if it's necessary. Splitting / and /usr does not make sense. Sure it does. Especially if you want / to be in a Flash disk and /usr to be somewhere else in the network. HTH Massa -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG wrote: Still, nobody has said. What filesystems available on Debian have a better than linear search time for open, reiserfs, ext2/3 (with dir_index), and probably others. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Tue, May 10, 2005 at 02:03:01PM -0300, Humberto Massa wrote: These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. ext2 doesn't. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Christoph Hellwig wrote: On Tue, May 10, 2005 at 02:03:01PM -0300, Humberto Massa wrote: These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. ext2 doesn't. With dir_index, yes it does. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Tue, May 10, 2005 at 02:21:50PM -0300, Humberto Massa wrote: ext2 doesn't. With dir_index, yes it does. If you want to forward port a three year old patch full of bugs and incompatible to the dir_index used in ext3 - all luck to you. All debian kernel-image packages don't have it for sure. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
GOMBAS Gabor [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 11:16:54AM +0200, Bernd Eckenfels wrote: the bootloader does not need to access the root filesystem. It only loads the kernel and the initrd from /boot. (I assume that /boot is on /. If not, the following still applies to /boot.) Well, grub _does_ access the filesystem directly so it needs explicit support for every filesystem you want to boot from. I think reiserfs is OK nowadays (I do not use it so I'm not sure). xfs + lilo (from the XFS FAQ): No, for root partition installations because the XFS superblock is written at block zero, where LILO would be installed. This is to maintain compatibility with the IRIX on-disk format, and will not be changed. That only applies to the bootloader on the partition or the FS on the drive directly. With partitions and the bootloader in the mbr that should not matter. What I was talking about is using a block to store the tail end of multiple files more efficiently. Didn't xfs do that? lvm, raid0, raid5: the boot loader must understand the lvm/raid descriptors to be able to determine where to load the kernel/initrd from (and in case of raid5, it has to be able to reconstruct the data using the parity disk if one of the non-parity chunks is inaccessible). AFAIK neither lilo nor Grub supports these. It would be nice enough if it would work with a working raid. The problem there (raid 0 and 5) is that it would need to gather the scattered blocks from multiple devices. Gabor -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Josselin Mouette [EMAIL PROTECTED] writes: Le mardi 10 mai 2005 à 17:27 +0200, Goswin von Brederlow a écrit : Almost all the schemas were already moved out to /usr/share. We plan to move the defaults directory structure to /var/lib/gconf after the release - at least, the defaults brought by package; we have to keep a defaults structure in /etc for local modifications. Some improvement but still way to big for my taste. Maybe /usr/etc isn't such a bad idea. Whoa? Once this move is made, there will be nothing in /etc unless the system administrator explicitly does something about it. Ok, so we are just not there yet (or I have garbage left in etc). MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 02:03:01PM -0300, Humberto Massa wrote: These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. ext2 doesn't. Convert it to utilize directory hashing. The ability is there but iirc isn't used by default. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes: On Wednesday 11 May 2005 01:39, Goswin von Brederlow [EMAIL PROTECTED] wrote: / on lvm is a major pain in case of error and if you already need a seperate / partition adding another for /boot is a bit stupid. / on LVM allows for snapshot backups which are the most convenient method of backup. Except that the kernel freezes the device because the DM lock and device node updating deadlock. Might work with udev or devfs, haven't tried that. - /usr can be easily network (shared accross the same arch) mounted while / (due to /etc) can't Why is this desirable in the days of large disks? There is no machine for which I am responsible which has a disk space issue other than my laptop. None of my machines has /usr taking any significant portion of the disk space. I have two recently bought systems without disk. Just a small flash to boot from. Those mips based wireless and dsl routers from linksys and similar are very popular as well as meshcubes. How much storage do the wireless and DSL routers have? Flash devices up to 1G in size are quite common nowadays. Distributions that can work with 32M of storage are still available (Familiar is one example). 16-64MB depending on model. Adding a big usb stick or CF card for /usr makes a very nice and quite router. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes:
On Wednesday 11 May 2005 01:28, Goswin von Brederlow
[EMAIL PROTECTED] wrote:
Why would it be desirable to have arch-os directories under libexec?
On fedora-devel Bill Nottingham suggested having /usr/lib vs /usr/lib64 for
programs that care about such things and /usr/libexec for programs that
don't.
32bit mozilla with flash plugin and 64bit mozilla without. A lot of
people seem to want that.
Bill's idea seems to work in that case. Although as you would need different
names in /usr/bin it might make sense to just name the libexec files with the
same extension as the file in /usr/bin that launches them.
What about mips O32, N32, N64 abis?
/lib, /lib32 and /lib64?
What about i386 knetbsd and linux?
The multiarch /arch-os/ path is already present in the toolchain for
many things including include files and libs and works for all cases
of multiarch in a clean way. The lib{,32,64} subdirs are different on
every arch, confusing and insuffient for the bsd case.
MfG
Goswin
--
To UNSUBSCRIBE, email to [EMAIL PROTECTED]
with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Le mardi 10 mai 2005 à 21:37 +0200, Goswin von Brederlow a écrit : Josselin Mouette [EMAIL PROTECTED] writes: Le mardi 10 mai 2005 à 17:27 +0200, Goswin von Brederlow a écrit : Almost all the schemas were already moved out to /usr/share. We plan to move the defaults directory structure to /var/lib/gconf after the release - at least, the defaults brought by package; we have to keep a defaults structure in /etc for local modifications. Some improvement but still way to big for my taste. Maybe /usr/etc isn't such a bad idea. Whoa? Once this move is made, there will be nothing in /etc unless the system administrator explicitly does something about it. Ok, so we are just not there yet (or I have garbage left in etc). Yes, this is planned for etch. The exact implementation hasn't been decided yet, because this is quite a drastic change. -- .''`. Josselin Mouette/\./\ : :' : [EMAIL PROTECTED] `. `'[EMAIL PROTECTED] `- Debian GNU/Linux -- The power of freedom signature.asc Description: This is a digitally signed message part
Re: /usr/lib vs /usr/libexec
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Martin Waitz [EMAIL PROTECTED] writes: hoi :) On Mon, May 09, 2005 at 03:45:32PM +1000, Russell Coker wrote: Should we change some of these to /usr/libexec? well, it would be against the FHS, I think. The BSDs use libexec but I don't really see a good reason why it exists. The only reason we don't have it is because of petty bickering and politics between the FHS folks (several years ago). There were few technical objections to it on the FHS list, but it was dropped for non-technical reasons. Given that the FHS is supposed to codify existing practice, it should be in there on that count alone. Every libexec-using package in Debian has been reconfigured not to use it; upstreams do use it, and I'd like to use it myself. I'd personally be very glad to have it, and would support using it in Debian. Regards, Roger - -- Roger Leigh Printing on GNU/Linux? http://gimp-print.sourceforge.net/ Debian GNU/Linuxhttp://www.debian.org/ GPG Public Key: 0x25BFB848. Please sign and encrypt your mail. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.0 (GNU/Linux) Comment: Processed by Mailcrypt 3.5.8 http://mailcrypt.sourceforge.net/ iD8DBQFCgRvZVcFcaSW/uEgRAlcpAJ920/c0RktK+9FjORJNfNEushYhsgCg7H8h tJo3qJd/a4eyOnGGHOXjdwc= =7Ppv -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Tue, May 10, 2005 at 08:12:38AM -0700, Thomas Bushnell BSG wrote: Andrew Suffield [EMAIL PROTECTED] writes: We do not have that bug, so it's not important to us. Still, nobody has said. What filesystems available on Debian have a better than linear search time for open, and are they used by a default Debian install? Who cares? What Debian systems spend a noticable amount of time waiting for ld to search for libraries? I've never seen ld be anything less than instantaneous to find libraries, even on old boxes with just about everything installed. It spends all its time linking, not waiting for open() to complete. Arguing about the theoretical performance is a bloody waste of time in the absence of an actual problem. As I said, we do not have that bug. -- .''`. ** Debian GNU/Linux ** | Andrew Suffield : :' : http://www.debian.org/ | `. `' | `- -- | signature.asc Description: Digital signature
Re: /usr/lib vs /usr/libexec
In article [EMAIL PROTECTED] you wrote: Still, nobody has said. What filesystems available on Debian have a better than linear search time for open, and are they used by a default Debian install? /etc/ld.so.cache Gruss Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
In article [EMAIL PROTECTED] you wrote: No lvm backup data available in case of superblock corruption. Bad idea. No booting with init=/bin/sh to patch things back together as / can't be mounted. Bad idea again. You can store the backup wherever you like, and an emergency boot via usb stick, netboot or standby volume is the default way to recover a server. Gruss Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
In article [EMAIL PROTECTED] you wrote: Why would it be desirable to have arch-os directories under libexec? For sharing the /usr tree among multiple machines with different architectures (I guess). Gruss Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas == Thomas Bushnell BSG [EMAIL PROTECTED] writes: Thomas You've missed the point. Split / and /boot, that makes Thomas sense if it's necessary. Splitting / and /usr does not Thomas make sense. Bad example. A better example might be if you want to mount /usr via NFS or some other network file-system. I have heard people use AFS for this purpose. I am sure there are other file-systems that could be used. Yes, there are ways you can mount / via a network file-system: * NFS Boot code in kernel that auto-configures the network. * initramfs (anybody written code to do this?) However, if all you want to do is share /usr between systems, currently the simplest approach (and the only way I know this is possible) is if /usr is a separate from /. For starters, it only requires an extra entry in /etc/fstab. No changes to the boot structure. A relevant factor is that some directories (i.e. /etc and /var) cannot always be shared, but must be available early on in the boot process (i.e. /etc). So it might make sense to have a local private copy of /, but have /usr shared. Yes, this gets a bit messy in places (e.g. keeping /etc, /lib, and /var synchronized with /usr), but my point is to prove that there still are benefits in keeping /usr and / split. The arguments presented hold true of any filesystem that is complicated enough to require user-level tools to initialize, and for some reason you don't want to use an initramfs to initialize it. Or if you want /usr to be shared between computers but don't want to share all of /. -- Brian May [EMAIL PROTECTED] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Humberto Massa [EMAIL PROTECTED] writes: with the possible exception of FAT and Minix. Q: are they used by a default? A: Last time I installed Debian (15 days ago), it asked me if I wanted my partition ext3, xfs, or reiserfs IIRC; I chose reiserfs, and I am pretty sure finding a file in a directory in reiserfs is O(log n) in the worse case. (Actually, I think that except for HUGE directories [far larger than /usr/lib] it accesses two or three blocks of disk in every case, hence being O(1)). How many directory entries do you think fit in a block? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Goswin von Brederlow [EMAIL PROTECTED] writes: Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 02:03:01PM -0300, Humberto Massa wrote: These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. ext2 doesn't. Convert it to utilize directory hashing. The ability is there but iirc isn't used by default. What does the default Debian install do? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Bernd Eckenfels [EMAIL PROTECTED] writes: In article [EMAIL PROTECTED] you wrote: Still, nobody has said. What filesystems available on Debian have a better than linear search time for open, and are they used by a default Debian install? /etc/ld.so.cache Um, no. ld.so.cache gives you the filename to open, but it's still linear on ext2 to search the directory. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Bernd Eckenfels [EMAIL PROTECTED] writes: In article [EMAIL PROTECTED] you wrote: No lvm backup data available in case of superblock corruption. Bad idea. No booting with init=/bin/sh to patch things back together as / can't be mounted. Bad idea again. You can store the backup wherever you like, and an emergency boot via usb stick, netboot or standby volume is the default way to recover a server. Gruss Bernd Sure. I can type with my toes using windows and putty too. But you can forget your usb rescue stick or misplace your rescue cd. Can't happen to / and for the most parts that is enough to rescue the system or dd | netcat a backup to another host. I used it often enought to not want to miss it the next time. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG dijo [Mon, May 09, 2005 at 03:08:57PM -0700]: If there is a reason to separate /usr from / (which so many people think there is, though I don't understand why, since it has no semantic significance at all), why separate /lib from /etc? I don't see a semantic difference between /bin and /usr/bin (or /lib and /usr/lib). IMHO, the only reason for /bin and /lib is that some programs and libraries need to be available before is /usr is mounted. That doesn't make sense. If you get rid of the /usr vs / distinction, then there is no before /usr is mounted. As far as I have always understood this, there is an important distinction: / should have everything needed for booting the system into a mode that can be used to solve problems. This means, if you are performing an installation on very reduced media, you only put / in it, and /usr is network-mounted. This was quite a common setup some years ago, and is still somewhat common. Greetings, -- Gunnar Wolf - [EMAIL PROTECTED] - (+52-55)1451-2244 / 5554-9450 PGP key 1024D/8BB527AF 2001-10-23 Fingerprint: 0C79 D2D1 2C4E 9CE4 5973 F800 D80E F35A 8BB5 27AF -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
In article [EMAIL PROTECTED] you wrote: How many directory entries do you think fit in a block? If I see this right I habe 80blocks for 756 entries: # ls -a | wc -l 756 # ls -lsd 80 drwxr-xr-x 122 root root 57344 May 10 06:34 ./ Most likely in dache. Still a lot to traverse. Ext2 direntry is 8bytes plus filename (or onlined symlinks, which you have a lot on /usr/lib). In my case 54bytes per entry. Of course this are 512byte blocks in 10 4 chunks. Greetings Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG [EMAIL PROTECTED] writes: Humberto Massa [EMAIL PROTECTED] writes: with the possible exception of FAT and Minix. Q: are they used by a default? A: Last time I installed Debian (15 days ago), it asked me if I wanted my partition ext3, xfs, or reiserfs IIRC; I chose reiserfs, and I am pretty sure finding a file in a directory in reiserfs is O(log n) in the worse case. (Actually, I think that except for HUGE directories [far larger than /usr/lib] it accesses two or three blocks of disk in every case, hence being O(1)). How many directory entries do you think fit in a block? Does that matter? Make a big hash table containing all filenames and spaning many blocks. Make index blocks containing the size and the block numbers of that hash table with primary, secondary and tertiary indirection. With 2 indirections you can easily cover 1 MB of data filled with 512 byte structures of filename, inode number and any other flags you want to keep in the hash directly. That is enough for 100 files (alowing for wasted space for hash collisions). And you still only need 3 reads to open a file. Do you think you have more than 100 libs? Say a dir entry has 1K, do you have more than 50 libs? Even with 4K that gives 125000 entries. MfG Goswin PS: I don't know if reiserfs is doing it this way, just wanted to show how it could be done. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG [EMAIL PROTECTED] writes: Goswin von Brederlow [EMAIL PROTECTED] writes: Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 02:03:01PM -0300, Humberto Massa wrote: These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. ext2 doesn't. Convert it to utilize directory hashing. The ability is there but iirc isn't used by default. What does the default Debian install do? Afaik the good, old, slow linear list. With that file open/stat is O(n) and ls also O(n) (cause you keep reading the dir instead of starting at the top each time). MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
In article [EMAIL PROTECTED] you wrote: Ext2 direntry is 8bytes plus filename (or onlined symlinks, which you have a lot on /usr/lib). In my case 54bytes per entry. Me bad - the symlinks are inlined in the inodes of course. Gruss Bernd -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Roger Leigh [EMAIL PROTECTED] writes: The only reason we don't have it is because of petty bickering and politics between the FHS folks (several years ago). That seems a good description of the FHS in general... -Miles -- In New York, most people don't have cars, so if you want to kill a person, you have to take the subway to their house. And sometimes on the way, the train is delayed and you get impatient, so you have to kill someone on the subway. [George Carlin] -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Goswin von Brederlow [EMAIL PROTECTED] writes: Thomas Bushnell BSG [EMAIL PROTECTED] writes: Humberto Massa [EMAIL PROTECTED] writes: with the possible exception of FAT and Minix. Q: are they used by a default? A: Last time I installed Debian (15 days ago), it asked me if I wanted my partition ext3, xfs, or reiserfs IIRC; I chose reiserfs, and I am pretty sure finding a file in a directory in reiserfs is O(log n) in the worse case. (Actually, I think that except for HUGE directories [far larger than /usr/lib] it accesses two or three blocks of disk in every case, hence being O(1)). How many directory entries do you think fit in a block? Does that matter? Not if it's hashed. (But then why is it O(log n) in the worst case? That sounds like a search tree strategy, not a hash.) PS: I don't know if reiserfs is doing it this way, just wanted to show how it could be done. Right. I have no doubt it can be done. It's been done for decades in non-Unixoid systems. I'm asking about Debian, here and now, under default installation options, not under what can be done in the abstract. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Goswin von Brederlow [EMAIL PROTECTED] writes: Thomas Bushnell BSG [EMAIL PROTECTED] writes: Goswin von Brederlow [EMAIL PROTECTED] writes: Christoph Hellwig [EMAIL PROTECTED] writes: On Tue, May 10, 2005 at 02:03:01PM -0300, Humberto Massa wrote: These are two questions: Q: What filesystems... ? A: Every one of them with the possible exception of FAT and Minix. ext2 doesn't. Convert it to utilize directory hashing. The ability is there but iirc isn't used by default. What does the default Debian install do? Afaik the good, old, slow linear list. With that file open/stat is O(n) and ls also O(n) (cause you keep reading the dir instead of starting at the top each time). In which case, we do have that bug. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG [EMAIL PROTECTED] writes: Goswin von Brederlow [EMAIL PROTECTED] writes: Thomas Bushnell BSG [EMAIL PROTECTED] writes: Humberto Massa [EMAIL PROTECTED] writes: with the possible exception of FAT and Minix. Q: are they used by a default? A: Last time I installed Debian (15 days ago), it asked me if I wanted my partition ext3, xfs, or reiserfs IIRC; I chose reiserfs, and I am pretty sure finding a file in a directory in reiserfs is O(log n) in the worse case. (Actually, I think that except for HUGE directories [far larger than /usr/lib] it accesses two or three blocks of disk in every case, hence being O(1)). How many directory entries do you think fit in a block? Does that matter? Not if it's hashed. (But then why is it O(log n) in the worst case? That sounds like a search tree strategy, not a hash.) If the hash is split up over multiple data blocks then you end up with a tree or list structure on top of it. The tree is O(log n), the later O(n). The base of the logarithm would be fairly big (the tree very wide) but that is just a constant and disapears in O(). PS: I don't know if reiserfs is doing it this way, just wanted to show how it could be done. Right. I have no doubt it can be done. It's been done for decades in non-Unixoid systems. I'm asking about Debian, here and now, under default installation options, not under what can be done in the abstract. It's doing something better than a plain list. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Wednesday 11 May 2005 05:47, Goswin von Brederlow [EMAIL PROTECTED] wrote: / on LVM allows for snapshot backups which are the most convenient method of backup. Except that the kernel freezes the device because the DM lock and device node updating deadlock. Might work with udev or devfs, haven't tried that. It works in Fedora and RHEL4, must be because of udev and tmpfs mounted on /dev. In that case we should make etch use tmpfs for /dev by default. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
hoi :) On Mon, May 09, 2005 at 03:45:32PM +1000, Russell Coker wrote: Should we change some of these to /usr/libexec? well, it would be against the FHS, I think. The BSDs use libexec but I don't really see a good reason why it exists. -- Martin Waitz signature.asc Description: Digital signature
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes: It seems that Red Hat has a lot of programs under /usr/libexec that are under /usr/lib in Debian. One example is /usr/lib/postfix vs /usr/libexec/postfix. It seems to me that /usr/libexec is a better name for such things, I disagree. Why is it important to distinguish between shared libraries and internal binaries (i.e. programs not supposed to be called directly by a user)? In principle, there could be files which can be used as both a shared library and an internal binary. Where would you put such files? Should we change some of these to /usr/libexec? I don't think so. Both FHS 2.1 (referenced by the current Policy) and FHS 2.3 (the latest FHS version) mandate /usr/lib (or a subdirectory) for internal binaries. Martin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes: It seems that Red Hat has a lot of programs under /usr/libexec that are under /usr/lib in Debian. One example is /usr/lib/postfix vs /usr/libexec/postfix. It seems to me that /usr/libexec is a better name for such things, and having the same directory names used across distributions provides real benefits The File Hierarchy Standard[0] uses /usr/lib for these kinds of things and LSB 2.1 explicitly referes to FHS for how the file hierarchy should be laied out. So unless Red HAt is pushing for the relevant standards to be changed I believe we should stick to the standards just so the same directory names may be used acress distributions. 0) http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLIBLIBRARIESFORPROGRAMMINGANDPA /usr/lib includes object files, libraries, and internal binaries that are not intended to be executed directly by users or shell scripts. -- Peter Makholm |According to the hacker ethic, the meaning of life [EMAIL PROTECTED] |is not Friday, but it is not Sunday either http://hacking.dk | -- Pekka Himanen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Martin Waitz [EMAIL PROTECTED] writes: Should we change some of these to /usr/libexec? well, it would be against the FHS, I think. The BSDs use libexec but I don't really see a good reason why it exists. GNU project stuff also uses libexec (by default; I don't know if that location gets overridden in debian builds). E.g., if you build your own emacs, notice it installs various helper programs in /usr/local/libexec/emacs/... I don't know if there's an argument for it other than clarity and warm fuzzies. [I personally think that if a good idea is against the FHS, the right answer is to change the FHS.] -Miles -- Come now, if we were really planning to harm you, would we be waiting here, beside the path, in the very darkest part of the forest? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
[Martin Waitz] The BSDs use libexec but I don't really see a good reason why it exists. Well, the reason */libexec exists is to avoid overloading the meaning of */lib to include things other than libraries. Just as /sbin was invented (way back in the day) to stop overloading /etc with things other than config files. I agree, though, that unless the FHS decides to adopt libexec, there's little point in Debian doing so. signature.asc Description: Digital signature
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes: Should we change some of these to /usr/libexec? Debian strives to follow the FHS (http://www.pathname.com/fhs), and this standard does not include /usr/libexec. See also http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=146023, which mentions the use of /usr/lib/package for storing support binaries out of the way. -- Stig Sandbeck Mathisen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Miles Bader [EMAIL PROTECTED] writes: I don't know if there's an argument for it other than clarity and warm fuzzies. Not that there is anything wrong with warm fuzzies. I prefer that to a file hierarchy layout that gives me the chills. [I personally think that if a good idea is against the FHS, the right answer is to change the FHS.] Since Debian's file system layout is based on FHS, that seems to be the correct way to change things. -- Stig Sandbeck Mathisen -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Martin Waitz [EMAIL PROTECTED] writes: The BSDs use libexec but I don't really see a good reason why it exists. It reduces search times in libraries, which is important. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Martin Dickopp [EMAIL PROTECTED] writes: It seems that Red Hat has a lot of programs under /usr/libexec that are under /usr/lib in Debian. One example is /usr/lib/postfix vs /usr/libexec/postfix. It seems to me that /usr/libexec is a better name for such things, I disagree. Why is it important to distinguish between shared libraries and internal binaries (i.e. programs not supposed to be called directly by a user)? lib is the place where ld searches for libraries. Linking is not speedy, and a non-trivial amount of the time is spent slogging through /usr/lib looking for the libraries wanted. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Monday 09 May 2005 17:17, Martin Dickopp [EMAIL PROTECTED] wrote: In principle, there could be files which can be used as both a shared library and an internal binary. Where would you put such files? Anything that's a shared object has to be in a directory that ldconfig knows about. There's nothing preventing a shared object in /usr/lib from being directly executed, there's a shared object in /lib that's regularly executed directly. -- http://www.coker.com.au/selinux/ My NSA Security Enhanced Linux packages http://www.coker.com.au/bonnie++/ Bonnie++ hard drive benchmark http://www.coker.com.au/postal/Postal SMTP/POP benchmark http://www.coker.com.au/~russell/ My home page -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Russell Coker [EMAIL PROTECTED] writes: It seems that Red Hat has a lot of programs under /usr/libexec that are under /usr/lib in Debian. One example is /usr/lib/postfix vs /usr/libexec/postfix. It seems to me that /usr/libexec is a better name for such things, and having the same directory names used across distributions provides real benefits (copying config files and binaries from other distributions when a bug stops a server working and it's REALLY important to get it fixed fast). Should we change some of these to /usr/libexec? That would be /usr/libexec/arch-os/postfix vs /usr/lib/arch-os/postfix then. If you consider any change then please include the multiarch changes at the same time. No point doing 2 transitions for etch. MfG Goswin PS: ln -s lib /usr/libexec -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Mon, May 09, 2005 at 08:39:10AM -0700, Thomas Bushnell BSG wrote: Martin Dickopp [EMAIL PROTECTED] writes: It seems that Red Hat has a lot of programs under /usr/libexec that are under /usr/lib in Debian. One example is /usr/lib/postfix vs /usr/libexec/postfix. It seems to me that /usr/libexec is a better name for such things, I disagree. Why is it important to distinguish between shared libraries and internal binaries (i.e. programs not supposed to be called directly by a user)? lib is the place where ld searches for libraries. Linking is not speedy, and a non-trivial amount of the time is spent slogging through /usr/lib looking for the libraries wanted. The number of directory entries in /usr/lib should not make any difference to a modern GNU linker on a modern filesystem, unless you have thousands or millions of them. -- Daniel Jacobowitz CodeSourcery, LLC -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG [EMAIL PROTECTED] writes: Martin Dickopp [EMAIL PROTECTED] writes: It seems that Red Hat has a lot of programs under /usr/libexec that are under /usr/lib in Debian. One example is /usr/lib/postfix vs /usr/libexec/postfix. It seems to me that /usr/libexec is a better name for such things, I disagree. Why is it important to distinguish between shared libraries and internal binaries (i.e. programs not supposed to be called directly by a user)? lib is the place where ld searches for libraries. Linking is not speedy, and a non-trivial amount of the time is spent slogging through /usr/lib looking for the libraries wanted. Is this an issue even with modern filesystems (like ext3 with the dir_index feature turned on) for which the kernel can find a directory entry faster than in O(n) time? Martin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
hoi :) On Mon, May 09, 2005 at 08:38:02AM -0700, Thomas Bushnell BSG wrote: The BSDs use libexec but I don't really see a good reason why it exists. It reduces search times in libraries, which is important. well, /usr/lib is not _that_ crowded. Any sane filesystem should handle that many files/subdirs. And they are very likely in RAM already. so, do you have any numbers? -- Martin Waitz signature.asc Description: Digital signature
Re: /usr/lib vs /usr/libexec
Daniel Jacobowitz [EMAIL PROTECTED] writes: The number of directory entries in /usr/lib should not make any difference to a modern GNU linker on a modern filesystem, unless you have thousands or millions of them. Why? Is there magic now? -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Mon, May 09, 2005 at 02:21:35PM -0700, Thomas Bushnell BSG wrote: Daniel Jacobowitz [EMAIL PROTECTED] writes: The number of directory entries in /usr/lib should not make any difference to a modern GNU linker on a modern filesystem, unless you have thousands or millions of them. Why? Is there magic now? What magic do you need? Most filesystems can open a file without doing an O(n) lookup, especially from the dentry cache. -- Daniel Jacobowitz CodeSourcery, LLC -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Daniel Jacobowitz [EMAIL PROTECTED] writes: On Mon, May 09, 2005 at 02:21:35PM -0700, Thomas Bushnell BSG wrote: Daniel Jacobowitz [EMAIL PROTECTED] writes: The number of directory entries in /usr/lib should not make any difference to a modern GNU linker on a modern filesystem, unless you have thousands or millions of them. Why? Is there magic now? What magic do you need? Most filesystems can open a file without doing an O(n) lookup, especially from the dentry cache. Huh? The dentry cache turns ls into O(n) instead of O(n^2), but that doesn't mean that searching every item in the directory isn't still necessary, unless the directory is hashed or stored in as a tree. I suppose the real question is why have a directory tree at all? If there is a reason to separate /usr from / (which so many people think there is, though I don't understand why, since it has no semantic significance at all), why separate /lib from /etc? Why not put every file in one place, actually? We could just have /, and /bin, put every package in / under its own name, and be done with it. Surely the reason who have these different directories is to make logical distinctions, keeping different kinds of things in different directories. If the argument for combining libexec and lib is that it does no harm, then I see why we should not put *everything* into lib. It would make it simpler. So the question is: why is it useful to make some distinctions (including non-sensical ones like /usr vs. /) but not this one? Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
On Mon, May 09, 2005 at 02:33:32PM -0700, Thomas Bushnell BSG wrote: Daniel Jacobowitz [EMAIL PROTECTED] writes: On Mon, May 09, 2005 at 02:21:35PM -0700, Thomas Bushnell BSG wrote: Daniel Jacobowitz [EMAIL PROTECTED] writes: The number of directory entries in /usr/lib should not make any difference to a modern GNU linker on a modern filesystem, unless you have thousands or millions of them. Why? Is there magic now? What magic do you need? Most filesystems can open a file without doing an O(n) lookup, especially from the dentry cache. Huh? The dentry cache turns ls into O(n) instead of O(n^2), but that doesn't mean that searching every item in the directory isn't still necessary, unless the directory is hashed or stored in as a tree. You asked why the GNU linker, which does not need to be 'ls' and does not need to look at the list of files in any directory, scaled well with the size of the directory. That's the question I answered. Surely the reason who have these different directories is to make logical distinctions, keeping different kinds of things in different directories. If the argument for combining libexec and lib is that it does no harm, then I see why we should not put *everything* into lib. It would make it simpler. So the question is: why is it useful to make some distinctions (including non-sensical ones like /usr vs. /) but not this one? That's a completely different question... which I don't think I need to answer. I was responding to your invalid criticism of the linker. -- Daniel Jacobowitz CodeSourcery, LLC -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Daniel Jacobowitz [EMAIL PROTECTED] writes: You asked why the GNU linker, which does not need to be 'ls' and does not need to look at the list of files in any directory, scaled well with the size of the directory. That's the question I answered. How does ld determine that -latoheun will fail, other than by listing the directory O(n)? How does ld find -lfoo, without searching through, on average, half the entries? Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
ma, 2005-05-09 kello 14:39 -0700, Thomas Bushnell BSG kirjoitti: Daniel Jacobowitz [EMAIL PROTECTED] writes: You asked why the GNU linker, which does not need to be 'ls' and does not need to look at the list of files in any directory, scaled well with the size of the directory. That's the question I answered. How does ld determine that -latoheun will fail, other than by listing the directory O(n)? How does ld find -lfoo, without searching through, on average, half the entries? I may be completely wrong here, but as far as I understand, ld turns -lfoo into /usr/lib/libfoo.a and then uses that if it can find it. It might look into some other directories as well, and it might fill in foo into some other patterns than lib%s.a, but basically that is it. It does not scan the /usr/lib directory, it merely looks up a filename it knows already. With modern filesystems, the kernel also does not need to read through the entires /usr/lib directory listing: modern filesystems user B-trees or other ways to speed up filename lookups. O(log n), that is, or even approximately O(1) if a good hash is used. I suspect this is what Daniel tried to say: that filename lookups aren't O(n) anymore. This means that the performance reason for keeping /usr/lib small is gone. This, of course, has no bearing on whether /usr/libexec should exist or not for other reasons. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Lars Wirzenius [EMAIL PROTECTED] writes: I may be completely wrong here, but as far as I understand, ld turns -lfoo into /usr/lib/libfoo.a and then uses that if it can find it. It might look into some other directories as well, and it might fill in foo into some other patterns than lib%s.a, but basically that is it. It does not scan the /usr/lib directory, it merely looks up a filename it knows already. Right, and open is O(n) on just about every system. If that's not true on ext2, then that's good news, and I'm surprised. With modern filesystems, the kernel also does not need to read through the entires /usr/lib directory listing: modern filesystems user B-trees or other ways to speed up filename lookups. O(log n), that is, or even approximately O(1) if a good hash is used. Actually, even systems as old as ITS used better than O(n) directories. It's Unix that has historically stunk. It's not a modern/old thing, it's a Just Do It thing. Which Linux filesystems have better than O(n) performance on open? Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG [EMAIL PROTECTED] writes: If there is a reason to separate /usr from / (which so many people think there is, though I don't understand why, since it has no semantic significance at all), why separate /lib from /etc? I don't see a semantic difference between /bin and /usr/bin (or /lib and /usr/lib). IMHO, the only reason for /bin and /lib is that some programs and libraries need to be available before is /usr is mounted. Surely the reason who have these different directories is to make logical distinctions, keeping different kinds of things in different directories. If the argument for combining libexec and lib is that it does no harm, then I see why we should not put *everything* into lib. It would make it simpler. That wasn't my argument. My argument is that I don't consider shared libraries and internal executables different kinds of things. They are both binaries loaded and executed by a program. If there is a _technical_ necessity to separate them (like directory search times), this would be similar to /bin vs /usr/bin, which are also separate for technical, but not semantic reasons. Martin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas, please read http://www.nl.debian.org/doc/developers-reference/ch-resources.en.html#s-mailing-lists-rules about not sending Cc's unless people explicitly ask to be copied. (Mail-Followup-To is non-standard and badly supported, and also unnecessary. Any decent mail user agent can deal with the default case of not doing a Cc, without help from M-F-T. Ctrl-L in Evolution.) ma, 2005-05-09 kello 14:53 -0700, Thomas Bushnell BSG kirjoitti: Which Linux filesystems have better than O(n) performance on open? I haven't studied all of them so I won't speak authoritatively. mke2fs(8) documents one. The option was just mentioned by Martin Dickopp earlier in this thread in the message archived at http://lists.debian.org/debian-devel/2005/05/msg00443.html . (As far as I care, this topic is dealt with. We can move on to other misunderstandings now.) -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Martin Dickopp [EMAIL PROTECTED] writes: Thomas Bushnell BSG [EMAIL PROTECTED] writes: If there is a reason to separate /usr from / (which so many people think there is, though I don't understand why, since it has no semantic significance at all), why separate /lib from /etc? I don't see a semantic difference between /bin and /usr/bin (or /lib and /usr/lib). IMHO, the only reason for /bin and /lib is that some programs and libraries need to be available before is /usr is mounted. That doesn't make sense. If you get rid of the /usr vs / distinction, then there is no before /usr is mounted. That wasn't my argument. My argument is that I don't consider shared libraries and internal executables different kinds of things. They are both binaries loaded and executed by a program. Sure, and documentation and libraries are both files opened by programs. The difference is that libraries are also generic things that are shared by many programs, and searched by the linker, whereas executables are not. In fact, a library is loaded and executed by only two programs (ld and ld.so) in the normal scheme of things. Thomas -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
Re: /usr/lib vs /usr/libexec
Thomas Bushnell BSG [EMAIL PROTECTED] writes: Lars Wirzenius [EMAIL PROTECTED] writes: I may be completely wrong here, but as far as I understand, ld turns -lfoo into /usr/lib/libfoo.a and then uses that if it can find it. It might look into some other directories as well, and it might fill in foo into some other patterns than lib%s.a, but basically that is it. It does not scan the /usr/lib directory, it merely looks up a filename it knows already. Right, and open is O(n) on just about every system. If that's not true on ext2, then that's good news, and I'm surprised. With modern filesystems, the kernel also does not need to read through the entires /usr/lib directory listing: modern filesystems user B-trees or other ways to speed up filename lookups. O(log n), that is, or even approximately O(1) if a good hash is used. Actually, even systems as old as ITS used better than O(n) directories. It's Unix that has historically stunk. It's not a modern/old thing, it's a Just Do It thing. Which Linux filesystems have better than O(n) performance on open? Thomas Which doesn't? Minix maybe. Even ext2/3 has hashes for dir if you format it that way. MfG Goswin -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
