Re: layout of filesystems on OpenBSD
Thanks a lot to all that replied either in private email (Francisco), or in the misc mailing list (Joachim, Robert, Will, Stuart, Peter, Damian, Marc and Matthias). I will try to reply to all in the thread right now. As I am not a member of this mailing list I have not received a copy of the emails. I will try to set up nmh to automatically classify email from two or three mailing lists to receive these lists as soon as possible. Sorry for the delay, but the first message was stalled because it included certain words that make it seem as spam (a word that I will not repeat here, because I do not want this message to be stalled again!) I agree with Francisco about the use of swap (wd0b) for both memory filesystems (/tmp and /var/run). Indeed, it is the way it is used in fstab(5), but only on a single mfs. Both Francisco, Damian and Joachim have observed that /var can grow when using databases or the system is a large mail or printer server. I should have observed it as I sometimes work for the Department of Geological Sciences, and they send huge files to the plotters. It is a very good point. Joachim is right when he writes that /home does not need to be too large. Currently it can grow up to, we say, 1 GB (e.g., when making an ISO image to be dumped to a CD-ROM) but large /home filesystems are a waste of space. Indeed, /usr/X11R6 is part of /usr. Even if some packages install files on it (e.g., OpenMotif), X11R6 does not grow a lot and can be considered static. I guess that I made it a separate filesystem because I do not see X.Org as a part of OpenBSD (another reason is that /usr/openwin was a separate filesystem on Solaris, and I am accustomed to this layout). A too large /usr/ports is something that should be avoided for me, as I really use binary packages. I just install /usr/ports because sometimes I needed to build packages from source (it happened only a few times.) Certainly a 24 GB /usr/ports is something I will probably avoid. On this matter, I feel that Damian is NOT doing something improperly when /usr/ports grows. It would be much better having these object files in /usr/obj or, at least, not lying around in /usr/ports. Building ports from source makes /usr/ports a real mess in only some days. I am sure, there are good technical reasons for not making the object and binary files in a separate directory: patches to source code tarballs would become unmanageable if third party applications need to be changed to support this feature. The advice on CCD is an excellent one (thanks Joachim!). I would suggest leaving some unused space on the disk drives too. It is not the same as Veritas Volume Manager, but can certainly help when a filesytem runs out of space. Certainly using CCD or RAIDframe is much better. I agree with most people in this thread. The setup I proposed is too complex. I like it and works fine for me, but it is certainly a nightmare when a new application changes the space requirements (and a database is a good example). Indeed, I do not run databases right now... but I should do it. Databases are not only useful as support tools for spam filters and intrusion detection systems, but excellent tools for our daily work. I will play with a database system as soon as I have some time. Again, thanks to all for the excellent advices on this matter. Cheers, Igor. --Boundary_(ID_b6vZWPMJJsiMXjf61E2EHw)--
Re: layout of filesystems on OpenBSD
Joachim Schipper wrote: On Fri, Nov 10, 2006 at 04:10:54PM -0600, Damian Wiest wrote: I've had the misfortune of running AIX for a short time and am aware of how Veritas Volume Manager encapsulates disks, but what's the equivalent in OpenBSD? One benefit of those systems is that they allow you to resize filesystems on the fly, which is helpful if you're not sure how much space you're going to need. I sometimes end up performing two installs. The first one lets me see how much space the OS distribution is likely to occupy and I then use those numbers when I redo the install. If you want to do the same in OpenBSD, allocate the maximum number of partitions and run ccd devices over appropriate subsets of the partitions. (Note growfs(8); there is no shrinkfs, though.) If this is not granular enough, add one ccd device per partition, and parition that one again [1]. This setup would allow dividing a disk in 15x15 = 255 not necessarily equally big slices, which Should Be Enough ^^^ oops For Everyone. (If not, repeat.) Interesing thought. Fragmented filesystems (not just files). Probably horrible in many ways and easy to make mistakes, but still interesting. But why not just start with a reasonably sized partition per ccd and then add additional partitions (of varied sizes) when needed? I see no need to predefine the partitions. Then, should you run out of partitions, you could make the last one (p?) use the rest of the disk and split it using the technique mentioned above. You'll probably want to build a custom kernel to increase the number of ccd devices. And wash your hands. Joachim [1] This is possible, but I haven't tested either performance or stability of this setup.
Re: layout of filesystems on OpenBSD
On Wed, Nov 15, 2006 at 12:06:20PM +0100, Alexander Hall wrote: Joachim Schipper wrote: On Fri, Nov 10, 2006 at 04:10:54PM -0600, Damian Wiest wrote: I've had the misfortune of running AIX for a short time and am aware of how Veritas Volume Manager encapsulates disks, but what's the equivalent in OpenBSD? One benefit of those systems is that they allow you to resize filesystems on the fly, which is helpful if you're not sure how much space you're going to need. I sometimes end up performing two installs. The first one lets me see how much space the OS distribution is likely to occupy and I then use those numbers when I redo the install. If you want to do the same in OpenBSD, allocate the maximum number of partitions and run ccd devices over appropriate subsets of the partitions. (Note growfs(8); there is no shrinkfs, though.) If this is not granular enough, add one ccd device per partition, and parition that one again [1]. This setup would allow dividing a disk in 15x15 = 255 not necessarily equally big slices, which Should Be Enough ^^^ oops Yet another mathematician that can't count. Oh well, I'm doing algebra anyway - counting is for engineers. ;-) For Everyone. (If not, repeat.) Interesing thought. Fragmented filesystems (not just files). Probably horrible in many ways and easy to make mistakes, but still interesting. But why not just start with a reasonably sized partition per ccd and then add additional partitions (of varied sizes) when needed? I see no need to predefine the partitions. Then, should you run out of partitions, you could make the last one (p?) use the rest of the disk and split it using the technique mentioned above. This would, indeed, work. Which is not to say it's necessarily a good idea, of course... though I must admit that, outside of being fugly and having some minor overhead, I find it hard to spot the downsides. Joachim
Re: layout of filesystems on OpenBSD
On Fri, Nov 10, 2006 at 04:10:54PM -0600, Damian Wiest wrote: I've had the misfortune of running AIX for a short time and am aware of how Veritas Volume Manager encapsulates disks, but what's the equivalent in OpenBSD? One benefit of those systems is that they allow you to resize filesystems on the fly, which is helpful if you're not sure how much space you're going to need. I sometimes end up performing two installs. The first one lets me see how much space the OS distribution is likely to occupy and I then use those numbers when I redo the install. If you want to do the same in OpenBSD, allocate the maximum number of partitions and run ccd devices over appropriate subsets of the partitions. (Note growfs(8); there is no shrinkfs, though.) If this is not granular enough, add one ccd device per partition, and parition that one again [1]. This setup would allow dividing a disk in 15x15 = 255 not necessarily equally big slices, which Should Be Enough For Everyone. (If not, repeat.) You'll probably want to build a custom kernel to increase the number of ccd devices. And wash your hands. Joachim [1] This is possible, but I haven't tested either performance or stability of this setup.
Re: layout of filesystems on OpenBSD
You wrote: try to outline the output of /etc/fstab, disklabel and df on a table: - Mounted on Type Sizebsize fsize - -- -- - - a / ffs64 MB 16384 2048 b swapswap 512 MB N/AN/A c all the disk d /altrootffs64 MB 16384 2048 e /varffs25 MB 16384 2048 f /usrffs 768 MB 16384 2048 g /var/logffs16 MB 16384 2048 h /var/mail ffs 128 MB 16384 2048 /var/runmfs 4 MB i /var/tmpffs24 MB 16384 2048 j /var/wwwffs32 MB 16384 2048 k /usr/X11R6 ffs 256 MB 16384 2048 l /usr/local ffs 2048 MB 16384 2048 m /usr/objffs 896 MB 16384 2048 n /usr/ports ffs 512 MB 4096 512 o /usr/srcffs 1024 MB 8192 1024 p /home ffs + MB 16384 2048 /tmpmfs64 MB to me, this just looks like a horrible mess. I have never understood why people should be so keen on creating thousands of microscopic filesystems. For me, the advantage of being able to have several classes of filesystem content all take advantage of the available free space of a filesystem/partition far outweighs any need to segregate classes of filesystem content into separate partitions. For example, how could /usr/X11R6 possibly represent a threat to eat all the space is /usr? X11R6 content is static. (yes, I know, software packages put stuff there, but for the purposes of this discussion it's static). Arguments can presumably be made for /var/www, and /var/mail, /home, /usr/src, and /tmp, but the rest just seems like a waste of energy. I imagine I'd do: / /var /usr and as necessary /var/mail /var/www /usr/src /home /tmp Rob Urban
Re: layout of filesystems on OpenBSD
On Fri, Nov 10, 2006 at 05:55:27PM +0100, Igor Sobrado wrote: Hello. I have some questions related with the layout of the filesystems on OpenBSD and hope someone will be able to help me. I must start saying that I never had problems with this setup, but I certainly would appreciate some advice on it. To make things clear, I will try to outline the output of /etc/fstab, disklabel and df on a table: - Mounted on Type Sizebsize fsize - -- -- - - a / ffs64 MB 16384 2048 b swapswap 512 MB N/AN/A c all the disk d /altrootffs64 MB 16384 2048 e /varffs25 MB 16384 2048 f /usrffs 768 MB 16384 2048 g /var/logffs16 MB 16384 2048 h /var/mail ffs 128 MB 16384 2048 /var/runmfs 4 MB i /var/tmpffs24 MB 16384 2048 j /var/wwwffs32 MB 16384 2048 k /usr/X11R6 ffs 256 MB 16384 2048 l /usr/local ffs 2048 MB 16384 2048 m /usr/objffs 896 MB 16384 2048 n /usr/ports ffs 512 MB 4096 512 o /usr/srcffs 1024 MB 8192 1024 p /home ffs + MB 16384 2048 /tmpmfs64 MB It is, approximately, the layout of all my systems. Not all computers have the same filesystems structure. For example, servers (currently a Soekris net4801 and a Dell PowerEdge 350) do not run X, nor have /usr/X11R6 on it. I do not extract the contents of XF4.tar.gz on any of my systems (workstations or servers) either. There are no filesystems with more than 50% space used and, to avoid running out of inodes, /usr/ports and /usr/src have no default block and fragment sizes either. My questions are: 1. /var/run and /tmp are memory filesystems right now. (/tmp does not need to survive to reboots and the contents of /var/run are better going away when the machines are restarted.) Both have entries on /etc/fstab with the SAME BLOCK SPECIAL DEVICE (/dev/wd0b). Is it a good practice? No idea; I suppose it doesn't really matter. I have /tmp on disk. I can see why you'd want to do it that way, but according to http://www.openbsd.org/faq/faq4.html#Partitioning, you'd want it to be larger when compiling software. 2. Are the sizes of the filesystems right ones? I am thinking on - /var (on the installation booklet provided with the OS it is recommended a size of 200 MB for this filesystem, I *never* found a /var filesystem larger than 10-20 MB). Can it really be so large? I suppose, but it does need something to actually get it that large. Large (mail|print) spools and log files come to mind. - /usr/obj (what is the maximum size required for making a full operating system rebuild?) I will certainly not do a full rebuild, only apply some patches, but it is better to know. Today, storage space is cheap and we can afford a 900 MB filesystem... my current estimation comes from some make world on NetBSD... it would be nice to know the amount of space that is required. Can we use the default block/fragment sizes on it? /usr/obj is 936M on my -current i386 box (and includes the result of 'make release'). Better up that estimate a little. - /usr/ports (is 512 MB enough for it?) I usually stay at binary packages, but sometimes I need to build software from source and would like to know the recommended space for this filesystem. /usr/ports is only 160M on my box. This should not include any build stuff. Depending on what you want to build, 512MB can be plenty or far too little (OpenOffice comes to mind...) I know that some filesystems (e.g., /var/mail and /var/www) are too small for most uses, but the size of these filesystems is very easy to know once we know how a computer will be used. On these machines I do not need a large /var/mail (only one of these machines receive standard email -not system reports- and it is stored in /home as soon as I run the inc(1) command in nmh.) Others (e.g., /usr, /usr/X11R6, /, /altroot -that is a dump of the root filesystem automatically done-) do not grow at all. 3. Any though about the filesystem layout? I know that it is certainly complex, but worked nicely for me in the last years. All these filesystems (except /var/tmp) are recommended in the documentation and, as I said, I do not remember any of these filesystem growing up to 60% in either space or inodes usage in the last years... but I never did a full rebuild of the operating system nor installed large packages from source code (mwm is soo nice, and it comes in the official CD-ROMs!) It's more complex than it needs to be; and I find
Re: layout of filesystems on OpenBSD
Robert Urban wrote: to me, this just looks like a horrible mess. I have never understood why people should be so keen on creating thousands of microscopic filesystems. For me, the advantage of being able to have several classes of filesystem content all take advantage of the available free space of a filesystem/partition far outweighs any need to segregate classes of filesystem content into separate partitions. I agree that it looks like overcomplication. I only create partitions when I will be using different mount options, for example noexec and nosuid stuff.
Re: layout of filesystems on OpenBSD
Robert Urban wrote: to me, this just looks like a horrible mess. I have never understood why people should be so keen on creating thousands of microscopic filesystems. For me, the advantage of being able to have several classes of filesystem content all take advantage of the available free space of a filesystem/partition far outweighs any need to segregate classes of filesystem content into separate partitions. For example, how could /usr/X11R6 possibly represent a threat to eat all the space is /usr? X11R6 content is static. (yes, I know, software packages put stuff there, but for the purposes of this discussion it's static). Arguments can presumably be made for /var/www, and /var/mail, /home, /usr/src, and /tmp, but the rest just seems like a waste of energy. I imagine I'd do: / /var /usr and as necessary /var/mail /var/www /usr/src /home /tmp Rob Urban I have to agree, except I would add a /var/log to the as necessary (and make it pretty big) as I often deal with firewalls and it's nice (I think) to limit the logs ability to totally run amuck Although it isn't strictly required since /var is in it's own partition. stuart
Re: layout of filesystems on OpenBSD
Joachim Schipper wrote: On Fri, Nov 10, 2006 at 05:55:27PM +0100, Igor Sobrado wrote: - /usr/ports (is 512 MB enough for it?) I usually stay at binary packages, but sometimes I need to build software from source and would like to know the recommended space for this filesystem. /usr/ports is only 160M on my box. This should not include any build stuff. Depending on what you want to build, 512MB can be plenty or far too little (OpenOffice comes to mind...) Have at least 24GB free space to build oo
Re: layout of filesystems on OpenBSD
On Fri, Nov 10, 2006 at 05:55:27PM +0100, Igor Sobrado wrote: Hello. [snip] 2. Are the sizes of the filesystems right ones? I am thinking on - /var (on the installation booklet provided with the OS it is recommended a size of 200 MB for this filesystem, I *never* found a /var filesystem larger than 10-20 MB). Can it really be so large? Don't some of the database ports install into /var? It seems to me that you're not running one, so it's probably not an issue. I've made the mistake before of only allocating a gigabyte of space to /var and then had someone install a database that eats up close to 80% of that. Needless to say, that can cause some problems if users aren't paying attention and fill the filesystem (ISTR some database ports keeping log files in the same filesystem as the data files by default). If you have any applications writing logs to /var, you may want to take a look at how much data they're writing in a given amount of time. I've seen systems where /var is large enough most of the time, but as soon as someone turns on the debugging flags for an application, /var is filled extremely quickly. Also, savecore will write crash dumps to /var/core by default, so you'll need to allocate at least as much space as you have RAM if you want to capture that information. [snip] - /usr/ports (is 512 MB enough for it?) I usually stay at binary packages, but sometimes I need to build software from source and would like to know the recommended space for this filesystem. I'm most likely doing something improperly, but if you're not careful about cleaning up after yourself, you can easily leave object files lying around in /usr/ports and run out of space during a large build. I don't separate out /usr/ports, but I typically allocate at least 10 gigabytes to /usr. Then again, that's for a workstation with a lot of dev. tools installed. [snip] 3. Any though about the filesystem layout? I know that it is certainly complex, but worked nicely for me in the last years. All these filesystems (except /var/tmp) are recommended in the documentation and, as I said, I do not remember any of these filesystem growing up to 60% in either space or inodes usage in the last years... but I never did a full rebuild of the operating system nor installed large packages from source code (mwm is soo nice, and it comes in the official CD-ROMs!) It seems to me that you have a relatively large number of filesystems present. This may cause you problems in the future if you're using that as a standard layout in a heterogeneous environment. I typically create only five filesystems: /, /usr, /var, /home and /tmp. ...I do not need a large /usr/local either. I just install mwm, firefox, nmh and a LaTeX distribution. Most utilities I need come with the operating system (nearly all, and for making figures I like MetaPost that comes with the LaTeX distribution.) No problem at all with these filesystems: its size is highly site dependent, but very easy to work out for me. As I said, I usually do not install third party applications. Thanks a lot for reading this post. Igor. I've had the misfortune of running AIX for a short time and am aware of how Veritas Volume Manager encapsulates disks, but what's the equivalent in OpenBSD? One benefit of those systems is that they allow you to resize filesystems on the fly, which is helpful if you're not sure how much space you're going to need. I sometimes end up performing two installs. The first one lets me see how much space the OS distribution is likely to occupy and I then use those numbers when I redo the install. HTH. -Damian
Re: layout of filesystems on OpenBSD
On Fri, Nov 10, 2006 at 07:32:38PM +0100, Robert Urban wrote: For example, how could /usr/X11R6 possibly represent a threat to eat all the space is /usr? X11R6 content is static. (yes, I know, software packages put stuff there, but for the purposes of this discussion it's static). Actually, no, on OpenBSD, software packages do not put stuff there.
Re: layout of filesystems on OpenBSD
On Sat, Nov 11, 2006 at 03:52:36AM +0100, Marc Espie wrote: space is /usr? X11R6 content is static. (yes, I know, software packages put stuff there, but for the purposes of this discussion it's static). Actually, no, on OpenBSD, software packages do not put stuff there. Opemmotif does (Motif.rules and Motif.tmpl). Ciao, Ki i'm still sleeping li -- I have to agree, Gentoo's install docs are some of the best out there and will allow just about anybody to install OpenBSD. -- Derek Tracy on misc@openbsd.org
