Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Le mercredi 13 avril 2011 à 10:51 +0100, Philip Hands a écrit : Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=245465 -- .''`. Josselin Mouette : :' : `. `' “If you behave this way because you are blackmailed by someone, `-[…] I will see what I can do for you.” -- Jörg Schilling signature.asc Description: This is a digitally signed message part
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Roger Leigh rle...@codelibre.net writes: On Fri, Apr 15, 2011 at 04:41:56PM +0200, Goswin von Brederlow wrote: Roger Leigh rle...@codelibre.net writes: If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). I hope you check the fstab first. If there is a entry for a non tmpfs /tmp filesystem then that should be used. I'm assuming you do but just to be sure... No, we don't check. It's up to the admin to configure their system properly. If there is an entry in in fstab, it'll be mounted on top of the tmpfs, so the system will be configured the way they asked, but there will be a hidden tmpfs mount. But they would have explicitly needed to set RAMTMP=yes to get into this situation. For new installs, where the default /etc/default/rcS files does set RAMTMP=yes by default, the fstab file will not yet contain any user-specific mounts. If they do want to manuall mount something on /tmp, then they simply set RAMTMP=no. Note this behaviour is exactly the same as existing practice for /dev/shm, /var/run and /var/lock. Then I don't get your 'is /not/ the default (except when root is read-only (ro) in fstab)'. To me that reads like you will mount a tmpfs on /tmp if root is read-only even if RAMTMP is not set. Which is wrong if the system has a /tmp filesystem in /etc/fstab. I already have a tmpfs for /tmp in my /etc/fstab and my root is read-only. Does that then mean I do get 2 tmpfs mounted for /tmp, one over the other? Also mount -a (in mountall.sh) fails, and therefore the whole boot, if a mountpoint already has something else mounted. If you unconditionally mount a tmpfs on /tmp if root is read-only then you just made systems unbootable that have /tmp in fstab. You do not get the behaviour you describe above. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87oc464pie.fsf@frosties.localnet
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Sat, Apr 16, 2011 at 09:35:53AM +0200, Goswin von Brederlow wrote: Roger Leigh rle...@codelibre.net writes: On Fri, Apr 15, 2011 at 04:41:56PM +0200, Goswin von Brederlow wrote: Roger Leigh rle...@codelibre.net writes: If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). I hope you check the fstab first. If there is a entry for a non tmpfs /tmp filesystem then that should be used. I'm assuming you do but just to be sure... No, we don't check. It's up to the admin to configure their system properly. If there is an entry in in fstab, it'll be mounted on top of the tmpfs, so the system will be configured the way they asked, but there will be a hidden tmpfs mount. But they would have explicitly needed to set RAMTMP=yes to get into this situation. For new installs, where the default /etc/default/rcS files does set RAMTMP=yes by default, the fstab file will not yet contain any user-specific mounts. If they do want to manuall mount something on /tmp, then they simply set RAMTMP=no. Note this behaviour is exactly the same as existing practice for /dev/shm, /var/run and /var/lock. Then I don't get your 'is /not/ the default (except when root is read-only (ro) in fstab)'. To me that reads like you will mount a tmpfs on /tmp if root is read-only even if RAMTMP is not set. Which is wrong if the system has a /tmp filesystem in /etc/fstab. This is a good point. I've added the ability to detect if a filesystem will be mounted, and skip the tmpfs mount on /tmp if an entry for /tmp exists in fstab (will_mount in /lib/init/mount-functions.sh) Also mount -a (in mountall.sh) fails, and therefore the whole boot, if a mountpoint already has something else mounted. If you unconditionally mount a tmpfs on /tmp if root is read-only then you just made systems unbootable that have /tmp in fstab. This is not correct. Have you actually tried it? I have, and other than the cosmetic issue of having a real filesystem mounted over the top of the tmpfs, the system is entirely functional, and boots error free. And with the above change, even this cosmetic issue is gone. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Sat, 16 Apr 2011 09:17:04 +0200, Josselin Mouette j...@debian.org wrote: Le mercredi 13 avril 2011 à 10:51 +0100, Philip Hands a écrit : Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=245465 Ah, good point -- so rather than just chatting about this I should probably get off my arse and start making some sort of contribution to d-i. I'll test your patch. Cheers, Phil. -- |)| Philip Hands [+44 (0)20 8530 9560]http://www.hands.com/ |-| HANDS.COM Ltd.http://www.uk.debian.org/ |(| 10 Onslow Gardens, South Woodford, London E18 1NE ENGLAND pgp3SdiDFr41A.pgp Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Sat, Apr 16, 2011 at 11:31:08AM +0100, Philip Hands wrote: On Sat, 16 Apr 2011 09:17:04 +0200, Josselin Mouette j...@debian.org wrote: Le mercredi 13 avril 2011 à 10:51 +0100, Philip Hands a écrit : Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=245465 Ah, good point -- so rather than just chatting about this I should probably get off my arse and start making some sort of contribution to d-i. By the way, now that initscripts has support for mounting tmpfs on /tmp, all d-i needs to do is to set RUNTMP=yes when creating /etc/default/rcS (it already needs to do this for UTC). It might also want to allow adjustment of the size set in /etc/default/tmpfs and ensure there's sufficient swap space configured, but it shouldn't actually require any fstab changes. (Though the mount logic should let it pick up the mount options from /etc/fstab, if present.) Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110416110822.gb13...@codelibre.net
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Roger Leigh rle...@codelibre.net writes: On Sat, Apr 16, 2011 at 09:35:53AM +0200, Goswin von Brederlow wrote: To me that reads like you will mount a tmpfs on /tmp if root is read-only even if RAMTMP is not set. Which is wrong if the system has a /tmp filesystem in /etc/fstab. This is a good point. I've added the ability to detect if a filesystem will be mounted, and skip the tmpfs mount on /tmp if an entry for /tmp exists in fstab (will_mount in /lib/init/mount-functions.sh) Thanks. Also mount -a (in mountall.sh) fails, and therefore the whole boot, if a mountpoint already has something else mounted. If you unconditionally mount a tmpfs on /tmp if root is read-only then you just made systems unbootable that have /tmp in fstab. This is not correct. Have you actually tried it? I have, and other than the cosmetic issue of having a real filesystem mounted over the top of the tmpfs, the system is entirely functional, and boots error free. And with the above change, even this cosmetic issue is gone. Hmm. This is strange. I extrapolated from my experience with proc (#603858): /proc/mounts: (proc mounted by initramfs-tools in the ramdisk) none on /proc type proc (rw,relatime) fstab: proc/proc procdefaults0 0 # mount -v -a -t nonfs,nfs4,smbfs,cifs,ncp,ncpfs,coda,ocfs2,gfs,gfs2 -O no_netdev mount: proc already mounted or /proc busy mount: according to mtab, none is already mounted on /proc # echo $? 32 BUT: /proc/mounts: none on /tmp2 type tmpfs (rw,relatime) fstab: tmpfs /tmp2 tmpfs defaults0 0 # mount -v -a -t nonfs,nfs4,smbfs,cifs,ncp,ncpfs,coda,ocfs2,gfs,gfs2 -O no_netdev tmpfs on /tmp2 type tmpfs (rw) # echo $? 0 It seems like proc is a special case that doesn't allow mounting something over it. With tmpfs it just mounts over the old entry. Sorry for raising the alarm. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87oc46dqy2.fsf@frosties.localnet
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, Apr 14, 2011 at 09:27:23AM -0400, Marvin Renich wrote: * Luca Capello l...@pca.it [110414 06:43]: Hi there! Disclaimer: this is my last post on this matter (i.e. the meaning of RAMLOCK), it seems there is a problem with myself or my understanding. Either I do not read `man rcS` as you read it or we do not understand each other, so here the situation before and after your patch for /run (/var/lock became useless, everything is in /run/lock now, but I kept using /var/lock for consistency with the previous state): [before] RAMLOCK=no - /var/lock on the same filesystem /var is on RAMLOCK=yes - /var/lock on tmpfs [after] RAMLOCK=no - /var/lock is on tmpfs, shared with /run (given that /var/lock is symlinked/bind-mounted to /run/lock) RAMLOCK=yes - /var/lock gets its own tmpfs, differently from the one mounted at /run So, the meaning of RAMLOCK, WRT `man rcS` and as I read it, *has* changed. This is where we do not agree on wordings, but given that English is not my mother language, maybe it is only my fault. Thx, bye, Gismo / Luca Luca, It appears to me that you understand perfectly what RAMLOCK does. Your issue appears to be with the wording of the man page. To me (as a native English speaker) the wording is explicit about what will happen if you set RAMLOCK=yes (i.e. a tmpfs will be allocated and mounted at /var/lock--soon to be /run/lock), but is ambiguous as to what will happen if RAMLOCK=no. The implicit meaning is that no tmpfs will be allocated _for /var/lock_ and it will be on the same file system as /var, whatever that may be (soon to be /run/lock and /run). So the implicit behavior is that if RAMLOCK=no, and /var is a tmpfs, /var/lock will simply be a part of the tmpfs on /var. This is indeed the current (pre-/run) behavior, and is exactly the same as what will soon be with /run. The current wording of the man page seems correct to me, even for the new /run directory structure (with appropriate changes from /var/lock to /run/lock), but some minor changes in wording would make the RAMLOCK=no behavior more explicit. I would add the word separate and a sentence describing the behavior when RAMLOCK=no: Make /var/lock/ available as a separate ram file system (tmpfs). Will also disable cleaning of /var/lock/ during boot. Set to 'yes' to enable, to 'no' to disable. When 'no', /var/lock is on the same file system as /var. The size of the tmpfs can be controlled using TMPFS_SIZE and LOCK_SIZE in /etc/default/tmpfs. Because of this, packages can not expect directories in /var/lock to exist after boot. Packages expecting this are buggy and need to be fixed. If you like this wording, you should file a wishlist bug on the initscripts package. (I'm not going to because the current wording doesn't bother me.) New text, which is hopefully clear enough: RAMLOCK Make /run/lock/ available as a ram file system (tmpfs). Set to 'yes' to enable, to 'no' to disable (defaults to yes). The size of the tmpfs can be controlled using TMPFS_SIZE and LOCK_SIZE in /etc/default/tmpfs. Note that irrespective of this setting /run/lock will be located on a tmpfs, either one mounted on /run/lock (if RAMLOCK=yes) or one mounted on /run (if RAM‐ LOCK=no), and as a result the contents of /var/lock will always be lost on system reboot, and it it is no longer explicitly cleaned at boot. Because of this, packages can not expect directories in /var/lock to exist after boot. Packages expect‐ ing this are buggy and need to be fixed. Note that /run/lock was previously /var/lock, and a compatibility symlink or bind mount will be created to allow the old path to continue to func‐ tion. -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
New text, which is hopefully clear enough: RAMLOCK Make /run/lock/ available as a ram file system (tmpfs). Set to 'yes' to enable, to 'no' to disable (defaults to yes). The size of the tmpfs can be controlled using TMPFS_SIZE and LOCK_SIZE in /etc/default/tmpfs. Note that irrespective of this setting /run/lock will be located on a tmpfs, either one mounted on /run/lock (if RAMLOCK=yes) or one mounted on /run (if RAM‐ LOCK=no), and as a result the contents of /var/lock will always be lost on system reboot, and it it is no longer explicitly cleaned at boot. Because of this, packages can not expect directories in /var/lock to exist after boot. Packages expect‐ ing this are buggy and need to be fixed. Note that /run/lock was previously /var/lock, and a compatibility symlink or bind mount will be created to allow the old path to continue to func‐ tion. In all the case could we create a mount point for /run/lock ? A bind mount for disk based and a true mount for ramfs ? Bastien -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTi=fkwvx+muhzb4hdjxhdo8-ecv...@mail.gmail.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Fri, Apr 15, 2011 at 12:58:19PM +0200, Bastien ROUCARIES wrote: New text, which is hopefully clear enough: RAMLOCK Make /run/lock/ available as a ram file system (tmpfs). Set to 'yes' to enable, to 'no' to disable (defaults to yes). The size of the tmpfs can be controlled using TMPFS_SIZE and LOCK_SIZE in /etc/default/tmpfs. Note that irrespective of this setting /run/lock will be located on a tmpfs, either one mounted on /run/lock (if RAMLOCK=yes) or one mounted on /run (if RAM‐ LOCK=no), and as a result the contents of /var/lock will always be lost on system reboot, and it it is no longer explicitly cleaned at boot. Because of this, packages can not expect directories in /var/lock to exist after boot. Packages expect‐ ing this are buggy and need to be fixed. Note that /run/lock was previously /var/lock, and a compatibility symlink or bind mount will be created to allow the old path to continue to func‐ tion. In all the case could we create a mount point for /run/lock ? A bind mount for disk based and a true mount for ramfs ? Sorry, but I'm not sure I get what you're saying here. Could you rephrase it more clearly? Thanks, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Bastien ROUCARIES roucaries.bast...@gmail.com writes: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition Bastien Then you wouldn't be setting RAMTMP (or whatever the variable is called). MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87y63bk29p.fsf@frosties.localnet
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Roger Leigh rle...@codelibre.net writes: On Thu, Apr 14, 2011 at 10:44:08AM +0200, Bastien ROUCARIES wrote: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). I hope you check the fstab first. If there is a entry for a non tmpfs /tmp filesystem then that should be used. I'm assuming you do but just to be sure... MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87tydzk24r.fsf@frosties.localnet
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
John D. Hendrickson and Sara Darnell johnandsa...@cox.net writes: I'm reading (can't spend allot of time though, I'll try) initscripts_2.88dsf-13.3_amd64.deb sysvinit_2.88dsf-13.3.dsc I'm thinking (I'm not sure) that Bastien is working on this. He'd mentioned issues between sysinit and running on certain vservers. While reading scripts it reminded me, /etc/default, I have an older bug / comment to mention! There shouldn't be any magic in /etc/default. It's a bad practice to have magic in /etc/default. Any magic an init script needs to remain right in the init script itself. Why? 1) so the two are never separated 2) so a lack of defaults doesn't ammount to broken initscript 3) so if I use MY init script, or an older one, it is not foo'd thanks I hope that helps anyone :) John Like using a construct like this? # Set default values # Do not change them here, edit /etc/default/foo to customize FOO=42 BAR=23 # Source customization [ -f /etc/default/foo ] . /etc/default/foo Is what you mean? At least for variable where being empty isn't a sane default. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/87pqonk1ys.fsf@frosties.localnet
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Fri, Apr 15, 2011 at 04:41:56PM +0200, Goswin von Brederlow wrote: Roger Leigh rle...@codelibre.net writes: If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). I hope you check the fstab first. If there is a entry for a non tmpfs /tmp filesystem then that should be used. I'm assuming you do but just to be sure... No, we don't check. It's up to the admin to configure their system properly. If there is an entry in in fstab, it'll be mounted on top of the tmpfs, so the system will be configured the way they asked, but there will be a hidden tmpfs mount. But they would have explicitly needed to set RAMTMP=yes to get into this situation. For new installs, where the default /etc/default/rcS files does set RAMTMP=yes by default, the fstab file will not yet contain any user-specific mounts. If they do want to manuall mount something on /tmp, then they simply set RAMTMP=no. Note this behaviour is exactly the same as existing practice for /dev/shm, /var/run and /var/lock. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Fri, 15 Apr 2011, Roger Leigh wrote: For new installs, where the default /etc/default/rcS files does set RAMTMP=yes by default, the fstab file will not yet contain any user-specific mounts. If they do want to manuall mount something on /tmp, then they simply set RAMTMP=no. Hopefully you don't set RAMTMP=yes on new installs where a /tmp partition has been configured (and thus will be in fstab)? -- Edward Allcutt -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/alpine.deb.2.00.1104151605100.30...@pandora.retrosnub.co.uk
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition Bastien kk -- Karl Goetz, (Kamping_Kaiser / VK5FOSS) Debian contributor / gNewSense Maintainer http://www.kgoetz.id.au No, I won't join your social networking group -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/BANLkTim8WEMBGzorTV=obey7ihxt9kq...@mail.gmail.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On to, 2011-04-14 at 10:44 +0200, Bastien ROUCARIES wrote: And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM That sounds like you'd be better off using /var/tmp instead, actually. Please do not put /tmp on tmpfs use a bind mount of a rw partition No configuration for /tmp is going to be suitable for every purpose, I'm afraid. That's why it's so great that we have a standard way of overriding the location for temporary files: $TMPDIR. (Me, I'd quite like to see /tmp go away completely.) -- Blog/wiki/website hosting with ikiwiki (free for free software): http://www.branchable.com/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1302772324.2921.17.ca...@havelock.liw.fi
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, Apr 14, 2011 at 10:44:08AM +0200, Bastien ROUCARIES wrote: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, Apr 14, 2011 at 11:50:52AM +0930, Karl Goetz wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: Proposal: [...] /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. Yeah, especially on small systems. What about using the size of swap instead? Heck, if RAM is not tight, even 100% of swap could be a good default maximum. Of course, no swap would mean /tmp on a slow general-purpose filesystem. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, Apr 14, 2011 at 11:15 AM, Roger Leigh rle...@codelibre.net wrote: On Thu, Apr 14, 2011 at 10:44:08AM +0200, Bastien ROUCARIES wrote: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). Could you bind mount /var/tmp under /tmp in this case ? Bastien that use is android phone sometime to solve math problem... Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `- GPG Public Key: 0x25BFB848 Please GPG sign your mail. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAk2muxsACgkQVcFcaSW/uEjw7gCgkYgVs+3SvHhF+8Sgk4SboCQF thgAn38DpDR+iJCv7YdlzTA1nBEfgb8G =2T+k -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/banlktinows0wv7zuhrfdpub6g_2qbu8...@mail.gmail.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, Apr 14, 2011 at 11:32 AM, Bastien ROUCARIES roucaries.bast...@gmail.com wrote: On Thu, Apr 14, 2011 at 11:15 AM, Roger Leigh rle...@codelibre.net wrote: On Thu, Apr 14, 2011 at 10:44:08AM +0200, Bastien ROUCARIES wrote: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). Could you bind mount /var/tmp under /tmp in this case ? And BTW it seems that since 2.6.14 (subtree bind mount) we could also mount bind mount of /tmp as noexec nosuid... Need to test but could work and improve your security Bastien Bastien that use is android phone sometime to solve math problem... Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `- GPG Public Key: 0x25BFB848 Please GPG sign your mail. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAk2muxsACgkQVcFcaSW/uEjw7gCgkYgVs+3SvHhF+8Sgk4SboCQF thgAn38DpDR+iJCv7YdlzTA1nBEfgb8G =2T+k -END PGP SIGNATURE- -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/banlktinlpf7u-9b7drrzhm_55skczbw...@mail.gmail.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Hi there! Disclaimer: this is my last post on this matter (i.e. the meaning of RAMLOCK), it seems there is a problem with myself or my understanding. On Tue, 12 Apr 2011 23:42:58 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 10:35:37PM +0200, Luca Capello wrote: On Tue, 12 Apr 2011 20:47:35 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 08:12:21PM +0200, Luca Capello wrote: Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). [...] Again, I found it changed: how can you define LOCK_SIZE if /run/lock is on the same tmpfs than /run? If you set RAMLOCK=yes, then /run/lock is a *separate* tmpfs of size LOCK_SIZE, *exactly* like the /var/lock tmpfs mount (it's the same code with s/var/run/g). So the actual behaviour of this option is entirely unchanged bar the switch from /var/lock to /run/lock. So by default, /run and /run/lock are on the same tmpfs. But if you set RAMLOCK=yes, you'll get a second tmpfs mounted on /run/lock. So yes, /run/lock will always be on a tmpfs filesystem, that's obviously the main point of the patch. Either I do not read `man rcS` as you read it or we do not understand each other, so here the situation before and after your patch for /run (/var/lock became useless, everything is in /run/lock now, but I kept using /var/lock for consistency with the previous state): [before] RAMLOCK=no - /var/lock on the same filesystem /var is on RAMLOCK=yes - /var/lock on tmpfs [after] RAMLOCK=no - /var/lock is on tmpfs, shared with /run (given that /var/lock is symlinked/bind-mounted to /run/lock) RAMLOCK=yes - /var/lock gets its own tmpfs, differently from the one mounted at /run So, the meaning of RAMLOCK, WRT `man rcS` and as I read it, *has* changed. This is where we do not agree on wordings, but given that English is not my mother language, maybe it is only my fault. Thx, bye, Gismo / Luca pgpinr9sbLCVE.pgp Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Hi there! Just to be sure everyone gets it correctly... On Thu, 14 Apr 2011 11:15:07 +0200, Roger Leigh wrote: If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). Sorry, having /tmp as a tmpfs can be ATM set *manually* (i.e. not managed by any Debian configuration file), I have asked for having RAMTMP quite a long time ago, but I have been ignored since then: http://bugs.debian.org/402828 On Wed, 13 Apr 2011 14:49:15 +0200, Roger Leigh wrote: I would very much appreciate it if anyone could take the time to install the new initscripts and test it out. http://people.debian.org/~rleigh/run/sysvinit_2.88dsf-13.3.dsc http://people.debian.org/~rleigh/run/initscripts_2.88dsf-13.3_amd64.deb [...] So, by default, you get (separate tmpfs mounts): /run /run/shm /lib/init/rw (RAMLOCK=no, RAMSHM=yes, RAMTMP=no) Bingo, thank you for finally supporting RAMTMP :-) However, as I was discussing at [1], I still think that RAMLOCK and RAMSHM are misleading names and they should be something like LOCK_OWN_TMPFS and SHM_OWN_TMPFS. RUNLOCK and RUNSHM would have been better, but this would mean that RUNLOCK=yes by default. [1] http://lists.debian.org/msgid-search/%3c8739ln6wdi.fsf%40gismo.pca.it%3e For additional safety and security, it's possible to mount everything as separate tmpfs filesystems: /run /run/shm /run/lock /lib/init/rw /tmp (RAMLOCK=yes, RAMSHM=yes, RAMTMP=yes). This lets one have separate size limits and mount modes for each mount. Alternatively, it's possible to have everything on a single /run tmpfs, including /tmp (excluding /lib/init/rw, which will be removed soon): /run /lib/init/rw /tmp → /run/tmp (RAMLOCK=no, RAMSHM=no, RAMTMP=no). Note that /tmp was symlinked to /run/tmp prior to restarting, and /run/tmp was created by the init scripts (mountkernfs). The symlink needs creating by hand should you wish to do this. Having /tmp as a symlink can be used whatever the setting of RAMTMP, so you could have a tmpfs mounted on /run/tmp if you chose. Sorry, but why does using RAMSHM=no causes /tmp to be a symlink? AFAIK these are two different and unrelated things. Continuing the very same comment above on variable names, while we are at it and if this could be a (sort of) common setup, could we add support for something like RUNTMP, i.e. symling /tmp to /run/tmp? Thx, bye, Gismo / Luca pgpDzeMdwKQ29.pgp Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
fuser(1) In the postinst (or other) it seems you wish to know if your impacting things, are not all sure about the vserver situation, and are using stat(1) and test -L and etc. You might try fuser(1) so you are sure if /var/run will impact something. Luca Capello wrote: Hi there! Disclaimer: this is my last post on this matter (i.e. the meaning of RAMLOCK), it seems there is a problem with myself or my understanding. On Tue, 12 Apr 2011 23:42:58 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 10:35:37PM +0200, Luca Capello wrote: On Tue, 12 Apr 2011 20:47:35 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 08:12:21PM +0200, Luca Capello wrote: Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). [...] Again, I found it changed: how can you define LOCK_SIZE if /run/lock is on the same tmpfs than /run? If you set RAMLOCK=yes, then /run/lock is a *separate* tmpfs of size LOCK_SIZE, *exactly* like the /var/lock tmpfs mount (it's the same code with s/var/run/g). So the actual behaviour of this option is entirely unchanged bar the switch from /var/lock to /run/lock. So by default, /run and /run/lock are on the same tmpfs. But if you set RAMLOCK=yes, you'll get a second tmpfs mounted on /run/lock. So yes, /run/lock will always be on a tmpfs filesystem, that's obviously the main point of the patch. Either I do not read `man rcS` as you read it or we do not understand each other, so here the situation before and after your patch for /run (/var/lock became useless, everything is in /run/lock now, but I kept using /var/lock for consistency with the previous state): [before] RAMLOCK=no - /var/lock on the same filesystem /var is on RAMLOCK=yes - /var/lock on tmpfs [after] RAMLOCK=no - /var/lock is on tmpfs, shared with /run (given that /var/lock is symlinked/bind-mounted to /run/lock) RAMLOCK=yes - /var/lock gets its own tmpfs, differently from the one mounted at /run So, the meaning of RAMLOCK, WRT `man rcS` and as I read it, *has* changed. This is where we do not agree on wordings, but given that English is not my mother language, maybe it is only my fault. Thx, bye, Gismo / Luca -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4da6f023.4010...@cox.net
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
I'm reading (can't spend allot of time though, I'll try) initscripts_2.88dsf-13.3_amd64.deb sysvinit_2.88dsf-13.3.dsc I'm thinking (I'm not sure) that Bastien is working on this. He'd mentioned issues between sysinit and running on certain vservers. While reading scripts it reminded me, /etc/default, I have an older bug / comment to mention! There shouldn't be any magic in /etc/default. It's a bad practice to have magic in /etc/default. Any magic an init script needs to remain right in the init script itself. Why? 1) so the two are never separated 2) so a lack of defaults doesn't ammount to broken initscript 3) so if I use MY init script, or an older one, it is not foo'd thanks I hope that helps anyone :) John Bastien ROUCARIES wrote: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition Bastien kk -- Karl Goetz, (Kamping_Kaiser / VK5FOSS) Debian contributor / gNewSense Maintainer http://www.kgoetz.id.au No, I won't join your social networking group -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4da6edf2.1080...@cox.net
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
I've noticed I have to check /etc carefully. Some rc.d scripts that packages install edit and or activate things in /etc (they make insertions into automatically actived scripts in /etc for ssh, ppp, perl, network (pre-ifupdown or what), exim, things or other possible phone home things). While most or all are innoculous (some DO look suspect) I don't allow that on my box - I boot thin is why. And whenever there is somethign I read it to see why I must run it. Due to that I use some of my own init scripts which are slackware or potatoe scripts :) Luca Capello wrote: Hi there! Disclaimer: this is my last post on this matter (i.e. the meaning of RAMLOCK), it seems there is a problem with myself or my understanding. On Tue, 12 Apr 2011 23:42:58 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 10:35:37PM +0200, Luca Capello wrote: On Tue, 12 Apr 2011 20:47:35 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 08:12:21PM +0200, Luca Capello wrote: Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). [...] Again, I found it changed: how can you define LOCK_SIZE if /run/lock is on the same tmpfs than /run? If you set RAMLOCK=yes, then /run/lock is a *separate* tmpfs of size LOCK_SIZE, *exactly* like the /var/lock tmpfs mount (it's the same code with s/var/run/g). So the actual behaviour of this option is entirely unchanged bar the switch from /var/lock to /run/lock. So by default, /run and /run/lock are on the same tmpfs. But if you set RAMLOCK=yes, you'll get a second tmpfs mounted on /run/lock. So yes, /run/lock will always be on a tmpfs filesystem, that's obviously the main point of the patch. Either I do not read `man rcS` as you read it or we do not understand each other, so here the situation before and after your patch for /run (/var/lock became useless, everything is in /run/lock now, but I kept using /var/lock for consistency with the previous state): [before] RAMLOCK=no - /var/lock on the same filesystem /var is on RAMLOCK=yes - /var/lock on tmpfs [after] RAMLOCK=no - /var/lock is on tmpfs, shared with /run (given that /var/lock is symlinked/bind-mounted to /run/lock) RAMLOCK=yes - /var/lock gets its own tmpfs, differently from the one mounted at /run So, the meaning of RAMLOCK, WRT `man rcS` and as I read it, *has* changed. This is where we do not agree on wordings, but given that English is not my mother language, maybe it is only my fault. Thx, bye, Gismo / Luca -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4da6f3b8.8050...@cox.net
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
* Luca Capello l...@pca.it [110414 06:43]: Hi there! Disclaimer: this is my last post on this matter (i.e. the meaning of RAMLOCK), it seems there is a problem with myself or my understanding. Either I do not read `man rcS` as you read it or we do not understand each other, so here the situation before and after your patch for /run (/var/lock became useless, everything is in /run/lock now, but I kept using /var/lock for consistency with the previous state): [before] RAMLOCK=no - /var/lock on the same filesystem /var is on RAMLOCK=yes - /var/lock on tmpfs [after] RAMLOCK=no - /var/lock is on tmpfs, shared with /run (given that /var/lock is symlinked/bind-mounted to /run/lock) RAMLOCK=yes - /var/lock gets its own tmpfs, differently from the one mounted at /run So, the meaning of RAMLOCK, WRT `man rcS` and as I read it, *has* changed. This is where we do not agree on wordings, but given that English is not my mother language, maybe it is only my fault. Thx, bye, Gismo / Luca Luca, It appears to me that you understand perfectly what RAMLOCK does. Your issue appears to be with the wording of the man page. To me (as a native English speaker) the wording is explicit about what will happen if you set RAMLOCK=yes (i.e. a tmpfs will be allocated and mounted at /var/lock--soon to be /run/lock), but is ambiguous as to what will happen if RAMLOCK=no. The implicit meaning is that no tmpfs will be allocated _for /var/lock_ and it will be on the same file system as /var, whatever that may be (soon to be /run/lock and /run). So the implicit behavior is that if RAMLOCK=no, and /var is a tmpfs, /var/lock will simply be a part of the tmpfs on /var. This is indeed the current (pre-/run) behavior, and is exactly the same as what will soon be with /run. The current wording of the man page seems correct to me, even for the new /run directory structure (with appropriate changes from /var/lock to /run/lock), but some minor changes in wording would make the RAMLOCK=no behavior more explicit. I would add the word separate and a sentence describing the behavior when RAMLOCK=no: Make /var/lock/ available as a separate ram file system (tmpfs). Will also disable cleaning of /var/lock/ during boot. Set to 'yes' to enable, to 'no' to disable. When 'no', /var/lock is on the same file system as /var. The size of the tmpfs can be controlled using TMPFS_SIZE and LOCK_SIZE in /etc/default/tmpfs. Because of this, packages can not expect directories in /var/lock to exist after boot. Packages expecting this are buggy and need to be fixed. If you like this wording, you should file a wishlist bug on the initscripts package. (I'm not going to because the current wording doesn't bother me.) ...Marvin -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110414132723.ga3...@cleo.wdw
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, 14 Apr 2011 10:15:07 +0100, Roger Leigh rle...@codelibre.net wrote: On Thu, Apr 14, 2011 at 10:44:08AM +0200, Bastien ROUCARIES wrote: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, And moreover for scientific computation /tmp need to be on an harddisk. I do not want my 16GiB matric to go to memory when I have only 8GiB of RAM Please do not put /tmp on tmpfs use a bind mount of a rw partition If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). I think that may have been in response to my suggestion that d-i should set that option in the case when people select a multi-partition install, using some or all of the space for that would have been used for the tmp partition as additional swap. It strikes me that in Bastien's example of having huge arrays on /tmp, that allocating the 16G+ that was being allocated to /tmp as additional swap would likely result in either no change, or in fact a speedup, since the kernel can be more relaxed about pushing the writes out to disk when it's doing it as swap, but I've not tested that, so I may be wrong. If the data he's talking about is at all compressible, adding compcache to the mix might make things much faster for him in this setup, which wouldn't help at all if he was still putting the data on a disk partition based /tmp. We should probably run tests on all these cases that people seem to be blithely assuming would be slower. Having run all my systems with tmpfs /tmp for some time, I've never seen any negative impact, and often it's clearly faster. It also ensures that things like laptops don't need to spin up their drives as often. In short, I'm wondering why we don't just say that the default be to have a tmpfs /tmp, set to be about as big as we would have set it if it was on disk, and backed by enough extra swap to ensure that we're no more likely to run out of RAM, and allow people to switch it off if they don't like it, probably via a debconf question that modifies the choices that we make for partition and swap sizing during installs. The only problem I can see might be that things that munch their way through memory (earlier versions of iceweasel come to mind) will take much longer to bump into limits if we have too much swap, and so will cause machines to grind to a halt as they swap all the drivel they've filled memory with. Not that having less swap made that much less painful -- more recent versions of iceweasel seem rather better though, so perhaps we should not pander to broken software in our choice of swap size. Cheers, Phil. -- |)| Philip Hands [+44 (0)20 8530 9560]http://www.hands.com/ |-| HANDS.COM Ltd.http://www.uk.debian.org/ |(| 10 Onslow Gardens, South Woodford, London E18 1NE ENGLAND pgpD1Fd4xtry8.pgp Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
One last rc.d comment. (noting I use a variety but try to stick with latest) For 20 yrs. every time I try NFS during boot scripts, no matter which linux, I tend to get my linux frozen when nfs can't mount. I have yet to see an NFS that offers file access in a suitable manner (ie, error checking, correct locking, roll-backs, etc), though I occasionally, sparingly, use it myself. Why you all want to make it default for everyone even if the don't have nfs configured I have no idea ! It's sure to cause problems and isn't required by everyone. Ok, you can now beat me with rusty spoons I'll try not to write anymore (this week) johnandsara2@cox.netJohn D. Hendrickson and Sara Darnell wrote: I've noticed I have to check /etc carefully. Some rc.d scripts that packages install edit and or activate things in /etc (they make insertions into automatically actived scripts in /etc for ssh, ppp, perl, network (pre-ifupdown or what), exim, things or other possible phone home things). While most or all are innoculous (some DO look suspect) I don't allow that on my box - I boot thin is why. And whenever there is somethign I read it to see why I must run it. Due to that I use some of my own init scripts which are slackware or potatoe scripts :) Luca Capello wrote: Hi there! Disclaimer: this is my last post on this matter (i.e. the meaning of RAMLOCK), it seems there is a problem with myself or my understanding. On Tue, 12 Apr 2011 23:42:58 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 10:35:37PM +0200, Luca Capello wrote: On Tue, 12 Apr 2011 20:47:35 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 08:12:21PM +0200, Luca Capello wrote: Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). [...] Again, I found it changed: how can you define LOCK_SIZE if /run/lock is on the same tmpfs than /run? If you set RAMLOCK=yes, then /run/lock is a *separate* tmpfs of size LOCK_SIZE, *exactly* like the /var/lock tmpfs mount (it's the same code with s/var/run/g). So the actual behaviour of this option is entirely unchanged bar the switch from /var/lock to /run/lock. So by default, /run and /run/lock are on the same tmpfs. But if you set RAMLOCK=yes, you'll get a second tmpfs mounted on /run/lock. So yes, /run/lock will always be on a tmpfs filesystem, that's obviously the main point of the patch. Either I do not read `man rcS` as you read it or we do not understand each other, so here the situation before and after your patch for /run (/var/lock became useless, everything is in /run/lock now, but I kept using /var/lock for consistency with the previous state): [before] RAMLOCK=no - /var/lock on the same filesystem /var is on RAMLOCK=yes - /var/lock on tmpfs [after] RAMLOCK=no - /var/lock is on tmpfs, shared with /run (given that /var/lock is symlinked/bind-mounted to /run/lock) RAMLOCK=yes - /var/lock gets its own tmpfs, differently from the one mounted at /run So, the meaning of RAMLOCK, WRT `man rcS` and as I read it, *has* changed. This is where we do not agree on wordings, but given that English is not my mother language, maybe it is only my fault. Thx, bye, Gismo / Luca -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4da70144.9010...@cox.net
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Thu, Apr 14, 2011 at 02:13:57PM +0100, Philip Hands wrote: On Thu, 14 Apr 2011 10:15:07 +0100, Roger Leigh rle...@codelibre.net wrote: On Thu, Apr 14, 2011 at 10:44:08AM +0200, Bastien ROUCARIES wrote: On Thu, Apr 14, 2011 at 4:20 AM, Karl Goetz k...@kgoetz.id.au wrote: On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) If it wasn't already clear, having /tmp as a tmpfs is a /configurable option/, and it is /not/ the default (except when root is read-only (ro) in fstab). [...] In short, I'm wondering why we don't just say that the default be to have a tmpfs /tmp, set to be about as big as we would have set it if it was on disk, and backed by enough extra swap to ensure that we're no more likely to run out of RAM, and allow people to switch it off if they don't like it, probably via a debconf question that modifies the choices that we make for partition and swap sizing during installs. I would personally be quite happy in making a tmpfs /tmp the default. I've used tmpfs /tmp on all my systems for many years, and never have any problems. Given that it would be quite controversial to introduce, I didn't do that in this patch, but doing this for new installs would be good. One factor to consider is that by default we don't currently make any guarantees about the size of /tmp. If it's not a separate filesystem, it's constrained by the size of the root filesystem, which might be quite small. So storing massive amounts of data there is really not a good plan unless you've taken steps to ensure there's enough free space there. In my case, I have 16 GiB of swap space and have an 8 GiB limit on my /tmp tmpfs. Other people might mount a large partition there. The general point I want to make is that if you want to store large amounts of data on /tmp, you need to take steps to do that; we don't support it by default. In consequence, supporting a tmpfs on /tmp by default shouldn't be particularly problematic, even if the size is small. If you want larger amounts of space then one can either increase the tmpfs size (and possibly add more swap), or disable the tmpfs mount and mount a separate partition there. The only problem I can see might be that things that munch their way through memory (earlier versions of iceweasel come to mind) will take much longer to bump into limits if we have too much swap, and so will cause machines to grind to a halt as they swap all the drivel they've filled memory with. Not that having less swap made that much less painful -- more recent versions of iceweasel seem rather better though, so perhaps we should not pander to broken software in our choice of swap size. We run into this issue with larger systems in general anyway--when you have many gigabytes of memory, a process leaking memory will take much longer to use up the available resources before being killed. There's not much we can realistically do about that (process resource limits can partially mitigate this). But in general, this is going to be an issue whether the /tmp filesystem is backed by a disc partition or by the VM. So long as you have a sensible limit on the size of the tmpfs, this isn't going to be more problematic than on a real filesystem in terms of I/O load and safety. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
Roger Leigh rle...@codelibre.net writes: One reason for doing this is to have a single writable mount on the system, which might be useful for tiny systems with minimal resources, where root is r/o. On such a system, it might be useful to pool the limited writable space (which might not be a tmpfs). Could this case be handled better by the fsprotect package? -- Stig Sandbeck Mathisen s...@debian.org pgpZY6Bxgti27.pgp Description: PGP signature
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wed, Apr 13, 2011 at 10:29:16AM +0200, Stig Sandbeck Mathisen wrote: Roger Leigh rle...@codelibre.net writes: One reason for doing this is to have a single writable mount on the system, which might be useful for tiny systems with minimal resources, where root is r/o. On such a system, it might be useful to pool the limited writable space (which might not be a tmpfs). Could this case be handled better by the fsprotect package? It's a nice idea, but for a somewhat different purpose. This is overlaying the read-only root with a writable overlay. Where I'd like to get to is a purely read-only setup where nothing is writing to e.g. /etc. fsprotect could be considered to be working around deficiencies with our current situation, where I would like to fix the underlying problems making an aufs overlay necessary in the first place. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: On Mon, Apr 11, 2011 at 08:01:42PM +0100, Roger Leigh wrote: With the transition to /run and /run/lock as tmpfs filesystems, it would be desirable to provide sensible default size limits. Currently, we default to the tmpfs default of ½ RAM. But with several tmpfs filesystems, this does have the potential for the system to be OOMed by a user filling up more than one of the filesystems. A sensible default size limit would be useful here, for at least some of the filesystems. We currently allow specification of size limits for: /run (RUN_SIZE) /run/lock (LOCK_SIZE, optional) /dev/shm (SHM_SIZE) /tmp (TMP_SIZE, optional) [from temporary git repo at http://git.debian.org/?p=collab-maint/sysvinit;a=summary] In order to default to a sensible size, I would appreciate it if I could have some figures for the useage of these filesystems: e.g. % sudo du -sk /var/run /var/lock /dev/shm OK, some results: /var/run /var/lock /dev/shm Min9 0 0 5% percentile 60 0 0 Mean 886 9 599 Median 120 8 0 95% percentile 61217 310 Max4769652 80744 Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: Switch the default for all tmpfs mounts from 50% to 20%; it's still very large, but you have to mount many more to be able to break your system. /run: Use 10% core. This gives 102MiB on 1GiB system; the largest observed user used 50MiB. Even a system with 512MiB would not have hit the observed maximum, and that was a very large system with presumably more memory than this. /run/lock and /lib/init/rw: 5MiB each. More than plenty, but far less than current usage. /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) Regards, Roger --- [/etc/default/tmpfs] # Defaults for tmpfs filesystems mounted in early boot, before # filesystems from /etc/fstab are mounted. # # _SIZE variables are the maximum size (in bytes) that tmpfs # filesystems can use. The size will be rounded down to a multiple of # the page size, 4096 bytes. If no size is set, TMPFS_SIZE will be # used as the default. _MODE variables are the initial access # permissions for the root of the tmpfs mount. If no mode is set, # TMPFS_MODE will be used as the default. # TMPFS_SIZE: maximum size for all tmpfs filesystems if no specific # size is provided. If no value is provided here, the kernel default # will be used. TMPFS_SIZE=20% TMPFS_MODE=755 # RUN_SIZE: maximum size of /run (/var/run) # # Defaults to 10% core memory; the size required varies widely # depending upon the demands of the software being run; this heuristic # scales /run usage on system size. Samba in particular has been seen # to use at least 50MiB in a large heavily used server. Typical usage # is hundreds of KiB, maximum is tens of MiB. RUN_SIZE=10% RUN_MODE=755 # LOCK_SIZE: maximum size of /run/lock (/var/lock) # # Typical usage: tens of KiB; maximum hundreds of KiB. Default of # 5MiB should ensure the limit is never reached. LOCK_SIZE=5242880 # 5MiB LOCK_MODE=1777 # SHM_SIZE: maximum size of /run/shm (/dev/shm) # # No default size; the size required varies widely depending upon the # demands of the software being run. SHM_SIZE= SHM_MODE=1777 # TMP_SIZE: maximum size of /tmp # # No default size. TMP_SIZE= TMP_MODE=1777 # RW_SIZE: maximum size of /lib/init/rw # # Note /lib/init/rw is deprecated and programs using is are migrating # to use /run. It will be removed once all users have migrated to # /run. # Typical usage: tens of KiB. Default of 5MiB should ensure the limit # is never reached. RW_SIZE=5242880 # 5 MiB RW_MODE=755 -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
First of all, thanks to Roger Leigh for leading this effort. Roger Leigh wrote: Proposal: Switch the default for all tmpfs mounts from 50% to 20%; it's still very large, but you have to mount many more to be able to break your system. He should have said ... but you have to mount *and fill* many more to be able to break your system. The current tmpfs size of 50% suffices to protect the system should any *one* tmpfs be completely filled by a wayward process. Is that not good enough? I.e., do we really need to worry about the case where multiple tmpfses get filled simultaneously? Does it matter whether the system fails due to filesystem full or due to OOM? Broken is broken. If we do need to worry about that case then the real solution is not arbitrarily to increase the number-of-tmpfses-to-fill-up-in- order-to-break-the-system from 2 to 5. One real solution is to limit the total amount of memory that all tmpfses can take up to some value less than 100%. Another is to look more closely at which tmpfses could reasonably be attacked and limit the sum of *their* sizes to something less than 100%. -- Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4da576b3.7010...@gmail.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, 12 Apr 2011 13:51:09 +0200, Michael Biebl bi...@debian.org wrote: Am 12.04.2011 13:38, schrieb Roger Leigh: this for /var/lock (/run/lock), which can be mounted as a separate tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. We could also do the same for /dev/shm (/run/shm) and /tmp (/run/tmp) as well. In the case of /tmp this would not be the default except when root is read-only. I don't think symlinking /tmp to /run would be a good idea, as one could fill up /tmp (accidentaly) pretty quick. If we want to make / ro, then a separate tmpfs for /tmp looks like a better idea to me. While were about this, for installs where the users select multiple partitions we currently create a separate /tmp partition. This strikes me as suboptimal, since one could use the disk space allocated to /tmp as extra swap and then allocate a tmpfs of that size to be mounted on /tmp with no effect other than allowing the system to have access to more swap than it would have otherwise had (of course, that's probably more than it needs, so instead you could just save some disk space that would otherwise be left generally unused by overloading the swap usage with /tmp usage. Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. Cheers, Phil. -- |)| Philip Hands [+44 (0)20 8530 9560]http://www.hands.com/ |-| HANDS.COM Ltd.http://www.uk.debian.org/ |(| 10 Onslow Gardens, South Woodford, London E18 1NE ENGLAND pgpsRy7739Pbe.pgp Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 09:30:53PM +0200, Jan Hauke Rahm wrote: On Tue, Apr 12, 2011 at 08:21:25PM +0100, Roger Leigh wrote: I know little about vservers. How do they currently deal with /dev/shm and /lib/init/rw? Interesting question. Actually, in my setup, I don't see /dev/shm at all and /lib/init/rw is a regular directory. They don't. / is writable. Bastian -- Insufficient facts always invite danger. -- Spock, Space Seed, stardate 3141.9 -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110413111310.ga29...@wavehammer.waldi.eu.org
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wed, Apr 13, 2011 at 10:51:50AM +0100, Philip Hands wrote: On Tue, 12 Apr 2011 13:51:09 +0200, Michael Biebl bi...@debian.org wrote: I don't think symlinking /tmp to /run would be a good idea, as one could fill up /tmp (accidentaly) pretty quick. If we want to make / ro, then a separate tmpfs for /tmp looks like a better idea to me. While were about this, for installs where the users select multiple partitions we currently create a separate /tmp partition. This strikes me as suboptimal, since one could use the disk space allocated to /tmp as extra swap and then allocate a tmpfs of that size to be mounted on /tmp with no effect other than allowing the system to have access to more swap than it would have otherwise had (of course, that's probably more than it needs, so instead you could just save some disk space that would otherwise be left generally unused by overloading the swap usage with /tmp usage. Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. Sounds like a great idea. I'd extend it to any setups with a swap. Tmpfs is a lot faster than regular filesystems: it doesn't have to care about preserving the data's integrity after a crash and thus has no barriers, journaling, fsync(). When there's plenty of unused memory, the data will not ever touch the disk, too -- gracefully getting written out when there is a better use for memory it takes. Since most files in /tmp/ are very short lived, it's a good optimization. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
* Philip Hands p...@hands.com [110413 12:54]: This strikes me as suboptimal, since one could use the disk space allocated to /tmp as extra swap and then allocate a tmpfs of that size to be mounted on /tmp with no effect other than allowing the system to have access to more swap than it would have otherwise had (of course, that's probably more than it needs, so instead you could just save some disk space that would otherwise be left generally unused by overloading the swap usage with /tmp usage. Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. This has both the disadvantage of a system then having swap (given the big memory sizes one currently has and the big difference between RAM and disk access times, having swap is often quite a disadvantage) and of mixing several different things (/tmp is usually something simply filling over time, so without low enough limits one risks that something important is sometime not working because of missing RAM). Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110413113412.ga3...@pcpool00.mathematik.uni-freiburg.de
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wed, 2011-04-13 at 13:34 +0200, Bernhard R. Link wrote: * Philip Hands p...@hands.com [110413 12:54]: This strikes me as suboptimal, since one could use the disk space allocated to /tmp as extra swap and then allocate a tmpfs of that size to be mounted on /tmp with no effect other than allowing the system to have access to more swap than it would have otherwise had (of course, that's probably more than it needs, so instead you could just save some disk space that would otherwise be left generally unused by overloading the swap usage with /tmp usage. Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. This has both the disadvantage of a system then having swap (given the big memory sizes one currently has and the big difference between RAM and disk access times, having swap is often quite a disadvantage) [...] Under Linux, swap space is a requirement to defragment RAM. (This may change in the future.) Without swap space, the kernel eventually becomes unable to make large physically-contiguous allocations. Don't turn it off. Ben. -- Ben Hutchings Once a job is fouled up, anything done to improve it makes it worse. signature.asc Description: This is a digitally signed message part
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wed, 13 Apr 2011 13:34:13 +0200, Bernhard R. Link brl...@debian.org wrote: * Philip Hands p...@hands.com [110413 12:54]: This strikes me as suboptimal, since one could use the disk space allocated to /tmp as extra swap and then allocate a tmpfs of that size to be mounted on /tmp with no effect other than allowing the system to have access to more swap than it would have otherwise had (of course, that's probably more than it needs, so instead you could just save some disk space that would otherwise be left generally unused by overloading the swap usage with /tmp usage. Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. This has both the disadvantage of a system then having swap (given the big memory sizes one currently has and the big difference between RAM and disk access times, having swap is often quite a disadvantage) and Are you suggesting that a system that has enough RAM to not need swap will become slower if you enable swap but don't use it? If not, then either the files in /tmp will sit in RAM and therefore be a lot faster, or if they need to go to disk, will have been staged via swap, which may well allow a lot of small writes to small files to be coalesced into larger swap writes, although I'm not familiar enough with that to compare the amount of disk activity provoked by writing a lot of small files onto an ext3 file system vs. taking the most elderly data From a tmpfs and deciding to swap it out. I can imagine that, for instance, removing a large file from /tmp when on a file system might well require several writes, whereas it seems possible that doing the same for a swapped out tmpfs might be a case of simply recording that the blocks are ready for reuse. On the other hand, it may be that one needs to read in each and every block in order to mark it ad being deleted, but I would hope that's not the way it works. of mixing several different things (/tmp is usually something simply filling over time, Really? Checking a couple of servers at random, one that's been running for about 8 months but is fairly tightly locked down, has 8Kb used, and another that's got rather more random stuff running on it, it has 17Mb in /tmp, and that turns out to be debris laying around after a process that does OCR on postfix files while scanning for SPAM -- so that's actually a bug by the looks of it. Even on servers where it is the case that /tmp grows over time, it strikes me that tmpfs gives the system a much better chance to make sensible decisions about when to write data out to disk. On a mounted file system the system will attempt to ensure that the data gets out to disk in a reasonably timely manner, since it's trying to ensure that it's not lost in the event of a crash -- in that case of /tmp, any effort to keep the normal file-system promise of post-crash integrity is wasted effort, since we're going to throw the data away anyway. With tmpfs on the other hand, the data will never hit the disk if there is enough RAM, and if there is a need to write it out, it'll generally write the oldest data out first, so the portions of the tmpfs that are being used for short-lived files will generally not be written before those files are deleted again, so that they never need to be written at all. so without low enough limits one risks that something important is sometime not working because of missing RAM). If you'd read what I wrote, I suggested that a reasonable start would be to allocate the /tmp filesystem space as swap, and then limit /tmp to that size -- even if you then fill /tmp it cannot exceed the extra size that you allocated to swap by giving it the disk that you'd otherwise be using for /tmp as a file system. Admittedly, I think that using the amount the we currently allocate to /tmp for swap would be a waste, but it's a reasonable starting point. In fact, it wouldn't surprise me if the act of mounting an additional filesystem consumes more memory than a tmpfs with 8kb of files on it, so for the first server mentioned above I may well be consuming less RAM than I would be mounting an empty /tmp from the disk. Cheers, Phil. -- |)| Philip Hands [+44 (0)20 8530 9560]http://www.hands.com/ |-| HANDS.COM Ltd.http://www.uk.debian.org/ |(| 10 Onslow Gardens, South Woodford, London E18 1NE ENGLAND pgpiEojmazFuq.pgp Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wednesday 13 April 2011, Ben Hutchings wrote: On Wed, 2011-04-13 at 13:34 +0200, Bernhard R. Link wrote: * Philip Hands p...@hands.com [110413 12:54]: This strikes me as suboptimal, since one could use the disk space allocated to /tmp as extra swap and then allocate a tmpfs of that size to be mounted on /tmp with no effect other than allowing the system to have access to more swap than it would have otherwise had (of course, that's probably more than it needs, so instead you could just save some disk space that would otherwise be left generally unused by overloading the swap usage with /tmp usage. Therefore, in the multi-partition setup, I think we should also default to having /tmp on tmpfs. This has both the disadvantage of a system then having swap (given the big memory sizes one currently has and the big difference between RAM and disk access times, having swap is often quite a disadvantage) [...] Under Linux, swap space is a requirement to defragment RAM. (This may change in the future.) Without swap space, the kernel eventually becomes unable to make large physically-contiguous allocations. Don't turn it off. Ben. I am surprised at this. I have several boxes which are small single board computers with solid state disks (MIDE or CF), so as I did not need swap space (the running set is fixed and the memory requirement was within the total available memory, I did not define any swap space. A few days ago I needed to move one of the boxes I noted its uptime at 594 days just before I switched it off. I grant you that it has 256MB of memory, and 120MB is currently free, but I have not noticed any problems growing over the time it was up. Maybe it just did not need to make any large physically contiguous allocations. BTW, /var/run is currently occupying a grand total of 27KB if that is relevant to the subject of this thread. David -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201104131544.37402.david.goodeno...@btconnect.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On 2011-04-13, David Goodenough david.goodeno...@btconnect.com wrote: I am surprised at this. I have several boxes which are small single board computers with solid state disks (MIDE or CF), so as I did not need swap space (the running set is fixed and the memory requirement was within the total available memory, I did not define any swap space. A few days ago I needed to move one of the boxes I noted its uptime at 594 days just before I switched it off. I grant you that it has 256MB of memory, and 120MB is currently free, but I have not noticed any problems growing over the time it was up. Maybe it just did not need to make any large physically contiguous allocations. Given that Linux does paging, you normally don't need large physically contiguous allocations. I think the exceptions are mainly I/O regions for DMA. And you're probably not using that heavily on such a machine. Kind regards Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrniqbfk4.rnn.tr...@kelgar.0x539.de
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
* Philip Hands p...@hands.com [110413 15:51]: Are you suggesting that a system that has enough RAM to not need swap will become slower if you enable swap but don't use it? If you don't use it it will hopefully make not much big difference. The difference is if it gets used. If some program goes harvoc and allocates to much stuff without swap it will most of the time simply get killed. With swap it will first make the computer unuseably slow for some time before it finally gets killed. of mixing several different things (/tmp is usually something simply filling over time, Really? Checking a couple of servers at random, one that's been running for about 8 months but is fairly tightly locked down, has 8Kb used, Servers of course usually have not that much (unless they are terminal/login servers). The big /tmp stuff are usually user programs and lab computers and other machines with many different users loging in are usually multi-partition. Some lab computers: /dev/sda7 6,5G 164M 6,0G 3% /tmp /dev/sda7 6,5G 157M 6,0G 3% /tmp /dev/sda7 6,5G 148M 6,0G 3% /tmp /dev/sda7 6,5G 145M 6,0G 3% /tmp /dev/sda7 6,5G 205M 5,9G 4% /tmp /dev/sda7 6,5G 321M 5,8G 6% /tmp /dev/sda7 6,5G 148M 6,0G 3% /tmp /dev/sda7 6,5G 225M 5,9G 4% /tmp /dev/sda7 6,5G 2,8G 3,4G 46% /tmp /dev/sda7 6,5G 152M 6,0G 3% /tmp /dev/sda7 6,5G 152M 6,0G 3% /tmp /dev/sda7 6,5G 150M 6,0G 3% /tmp /dev/sda7 6.5G 162M 6.0G 3% /tmp A login server: /dev/vda4 21G 173M 18G 1% /tmp A login server from another institute: /dev/hda5 9,9G 1,2G 8,2G 13% /tmp If you'd read what I wrote, I suggested that a reasonable start would be to allocate the /tmp filesystem space as swap, and then limit /tmp to that size -- even if you then fill /tmp it cannot exceed the extra size that you allocated to swap by giving it the disk that you'd otherwise be using for /tmp as a file system. And how do you make sure those metrics are correct if that is enabled as default in the installer? In fact, it wouldn't surprise me if the act of mounting an additional filesystem consumes more memory than a tmpfs with 8kb of files on it, so for the first server mentioned above I may well be consuming less RAM than I would be mounting an empty /tmp from the disk. RAM is nowadays for most uses usually there in abundancy. It's not that it is scarce. The problems are faulty programs trying to get as much as they can. If things grow unlimited there will always hit the limit. And having different limits usually makes things more easy to control. Bernhard R. Link -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110413152716.ga5...@pcpool00.mathematik.uni-freiburg.de
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
I just realized that I misunderstood Roger Leigh's posting and so my previous message was mostly superfluous. My apologies. 1. His statement but you have to mount many more to be able to break your system was correct (and can be made more explicit by adding ... by filling them all). 2. His proposal *was* to reduce the size of all tmpfses (together) to 50%, as follows: /run 10% /run/shm 20% /tmp 20% /run/lock, /lib/init/rw very small What remains of my previous message are my two questions: 1. Is OOM importantly worse than a full system tmpfs? I think so too, but 2. Even if so, do we have to worry about OOM happening because *multiple* tmpfses got filled? We are already safe from OOM if one tmpfs (whose size is 50% of memory) gets filled. -- Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4da5bf6e.9090...@gmail.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wed, Apr 13, 2011 at 03:17:24PM +, Philipp Kern wrote: On 2011-04-13, David Goodenough david.goodeno...@btconnect.com wrote: I am surprised at this. I have several boxes which are small single board computers with solid state disks (MIDE or CF), so as I did not need swap space (the running set is fixed and the memory requirement was within the total available memory, I did not define any swap space. A few days ago I needed to move one of the boxes I noted its uptime at 594 days just before I switched it off. I grant you that it has 256MB of memory, and 120MB is currently free, but I have not noticed any problems growing over the time it was up. Maybe it just did not need to make any large physically contiguous allocations. Given that Linux does paging, you normally don't need large physically contiguous allocations. I think the exceptions are mainly I/O regions for DMA. Heap allocations also have to be contiguous. And every thread needs a kernel stack which is at least 2 contiguous pages on most architectures. And you're probably not using that heavily on such a machine. Evidently. Ben. -- Ben Hutchings We get into the habit of living before acquiring the habit of thinking. - Albert Camus -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110413160825.gi2...@decadent.org.uk
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wednesday 13 April 2011, Ben Hutchings wrote: On Wed, Apr 13, 2011 at 03:17:24PM +, Philipp Kern wrote: On 2011-04-13, David Goodenough david.goodeno...@btconnect.com wrote: I am surprised at this. I have several boxes which are small single board computers with solid state disks (MIDE or CF), so as I did not need swap space (the running set is fixed and the memory requirement was within the total available memory, I did not define any swap space. A few days ago I needed to move one of the boxes I noted its uptime at 594 days just before I switched it off. I grant you that it has 256MB of memory, and 120MB is currently free, but I have not noticed any problems growing over the time it was up. Maybe it just did not need to make any large physically contiguous allocations. Given that Linux does paging, you normally don't need large physically contiguous allocations. I think the exceptions are mainly I/O regions for DMA. Heap allocations also have to be contiguous. And every thread needs a kernel stack which is at least 2 contiguous pages on most architectures. And you're probably not using that heavily on such a machine. Its acting as a network router. So presumably once everything is allocated, it keeps reusing them. David Evidently. Ben. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201104131720.18057.david.goodeno...@btconnect.com
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wed, Apr 13, 2011 at 05:21:18PM +0200, Thomas Hood wrote: I just realized that I misunderstood Roger Leigh's posting and so my previous message was mostly superfluous. My apologies. 1. His statement but you have to mount many more to be able to break your system was correct (and can be made more explicit by adding ... by filling them all). Yes, I did word it poorly, I'm afraid. 2. His proposal *was* to reduce the size of all tmpfses (together) to 50%, as follows: /run 10% /run/shm 20% /tmp 20% /run/lock, /lib/init/rw very small I should admit that the 50% here is more by coincidence than intent, but the general idea is that it's restricted overall to a sensible amount. Given that tmpfs on /tmp is optional and disabled by default, it would be 30% + 10MiB for the last two. What I should have stated more clearly in my earlier messages about the limits is that we now have the flexibility to choose exactly how to manage the space. We can have a single large tmpfs (less flexible and can be filled up by users, but there's much less chance of a single part running out of space cf multiple mounts) or we can have a collection of separate tmpfs mounts with different size limits (less chance of a user filling a system-only part, but an increased possibility of a single mount filling up). Or we can choose something in between those two extremes. For the defaults, I'd like something which will work in all cases, and which can be easily tweaked for special/non-standard uses. If we have a single shared tmpfs, 50% core is eminently sensible, providing you have some swap to back it. If we have multiple tmpfs mounts, we do want to consider restricting it further. The 10% and 20% figures are fairly arbitrary (the 10% comes from the max. samba usage seen with a big margin added to it; it's still three orders of magnitude bigger than the average usage of /run). The 20% is a bit more fuzzy; it's smaller than the 50% default, but still large enough that the chance of using it all is extremely remote. I'm quite happy to adjust these values if needed. What remains of my previous message are my two questions: 1. Is OOM importantly worse than a full system tmpfs? I think so too, but They are both undesirable. OOM is worse because it's potentially unrecoverable (the OOM killer can't kill a process to reclaim space on tmpfs [with the exception of POSIX SHM, but I don't think the kernel knows about that--it's purely a userspace implementation], so you can lock up the whole machine). But equally we don't want to allow users to interfere with system processes by denying them the resources they need (e.g. not being able to create a lockfile/pidfile); but in this situation, it is rather easier to recover since we can just delete the offending files to free up some space. [It would be nice if tmpfs offered a superuser-only allocation of e.g. 5% like ext does, which would go a long way to mitigate user DoS.] 2. Even if so, do we have to worry about OOM happening because *multiple* tmpfses got filled? We are already safe from OOM if one tmpfs (whose size is 50% of memory) gets filled. This is really down to the limits we have set on them. If we have sensible limits, even far too large for what we expect to be typical usage, we're safe from OOM. With no limit (or more accurately the default kernel limit), as used at present, adding multiple tmpfs mounts does make this more likely. The proposed default values are intended to satisfy these requirements, but they may well not be optimal, and I'll be happy to adjust them as needed. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Wed, 13 Apr 2011 10:32:42 +0100 Roger Leigh rle...@codelibre.net wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: Following the discussion yesterday, I'd like to propose doing something like the example below. It's possible to size a tmpfs as a percentage of core memory, the default being -o size=50%. Rather than setting an absolute value, we can size e.g. /run as a percentage of total memory, which should scale with /run usage better than a fixed value. Proposal: [...] /run/shm: No default (use general tmpfs default of 20%) /tmp: No default (use general tmpfs default of 20%) 20% doesn't seem like a lot for /tmp when people try and compile something. While its not something most people end up doing, it does seem odd to make people change their tempfs size before they can start building packages for debian/themselves. just a thought, kk -- Karl Goetz, (Kamping_Kaiser / VK5FOSS) Debian contributor / gNewSense Maintainer http://www.kgoetz.id.au No, I won't join your social networking group signature.asc Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Mon, Apr 11, 2011 at 08:01:42PM +0100, Roger Leigh wrote: With the transition to /run and /run/lock as tmpfs filesystems, it would be desirable to provide sensible default size limits. Currently, we default to the tmpfs default of ½ RAM. But with several tmpfs filesystems, this does have the potential for the system to be OOMed by a user filling up more than one of the filesystems. A sensible default size limit would be useful here, for at least some of the filesystems. We currently allow specification of size limits for: /run (RUN_SIZE) /run/lock (LOCK_SIZE, optional) /dev/shm (SHM_SIZE) /tmp (TMP_SIZE, optional) [from temporary git repo at http://git.debian.org/?p=collab-maint/sysvinit;a=summary] In order to default to a sensible size, I would appreciate it if I could have some figures for the useage of these filesystems: e.g. % sudo du -sk /var/run /var/lock /dev/shm OK, some results: /var/run /var/lock /dev/shm Min9 0 0 5% percentile 60 0 0 Mean 886 9 599 Median 120 8 0 95% percentile 61217 310 Max4769652 80744 I was going to do this separately for different types of system (desktop, server, etc.), but there really wasn't much difference except for a few outliers. /var/run: Most systems use just 1-200 KiB. A few use a lot (tens of MiB). The large users appear to be Samba, which creates a number of .tdb files under /var/run in addition to pidfiles; presumably on very busy systems, these can grow to be quite large. I guess they are transient state, but is /var/run the appropriate place for them? If so, we need to take this into account. One system had a 1MiB /var/run/samba/namelist.debug file, which could possibly have been put elsewhere. wins.tdb, connections.tdb and locking.tdb in particular look like they can potentially grow very large; would keeping these on /var and deleting them on server restart be more appropriate than using /run? utmp could also potentially grow to several MiB. Without taking Samba into consideration, 10MiB would be more than plenty for all but the most extreme uses. Allowing for Samba, we need at least 30MiB, and potentially 50MiB for a good safety margin. Any comments from the Samba maintainers? /var/lock: Tiny usage under all circumstances. One MiB would be more than plenty. /tmp, /dev/shm: Can vary massively depending on what's using it; we can't set a sensible default at all. Josh Triplett suggested that we could use a single tmpfs on /run and have the rest as symlinks into /run, with potentially a separate tmpfs for user-writable filesystems to prevent a user DoS. This idea does have merit, and we could make it the default. We currently do this for /var/lock (/run/lock), which can be mounted as a separate tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. We could also do the same for /dev/shm (/run/shm) and /tmp (/run/tmp) as well. In the case of /tmp this would not be the default except when root is read-only. This would mean we don't have to choose a default size for each filesystem separately. But the sysadmin would have the ability to enable it should they choose. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: On Mon, Apr 11, 2011 at 08:01:42PM +0100, Roger Leigh wrote: With the transition to /run and /run/lock as tmpfs filesystems, it would be desirable to provide sensible default size limits. Currently, we default to the tmpfs default of ½ RAM. But with several tmpfs filesystems, this does have the potential for the system to be OOMed by a user filling up more than one of the filesystems. A sensible default size limit would be useful here, for at least some of the filesystems. We currently allow specification of size limits for: /run (RUN_SIZE) /run/lock (LOCK_SIZE, optional) /dev/shm (SHM_SIZE) /tmp (TMP_SIZE, optional) [from temporary git repo at http://git.debian.org/?p=collab-maint/sysvinit;a=summary] In order to default to a sensible size, I would appreciate it if I could have some figures for the useage of these filesystems: e.g. % sudo du -sk /var/run /var/lock /dev/shm OK, some results: /var/run /var/lock /dev/shm Min9 0 0 5% percentile 60 0 0 Mean 886 9 599 Median 120 8 0 95% percentile 61217 310 Max4769652 80744 Forgot to mention, this is based on a sample set of 156 separate systems. Many thanks to everyone who mailed me with the stats! -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Am 12.04.2011 13:38, schrieb Roger Leigh: this for /var/lock (/run/lock), which can be mounted as a separate tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. We could also do the same for /dev/shm (/run/shm) and /tmp (/run/tmp) as well. In the case of /tmp this would not be the default except when root is read-only. I don't think symlinking /tmp to /run would be a good idea, as one could fill up /tmp (accidentaly) pretty quick. If we want to make / ro, then a separate tmpfs for /tmp looks like a better idea to me. Cheers, Michael -- Why is it that all of the instruments seeking intelligent life in the universe are pointed away from Earth? signature.asc Description: OpenPGP digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Mon, Apr 11, 2011 at 08:01:42PM +0100, Roger Leigh wrote: With the transition to /run and /run/lock as tmpfs filesystems, it would be desirable to provide sensible default size limits. Currently, we default to the tmpfs default of ½ RAM. But with several tmpfs filesystems, this does have the potential for the system to be OOMed by a user filling up more than one of the filesystems. A sensible default size limit would be useful here, for at least some of the filesystems. What about 5% of RAM + 20% of swap? Perhaps even just 5% of (RAM + swap). Unused space doesn't cost you anything, and when there's swap available, tmpfs is strictly better than any real filesystem could possibly be. Unlike a fixed limit, a ratio would both prevent a runaway daemon from OOMing the system and ensure non-embedded systems have plenty of space. Ie, that 50MB of samba files would fit comfortably on any system that's not starved for memory. Samba typically means a file server -- ie, often very little physical RAM but plenty of disk space. A static number would be either too big on a smartphone or too small on regular machines. -- 1KB // Microsoft corollary to Hanlon's razor: // Never attribute to stupidity what can be // adequately explained by malice. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110412120446.ga2...@angband.pl
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: On Mon, Apr 11, 2011 at 08:01:42PM +0100, Roger Leigh wrote: With the transition to /run and /run/lock as tmpfs filesystems, it would be desirable to provide sensible default size limits. Currently, we default to the tmpfs default of ½ RAM. But with several tmpfs filesystems, this does have the potential for the system to be OOMed by a user filling up more than one of the filesystems. A sensible default size limit would be useful here, for at least some of the filesystems. We currently allow specification of size limits for: /run (RUN_SIZE) /run/lock (LOCK_SIZE, optional) /dev/shm (SHM_SIZE) /tmp (TMP_SIZE, optional) [from temporary git repo at http://git.debian.org/?p=collab-maint/sysvinit;a=summary] In order to default to a sensible size, I would appreciate it if I could have some figures for the useage of these filesystems: e.g. % sudo du -sk /var/run /var/lock /dev/shm /var/run: Most systems use just 1-200 KiB. A few use a lot (tens of MiB). The large users appear to be Samba, which creates a number of .tdb files under /var/run in addition to pidfiles; presumably on very busy systems, these can grow to be quite large. I guess they are transient state, but is /var/run the appropriate place for them? Yes, it is, per the FHS. If so, we need to take this into account. One system had a 1MiB /var/run/samba/namelist.debug file, which could possibly have been put elsewhere. It fits the FHS definition for /var/run. It doesn't belong elsewhere. wins.tdb, connections.tdb and locking.tdb in particular look like they can potentially grow very large; would keeping these on /var and deleting them on server restart be more appropriate than using /run? utmp could also potentially grow to several MiB. Keep them *where* on /var? They're already exactly where the FHS says they should be. The server will null them out on startup, which is precisely the sort of file that's *meant to be stored in /var/run*. Without taking Samba into consideration, 10MiB would be more than plenty for all but the most extreme uses. Allowing for Samba, we need at least 30MiB, and potentially 50MiB for a good safety margin. Any comments from the Samba maintainers? I would like to know what real-world problem you're trying to solve by limiting the size of these filesystems. Have you had actual reports of users running into trouble with the current half-of-RAM default filling up? I've never heard such a thing. This is a root-only filesystem, so if the user has a service that wants that much space for temp files, why impose additional limits? If the problem is that multiple tmpfs are mounted and each can expand to half-of-RAM, either reduce the number of tmpfses presented (as discussed), or limit the *whole* allocation for known mount points to half-of-RAM and partition appropriately, or both. But imposing stricter limits than necessary based on survey data is just wrong. -- Steve Langasek Give me a lever long enough and a Free OS Debian Developer to set it on, and I can move the world. Ubuntu Developerhttp://www.debian.org/ slanga...@ubuntu.com vor...@debian.org -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20110412144454.gc9...@virgil.dodds.net
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Hi there! On Tue, 12 Apr 2011 13:38:03 +0200, Roger Leigh wrote: Josh Triplett suggested that we could use a single tmpfs on /run and have the rest as symlinks into /run, with potentially a separate tmpfs for user-writable filesystems to prevent a user DoS. This idea does have merit, and we could make it the default. We currently do this for /var/lock (/run/lock), which can be mounted as a separate tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. Do you mean that the meaning of RAMLOCK has completely changed? Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). Will also disable cleaning of /var/lock/ during boot. Set to 'yes' to enable, to 'no' to disable. The size of the tmpfs can be controlled using TMPFS_SIZE and LOCK_SIZE in /etc/default/tmpfs. Because of this, packages can not expect directories in /var/lock to exist after boot. Packages expecting this are buggy and need to be fixed. I consider completely changing it a serious bug, may I suggest deprecating it completely and adding a new variable instead? I guess the same should be applied to RAMRUN, i.e. simply deprecate it. Thx, bye, Gismo / Luca pgp1Sal8WK3tP.pgp Description: PGP signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 08:12:21PM +0200, Luca Capello wrote: Hi there! On Tue, 12 Apr 2011 13:38:03 +0200, Roger Leigh wrote: Josh Triplett suggested that we could use a single tmpfs on /run and have the rest as symlinks into /run, with potentially a separate tmpfs for user-writable filesystems to prevent a user DoS. This idea does have merit, and we could make it the default. We currently do this for /var/lock (/run/lock), which can be mounted as a separate tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. Do you mean that the meaning of RAMLOCK has completely changed? Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). Will also disable cleaning of /var/lock/ during boot. Set to 'yes' to enable, to 'no' to disable. The size of the tmpfs can be controlled using TMPFS_SIZE and LOCK_SIZE in /etc/default/tmpfs. Because of this, packages can not expect directories in /var/lock to exist after boot. Packages expecting this are buggy and need to be fixed. I consider completely changing it a serious bug, may I suggest deprecating it completely and adding a new variable instead? I guess the same should be applied to RAMRUN, i.e. simply deprecate it. With the patch as it stands at present, RAMRUN is deprecated. /run is always a tmpfs; RUN_SIZE will set its size, as before. RAMLOCK is unchanged, except for the fact that it's mounted on /run/lock rather than /var/lock. Likewise, LOCK_SIZE is unchanged in its meaning. We could introduce new variables for /run such as RUNLOCK, but given that it does exactly what it used to do, I don't think it gains us anything. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 07:47:35PM +0100, Roger Leigh wrote: With the patch as it stands at present, RAMRUN is deprecated. /run is always a tmpfs; RUN_SIZE will set its size, as before. Hmm, just thinking... vServers don't really allow mounting AFAIK as that would be the host's job. Wouldn't making /run a tmpfs in every case conflict here? Or am I jumping around nonsense now...? Hauke -- .''`. Jan Hauke Rahm j...@debian.org www.jhr-online.de : :' : Debian Developer www.debian.org `. `'` Member of the Linux Foundationwww.linux.com `- Fellow of the Free Software Foundation Europe www.fsfe.org signature.asc Description: Digital signature
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 07:44:54AM -0700, Steve Langasek wrote: On Tue, Apr 12, 2011 at 12:38:03PM +0100, Roger Leigh wrote: On Mon, Apr 11, 2011 at 08:01:42PM +0100, Roger Leigh wrote: With the transition to /run and /run/lock as tmpfs filesystems, it would be desirable to provide sensible default size limits. Currently, we default to the tmpfs default of ½ RAM. But with several tmpfs filesystems, this does have the potential for the system to be OOMed by a user filling up more than one of the filesystems. A sensible default size limit would be useful here, for at least some of the filesystems. We currently allow specification of size limits for: /run (RUN_SIZE) /run/lock (LOCK_SIZE, optional) /dev/shm (SHM_SIZE) /tmp (TMP_SIZE, optional) [from temporary git repo at http://git.debian.org/?p=collab-maint/sysvinit;a=summary] In order to default to a sensible size, I would appreciate it if I could have some figures for the useage of these filesystems: e.g. % sudo du -sk /var/run /var/lock /dev/shm /var/run: Most systems use just 1-200 KiB. A few use a lot (tens of MiB). The large users appear to be Samba, which creates a number of .tdb files under /var/run in addition to pidfiles; presumably on very busy systems, these can grow to be quite large. I guess they are transient state, but is /var/run the appropriate place for them? Yes, it is, per the FHS. I thought so at well; I just wanted to check, given that it's the main user of space under /run. Without taking Samba into consideration, 10MiB would be more than plenty for all but the most extreme uses. Allowing for Samba, we need at least 30MiB, and potentially 50MiB for a good safety margin. Any comments from the Samba maintainers? I would like to know what real-world problem you're trying to solve by limiting the size of these filesystems. Have you had actual reports of users running into trouble with the current half-of-RAM default filling up? I've never heard such a thing. This is a root-only filesystem, so if the user has a service that wants that much space for temp files, why impose additional limits? Resources are limited, and there's always a size limit. But we currently ignore the issue by simply accepting the kernel defaults, which are far, far too large (but on a very low memory system, might equally be too small). We can of course continue to do so; this doesn't directly cause problems--the space isn't actually claimed until used--but there is the potential for the lack of consideration for the real usage requirements to be abused. For filesystems such as /run/lock, we can safely set a limit for it. e.g. 2MiB would be fine; it's several hundred times the typical usage. Having it sized at 4GiB (as is the case on my 8GiB desktop) is clearly just too large. We should be able to set a more sensible default than that. Current usage on my system: 12 KiB... The same applies to /lib/init/rw (currently 4GiB also; actual usage 8 KiB). Having multiple tmpfses with the kernel defaults means that a user or badly written program could intentionally or accidentally lock up the machine by using all available memory by filling up one or more of the tmpfses. And the majority /are/ user writable by default, even /run (via /var/lock, which is not a separate mount by default--maybe it should be?). /dev/shm is user writable, /tmp is user writable. This isn't a new problem; users could always fill up /var before now as well (by default). But now they can lock up the system rather than just using up free disc space. If the problem is that multiple tmpfs are mounted and each can expand to half-of-RAM, either reduce the number of tmpfses presented (as discussed), or limit the *whole* allocation for known mount points to half-of-RAM and partition appropriately, or both. But imposing stricter limits than necessary based on survey data is just wrong. I wasn't wanting to impose strict limits, but a sensible default based upon what was commonly used, with a big margin on top of that, to ensure the limit would never be hit in practice. For /var/run, 100MiB would be plenty more than would ever be required for the vast majority of users. But the suggestion of computing it as a fraction of core+swap is a good one (perhaps with a minimum limit). As you state above, reducing the total number of tmpfses would be useful--we can then have one or two with sysadmin (or kernel) set limits, so an individual directory won't have an outrageously small limit that might be reached. For this reason, I've adapted the patch to move /dev/shm to /run/shm; it's configurable whether this is a separate tmpfs mount, or simply a subdirectory of /run, and the size is also configurable as before (SHM_SIZE, with RAMSHM as the option to toggle the mounting). We could additionally allow /tmp to be moved under /run/tmp, so that all existing tmpfs mounts could share a single
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 09:07:48PM +0200, Jan Hauke Rahm wrote: On Tue, Apr 12, 2011 at 07:47:35PM +0100, Roger Leigh wrote: With the patch as it stands at present, RAMRUN is deprecated. /run is always a tmpfs; RUN_SIZE will set its size, as before. Hmm, just thinking... vServers don't really allow mounting AFAIK as that would be the host's job. Wouldn't making /run a tmpfs in every case conflict here? Or am I jumping around nonsense now...? I know little about vservers. How do they currently deal with /dev/shm and /lib/init/rw? Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 08:21:25PM +0100, Roger Leigh wrote: On Tue, Apr 12, 2011 at 09:07:48PM +0200, Jan Hauke Rahm wrote: On Tue, Apr 12, 2011 at 07:47:35PM +0100, Roger Leigh wrote: With the patch as it stands at present, RAMRUN is deprecated. /run is always a tmpfs; RUN_SIZE will set its size, as before. Hmm, just thinking... vServers don't really allow mounting AFAIK as that would be the host's job. Wouldn't making /run a tmpfs in every case conflict here? Or am I jumping around nonsense now...? I know little about vservers. How do they currently deal with /dev/shm and /lib/init/rw? Interesting question. Actually, in my setup, I don't see /dev/shm at all and /lib/init/rw is a regular directory. Hauke -- .''`. Jan Hauke Rahm j...@debian.org www.jhr-online.de : :' : Debian Developer www.debian.org `. `'` Member of the Linux Foundationwww.linux.com `- Fellow of the Free Software Foundation Europe www.fsfe.org signature.asc Description: Digital signature
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 08:19:59PM +0100, Roger Leigh wrote: On Tue, Apr 12, 2011 at 07:44:54AM -0700, Steve Langasek wrote: If the problem is that multiple tmpfs are mounted and each can expand to half-of-RAM, either reduce the number of tmpfses presented (as discussed), or limit the *whole* allocation for known mount points to half-of-RAM and partition appropriately, or both. For this reason, I've adapted the patch to move /dev/shm to /run/shm; it's configurable whether this is a separate tmpfs mount, or simply a subdirectory of /run, and the size is also configurable as before (SHM_SIZE, with RAMSHM as the option to toggle the mounting). We could additionally allow /tmp to be moved under /run/tmp, so that all existing tmpfs mounts could share a single tmpfs (I haven't done this yet). Currently we mount a tmpfs on /tmp if RAMTMP=yes or root is mounted read-only, but we could move it under /run. I should add here that while the other distributions have moved /var/run and /var/lock under /run, they have not moved /dev/shm or /tmp. I'm not sure what the consensus is on doing either of these, so I haven't put either into the proposed patch yet. I think the question to answer here is whether or not /dev/shm and /tmp offer the same lifetime policies as /var/run and /var/lock, and whether or not they logically fit under the same heirarchy. The first is clearly the case: they can all be on tmpfs. Whether they logically fit is not so clear. I think that if we have /run/lock, /run/shm makes sense (how different are locks and POSIX semaphores? They are just a different type of lock (broadly). And shared memory is ephemeral state, just like samba's state etc.). So I would argue that it does fit. But this isn't a universally held opinion. Is there any rationale against doing this? I'm not as sure about /run/tmp, though all the files under /run are strictly temporary, they are pretty much all system files rather than being owned by users (though /run/lock and /run/shm would be user-writable; however, there are proposals to restrict access to /lock as on Fedora). Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
]] Roger Leigh | I think that if we have /run/lock, /run/shm makes sense (how different | are locks and POSIX semaphores? They are just a different type of | lock (broadly). And shared memory is ephemeral state, just like | samba's state etc.). So I would argue that it does fit. But this | isn't a universally held opinion. Is there any rationale against | doing this? /dev/shm is POSIX shared memory, not just semaphores, afaik? | I'm not as sure about /run/tmp, though all the files under /run | are strictly temporary, they are pretty much all system files | rather than being owned by users (though /run/lock and /run/shm | would be user-writable; however, there are proposals to restrict | access to /lock as on Fedora). I'd rather not make /tmp a tmpfs, and there's no reason for it to live under /run. It might also reasonably be namespaced for different users and so on, so moving it somewhere else than /tmp is, IMO, silly. -- Tollef Fog Heen UNIX is user friendly, it's just picky about who its friends are -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/8739lns05d@qurzaw.varnish-software.com
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
On 2011-04-12, Roger Leigh rle...@codelibre.net wrote: Having multiple tmpfses with the kernel defaults means that a user or badly written program could intentionally or accidentally lock up the machine by using all available memory by filling up one or more of the tmpfses. And the majority /are/ user writable by default, even /run (via /var/lock, which is not a separate mount by default--maybe it should be?). /dev/shm is user writable, /tmp is user writable. How is that different from lock-ups due to fork bombs? If the admin cares, he can still fence his users. (Like DSA do on their machines by setting a sane tmpfs size limit.) Kind regards Philipp Kern -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/slrniq9d38.phd.tr...@kelgar.0x539.de
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
Hi there! On Tue, 12 Apr 2011 20:47:35 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 08:12:21PM +0200, Luca Capello wrote: On Tue, 12 Apr 2011 13:38:03 +0200, Roger Leigh wrote: Josh Triplett suggested that we could use a single tmpfs on /run and have the rest as symlinks into /run, with potentially a separate tmpfs for user-writable filesystems to prevent a user DoS. This idea does have merit, and we could make it the default. We currently do this for /var/lock (/run/lock), which can be mounted as a separate ^^ tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. ^ Do you mean that the meaning of RAMLOCK has completely changed? Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). ^^^ [...] I consider completely changing it a serious bug, may I suggest deprecating it completely and adding a new variable instead? I guess the same should be applied to RAMRUN, i.e. simply deprecate it. With the patch as it stands at present, RAMRUN is deprecated. /run is always a tmpfs; RUN_SIZE will set its size, as before. Fair enough and, FWIW, fully agree. RAMLOCK is unchanged, except for the fact that it's mounted on /run/lock rather than /var/lock. No, /var/lock changed *substantially*, given that it is now *by default* (on) a tmpfs, regardless if RAMLOCK is set or not. Which means that RAMLOCK as it was explained is useless, thus my question about your words underlined above: which can be mounted as a separate tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. I do not have any idea which variable could be straightforward, something like LOCK_OWN_TMPFS... Likewise, LOCK_SIZE is unchanged in its meaning. Again, I found it changed: how can you define LOCK_SIZE if /run/lock is on the same tmpfs than /run? We could introduce new variables for /run such as RUNLOCK, but given that it does exactly what it used to do, I don't think it gains us anything. Fully agree, and FWIW it was not what I was asking for ;-) Thx, bye, Gismo / Luca pgpSzEfAuUZ7U.pgp Description: PGP signature
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 10:08:30PM +0200, Tollef Fog Heen wrote: ]] Roger Leigh | I think that if we have /run/lock, /run/shm makes sense (how different | are locks and POSIX semaphores? They are just a different type of | lock (broadly). And shared memory is ephemeral state, just like | samba's state etc.). So I would argue that it does fit. But this | isn't a universally held opinion. Is there any rationale against | doing this? /dev/shm is POSIX shared memory, not just semaphores, afaik? Yes, it's used for both. | I'm not as sure about /run/tmp, though all the files under /run | are strictly temporary, they are pretty much all system files | rather than being owned by users (though /run/lock and /run/shm | would be user-writable; however, there are proposals to restrict | access to /lock as on Fedora). I'd rather not make /tmp a tmpfs, and there's no reason for it to live under /run. It might also reasonably be namespaced for different users and so on, so moving it somewhere else than /tmp is, IMO, silly. One reason for doing this is to have a single writable mount on the system, which might be useful for tiny systems with minimal resources, where root is r/o. On such a system, it might be useful to pool the limited writable space (which might not be a tmpfs). This isn't a common use case, but one of the considerations in the patch is to make read-only root possible, and since one of the changes is to automatically mount a tmpfs on /tmp if root is read-only, I thought it was worth asking the question if it could be shared with the existing tmpfs filesystems on the system. While a tmpfs on /tmp isn't going to be the default, the patch does add support for it (RAMTMP). Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: [Pkg-samba-maint] Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 08:22:00PM +, Philipp Kern wrote: On 2011-04-12, Roger Leigh rle...@codelibre.net wrote: Having multiple tmpfses with the kernel defaults means that a user or badly written program could intentionally or accidentally lock up the machine by using all available memory by filling up one or more of the tmpfses. And the majority /are/ user writable by default, even /run (via /var/lock, which is not a separate mount by default--maybe it should be?). /dev/shm is user writable, /tmp is user writable. How is that different from lock-ups due to fork bombs? If the admin cares, he can still fence his users. (Like DSA do on their machines by setting a sane tmpfs size limit.) It's something which is entirely preventable, and while it's possible for sysadmins to set the limits to something sane, I would really like to have something sane by default when this is possible. And for some of the filesystems in question, this is totally safe to do. Others like /var/run do vary somewhat more, but it should still be possible to do better than existing practice. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Re: Default size limits for /run (/var/run) and /run/lock (/var/lock)
On Tue, Apr 12, 2011 at 10:35:37PM +0200, Luca Capello wrote: Hi there! On Tue, 12 Apr 2011 20:47:35 +0200, Roger Leigh wrote: On Tue, Apr 12, 2011 at 08:12:21PM +0200, Luca Capello wrote: On Tue, 12 Apr 2011 13:38:03 +0200, Roger Leigh wrote: Josh Triplett suggested that we could use a single tmpfs on /run and have the rest as symlinks into /run, with potentially a separate tmpfs for user-writable filesystems to prevent a user DoS. This idea does have merit, and we could make it the default. We currently do this for /var/lock (/run/lock), which can be mounted as a separate ^^ tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. ^ Do you mean that the meaning of RAMLOCK has completely changed? Currently, `man rcS` gives: RAMLOCK Make /var/lock/ available as a ram file system (tmpfs). ^^^ [...] I consider completely changing it a serious bug, may I suggest deprecating it completely and adding a new variable instead? I guess the same should be applied to RAMRUN, i.e. simply deprecate it. With the patch as it stands at present, RAMRUN is deprecated. /run is always a tmpfs; RUN_SIZE will set its size, as before. Fair enough and, FWIW, fully agree. RAMLOCK is unchanged, except for the fact that it's mounted on /run/lock rather than /var/lock. No, /var/lock changed *substantially*, given that it is now *by default* (on) a tmpfs, regardless if RAMLOCK is set or not. Which means that RAMLOCK as it was explained is useless, thus my question about your words underlined above: which can be mounted as a separate tmpfs on /run/lock if RAMLOCK is set in /etc/defaults/rcS. I do not have any idea which variable could be straightforward, something like LOCK_OWN_TMPFS... Likewise, LOCK_SIZE is unchanged in its meaning. Again, I found it changed: how can you define LOCK_SIZE if /run/lock is on the same tmpfs than /run? If you set RAMLOCK=yes, then /run/lock is a *separate* tmpfs of size LOCK_SIZE, *exactly* like the /var/lock tmpfs mount (it's the same code with s/var/run/g). So the actual behaviour of this option is entirely unchanged bar the switch from /var/lock to /run/lock. So by default, /run and /run/lock are on the same tmpfs. But if you set RAMLOCK=yes, you'll get a second tmpfs mounted on /run/lock. So yes, /run/lock will always be on a tmpfs filesystem, that's obviously the main point of the patch. But the sysadmin can configure the size of /run/lock separately should they choose to do so (especially since it's user-writable by default). Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
Default size limits for /run (/var/run) and /run/lock (/var/lock)
With the transition to /run and /run/lock as tmpfs filesystems, it would be desirable to provide sensible default size limits. Currently, we default to the tmpfs default of ½ RAM. But with several tmpfs filesystems, this does have the potential for the system to be OOMed by a user filling up more than one of the filesystems. A sensible default size limit would be useful here, for at least some of the filesystems. We currently allow specification of size limits for: /run (RUN_SIZE) /run/lock (LOCK_SIZE, optional) /dev/shm (SHM_SIZE) /tmp (TMP_SIZE, optional) [from temporary git repo at http://git.debian.org/?p=collab-maint/sysvinit;a=summary] In order to default to a sensible size, I would appreciate it if I could have some figures for the useage of these filesystems: e.g. % sudo du -sk /var/run /var/lock /dev/shm for a variety of systems (desktop, server, large samba setup, very large and busy systems, minimal environments etc.). We want to be sure the default is sane for all but the most extreme outliers, where the sysadmin would need to configure a bigger value as the default. A sensible size for /tmp would also be useful, but this obviously varies widely. Note this is for a tmpfs /tmp, which is not mounted by default (only defaults if root is read-only). Thanks for any feedback (any stats can be sent directly to me to avoid flooding the list). I'd just like the output of the above command, plus a small description of what the system is for (size, major services etc.) for context. Regards, Roger -- .''`. Roger Leigh : :' : Debian GNU/Linux http://people.debian.org/~rleigh/ `. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/ `-GPG Public Key: 0x25BFB848 Please GPG sign your mail. signature.asc Description: Digital signature
