Re: Lets (eventually) find a good solution for /tmp
Don is right. I'd like to step up a level and think about the situation. The real issue here is that having /tmp be just another directory in a writable partition filesystem, like / or /home or whatever, means that /tmp gets all the associated properties. One such property (A) is that files can be large and can bleed off to disk smoothly without thrashing the system, both of which are problems with tmpfs. On the other hand, another property (B) is that a great deal of effort is going into ensuring that files get written to disk in a timely fashion, that data and metadata survives crashes, and that structures on disk are consistent, up-to-date, and if necessary, quick to check. We want /tmp to have (A), but it need not have (B). So this is a technical question. Can we implement a filesystem which provides properties A, but which (C) has zero data surviving after a crash or umount, and takes advantage of property C to make things faster. This seems like a much easier problem than the problems that journaling filesystems and their ilk solve. But even though it may be an easy problem, to my knowledge, no one has implemented a solution. IDEAS One thought would be to make a fuse filesystem which keeps small files in the fuse process itself (like tmpfs) and puts big files on a big filesystem but *unlinks* the files so their only reference is via an open handle held by the fuse process itself. The files are thus backed on disk as usual, and can be large, but when the fuse process dies *bam* they're gone, and the usual recovery mechanisms scavenge their space after a crash. Another thought would be to hack tmpfs to keep only big files in some backing filesystem, perhaps using handles to unlinked files as above, thus having the advantages of tmpfs for small files while also handling large files well. --Barak. -- Barak A. Pearlmutter Hamilton Institute Dept Comp Sci, NUI Maynooth, Co. Kildare, Ireland http://www.bcl.hamilton.ie/~barak/ -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/E1SeSpB-00060c-Cg@port-kdr.hamilton.local
Re: Lets (eventually) find a good solution for /tmp
2012/6/12 Barak A. Pearlmutter wrote: The real issue here is that having /tmp be just another directory in a writable partition filesystem, like / or /home or whatever, means that /tmp gets all the associated properties. One such property (A) is that files can be large and can bleed off to disk smoothly without thrashing the system, both of which are problems with tmpfs. On the other hand, another property (B) is that a great deal of effort is going into ensuring that files get written to disk in a timely fashion, that data and metadata survives crashes, and that structures on disk are consistent, up-to-date, and if necessary, quick to check. We want /tmp to have (A), but it need not have (B). Ext3 has (A) and does not have (B). :) You don't think that writing some bytes of ext3 journal is a great deal of effort for /tmp, do you? ;) So this is a technical question. Can we implement a filesystem which provides properties A, but which (C) has zero data surviving after a crash or umount, and takes advantage of property C to make things faster. Whatever filesystem you invent, you'll need to recreate it as empty on every boot. This is not going to be much faster than replaying ext3 journal. And since you're replaying journal for your root partition anyway, keeping /tmp on your root partition is the fastest thing you can get, I guess. :) But even though it may be an easy problem, to my knowledge, no one has implemented a solution. Probably that's because nobody really needed such a filesystem yet. :) IDEAS One thought would be to make a fuse filesystem [...] Another thought would be to hack tmpfs [...] BTW, what do you need all this for? I mean, it's an interesting idea about inventing a special limited version of tmpfs. I was thinking about that myself (e.g. aufs-mount with writable branches on tmpfs and disk), it's quite possible to implement. There're even fuse-filesystems partially implementing this feature. But how are you going to use it? How can you check that this new InventedFS is better than plain ext3? What do you want to use it for? I mean, there're plenty of different filesystems already, if you describe some use-case, or some problem to solve, then maybe, I could find a filesystem that will help you? For example if you need it to be fast, you can forget about fuse, I can suggest you to look at aufs3 patches. Hacking tmpfs won't be easy, unless you're good at understanding kernel/vfs code. -- Serge -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/caoveneobqcovjsvretryzjd3m_yn3bh378jrjsjgsoqe5tg...@mail.gmail.com
Re: Lets (eventually) find a good solution for /tmp
On 10 June 2012 19:31, Thomas Goirand z...@debian.org wrote: On 06/11/2012 12:06 AM, Don Armstrong wrote: swap file on / [...] is really the direction that we should be going NO ! Does this need to be explained? :/ Not quite sure what you're objecting to. If you are against the use of swap files rather than swap partitions then yes, it does need to be explained, because as far as I am aware a swap file is the better choice in virtually all situations (and is what I've been using exclusively since Linux 2.6 removed the downsides). If in fact there are any remaining downsides I'm unaware of, it would be good to see them documented. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAHb+SPC=jww-faxu_uc9xsotiy3p69wudznfwemrog0y983...@mail.gmail.com
Re: Lets (eventually) find a good solution for /tmp
On 11/06/12 15:01, Aneurin Price wrote: as far as I am aware a swap file is the better choice in virtually all situations Assuming http://wiki.debian.org/Hibernation/Hibernate_Without_Swap_Partition is still current: If you want to use hibernation (suspend-to-disk), you need roughly[0] as much partion-based swap as you have RAM, unless you're using the setup on that wiki page (using uswsusp, which is not installed by default; have allocated a contiguous swap file; have made sure to use a filesystem which will not move that file; done some dangerous[1] manual configuration for uswsusp). By default, GNOME's gnome-power-manager will just suspend (suspend-to-RAM) under normal conditions; but if your laptop battery gets critically low, it will automatically hibernate, because that's the only way it can preserve what you were working on across total power failure (i.e. not enough battery to keep your RAM alive). I think this is a sensible approach. S [0] I believe the hibernation image is compressed, and doesn't include disk cache, so you only need as much swap as you have real data in RAM [1] dangerous as in if you got the numbers wrong, you will trash arbitrary parts of your filesystem next time you hibernate, as far as I can see -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fd5fef0.9080...@debian.org
Re: Lets (eventually) find a good solution for /tmp
On Monday 11 June 2012 16:01:10 Aneurin Price wrote: On 10 June 2012 19:31, Thomas Goirand z...@debian.org wrote: On 06/11/2012 12:06 AM, Don Armstrong wrote: swap file on / [...] is really the direction that we should be going NO ! Does this need to be explained? :/ Hi, Not quite sure what you're objecting to. If you are against the use of swap files rather than swap partitions then yes, it does need to be explained, because as far as I am aware a swap file is the better choice in virtually all situations (and is what I've been using exclusively since Linux 2.6 removed the downsides). data point: a swap file would not work with btrfs [1]. [1]https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F -- pub 4096R/0E4BD0AB people.fccf.net/danchev/key pgp.mit.edu -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201206111630.53595.danc...@spnet.net
Re: Lets (eventually) find a good solution for /tmp
On 11 June 2012 15:21, Simon McVittie s...@debian.org wrote: On 11/06/12 15:01, Aneurin Price wrote: as far as I am aware a swap file is the better choice in virtually all situations Assuming http://wiki.debian.org/Hibernation/Hibernate_Without_Swap_Partition is still current: If you want to use hibernation (suspend-to-disk), you need roughly[0] as much partion-based swap as you have RAM, unless you're using the setup on that wiki page (using uswsusp, which is not installed by default; have allocated a contiguous swap file; have made sure to use a filesystem which will not move that file; done some dangerous[1] manual configuration for uswsusp). snip Thanks. that's useful information. It's not relevant to me personally which is why I was unaware of it, but certainly that would count as a good reason not to use a file by default (at least until that limitation is overcome). -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/CAHb+SPDNnZkSHcuOm6w2de=NuBt_jrSUFEisRaR+_0MMcQ=w...@mail.gmail.com
Re: Lets (eventually) find a good solution for /tmp
On Mon, 11 Jun 2012, Thomas Goirand wrote: On 06/11/2012 12:06 AM, Don Armstrong wrote: swap file on / [...] is really the direction that we should be going NO ! Some imprecise language on my part has apparently lead to some misunderstanding of what I am suggesting. I don't mean a swap file that gets used as swap. I meant file(s) which are on / (or some other large filesystem if / is not large) which are used to handle /tmp growing beyond the bounds of the memory assigned to it. These file(s) should be orthogonal to the current memory management system. [I know that using a swap file can currently do this to some extent, but it has other problems.] Don Armstrong -- Maybe I did steal your heart and I am such a perfect criminal that you never noticed -- a softer world #481 http://www.asofterworld.com/index.php?id=481 http://www.donarmstrong.com http://rzlab.ucr.edu -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120611171326.gp32...@rzlab.ucr.edu
Re: Lets (eventually) find a good solution for /tmp
On 06/11/2012 12:06 AM, Don Armstrong wrote: swap file on / [...] is really the direction that we should be going NO ! Does this need to be explained? :/ Thomas -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/4fd4e808.80...@debian.org
Re: Lets (eventually) find a good solution for /tmp
On Mon, Jun 11, 2012 at 02:31:36AM +0800, Thomas Goirand wrote: On 06/11/2012 12:06 AM, Don Armstrong wrote: swap file on / [...] is really the direction that we should be going NO ! Does this need to be explained? :/ Perhaps? Please point me at the msg-id of the explanation if I missed it. -- To UNSUBSCRIBE, email to debian-devel-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120610184646.GB16293@debian
