Re: [PATCH 0/2] LogFS take two
On Tue, 8 May 2007 01:53:38 -0400, Albert Cahalan wrote: > > You seem to be missing the immutable bit. This is really useful > for dealing with buggy or badly-designed things running as root. > I've used to to protect /dev/null from becoming a normal file > filled with junk, and to protect /etc/resolv.conf from "helpful" > network management daemons that don't know my DNS servers. Sounds useful. Onto my todo list. And since the list is slowly getting too long to be memorized, I've added it to my wiki: http://logfs.org/logfs/todo > Anything else missing? > > BTW, BSD offers an unprivileged immutable bit as well. I'm sure > it's useful for the apps that trash their own config files. > Actually, this bit alone would do fine, and we could really use > a way to protect writable device files from deletion or permission > bit changes. It would be relatively easy to add this as well. The biggest obstacle I see is getting support in chattr(1). Adding Ted to Cc:, as he is the maintainer. What remains to be decided is whether such a flag is a useful addition. My gut feeling is yes, but I would like to have more than two votes in favor. Jörn -- Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it. -- Perlis's Programming Proverb #58, SIGPLAN Notices, Sept. 1982 - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/2] LogFS take two
On Tue, 8 May 2007 09:39:37 +0200, Thomas Gleixner wrote: > > > Motivation 2: > > > > Flash is becoming increasingly common in standard PC hardware. Nearly > > a dozen different manufacturers have announced Solid State Disks > > (SSDs), the OLPC and the Intel Classmate no longer contain hard disks > > and ASUS announced a flash-only Laptop series for regular consumers. > > With a hardware controller which allows no direct access to the flash. > > > And that doesn't even mention the ubiquitous USB-Sticks, SD-Cards, > > etc. > > which again do not allow direct access to the flash I know that and I have talked to manufacturers. Not allowing direct access is common practice today, but I didn't encounter much opposition against allowing it in the future. What appears to be holding them back is that there would be absolutely no value in it right now. With direct flash access, which filesystem should users choose for their 32GB SSD? > > Flash behaves significantly different to hard disks. In order to use > > flash, the current standard practice is to add an emulation layer and > > an old-fashioned hard disk filesystem. As can be expected, this is > > eating up some of the benefits flash can offer over hard disks. > > > > In principle it is possible to achieve better performance with a flash > > filesystem than with the current emulated approach. > > Err, where does JFFS2 use a block emulation layer ? It doesn't. Motivation 2 is about SSDs, USB sticks, SD-Cards, etc. JFFS2 is motivation 1. > Are you going to make logfs play with UBI ? It is not very high on my priority list. > > Handling of read/write/erase errors currently is BUG(). It is on my > > list, no need to remind me. :) > > > > Overall I consider this to be -mm material. > > I don't. It seems fs developers tend to have their own view of how to > get stuff mainline. Maybe. My view is that I have to solve any problems found until people consider the code good enough by whatever metric. The final criterium appears to be quite fuzzy. > The code is far from being useful on real world hardware. The error > handling via BUG() is just making it useless. On NOR hardware? How many write/erase failures does one commonly encounter there? Those things will need to get sorted, sure. But I doubt whether LogFS is useless on _all_ hardware because of this. > Also please fix the coding style and other issues from the seperate > review. Sure. > Some useful comments would make a functional review way easier. Common problem. Implementor doesn't know what comments would be useful and reviewer doesn't know where to start without useful comments. I will try to add some and would love to see suggestions. > > It would be good to get > > some review and have the usual allyesconfig crowd build it > > make allyesconfig does not work for you ? It does. But I don't have a coverity license, just to give one example. Jörn -- The wise man seeks everything in himself; the ignorant man tries to get everything from somebody else. -- unknown - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/2] LogFS take two
On Mon, 2007-05-07 at 23:59 +0200, Jörn Engel wrote: > LogFS has an on-medium tree, fairly similar to Ext2 in structure, so > mount times are O(1). In absolute terms, the OLPC system has mount > times of ~3.3s for JFFS2 and ~60ms for LogFS. Impressive number > Motivation 2: > > Flash is becoming increasingly common in standard PC hardware. Nearly > a dozen different manufacturers have announced Solid State Disks > (SSDs), the OLPC and the Intel Classmate no longer contain hard disks > and ASUS announced a flash-only Laptop series for regular consumers. With a hardware controller which allows no direct access to the flash. > And that doesn't even mention the ubiquitous USB-Sticks, SD-Cards, > etc. which again do not allow direct access to the flash > Flash behaves significantly different to hard disks. In order to use > flash, the current standard practice is to add an emulation layer and > an old-fashioned hard disk filesystem. As can be expected, this is > eating up some of the benefits flash can offer over hard disks. > > In principle it is possible to achieve better performance with a flash > filesystem than with the current emulated approach. Err, where does JFFS2 use a block emulation layer ? > Current state: > > LogFS works and survives my testcases. It has fairly good chances of > not eating your data during regular operation. There are still two > known bugs that will eat data if the filesystem is uncleanly > unmounted. Also still missing is wear leveling. Are you going to make logfs play with UBI ? > Handling of read/write/erase errors currently is BUG(). It is on my > list, no need to remind me. :) > > Overall I consider this to be -mm material. I don't. It seems fs developers tend to have their own view of how to get stuff mainline. The code is far from being useful on real world hardware. The error handling via BUG() is just making it useless. Also please fix the coding style and other issues from the seperate review. Some useful comments would make a functional review way easier. > It would be good to get > some review and have the usual allyesconfig crowd build it make allyesconfig does not work for you ? tglx - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/2] LogFS take two
On Mon, 2007-05-07 at 23:59 +0200, Jörn Engel wrote: LogFS has an on-medium tree, fairly similar to Ext2 in structure, so mount times are O(1). In absolute terms, the OLPC system has mount times of ~3.3s for JFFS2 and ~60ms for LogFS. Impressive number Motivation 2: Flash is becoming increasingly common in standard PC hardware. Nearly a dozen different manufacturers have announced Solid State Disks (SSDs), the OLPC and the Intel Classmate no longer contain hard disks and ASUS announced a flash-only Laptop series for regular consumers. With a hardware controller which allows no direct access to the flash. And that doesn't even mention the ubiquitous USB-Sticks, SD-Cards, etc. which again do not allow direct access to the flash Flash behaves significantly different to hard disks. In order to use flash, the current standard practice is to add an emulation layer and an old-fashioned hard disk filesystem. As can be expected, this is eating up some of the benefits flash can offer over hard disks. In principle it is possible to achieve better performance with a flash filesystem than with the current emulated approach. Err, where does JFFS2 use a block emulation layer ? Current state: LogFS works and survives my testcases. It has fairly good chances of not eating your data during regular operation. There are still two known bugs that will eat data if the filesystem is uncleanly unmounted. Also still missing is wear leveling. Are you going to make logfs play with UBI ? Handling of read/write/erase errors currently is BUG(). It is on my list, no need to remind me. :) Overall I consider this to be -mm material. I don't. It seems fs developers tend to have their own view of how to get stuff mainline. The code is far from being useful on real world hardware. The error handling via BUG() is just making it useless. Also please fix the coding style and other issues from the seperate review. Some useful comments would make a functional review way easier. It would be good to get some review and have the usual allyesconfig crowd build it make allyesconfig does not work for you ? tglx - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/2] LogFS take two
On Tue, 8 May 2007 09:39:37 +0200, Thomas Gleixner wrote: Motivation 2: Flash is becoming increasingly common in standard PC hardware. Nearly a dozen different manufacturers have announced Solid State Disks (SSDs), the OLPC and the Intel Classmate no longer contain hard disks and ASUS announced a flash-only Laptop series for regular consumers. With a hardware controller which allows no direct access to the flash. And that doesn't even mention the ubiquitous USB-Sticks, SD-Cards, etc. which again do not allow direct access to the flash I know that and I have talked to manufacturers. Not allowing direct access is common practice today, but I didn't encounter much opposition against allowing it in the future. What appears to be holding them back is that there would be absolutely no value in it right now. With direct flash access, which filesystem should users choose for their 32GB SSD? Flash behaves significantly different to hard disks. In order to use flash, the current standard practice is to add an emulation layer and an old-fashioned hard disk filesystem. As can be expected, this is eating up some of the benefits flash can offer over hard disks. In principle it is possible to achieve better performance with a flash filesystem than with the current emulated approach. Err, where does JFFS2 use a block emulation layer ? It doesn't. Motivation 2 is about SSDs, USB sticks, SD-Cards, etc. JFFS2 is motivation 1. Are you going to make logfs play with UBI ? It is not very high on my priority list. Handling of read/write/erase errors currently is BUG(). It is on my list, no need to remind me. :) Overall I consider this to be -mm material. I don't. It seems fs developers tend to have their own view of how to get stuff mainline. Maybe. My view is that I have to solve any problems found until people consider the code good enough by whatever metric. The final criterium appears to be quite fuzzy. The code is far from being useful on real world hardware. The error handling via BUG() is just making it useless. On NOR hardware? How many write/erase failures does one commonly encounter there? Those things will need to get sorted, sure. But I doubt whether LogFS is useless on _all_ hardware because of this. Also please fix the coding style and other issues from the seperate review. Sure. Some useful comments would make a functional review way easier. Common problem. Implementor doesn't know what comments would be useful and reviewer doesn't know where to start without useful comments. I will try to add some and would love to see suggestions. It would be good to get some review and have the usual allyesconfig crowd build it make allyesconfig does not work for you ? It does. But I don't have a coverity license, just to give one example. Jörn -- The wise man seeks everything in himself; the ignorant man tries to get everything from somebody else. -- unknown - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/2] LogFS take two
On Tue, 8 May 2007 01:53:38 -0400, Albert Cahalan wrote: You seem to be missing the immutable bit. This is really useful for dealing with buggy or badly-designed things running as root. I've used to to protect /dev/null from becoming a normal file filled with junk, and to protect /etc/resolv.conf from helpful network management daemons that don't know my DNS servers. Sounds useful. Onto my todo list. And since the list is slowly getting too long to be memorized, I've added it to my wiki: http://logfs.org/logfs/todo Anything else missing? BTW, BSD offers an unprivileged immutable bit as well. I'm sure it's useful for the apps that trash their own config files. Actually, this bit alone would do fine, and we could really use a way to protect writable device files from deletion or permission bit changes. It would be relatively easy to add this as well. The biggest obstacle I see is getting support in chattr(1). Adding Ted to Cc:, as he is the maintainer. What remains to be decided is whether such a flag is a useful addition. My gut feeling is yes, but I would like to have more than two votes in favor. Jörn -- Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it. -- Perlis's Programming Proverb #58, SIGPLAN Notices, Sept. 1982 - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/2] LogFS take two
[EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED], linux-kernel@vger.kernel.org, [EMAIL PROTECTED], [EMAIL PROTECTED] Re: [PATCH 0/2] LogFS take two You seem to be missing the immutable bit. This is really useful for dealing with buggy or badly-designed things running as root. I've used to to protect /dev/null from becoming a normal file filled with junk, and to protect /etc/resolv.conf from "helpful" network management daemons that don't know my DNS servers. Anything else missing? BTW, BSD offers an unprivileged immutable bit as well. I'm sure it's useful for the apps that trash their own config files. Actually, this bit alone would do fine, and we could really use a way to protect writable device files from deletion or permission bit changes. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/2] LogFS take two
Motivation: Linux currently has 1-2 flash filesystems to choose from, JFFS2 and YAFFS. The latter has never made a serious attempt of kernel integration, which may disqualify it to some. The two main problems of JFFS2 are memory consumption and mount time. Unlike most filesystems, there is no tree structure of any sorts on the medium, so the complete medium needs to be scanned at mount time and a tree structure kept in-memory while the filesystem is mounted. With bigger devices, both mount time and memory consumption increase linearly. JFFS2 has recently gained summary support, which helps reduce mount time by a constant factor. Linear scalability remains. YAFFS also appears to be better by a constant factor, yet still scales linearly. LogFS has an on-medium tree, fairly similar to Ext2 in structure, so mount times are O(1). In absolute terms, the OLPC system has mount times of ~3.3s for JFFS2 and ~60ms for LogFS. Motivation 2: Flash is becoming increasingly common in standard PC hardware. Nearly a dozen different manufacturers have announced Solid State Disks (SSDs), the OLPC and the Intel Classmate no longer contain hard disks and ASUS announced a flash-only Laptop series for regular consumers. And that doesn't even mention the ubiquitous USB-Sticks, SD-Cards, etc. Flash behaves significantly different to hard disks. In order to use flash, the current standard practice is to add an emulation layer and an old-fashioned hard disk filesystem. As can be expected, this is eating up some of the benefits flash can offer over hard disks. In principle it is possible to achieve better performance with a flash filesystem than with the current emulated approach. In practice our current flash filesystems are not even near that theoretical goal. LogFS in its current state is already closer. Current state: LogFS works and survives my testcases. It has fairly good chances of not eating your data during regular operation. There are still two known bugs that will eat data if the filesystem is uncleanly unmounted. Also still missing is wear leveling. Handling of read/write/erase errors currently is BUG(). It is on my list, no need to remind me. :) Overall I consider this to be -mm material. It would be good to get some review and have the usual allyesconfig crowd build it and find coverity bugs and the like. http://logfs.org/logfs/ may have some further information. Shameless plug: I have quit my job last November to concentrate on LogFS. While I have found one sponsor kind enough to fund me, my monetary reserves are fairly stressed. Fairly soon I will be forced to take an old-fashioned job again and work on other less exciting stuff. So if anyone needs a fast flash filesystem and has spare money to spend, please contact me. Jörn -- Everything should be made as simple as possible, but not simpler. -- Albert Einstein - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 0/2] LogFS take two
Motivation: Linux currently has 1-2 flash filesystems to choose from, JFFS2 and YAFFS. The latter has never made a serious attempt of kernel integration, which may disqualify it to some. The two main problems of JFFS2 are memory consumption and mount time. Unlike most filesystems, there is no tree structure of any sorts on the medium, so the complete medium needs to be scanned at mount time and a tree structure kept in-memory while the filesystem is mounted. With bigger devices, both mount time and memory consumption increase linearly. JFFS2 has recently gained summary support, which helps reduce mount time by a constant factor. Linear scalability remains. YAFFS also appears to be better by a constant factor, yet still scales linearly. LogFS has an on-medium tree, fairly similar to Ext2 in structure, so mount times are O(1). In absolute terms, the OLPC system has mount times of ~3.3s for JFFS2 and ~60ms for LogFS. Motivation 2: Flash is becoming increasingly common in standard PC hardware. Nearly a dozen different manufacturers have announced Solid State Disks (SSDs), the OLPC and the Intel Classmate no longer contain hard disks and ASUS announced a flash-only Laptop series for regular consumers. And that doesn't even mention the ubiquitous USB-Sticks, SD-Cards, etc. Flash behaves significantly different to hard disks. In order to use flash, the current standard practice is to add an emulation layer and an old-fashioned hard disk filesystem. As can be expected, this is eating up some of the benefits flash can offer over hard disks. In principle it is possible to achieve better performance with a flash filesystem than with the current emulated approach. In practice our current flash filesystems are not even near that theoretical goal. LogFS in its current state is already closer. Current state: LogFS works and survives my testcases. It has fairly good chances of not eating your data during regular operation. There are still two known bugs that will eat data if the filesystem is uncleanly unmounted. Also still missing is wear leveling. Handling of read/write/erase errors currently is BUG(). It is on my list, no need to remind me. :) Overall I consider this to be -mm material. It would be good to get some review and have the usual allyesconfig crowd build it and find coverity bugs and the like. http://logfs.org/logfs/ may have some further information. Shameless plug: I have quit my job last November to concentrate on LogFS. While I have found one sponsor kind enough to fund me, my monetary reserves are fairly stressed. Fairly soon I will be forced to take an old-fashioned job again and work on other less exciting stuff. So if anyone needs a fast flash filesystem and has spare money to spend, please contact me. Jörn -- Everything should be made as simple as possible, but not simpler. -- Albert Einstein - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 0/2] LogFS take two
[EMAIL PROTECTED], [EMAIL PROTECTED], [EMAIL PROTECTED], linux-kernel@vger.kernel.org, [EMAIL PROTECTED], [EMAIL PROTECTED] Re: [PATCH 0/2] LogFS take two You seem to be missing the immutable bit. This is really useful for dealing with buggy or badly-designed things running as root. I've used to to protect /dev/null from becoming a normal file filled with junk, and to protect /etc/resolv.conf from helpful network management daemons that don't know my DNS servers. Anything else missing? BTW, BSD offers an unprivileged immutable bit as well. I'm sure it's useful for the apps that trash their own config files. Actually, this bit alone would do fine, and we could really use a way to protect writable device files from deletion or permission bit changes. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
