Re: [PATCH 06/14] Pramfs: Include files
2009/6/22 Arnd Bergmann : > On Monday 22 June 2009, Marco wrote: >> >> Sorry, I meant it's not currently possible. At the moment the only way >> to use it as rootfs it's to copy all the data in an already mounted >> (empty) ram partition and reboot. However it's not my first item on my >> todo list because I think that it's possible to use it as rootfs but it >> isn't the standard use for this fs. > > Well, it doesn't have to work right away. What I'm asking to > define the data structures in a way that keeps the layout stable > across kernel updates. Since a future version of the file system > might support cross-endian image creation, it would be good to > define the data structures in a fixed endian mode already, so > you don't have to change it in the future. > > Arnd <>< > I was refering to create a tool "mkpramfs". Marco -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
On Mon, 22 June 2009 23:20:39 +0100, David Woodhouse wrote: > On Mon, 2009-06-22 at 23:41 +0200, Jörn Engel wrote: > > Four loops doing the same increment with different data types: long, > > u64, we32 (wrong-endian) and we64. Compile with _no_ optimizations. > > That's a bit of a poor test then. Especially on architectures with a > load-and-swap instruction where it really shouldn't be any slower at > all. > > (Although since GCC doesn't have an __attribute__((littleendian)) I'm > not entirely sure how to entice it into _using_ said instruction for the > purpose of the test... I think the kernel does manage somehow though, if > you get the sources _just_ right.) Feel free to improve the test. It is admittedly crap and designed to support Chris' argument. But seeing that it still fails to do so and Arnd has already shown one improvement that weakened Chris' argument, I guess we can all agree that further improvments won't change the conclusion, can we? ;) Jörn -- It's just what we asked for, but not what we want! -- anonymous -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
> > > It should be possible to read a file-system on your x86 64bit > > box that you wrote with your small powerpc target. > For a (NV)RAM-based filesystem?? WTH??? dd the full image - dig into it. Usefull is you have post-mortem info there. Sam -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
On Mon, Jun 22, 2009 at 05:26:45PM -0600, Chris Friesen wrote: > Pavel Machek wrote: > > > More politely said: "I believe you would be better off modifying > > ramdisk to include the functionality for persistence." New filesystem > > should not really be neccessary. ext2 for performance, ext3 if you > > need robustness from journalling, maybe something else makes sense, > > too. > > I agree that a persistent block device makes more sense. > > However, as someone with some experience in using this type of > mechanism, a special filesystem (separate from the special block device) > could also be useful. We often use things like circular buffers, > per-cpu areas, both log-structured and variable-length records, etc. A > filesystem that would take care of this type of thing under the hood > might simplify a few things. > > Also note that it's very useful for the kernel itself to be able to > access the contents of this persistent area...we use it for parts of the > log stream, various forms of "flight recorder" information, panic > tracebacks, etc. as well as making it available to select userspace apps. I think we might step back from this particular patch and look at what people actually want. From my standpoint, we use, or intend to use, persistent memory in three ways: 1. Recording crash logs from user space 2. Recording crash logs from kernel space 3. Flight recorder/continuous logging. The first two uses share the same chunk of memory. In the first case, the interface is through filesystem operations from user space, but simple memcpys in kernel space. Neither of these uses is performance critical The third use is a different memory chunk and, depending on just what you log, can have a significant performance impact. It is helpful if this can be a single chunk of pram that is contiguous in kernel virtual space. Depending on how you use it, you might even want it to be uncached. I'm not clear on whether Pramfs supports this. All three use cases can be satisfied with a persistent memory block device, as others have mentioned, so long as multiple pram partitions are supported. We might steal the mtdparts syntax for this (see drivers/mtd/cmdlinepart.c). Another consideration--we actually stopped using a checksum on our crash logs. The crash logs are in text, so if they are a little bit stomped on, they are still usable. If you require a matching checksum, you lose the ability to read those slightly dented logs. So, all the work that's being done to ensure data integrity in the pram filesystem is actually harmful in this case. I don't know whether a pram block device helps at all with a pramfs, but I wouldn't want to lose ability to choose to use the block device directly or switch to a pramfs. > Chris -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
Pavel Machek wrote: > More politely said: "I believe you would be better off modifying > ramdisk to include the functionality for persistence." New filesystem > should not really be neccessary. ext2 for performance, ext3 if you > need robustness from journalling, maybe something else makes sense, > too. I agree that a persistent block device makes more sense. However, as someone with some experience in using this type of mechanism, a special filesystem (separate from the special block device) could also be useful. We often use things like circular buffers, per-cpu areas, both log-structured and variable-length records, etc. A filesystem that would take care of this type of thing under the hood might simplify a few things. Also note that it's very useful for the kernel itself to be able to access the contents of this persistent area...we use it for parts of the log stream, various forms of "flight recorder" information, panic tracebacks, etc. as well as making it available to select userspace apps. Chris -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
On Monday 22 June 2009, Jörn Engel wrote: > Four loops doing the same increment with different data types: long, > u64, we32 (wrong-endian) and we64. Compile with no optimizations. > > Results on my i386 notebook: > long: 453953 us > we32: 880273 us > u64: 504214 us > we64:2259953 us > loops: 1 (couldn't resist) The we64 number is artificially high because the glibc bswap_64 implementation forces the conversion to be done on the stack. Using __builtin_bswap64 make this look more logical, and makes your point even stronger (on core 2, using -m32): long: 236792 us we32: 500827 us u64: 265990 us we64: 757380 us loops: 1 Arnd <>< -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
On Mon 2009-06-22 23:57:53, Pavel Machek wrote: > On Mon 2009-06-22 14:50:01, Tim Bird wrote: > > Pavel Machek wrote: > > >> block of fast non-volatile RAM that need to access data on it using a > > >> standard filesytem interface." > > > > > > Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are > > > better with ext2. > > > > Not if you want the RAM-based filesystem to persist over a kernel > > invocation. > > Yes, you'll need to code Persistent, RAM-based _block_device_. More politely said: "I believe you would be better off modifying ramdisk to include the functionality for persistence." New filesystem should not really be neccessary. ext2 for performance, ext3 if you need robustness from journalling, maybe something else makes sense, too. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
On Mon, 2009-06-22 at 23:41 +0200, Jörn Engel wrote: > Four loops doing the same increment with different data types: long, > u64, we32 (wrong-endian) and we64. Compile with _no_ optimizations. That's a bit of a poor test then. Especially on architectures with a load-and-swap instruction where it really shouldn't be any slower at all. (Although since GCC doesn't have an __attribute__((littleendian)) I'm not entirely sure how to entice it into _using_ said instruction for the purpose of the test... I think the kernel does manage somehow though, if you get the sources _just_ right.) -- David WoodhouseOpen Source Technology Centre david.woodho...@intel.com Intel Corporation -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
Pavel Machek wrote: >> block of fast non-volatile RAM that need to access data on it using a >> standard filesytem interface." > > Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are > better with ext2. Not if you want the RAM-based filesystem to persist over a kernel invocation. = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
Pavel Machek wrote: > You are saying top goal is robustness, while the web page says (home > page, stop using frames!): > > "embedded systems have a block of non-volatile RAM seperate from > normal system memory, i.e. of which the kernel maintains no memory > page descriptors. For such systems it would be beneficial to mount a > fast read/write filesystem over this "I/O memory", for storing > frequently accessed data that must survive system reboots and power > cycles" > > Note the "frequently accessed" and "fast". > > IOW the web page is confusing. It does not talk about robustness at > all. OK, thanks. = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
Sam Ravnborg wrote: > It is not that we are talking big and complex stuff here. I agree completely. > pramfs is likely to be used for small things and then having to > fix endian on a few headers in the on-dsk format does not matter. I agree with this, but mostly out of exhaustion. > Not compared to the potential disadvantages. I can see no potential disadvantages. > It should be possible to read a file-system on your x86 64bit > box that you wrote with your small powerpc target. For a (NV)RAM-based filesystem?? WTH??? This is not my file system, so I don't have a dog in this fight. I just wanted to clarify what I thought were some misconceptions about the use cases and priorities for the FS. My "advocacy" may be interfering with understanding this system and its purpose. I'll be quiet now. -- Tim = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
On Mon 2009-06-22 14:50:01, Tim Bird wrote: > Pavel Machek wrote: > >> block of fast non-volatile RAM that need to access data on it using a > >> standard filesytem interface." > > > > Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are > > better with ext2. > > Not if you want the RAM-based filesystem to persist over a kernel > invocation. Yes, you'll need to code Persistent, RAM-based _block_device_. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
On Mon, 22 June 2009 20:31:10 +0100, Chris Simmonds wrote: > > I disagree: that adds an unnecessary overhead for those architectures > where the cpu byte order does not match the data structure ordering. I > think the data structures should be native endian and when mkpramfs is > written it can take a flag (e.g. -r) in the same way mkcramfs does. Just to quantify this point, I've written a small crap program: #include #include #include #include long long delta(struct timeval *t1, struct timeval *t2) { long long delta; delta = 100ull * t2->tv_sec + t2->tv_usec; delta -= 100ull * t1->tv_sec + t1->tv_usec; return delta; } #define LOOPS 1 int main(void) { long native = 0; uint32_t narrow = 0; uint64_t wide = 0, native_wide = 0; struct timeval t1, t2, t3, t4, t5; int i; gettimeofday(&t1, NULL); for (i = 0; i < LOOPS; i++) native++; gettimeofday(&t2, NULL); for (i = 0; i < LOOPS; i++) narrow = bswap_32(bswap_64(narrow) + 1); gettimeofday(&t3, NULL); for (i = 0; i < LOOPS; i++) native_wide++; gettimeofday(&t4, NULL); for (i = 0; i < LOOPS; i++) wide = bswap_64(bswap_64(wide) + 1); gettimeofday(&t5, NULL); printf("long: %9lld us\n", delta(&t1, &t2)); printf("we32: %9lld us\n", delta(&t2, &t3)); printf("u64: %9lld us\n", delta(&t3, &t4)); printf("we64: %9lld us\n", delta(&t4, &t5)); printf("loops: %9d\n", LOOPS); return 0; } Four loops doing the same increment with different data types: long, u64, we32 (wrong-endian) and we64. Compile with _no_ optimizations. Results on my i386 notebook: long: 453953 us we32: 880273 us u64: 504214 us we64:2259953 us loops: 1 Or thereabouts, not completely stable. Increasing the data width is 10% slower, 32bit endianness conversions is 2x slower, 64bit conversion is 5x slower. However, even the we64 loop still munches through 353MB/s (100M conversions in 2.2s, 8bytes per converion. Double the number if you count both conversion to/from wrong endianness). Elsewhere in this thread someone claimed the filesystem peaks out at 13MB/s. One might further note that only filesystem metadata has to go through endianness conversion, so on this particular machine it is completely lost in the noise. Feel free to run the program on any machine you care about. If you get numbers to back up your position, I'm willing to be convinced. Until then, I consider the alleged overhead of endianness conversion a prime example of premature optimization. Jörn -- Joern's library part 7: http://www.usenix.org/publications/library/proceedings/neworl/full_papers/mckusick.a -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
On Mon 2009-06-22 11:55:04, Tim Bird wrote: > Pavel Machek wrote: > > On Mon 2009-06-22 10:31:28, Tim Bird wrote: > >> Pavel Machek wrote: > >>> I did not see that in the changelog. If it is not general purpose > >>> filesystem, it is lot less interesting. > >> PRAMFS is not a general purpose filesystem. Please read > >> the introductory post to this thread, or look at > >> http://pramfs.sourceforge.net/ for more information. > > > > Yeah, I seen that. It directly contradicts what you say. > > Could you be more specific? In what way does the > description on the website contradict what I said? You are saying top goal is robustness, while the web page says (home page, stop using frames!): "embedded systems have a block of non-volatile RAM seperate from normal system memory, i.e. of which the kernel maintains no memory page descriptors. For such systems it would be beneficial to mount a fast read/write filesystem over this "I/O memory", for storing frequently accessed data that must survive system reboots and power cycles" Note the "frequently accessed" and "fast". IOW the web page is confusing. It does not talk about robustness at all. > >> Since the purpose of PRAMFS is to provide a filesystem > >> that is persistent across kernel instantions, it is not > >> designed for high speed. Robustness in the face of > >> kernel crashes or bugs is the highest priority, so > >> PRAMFS has significant overhead to make the window > >> of writability to the filesystem RAM as small as possible. > > > > Really? So why don't you use well known, reliable fs like ext3? > > Are you sure you read the web site? It directly addresses this > question. From the web site: "1. Disk-based filesystems such as No, it does not. It explains that ext2 would be too slow on this, and explains that it will eat too much disk space. Please back that claims with numbers. If reliability is top concern, explain how you get away w/o journalling. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
On Mon 2009-06-22 20:07:06, Marco wrote: > Pavel Machek wrote: > > On Mon 2009-06-22 10:31:28, Tim Bird wrote: > >> Pavel Machek wrote: > > How do you handle hard-links, then? > Indeed hard-links are not supported :) Due to the design of this fs > there are some limitations explained in the documentation as not > hard-link, only private memory mapping and so on. However this > limitations don't limit the fs itself because you must consider the > special goal of this fs. > >>> I did not see that in the changelog. If it is not general purpose > >>> filesystem, it is lot less interesting. > >> PRAMFS is not a general purpose filesystem. Please read > >> the introductory post to this thread, or look at > >> http://pramfs.sourceforge.net/ for more information. > > > > Yeah, I seen that. It directly contradicts what you say. > > > > I don't think, I think it's very clear: > > "In summary, PRAMFS is a light-weight, full-featured, and > space-efficient special filesystem that is ideal for systems with a Except that it is not full-featured. No hardlinks. (What about ACLs, etc?) > block of fast non-volatile RAM that need to access data on it using a > standard filesytem interface." Turns a block of fast RAM into 13MB/sec disk. Hmm. I believe you are better with ext2. -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
On Mon, 22 Jun 2009, Marco wrote: > I don't think, I think it's very clear: > > "In summary, PRAMFS is a light-weight, full-featured, and > space-efficient special filesystem that is ideal for systems with a > block of fast non-volatile RAM that need to access data on it using a > standard filesytem interface." It is not full-featured if it doesn't have support for hardlinks, security labels, extended attributes, etc. Please call it a specialized filesystem instead, that seems to be much more in line with the comments about pramfs use cases in this thread... Oh, and that should be in the Kconfig help text, as well as the stuff that goes in Documentation/*, obviously. Maybe it already is, I didn't look. -- "One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie." -- The Silicon Valley Tarot Henrique Holschuh -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
On Mon, Jun 22, 2009 at 08:31:10PM +0100, Chris Simmonds wrote: > Arnd Bergmann wrote: > >On Monday 22 June 2009, Marco wrote: > >>Sorry, I meant it's not currently possible. At the moment the only way > >>to use it as rootfs it's to copy all the data in an already mounted > >>(empty) ram partition and reboot. However it's not my first item on my > >>todo list because I think that it's possible to use it as rootfs but it > >>isn't the standard use for this fs. > > > >Well, it doesn't have to work right away. What I'm asking to > >define the data structures in a way that keeps the layout stable > >across kernel updates. Since a future version of the file system > >might support cross-endian image creation, it would be good to > >define the data structures in a fixed endian mode already, so > >you don't have to change it in the future. > > > > Arnd <>< > >-- > >To unsubscribe from this list: send the line "unsubscribe linux-embedded" > >in > >the body of a message to majord...@vger.kernel.org > >More majordomo info at http://vger.kernel.org/majordomo-info.html > > > > I disagree: that adds an unnecessary overhead for those architectures > where the cpu byte order does not match the data structure ordering. It is not that we are talking big and complex stuff here. pramfs is likely to be used for small things and then having to fix endian on a few headers in the on-dsk format does not matter. Not compared to the potential disadvantages. It should be possible to read a file-system on your x86 64bit box that you wrote with your small powerpc target. Sam -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
Arnd Bergmann wrote: On Monday 22 June 2009, Marco wrote: Sorry, I meant it's not currently possible. At the moment the only way to use it as rootfs it's to copy all the data in an already mounted (empty) ram partition and reboot. However it's not my first item on my todo list because I think that it's possible to use it as rootfs but it isn't the standard use for this fs. Well, it doesn't have to work right away. What I'm asking to define the data structures in a way that keeps the layout stable across kernel updates. Since a future version of the file system might support cross-endian image creation, it would be good to define the data structures in a fixed endian mode already, so you don't have to change it in the future. Arnd <>< -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html I disagree: that adds an unnecessary overhead for those architectures where the cpu byte order does not match the data structure ordering. I think the data structures should be native endian and when mkpramfs is written it can take a flag (e.g. -r) in the same way mkcramfs does. Chris Simmonds -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
Pavel Machek wrote: > On Mon 2009-06-22 10:31:28, Tim Bird wrote: >> Pavel Machek wrote: >>> I did not see that in the changelog. If it is not general purpose >>> filesystem, it is lot less interesting. >> PRAMFS is not a general purpose filesystem. Please read >> the introductory post to this thread, or look at >> http://pramfs.sourceforge.net/ for more information. > > Yeah, I seen that. It directly contradicts what you say. Could you be more specific? In what way does the description on the website contradict what I said? >> Since the purpose of PRAMFS is to provide a filesystem >> that is persistent across kernel instantions, it is not >> designed for high speed. Robustness in the face of >> kernel crashes or bugs is the highest priority, so >> PRAMFS has significant overhead to make the window >> of writability to the filesystem RAM as small as possible. > > Really? So why don't you use well known, reliable fs like ext3? Are you sure you read the web site? It directly addresses this question. From the web site: "1. Disk-based filesystems such as ext2/ext3 were designed for optimum performance on spinning disk media, so they implement features such as block groups, which attempts to group inode data into a contiguous set of data blocks to minimize disk seeking when accessing files. For RAM there is no such concern; a file's data blocks can be scattered throughout the media with no access speed penalty at all. So block groups in a filesystem mounted over RAM just adds unnecessary complexity. A better approach is to use a filesystem specifically tailored to RAM media which does away with these disk-based features. This increases the efficient use of space on the media, i.e. more space is dedicated to actual file data storage and less to meta-data needed to maintain that file data." >> This is not a file system one would do kernel compiles on. >> This is where someone would keep a small amount of sensitive >> data, or crash logs that one needed to preserve over kernel >> invocations. > > Really? Web page says: > > #2. If the backing-store RAM is comparable in access speed to system > #memory, there's really no point in caching the file I/O data in the > #page cache. Better to move file data directly between the user buffers > #and the backing store RAM, i.e. use direct I/O. This prevents the > #unnecessary > > So you don't cache it "because its fast", and then it is 13MB/sec? I'm not sure what you're quoting. The paragraph you quoted doesn't have the words "because its fast" [sic]. Speed is not a primary consideration for this filesystem. Due to the overhead of changing the page flags, it's possible that using the page cache would speed up overall access to this filesystem. (This would depend on the cost of changing the access flags, and the pattern of operations on the filesystem.) However, using the page cache would add a level of indirection which would detract from the intended robustness. -- Tim = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
On Monday 22 June 2009, Marco wrote: > > Sorry, I meant it's not currently possible. At the moment the only way > to use it as rootfs it's to copy all the data in an already mounted > (empty) ram partition and reboot. However it's not my first item on my > todo list because I think that it's possible to use it as rootfs but it > isn't the standard use for this fs. Well, it doesn't have to work right away. What I'm asking to define the data structures in a way that keeps the layout stable across kernel updates. Since a future version of the file system might support cross-endian image creation, it would be good to define the data structures in a fixed endian mode already, so you don't have to change it in the future. Arnd <>< -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
Tim Bird wrote: > Pavel Machek wrote: How do you handle hard-links, then? >>> Indeed hard-links are not supported :) Due to the design of this fs >>> there are some limitations explained in the documentation as not >>> hard-link, only private memory mapping and so on. However this >>> limitations don't limit the fs itself because you must consider the >>> special goal of this fs. >> I did not see that in the changelog. If it is not general purpose >> filesystem, it is lot less interesting. > > PRAMFS is not a general purpose filesystem. Please read > the introductory post to this thread, or look at > http://pramfs.sourceforge.net/ for more information. > > Since the purpose of PRAMFS is to provide a filesystem > that is persistent across kernel instantions, it is not > designed for high speed. Robustness in the face of > kernel crashes or bugs is the highest priority, so > PRAMFS has significant overhead to make the window > of writability to the filesystem RAM as small as possible. > > This is not a file system one would do kernel compiles on. > This is where someone would keep a small amount of sensitive > data, or crash logs that one needed to preserve over kernel > invocations. > Yep, I quite agree. Marco -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
Pavel Machek wrote: > On Mon 2009-06-22 10:31:28, Tim Bird wrote: >> Pavel Machek wrote: > How do you handle hard-links, then? Indeed hard-links are not supported :) Due to the design of this fs there are some limitations explained in the documentation as not hard-link, only private memory mapping and so on. However this limitations don't limit the fs itself because you must consider the special goal of this fs. >>> I did not see that in the changelog. If it is not general purpose >>> filesystem, it is lot less interesting. >> PRAMFS is not a general purpose filesystem. Please read >> the introductory post to this thread, or look at >> http://pramfs.sourceforge.net/ for more information. > > Yeah, I seen that. It directly contradicts what you say. > I don't think, I think it's very clear: "In summary, PRAMFS is a light-weight, full-featured, and space-efficient special filesystem that is ideal for systems with a block of fast non-volatile RAM that need to access data on it using a standard filesytem interface." >> Since the purpose of PRAMFS is to provide a filesystem >> that is persistent across kernel instantions, it is not >> designed for high speed. Robustness in the face of >> kernel crashes or bugs is the highest priority, so >> PRAMFS has significant overhead to make the window >> of writability to the filesystem RAM as small as possible. > > Really? So why don't you use well known, reliable fs like ext3? > >> This is not a file system one would do kernel compiles on. >> This is where someone would keep a small amount of sensitive >> data, or crash logs that one needed to preserve over kernel >> invocations. > > Really? Web page says: > > #2. If the backing-store RAM is comparable in access speed to system > #memory, there's really no point in caching the file I/O data in the > #page cache. Better to move file data directly between the user buffers > #and the backing store RAM, i.e. use direct I/O. This prevents the > #unnecessary > > So you don't cache it "because its fast", and then it is 13MB/sec? > > Pavel -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
Arnd Bergmann wrote: > On Monday 22 June 2009, Marco Stornelli wrote: >>> It's still a problem. You might be creating a file system image >>> for an embedded board with a different endianess. >> It's not possible to create an "image" with pramfs, it's like tmpfs. > > But the data is persistant, you even support using it as a root file > system, so the data has to have a way to get there. Even if you > don't do it right now, I don't see any fundamental limitation that > prevents you from creating an image on one machine and dumping it > into the nvram of another machine as part of manufacturing or testing. > Sorry, I meant it's not *currently* possible. At the moment the only way to use it as rootfs it's to copy all the data in an already mounted (empty) ram partition and reboot. However it's not my first item on my todo list because I think that it's possible to use it as rootfs but it isn't the standard use for this fs. >>> Or even on the same machine, you could be looking at the file system >>> contents >>> with a 32 bit process running on a 64 bit kernel. >> Yes, indeed the most important thing is to be sure that a 64bit kernel >> works well. I'll try to test it in this environment. If there are >> "64bit guys" to help me to test it, it'd be great. > > This particular problem (__kernel_off_t on 64-bit machines) can be avoided > by just switching to __kernel_loff_t, which is 64 bit long on all machines, > while __kernel_off_t is always the register length (32 or 64 bits). > > Arnd <>< > Yep, it can be a good idea, thanks for the tip. Marco -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
On Mon 2009-06-22 10:31:28, Tim Bird wrote: > Pavel Machek wrote: > >>> How do you handle hard-links, then? > >> Indeed hard-links are not supported :) Due to the design of this fs > >> there are some limitations explained in the documentation as not > >> hard-link, only private memory mapping and so on. However this > >> limitations don't limit the fs itself because you must consider the > >> special goal of this fs. > > > > I did not see that in the changelog. If it is not general purpose > > filesystem, it is lot less interesting. > > PRAMFS is not a general purpose filesystem. Please read > the introductory post to this thread, or look at > http://pramfs.sourceforge.net/ for more information. Yeah, I seen that. It directly contradicts what you say. > Since the purpose of PRAMFS is to provide a filesystem > that is persistent across kernel instantions, it is not > designed for high speed. Robustness in the face of > kernel crashes or bugs is the highest priority, so > PRAMFS has significant overhead to make the window > of writability to the filesystem RAM as small as possible. Really? So why don't you use well known, reliable fs like ext3? > This is not a file system one would do kernel compiles on. > This is where someone would keep a small amount of sensitive > data, or crash logs that one needed to preserve over kernel > invocations. Really? Web page says: #2. If the backing-store RAM is comparable in access speed to system #memory, there's really no point in caching the file I/O data in the #page cache. Better to move file data directly between the user buffers #and the backing store RAM, i.e. use direct I/O. This prevents the #unnecessary So you don't cache it "because its fast", and then it is 13MB/sec? Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
Pavel Machek wrote: >>> How do you handle hard-links, then? >> Indeed hard-links are not supported :) Due to the design of this fs >> there are some limitations explained in the documentation as not >> hard-link, only private memory mapping and so on. However this >> limitations don't limit the fs itself because you must consider the >> special goal of this fs. > > I did not see that in the changelog. If it is not general purpose > filesystem, it is lot less interesting. PRAMFS is not a general purpose filesystem. Please read the introductory post to this thread, or look at http://pramfs.sourceforge.net/ for more information. Since the purpose of PRAMFS is to provide a filesystem that is persistent across kernel instantions, it is not designed for high speed. Robustness in the face of kernel crashes or bugs is the highest priority, so PRAMFS has significant overhead to make the window of writability to the filesystem RAM as small as possible. This is not a file system one would do kernel compiles on. This is where someone would keep a small amount of sensitive data, or crash logs that one needed to preserve over kernel invocations. > >From performance point of view: Sometimes ago I uploaded here (http://elinux.org/Pram_Fs) some benchmark results to compare the performance with and without XIP in a real embedded environment with bonnie++. You could use it as reference point. >>> Well, so XIP helps. ext2 can do XIP too, IIRC. Is your performance >>> better than ext2? >>> >>> Wait... those numbers you pointed me... claim to be as slow as >>> 13MB/sec. That's very very bad. My harddrive is faster than that. >> As I said I did the test in a real embedded environment so to have >> comparable result you should use the same environmente with the same >> tools, with the same workload and so on. > > Even on real embedded hardware you should get better than 13MB/sec > writing to _RAM_. I guess something is seriously wrong with pramfs. See above. = Tim Bird Architecture Group Chair, CE Linux Forum Senior Staff Engineer, Sony Corporation of America = -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/14] Pramfs: Persistent and protected ram filesystem
> > How do you handle hard-links, then? > > Indeed hard-links are not supported :) Due to the design of this fs > there are some limitations explained in the documentation as not > hard-link, only private memory mapping and so on. However this > limitations don't limit the fs itself because you must consider the > special goal of this fs. I did not see that in the changelog. If it is not general purpose filesystem, it is lot less interesting. > >> >From performance point of view: > >> > >> Sometimes ago I uploaded here (http://elinux.org/Pram_Fs) some benchmark > >> results to compare the performance with and without XIP in a real > >> embedded environment with bonnie++. You could use it as reference point. > > > > Well, so XIP helps. ext2 can do XIP too, IIRC. Is your performance > > better than ext2? > > > > Wait... those numbers you pointed me... claim to be as slow as > > 13MB/sec. That's very very bad. My harddrive is faster than that. > > As I said I did the test in a real embedded environment so to have > comparable result you should use the same environmente with the same > tools, with the same workload and so on. Even on real embedded hardware you should get better than 13MB/sec writing to _RAM_. I guess something is seriously wrong with pramfs. Pavel -- (english) http://www.livejournal.com/~pavelmachek (cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 06/14] Pramfs: Include files
On Monday 22 June 2009, Marco Stornelli wrote: > > It's still a problem. You might be creating a file system image > > for an embedded board with a different endianess. > > It's not possible to create an "image" with pramfs, it's like tmpfs. But the data is persistant, you even support using it as a root file system, so the data has to have a way to get there. Even if you don't do it right now, I don't see any fundamental limitation that prevents you from creating an image on one machine and dumping it into the nvram of another machine as part of manufacturing or testing. > > Or even on the same machine, you could be looking at the file system > > contents > > with a 32 bit process running on a 64 bit kernel. > > Yes, indeed the most important thing is to be sure that a 64bit kernel > works well. I'll try to test it in this environment. If there are > "64bit guys" to help me to test it, it'd be great. This particular problem (__kernel_off_t on 64-bit machines) can be avoided by just switching to __kernel_loff_t, which is 64 bit long on all machines, while __kernel_off_t is always the register length (32 or 64 bits). Arnd <>< -- To unsubscribe from this list: send the line "unsubscribe linux-embedded" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html