Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Mon, 15 Sep 2008, Jared Hulbert wrote: I'm using Spansion MirrorBit S29GL128N, which reads at about 0.6 MByte/s. If you are using a GL, you'll probably find our P33 a good fit and at competitive prices to GL as I understand it. That's I think 50Mhz. What's a GL? Never heard of it - all I can think of is OpenGL :-) S29_GL_128N I'm using a S29GL064N chip. Going through linux and /dev/mtd I get 13.5 MB/sec and reading directly from the chips give 15 MB/sec. I've not mapped the chip cached and I'm not using the page burst mode. That would help a lot certainly, but the current flash speed isn't much of a bottleneck. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
NAND is significantly faster when writing than NOR, read speed is of the same magnitude, possibly slower in many cases. Right. Specifically, read bandwidth is on the same order of magnitude. However the read latency of NAND is a couple orders of magnitude higher (100ns vs 20,000ns) so it depends on what you are doing. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Jared Hulbert wrote: What kind of NOR you using? That is not what I measure with fast synchronous burst NOR's. I think the fast in fast synchronous gives it away :-) I'm using Spansion MirrorBit S29GL128N, which reads at about 0.6 MByte/s. By the way, what speeds do you get on fast synchronous burst NORs - and which chips are those? I am only familiar with the Numonyx product line up. If you are using a GL, you'll probably find our P33 a good fit and at competitive prices to GL as I understand it. That's I think 50Mhz. What's a GL? Never heard of it - all I can think of is OpenGL :-) I'm using a Sigma Designs 862x media processor. It clocks at 166MHz to main RAM, has an ARM internally to run Linux, and the intensive work happens in coprocessors. The NOR is not on the RAM bus, it's on a peripheral bus. About the only thing I know about the bus is it's 16 bits wide - I have the schematic, but only the board supplier has access to Sigma chip documentation. A rule of thumb is to say that copying to RAM will reduce this value less than 50% because RAM should be at least a little faster than NOR. In our case, RAM is at least 100x faster :-) Nevertheless, If you can't use a simple calculation like this to explain the numbers then you have a poorly configured bus timings or have your cache off. I'm not sure if cache is an option with this device - but would it make a difference anyway? Launching executables like Busybox - those are much larger than the cache anyway, so launch time is dominated by bulk streaming copy speed. Thanks for the idea, I'll look into whether it's possible to access this 'peripheral bus' through the ARM's cache and see if that speeds up streaming read time. So, If you are only getting 0.6MB/s out of your NOR. You're using it wrong. Interesting, thanks. I'm not sure it's possible to change the way NOR is being used with this chip, and it'll be a while before it's economical to replace the board with a new design. This is all very interesting - I had no prior experience with NOR, so didn't know that 0.6MB/s was slow. It's fast compared with older EEPROMs after all, and had imagined that people wanting fast flash would use NAND. On looking at the datasheet, I see it's quite a lot faster. I'm suspecting the Sigma Designs perpheral bus and the way it's wired up not doing it any favours. We already have the weirdness that we have to patch the Linux CFI-0002 MTD code: the CPU locks up when polling the erase status byte, until erase is finished. Unfortunately this is difficult to change now - I'm programming hardware which is already out in the field and cannot be redesigned. Thanks for your thoughts. -- Jamie -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
I'm using Spansion MirrorBit S29GL128N, which reads at about 0.6 MByte/s. If you are using a GL, you'll probably find our P33 a good fit and at competitive prices to GL as I understand it. That's I think 50Mhz. What's a GL? Never heard of it - all I can think of is OpenGL :-) S29_GL_128N I'm not sure if cache is an option with this device - but would it make a difference anyway? Well the first read takes 100ns (plus the other chipset overhead 300ns) but other reads in a page are only an extra 25ns each. So your benefit is not from having the entire executable in cache it's from having the next 7 instructions in the cacheline for only an extra 25ns each instead of 400ns. Interesting, thanks. I'm not sure it's possible to change the way NOR is being used with this chip, and it'll be a while before it's economical to replace the board with a new design. Usually these things can be fixed in the bootloader or by hacking the kernel to tweak the relevant chipset registers. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
I think the fast in fast synchronous gives it away :-) Yes, I suppose it does. I'm using Spansion MirrorBit S29GL128N, which reads at about 0.6 MByte/s. I think you should get more like an order of magnitude higher Get an expert to look at your timings in the bootloader. Make sure things are cached too. ioremap_cached()... Not because they're good, but because that's what the board I'm coding for has on it. I presume they were cheap and familiar to the board designers. (There is 32MB of RAM to play with after all.) So start a sequence of Busybox processes from a shell script is noticable, if it reads from NOR each time. Oh, and it's a 166MHz ARM, so it's quite capable of decompressing faster than the NOR can deliver. Depends on how you are measuring it. You ought to be able to get at least 2 orders of magnitude higher read speeds with a good sync Flash. Some of the newer stuff is even faster. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Fri, Aug 22, 2008 at 11:43 AM, Jamie Lokier [EMAIL PROTECTED] wrote: Jamie Lokier wrote: Jared Hulbert wrote: What kind of NOR you using? That is not what I measure with fast synchronous burst NOR's. I think the fast in fast synchronous gives it away :-) I'm using Spansion MirrorBit S29GL128N, which reads at about 0.6 MByte/s. By the way, what speeds do you get on fast synchronous burst NORs - and which chips are those? I am only familiar with the Numonyx product line up. If you are using a GL, you'll probably find our P33 a good fit and at competitive prices to GL as I understand it. That's I think 50Mhz. M18 is 100Mhz maybe a little higher. And we just announced our LPDDR 266 part, Velocity LP. A good way of making a rough estimate of read performance is to measure a cache miss latency and convert that to bandwidth. It's usually fairly close. 32Byte cache size / 16 bit bus = 16 word transfers memory controller latency (time from Load instruction to bus activity) = ~300ns (upto 450ns for some processors) initial latency (time to read first word) = ~100ns (60ns - 120ns) clock frequency (time between words) = 50Mhz = 20ns per word bus clean up = ~50ns 32Bytes = 300ns + 100ns + (16 - 1) * 20ns + 50ns = 750ns = 32B/750ns = 40MB/s This is a very simple model and reality is much more nuanced. You also need to check my assume numbers with the reality of your system. Also this doesn't take copying the data to RAM into account which is usually what you are really measuring. That's easy to model though. A rule of thumb is to say that copying to RAM will reduce this value less than 50% because RAM should be at least a little faster than NOR. Nevertheless, If you can't use a simple calculation like this to explain the numbers then you have a poorly configured bus timings or have your cache off. Cache is important because it would make the equation look like this, or worse. 4Bytes = 300ns + 100ns + 1 * 20ns + 50ns = 470ns =4B/470ns =8MB/s For a PXA270 if you go with the defaults it can look like this. 2Bytes = 300ns + 250ns + 250ns = 800ns = 2MB/s So, If you are only getting 0.6MB/s out of your NOR. You're using it wrong. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
I found what's wrong. The size of an AxFS image created by mkfs.axfs is always n*4096+4 bytes large. So when it wants to check the magic value in the last 4 bytes, the block layer tries to read a whole 512-byte sector, which fails for loop-mounted images. If you test on real FLASH, additional bytes after the end of the AxFS image can be read, hence it works. By padding the image with 508 zero bytes, I can mount it, on both PS3 (ppc64) and UML (ai32). I can even read images created on PS3. Right. We haven't tested loopback since we added the magic end value. How is one expected to read those last 4 bytes of a loopbacked file? Are they unreadable? We can add the padding. I am just wondering if this is a bug or a known limitation in the loopback handling or if there is a different safer way of reading block devs with truncated last blocks. However, there still are weird things going on, like `find' not seeing all files and directories, or just aborting, and `ls -lR' showing actual file contents in its output. Do you see this behavior for all builds for just the PS3? -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Tue, 2 September 2008 09:44:19 -0700, Jared Hulbert wrote: How is one expected to read those last 4 bytes of a loopbacked file? Are they unreadable? We can add the padding. I am just wondering if this is a bug or a known limitation in the loopback handling or if there is a different safer way of reading block devs with truncated last blocks. Can't you just include the final magic into the last block, thereby making the size a clean multiple of 4k? It looks as if you have some padding before the magic anyway. So you just have to make sure the padding is at least 4 bytes and write the magic to the end of it. Apart from solving this bug, it should also save you some space. ;) Jörn -- Invincibility is in oneself, vulnerability is in the opponent. -- Sun Tzu -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
How is one expected to read those last 4 bytes of a loopbacked file? Are they unreadable? We can add the padding. I am just wondering if this is a bug or a known limitation in the loopback handling or if there is a different safer way of reading block devs with truncated last blocks. Can't you just include the final magic into the last block, thereby making the size a clean multiple of 4k? It looks as if you have some padding before the magic anyway. So you just have to make sure the padding is at least 4 bytes and write the magic to the end of it. Apart from solving this bug, it should also save you some space. ;) I'm going to have to look into this n*4K thing. The image doesn't need to be aligned. There shouldn't be any last block to put the magic in. But I haven't messed with the mkfs.axfs code for a while. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Tue, 2 Sep 2008, Jared Hulbert wrote: However, there still are weird things going on, like `find' not seeing all files and directories, or just aborting, and `ls -lR' showing actual file contents in its output. Do you see this behavior for all builds for just the PS3? The `find' issue I saw on both PS3 (ppc64) and UML (ia32). The `ls -lR' I tried on UML (ia32) only for now. With kind regards, Geert Uytterhoeven Software Architect Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium Phone:+32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [EMAIL PROTECTED] Internet: http://www.sony-europe.com/ A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Fri, 22 Aug 2008, Jared Hulbert wrote: The version in SVN seems to be slightly older than the one you submitted? Oops. Okay I must have neglected to sync at the very end. Thanks. I forgot, there is also a git repo at git://git.infradead.org/users/jehulber/axfs.git Which platform(s) do you use for testing? ARM, x86 I gave AxFS a try on PS3 (ppc64, always use big-endian 64-bit for testing new code ;-). Smart. Hmmm, If only I had a PS3 I heard you got one? ;-) When mounting the image, I got the crash below: | attempt to access beyond end of device | loop0: rw=0, want=4920, limit=4912 Interestingly, it also doesn't work on UserModeLinux (x86, 32-bit): | attempt to access beyond end of device | loop0: rw=0, want=24, limit=16 | | EIP: 0073:[0811ec67] CPU: 0 Not tainted ESP: 007b:19515c38 EFLAGS: 00210212 | Not tainted | EAX: EBX: 1000 ECX: 19484aa0 EDX: 190d9f0c | ESI: 195ee000 EDI: 19515cd0 EBP: 19515c6c DS: 007b ES: 007b | 08247af0: [08069ba3] show_regs+0xb4/0xb9 | 08247b1c: [080591ee] segv+0x222/0x23a | 08247bbc: [08059296] segv_handler+0x90/0x9a | 08247c68: [080649b8] sig_handler_common+0x63/0x72 | 08247ce0: [08064cac] sig_handler+0x31/0x3d | 08247cec: [08064c0b] handle_signal+0x4c/0x7a | 08247d0c: [08066327] hard_handler+0xf/0x14 | 08247d1c: [005c0420] 0x5c0420 | | Kernel panic - not syncing: Kernel mode fault at addr 0x14, ip 0x811ec67 | | EIP: 0073:[4010a44e] CPU: 0 Not tainted ESP: 007b:bfa323a0 EFLAGS: 00200246 | Not tainted | EAX: ffda EBX: 080595f8 ECX: 080595c8 EDX: 080595d8 | ESI: c0ed EDI: EBP: bfa323d8 DS: 007b ES: 007b | 08247a5c: [08069ba3] show_regs+0xb4/0xb9 | 08247a88: [08059462] panic_exit+0x25/0x3b | 08247a9c: [08083642] notifier_call_chain+0x27/0x4c | 08247ac4: [0808367e] __atomic_notifier_call_chain+0x17/0x19 | 08247ad4: [08083695] atomic_notifier_call_chain+0x15/0x17 | 08247af0: [0806fd87] panic+0x52/0xd8 | 08247b10: [080591fc] segv+0x230/0x23a | 08247bbc: [08059296] segv_handler+0x90/0x9a | 08247c68: [080649b8] sig_handler_common+0x63/0x72 | 08247ce0: [08064cac] sig_handler+0x31/0x3d | 08247cec: [08064c0b] handle_signal+0x4c/0x7a | 08247d0c: [08066327] hard_handler+0xf/0x14 | 08247d1c: [005c0420] 0x5c0420 Commandline is `mount image.axfs /mnt -o loop -t axfs'. Is there something wrong with the axfs version you submitted, or with mkfs.axfs? With kind regards, Geert Uytterhoeven Software Architect Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium Phone:+32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [EMAIL PROTECTED] Internet: http://www.sony-europe.com/ A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Hi Jared, Jared Hulbert wrote: That would be enough I think. If you could manually select which files are contiguous-and-uncompressed that would be useful for some too here. So If you don't have an MMU when do you call -fault? Does the noMMU code just loop through -fault()ing all the pages in an mmap()? Sort of. It actually just uses a single -read to bring in the entire file contents. There is a few limitations on the use of mmap() for non-mmu. Documentation/nommu-mmap.txt gives more details. With no MMU it does rely on being able to kmalloc() a single RAM region big enough to hold the entire file. One thing for sure is that many people who do non-MMU setups are interested in XIP to get the space savings. These are very often small devices with very constrained RAM and flash. (For whatever it is worth single NOR flash only boards are common in these smaller form factors :-) True. Regards Greg Greg Ungerer -- Chief Software Dude EMAIL: [EMAIL PROTECTED] SnapGear -- a Secure Computing Company PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Fri, 22 Aug 2008, Will Marone wrote: Geert Uytterhoeven wrote: I gave AxFS a try on PS3 (ppc64, always use big-endian 64-bit for testing new code ;-). When mounting the image, I got the crash below: | attempt to access beyond end of device | loop0: rw=0, want=4920, limit=4912 | Unable to handle kernel paging request for data at address 0x0028 | Faulting instruction address: 0xd0037988 | Oops: Kernel access of bad area, sig: 11 [#1] | SMP NR_CPUS=2 PS3 When mounting (also on PS3) an image created on ia32, I get a different crash: | axfs: wrong magic ^ | Unable to handle kernel paging request for data at address 0x03a8 | Faulting instruction address: 0xd00355f0 | Oops: Kernel access of bad area, sig: 11 [#1] | SMP NR_CPUS=2 PS Geert, Thanks for giving it a spin, especially on a platform as different from ours as the PS3. Before I dig more into what happened, I was wondering if you could tell me a bit more about your environment, particularly how you supplied the filesystem to the kernel and your mount commandline (also, if you used a boot commandline, what it was.) My first guess would be a ppc64 compiled UML session, but I'd like to be a bit more sure. Nope, I just built axfs as a module and insmoded it. After that mount image.axfs /mnt -o loop -t axfs So nothing fancy. With kind regards, Geert Uytterhoeven Software Architect Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium Phone:+32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [EMAIL PROTECTED] Internet: http://www.sony-europe.com/ A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Fri, 22 Aug 2008, Jared Hulbert wrote: The version in SVN seems to be slightly older than the one you submitted? Oops. Okay I must have neglected to sync at the very end. Thanks. I forgot, there is also a git repo at git://git.infradead.org/users/jehulber/axfs.git Which platform(s) do you use for testing? ARM, x86 Ah, little endian. From your good relationship with the s390 developers, I had hoped you would have done some tests on s390 ;-) I gave AxFS a try on PS3 (ppc64, always use big-endian 64-bit for testing new code ;-). Smart. Hmmm, If only I had a PS3 When mounting the image, I got the crash below: When mounting (also on PS3) an image created on ia32, I get a different crash: | axfs: wrong magic So I guess some parts are not yet 64-bit or endian clean. Can you run mkfs.axfs on the same trivial directory on both ia32 and PPC64 and then get me the resulting images? I'll send them by private email. With kind regards, Geert Uytterhoeven Software Architect Sony Techsoft Centre Europe The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium Phone:+32 (0)2 700 8453 Fax: +32 (0)2 700 8622 E-mail: [EMAIL PROTECTED] Internet: http://www.sony-europe.com/ A division of Sony Europe (Belgium) N.V. VAT BE 0413.825.160 · RPR Brussels Fortis · BIC GEBABEBB · IBAN BE41293037680010
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Geert Uytterhoeven wrote: From your good relationship with the s390 developers, I had hoped you would have done some tests on s390 ;-) Haha, we let you sort out the endianess issues first and then take the easy path :-). We have'nt tried it yet so far. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Fri, 2008-08-22 at 09:51 -0700, Jared Hulbert wrote: Can you run mkfs.axfs on the same trivial directory on both ia32 and PPC64 and then get me the resulting images? git.infradead.org is a big-endian box, and I know you have an account there... -- David WoodhouseOpen Source Technology Centre [EMAIL PROTECTED] Intel Corporation -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Hi Jamie, Jamie Lokier wrote: Greg Ungerer wrote: Sort of. It actually just uses a single -read to bring in the entire file contents. There is a few limitations on the use of mmap() for non-mmu. Documentation/nommu-mmap.txt gives more details. With no MMU it does rely on being able to kmalloc() a single RAM region big enough to hold the entire file. That's unfortunate, if you're using FDPIC-ELF or BFLT-XIP, you really want to kmalloc() one region for code (i.e. mmap not the whole file), and one separate for data. That is what the BFLT loader does. For the XIP case it mmap()s the text directly from the file, and then mmap()s a second region for the data/bss (reading the data into that region). I was referring to general mmap() of a file case above, not the exec path. Asking for a single larger region sometimes creates much higher memory pressure while kmalloc() attempts to defragment by evicting everything. Sure. But that's fiddly to do right in general. The natural thing for AXFS to do to support no-MMU FDPIC-ELF or BFLT-XIP is store the code segment uncompressed and contiguous, and the data segment however the filesystem prefers, and the profiling information to work out where these are is readily available from the mmap() calls, which are always the same when an executable is run. Yep. Regards Greg Greg Ungerer -- Chief Software Dude EMAIL: [EMAIL PROTECTED] SnapGear -- a Secure Computing Company PHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Hi Jared, On Wed, 20 Aug 2008, Jared Hulbert wrote: I'd like to get a first round of review on my AXFS filesystem. This is a simple See http://axfs.sourceforge.net for more info. The version in SVN seems to be slightly older than the one you submitted? Which platform(s) do you use for testing? I gave AxFS a try on PS3 (ppc64, always use big-endian 64-bit for testing new code ;-). When mounting the image, I got the crash below: | attempt to access beyond end of device | loop0: rw=0, want=4920, limit=4912 | Unable to handle kernel paging request for data at address 0x0028 | Faulting instruction address: 0xd0037988 | Oops: Kernel access of bad area, sig: 11 [#1] | SMP NR_CPUS=2 PS3 | Modules linked in: axfs zlib_inflate nfsd exportfs dm_crypt dm_mod sg joydev evdev | NIP: d0037988 LR: d0037974 CTR: | REGS: cc1e3240 TRAP: 0300 Not tainted (2.6.27-rc4-dirty) | MSR: 80008032 EE,IR,DR CR: 24044482 XER: 2000 | DAR: 0028, DSISR: 4000 | TASK = c68d4e40[1744] 'mount' THREAD: cc1e CPU: 0 | GPR00: d0037974 cc1e34c0 d0043f30 cc1e36a0 | GPR04: 013e 013e cc1e2eb0 0002 | GPR08: c058de80 0001 c68d4e40 cc1e34c0 | GPR12: 80008032 c0671300 1002 ff80bec1 | GPR16: 10023dc8 10023db8 ff80bed1 10023e00 | GPR20: 0001 10023e38 cc1e36a0 cc1d5000 | GPR24: 0004 00266000 | GPR28: 1000 0004 d00438e0 cc1e34c0 | NIP [d0037988] .axfs_copy_block+0xa0/0x144 [axfs] | LR [d0037974] .axfs_copy_block+0x8c/0x144 [axfs] | Call Trace: | [cc1e34c0] [d0037974] .axfs_copy_block+0x8c/0x144 [axfs] (unreliable) | [cc1e3580] [d0035f20] .axfs_copy_metadata+0x154/0x1cc [axfs] | [cc1e3630] [d0035fd8] .axfs_verify_eofs_magic+0x40/0xa0 [axfs] | [cc1e36c0] [d0036678] .axfs_fill_super+0x3c0/0x7c8 [axfs] | [cc1e3780] [c00d1670] .get_sb_bdev+0x154/0x1ec | [cc1e3860] [d0037a94] .axfs_get_sb_bdev+0x34/0x6c [axfs] | [cc1e38f0] [d0035d0c] .axfs_get_sb+0x320/0x394 [axfs] | [cc1e3a00] [c00d1318] .vfs_kern_mount+0x88/0x108 | [cc1e3ab0] [c00d143c] .do_kern_mount+0x68/0x148 | [cc1e3b60] [c00f0a10] .do_new_mount+0x90/0xf4 | [cc1e3c10] [c00f0c5c] .do_mount+0x1e8/0x23c | [cc1e3d60] [c0114778] .compat_sys_mount+0x21c/0x2ac | [cc1e3e30] [c00074dc] syscall_exit+0x0/0x40 | Instruction dump: | 3b60 409e0084 4890 80b7001c e87701d0 7c84e392 48000799 e8410028 | 2fbb 7c781b78 7f3de040 7ec3b378 e8980028 409e002c 7f3dcb78 7c1ae392 | ---[ end trace 7f5bc7e7ad0c4386 ]--- When mounting (also on PS3) an image created on ia32, I get a different crash: | axfs: wrong magic ^ | Unable to handle kernel paging request for data at address 0x03a8 | Faulting instruction address: 0xd00355f0 | Oops: Kernel access of bad area, sig: 11 [#1] | SMP NR_CPUS=2 PS3 | Modules linked in: axfs zlib_inflate nfsd exportfs dm_crypt dm_mod sg joydev evdev | NIP: d00355f0 LR: c00d1250 CTR: d00355d0 | REGS: cc0b73d0 TRAP: 0300 Not tainted (2.6.27-rc4-dirty) | MSR: 80008032 EE,IR,DR CR: 24044482 XER: | DAR: 03a8, DSISR: 4000 | TASK = c6814b40[1745] 'mount' THREAD: cc0b4000 CPU: 1 | GPR00: c00d1250 cc0b7650 d0043f30 c652f800 | GPR04: c652f8b8 c6815480 0002 c6815480 | GPR08: c6815480 01ea | GPR12: d0037e68 c0671500 1002 ffc18eee | GPR16: 10023d98 10023d88 ffc18efe 10023db0 | GPR20: 0001 10023dc8 c634f280 c65b5000 | GPR24: f000 d003bd38 d003b278 | GPR28: c652f800 c652f800 c05d66e8 cc0b7650 | NIP [d00355f0] .axfs_kill_super+0x20/0x9c [axfs] | LR [c00d1250] .deactivate_super+0xd4/0x114 | Call Trace: | [cc0b7650] [c03d2c48] .down_write+0x5c/0xb8 (unreliable) | [cc0b76e0] [c00d1250] .deactivate_super+0xd4/0x114 | [cc0b7780] [c00d1690] .get_sb_bdev+0x174/0x1ec | [cc0b7860] [d0037a94] .axfs_get_sb_bdev+0x34/0x6c [axfs] | [cc0b78f0] [d0035d0c] .axfs_get_sb+0x320/0x394 [axfs] | [cc0b7a00] [c00d1318] .vfs_kern_mount+0x88/0x108 | [cc0b7ab0] [c00d143c] .do_kern_mount+0x68/0x148 | [cc0b7b60]
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Friday 22 August 2008, Geert Uytterhoeven wrote: I gave AxFS a try on PS3 (ppc64, always use big-endian 64-bit for testing new code ;-). When mounting the image, I got the crash below: | attempt to access beyond end of device | loop0: rw=0, want=4920, limit=4912 | Unable to handle kernel paging request for data at address 0x0028 Offset 0x28 is buffer_head-b_data, so it seems like sb_bread returns NULL, which it does for out of range block numbers. I guess axfs_copy_block should check for that condition, as it can happen on malicious file system images. I agree that this is likely to get caused by an endianess bug. A good help for finding endianess bugs is to use __be64 like data types everywhere and test with sparse -D__CHECK_ENDIAN__. Arnd -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
The version in SVN seems to be slightly older than the one you submitted? Oops. Okay I must have neglected to sync at the very end. Thanks. I forgot, there is also a git repo at git://git.infradead.org/users/jehulber/axfs.git Which platform(s) do you use for testing? ARM, x86 I gave AxFS a try on PS3 (ppc64, always use big-endian 64-bit for testing new code ;-). Smart. Hmmm, If only I had a PS3 When mounting the image, I got the crash below: | attempt to access beyond end of device | loop0: rw=0, want=4920, limit=4912 | [cc1e34c0] [d0037974] .axfs_copy_block+0x8c/0x144 [axfs] (unreliable) | [cc1e3580] [d0035f20] .axfs_copy_metadata+0x154/0x1cc [axfs] | [cc1e3630] [d0035fd8] .axfs_verify_eofs_magic+0x40/0xa0 [axfs] | [cc1e36c0] [d0036678] .axfs_fill_super+0x3c0/0x7c8 [axfs] | [cc1e3780] [c00d1670] .get_sb_bdev+0x154/0x1ec | [cc1e3860] [d0037a94] .axfs_get_sb_bdev+0x34/0x6c [axfs] | [cc1e38f0] [d0035d0c] .axfs_get_sb+0x320/0x394 [axfs] | [cc1e3a00] [c00d1318] .vfs_kern_mount+0x88/0x108 | [cc1e3ab0] [c00d143c] .do_kern_mount+0x68/0x148 | [cc1e3b60] [c00f0a10] .do_new_mount+0x90/0xf4 | [cc1e3c10] [c00f0c5c] .do_mount+0x1e8/0x23c | [cc1e3d60] [c0114778] .compat_sys_mount+0x21c/0x2ac | [cc1e3e30] [c00074dc] syscall_exit+0x0/0x40 Yeah we've had this problem before. I'm not so sure this is an endian bug, though it is likely. When mounting (also on PS3) an image created on ia32, I get a different crash: | axfs: wrong magic So I guess some parts are not yet 64-bit or endian clean. Can you run mkfs.axfs on the same trivial directory on both ia32 and PPC64 and then get me the resulting images? -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Jared Hulbert wrote: On Fri, Aug 22, 2008 at 11:13 AM, Jamie Lokier [EMAIL PROTECTED] wrote: Greg Ungerer wrote: One thing for sure is that many people who do non-MMU setups are interested in XIP to get the space savings. These are very often small devices with very constrained RAM and flash. (For whatever it is worth single NOR flash only boards are common in these smaller form factors :-) I'm using XIP on a device with 32MB RAM. The reason I use it is _partly_ to save RAM, partly because programs start about 10 times faster (reading NOR flash is slow and I keep the XIP region in RAM) What kind of NOR you using? That is not what I measure with fast synchronous burst NOR's. I think the fast in fast synchronous gives it away :-) I'm using Spansion MirrorBit S29GL128N, which reads at about 0.6 MByte/s. Not because they're good, but because that's what the board I'm coding for has on it. I presume they were cheap and familiar to the board designers. (There is 32MB of RAM to play with after all.) So start a sequence of Busybox processes from a shell script is noticable, if it reads from NOR each time. Oh, and it's a 166MHz ARM, so it's quite capable of decompressing faster than the NOR can deliver. -- Jamie -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Geert Uytterhoeven wrote: I gave AxFS a try on PS3 (ppc64, always use big-endian 64-bit for testing new code ;-). When mounting the image, I got the crash below: | attempt to access beyond end of device | loop0: rw=0, want=4920, limit=4912 | Unable to handle kernel paging request for data at address 0x0028 | Faulting instruction address: 0xd0037988 | Oops: Kernel access of bad area, sig: 11 [#1] | SMP NR_CPUS=2 PS3 When mounting (also on PS3) an image created on ia32, I get a different crash: | axfs: wrong magic ^ | Unable to handle kernel paging request for data at address 0x03a8 | Faulting instruction address: 0xd00355f0 | Oops: Kernel access of bad area, sig: 11 [#1] | SMP NR_CPUS=2 PS Geert, Thanks for giving it a spin, especially on a platform as different from ours as the PS3. Before I dig more into what happened, I was wondering if you could tell me a bit more about your environment, particularly how you supplied the filesystem to the kernel and your mount commandline (also, if you used a boot commandline, what it was.) My first guess would be a ppc64 compiled UML session, but I'd like to be a bit more sure. Will Marone -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Jared, nice work! I've also read your paper from the linux symposium (http://ols.fedoraproject.org/OLS/Reprints-2008/hulbert-reprint.pdf) A few questions: - how does this benchmark compared to cramfs and squashfs in a NAND-only system (or is it just not a good plan to use this with NAND-only (of course I won't get XIP with NAND, I understand that) - would axfs be suitable as a filesystem on a ram disk? Background for the last question is that if you do not have the memory to retain all pages uncompressed (as you would with ramfs), this could be a nice intermediate format. Furthermore compared to ramfs, a filesystem on a ramdisk does not need the initialisation during startup (decompressing the cpio file, creating the files, copying the data), so when it comes to boot times a filesystem on a ramdisk (e.g. axfs) could be a better choice. Appreciate your feedback. Frans. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Jared Hulbert wrote: I'd like to get a first round of review on my AXFS filesystem. I like the general approach of it. It's much more flexible than the ext2 extension I've done, and the possibility to select XIP vs. compression per page is really really neat. I can imagine that people will prefer this over the ext2 implementation on s390. It is unclear to me how the secondary block device thing is supposed to work. Could you elaborate a bit on that? -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
On Thursday 21 August 2008 20:25, Carsten Otte wrote: Jared Hulbert wrote: I'd like to get a first round of review on my AXFS filesystem. I like the general approach of it. It's much more flexible than the ext2 extension I've done, and the possibility to select XIP vs. compression per page is really really neat. I can imagine that people will prefer this over the ext2 implementation on s390. It is unclear to me how the secondary block device thing is supposed to work. Could you elaborate a bit on that? Agreed. I haven't had a good look through it yet, but at a glance it looks pretty neat. The VM side of things looks pretty reasonable (I fear XIP faulting might have another race or two, but that's a core mm issue rather than filesystem specific). -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Jared Hulbert wrote: The biggest improvement is in the way AXFS allows for each page to be XIP or not. First, a user collects information about which pages are accessed on a compressed image for each mmap()ed region from /proc/axfs/volume0. That 'profile' is used as an input to the image builder. The resulting image has only the relevant pages uncompressed and XIP. The result is smaller memory sizes and faster launches. Sounds great, really nice idea. How does it fare with no MMU? Can the profiler and image builder lay out the XIP pages in such a way that no-MMU mmaps can map those regions? No complaint if not, it would be a nice bonus though. -- Jamie -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
I like the general approach of it. It's much more flexible than the ext2 extension I've done, and the possibility to select XIP vs. compression per page is really really neat. I can imagine that people will prefer this over the ext2 implementation on s390. It is unclear to me how the secondary block device thing is supposed to work. Could you elaborate a bit on that? First off we don't yet support direct_access(), but I am planning on that soon. Sure. For a system that has say a NOR Flash and a NAND or a embedded MMC, one can split a filesystem image such that only the XIP parts of the image are on the NOR while the compressed bits are on the NAND / eMMC. The NOR part is accessed as directly addressable memory, while the NAND would use mtd-read() and the eMMC would use block device access API's. In this case I would call this NAND or eMMC the secondary device because the primary device is the NOR. Assuming my NOR was at /dev//mtd2 and my NAND at /dev/mtd5. I would call the following to mount such a system: mount -t axfs -o second_dev=/dev/mtd5 /dev/mtd2 /mnt/axfs -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
Hi Jared, Jared Hulbert wrote: How does it fare with no MMU? Can the profiler and image builder lay out the XIP pages in such a way that no-MMU mmaps can map those regions? No complaint if not, it would be a nice bonus though. Sorry. I don't believe it will work on no-MMU as is. That said you _could_ tweak the mkfs tool to lay mmap()'ed regions down contiguously but then if you mmap() an unprofiled region, well that would be bad. I suppose you could make axfs_mmap smart enough to handle that. I guess the cleanest way would be to just make files lay down contiguously, you lose some of the space saving but it would work. That would be enough I think. If you could manually select which files are contiguous-and-uncompressed that would be useful for some too here. I'm not plannin to get to this anytime soon. But I'd be willing merge patches. Can anybody convince me offline that working on no-MMU this makes financial sense for my employer? This is getting to be a common question. How many noMMU users are out there and why are you so interested? One of those unknown factors, how many are there? Who knows, pretty much impossible to tell. One thing for sure is that many people who do non-MMU setups are interested in XIP to get the space savings. These are very often small devices with very constrained RAM and flash. (For whatever it is worth single NOR flash only boards are common in these smaller form factors :-) Regards Greg Greg Ungerer -- Chief Software Dude EMAIL: [EMAIL PROTECTED] Secure Computing CorporationPHONE: +61 7 3435 2888 825 Stanley St, FAX: +61 7 3891 3630 Woolloongabba, QLD, 4102, Australia WEB: http://www.SnapGear.com -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 00/10] AXFS: Advanced XIP filesystem
That would be enough I think. If you could manually select which files are contiguous-and-uncompressed that would be useful for some too here. So If you don't have an MMU when do you call -fault? Does the noMMU code just loop through -fault()ing all the pages in an mmap()? One thing for sure is that many people who do non-MMU setups are interested in XIP to get the space savings. These are very often small devices with very constrained RAM and flash. (For whatever it is worth single NOR flash only boards are common in these smaller form factors :-) True. -- To unsubscribe from this list: send the line unsubscribe linux-embedded in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
