Re: [U-Boot] Fastboot behaviour with sparse images
Hi Colin, On Tue, Oct 27, 2015 at 02:00:24PM -0700, Colin Cross wrote: > On Monday, October 12, 2015 at 6:43:37 AM UTC-7, Maxime Ripard wrote: > > > > Hi, > > > > I'm currently writing the support in U-Boot for NAND-backed devices > > using fastboot [1], and that work derived a bit to supporting the > > sparse images. > > > > For "regular" images that are being stored, we expect a pair of > > download and flash commands. Simple. > > > > Things start to get a bit more complex with sparse images that have > > been split because of a max-download-size lower than the actual image > > size. > > > > Here, from what I could gather from various random blog posts, the > > fastboot client implementation and dumping a few USB sessions, the > > client simply creates several download / flash pairs, always on the > > same partition, without any way to distinct that from several > > subsequent writes issued by the user. > > > > So, I'm guessing that the expectation is that the bootloader > > implementation should store the last offset it wrote to, and simple > > resume from there if the partition names in the flash commands are the > > same, which would prevent two subsequent write on the same partition > > by any client. Am I right? > > > > No, each blob passed to the bootloader will begin with a sparse "skip" > chunk that will seek to the correct place to resume writing. The > bootloader shouldn't need to store any metadata across commands. Just read > in the blob from the data command, then write it out using a port of the > Apache-licensed libsparse during the flash command. Oh, so that's how it works. Great. I guess however that you still need to scan out the area you skip for bad blocks to account them in the offset calculation as well then (when you're using NAND). > > A related question is when should we erase the NAND partition? Only > > when doing fastboot erase, or also when doing fastboot write (which, > > combined with the issue raised above, would also mean that we don't > > want to do an erase on the whole partition everytime there's a flash > > command on it). > > Fastboot should send an erase command before every sequence of writes. > Erase the whole partition on the erase command, and don't erase anything > on the flash command. Ack. Thanks! Maxime -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] Fastboot behaviour with sparse images
On Monday, October 12, 2015 at 6:43:37 AM UTC-7, Maxime Ripard wrote: > > Hi, > > I'm currently writing the support in U-Boot for NAND-backed devices > using fastboot [1], and that work derived a bit to supporting the > sparse images. > > For "regular" images that are being stored, we expect a pair of > download and flash commands. Simple. > > Things start to get a bit more complex with sparse images that have > been split because of a max-download-size lower than the actual image > size. > > Here, from what I could gather from various random blog posts, the > fastboot client implementation and dumping a few USB sessions, the > client simply creates several download / flash pairs, always on the > same partition, without any way to distinct that from several > subsequent writes issued by the user. > > So, I'm guessing that the expectation is that the bootloader > implementation should store the last offset it wrote to, and simple > resume from there if the partition names in the flash commands are the > same, which would prevent two subsequent write on the same partition > by any client. Am I right? > No, each blob passed to the bootloader will begin with a sparse "skip" chunk that will seek to the correct place to resume writing. The bootloader shouldn't need to store any metadata across commands. Just read in the blob from the data command, then write it out using a port of the Apache-licensed libsparse during the flash command. > A related question is when should we erase the NAND partition? Only > when doing fastboot erase, or also when doing fastboot write (which, > combined with the issue raised above, would also mean that we don't > want to do an erase on the whole partition everytime there's a flash > command on it). Fastboot should send an erase command before every sequence of writes. Erase the whole partition on the erase command, and don't erase anything on the flash command. > > Thanks! > Maxime > > 1: http://lists.denx.de/pipermail/u-boot/2015-August/226053.html > > -- > Maxime Ripard, Free Electrons > Embedded Linux, Kernel and Android engineering > http://free-electrons.com > ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] Fastboot behaviour with sparse images
Hi Tom, On Wed, Oct 21, 2015 at 10:09:28AM -0400, Tom Rini wrote: > On Mon, Oct 12, 2015 at 03:43:01PM +0200, Maxime Ripard wrote: > > Hi, > > > > I'm currently writing the support in U-Boot for NAND-backed devices > > using fastboot [1], and that work derived a bit to supporting the > > sparse images. > > > > For "regular" images that are being stored, we expect a pair of > > download and flash commands. Simple. > > > > Things start to get a bit more complex with sparse images that have > > been split because of a max-download-size lower than the actual image > > size. > > > > Here, from what I could gather from various random blog posts, the > > fastboot client implementation and dumping a few USB sessions, the > > client simply creates several download / flash pairs, always on the > > same partition, without any way to distinct that from several > > subsequent writes issued by the user. > > > > So, I'm guessing that the expectation is that the bootloader > > implementation should store the last offset it wrote to, and simple > > resume from there if the partition names in the flash commands are the > > same, which would prevent two subsequent write on the same partition > > by any client. Am I right? > > > > A related question is when should we erase the NAND partition? Only > > when doing fastboot erase, or also when doing fastboot write (which, > > combined with the issue raised above, would also mean that we don't > > want to do an erase on the whole partition everytime there's a flash > > command on it). > > I think for this last question, some experimentation with the existing > tools might be required. As there's no required explicit erase for MMC, > I think it might make sense to say you erase nand up front and then > write as anything else starts getting really tricky and we're just > second-guessing the user. Actually, the only FS the fastboot tool seems to be doing it for the moment are ext4 and F2FS. It can probably be extended to UBI and raw partitions, but that won't fix the tools that are bundled by the distros at the moment. So I guess we can always erase it now using the session counter: if we are writing the first chunk, erase the whole partition, if we're not, then simply flash it at the previous offset. How does it sound? Maxime -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] Fastboot behaviour with sparse images
On Thu, Oct 22, 2015 at 07:22:54AM -0400, Tom Rini wrote: > On Thu, Oct 22, 2015 at 09:57:05AM +0200, Maxime Ripard wrote: > > Hi Tom, > > > > On Wed, Oct 21, 2015 at 10:09:28AM -0400, Tom Rini wrote: > > > On Mon, Oct 12, 2015 at 03:43:01PM +0200, Maxime Ripard wrote: > > > > Hi, > > > > > > > > I'm currently writing the support in U-Boot for NAND-backed devices > > > > using fastboot [1], and that work derived a bit to supporting the > > > > sparse images. > > > > > > > > For "regular" images that are being stored, we expect a pair of > > > > download and flash commands. Simple. > > > > > > > > Things start to get a bit more complex with sparse images that have > > > > been split because of a max-download-size lower than the actual image > > > > size. > > > > > > > > Here, from what I could gather from various random blog posts, the > > > > fastboot client implementation and dumping a few USB sessions, the > > > > client simply creates several download / flash pairs, always on the > > > > same partition, without any way to distinct that from several > > > > subsequent writes issued by the user. > > > > > > > > So, I'm guessing that the expectation is that the bootloader > > > > implementation should store the last offset it wrote to, and simple > > > > resume from there if the partition names in the flash commands are the > > > > same, which would prevent two subsequent write on the same partition > > > > by any client. Am I right? > > > > > > > > A related question is when should we erase the NAND partition? Only > > > > when doing fastboot erase, or also when doing fastboot write (which, > > > > combined with the issue raised above, would also mean that we don't > > > > want to do an erase on the whole partition everytime there's a flash > > > > command on it). > > > > > > I think for this last question, some experimentation with the existing > > > tools might be required. As there's no required explicit erase for MMC, > > > I think it might make sense to say you erase nand up front and then > > > write as anything else starts getting really tricky and we're just > > > second-guessing the user. > > > > Actually, the only FS the fastboot tool seems to be doing it for the > > moment are ext4 and F2FS. It can probably be extended to UBI and raw > > partitions, but that won't fix the tools that are bundled by the > > distros at the moment. > > > > So I guess we can always erase it now using the session counter: if we > > are writing the first chunk, erase the whole partition, if we're not, > > then simply flash it at the previous offset. > > > > How does it sound? > > Sounds workable but testing with the existing tools will be the key and > the hard part here :( Well, if we always erase when we write, the worst case scenario would be one erase too many. It doesn't sound that bad, or hard. Maxime -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] Fastboot behaviour with sparse images
On Thu, Oct 22, 2015 at 09:57:05AM +0200, Maxime Ripard wrote: > Hi Tom, > > On Wed, Oct 21, 2015 at 10:09:28AM -0400, Tom Rini wrote: > > On Mon, Oct 12, 2015 at 03:43:01PM +0200, Maxime Ripard wrote: > > > Hi, > > > > > > I'm currently writing the support in U-Boot for NAND-backed devices > > > using fastboot [1], and that work derived a bit to supporting the > > > sparse images. > > > > > > For "regular" images that are being stored, we expect a pair of > > > download and flash commands. Simple. > > > > > > Things start to get a bit more complex with sparse images that have > > > been split because of a max-download-size lower than the actual image > > > size. > > > > > > Here, from what I could gather from various random blog posts, the > > > fastboot client implementation and dumping a few USB sessions, the > > > client simply creates several download / flash pairs, always on the > > > same partition, without any way to distinct that from several > > > subsequent writes issued by the user. > > > > > > So, I'm guessing that the expectation is that the bootloader > > > implementation should store the last offset it wrote to, and simple > > > resume from there if the partition names in the flash commands are the > > > same, which would prevent two subsequent write on the same partition > > > by any client. Am I right? > > > > > > A related question is when should we erase the NAND partition? Only > > > when doing fastboot erase, or also when doing fastboot write (which, > > > combined with the issue raised above, would also mean that we don't > > > want to do an erase on the whole partition everytime there's a flash > > > command on it). > > > > I think for this last question, some experimentation with the existing > > tools might be required. As there's no required explicit erase for MMC, > > I think it might make sense to say you erase nand up front and then > > write as anything else starts getting really tricky and we're just > > second-guessing the user. > > Actually, the only FS the fastboot tool seems to be doing it for the > moment are ext4 and F2FS. It can probably be extended to UBI and raw > partitions, but that won't fix the tools that are bundled by the > distros at the moment. > > So I guess we can always erase it now using the session counter: if we > are writing the first chunk, erase the whole partition, if we're not, > then simply flash it at the previous offset. > > How does it sound? Sounds workable but testing with the existing tools will be the key and the hard part here :( -- Tom signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
Re: [U-Boot] Fastboot behaviour with sparse images
On Mon, Oct 12, 2015 at 03:43:01PM +0200, Maxime Ripard wrote: > Hi, > > I'm currently writing the support in U-Boot for NAND-backed devices > using fastboot [1], and that work derived a bit to supporting the > sparse images. > > For "regular" images that are being stored, we expect a pair of > download and flash commands. Simple. > > Things start to get a bit more complex with sparse images that have > been split because of a max-download-size lower than the actual image > size. > > Here, from what I could gather from various random blog posts, the > fastboot client implementation and dumping a few USB sessions, the > client simply creates several download / flash pairs, always on the > same partition, without any way to distinct that from several > subsequent writes issued by the user. > > So, I'm guessing that the expectation is that the bootloader > implementation should store the last offset it wrote to, and simple > resume from there if the partition names in the flash commands are the > same, which would prevent two subsequent write on the same partition > by any client. Am I right? > > A related question is when should we erase the NAND partition? Only > when doing fastboot erase, or also when doing fastboot write (which, > combined with the issue raised above, would also mean that we don't > want to do an erase on the whole partition everytime there's a flash > command on it). I think for this last question, some experimentation with the existing tools might be required. As there's no required explicit erase for MMC, I think it might make sense to say you erase nand up front and then write as anything else starts getting really tricky and we're just second-guessing the user. -- Tom signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
[U-Boot] Fastboot behaviour with sparse images
Hi, I'm currently writing the support in U-Boot for NAND-backed devices using fastboot [1], and that work derived a bit to supporting the sparse images. For "regular" images that are being stored, we expect a pair of download and flash commands. Simple. Things start to get a bit more complex with sparse images that have been split because of a max-download-size lower than the actual image size. Here, from what I could gather from various random blog posts, the fastboot client implementation and dumping a few USB sessions, the client simply creates several download / flash pairs, always on the same partition, without any way to distinct that from several subsequent writes issued by the user. So, I'm guessing that the expectation is that the bootloader implementation should store the last offset it wrote to, and simple resume from there if the partition names in the flash commands are the same, which would prevent two subsequent write on the same partition by any client. Am I right? A related question is when should we erase the NAND partition? Only when doing fastboot erase, or also when doing fastboot write (which, combined with the issue raised above, would also mean that we don't want to do an erase on the whole partition everytime there's a flash command on it). Thanks! Maxime 1: http://lists.denx.de/pipermail/u-boot/2015-August/226053.html -- Maxime Ripard, Free Electrons Embedded Linux, Kernel and Android engineering http://free-electrons.com signature.asc Description: Digital signature ___ U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
