On Fri, Apr 8, 2016 at 12:14 PM, Joshua Lock <[email protected]> wrote:
> From: Joshua Lock <[email protected]> > > The individual partitions created by wic are sparse but without > this change the assembled image is written as one (potentially > very) large file. > > Preserve sparseness in the assembled image by passing the sparse > conversion symbol. > > [YOCTO #9099] > > Signed-off-by: Joshua Lock <[email protected]> > --- > scripts/lib/wic/utils/partitionedfs.py | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/scripts/lib/wic/utils/partitionedfs.py > b/scripts/lib/wic/utils/partitionedfs.py > index 5a103bb..3e2b420 100644 > --- a/scripts/lib/wic/utils/partitionedfs.py > +++ b/scripts/lib/wic/utils/partitionedfs.py > @@ -340,7 +340,7 @@ class Image(object): > source = part['source_file'] > if source: > # install source_file contents into a partition > - cmd = "dd if=%s of=%s bs=%d seek=%d count=%d > conv=notrunc" % \ > + cmd = "dd if=%s of=%s bs=%d seek=%d count=%d > conv=notrunc,sparse" % \ > (source, image_file, self.sector_size, > part['start'], part['size']) > exec_cmd(cmd) > This might be dangerous. Even in recent version of DD (coreutils 8.25), it might not detect in source file mapped/unmapped blocks properly, (just use is_nul() function to check if block contains only zeros, instead of e.g. fiemap). Result might be so, that in output file block which must be zeros is actually become unmapped. And as consequence, if it might not be written to destination drive. Let me show example based on Ostro Project sparse images: kad@orava:/tmp> ls -ls 2222 308416 -rw-r--r-- 1 kad users 5280628736 Apr 6 00:53 2222 kad@orava:/tmp> grep /MappedBlocksCount 2222.bmap <MappedBlocksCount> 77104 </MappedBlocksCount> Now, let's copy it with dd: kad@orava:/tmp> ~/cu/coreutils-8.25/src/dd if=2222 of=4444 conv=notrunc,sparse bs=4096 1289216+0 records in 1289216+0 records out 5280628736 bytes (5.3 GB, 4.9 GiB) copied, 1.31239 s, 4.0 GB/s kad@orava:/tmp> bmaptool create 4444 -o 4444.bmap kad@orava:/tmp> ls -ls 4444 308348 -rw-r--r-- 1 kad users 5280628736 Apr 9 13:22 4444 kad@orava:/tmp> grep /MappedBlocksCount 4444.bmap <MappedBlocksCount> 77087 </MappedBlocksCount> kad@orava:/tmp> As you can see, in result of dd copied file we have less blocks mapped than in original. Diffing block ranges: (- original file, + dd copied) <BlockMap> - <Range> 0-4 </Range> - <Range> 768-3321 </Range> - <Range> 4608-7161 </Range> + <Range> 0 </Range> + <Range> 768-770 </Range> + <Range> 772-774 </Range> + <Range> 776 </Range> + <Range> 780-3321 </Range> + <Range> 4608-4610 </Range> + <Range> 4612-4614 </Range> + <Range> 4616 </Range> + <Range> 4620-7161 </Range> - <Range> 1289211-1289215 </Range> + <Range> 1289211 </Range> + <Range> 1289215 </Range> </BlockMap> As you can see, some blocks that must be cleaned (e.g. 1-3 around MBR or in the end, 1289211-1289215 where GPT bits located) are not marked as mapped, thus in case of sparse aware writing tool used to write to destination drive it might result as unwritten zero blocks, thus some old data would left on device. I would suggest not to accept this patch right now, and do copy function that would be aware about real mapped/unmapped blocks. Something similar that I've did for Ostro Project here: https://github.com/ostroproject/meta-ostro/commit/d883169c3266c3c44f22db5f75ae94c4cf0f2924 but adopted to be generic for use inside wic. -- br, Alexander Kanevskiy
-- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
