Re: Tools in Debian to create whole disk image (multiple partitions)?
On Lu, 23 iul 12, 10:48:29, Mark Neidorff wrote: On Monday 23 July 2012 2:07:42 am Andrei POPESCU wrote: On Du, 22 iul 12, 22:41:52, Gary Dale wrote: So what you really need is a copy of the files on /boot and /. You don't need the swap space and you don't need the empty space in the main partition. Nope, what I really need is something that would fit here: http://www.raspberrypi.org/downloads I think what you want to do is here: http://darkdust.net/writings/diskimagesminihowto I thought about zeroing the card in advance and also about creating the big partition smaller. There are tips in there to not copy the unpartitioned space which should help in this case ;) Another suggestionask the kind folks at RPi. (They are almost as nice as the folks on the Debian list) Sure, but I don't like forums at all :p Kind regards Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Du, 22 iul 12, 22:41:52, Gary Dale wrote: So what you really need is a copy of the files on /boot and /. You don't need the swap space and you don't need the empty space in the main partition. Nope, what I really need is something that would fit here: http://www.raspberrypi.org/downloads Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Du, 22 iul 12, 19:28:35, Rick Thomas wrote: If all the empty space is filled with something redundant (like, zeroes?) then you can use almost any compress program (gzip comes to mind...) and it will all be compressed out. If the empty space is filled with random junk, it will depend on just how random the junk is. Does that help? A compressor of course helps reduce the size a *lot* (it's only 368 MiB gziped), but this introduces an additional step that I was trying to avoid. I'll look at Colenzilla, but just creating the live USB is a pain[1] and having to reboot each time I want to generate a new image is also not something I look forward to :( [1] http://clonezilla.org/liveusb.php Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Lu, 23 iul 12, 09:15:36, Andrei POPESCU wrote: A compressor of course helps reduce the size a *lot* (it's only 368 MiB gziped), but this introduces an additional step that I was trying to avoid. ... and a gzip/gunzip cycle makes the file non-sparse. I fixed it with cp --sparse always, but this is definitely a show stopper. Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Jul 22, 2012, at 11:15 PM, Andrei POPESCU wrote: On Du, 22 iul 12, 19:28:35, Rick Thomas wrote: If all the empty space is filled with something redundant (like, zeroes?) then you can use almost any compress program (gzip comes to mind...) and it will all be compressed out. If the empty space is filled with random junk, it will depend on just how random the junk is. Does that help? A compressor of course helps reduce the size a *lot* (it's only 368 MiB gziped), but this introduces an additional step that I was trying to avoid. You can do it all in a shell one liner that does the compression on- the-fly. E.g. (assuming your SD card is on /dev/sdx, and you want to call your image raspbian.img): dd bs=1M if=/dev/sdx | gzip | dd bs=1M of=raspbian.img.gz If the swap partition has been used, it will have random garbage in it. You may want to zero it out and re-initialize it before you create the image. That will give you maximum compression. E.g. (again, assuming your SD card is on /dev/sdx, and the partition setup you gave in your previous email is still in effect): dd bs=1M if=/dev/zero of=/dev/sdx2 count=256 mkswap /dev/sdx2 Note, I don't have an Rpi and none of the above have been tested, so be careful. I'll look at Colenzilla, but just creating the live USB is a pain[1] and having to reboot each time I want to generate a new image is also not something I look forward to :( [1] http://clonezilla.org/liveusb.php I don't think clonezilla will help. The wiki page [2], which tells people how to use these images, pretty much assumes that they were created exactly as I have described above. [2] http://elinux.org/RPi_Easy_SD_Card_Setup Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic HTH, Rick -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/d56c8933-a67c-4000-9d97-fd18c38f8...@pobox.com
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Jul 23, 2012, at 12:15 AM, Andrei POPESCU wrote: On Lu, 23 iul 12, 09:15:36, Andrei POPESCU wrote: A compressor of course helps reduce the size a *lot* (it's only 368 MiB gziped), but this introduces an additional step that I was trying to avoid. ... and a gzip/gunzip cycle makes the file non-sparse. I fixed it with cp --sparse always, but this is definitely a show stopper. Well... making it non-sparse is kinda the point. The parts that -- sparse removes are a subset of the parts that gzip will give maximum compression to. Again, when I read the wiki instructions for using the images, it pretty much assumes that the image was produced with dd and gzip as I have described. Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic HTH Rick -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/fadd3e24-6ad4-4f21-a113-32a23bd5f...@pobox.com
Re: Tools in Debian to create whole disk image (multiple partitions)?
On 23/07/12 02:07 AM, Andrei POPESCU wrote: On Du, 22 iul 12, 22:41:52, Gary Dale wrote: So what you really need is a copy of the files on /boot and /. You don't need the swap space and you don't need the empty space in the main partition. Nope, what I really need is something that would fit here: http://www.raspberrypi.org/downloads Kind regards, Andrei In that case, simply do a dd but limit the number of blocks written to 2G worth. If you zero the swap partition first, this will make the file more compressible. You can also zero the empty space on the main partition, which should make it also compress quite well. BitShredder can do this. If the partition has the free space zeroed, it should compress down to a manageable size. After you've created the image, compress it. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/500d0b09.6010...@rogers.com
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Mon, 23 Jul 2012, Gary Dale wrote: compressible. You can also zero the empty space on the main partition, which should make it also compress quite well. BitShredder can do this. If the partition has the free space zeroed, it should compress down to a manageable size. On most filesystems (not on btrfs, for example), dd if=/dev/zero of=somefile bs=1M ; rm somefile (as root) will happily zero all the data area in the filesystem that hosts somefile (provided it is otherwise idle). Zeroing stale metadata areas is filesystem-version-specific, dangerous, and not worth the effort. -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120723111357.gb9...@khazad-dum.debian.net
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Mon, 23 Jul 2012, Henrique de Moraes Holschuh wrote: On Mon, 23 Jul 2012, Gary Dale wrote: compressible. You can also zero the empty space on the main partition, which should make it also compress quite well. BitShredder can do this. If the partition has the free space zeroed, it should compress down to a manageable size. On most filesystems (not on btrfs, for example), dd if=/dev/zero of=somefile bs=1M ; rm somefile (as root) will happily zero all the data area in the filesystem that hosts somefile (provided it is otherwise idle). Zeroing stale metadata areas is filesystem-version-specific, dangerous, and not worth the effort. Ah yes, better disclaim it: that's for spinning rust and RAM-based drives only! Don't do it on solid-state devices with limited write-cycles such as FLASH-based devices (i.e. SSDs, pendrives, SD cards...). -- One disk to rule them all, One disk to find them. One disk to bring them all and in the darkness grind them. In the Land of Redmond where the shadows lie. -- The Silicon Valley Tarot Henrique Holschuh -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20120723112842.gc9...@khazad-dum.debian.net
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Monday 23 July 2012 2:07:42 am Andrei POPESCU wrote: On Du, 22 iul 12, 22:41:52, Gary Dale wrote: So what you really need is a copy of the files on /boot and /. You don't need the swap space and you don't need the empty space in the main partition. Nope, what I really need is something that would fit here: http://www.raspberrypi.org/downloads I think what you want to do is here: http://darkdust.net/writings/diskimagesminihowto Seems to me that the RPi folks used this technique to create the images. Rick Thomas suggested a one-liner to create the img file that looks like it should work well. Another suggestionask the kind folks at RPi. (They are almost as nice as the folks on the Debian list) Mark -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/201207231048.29216.m...@neidorff.com
Tools in Debian to create whole disk image (multiple partitions)?
Hello list, My Raspberry Pi arrived a few days ago and yesterday I finally managed to run the installer for Raspbian (Debian wheezy armhf recompiled for the Raspberry Pi). Since the installation is not very fast due to the speed of the SD card (and I may want to contribute images anyway) I want to create an image of the SD card used for installation, possibly in different stages (fresh install, all updates, etc.). Unfortunately dd creates an image as big as the SD card (7,5 GiB) and GNU ddrescue with the --sparse option will create a file of 5,7 GiB. (the base + SSH server install is ~ 0,6 GiB) Partimage, as far as I can tell, only works on individual partitions, but I want to image the whole card. A web search seems to indicate CloneZilla might do the trick, but it's not in Debian. Any suggestions? Kind regards, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Tools in Debian to create whole disk image (multiple partitions)?
Good time of the day, Andrei. You worte: Any suggestions? Why don't You copy Your installation w/ cp -a and reconfiguring then grub for the copy - to another disk (USB one?). OR I'm missing something? Sthu. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/500bd84b.c76c980a.3f31.8...@mx.google.com
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Sun, 22 Jul 2012 10:10:04 +0300, Andrei POPESCU wrote: (...) A web search seems to indicate CloneZilla might do the trick, but it's not in Debian. Any suggestions? CloneZilla did a good job for me time ago and it can be run from a live medium, nothing needs to be installed. Greetings, -- Camaleón -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/juh03d$nk6$6...@dough.gmane.org
Re: Tools in Debian to create whole disk image (multiple partitions)?
On Du, 22 iul 12, 17:38:58, Sthu Deus wrote: Good time of the day, Andrei. You worte: Any suggestions? Why don't You copy Your installation w/ cp -a and reconfiguring then grub for the copy - to another disk (USB one?). OR I'm missing something? The Raspberry Pi can only boot from an SD card and the partition layout matters (e.g. /boot needs to be primary, FAT32, bootable and probably the first partition -- there is no bootloader like grub AFAICT). # parted -l Model: SD SD08G (sd/mmc) Disk /dev/mmcblk0: 7948MB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End SizeType File system Flags 1 1049kB 79,7MB 78,6MB primary fat32 boot 2 79,7MB 336MB 256MB primary linux-swap(v1) 3 336MB 7947MB 7612MB primary ext4 The ext4 partition only holds about 630 MiB of data (Debian base install + SSH server). I want to create an image that I can: * reuse myself later (just dd to some SD card) * distribute to possibly not very knowledgeable people Actually, I already have the image (with GNU ddrescue --sparse), but it's about 5 times bigger than expected, which makes it difficult to store and/or distribute :( Hope this explains, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic signature.asc Description: Digital signature
Re: Tools in Debian to create whole disk image (multiple partitions)?
If all the empty space is filled with something redundant (like, zeroes?) then you can use almost any compress program (gzip comes to mind...) and it will all be compressed out. If the empty space is filled with random junk, it will depend on just how random the junk is. Does that help? Rick On Jul 22, 2012, at 11:05 AM, Andrei POPESCU wrote: On Du, 22 iul 12, 17:38:58, Sthu Deus wrote: Good time of the day, Andrei. You worte: Any suggestions? Why don't You copy Your installation w/ cp -a and reconfiguring then grub for the copy - to another disk (USB one?). OR I'm missing something? The Raspberry Pi can only boot from an SD card and the partition layout matters (e.g. /boot needs to be primary, FAT32, bootable and probably the first partition -- there is no bootloader like grub AFAICT). # parted -l Model: SD SD08G (sd/mmc) Disk /dev/mmcblk0: 7948MB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End SizeType File system Flags 1 1049kB 79,7MB 78,6MB primary fat32 boot 2 79,7MB 336MB 256MB primary linux-swap(v1) 3 336MB 7947MB 7612MB primary ext4 The ext4 partition only holds about 630 MiB of data (Debian base install + SSH server). I want to create an image that I can: * reuse myself later (just dd to some SD card) * distribute to possibly not very knowledgeable people Actually, I already have the image (with GNU ddrescue --sparse), but it's about 5 times bigger than expected, which makes it difficult to store and/or distribute :( Hope this explains, Andrei -- Offtopic discussions among Debian users and developers: http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/f1aabf71-4355-49db-9cfd-9dc093a90...@pobox.com
Re: Tools in Debian to create whole disk image (multiple partitions)?
On 22/07/12 02:05 PM, Andrei POPESCU wrote: On Du, 22 iul 12, 17:38:58, Sthu Deus wrote: Good time of the day, Andrei. You worte: Any suggestions? Why don't You copy Your installation w/ cp -a and reconfiguring then grub for the copy - to another disk (USB one?). OR I'm missing something? The Raspberry Pi can only boot from an SD card and the partition layout matters (e.g. /boot needs to be primary, FAT32, bootable and probably the first partition -- there is no bootloader like grub AFAICT). # parted -l Model: SD SD08G (sd/mmc) Disk /dev/mmcblk0: 7948MB Sector size (logical/physical): 512B/512B Partition Table: msdos Number Start End SizeType File system Flags 1 1049kB 79,7MB 78,6MB primary fat32 boot 2 79,7MB 336MB 256MB primary linux-swap(v1) 3 336MB 7947MB 7612MB primary ext4 The ext4 partition only holds about 630 MiB of data (Debian base install + SSH server). I want to create an image that I can: * reuse myself later (just dd to some SD card) * distribute to possibly not very knowledgeable people Actually, I already have the image (with GNU ddrescue --sparse), but it's about 5 times bigger than expected, which makes it difficult to store and/or distribute :( Hope this explains, Andrei So what you really need is a copy of the files on /boot and /. You don't need the swap space and you don't need the empty space in the main partition. You can use fdisk to copy the partition information and a couple of cp -a or tar's to copy the other files. You could even use dd to copy the boot partition since it is pretty full. This could be put into a restore script where you just need to identify the SD destination card. However, unless the SD card is the same size, the main partition won't fill the space. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/500cb9f0.4030...@rogers.com
