Re: [Dorset] Duplicating uSD cards
On 16/06/2020 07:27, PeterMerchant wrote: On 15/06/2020 22:46, Andrew wrote: I've just remembered, for a Raspberry Pi you don't need to duplicate the entire contents of the µSD card - you can create a new partition table and filesystems (FAT32 for boot, EXT4 for root) on the blank µSD card and copy everything using regular file copy commands. You might need to edit what will become /etc/fstab to update the UUID of the new root filesystem. (And any other filesystems it needs to mount.) It's not quite that simple somehow. To get the two partitions to the right size, I just installed an image of Raspbian stretch on the SD card, Then connected the 'master' card to the computer. I used Dolphin in split mode and (Being extra careful) I was able to copy the contents of the boot partition across ok, but I was not permitted to write to the rootfs partition except in the home directory. I tried all sorts of ways of changing the permissions without success. By going into terminal mode I was able to edit things like dhcpcd.conf so that I can fire up the R-Pi and connect to a known IP address. So I am now updating and going to go through the rebuild process to get to my desired starting point for playing. Peter -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
On Tuesday, 16 June 2020 07:27:51 BST PeterMerchant wrote: > On 15/06/2020 22:46, Andrew wrote: > > I've just remembered, for a Raspberry Pi you don't need to duplicate the > > entire contents of the µSD card - you can create a new partition table > > and filesystems (FAT32 for boot, EXT4 for root) on the blank µSD card and > > copy everything using regular file copy commands. You might need to edit > > what will become /etc/fstab to update the UUID of the new root > > filesystem. (And any other filesystems it needs to mount.) > > > > The Raspberry Pi bootloader knows how to read a partition table and FAT32 > > filesystem to load the kernel, so there's no magic hidden data on the > > disk like with IBM PC-BIOS type booting. > > > > The same goes for UEFI system bootloaders - they understand partition > > tables and filesystems. > That's cool Andrew, I had wondered about that. If what I am doing doesn't > work, I'll clear off all the files on the SD card and copy over the files > from the good one. Unfortunately, on it's own, that doesn't help if you have a (slightly) too large image and a (slightly) too small new SD Card to copy it to. You need to be able to see the files to copy them. It is possible to mount the image as loopback and grab the files that way. That can be a bit of a faff unless your bash knowledge is at your fingertips. In Gnome, I think there is a feature in the File Manager that will do that for you. Otherwise, Hamish's DD_Rescue_GUI tool does it as he demonstrated at the first online meeting. -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
On 15/06/2020 22:46, Andrew wrote: I've just remembered, for a Raspberry Pi you don't need to duplicate the entire contents of the µSD card - you can create a new partition table and filesystems (FAT32 for boot, EXT4 for root) on the blank µSD card and copy everything using regular file copy commands. You might need to edit what will become /etc/fstab to update the UUID of the new root filesystem. (And any other filesystems it needs to mount.) The Raspberry Pi bootloader knows how to read a partition table and FAT32 filesystem to load the kernel, so there's no magic hidden data on the disk like with IBM PC-BIOS type booting. The same goes for UEFI system bootloaders - they understand partition tables and filesystems. That's cool Andrew, I had wondered about that. If what I am doing doesn't work, I'll clear off all the files on the SD card and copy over the files from the good one. Cheers, Peter M. -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
I've just remembered, for a Raspberry Pi you don't need to duplicate the entire contents of the µSD card - you can create a new partition table and filesystems (FAT32 for boot, EXT4 for root) on the blank µSD card and copy everything using regular file copy commands. You might need to edit what will become /etc/fstab to update the UUID of the new root filesystem. (And any other filesystems it needs to mount.) The Raspberry Pi bootloader knows how to read a partition table and FAT32 filesystem to load the kernel, so there's no magic hidden data on the disk like with IBM PC-BIOS type booting. The same goes for UEFI system bootloaders - they understand partition tables and filesystems. -- Andrew. -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
On 15/06/2020 11:36, Terry Coles wrote: On Wednesday, 10 June 2020 16:06:01 BST Terry Coles wrote: Nowadays I try to avoid the problem by shrinking the original partition a bit before I dd it to create the image; you should be able to do it with GParted. Then, when I copy it to the new card, the image should fit. I think I was fooling myself when I did this back in the day. We bought a supply of Patriot 8 GB Micro SD Cards and I suspect that these had the most capacity out of the selection that I had from Verbatim, Sandisk, etc. I was probably using those... Anyway, I took an image of a card at the beginning of the month and when I tried to copy it to a new card, it wouldn't fit ;-( After various shenanigans I posted a query on the Raspberry Pi Forums and got several responses see: https://www.raspberrypi.org/forums/viewtopic.php? f=28=277218=1679441#p1679441 Out of the several options offered, I managed to fix the problem using a tool called image-backup. It turns out that image-backup is only part of a suite of utilities called image-utils; see the discussion at the link for more info. I downloaded this and initially used the included image-shrink utility to reduce the size of the image that I had saved earlier. The process completed without error, but it wouldn't mount, so there is clearly something wrong with the original image. However I was able to successfully write it to a 16 GB card and it subsequently booted with no problems. So then, from within the running system, I used image-backup to save a backup of the running system onto a memory stick plugged into a spare USB connector on the Pi. This produced a 2.3 GB image instead of a 7.5 GB one and I was able to successfully write that to an 8 GB SD Card and boot from that too. (Note: Patience is needed because there is no feedback and it checks the resultant image before exiting.) There is an option to not expand the file-system on first boot, but now I know I can create usable images using this tool, I didn't use that. A highly recommended tool. Thanks for that. I gave up trying to duplicate the cards, and instead ran through my procedure for creating a new system, but using the latest Raspberry Pi OS. There seems to be a fault with that or maybe I am just premature in that comment, as I had everything working until I tried to add the software to make the Pi a wifi access point. As a PS to that the R-Pi documentation for doing that has disappeared and been replaced by a different set of instructions with more capability. When you have a standalone Pi configured for a different ESSID and IP addresses and the Wifi doesn't work, you are a bit stuffed. So now I have reverted to the latest Raspbian Buster and am trying again. I don't let it keep me awake nights. Peter -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
On Wednesday, 10 June 2020 16:06:01 BST Terry Coles wrote: > Nowadays I try to avoid the problem by shrinking the original partition a > bit before I dd it to create the image; you should be able to do it with > GParted. Then, when I copy it to the new card, the image should fit. I think I was fooling myself when I did this back in the day. We bought a supply of Patriot 8 GB Micro SD Cards and I suspect that these had the most capacity out of the selection that I had from Verbatim, Sandisk, etc. I was probably using those... Anyway, I took an image of a card at the beginning of the month and when I tried to copy it to a new card, it wouldn't fit ;-( After various shenanigans I posted a query on the Raspberry Pi Forums and got several responses see: https://www.raspberrypi.org/forums/viewtopic.php? f=28=277218=1679441#p1679441 Out of the several options offered, I managed to fix the problem using a tool called image-backup. It turns out that image-backup is only part of a suite of utilities called image-utils; see the discussion at the link for more info. I downloaded this and initially used the included image-shrink utility to reduce the size of the image that I had saved earlier. The process completed without error, but it wouldn't mount, so there is clearly something wrong with the original image. However I was able to successfully write it to a 16 GB card and it subsequently booted with no problems. So then, from within the running system, I used image-backup to save a backup of the running system onto a memory stick plugged into a spare USB connector on the Pi. This produced a 2.3 GB image instead of a 7.5 GB one and I was able to successfully write that to an 8 GB SD Card and boot from that too. (Note: Patience is needed because there is no feedback and it checks the resultant image before exiting.) There is an option to not expand the file-system on first boot, but now I know I can create usable images using this tool, I didn't use that. A highly recommended tool. -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
On Wednesday, 10 June 2020 16:04:44 BST PeterMerchant wrote: > I am sure that I used the Raspbian operation to expand the filesystem when I > first used the card. Older versions of Raspbian used to leave it up to you to expand the filesystem. I never used to do it to prevent this very problem. Newer versions, including the new Raspberry Pi OS, just do it on first boot ;-( -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
On Wednesday, 10 June 2020 15:25:03 BST PeterMerchant wrote: > I have been trying to duplicate a 16GB MicroSD card to another and it > hasn't worked. The original was a TDK 16GB card, which I copied to my > computer with DD. > > I then tried to use DD to copy it to the Toshiba card, with an error message > [now lost]. > > So I tried to use etcher and it tells me that the image is too large for the > destination card. > > What has happened? Do I need to add some parameters when backing up the > disk? Peter, This is an old problem with SD Cards. Manufacturers sell these devices with a nominal capacity of 8 GB, 16 GB, etc, but in reality they are not all the same. So if you originally ran dd to save an image of a card that had a capacity of 15.91 GB and then try to copy it to a card that has a capacity of 15.9 GB, it will fail. (Even if the partition is not full, dd will faithfully copy all the empty sectors.) There are ways of shrinking the size of the image, but that involves some magic with GParted, some remounting of the image in loopback mode plus some mind-bending arithmetic. I've never had any success with that. Nowadays I try to avoid the problem by shrinking the original partition a bit before I dd it to create the image; you should be able to do it with GParted. Then, when I copy it to the new card, the image should fit. Of course if the disc is full... -- Terry Coles -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
On 10/06/2020 15:54, Ralph Corderoy wrote: Hi Peter, So I tried to use etcher and it tells me that the image is too large for the destination card. Perhaps they're different sizes. Use ‘sudo lsblk -b’ to get a list of current block devices and their size in bytes. You can ignore the partitions. Gather the size of both SD cards and compare. If source > destination then that explains the error. You can also compare source size in bytes to the size of the image file you've created, just to see they match. It might be that you can shrink the source's filesystem(s) and partition(s) to leave a run of sectors at the end of the device unused. The used sectors starting from the beginning of the block device may then fit in the destination. I see what you mean: NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT (TDK) sdb 8:16 1 15720251392 0 disk ├─sdb1 8:17 1 46005248 0 part └─sdb2 8:18 1 15669919744 0 part sr0 11:0 1 1073741312 0 rom sr1 11:1 1 1073741312 0 rom peterm@peterm-MBB-34204H:~$ sudo lsblk -b NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT (Toshiba) sdb 8:16 1 15552479232 0 disk ├─sdb1 8:17 1 46005248 0 part └─sdb2 8:18 1 15502147584 0 part sr0 11:0 1 1073741312 0 rom sr1 11:1 1 1073741312 0 rom I am sure that I used the Raspbian operation to expand the filesystem when I first used the card. I just wondered if the bit that is missing was needed, and it must be because the card won't boot.! Peter -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
Re: [Dorset] Duplicating uSD cards
Hi Peter, > So I tried to use etcher and it tells me that the image is too large > for the destination card. Perhaps they're different sizes. Use ‘sudo lsblk -b’ to get a list of current block devices and their size in bytes. You can ignore the partitions. Gather the size of both SD cards and compare. If source > destination then that explains the error. You can also compare source size in bytes to the size of the image file you've created, just to see they match. It might be that you can shrink the source's filesystem(s) and partition(s) to leave a run of sectors at the end of the device unused. The used sectors starting from the beginning of the block device may then fit in the destination. -- Cheers, Ralph. -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
[Dorset] Duplicating uSD cards
I have been trying to duplicate a 16GB MicroSD card to another and it hasn't worked. The original was a TDK 16GB card, which I copied to my computer with DD. I then tried to use DD to copy it to the Toshiba card, with an error message [now lost]. So I tried to use etcher and it tells me that the image is too large for the destination card. What has happened? Do I need to add some parameters when backing up the disk? Peter -- Next meeting: Online, Jitsi, Tuesday, 2020-07-07 20:00 Check to whom you are replying Meetings, mailing list, IRC, ... http://dorset.lug.org.uk New thread, don't hijack: mailto:dorset@mailman.lug.org.uk
