Re: [BackupPC-users] Moving /var/lib/BackupPC to a new disk fails with rsync and OOM

[Johan Ehnberg]

On 09/06/2016 04:46 PM, Colin wrote:
> Thank you Benjamin.
>
> I ended doing the dd like:
> dd if=/dev/sdb1 | pv -s 400G | dd of=/dev/sdc1 bs=16M
>
> That took around 5h total, which was great!
> Being stuck with the same filesystem sucks but it seems without any
> bigger window of downtime, there isn't any tool that would be possible
> to use.
>
> Thank you all!
>
> Cheers,
> Colin
>

For future reference, there's Holger's script and this one I created for 
migrations such as your case:

http://johan.ehnberg.net/backuppc-pre-loading-seeding-and-migrating-moving-script/

Giving up the pool and backup history may not be feasible though, so 
beware: my script starts a fresh pool.

Best regards,
Johan Ehnberg

--
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Moving /var/lib/BackupPC to a new disk fails with rsync and OOM

[Holger Parplies]

Hi,

Colin wrote on 2016-09-05 17:46:34 +0100 [Re: [BackupPC-users] Moving 
/var/lib/BackupPC to a new disk fails with rsync and OOM]:
> Thank you Les!
> I'll have a go tomorrow after backups with something similar as:
> dd if=/dev/sdb1 of=/dev/sdc1 bs=16M
> [...]
> On 5 September 2016 at 17:03, Les Mikesell  wrote:
> > On Mon, Sep 5, 2016 at 10:07 AM, Colin  wrote:
> > > [...]
> > > I currently have lots of data on a normal parition (no LVM) and when
> > trying
> > > to copying to a new disk, it always fail after a while:
> > > [...]
> > Your best shot would be to use dd to copy the partition to the new
> > disk, then grow the filesystem to fill the space using the filesystem
> > tools. [...]

if that (at the top ... would have been at the bottom without top-posting)
doesn't work for some reason or you want to change file system types, then
please get back to us. I have a script that handles it at the file level,
though not well tested. And dd will definitely be faster, so if that works
for you, it is certainly the better option.

Regards,
Holger

--
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Moving /var/lib/BackupPC to a new disk fails with rsync and OOM

[Colin]

Thank you Benjamin.

I ended doing the dd like:
dd if=/dev/sdb1 | pv -s 400G | dd of=/dev/sdc1 bs=16M

That took around 5h total, which was great!
Being stuck with the same filesystem sucks but it seems without any bigger
window of downtime, there isn't any tool that would be possible to use.

Thank you all!

Cheers,
Colin

On 6 September 2016 at 09:36, Benjamin Redling 
wrote:

> Am 05.09.2016 um 17:07 schrieb Colin:
> > # df -h /var/lib/BackupPC
> > Filesystem  Size  Used Avail Use% Mounted on
> > /dev/sdb1   493G  394G   74G  85% /var/lib/BackupPC
> >
> > # rsync -aH /var/lib/BackupPC/. /mnt/.
>
> > So far I've tried just rsync'ing individual directories under
> > /var/lib/BackupPC but in the end, the destination appears to use almost
> > double of the space of the current one.
>
> rsync to an empty dir/disks never(?) makes any sense.
> Additionally any file based approach will be hit hard  by hard links.
>
> > Another solution I tried was dump/restore but it just takes too long:
> > longer than a day and backup are daily so this solution doesn't fit.
> >
> > Any ideas ?
>
> At least an idea to get the same file system to a bigger disk:
> dd
> -- or anything else for cloning a partition.
>
> I migrated a full 1TB partition via dd to a 2.2TB relatively fast --
> from a remote VM to a local host, the throughput was limited by that VM
> anyway.
> I only had to resize it afterwards.
> The main drawback of that method is, that you would be stuck with the
> same file system type.
>
> Regards,
> Benjamin
> --
> FSU Jena | JULIELab.de/Staff/Benjamin+Redling.html
> vox: +49 3641 9 44323 | fax: +49 3641 9 44321
>
> 
> --
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
--
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Moving /var/lib/BackupPC to a new disk fails with rsync and OOM

[Benjamin Redling]

Am 05.09.2016 um 17:07 schrieb Colin:
> # df -h /var/lib/BackupPC
> Filesystem  Size  Used Avail Use% Mounted on
> /dev/sdb1   493G  394G   74G  85% /var/lib/BackupPC
> 
> # rsync -aH /var/lib/BackupPC/. /mnt/.

> So far I've tried just rsync'ing individual directories under
> /var/lib/BackupPC but in the end, the destination appears to use almost
> double of the space of the current one.

rsync to an empty dir/disks never(?) makes any sense.
Additionally any file based approach will be hit hard  by hard links.

> Another solution I tried was dump/restore but it just takes too long:
> longer than a day and backup are daily so this solution doesn't fit.
> 
> Any ideas ?

At least an idea to get the same file system to a bigger disk:
dd
-- or anything else for cloning a partition.

I migrated a full 1TB partition via dd to a 2.2TB relatively fast --
from a remote VM to a local host, the throughput was limited by that VM
anyway.
I only had to resize it afterwards.
The main drawback of that method is, that you would be stuck with the
same file system type.

Regards,
Benjamin
-- 
FSU Jena | JULIELab.de/Staff/Benjamin+Redling.html
vox: +49 3641 9 44323 | fax: +49 3641 9 44321

--
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/