Re: [rdiff-backup-users] initial rdiff-backup for large repository over Internet (and connection lost)

[Alvin Starr]

This is actually an excellent idea.
I do some support work for a backup company and they regularly do 
something like this.
They have a courier deliver a USB drive to the customer and an "initial 
backup" copy is put on the drive and this drive is sent back to the 
company where someone plugs it in and copies the data.


Rdiff onto a USB drive would be a good way to back the system up and 14G 
of data would fit nicely onto a USB flash drive.



On 02/25/2016 05:34 AM, David Precious wrote:

Hi Nicolas,

On Thu, 25 Feb 2016 08:57:20 +1100
Nicolas  wrote:

Now /var/www is 14G large and I need to fresh start the backup
(destination directory has been deleted).

[...]

What would be proper way to first sync rdiff-backup ?
Would a previous rsync help, so that only meta-datas would be written
by rdiff ?

If I remember rightly, what I did when I set up my large backup was use
rdiff-backup to back up to a local rdiff-backup target on a spare drive
(so all data and metadata was there), then sneakernet that to the box
in a datacentre that I wanted to back up to, then future backups over
the Internet to that target worked fine.

For 14G, I'd say you could just do the backup locally on the source
server, assuming you have enough disk space, rsync the whole thing to
the remote box (being able to continue if it's interrupted), then point
rdiff-backup at it afterwards - I'd expect that to work fine.

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki



--
Alvin Starr   ||   voice: (905)513-7688
Netvel Inc.   ||   Cell:  (416)806-0133
al...@netvel.net  ||


___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] initial rdiff-backup for large repository over Internet (and connection lost)

[David Precious]

Hi Nicolas,

On Thu, 25 Feb 2016 08:57:20 +1100
Nicolas  wrote:
> Now /var/www is 14G large and I need to fresh start the backup 
> (destination directory has been deleted).
[...]
> What would be proper way to first sync rdiff-backup ?
> Would a previous rsync help, so that only meta-datas would be written
> by rdiff ?

If I remember rightly, what I did when I set up my large backup was use
rdiff-backup to back up to a local rdiff-backup target on a spare drive
(so all data and metadata was there), then sneakernet that to the box
in a datacentre that I wanted to back up to, then future backups over
the Internet to that target worked fine.

For 14G, I'd say you could just do the backup locally on the source
server, assuming you have enough disk space, rsync the whole thing to
the remote box (being able to continue if it's interrupted), then point
rdiff-backup at it afterwards - I'd expect that to work fine.

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] initial rdiff-backup for large repository over Internet (and connection lost)

[Adrian Klaver]

On 02/24/2016 01:57 PM, Nicolas wrote:

Hi all,

I'm an happy rdiff-backup user for many years.

Years ago I setup a rdiff-backup from a WWW server to a local server.
Internet connection is limited to 2Mbps between the 2 servers.
As /var/www was initially empty, rdiff-backup as done its backup job
everyday without problem, with many sites added in /var/www.

Now /var/www is 14G large and I need to fresh start the backup
(destination directory has been deleted).

Bad thing is that trying to first sync source and destination via the
command :
nice -n 19 rdiff-backup --force $SOURCE $DEST::$DESTDIR
always ends with network (Internet error) like these :
Found interrupted initial backup. Removing...
Write failed: Broken pipe
Fatal Error: Lost connection to the remote system

What would be proper way to first sync rdiff-backup ?
Would a previous rsync help, so that only meta-datas would be written by
rdiff ?


I do not use rdiff much any more, as I switched over to 
borg-backup(http://borgbackup.readthedocs.org/en/stable/), so I am a 
little rusty on my advice. In addition to my previous post, I now 
remember something else I used to do. That is rdiff from the source to a 
local destination, then rsync the local destination to the remote 
destination, finally switch the backup to use the remote instead of the 
local destination. rsync is more forgiving of network interruptions.




Thanks in advance for your thought.
Nicolas


___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki



--
Adrian Klaver
adrian.kla...@aklaver.com

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

Re: [rdiff-backup-users] initial rdiff-backup for large repository over Internet (and connection lost)

[Adrian Klaver]

On 02/24/2016 01:57 PM, Nicolas wrote:

Hi all,

I'm an happy rdiff-backup user for many years.

Years ago I setup a rdiff-backup from a WWW server to a local server.
Internet connection is limited to 2Mbps between the 2 servers.
As /var/www was initially empty, rdiff-backup as done its backup job
everyday without problem, with many sites added in /var/www.

Now /var/www is 14G large and I need to fresh start the backup
(destination directory has been deleted).

Bad thing is that trying to first sync source and destination via the
command :
nice -n 19 rdiff-backup --force $SOURCE $DEST::$DESTDIR
always ends with network (Internet error) like these :
Found interrupted initial backup. Removing...
Write failed: Broken pipe
Fatal Error: Lost connection to the remote system

What would be proper way to first sync rdiff-backup ?
Would a previous rsync help, so that only meta-datas would be written by
rdiff ?


Take a look here:

http://www.nongnu.org/rdiff-backup/rdiff-backup.1.html

FILE SELECTION

I would use the --include-filelist option and associated file to have 
rdiff back up /var/www a subdirectory at a time. In other words add a 
subdirectory to the file and run rdiff. After that completes, add 
another subdirectory and rerun, etc. Basically build up the destination 
directory incrementally.




Thanks in advance for your thought.
Nicolas


___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL:
http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki



--
Adrian Klaver
adrian.kla...@aklaver.com

___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki

[rdiff-backup-users] initial rdiff-backup for large repository over Internet (and connection lost)

[Nicolas]

Hi all,

I'm an happy rdiff-backup user for many years.

Years ago I setup a rdiff-backup from a WWW server to a local server.
Internet connection is limited to 2Mbps between the 2 servers.
As /var/www was initially empty, rdiff-backup as done its backup job 
everyday without problem, with many sites added in /var/www.


Now /var/www is 14G large and I need to fresh start the backup 
(destination directory has been deleted).


Bad thing is that trying to first sync source and destination via the 
command :

nice -n 19 rdiff-backup --force $SOURCE $DEST::$DESTDIR
always ends with network (Internet error) like these :
Found interrupted initial backup. Removing...
Write failed: Broken pipe
Fatal Error: Lost connection to the remote system

What would be proper way to first sync rdiff-backup ?
Would a previous rsync help, so that only meta-datas would be written by 
rdiff ?


Thanks in advance for your thought.
Nicolas


___
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki