Re: [BackupPC-users] Backing up the BackupPC pool

[Bzzzz]

On Fri, 11 Aug 2017 07:12:30 +0200
Hannes Elvemyr  wrote:

> I'm taking about BPC itself. Why? Let's take an example.
> 
> 00:30 My nightly rsync script starts to sync pool to off-site storage.
> This night, there happens to be a lot of new data an it takes time.
> 01:00 BackupPC_Nightly starts cleaning the pool! Rsync is still
> running! 01:15 My nightly rsync script is done syncing

(disclaimer: I'm using v3) this isn't a problem, I've several machines
that keep BPC busy for +22 HRS w/  BackupPC_Nightly running behind;
the only thing is that makes the unfinished backups loger to achieve.

> This night I would end up with a corrupt copy since BPC wrote to the
> pool while I was copying it.

I never saw any corruption in former backups.
>From what I've put on the back of BPC, I can tell you it is _very_
reliable in any normal situation (power outage included; I exclude any
hardware fault or failure of course.)

> One suggestion to prevent BPC to touch
> the pool while copying it was to stop the BPC service temporarily.

I don't see why, unless you use v4 and it is know to corrupt in such
a situation.

> Sounds good, but would that lead to new problems for BPC?

As far as your backups have ended that must not be the case.

At home, my BPC server is an old machine that eats a lot of energy, so it
is off most of the time. I do a backup every 4 days, colliding happily
running backups and nightly, and, as written above, I never saw any
corruption occurring (I recently had to reinstall 2 machines completely
from a minimal installation then a BPC restore, so if there was any
problem I would have meet them.)

JY

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Backing up the BackupPC pool

[Hannes Elvemyr]

I'm taking about BPC itself. Why? Let's take an example.00:30 My nightly rsync script starts to sync pool to off-site storage. This night, there happens to be a lot of new data an it takes time.01:00 BackupPC_Nightly starts cleaning the pool! Rsync is still running!01:15 My nightly rsync script is done syncingThis night I would end up with a corrupt copy since BPC wrote to the pool while I was copying it. One suggestion to prevent BPC to touch the pool while copying it was to stop the BPC service temporarily. Sounds good, but would that lead to new problems for BPC?If BPC never runs between say 00.00 and 01.30, will this prevent certain tasks in the BPC_Nightly to never run for instance?And what if BPC happens to running a cleaning job when I stop it and start my rsync script? Will I then copy a corrupted pool?Many questions. This approach seems fragile to me. Am I right?/Hannes Original Message Subject: Re: [BackupPC-users] Backing up the BackupPC poolFrom: B To: backuppc-users@lists.sourceforge.netCC: On Fri, 11 Aug 2017 00:16:27 +0200Hannes Elvemyr  wrote:> Would it be wise to stop/start BackupPC every night? Can it introduce> any problem? What if my copy process (for instance rsync over Internet> to an off-site storage) takes 1 hour, maybe 2 hours some days, well> that interfere with BackupPC_Nightly?Do you speak about BPC itself or it's machine?If BPC, why, as it almost eat nothing while sleeping,if machine (energy spare, I guess), it isn't recommended as electronics and HDz really hate shocks, either electric(start draw more current as you've got to charge capacitors, thisproduce an overall peak) or thermal, especially HDz.Jean-Yves--Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! http://sdm.link/slashdot___BackupPC-users mailing listBackupPC-users@lists.sourceforge.netList:https://lists.sourceforge.net/lists/listinfo/backuppc-usersWiki:http://backuppc.wiki.sourceforge.netProject: http://backuppc.sourceforge.net/--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] V4, Set port for SSH

[Craig Barratt via BackupPC-users]

Pelle,

In 4.x you should use $Conf{RsyncSshArgs}.

Craig

On Thu, Aug 10, 2017 at 11:51 AM, Pelle Hanses  wrote:

> Hi,
> I have upgraded to V4 (4.1.2) and everything seems to work except from
> setting SSH port. How can I set the port SSH will connect to?
> In version 3 I used "$Conf{RsyncClientCmd} = '$sshPath -q -x -l backuppc
> -p 2022 $host sudo $rsyncPath $argList+';" but I can't find out how to set
> -p 2022 in the new settings.
>
>
>
> /Pelle
>
>
> 
>
> *Pelle Hanses*
> Sweden
> E-post: pe...@masendata.com
>
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
>
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Backing up the BackupPC pool

[Les Mikesell]

On Thu, Aug 10, 2017 at 5:16 PM, Hannes Elvemyr  wrote:
> Would it be wise to stop/start BackupPC every night? Can it introduce any
> problem? What if my copy process (for instance rsync over Internet to an
> off-site storage) takes 1 hour, maybe 2 hours some days, well that interfere
> with BackupPC_Nightly?

I thought v4 was supposed to be designed to be rsync-friendly so you
should get whatever had been processed at the time of your rsync run -
or at least not have problems with out-of-sync data.You might also
try doing 2 rsync runs per night where you expect one to move most of
the data so the 2nd one should go very fast and catch anything the 1st
one missed.

-- 
  Les Mikesell
 lesmikes...@gmail.com



>

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Backing up the BackupPC pool

[Bzzzz]

On Fri, 11 Aug 2017 00:16:27 +0200
Hannes Elvemyr  wrote:

> Would it be wise to stop/start BackupPC every night? Can it introduce
> any problem? What if my copy process (for instance rsync over Internet
> to an off-site storage) takes 1 hour, maybe 2 hours some days, well
> that interfere with BackupPC_Nightly?

Do you speak about BPC itself or it's machine?
If BPC, why, as it almost eat nothing while sleeping,
if machine (energy spare, I guess), it isn't recommended as 
electronics and HDz really hate shocks, either electric
(start draw more current as you've got to charge capacitors, this
produce an overall peak) or thermal, especially HDz.

Jean-Yves

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Backing up the BackupPC pool

[Hannes Elvemyr]

Hi!Thanks for great response!tar + copy will of course take a long time and be inefficient due to the amount of data. What about rsync? Is there any reason why the normal benefits of rsync over tar+copy would not apply for the BackupPC v4 pool?Is there anything, any options to the command, to think about when using rsync? (For instance, can I prevent BackupPC to write to the pool during my rsync job is running?)I will have a look at wireguard. Anyone got a clue of performance compared to openvpn?"I think he meant making a snapshot then convey it to he's 2nd site. " - yes, I did :)/Hannes Original Message Subject: Re: [BackupPC-users] Backing up the BackupPC poolFrom: Ray Frush To: "General list for user discussion, questions and support" CC: Hannes-Option 2, Copying the the pool (tar, then copy) to a remote server appears to be viable for small installations, and we've tested that with success.  But at a certain point, if you have a lot of machines, and your pool gets large, that process becomes un-manageable.  (Our pool is about 3.5TiB for just over 100 hosts backed up).  Just  tar-ing the data store up takes most of the day.So, Option 1 starts to look more appealing.   It may be that your primary instance (local) has a more aggressive retention schedule than the remote copy, or other differences like how frequently backups run.   --Ray FrushOn Wed, Aug 9, 2017 at 2:47 PM, Hannes Elvemyr  wrote:
  

  
  

  

Hi!
I'm using BackupPC for all my machines and it's great! I would
  now like to protect my BackupPC pool somehow (if my BackupPC
  server crashes, gets stolen or burns up I don't want to loose the
  data). As I see it, I have at least two options:
1. Run a second instance of BackupPC off-site
This of course creates a new second pool, but that could actually
  be an advantage if one of them somehow gets corrupted.
Pros: Two independent pools.
Cons: Complicated setup. I need a VPN between my network and the
  off-site machine to get this to work, which turned out to be more
  complicated than I first thought (my current router does not
  support static routing, so a new router would be the first step).
  I would also need to sync the BackupPC configuration from my main
  BackupPC instance to the second one (if I change some
  configuration on my main instance, I would also like the second
  instance to get that change).
2. Copying the pool and send it off-site
Pros: No need for a second BackupPC instance. Seems to be the
  easier solution if I can find out how to make a reliable copy of
  the pool.
Cons: How to copy the pool? The version 4 documentation says that “In V4, since
  hardlinks are not used permanently, duplicating a V4 pool is much
  easier, allowing remote copying of the pool.”. Sound great, but
  how do I know that BackupPC is not reading/writing to the pool
  during the copying process (maybe some backup is running or
  BackupPC_Nightly could start doing some cleaning). Copying a large
  pool over a bad Internet connection could take hours…
Any thought on this? How do you get redundancy of your BackupPC
  data?
Thanks!
-- 
/Hannes Elvemyr
  

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:    https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:    http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
-- Time flies like an arrow, but fruit flies like a banana.

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Backing up the BackupPC pool

[Hannes Elvemyr]

Would it be wise to stop/start BackupPC every night? Can it introduce any problem? What if my copy process (for instance rsync over Internet to an off-site storage) takes 1 hour, maybe 2 hours some days, well that interfere with BackupPC_Nightly? /Hannes Original Message Subject: Re: [BackupPC-users] Backing up the BackupPC poolFrom: Alexander Moisseev via BackupPC-users To: backuppc-users@lists.sourceforge.netCC: Alexander Moisseev On 8/9/2017 11:47 PM, Hannes Elvemyr wrote:> Sound great, but how do I know that BackupPC is not reading/writing to the pool during the copying process (maybe some backup is running or BackupPC_Nightly could start doing some cleaning). Copying a large pool over a bad Internet connection could take hours…> Option 1. Stop BackupPC.Option 2. Make a snapshot of the file system.--Check out the vibrant tech community on one of the world's mostengaging tech sites, Slashdot.org! http://sdm.link/slashdot___BackupPC-users mailing listBackupPC-users@lists.sourceforge.netList:https://lists.sourceforge.net/lists/listinfo/backuppc-usersWiki:http://backuppc.wiki.sourceforge.netProject: http://backuppc.sourceforge.net/--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] V4, Set port for SSH

[Pelle Hanses]

Hi,
I have upgraded to V4 (4.1.2) and everything seems to work except from 
setting SSH port. How can I set the port SSH will connect to?
In version 3 I used "$Conf{RsyncClientCmd} = '$sshPath -q -x -l backuppc 
-p 2022 $host sudo $rsyncPath $argList+';" but I can't find out how to 
set -p 2022 in the new settings.




/Pelle




*/Pelle Hanses/*
Sweden
E-post: pe...@masendata.com 

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/