Re: [BackupPC-users] Necro'd: Use S3 buckets for the pool

2020-07-21 Thread Kris Lou via BackupPC-users 
All,

Thanks for the suggestions and links.  There's a lot of interesting reading
to be done.  But as noted, checksum matching and storage latency are
probably prohibitive.

I hope to have access to a colo with gigabit bandwidth in the near future.
Maybe I'll spin up an instance just to see how it goes -- especially since
BPC4 should be less dependent upon bare-metal installations.

Thanks,
-Kris
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

Re: [BackupPC-users] Sign out

2020-07-21 Thread Craig Barratt via BackupPC-users 
I appreciate the feedback.  Please use the "List
" link in the
footer of any email to unsubscribe.

Mailing to the list mails everyone on the list.  It doesn't help you
unsubscribe.

Craig

On Tue, Jul 21, 2020 at 2:22 PM Ants Mark  wrote:

> ThNK YOU 4 providing me good info. But i don't wanna received any mor
> mails. Congrats for having such useful program
> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/
>
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

[BackupPC-users] Sign out

2020-07-21 Thread Ants Mark 
ThNK YOU 4 providing me good info. But i don't wanna received any mor
mails. Congrats for having such useful program
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Necro'd: Use S3 buckets for the pool

2020-07-21 Thread Marcelo Ricardo Leitner 
Hi,

Maybe a solution composed by backuppc + rclone is the way forward.
I never used rclone myself, but AFAICT it handles all the issues
you raised, plus more:

>From their page: "Virtual backends wrap local and cloud file systems
to apply encryption, caching, chunking and joining."

https://rclone.org/

Been willing to try it, but didn't have the chance so far.

Best regards,
Marcelo

On Tue, Jul 21, 2020 at 10:42:30AM +0300, Johan Ehnberg wrote:
> Hi Kris,
> 
> Indeed the object storage transformation has not been a hot topic for some
> time. I explored this some years ago and did some proof of concept testing.
> It requires a fairly different architecture:
> 
> https://molnix.com/proposal-new-open-source-backup-solution/
> 
> Essentially, the object storage to fuse approach is not workable as-is from
> a performance standpoint, at least in almost any production scenario I can
> imagine.
> 
> The compute node needs to independently do at least the checksum matching,
> storage buffering or tiering, and object storage rate limiting.
> 
> With the fuse approach you can get pretty close to that by putting only pool
> or cpool on fuse, and keeping pc locally, added with some configuration
> tweaks. However, the object storage needs to be asynchronous to the backup
> for it to really make sense.
> 
> I believe the efforts to create FS-to-object layers to transform existing
> software to cloud concepts without actual code changes quieted down exactly
> because of these types of issues.
> 
> Best regards,
> 
> Johan
> 
> 
> On 21/07/2020 02.37, Kris Lou via BackupPC-users wrote:
> > This hasn't been addressed for a while, and I didn't find anything in
> > recent archives.
> > 
> > Anybody have any experience or hypothetical issues with writing the BPC4
> > Pool over s3fs-fuse to S3 or something similar?  Pros, Cons?
> > 
> > Thanks,
> > -Kris
> > 
> > 
> > 
> > Kris Lou
> > k...@themusiclink.net 
> > 
> > 
> > ___
> > BackupPC-users mailing list
> > BackupPC-users@lists.sourceforge.net
> > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> > Wiki:http://backuppc.wiki.sourceforge.net
> > Project: http://backuppc.sourceforge.net/
> -- 
> Signature
> *Johan Ehnberg*
> 
> Founder, CEO
> 
> Molnix Oy
> 
> 
> jo...@molnix.com 
> 
> +358 50 320 96 88
> 
> molnix.com 
> 
> 
> /The contents of this e-mail and its attachments are for the use of the
> intended recipient only, and are confidential and may contain legally
> privileged information. If you are not the intended recipient or have
> otherwise received the e-mail in error, please notify the sender by replying
> to this e-mail immediately and then delete it immediately from your system.
> Any dissemination, distribution, copying or use of this communication
> without prior and explicit permission of the sender is strictly prohibited./
> 
> /*Please consider the environment - do not print this e-mail unless you
> really need to.*/
> 


> ___
> BackupPC-users mailing list
> BackupPC-users@lists.sourceforge.net
> List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
> Wiki:http://backuppc.wiki.sourceforge.net
> Project: http://backuppc.sourceforge.net/



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Windows 10 Subsystem For Linux

2020-07-21 Thread backuppc 
Stan Larson wrote at about 12:57:40 -0400 on Tuesday, July 21, 2020:
 > We've been successfully using BackupPC 3.3 and the Windows 10 WSL feature to 
 > access Windows 10 PCs using rsync (without the cgwin plugin).  We've been 
 > using this method on our production BackupPC server to back up about 30 
 > Win10 Pro clients.  We just back up the C:/Users folder, which picks up the 
 > User's Desktop, Documents, AppData folders, etc.  This method has proven to 
 > be very reliable using BackupPC 3.3.
 > 
 > We are testing with BackupPC 4.4 so that we can overcome the filesystem 
 > issues that BackupPC 3.3  hard links present.
 > 
 > We are running into a couple of problems that seem to be related to 
 > rsync_bpc.
 > 
 > 1.  On our BackupPC 3.3 server, we are able to use an alternate ssh port for 
 > our Windows 10 clients.  We actually run ssh on port  on the clients 
 > with no problem.  With BackupPC 4.4 (rsync_bpc), we get errors when trying 
 > to run on alternate ports.  The errors seem to indicate that even though we 
 > are specifying a different port, rsync_bpc is ignoring the alternate port 
 > and trying to use port 22.  Here's the config declaration, which works on 
 > 3.3 but not 4.4... $Conf{RsyncClientCmd} = '$sshPath -p  -q -x -l root 
 > $host $rsyncPath $argList+';

RsyncClientCmd is not a configurable variable for 4.x so not
surprising that you are having a problem...

You probably want to use: RsyncSshArgs.
For example:
$Conf{RsyncSshArgs} = ['-e', '$sshPath -p  -q -x -l root'];
Though not sure you need '-q -x'

 > 
 > 2.  When we use port 22 on the client instead of port  (see above), we 
 > get a successful backup, but we have a different problem.  On the Win 10 WSL 
 > client, the C:\ drive is a separate filesystem presented as /mnt/c.  On our 
 > BackupPC 3.3 server, we are able to cross this mount point successfully with 
 > no special configurations, using the config declaration...   
 > "$Conf{BackupFilesOnly} = ['/mnt/c/'];".  On our BackupPC 4.4 server, the 
 > backup will run successfully, but no files below /mnt/c are included.  It's 
 > as if BackupPC is refusing to cross from the / filesystem to the /mnt/c 
 > filesystem.
 > 

Suggest you test manually by running from the command line:

  sudo -u backuppc rsync -navxH -p  -l root :/mnt/c

 > For the new server, we are using CentOS 8 and the default BackupPC yum 
 > packages.
 > 
 > Any thoughts on either problem would be much appreciated.
 > 
 > --
 > [Freedom] 
 > Stan Larson  |  IT Manager
 > Freedom  |  www.freedomsales.com
 > 11225 Challenger Avenue Odessa, FL 33556
 > PH: 813-855-2671 x206  |
 > Direct Line: 1-727-835-1157
 > 
 > 
 > All commodities purchased from Freedom Sales are to be handled in accordance 
 > with US law including but not limited to the Export Administration 
 > Regulations, International Traffic in Arms Regulations, US Department of 
 > State, US Department of Homeland Security, US Department of Commerce, and US 
 > Office of Foreign Assets Control. Diversion Contrary to US law is prohibited.
 > ___
 > BackupPC-users mailing list
 > BackupPC-users@lists.sourceforge.net
 > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
 > Wiki:http://backuppc.wiki.sourceforge.net
 > Project: http://backuppc.sourceforge.net/


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] Windows 10 Subsystem For Linux

2020-07-21 Thread Stan Larson 

We've been successfully using BackupPC 3.3 and the Windows 10 WSL feature to 
access Windows 10 PCs using rsync (without the cgwin plugin).  We've been using 
this method on our production BackupPC server to back up about 30 Win10 Pro 
clients.  We just back up the C:/Users folder, which picks up the User's 
Desktop, Documents, AppData folders, etc.  This method has proven to be very 
reliable using BackupPC 3.3.

We are testing with BackupPC 4.4 so that we can overcome the filesystem issues 
that BackupPC 3.3  hard links present.

We are running into a couple of problems that seem to be related to rsync_bpc.

1.  On our BackupPC 3.3 server, we are able to use an alternate ssh port for 
our Windows 10 clients.  We actually run ssh on port  on the clients with 
no problem.  With BackupPC 4.4 (rsync_bpc), we get errors when trying to run on 
alternate ports.  The errors seem to indicate that even though we are 
specifying a different port, rsync_bpc is ignoring the alternate port and 
trying to use port 22.  Here's the config declaration, which works on 3.3 but 
not 4.4... $Conf{RsyncClientCmd} = '$sshPath -p  -q -x -l root $host 
$rsyncPath $argList+';

2.  When we use port 22 on the client instead of port  (see above), we get a 
successful backup, but we have a different problem.  On the Win 10 WSL client, the C:\ 
drive is a separate filesystem presented as /mnt/c.  On our BackupPC 3.3 server, we are 
able to cross this mount point successfully with no special configurations, using the 
config declaration...   "$Conf{BackupFilesOnly} = ['/mnt/c/'];".  On our 
BackupPC 4.4 server, the backup will run successfully, but no files below /mnt/c are 
included.  It's as if BackupPC is refusing to cross from the / filesystem to the /mnt/c 
filesystem.

For the new server, we are using CentOS 8 and the default BackupPC yum packages.

Any thoughts on either problem would be much appreciated.

--
[Freedom] 
Stan Larson  |  IT Manager
Freedom  |  www.freedomsales.com
11225 Challenger Avenue Odessa, FL 33556
PH: 813-855-2671 x206  |
Direct Line: 1-727-835-1157


All commodities purchased from Freedom Sales are to be handled in accordance 
with US law including but not limited to the Export Administration Regulations, 
International Traffic in Arms Regulations, US Department of State, US 
Department of Homeland Security, US Department of Commerce, and US Office of 
Foreign Assets Control. Diversion Contrary to US law is prohibited.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] Necro'd: Use S3 buckets for the pool

2020-07-21 Thread Johan Ehnberg 

Hi Kris,

Indeed the object storage transformation has not been a hot topic for 
some time. I explored this some years ago and did some proof of concept 
testing. It requires a fairly different architecture:


https://molnix.com/proposal-new-open-source-backup-solution/

Essentially, the object storage to fuse approach is not workable as-is 
from a performance standpoint, at least in almost any production 
scenario I can imagine.


The compute node needs to independently do at least the checksum 
matching, storage buffering or tiering, and object storage rate limiting.


With the fuse approach you can get pretty close to that by putting only 
pool or cpool on fuse, and keeping pc locally, added with some 
configuration tweaks. However, the object storage needs to be 
asynchronous to the backup for it to really make sense.


I believe the efforts to create FS-to-object layers to transform 
existing software to cloud concepts without actual code changes quieted 
down exactly because of these types of issues.


Best regards,

Johan


On 21/07/2020 02.37, Kris Lou via BackupPC-users wrote:
This hasn't been addressed for a while, and I didn't find anything in 
recent archives.


Anybody have any experience or hypothetical issues with writing the 
BPC4 Pool over s3fs-fuse to S3 or something similar?  Pros, Cons?


Thanks,
-Kris



Kris Lou
k...@themusiclink.net 


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

--
Signature
*Johan Ehnberg*

Founder, CEO

Molnix Oy


jo...@molnix.com 

+358 50 320 96 88

molnix.com 


/The contents of this e-mail and its attachments are for the use of the 
intended recipient only, and are confidential and may contain legally 
privileged information. If you are not the intended recipient or have 
otherwise received the e-mail in error, please notify the sender by 
replying to this e-mail immediately and then delete it immediately from 
your system. Any dissemination, distribution, copying or use of this 
communication without prior and explicit permission of the sender is 
strictly prohibited./


/*Please consider the environment - do not print this e-mail unless you 
really need to.*/


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/