Re: [BackupPC-users] errors on network-mounted pool filesystem

2022-09-02 Thread Les Mikesell
On Fri, Sep 2, 2022 at 1:34 PM  wrote:
>
> Sshfs may have other limitations as it is not a full-fledged
> filesystem...
>

And conceptually, think about what has to happen when using rysnc with
a remote filesystem.  Computing the block checksums that is supposed
to be a quick local disk operation now has to run the whole contents
of each file over the network.

--
   Les Mikesell
 lesmikes...@gmail.com


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/


Re: [BackupPC-users] errors on network-mounted pool filesystem

2022-09-02 Thread backuppc
Are you sure that BackupPC is compatible with sshfs?
While v4 drastically reduces number of hard links by eliminating the
hard-link pool scheme, it still uses hard links for *temporary* atomic
renames.
See: http://backuppc.sourceforge.net/BackupPC-4.2.1.html

Sshfs may have other limitations as it is not a full-fledged
filesystem...


Sandro Bordacchini wrote at about 15:01:35 +0200 on Friday, September 2, 2022:
 > Hello all.
 > 
 > Just wanted to share with you what happened to me and check if you have any
 > idea on what is going on.
 > I have been a satisfied user of backuppc for a long time. So far I am using
 > a 3.3.0 version on a debian 7 on a local vm (a local vmware esxi i have
 > full control on): no problem with this approach.
 > 
 > I needed to switch to a cloud solution, though.
 > 
 > No need to migrate the pool. I thought about installing backuppc v4 and
 > just starting from that, adding my old hosts (to avoid v3/v4 pool version
 > issues)
 > My pool is 0.7 TB so to keep the budget low I planned to buy a low-middle
 > sized vm plus some "cloud storage space".
 > I read that hardlinks support is no longer strictly mandatory (but it can
 > affect performance) on v4 so I was about to give it a try. I hope I
 > understood this crucial point correctly.
 > 
 > So i installed backuppc 4.4.0-3 (standard repo version on debian 11)
 > I mounted the cloud space with sshfs as user backuppc, this are the options
 > in the fstab line:
 > 
 > idmap=user,allow_other,default_permissions,IdentityFile=/root/.ssh/,port=23,ServerAliveInterval=15,reconnect,_netdev,uid=108,gid=114
 > 
 > uid and gid are backuppc user and group
 > I got a working installation and I tried a successful backup (full and
 > incr) of localhost (tar method), with no errors.
 > 
 > Then I tried to add some remote linux servers (rsync over ssh method) and I
 > got several transfer errors... very similar across all the servers... for
 > example a sample from xferLOG:
 > 
 > [...]
 > This is the rsync child about to exec /usr/libexec/backuppc-rsync/rsync_bpc
 > Xfer PIDs are now 39411,39414
 > xferPids 39411,39414
 > G bpc_path_create: can't create f%2fetc%2f/attrib (errno 2)
 > G bpc_path_create: can't create f%2fetc%2f/attrib (errno 2)
 > => [lots of repeated lines]
 > rsync_bpc: recv_generator: mkdir "/NetworkManager" failed: No such file or
 > directory (2)
 > *** Skipping any contents from this failed directory ***
 > G bpc_path_create: can't create f%2fetc%2f/fX11 (errno 2)
 > G bpc_path_create: can't create f%2fetc%2f/fX11 (errno 2)
 > rsync_bpc: recv_generator: mkdir "/X11/xkb" failed: No such file or
 > directory (2)
 > *** Skipping any contents from this failed directory ***
 > G bpc_path_create: can't create f%2fetc%2f/facpi (errno 2)
 > G bpc_path_create: can't create f%2fetc%2f/facpi (errno 2)
 > rsync_bpc: recv_generator: mkdir "/acpi/events" failed: No such file or
 > directory (2)
 > *** Skipping any contents from this failed directory ***
 > G bpc_fileReadAll: can't open
 > /var/lib/backuppc/cpool/36/16/37170a6409f1fa9cf67d20660b1d0f2d (from
 > alternatives/ALTER_OPERATOR_CLASS.7.gz)
 > rsync_bpc: symlink "/alternatives/ALTER_OPERATOR_CLASS.7.gz" ->
 > "/usr/share/postgresql/13/man/man7/ALTER_OPERATOR_CLASS.7.gz" failed: File
 > exists (17)
 > rsync_bpc: symlink "/alternatives/ALTER_OPERATOR_FAMILY.7.gz" ->
 > "/usr/share/postgresql/13/man/man7/ALTER_OPERATOR_FAMILY.7.gz" failed: File
 > exists (17)
 > G bpc_fileReadAll: can't open
 > /var/lib/backuppc/cpool/e2/48/e34810c68ce29a133fbc30d73c6ad6a0 (from
 > alternatives/ALTER_TABLESPACE.7.gz)
 > [...]
 > 
 > All the remote servers i added so far (all debian servers) got several
 > errors and if i try to browse their backups, the web interface tells me
 > they are empty.
 > 
 > My main concern is: why the backup with tar method is working and the other
 > with rsync is not?
 > If it was something about the pool filesystem ... should not affect all the
 > backups?
 > Is something about the options i mount the sshfs filesystem? Should I set
 > write syncs?
 > Seeing the log someone could argue user backuppc has no rw access but i can
 > tell you that backuppc created a lot of files and folders in that fs.
 > 
 > Thank you in advance.
 > S
 > ___
 > BackupPC-users mailing list
 > BackupPC-users@lists.sourceforge.net
 > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
 > Wiki:https://github.com/backuppc/backuppc/wiki
 > Project: https://backuppc.github.io/backuppc/


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/


Re: [BackupPC-users] errors on network-mounted pool filesystem

2022-09-02 Thread Greg Harris
I’d suspect that the rsync_bpc call is using a user that isn’t what you expect. 
 However, I can’t find the exact setting I was expecting.  In my setup, the 
group that owns the backup location is the web group, (I think www for Ubuntu), 
not backuppc, and backuppc is part of the web group.

Thanks,

Greg Harris

On Sep 2, 2022, at 9:01 AM, Sandro Bordacchini 
mailto:sandro.bordacch...@gmail.com>> wrote:

Hello all.

Just wanted to share with you what happened to me and check if you have any 
idea on what is going on.
I have been a satisfied user of backuppc for a long time. So far I am using a 
3.3.0 version on a debian 7 on a local vm (a local vmware esxi i have full 
control on): no problem with this approach.

I needed to switch to a cloud solution, though.

No need to migrate the pool. I thought about installing backuppc v4 and just 
starting from that, adding my old hosts (to avoid v3/v4 pool version issues)
My pool is 0.7 TB so to keep the budget low I planned to buy a low-middle sized 
vm plus some "cloud storage space".
I read that hardlinks support is no longer strictly mandatory (but it can 
affect performance) on v4 so I was about to give it a try. I hope I understood 
this crucial point correctly.

So i installed backuppc 4.4.0-3 (standard repo version on debian 11)
I mounted the cloud space with sshfs as user backuppc, this are the options in 
the fstab line:

idmap=user,allow_other,default_permissions,IdentityFile=/root/.ssh/,port=23,ServerAliveInterval=15,reconnect,_netdev,uid=108,gid=114

uid and gid are backuppc user and group
I got a working installation and I tried a successful backup (full and incr) of 
localhost (tar method), with no errors.

Then I tried to add some remote linux servers (rsync over ssh method) and I got 
several transfer errors... very similar across all the servers... for example a 
sample from xferLOG:

[...]
This is the rsync child about to exec /usr/libexec/backuppc-rsync/rsync_bpc
Xfer PIDs are now 39411,39414
xferPids 39411,39414
G bpc_path_create: can't create f%2fetc%2f/attrib (errno 2)
G bpc_path_create: can't create f%2fetc%2f/attrib (errno 2)
=> [lots of repeated lines]
rsync_bpc: recv_generator: mkdir "/NetworkManager" failed: No such file or 
directory (2)
*** Skipping any contents from this failed directory ***
G bpc_path_create: can't create f%2fetc%2f/fX11 (errno 2)
G bpc_path_create: can't create f%2fetc%2f/fX11 (errno 2)
rsync_bpc: recv_generator: mkdir "/X11/xkb" failed: No such file or directory 
(2)
*** Skipping any contents from this failed directory ***
G bpc_path_create: can't create f%2fetc%2f/facpi (errno 2)
G bpc_path_create: can't create f%2fetc%2f/facpi (errno 2)
rsync_bpc: recv_generator: mkdir "/acpi/events" failed: No such file or 
directory (2)
*** Skipping any contents from this failed directory ***
G bpc_fileReadAll: can't open 
/var/lib/backuppc/cpool/36/16/37170a6409f1fa9cf67d20660b1d0f2d (from 
alternatives/ALTER_OPERATOR_CLASS.7.gz)
rsync_bpc: symlink "/alternatives/ALTER_OPERATOR_CLASS.7.gz" -> 
"/usr/share/postgresql/13/man/man7/ALTER_OPERATOR_CLASS.7.gz" failed: File 
exists (17)
rsync_bpc: symlink "/alternatives/ALTER_OPERATOR_FAMILY.7.gz" -> 
"/usr/share/postgresql/13/man/man7/ALTER_OPERATOR_FAMILY.7.gz" failed: File 
exists (17)
G bpc_fileReadAll: can't open 
/var/lib/backuppc/cpool/e2/48/e34810c68ce29a133fbc30d73c6ad6a0 (from 
alternatives/ALTER_TABLESPACE.7.gz)
[...]

All the remote servers i added so far (all debian servers) got several errors 
and if i try to browse their backups, the web interface tells me they are empty.

My main concern is: why the backup with tar method is working and the other 
with rsync is not?
If it was something about the pool filesystem ... should not affect all the 
backups?
Is something about the options i mount the sshfs filesystem? Should I set write 
syncs?
Seeing the log someone could argue user backuppc has no rw access but i can 
tell you that backuppc created a lot of files and folders in that fs.

Thank you in advance.
S

___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/

___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/


[BackupPC-users] errors on network-mounted pool filesystem

2022-09-02 Thread Sandro Bordacchini
Hello all.

Just wanted to share with you what happened to me and check if you have any
idea on what is going on.
I have been a satisfied user of backuppc for a long time. So far I am using
a 3.3.0 version on a debian 7 on a local vm (a local vmware esxi i have
full control on): no problem with this approach.

I needed to switch to a cloud solution, though.

No need to migrate the pool. I thought about installing backuppc v4 and
just starting from that, adding my old hosts (to avoid v3/v4 pool version
issues)
My pool is 0.7 TB so to keep the budget low I planned to buy a low-middle
sized vm plus some "cloud storage space".
I read that hardlinks support is no longer strictly mandatory (but it can
affect performance) on v4 so I was about to give it a try. I hope I
understood this crucial point correctly.

So i installed backuppc 4.4.0-3 (standard repo version on debian 11)
I mounted the cloud space with sshfs as user backuppc, this are the options
in the fstab line:

idmap=user,allow_other,default_permissions,IdentityFile=/root/.ssh/,port=23,ServerAliveInterval=15,reconnect,_netdev,uid=108,gid=114

uid and gid are backuppc user and group
I got a working installation and I tried a successful backup (full and
incr) of localhost (tar method), with no errors.

Then I tried to add some remote linux servers (rsync over ssh method) and I
got several transfer errors... very similar across all the servers... for
example a sample from xferLOG:

[...]
This is the rsync child about to exec /usr/libexec/backuppc-rsync/rsync_bpc
Xfer PIDs are now 39411,39414
xferPids 39411,39414
G bpc_path_create: can't create f%2fetc%2f/attrib (errno 2)
G bpc_path_create: can't create f%2fetc%2f/attrib (errno 2)
=> [lots of repeated lines]
rsync_bpc: recv_generator: mkdir "/NetworkManager" failed: No such file or
directory (2)
*** Skipping any contents from this failed directory ***
G bpc_path_create: can't create f%2fetc%2f/fX11 (errno 2)
G bpc_path_create: can't create f%2fetc%2f/fX11 (errno 2)
rsync_bpc: recv_generator: mkdir "/X11/xkb" failed: No such file or
directory (2)
*** Skipping any contents from this failed directory ***
G bpc_path_create: can't create f%2fetc%2f/facpi (errno 2)
G bpc_path_create: can't create f%2fetc%2f/facpi (errno 2)
rsync_bpc: recv_generator: mkdir "/acpi/events" failed: No such file or
directory (2)
*** Skipping any contents from this failed directory ***
G bpc_fileReadAll: can't open
/var/lib/backuppc/cpool/36/16/37170a6409f1fa9cf67d20660b1d0f2d (from
alternatives/ALTER_OPERATOR_CLASS.7.gz)
rsync_bpc: symlink "/alternatives/ALTER_OPERATOR_CLASS.7.gz" ->
"/usr/share/postgresql/13/man/man7/ALTER_OPERATOR_CLASS.7.gz" failed: File
exists (17)
rsync_bpc: symlink "/alternatives/ALTER_OPERATOR_FAMILY.7.gz" ->
"/usr/share/postgresql/13/man/man7/ALTER_OPERATOR_FAMILY.7.gz" failed: File
exists (17)
G bpc_fileReadAll: can't open
/var/lib/backuppc/cpool/e2/48/e34810c68ce29a133fbc30d73c6ad6a0 (from
alternatives/ALTER_TABLESPACE.7.gz)
[...]

All the remote servers i added so far (all debian servers) got several
errors and if i try to browse their backups, the web interface tells me
they are empty.

My main concern is: why the backup with tar method is working and the other
with rsync is not?
If it was something about the pool filesystem ... should not affect all the
backups?
Is something about the options i mount the sshfs filesystem? Should I set
write syncs?
Seeing the log someone could argue user backuppc has no rw access but i can
tell you that backuppc created a lot of files and folders in that fs.

Thank you in advance.
S
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/