Re: [BackupPC-users] Backuppc Errors

2020-08-04 Thread Michael Stowe

On 2020-08-04 03:55, s.chimere--- via BackupPC-users wrote:





Hello, Good day 

Hope everyone is doing okay amid the pandemic, 

I have been using backuppc since last year and it has been working okay, but recently pings from the backuppc started failing 

I fixed that using this https://u2182357.ct.sendgrid.net/ls/click?upn=UlfI6r-2FmuicX-2BnC5-2BZ3I6t4hto13icElebmTufqZ1T1YnGHZ-2FjD3nLfmUtHCO97XgoUxTh8s-2FfxzfziVTV04vLrzkdnfHsSUREcSKK59i-2BGkE2k246AzNnwCwWIvb8hWpt7J_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk00ZHxb76gvYWlcaRBYHBhk8Mz2U5ddB1JP1P0Z4ZOguEfTHjysl-2F3g-2F-2FJVHTQQE8-2F5Jrdos3X0c4-2F5a9Jgsf9uk83-2BiHAbhbZxGXGWnN49JBzFg5k9NoJ4nNQh3GXXpxzflVnPzTE8v9tx4BH-2BeWtEz-2BSQISZlm2cn7BIagtJz04SCQ10-2FvwO3GfCStyiKKHZV [1] 

But now I get this error instead.  

Connection to x.x.x.x failed (Error NT_STATUS_IO_TIMEOUT) 

Connection to x.x.x.x failed (Error NT_STATUS_IO_TIMEOUT) 

tarExtract: Done: 0 errors, 0 filesExist, 0 sizeExist, 0 sizeExistComp, 0 filesTotal, 0 sizeTotal 

Got fatal error during xfer (No files dumped for share Virtual hard disks) 

Backup aborted (No files dumped for share Virtual hard disks) 

Not saving this as a partial backup since it has fewer files than the prior one (got 0 and 0 files versus 14) 

Please anyone have idea how I can quickly resolve this ? 

Thanks 

Symbol Chimere 


Team Lead Infrastructure


Replacing the ping with an echo is pretty terrible advice.  While
Windows blocks pings by default, it's easy enough to turn on, and
there's rarely a good reason not to. 


Of course, the most likely explanation is that the system you're trying
to back up is _unreachable_, which is impossible to give a blanket
solution for, but fix that first.  Chances are, the IP changed, or the
same thing that made the system no longer responding to pings is making
it no longer respond to CIFS traffic. 


Links:
--
[1]
https://u2182357.ct.sendgrid.net/ls/click?upn=UlfI6r-2FmuicX-2BnC5-2BZ3I6t4hto13icElebmTufqZ1T1YnGHZ-2FjD3nLfmUtHCO97XgoUxTh8s-2FfxzfziVTV04vLrzkdnfHsSUREcSKK59i-2BGkE2k246AzNnwCwWIvb8hW9QYB_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk00ZHxb76gvYWlcaRBYHBhk8FGWGdJ3lrtCD5wscVs9lowlub8Ag1UcY64kBw1zO2ZES1DiWsGgiwm8FEm8Z2tE2CL4PAmCcL92cP5DGKtFLT-2Bx3UHOdfA-2FcALw8aaPXW892MdLmneKCHx2OOLDqb7-2BF0eAxdCHzCkJPnBC-2Fkz4dnLTkqEBKf5njawEMHkjJ6uM___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:https://github.com/backuppc/backuppc/wiki
Project: https://backuppc.github.io/backuppc/


Re: [BackupPC-users] config.pl "built-in" script to backup Windows shares with Virtual Shadow Service Copy

2020-07-01 Thread Michael Stowe

On 2020-06-30 19:35, backu...@kosowsky.org wrote:

Michael Stowe wrote at about 23:09:55 + on Tuesday, June 30, 2020:
 > On 2020-06-29 21:51, backu...@kosowsky.org wrote:
 > > Over the years, many have asked and struggled with backing up 
remote
 > > Windows shares with shadow copies. Shadow copies are useful since 
they
 > > allow both the backup to be 'consistent' and allow for reading 
files
 > > that are otherwise 'busy' and unreadable when part of active 
Windows

 > > partitions.
 > >
 > > Various solutions (including one I proposed almost a decade ago) 
use

 > > additional scripts and hacks to create the shadow copy.
 > > Such solutions are kludgy and require the maintenance of separate
 > > scripts either on the server or client.
 > >
 > > I have written a combination of perl and bash code that can be 
stored

 > > in the host.pl configuration file that does everything you need to
 > > automagically create shadow copies for each share (where possible)
 > > with minimal to no special configuration in host.pl and nothing to
 > > configure on the Windows client (other than having 
cygwin+ssh+rsync

 > > and an accessible account on your Windows client).
 > >
 > > The only thing you need to do is to set up the hash
 > > Conf{ClientShareName2Path} to map share names to their
 > > (unshadowed) Windows paths. The attached script will then set up 
and

 > > interpolate the appropriate shadow paths.
 > >
 > > It should just work...
 > > Just cut-and-paste the attachment into your host.pl code for 
Windows

 > > clients.
 > >
 > > Note: I included a fair amount of debugging & error messages in 
case

 > > any shadows or links fail to get created or unwound.
 >
 > What ssh do you use?

I just use stock cygwin ssh & rsync.


 > When I updated my server-side scripts to work with ssh as well as 
the

 > venerable winexe, and was alerted to this:
 >
https://u2182357.ct.sendgrid.net/ls/click?upn=UlfI6r-2FmuicX-2BnC5-2BZ3I6n1L3gGqkO4CpkrIpm-2FMXVdh7clCIBIMiVaW-2Fu4zJf1mu24301LW0nUuAE3Gedwg9vUvdiSjYTlleQn605qsb0FRbW6N7V7wfRFUjfSABVFV0RoZJ-2FyEfjb4YYEr-2F8znrwZSnSJzHNGDagX1i-2B8oxVVwIZ-2B-2BI-2B7fMNzIKJC5kqSEW-2F9D0DtlKHitvY1YisbM5pu2luUF-2B-2BItSFT8NuOX9Vq2dCJdKdjvFvOyTyyHxp1wW32cXKbNHSuWQQmnAIy9oD35Wau2-2B7WszpGgfR0cY-2B0pevu2W-2B-2B6fJJEznZrhlRitkK4PIwEk6rpHv5pNEE3zE6plvEiSsBUAzjqpeaPD-2BxvaMsO6aQJ9CXFTtDn8dUS5m6iatIhO0OYIAr3rlIP8HOuM8UxphdnUS-2Bx3cno2jam0S-2B-2B5lEewh0xiHIFUJ6ol867CfD3znalR13b0QgQk4L2VHC3hqf6DGABwIsLeze9isvhCs7r1nk59mpMhuTky2t-2BRoH9Mdm1KhWkz0czqEeXpfbtZjuSwPZeYgTcqMM-3D_4d7_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk00pkNGxGGLZoFxeZBePnNNfQwoEtLwJoSFE-2FLY1sVDBZbSrzu7m0AdClCUvxzYM7KNHA9ZIEYhwveViz3D0A1qxuZ0VdWwC2Z2fJVSIP9x42x5tUhxpcmHt-2FjoSKP46YtZux-2F159UIJgD7q6ONjcVYeYEQdVYxxGrBNusnkQIrUHReWk9DtR-2B30KsApgurmA19

I don't use winexe - it's also not particularly secure.


No, nor do I.  But I do use the ssh that comes with Windows, which 
required some special handling to get the permissions right.  I don't 
know if your script will work with it, but I can give it a whirl within 
the week.



I pipe a server-side bash script onto ssh as part of the Dump Pre/Post
User Commands to setup and takedown the shadows.

Not sure why you would want to use a custom version of rsync when my
pretty simple scripts do all that with a lot more transparency to how
they are setup.


I think because it's just one binary (vsshadow not needed, nor anything 
else)



Can you test out my script?


I'll give it a shot (see above)


I believe it's far simpler and cleaner than either:
- My old approach for WinXP (using a client-side triggered script,
  rsyncd setup, dosdev, 'at' recursion to elevate privileges, etc.)
- Your version requiring win.exe
- Other versions requiring a custom/non-standard rsync


N.B.: my version works using ssh now, it doesn't require winexe


My version only requires a basic cygwin install with rsync/ssh and
basic linux utils plus built-in windows functions.

BTW, I still need to add back in the ability to dump all the acl's
(using subinacl) since rsync only syncs POSIX acls and I believe ntfs
has additional acl's.

In any case, my ultimate holy-grail is to be able to use BackupPC to 
allow for

a full bare-metal restore by combining:
- Full VSS file backup
- Restore of all ACLs from a subinacl dump
- Anything else I may need to recreate the full NTFS filesystem for
  windows (maybe disk signatures???)


I fully support this notion; NTFS has a lot of weirdness that doesn't 
translate well to rsync, like junction points.  Last time I tried these, 
rsync would convert them to symlinks, and restore them as symlinks.  
YMMV



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] config.pl "built-in" script to backup Windows shares with Virtual Shadow Service Copy

2020-06-30 Thread Michael Stowe

On 2020-06-29 21:51, backu...@kosowsky.org wrote:

Over the years, many have asked and struggled with backing up remote
Windows shares with shadow copies. Shadow copies are useful since they
allow both the backup to be 'consistent' and allow for reading files
that are otherwise 'busy' and unreadable when part of active Windows
partitions.

Various solutions (including one I proposed almost a decade ago) use
additional scripts and hacks to create the shadow copy.
Such solutions are kludgy and require the maintenance of separate
scripts either on the server or client.

I have written a combination of perl and bash code that can be stored
in the host.pl configuration file that does everything you need to
automagically create shadow copies for each share (where possible)
with minimal to no special configuration in host.pl and nothing to
configure on the Windows client (other than having cygwin+ssh+rsync
and an accessible account on your Windows client).

The only thing you need to do is to set up the hash
Conf{ClientShareName2Path} to map share names to their
(unshadowed) Windows paths. The attached script will then set up and
interpolate the appropriate shadow paths.

It should just work...
Just cut-and-paste the attachment into your host.pl code for Windows
clients.

Note: I included a fair amount of debugging & error messages in case
any shadows or links fail to get created or unwound.


What ssh do you use?

When I updated my server-side scripts to work with ssh as well as the 
venerable winexe, and was alerted to this:

https://u2182357.ct.sendgrid.net/ls/click?upn=UlfI6r-2FmuicX-2BnC5-2BZ3I6hSUAGKA-2FZ4EXu0KbZUYtfPMmjaEDrGQFZ-2BTO1Kw4YUsENnB-2BYHtkE8jsm5y9ZKsZw-3D-3DOkUp_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk00j-2B342fKCvTk2gVMjxlLeGQjGpYP9XfaFhqsh78hDEKLdOUOH4lHptqZI7uoYJ7-2BtzZLqYUcVXdPlHNtfoD3QBSgt3UEs1MGWPQzmfb3fOH7HbGctnY54CDgrSwj0qmmrglWL3EjG0PDiNOHaMCd6Ca1oincEGz6gUk00qqpETnyNz9sH0ZleDvOZCq6-2BLfbW

Which points to this:
https://u2182357.ct.sendgrid.net/ls/click?upn=UlfI6r-2FmuicX-2BnC5-2BZ3I6ixXxMXwHczMOYAFDcPlTkEXRK7t-2BnyUlyzDQoG5GZdN-2B0DCDb-2Fa6IOvkPW8bMfXFEiMGZijC6vabaR5CPfjOJA-3DZinR_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk00j-2B342fKCvTk2gVMjxlLeGZHu7cPKYWx5IlIr5yRpXtHWrn5iGQubBaMv62yVZDPCU8oYXWCq30DGKiQ6yzxPxdV1ygz7XdIc-2FN9v6lUIeB9MDuiLXLcnutCzb-2FQPd-2Fcg-2FefTLzrbRTi1OxMVO1MSFVoLv4y1kwBN0s-2FUA7JfoepEk2hS1-2BVqr4yp7NwB4uvW

Fundamentally, it's a customized copy of rsync that automatically 
handles the VSS side.  At any rate, I haven't finished testing it (have 
a large project backlog) but thought you (or others here) might be 
interested.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BUG? Using --omit-dir-times in rsync backup sets all dir dates to beginning of Epoch

2020-05-24 Thread Michael Stowe

On 2020-05-23 23:47, backu...@kosowsky.org wrote:

Regarding your comment:

 However, I don't think it makes sense to
"fix" it, since a backup shouldn't add metadata that changes each time 
you

backup some data that hasn't changed.


Actually, the whole point of --ignore-dir-times is that the dir mod
time is not changed each time you rsync.

Behavior is as follows:
1. When created, the mod time is set to 'rsync' run time
2. This mod time is *not* changed on subsequent 'rsyncs' unless a
   change to the directory contents occurs in which case the
   directory timestamp is updated to the *current* time

Indeed, one reason to use --ignore-dir-times is to avoid changing the
mod time every time the directory mod time changes.

This is different from the absence of --times that changes the mod time 
of

files/links/devices to the current time on every rsync since the files
show up as changed (unless you also use --ignore-times)

This behavior can be verified by using rsync.
Also, it can be gleaned from the following pulled from the man pages:


--ignore-dir-times and --omit-dir-times are two different options


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Testing full restore of backuppc... MULTIPLE BUGS???

2020-05-22 Thread Michael Stowe

On 2020-05-22 16:49, backu...@kosowsky.org wrote:

Michael Stowe wrote at about 22:18:50 + on Friday, May 22, 2020:
 > On 2020-05-22 11:42, backu...@kosowsky.org wrote:
 > > 1. Sockets are restored as regular files not special files --> 
BUG?

 >
 > Why would one back up a socket?
I am testing the fidelity of the backup/restore cycle..


 > If you really think this is sensible, you should be able to 
accomplish

 > it with "--devices --specials" as part of your rsync command lines.
 >  From the symptoms, you have this in backup but not restore.

Actually, in the original text (which you snipped), I shared my
rsync_bpc commands for both 'dump' and 'restore', which include the
'-D' flag (actually it's the default in the config.pl for both rsync
dump and restore)... and '-D' is *equivalent* to '--devices
--specials'

And since I suspected some readers might miss that, I even noted in
the text that:
   "Also, special files (--specials) should be included under the -D
   flag that I use for both rsync dump and restore commands (see
   below)"

Hence, why I suggested this is a *BUG* vs. user error or lack of
knowledge :)


You've mistaken my point -- sure, the -D flag is there, but it's 
behaving like it isn't.  Let's review:


--devices
   This option causes rsync to transfer character and block  device
   files  to  the  remote  system  to recreate these devices.  This
   option has no effect if the receiving rsync is not  run  as  the
   super-user (see also the --super and --fake-super options).

Naturally this begs the question as to whether you're running it as the 
super-user, and if you've seen the options as referred to in the man 
page, which I've quoted above.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BUG? Using --omit-dir-times in rsync backup sets all dir dates to beginning of Epoch

2020-05-22 Thread Michael Stowe

On 2020-05-22 16:52, backu...@kosowsky.org wrote:

Michael Stowe wrote at about 23:46:54 + on Friday, May 22, 2020:
 > On 2020-05-22 16:19, backu...@kosowsky.org wrote:
 > > Michael Stowe wrote at about 22:24:13 + on Friday, May 22, 
2020:

 > >  > On 2020-05-22 09:15, backu...@kosowsky.org wrote:
 > >  > What it does is omit directories from the modification times 
that it
 > >  > sets.  In other words, you're telling it not to set the times 
on

 > >  > directories it copies.  The beginning of the epoch is pretty
 > > reasonable
 > >  > for directories which have no specific time set.
 > >  >
 > >
 > > Actually, at least the manpage is unclear.
 > > And *differs* from the default behavior of native rsync (at lesat 
on
 > > Ubuntu) that sets the dir time to the current time -- which is 
more

 > > reasonable than some arbitrary epoch = 0 time.
 > >
 > > That is what I would have expected and I believe should be the 
default

 > > behavior...
 > >
 > >  > This option has no implications for which directories are 
selected

 > > to be
 > >  > copied.
 >
 > Unset is unset, it's not the option to use if you want the directory
 > modification time set.

Regardless, behavior should be consistent with normal rsync...

If you can show me a standard *nix version of rsync that uses Epoch as
the default then I would retract my point... but otherwise Epoch is
totally arbitrary and illogical... while at least the current time has
a good rationale... Choosing 1/1/1970 not so much...


It's not that "epoch is the default" it's that that's what a timestamp 
of 0 is.  When you tell rsync not to set the timestamps, it doesn't.


If you want to touch the directories and update their timestamps to the 
current time, you can do that, but it's an odd thing to expect rsync to 
take care of for you when you explicitly tell it not to.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BUG? Using --omit-dir-times in rsync backup sets all dir dates to beginning of Epoch

2020-05-22 Thread Michael Stowe

On 2020-05-22 16:19, backu...@kosowsky.org wrote:

Michael Stowe wrote at about 22:24:13 + on Friday, May 22, 2020:
 > On 2020-05-22 09:15, backu...@kosowsky.org wrote:
 > What it does is omit directories from the modification times that it
 > sets.  In other words, you're telling it not to set the times on
 > directories it copies.  The beginning of the epoch is pretty 
reasonable

 > for directories which have no specific time set.
 >

Actually, at least the manpage is unclear.
And *differs* from the default behavior of native rsync (at lesat on
Ubuntu) that sets the dir time to the current time -- which is more
reasonable than some arbitrary epoch = 0 time.

That is what I would have expected and I believe should be the default
behavior...

 > This option has no implications for which directories are selected 
to be

 > copied.


Unset is unset, it's not the option to use if you want the directory 
modification time set.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BUG? Using --omit-dir-times in rsync backup sets all dir dates to beginning of Epoch

2020-05-22 Thread Michael Stowe

On 2020-05-22 09:15, backu...@kosowsky.org wrote:

If I add '--omit-dir-times' to $Conf{RsyncArgsExtra}, then the backups
set all the directory dates to the beginning of the Epoch.

For example
drwxr-xr-x 3 backuppc www-data  1024 Dec 31  1969 pc/

(note this is 1/1/70 00:00:00 GMT)

This is inconsistent with normal rsync wich just uses --omit-dir-times
to omit directories from --times when looking for *changes*

I was expecting and would have liked the normal behavior of
--omit-dir-times to speed up backups...

Is this a bug???


No, it's expected behavior, you seem to have misunderstood what this 
rsync option does.


What it does is omit directories from the modification times that it 
sets.  In other words, you're telling it not to set the times on 
directories it copies.  The beginning of the epoch is pretty reasonable 
for directories which have no specific time set.


This option has no implications for which directories are selected to be 
copied.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Testing full restore of backuppc... MULTIPLE BUGS???

2020-05-22 Thread Michael Stowe

On 2020-05-22 11:42, backu...@kosowsky.org wrote:

1. Sockets are restored as regular files not special files --> BUG?


Why would one back up a socket?

If you really think this is sensible, you should be able to accomplish 
it with "--devices --specials" as part of your rsync command lines.  
From the symptoms, you have this in backup but not restore.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] V4 - eventual effects of wrong sequence when removing Host

2020-04-18 Thread Michael Stowe

On 2020-04-17 03:20, R.C. wrote:

Hi all

Is the following Host remove sequence correct?
- remove Host backups (one by one) with BackupPC_backupDelete
- run BackupPC_nightly 0 255
- remove host from hosts file
- reload server configuration
- remove pc/ folder manually

What would happen if a wrong sequence is followed?
For example:
- remove host from hosts file
- reload server configuration
- remove pc/ folder manually
- run BackupPC_nightly 0 255

Would this leave orphan files in the pool?

Thank you

Raf


I usually follow this sequence:

- remove host from hosts file

That's it.  The nightly job will usually take care of the rest, although 
you can speed things up by running steps manually if you really need the 
space.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] {Disarmed} Using ssh to enable shadow copies, rsyncd on Windows

2020-03-12 Thread Michael Stowe
Sorry -- I didn't mean to be vague, they're here: 
https://u2182357.ct.sendgrid.net/ls/click?upn=iokNmAdIkTJdVIFu9GuODbAWEHZ78jI1cPpfvthoVhC4BaEf72XvCyGKw-2FIr2B-2BW5WbR_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk007fDdKgBV4BVHzrDFRg9PbDYxYUEhEKzW45jh1oFPLgbE-2FTKoWmYsLoI18b7XVoeEuYEC8BCMglq9KO1arAkc4DKANxQ8GtFsmr65UZ9uUGeOs7RkV3tBoZ3-2F2qrOseJ2iFmvKtoSCrVYzcw6EYt9aYA-2BGDXFWZk-2Bo0vy647mQFRYCQ9V8RfO39tcIMcLjQepDagH6kUPUVX-2FZadP-2BodUkQ-3D-3D 


In a nutshell, the installer will select the appropriate vshadow and
rsync for the version of Windows, and install a set of scripts on the
Windows host. 


On 2020-03-12 03:15, Pete Geenhuizen wrote:


Michael,
Which binaries that you've packaged are you referring to and if needed where 
can I find them?
Thanks
Pete

On 3/12/20 1:26 AM, Michael Stowe wrote: 

With each security update and version bump of Samba, winexe is becoming more and more finicky, so I thought it time to see if Window's own ssh could be used for the same purpose.  While it's not as straightforward as I'd hoped, it does work and doesn't require anything other than the binaries I'd already packaged for the purpose. 

This is the script executed with $Conf{DumpPreUserCmd}:  


#!/bin/bash
# 
# Takes one parameter, the name of the system

#
BOX=$1
/usr/bin/ssh Administrator@$BOX 
'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe Invoke-WmiMethod -Class 
Win32_Process -EnableAllPrivileges -Name Create -ArgumentList \"cmd /c 
c:\backuppc\pre-exec.cmd\"'
# The above command is running in the background, we need to wait for rsync
# to start
let i=0
while [ $i -le 72 ]; do
PIDEXISTS=$(ssh Administrator@carrot 'dir C:\BackupPC\rsyncd.pid' |& grep -v "^ Volume" | grep -v "^ Directory" | grep -o rsyncd.pid) 
if [ "$PIDEXISTS" == "rsyncd.pid" ]; then

i=-1
break
fi
echo -n "."
sleep 5
done

if [ $i -eq -1 ]; then
echo ""
echo "Rsync and shadow copy loaded"
exit 0
else
echo "Error loading rsync"
exit 1
fi 

This assumes that one has installed the ssh server that comes with Windows, and set up key authentication.  Note that it's necessary to use an account with Administrative privileges in order to have the requisite permissions. 

$Conf{DumpPostUserCmd} is more straightforward: 


#!/bin/bash
/usr/bin/ssh Administrator@$BOX 'cmd /c echo '1' > c:\backuppc\wake.up' <&-

--
This message has been scanned for viruses and 
dangerous content by MailScanner [1], and is 
believed to be clean. 


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:
https://u2182357.ct.sendgrid.net/ls/click?upn=UlfI6r-2FmuicX-2BnC5-2BZ3I6jqURule12gviJtSX09ijM6elDXB-2FCaheeVayym-2Bos-2FoVpjW9-2FkKQnLXMxydGZB-2FDtFXVbRRZpUOAN0UDXsHbqQ-3DJc3B_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk007fDdKgBV4BVHzrDFRg9PbN8hnA7gGe66-2B-2Fh62fxWXN1S-2FEJRd0SvNCHFDc6b-2Bi3ZV6FG1BUqoaHIXMorzZkOgiO0rhZ480JQTWwL6if95pfz8h6OE41RZfDMtxZqzRjqXL2xOdeAdCUupKXbtdZ7aPuh7BRfc8UA-2Fc6eEZNY2zRa6BjNYbOSi2D9V5K8t9JegiK65XGs8i9IigbKWiTV0w-3D-3D
Wiki:
https://u2182357.ct.sendgrid.net/ls/click?upn=sTLFCyLDXowsKqyboHaFJ7mRQqgCaLHaHEBsMBgsjWJZYA9C6tXX0QCFvWN5V9qyuz9Z_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk007fDdKgBV4BVHzrDFRg9PbIn8rVfMRg-2F5jzzHw0balFODRoWZQ8c3mAiW-2BeZaeOKt4gLVP21DLNmXRPbO3qOq5-2BAUy76143JTRQDoJdvLzZf18h3QOVAOBF9J6MYP5Y10Tt5KFp3zq107kNXUFGrKN9qP3N5MwELC1K2XC4DPxzmXF8HR9Y7lr2YhBoejUg-2ByUg4afh-2FVNJAK3gYZSI5CUg-3D-3D
Project: 
https://u2182357.ct.sendgrid.net/ls/click?upn=sTLFCyLDXowsKqyboHaFJxCQh5xR48engw5-2BOaCnPIaErGA9O-2Bd-2BJOVXIS1FosZ4wpk0_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk007fDdKgBV4BVHzrDFRg9PbNUdPgL6CYDJA36Lxqqmu-2Blz3SxA2VgbJjGT0-2Fb8lBadmAcWAwLtqw0DCXogfEIBkYtFpAal59cHLE84RgIzKutPY7gDZg2Hs-2FWgD1cHw6O27muwwXG4h27r1ZDa8lB-2BAMt5lgzBCBD6Rd3U6K3Sfi3RvjU-2FBG2S7MHnjeetPWSV-2BMpg-2FdjVHoKvGQI-2BkI87sg-3D-3D


--
Unencumbered by the thought process.  
-- Click and Clack the Tappet brothers 


--
This message has been scanned for viruses and 
dangerous content by MailScanner [1], and is 
believed to be clean.




Links:
--
[1] 
https://u2182357.ct.sendgrid.net/ls/click?upn=iokNmAdIkTJdVIFu9GuODcXHWCd42qqD-2BDyXm9ee-2FQQFH-2F6ZG8Lok-2FKPKOw2Uvwq_LO7_ukiVZyKkp5Cjvx76jsH50UVUlXkseXbKCRkPqSLeHuHGH3U7CETSxfS6wuOOBk007fDdKgBV4BVHzrDFRg9PbIi2u2avH4WxQPF9PvXXVRG6SEPxOMHctDfpinjcU5lIioxLY11hQGrWcOQm8YZeHx8f-2BLVwZcjBl40KbgIXpno-2FC-2BMmhn1ZA3xsbNmwZ1YxcgKBl6bfAThJ2cCcqY-2F9Xk7F6RaCh4j5KexpoQe0z0WPqsFoniNc4HVkIVAnAx9kQSikSn2czxTQrp4gXDFFNg-3D-3D___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


[BackupPC-users] Using ssh to enable shadow copies, rsyncd on Windows

2020-03-11 Thread Michael Stowe

With each security update and version bump of Samba, winexe is becoming
more and more finicky, so I thought it time to see if Window's own ssh
could be used for the same purpose.  While it's not as straightforward
as I'd hoped, it does work and doesn't require anything other than the
binaries I'd already packaged for the purpose. 

This is the script executed with $Conf{DumpPreUserCmd}:  


#!/bin/bash
# 
# Takes one parameter, the name of the system

#
BOX=$1
/usr/bin/ssh Administrator@$BOX
'C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
Invoke-WmiMethod -Class Win32_Process -EnableAllPrivileges -Name Create
-ArgumentList \"cmd /c c:\backuppc\pre-exec.cmd\"'
# The above command is running in the background, we need to wait for
rsync
# to start
let i=0
while [ $i -le 72 ]; do
 PIDEXISTS=$(ssh Administrator@carrot 'dir C:\BackupPC\rsyncd.pid' |&
grep -v "^ Volume" | grep -v "^ Directory" | grep -o rsyncd.pid) 
 if [ "$PIDEXISTS" == "rsyncd.pid" ]; then

   i=-1
   break
 fi
 echo -n "."
 sleep 5
done

if [ $i -eq -1 ]; then
 echo ""
 echo "Rsync and shadow copy loaded"
 exit 0
else
 echo "Error loading rsync"
 exit 1
fi 


This assumes that one has installed the ssh server that comes with
Windows, and set up key authentication.  Note that it's necessary to use
an account with Administrative privileges in order to have the requisite
permissions. 

$Conf{DumpPostUserCmd} is more straightforward: 


#!/bin/bash
/usr/bin/ssh Administrator@$BOX 'cmd /c echo '1' > c:\backuppc\wake.up'
<&-___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Full backup will not complete

2020-03-10 Thread Michael Stowe

On 2020-03-09 02:11, George Campbell wrote:

Hello, I am new to this list, so please let me know if there is anything missing from my question... 

I have setup backuppc as a docker (version 4.3.2) on an Ubuntu server. My client is a Windows 10 running RPC.  

A full backup runs and I can see lots of files on the server. But, the backup never seems to finish and gets stuck. I can't see that the backup in incomplete, though I have not tried to test it.  

A few observations:  
- on the client I can see the RPC processing running, but it does not seem to do anything withe with the disk or network 
- the docker is setup to use 8 threads, but only ever uses one. I have changed the MaxBackupPCNightlyJobs to 8.  
- The server and client are both massive with 64/16gb of memory and 12/4 cores respectively. Both, and the network, seem untaxed.  
- I can open a file and restore back to the client, so the config seems to all be working.  

Can someone please suggest what could be wrong, or where I should look next? 


Thanks, George


There's a lot to unpack here, so please bear with me a bit: 


First, I don't know what you mean by RPC -- windows has an RPC service,
but I'm not sure how that relates to one of the BackupPC transports
(e.g., smb, rsync, ftp) and to make use of the RPC service, you'd need
to do some custom work or scripting.  (If this is the case, you may want
to elaborate.) 


Second, ignoring the above and assuming you're using smb or rsync,
you'll need a strategy to deal with windows filesystem semantics and
features.  It's pretty easy to hang either transport by trying to back
up an open file or junction.  Tried and true strategies involve shadow
volumes or ... not doing that. 


MaxBackupPCNightlyJobs will run backup jobs in parallel, but it won't
automatically split your job for you. 


While 16G of memory and 4 cores isn't terrible, it's worth noting that
IO speed is a significant factor, so unless that's enough RAM to cache
your entire backup (or filesystem) then expect to run into IO
constraints -- and note that the time for an individual backup job is
usually proportional to the size of the data. 


And finally, note that there have been various bugs in transports, so
it's probably not a bad idea to be specific about the version in the
case that somebody recognizes it as being problematic in some way.___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Windows 10 fell into hibernation when "rsyncd" backup is done

2020-01-20 Thread Michael Stowe

On 2020-01-17 10:05, DI Harald ARNOLD wrote:

Hi to all, I am a newbee at this list. I found nothing
about this thread in old archives.

I installed backuppc on a linux openSuSE Tumbleweed box
to backup two methods of a "Windows 10" box by "rsyncd".

Everything is working fine, both "methods" are starting.

But the Windows 10 box is going into hibernation mode.
I would like that the windows box is using hibernation,
but not when "rsync" (the backuppc job) is running.

I switched of hibernation mode an everything was fine.

Is there a configuration that windows uses hibernation
and will wake up or do not go into hibernation mode,
when backup is starting/running.

Till now I have no configuration for waking up windows
and to ignore hibernation at backup time.

Thanks Harald


Depending on how you're managing rsync, it's relatively easy to manage 
this through a script:


powercfg.exe /hibernate off


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Win10 rsync and shadow

2020-01-11 Thread Michael Stowe

On 2020-01-09 12:23, Greg Harris wrote:

Anyone tried enabling the in-built Win10 SSH server and then using straight rsync, not rsyncd, with cygwin-rsync client?   

Additionally, and maybe I should create a separate thread, but does the cygwin-rsync client that's bundled on the GitHub page use the shadow volume?  I can't see anywhere that it discusses this, nor find any forum posts.  However, this forum post says there are at least two solutions for doing shadow copies with rsync: 

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7YQ8BYSHbrWZS1jKQzKBLmHDvD-2BcaXOPRSW8DuNIHKGhDNdyUJdGmEZGMgRHKaX7P1K7kmZ1gTYsRz7rQadIHYI-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91C9VErJ0fkm9L7N1YEY7J0kzi5n1ImaaJEq67DMUehVuqG-2FePD7-2Bl-2FX-2ByZ9cHRHZ-2FDIk3AEthc7CU8gT9hvmsidxkLFlx53NjOg2g1v2ukFttXfrvM5-2FmseAVt1e8wtNi7y6YLFS3vu1hdcPuCZeI7Lyw3t6fZ9Jmwjz50zp-2Fraxtk-2FUexRGsHh0BxBnnCeCEjf5klOAVG6GRbAiOJRgaohQ-3D-3D 

Yet, searching around, I've only been able to find: 

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7Zaxtwab-2FG40JLJH69HA23Ye53aTPmpurGS-2FLnwtQoz5_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91C9VErJ0fkm9L7N1YEY7J0k428nU-2FnEzRAlLUeBjcbJaT5-2BsEnyAFI6H-2BPVsEfJPzA-2BxtlSI0c37BOfIEQxa5qSBr6Fc8A412khMFvNbKQpMstKHBvUc4Xvo9TdaK5NHv4-2FuA9n6wIVEGaJhJS8l0-2BnxIbRE1smXmlark8NAjcHGEBLvqAwrJcloSbdS6SoyOF3HU-2B1UPlRGRTlptwDiCZYg-3D-3D 


Thoughts?  Ideas?  Tomatoes?

Thanks, 


Greg Harris


It has been a while since I tried it, but last time I did, I had trouble
with permissions (i.e., there were many files that it simply did not
allow reading) and with shadow volumes -- there's a bit of a Catch-22
where the shadow volume is unusable unless it's created before the ssh
service is started, which kind of defeats the point of using it to
control the shadow volume. 


To explain a bit more, there are two kinds of shadow copies:  persistent
and transient.  One could create a persistent shadow copy and back it
up; it will have its own path (that is set at the time it's created.) 
Transient shadow copies belong to their context, and disappear when

rebooting or when whatever process owns them terminates.  I prefer the
latter, though the differences are really timing (in the sense that if
one created a persistent shadow copy last Thursday and backs it up, the
backup would only be current as of last Thursday) and management.___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Issue with configure.pl not recognizing user

2020-01-02 Thread Michael Stowe

On 2020-01-02 14:14, White FrosT wrote:


Thanks Michaelm "passwd backuppc" seems to work fine, but the configure script 
does not. I did the modifications using usermod, useradd etc. Is there anyone that knows 
the configure script in depth and can help me with a step by step debug of this issue?


I'm fairly familiar with the configure scripts, though the first place
I'd look is that there might be a clue in the package for vipw not
looking for the passwd file in the right place -- by a similar token, if
the module perl uses to access user information is looking in the wrong
place, you'll have the same symptoms.  Was perl and its modules
installed from the same repository?___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Issue with configure.pl not recognizing user

2019-12-31 Thread Michael Stowe

On 2019-12-31 09:03, White FrosT wrote:

Hi All, 
Thanks for the feedback, but I'm not getting there yet. 

I tried vipw, but this doesn't work. It was not on my system, installed it with opkg install shadow-vipw. But then running vipw gives me: 

vipw: /opt/etc/passwd: No such file or directory 
vipw: /opt/etc/passwd is unchanged 

Neither does adding a shell to the user (/sbin/nologin) change anything. 

Thanks for any additional idea's! 

Op di 31 dec. 2019 om 16:54 schreef mlm : 

On 12/30/2019 6:57 PM, Michael Stowe wrote: 

On 2019-12-30 12:24, White FrosT wrote: 

Dear list, 
I have an issue installing backuppc on my new network disk running Linux 4.2.8 ARMv71. I think I have managed all preconditions in perl (v5.28.1) and are ready to install, but I run into an issue with configure.pl [1]. It doesn't let me set the user to run BackupPC with. See the snapshot of the script output below: 

--> BackupPC should run as user [backuppc] 

getpwnam() says that user backuppc doesn't exist.  Please 
check the name and verify that this user is in the passwd file. 

But the user does exist on my system, see: 

[/.../BackupPC-4.3.1] # grep backuppc /etc/passwd 
backuppc:x:1000:1000::/share/homes/Backupuser/: 

I cannot seem to workaround this problem. Does anyone have any leads? 

Thanks a lot! 


For what it's worth, I've seen this behavior when shadow hasn't properly been 
updated, and it took (takes?) running vipw, then rebooting.  YMMV


hi, 

Might need an entry where the shell for the user normally goes. 

This is what my entry looks like: 

backuppc:x:988:983::/var/lib/BackupPC:/sbin/nologin 

Mark 


What vipw does is set the appropriate locks and prevent corruption while
editing the password file -- under the covers, it's often vi or vim,
anyway.  Depending on the platform, this will help trigger the shadow
update (thus my suggestion to use vipw and reboot.)  In your case, it
seems the package for vipw doesn't fit with the actual location of
passwd and shadow on your system. 


Usually it's best not to directly interact with either file, as a syntax
error can result in accounts not working, instead it is better to use
tools like adduser and passwd.  Officially, authconfig (on Linux) is
probably the right way to update whatever files must be coordinated. 
You could also just use "passwd backuppc" as root. 


Links:
--
[1]
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7Z4Qyz-2FdRVTsykwiKMAKNXGM7CUf5juiBco6vpfp2jz1b2ywz5iHhGuMBRW-2BbsRpyriQYngu5p4EU0mD2E0mSA-2Frx4vGZ-2Fp1CIYOyKiFCqZL8eCJP1ev5gbx6IyiE2BTfJkKHjUD3jIlJreydTMKH-2Bqv1D4uuq43VMS0FaikvKy7VqAPUlvbdOkdgQpiX1eIZsFi7E0PGALblqcD6XODGTiQfrxf2Ww-2BoZuWZLFU2X31h1Hj-2B7GJrneyiX1-2FFvnlxU4I7wTHt8-2BJvq-2BuE5zLiTr95cPfCmN-2BgBrpaxf7gPSN-2BTzAdyMfJ5sUdtb9PCHWXN7wScI7pZTjmfQO3D5gZUHGRa2CsHJXBJdrhwmu7dTlpR03aIEMYoo8yZwBwNj4gjsJIsG7Su84FJqvcIyIpflXD6tIE5sOA4Mgz7Gm4AxuGSD7ZuLRgeSz4BobtEiICRLRUkfskUAU9FWfXtMaQEfFHl4E9e5lVErw-2BKLj8Wso_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91C4PR-2FeKP-2FLdO8ln6xWRdGp8cnfpoH5V4NVAZe-2Fq-2Bh9z0g8pmaI9I2RofHWs4abKW0hMxP-2FYgDt6jpedl3dEvSCf5KgwV1hHupA-2BE700SkCEX7qGo18pLl7f4olsiV1CBAjtJ7lRt4MzjhRnMz1A0wajuroxcwbv1LttHgNJR5ZfPazVedd3kwRdltgqwFurVY___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Issue with configure.pl not recognizing user

2019-12-30 Thread Michael Stowe

On 2019-12-30 12:24, White FrosT wrote:

Dear list, 
I have an issue installing backuppc on my new network disk running Linux 4.2.8 ARMv71. I think I have managed all preconditions in perl (v5.28.1) and are ready to install, but I run into an issue with configure.pl [1]. It doesn't let me set the user to run BackupPC with. See the snapshot of the script output below: 

--> BackupPC should run as user [backuppc] 

getpwnam() says that user backuppc doesn't exist.  Please 
check the name and verify that this user is in the passwd file. 

But the user does exist on my system, see: 

[/.../BackupPC-4.3.1] # grep backuppc /etc/passwd 
backuppc:x:1000:1000::/share/homes/Backupuser/: 

I cannot seem to workaround this problem. Does anyone have any leads? 


Thanks a lot!


For what it's worth, I've seen this behavior when shadow hasn't properly
been updated, and it took (takes?) running vipw, then rebooting.  YMMV 


Links:
--
[1] 
https://u2182357.ct.sendgrid.net/wf/click?upn=ktc6Sw27myTf4Fow3Cmoe104eSS9eTV3JAv92P6q3YM-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CDs-2BYzdt2HE8zUz4v3W9zJ5KQFWgpKYFRloa8R5pL1JpkwdFsV6MyURoVYyORP5KbjcNIro-2BSppu3j4gfiCMhsryu4Rz4gSeys4CPAHiTL-2BIth93gfmW0PKHhfw79s4T4x5w-2FVI2oizAulx2K-2BFLtcKonOQRVB4FWb2ZkKQJqehg-2B-2FlScX881ft1lM6769ofr___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Does backuppc support bare metal restore?

2019-11-27 Thread Michael Stowe

On 2019-11-25 05:51, Marek Grossman wrote:

Hello,
I can't find the clear answer on backuppc WEB site, so I ask you here.

Does BackupPC support bare metal restore in nowadays or not (last
notice is from 2007)?

Is there any other way to restore completely Windows 7/10 installation
with more users after, for example, I change HDD?

Thanks
--
Mareg


This works for W7:  
http://u2182357.ct.sendgrid.net/wf/click?upn=BlpU7GL5eKRV06yRQzHbsq14yYFgT-2FzyzGIXamlhpJ3dEyk-2BwuSj-2B8ne9ib3HujZ_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91C2EZuS-2FVPVK6LQIOlb6Sv7cJCWOfnr7sSpDHd1mXDrhnm-2Ftl-2Bqd9MfO6GD6Rt3JzV8XVFSSt6OgtEBR7wpPyMgBO4QYG7-2BKRhRaDQUorvx57gsnRkEL2UB5pi9MIeZ9doxNvNtjZ6KjfVaJe4huwmFrush-2FYmNSbi2-2B6uQBBFbZgxEVkLEfMiYBXtxqTOzU-2FE

YMMV
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] home directory empty

2019-09-28 Thread Michael Stowe
On 2019-09-28 04:34, Bob Wooden wrote:

> I recently built (on newer hardware) a BackupPC v.4.3.1 on CentOS 7 using 
> hobbes1069/BackupPC repo. 
> 
> Letting BackupPC do backups and time has passed (couple of weeks) and this 
> morning I discover, while "browsing backups" that the /home directory is 
> empty on one (2) if the clients and are NOT being backed up. Directory /home 
> has NOT been excluded, therefore, I do not understand why it is not backing 
> up. 
> 
> There are seven (7) clients being backed up and only two are not backing up 
> /home, the others show files in /home. 
> 
> I have increased the loglevel to "2" and ran a "Incr Backup" on one client 
> and the logs say:
> 
> 2019-09-28 06:23:12 incr backup started for directory /
> 2019-09-28 06:23:18 incr backup 40 complete, 66902 files, 2977658199 bytes, 0 
> xferErrs (0 bad files, 0 bad shares, 0 other)
> 
> Surely I need to look at other logs and adjust log settings to determine what 
> is happening. 
> 
> Any suggestions on what to look at are welcome, please. 
> 
> Help?

rsync --one-file-system___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] smbclient gets NT_STATUS_ACCESS_DENIED

2019-08-30 Thread Michael Stowe

On 2019-08-29 19:55, Bob McClure Jr wrote:

On Thu, Aug 29, 2019 at 10:48:45PM +, Michael Stowe wrote:

On 2019-08-29 14:47, Michael Stowe wrote:

> On 2019-08-29 13:02, Bob McClure Jr wrote:
>
>> I'm at wit's end -- a short trip. I have a very simple setup of a Linux server (CentOS 6) 
running BackupPC to back up a Win 10 Pro machine. BackupPC is BackupPC-3.3.1-5.el6.x86_64 from the EPEL 
repo. On the Win 10 box, I created a normal user "backuppc" with a password, and then put him 
in the local "Backup Operators" group. I'm using SMB for the backup method.
>>
>> # smbclient '//nate/C$' -U backuppc -c pwd Enter backuppc's password: 
Domain=[NATE] OS=[Windows 10 Pro 18362] Server=[Windows 10 Pro 6.3] tree connect 
failed: NT_STATUS_ACCESS_DENIED
>>
>> And of course, the backups fail. I've done a Web search from the Win 10 
aspect, the samba aspect, and the BackupPC aspect, and have come up dry.
>>
>> Here is the hosts file:
>>
>> host dhcp user moreUsers nate 0 bob
>
> You can debug this entirely at the command line; you're likely to find "-d 
debuglevel" helpful as it can help you pinpoint exactly what's going wrong.
>
> As another response alluded to, this is often due to smbclient not supporting 
recent enough protocol versions; ratcheting up the debug level should allow you to 
see the negotiation. If this is indeed the case, this can be mitigated by 
upgrading smbclient (preferred) or allowing Win10 to fall back to older versions.

I'd still recommend that as a debugging method, but one other 
potential

cause is that Win10 now disables remote access to the administrative
shares by default.  They can be specifically re-enabled, but without
doing so, would generate an access denied error no matter what
credentials were used.


I tried the debug level at 3 (the highest recommended for mere
mortals) and found nothing educational.

I rebooted my workstation to CentOS 7 and tried smbclient and it, too,
failed, so I guess it's the remote access problem.  Does that mean
rsync won't do any better than smb or does rsync get past the remote
restrictions?

Any clues on how to enable remote access?  I'm not concerned about
security since we're behind a good firewall.

Cheers,


This is how to enable remote shares:
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7fB0NECgtVa3tQh4O52BIRkc9-2FxI35iS9RIffWOpxMiHDlA6Rik7ZqM5ULeaWsMfPcBwDahMJxrjwfA-2Bq1J0hWE-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CG4cTrGD7J6lbr2S8rKruFJaFr3HD6Acrq1e6JDqWs-2FonTyT-2FvpG6YsaxO0wo23nI447AMCTK6PKUWErD5zqkMRMsxa8Ddmwq8fsQ-2FvPNPOCEsHs91K0AtI7-2F5WSzLZQTq8MtcMjojhu-2F1yJMraKlBa7tG7PrVsU4scW1-2Fpdy0lM-3D

This is how to enable older versions of the smb protocol:
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7Qgbyc8hNp-2BwXA9Wwf3uSMz-2Ff-2FFndnY61cbQ417VQmoH-2B0F2LWOyHDRpeiOS8m292oOfPzsb7Fbw-2BpFvCQ4SMmEqFawMqNFqVrHVPS2yX10mpJmRVkJybwWX3dKfBOsk-2F0Rpp6D736v4IoJiA6yBb1p9HsUWLqx4w-2BSM0-2BPMezbtgEewkV997QtEiwhrNT6Vxw-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CG4cTrGD7J6lbr2S8rKruFEIXTbsXfWZtSB0lat-2FM56v2zfxbuFYPR-2BhlM0MH36fnZPUnFTGBGGlFHyw6-2FQjUfS5XMT-2FrAJDDXA4Tcz63xPF1i7gPRaiWybANdab5VDe-2BZaFo-2FUr7NqRmjBN6xZ7J-2B4iPExEafxfpUjDBwMuLm7c-3D

Though to answer your question rsync doesn't rely (or is aware of in any 
way) the default shares, and it doesn't use the smb protocol, so you 
wouldn't have these problems.  You may have a different set of problems, 
mainly revolving around getting permissions and modules right, and (if 
you use it) properly engaging shadowing.  That said, SMB can't use 
shadowing at all, so I see rsync as a major improvement in many ways and 
use it for all my windows systems.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] smbclient gets NT_STATUS_ACCESS_DENIED

2019-08-29 Thread Michael Stowe
On 2019-08-29 14:47, Michael Stowe wrote:

> On 2019-08-29 13:02, Bob McClure Jr wrote:
> 
>> I'm at wit's end -- a short trip. I have a very simple setup of a Linux 
>> server (CentOS 6) running BackupPC to back up a Win 10 Pro machine. BackupPC 
>> is BackupPC-3.3.1-5.el6.x86_64 from the EPEL repo. On the Win 10 box, I 
>> created a normal user "backuppc" with a password, and then put him in the 
>> local "Backup Operators" group. I'm using SMB for the backup method. 
>> 
>> # smbclient '//nate/C$' -U backuppc -c pwd Enter backuppc's password: 
>> Domain=[NATE] OS=[Windows 10 Pro 18362] Server=[Windows 10 Pro 6.3] tree 
>> connect failed: NT_STATUS_ACCESS_DENIED 
>> 
>> And of course, the backups fail. I've done a Web search from the Win 10 
>> aspect, the samba aspect, and the BackupPC aspect, and have come up dry. 
>> 
>> Here is the hosts file: 
>> 
>> host dhcp user moreUsers nate 0 bob
> 
> You can debug this entirely at the command line; you're likely to find "-d 
> debuglevel" helpful as it can help you pinpoint exactly what's going wrong. 
> 
> As another response alluded to, this is often due to smbclient not supporting 
> recent enough protocol versions; ratcheting up the debug level should allow 
> you to see the negotiation. If this is indeed the case, this can be mitigated 
> by upgrading smbclient (preferred) or allowing Win10 to fall back to older 
> versions.

I'd still recommend that as a debugging method, but one other potential
cause is that Win10 now disables remote access to the administrative
shares by default.  They can be specifically re-enabled, but without
doing so, would generate an access denied error no matter what
credentials were used.___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] smbclient gets NT_STATUS_ACCESS_DENIED

2019-08-29 Thread Michael Stowe

On 2019-08-29 13:02, Bob McClure Jr wrote:

I'm at wit's end -- a short trip.  I have a very simple setup of a
Linux server (CentOS 6) running BackupPC to back up a Win 10 Pro
machine.  BackupPC is BackupPC-3.3.1-5.el6.x86_64 from the EPEL repo.
On the Win 10 box, I created a normal user "backuppc" with a password,
and then put him in the local "Backup Operators" group.  I'm using SMB
for the backup method.

# smbclient '//nate/C$' -U backuppc -c pwd
Enter backuppc's password:
Domain=[NATE] OS=[Windows 10 Pro 18362] Server=[Windows 10 Pro 6.3]
tree connect failed: NT_STATUS_ACCESS_DENIED

And of course, the backups fail.  I've done a Web search from the Win
10 aspect, the samba aspect, and the BackupPC aspect, and have come up
dry.

Here is the hosts file:

hostdhcpusermoreUsers
nate0   bob


You can debug this entirely at the command line; you're likely to find 
"-d debuglevel" helpful as it can help you pinpoint exactly what's going 
wrong.


As another response alluded to, this is often due to smbclient not 
supporting recent enough protocol versions; ratcheting up the debug 
level should allow you to see the negotiation.  If this is indeed the 
case, this can be mitigated by upgrading smbclient (preferred) or 
allowing Win10 to fall back to older versions.


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] PoolSizeNightlyUpdatePeriod and BackupPCNightlyPeriod

2019-08-08 Thread Michael Stowe

On 2019-08-08 02:08, Gandalf Corvotempesta wrote:

Any advice on this ? I have to keep the nighly job as light as possible

Il giorno ven 19 lug 2019 alle ore 11:31 Gandalf Corvotempesta
 ha scritto:


On official docs, max value is 16. As I have a very slow server with a
very big pool, is 16 the max value I can set or it's just an example ?

Can I set it to 32 ? cntUpdate phase is taking ages... (3 days)


You seem to have confused the default and the max value.

It's a calculation of pool size, it's the denominator of the fraction of 
how much to refresh every night.  If you're happy with 1/32 every night, 
set it to 32.


Or just set it to 0 to disable it.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Starting BackupPC server manually

2019-08-03 Thread Michael Stowe
On 2019-08-03 14:14, Norman Goldstein wrote:

> On 2019-08-03 2:08 p.m., Michael Stowe wrote: 
> 
> On 2019-08-03 08:24, Norman Goldstein wrote: Am running Fedora 30 x86-64, BP 
> 4.3.1
> Am not able to start the BackupPC server manually.
> 
> I would like to be able to run the BC server manually, to be able to debug 
> into it.  When I do, as root:
> 
> systemctl stop backuppc
> 
> this stops the server.  Then, as user backuppc, to start the server manually:
> 
> /usr/share/BackupPC/bin/BackupPC
> 
> but the BP web page does not come back to life.  I kill this, and do:
> 
> systemctl start backuppc
> 
> and the BP web page comes back to life.  I checked the sysd file
> 
>> cat /usr/lib/systemd/system/backuppc.service
> [Unit]
> Description= BackupPC server
> After=syslog.target local-fs.target remote-fs.target
> 
> [Service]
> Type=simple
> User=backuppc
> Group=backuppc
> ExecStart=/usr/share/BackupPC/bin/BackupPC
> ExecReload=/bin/kill -HUP $MAINPID
> PIDFile=/var/run/BackupPC/BackupPC.pid so there does not seem to be anything 
> special about invoking the BP server.  The same problem if I use the  -d  
> flag when invoking BackupPC .

Perhaps not special enough, but surely you've noticed these lines? 

User=backuppc
Group=backuppc Yes, I invoke BackupPC from a backuppc xterm:
$ groups
backuppc
$ whoami
backuppc

Thanks for raising that possibility.

Since it's a simple type, there's not anything special that systemd does
except ... exactly what you did at the command line.  (N.B.:  it's
possible to override systemd with unit files that take priority, but I
imagine if something like that were at play you'd know about it.) 

What this leaves is environmental differences between the user
backuppc's shell and systemd's backuppc environment -- I can't imagine
what that would be that would be that would affect only the web
service...  What is the configuration for that (from backuppc's config)?___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Starting BackupPC server manually

2019-08-03 Thread Michael Stowe
On 2019-08-03 08:24, Norman Goldstein wrote:

> Am running Fedora 30 x86-64, BP 4.3.1
> Am not able to start the BackupPC server manually.
> 
> I would like to be able to run the BC server manually, to be able to debug 
> into it.  When I do, as root:
> 
> systemctl stop backuppc
> 
> this stops the server.  Then, as user backuppc, to start the server manually:
> 
> /usr/share/BackupPC/bin/BackupPC
> 
> but the BP web page does not come back to life.  I kill this, and do:
> 
> systemctl start backuppc
> 
> and the BP web page comes back to life.  I checked the sysd file
> 
>> cat /usr/lib/systemd/system/backuppc.service
> 
>> [Unit]
>> Description= BackupPC server
>> After=syslog.target local-fs.target remote-fs.target
>> 
>> [Service]
>> Type=simple
>> User=backuppc
>> Group=backuppc
>> ExecStart=/usr/share/BackupPC/bin/BackupPC
>> ExecReload=/bin/kill -HUP $MAINPID
>> PIDFile=/var/run/BackupPC/BackupPC.pid
> so there does not seem to be anything special about invoking the BP server.  
> The same problem if I use the  -d  flag when invoking BackupPC .

Perhaps not special enough, but surely you've noticed these lines? 

User=backuppc
Group=backuppc___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Problem with multibyte character

2019-06-15 Thread Michael Stowe

On 2019-06-15 19:20, backu...@kosowsky.org wrote:

I am running backuppc 4.3.0 on Linux Ubuntu 18.04

I have a file copied over from an old Windows installation to my Linux
server with name:

-rwx-- 1 user1 user1  28672 May 11  2005 'r'$'\351''ponse.doc'

This file gives an error under backuppc as follows:
 [sender] cannot convert filename:
 user1/docs/r\#351ponse.doc (Invalid or incomplete multibyte or
wide character)

Note that rsync itself has no trouble copying this file when I run
rsync manually.

Also, interestingly, this same file backed up fine on my old Fedora 12 
server

running backuppc 3.x.

Any ideas why backuppc is having trouble with this file?


Almost definitely because the encoding is invalid relative to the 
locale.  The simplest thing to do is to rename the file so that it's a 
valid filename.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BACKUPPC V4 - rsync-bpc error connecting to client

2019-06-07 Thread Michael Stowe
On 2019-06-05 12:52, David Wynn via BackupPC-users wrote:

> Hi Mike ... don't know if I replied correctly to this or not  And NO - I 
> have not changed the RsyncBackupPCPath, it is still /usr/local/bin/rsync_bpc. 
>  And YES, the 192.168.1.6 is the IP address to our Netstore device.   Just 
> can't figure out why it is appearing in the command line ... aaarrrggghh (I 
> almost hate technology!)

One minor thing, it actually looks like RsyncClientPath is set to
192.168.1.6.  It should look like this: 

$Conf{RsyncClientPath} = '/usr/bin/rsync';

Also worth checking: 

$Conf{RsyncSshArgs} = [
'-e', '$sshPath -l root',
];___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] restore issue on backuppc 4.1.5

2019-06-04 Thread Michael Stowe
On 2019-06-04 07:33, Philip Parsons (Velindre - Medical Physics) wrote:

> Thanks for the reply Michael. 
> 
> I've tried to restore to another share on another server and am getting 
> NT_STATUS_BAD_NETWORK_NAME 
> 
> I don't want to overwrite the existing data, I just want to be able to 
> download data to another share so that I can QA our backups. 
> 
> Running: /usr/bin/smbclient destinationserver\\share -U domainaccount -E 
> -d 1 -c tarmode\ full -mSMB3 -Tx - 
> 
> Running: /usr/local/BackupPC/bin/BackupPC_tarCreate -h sourceserver -n 510 -s 
> sourceshare -t /sourcefolder 
> 
> Xfer PIDs are now 104823,104824 
> 
> WARNING: The "syslog" option is deprecated 
> 
> [ skipped 1 lines ] 
> 
> This backup will fail because: tree connect failed: 
> NT_STATUS_BAD_NETWORK_NAME 
> 
> tree connect failed: NT_STATUS_BAD_NETWORK_NAME 
> 
> [ skipped 1 lines ] 
> 
> tree connect failed: NT_STATUS_BAD_NETWORK_NAME 
> 
> readOutput: sysread returns 0 and got EOF (exit ok = , ) 
> 
> XferErr Non-zero exit status from smbclient 
> 
> restore failed: tree connect failed: NT_STATUS_BAD_NETWORK_NAME 
> 
> thanks again for any advice! 
> 
> Phil

Samba errors are maddeningly unspecific.  In this case, the error
translates to "something about that share ain't working for me."  It
could be anything from the share name being misspelled, or the wrong
part of the tree, or permission errors. 

At any rate, the simplest way to diagnose this is to start with what the
share names are: 

smbclient -L //destinationserver 

This should give you a list of valid shares -- if the sharename you
tried doesn't exactly match one of the shares in that list, viola, that
is the trouble.  (N.B. I do mean *exactly.*  There's no support for
joining the tree at a different point in the hierarchy.)___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] restore issue on backuppc 4.1.5

2019-05-22 Thread Michael Stowe

On 2019-05-22 05:32, Philip Parsons (Velindre - Medical Physics) wrote:

Hi,

I'm having an issue with restoring to a share and am trying to deal
with this before it becomes a need to restore situation!

We're using BackupPC v4.1.5.

I've had the following error when trying to restore:

Running: /usr/bin/smbclient servername\\sharename -U accountname
-E -d 1 -c tarmode\ full -mSMB3 -Tx -
Running: /usr/local/BackupPC/bin/BackupPC_tarCreate -h servername -n
211 -s sharename -t -r / -p /temp/ /restorefolder
Xfer PIDs are now 48288,48289
WARNING: The "syslog" option is deprecated
[ skipped 1 lines ]
tar:316 tarmode is now full, system, hidden, noreset, quiet
tar:1597 Can't mkdir temp: NT_STATUS_OBJECT_NAME_COLLISION
readOutput: sysread returns 0 and got EOF (exit ok = , )
XferErr Non-zero exit status from smbclient
restore failed: BackupPC_tarCreate failed

Does anyone know what could be causing this?

Thanks in advance,

Phil


It looks like you're trying to restore to a directory called temp, which 
already exists, but is hidden.


You probably just want to choose a different directory to restore to.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Backups missing entire directories: "file has vanished"

2019-05-07 Thread Michael Stowe

On 2019-05-07 13:39, Raman Gupta wrote:

Certain directories (and their contents) on one of my hosts are not
getting backed up at all, even with a "Full" backup.

I use rsync as my Xfer method, with BackupPC 4.3.0 on Fedora (rpms
BackupPC-4.3.0-1.fc29.x86_64, BackupPC-XS-0.58-1.fc29.x86_64).

Looking at the backup logs, I see messages like the following related
to the directories that are not being backed up:

file has vanished: "/home/raman/x/y/a"
file has vanished: "/home/raman/x/y/b"
file has vanished: "/home/raman/x/y/c"

I have other directories and files successfully backed up in
"/home/raman/x/y", but the directories "a", "b", and "c" (and their
content) are not being backed up.

Note that these files have *not* vanished -- they are not ephemeral
and they haven't been touched in days. For example:

  File: /home/raman/x/y/a
 Size: 4096Blocks: 8  IO Block: 4096   directory
Device: fd08h/64776dInode: 33037482Links: 5
Access: (0775/drwxrwxr-x)  Uid: ( 1000/   raman)   Gid: ( 1000/   
raman)

Context: unconfined_u:object_r:user_home_t:s0
Access: 2019-05-07 05:05:17.288857497 -0400
Modify: 2019-04-30 00:56:22.914849594 -0400
Change: 2019-04-30 00:56:22.914849594 -0400
Birth: -

Any idea what might be happening here?

Regards,
Raman


"File has vanished" issues can be tricky to diagnose if the file appears 
to be there.  What rsync is really telling you is that it built a file 
list, and some of the files or directories from that list are not 
accessible when it actually went to read them.  Actually being deleted 
or ephemeral files are two reasons, but there are others, from filename 
encoding issues to inode changes to complications with remotely mounted 
filesystems to corruption issues to complex file permissions.


While I might check the file's details both before and after the rsync 
run to look for changes, I recommend ensuring that these files are 
reliably accessible by the rsync user, check the logs for any problems, 
and working through filesystem issues.  (XFS is notorious for this sort 
of thing.)  Also, if the volume is anything other than a local mount, 
that's where I'd look first for issues; be aware that rsync's high read 
volume often exposes issues not evident under less stressful usage.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Rsync backup does not start

2019-05-06 Thread Michael Stowe

On 2019-05-06 00:26, THÉBAULT Arnaud wrote:

HI,

I am  new to backuppc.. again ( I was used to work with it nearly  15
years ago…)

I have an issue when trying to backup a Windows 10 computer with
rsyncd.

When initiating the backup from the backupPC GUI, it never starts.

There is no error message in the log file.

I have tried to launch it manually  (
__INSTALLDIR__/bin/BackupPC_dump -v -f hostname ) and it works fine
without any error message.

So I have no idea why it doesn't work when I start it from the GUI.

Best Regards,

Arnaud THEBAULT


The logs should tell you for sure, but the most likely reason is that 
the Windows 10 system isn't pingable, which is the default.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Partial backup is not saved

2019-04-29 Thread Michael Stowe

On 2019-04-29 08:08, Tapio Lehtonen wrote:

I split a host to two backups, because dump directory contains virtual
machine dump files which are big and are taken every night. So I needed
shorter keep time for those backups to not run out of disk on the 
backup

host. But this new backup setup does not seem to work. I takes a long
time but XferLOGbad shows this:

  create   644   0/0 100609398737 
rottank/bupool/dump/vzdump-qemu-5099-2019_04_25-00_16_31.vma.lzo
  pool 644   0/0   11286 
rottank/bupool/dump/vzdump-qemu-5099-2019_04_26-00_16_26.log
finish: removing in-process file 
rottank/bupool/dump/vzdump-qemu-5099-2019_04_26-00_16_26.vma.lzo

Child is aborting
Done: 89 files, 910772510921 bytes
Got fatal error during xfer (aborted by signal=ALRM)
Backup aborted by user signal
Not saving this as a partial backup since it has fewer files than the 
prior one (got 89 and 89 files versus 176002)


Does the last line mean BackupPC things this backup ought to have 
176002
files, and since only 89 files got backed up this backup is thrown 
away?


The same host is backed up twice, I added exclude for that dump/
director and this new backup only backs up that dump/ directory.


There are a few things to address here:
* The removed in-process file -- presumably, you want to back this up; 
since it's a huge file, there may be issues with timeout settings, et 
al. but the log is saying it's not backed up.
* "Splitting a host into two backups" isn't necessarily the best idea.  
While it can work for non-overlapping shares, you're working against the 
grain in terms of fills, etc.  What you probably want to do is split the 
host into two hosts and manage them independently, so BackupPC is making 
the right assumptions in terms of how to do fills, what to keep, etc.
* As you work through these issues, you probably want to include the 
relevant configs, e.g., rsync parameters for 100G files and how you're 
splitting the hosts.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Windows client restore

2019-04-28 Thread Michael Stowe

On 2019-04-28 09:15, Gopal Dhapa wrote:

Hello support Team.

When I use backuppc in Linux client it's working okay but windows
client is not specify director backup it's take full c drive backup.
Also windows client backup not restore it's given my status access
denied using smb xfer method.

Please guide me.

Thanks in advance.

Thank you
Gopal Dhapa


You may just want to go one of the "download archive" options instead of 
a direct restore.


Direct restore via smb *can* work, but Windows can be picky about 
permissions and semantics for overwriting files, and due to the nature 
of the SMB protocol, if you do get it working, there may be additional 
work to do to make the files readable & writable by others.  If you're 
backing up using one of the Administrative Shares (e.g., C$ or Admin$) 
you'll run into additional problems; it's better not to back up or 
restore via these shares.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] ssh tunnel established, but backuppc doesn't begin backup

2019-04-25 Thread Michael Stowe

On 2019-04-24 14:21, Justin Magers wrote:

I'm trying to tunnel through a computer (HOST) to which I (justin)
have ssh access to so that rsyncd has a secure connection for
BackupPC.  The tunnel is setup using DumpPreUserCmd:

$sshPath -M -S ~/.ssh/sockets/stradella -fN -l justin -L
8874:stradella.local:873 HOST -p PORT -v

The tunnel is successfully established, but then BackupPC just sits.
The log looks like this:

2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: Authentication
succeeded (publickey).
2019-04-24 13:42:17 Output from DumpPreUserCmd: Authenticated to HOST
([XXX.XXX.XXX.XXX]:PORT).
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: Local
connections to LOCALHOST:8874 forwarded to remote address
stradella.local:873
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: Local
forwarding listening on ::1 port 8874.
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: channel 0: new
[port listener]
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: Local
forwarding listening on 127.0.0.1 port 8874.
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: channel 1: new
[port listener]
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: setting up
multiplex master socket
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: channel 2: new
[/var/lib/backuppc/.ssh/sockets/stradella]
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: forking to
background
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: Entering
interactive session.
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1: pledge: id
2019-04-24 13:42:17 Output from DumpPreUserCmd: debug1:
client_input_global_request: rtype hostkeys...@openssh.com want_reply
0

Does anyone have any ideas on why nothing else happens? I would expect
a backup to start ("incr backup started for directory ...")

It should be noted that at this point I can use rsync to list the
files in the users module while logged into HOST:

rsync -n rsync://backuppc@localhost:8874/users

Thanks!


Are you running this from a script, or executing it directly?  (I 
haven't tried it, but it seems plausible that forking ssh to the 
background without a shell might cause DumpPreUserCmd not to complete, 
which would explain the symptoms.  This would be easy to verify.)


However, this seems a lot more complex than it needs to be.  Why not 
just use rsync (instead of rsyncd) which already sets up an ad hoc ssh 
tunnel?
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on another host

2019-04-19 Thread Michael Stowe

On 2019-04-19 06:11, Mathew Perry wrote:

one step further. I now can access the web interface partly.
difficult to describe. everything is top down scrollable, over 3 pages
downwards. no css or so formating.

-

FROM: Michael Stowe 
SENT: Friday, April 19, 2019 5:25 AM
TO: General list for user discussion, questions and support
CC: Mathew Perry
SUBJECT: Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is
on another host

On 2019-04-19 02:41, Mathew Perry wrote:


…and just for the record switched backend to https at the very
moment:

Outcome: The Requested URL /BackupPC_Admin was not found on this

server

-

FROM: Mathew Perry  SENT: Friday, April 19,
2019 1:25 AM TO: General list for user discussion, questions and
support SUBJECT: Re: [BackupPC-users] Fwd: Re: backuppc behind nginx
which is on another host

404 NOT FOUND

I know you're trying to help, which is very nice, but be advised
that

over this nginx server about 20 services are running and working.
backends behind nginx are both http or https. As for example chrome
is using HSTS, the domain runs https for a long time.

I don't think that i need to change a whole working and running
nginx

conf on higher level than on the backuppc directive.

-

FROM: Michael Stowe  SENT: Thursday,
April 18, 2019 7:39 AM TO: General list for user discussion,
questions and support CC: Mathew Perry SUBJECT: Re: [BackupPC-users]
Fwd: Re: backuppc behind nginx which is on another host

On 2019-04-18 01:40, Mathew Perry wrote:


hmmm… no!

The Requested URL /BackupPC_Admin was not found on this server

FYI: the url which “comes back” from nginx looks like:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7T8nrzTGGnwx9UNS1WVjQxnVSBGmhdBbJoi5j5f7UDEJ_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CtHqx5ajFKrIeEAMj67C-2FkYZTfB4c6svpzlmD5aShbi8PRTCt9qZsUXkwXQk48NyHdZiYE5NgidiQjbuMjy-2FMSxNopfjI6SR6MgSrVA8ZTmmsCBE-2FHu4CyKwJUXvjLeATdd05cYm-2Fe4Ui49nzO9OQHSHaNdNmWqNdNu-2Ftv4jhlh813W0Lg0ntRiFnigIfw6bs
 [1] [1]

nginx testet internally (LAN) and externally. Same result


And what happens when you navigate to
https://u2182357.ct.sendgrid.net/wf/click?upn=gHC-2FQ4aFf0eQjtLJSQQ-2BdZtT-2B1GrVoFn98d8hTvP5qh8-2FeTY2bBOm6Nhq8PRhzM1JS12fyMe1gWKiZkCKxHdYg-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CtHqx5ajFKrIeEAMj67C-2FkYzYtahxwXHsQCgpCInZBJ8OmEWNAriW0H3iFXN4DKYg0-2Bkz3AWdChNzU5qjNgYnWjcdHqfPJGzE67P3IYbavJ-2BbLPQtyWo1rgLFGdt-2FuvWFTRu1rzLHaem0-2F5YdXFb97EjYJKbeCGSSljpXj-2FsN712g-2BHSKbaUYqajfqTrRqrbr
 [2] [1]

Also, have you actually set up the back end for https?


To clarify: I don't think it's an http/https problem, but it's worth
ensuring that you're matching what works on the host side.

At any rate, have you tried:

location /BackupPC_Admin {

proxy_pass 
https://u2182357.ct.sendgrid.net/wf/click?upn=OIcehkuoae0CzVKCEzanymOLCu-2BVSO75-2FnYwJXg-2FV4e-2B28ucf3rSWzrrbMJduJe-2F_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CtHqx5ajFKrIeEAMj67C-2FkbdmzmsxJIXCKDZPIEFsi7XhcRK1qs7-2FyaMSbiB8OB7nwDVKL5zYN8CarktZhKqVmGwVlAd557-2FpdTHnpHHHY2mkajjeLDkq5vsWQTCKI0DVz6Lai1kpbvN-2BFn1UinWGMDedawwF26pgxcKN9jVapOIPlIuiguI4vmhiAfOSTsXY
 [3]; }


That's good -- so we know that proxying itself is working, but there's 
something goofy going on in the redirect/rewrite.  (Yes, it sounds like 
where we started.)


At any rate, two things worth bringing up:

1) Have you tried scgi_pass instead?  It removes one step between nginx 
and BackupPC.  I don't know that it will solve your problem, but at 
least it's worth being aware that you don't actually need an http server 
on the back end.


2) I don't know why the default proxy_redirect wouldn't be behaving 
itself, but it clearly isn't.  You may just want to make this explicit:


proxy_redirect off
proxy_redirect default
proxy_redirect 
https://u2182357.ct.sendgrid.net/wf/click?upn=YahQ8tVjTExAt0gOS-2Bui2HSH9NT15auMwLJik22QXzw-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CtHqx5ajFKrIeEAMj67C-2FkUG7NmDKMSEU7PilrxT3OH65Vshj3AhW5ePICO-2FDzpfkLPaDmNLNDkhqAW3HrUEQl7JrbLx02LOU1Tu01GGzWqUimqovmDR4VRCIoqHBotal-2FozNYvwNekmtSa8a5aly25VOr4dlXIwqNLp20MzngzOFsn89o9XLqVCOhM0ajfWy
 
https://u2182357.ct.sendgrid.net/wf/click?upn=qCUd6WcJ5OOirVHNEJOdx3XSC1QOgTCDRYnVJypQwOOMQMWSeLTfhLAVAPhpR1-2Fj_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CtHqx5ajFKrIeEAMj67C-2FkRNTVIOldWA4lG4lSUjuccGDrEgvZ-2F9pCQFtQNDubBQLcGsfBRVjfsXb5R-2F3gmju7yJGutjlKh95SN0TSeIDmA3aOrAWfRoGXeaJaOUlP7XoJ93RPwRDnsWw8QBeUwTwQODlnKiLPuZKbWyG4P2KpqSO5ihDdxYs9w1qVT487BT-2B

(Order is important, of course, and the point of off and default is to 
rule out global rules which are taking precedence.)
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http

Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on another host

2019-04-19 Thread Michael Stowe

On 2019-04-19 02:41, Mathew Perry wrote:

...and just for the record switched backend to https at the very
moment:

 Outcome: The Requested URL /BackupPC_Admin was not found on this
server

-

FROM: Mathew Perry 
SENT: Friday, April 19, 2019 1:25 AM
TO: General list for user discussion, questions and support
SUBJECT: Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is
on another host

404 NOT FOUND

 I know you're trying to help, which is very nice, but be advised that
over this nginx server about 20 services are running and working.
backends behind nginx are both http or https. As for example chrome is
using HSTS, the domain runs https for a long time.
 I don't think that i need to change a whole working and running nginx
conf on higher level than on the backuppc directive.

-

FROM: Michael Stowe 
SENT: Thursday, April 18, 2019 7:39 AM
TO: General list for user discussion, questions and support
CC: Mathew Perry
SUBJECT: Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is
on another host

On 2019-04-18 01:40, Mathew Perry wrote:


hmmm… no!

The Requested URL /BackupPC_Admin was not found on this server

FYI: the url which "comes back" from nginx looks like:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7T8nrzTGGnwx9UNS1WVjQxnVSBGmhdBbJoi5j5f7UDEJ_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CKfGiJvA1PcFpefW3bqr-2BdpF9WaxP-2Fz-2BKPwGTfcmoyofFnLfBLZ5VYphw53qYzl0gbeZuORygwmWGwDvpDlQSdNawmVkYGCuTF5dwS7DRvdIANbkAxXStctxkvXPATRktU8vxmC-2F035IClcRSAjrBOao8FWhvoG1A-2BD3zlepaz3s-3D
 [1]

nginx testet internally (LAN) and externally. Same result


And what happens when you navigate to
https://u2182357.ct.sendgrid.net/wf/click?upn=gHC-2FQ4aFf0eQjtLJSQQ-2BdZtT-2B1GrVoFn98d8hTvP5qh8-2FeTY2bBOm6Nhq8PRhzM1JS12fyMe1gWKiZkCKxHdYg-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CKfGiJvA1PcFpefW3bqr-2BdoC27FIaHtLukBYdeH0fOfH4TejVaEfG-2BLhAjihRo-2BphmoXvTgTJ2HAfQDEHhhua-2Bs747R1cpnyuL5ipTte1Xs8ElR6m8Xob1ayOwPQ0CyL6Vpn92Ag-2BVIFfmJchYsL3fvHVTc7Gt5ZEld7a8Mk1sCg-3D
 [1]

Also, have you actually set up the back end for https?


To clarify:  I don't think it's an http/https problem, but it's worth 
ensuring that you're matching what works on the host side.


At any rate, have you tried:

location /BackupPC_Admin {

proxy_pass 
https://u2182357.ct.sendgrid.net/wf/click?upn=OIcehkuoae0CzVKCEzanymOLCu-2BVSO75-2FnYwJXg-2FV4ediqxr1VtC8L6pBcvPM92z_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CKfGiJvA1PcFpefW3bqr-2BdsPURAPOK4uZ9PbBpJOoIHNp8szRytp4rI3pjnVfTyObKR7xW3CAaBmQyTbZwK32fG2Gp72-2BGUzk3-2BQ-2FY4Um3NjWEVhVV64Jmj-2Bz4dk12QrNfzY04GEUEYfCvwm9h7oHwwxvMvsOikkLeIgMIK62bY0-3D
}
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on another host

2019-04-19 Thread Michael Stowe

On 2019-04-19 01:25, Mathew Perry wrote:

404 NOT FOUND

 I know you're trying to help, which is very nice, but be advised that
over this nginx server about 20 services are running and working.
backends behind nginx are both http or https. As for example chrome is
using HSTS, the domain runs https for a long time.
 I don't think that i need to change a whole working and running nginx
conf on higher level than on the backuppc directive.

-

FROM: Michael Stowe 
SENT: Thursday, April 18, 2019 7:39 AM
TO: General list for user discussion, questions and support
CC: Mathew Perry
SUBJECT: Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is
on another host

On 2019-04-18 01:40, Mathew Perry wrote:


hmmm… no!

The Requested URL /BackupPC_Admin was not found on this server

FYI: the url which "comes back" from nginx looks like:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7T8nrzTGGnwx9UNS1WVjQxnVSBGmhdBbJoi5j5f7UDEJ_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CDUbOmcsxmenBQMf51gjd4USN4gIiwaoql9mVcjNIqzj8hcBHEQlyR24336jD7LcPFz2GieWLMWjSoXdgIng8hqnkLXJDrudqTHv9huJoOsXtq-2FhKEuSZvqLfmc50I-2BNuaWquWpEoMZg48Fet0BHpkDN2ZIjxrjQwSO4G0QshyPo-3D
 [1]

nginx testet internally (LAN) and externally. Same result


And what happens when you navigate to
https://u2182357.ct.sendgrid.net/wf/click?upn=gHC-2FQ4aFf0eQjtLJSQQ-2BdZtT-2B1GrVoFn98d8hTvP5qh8-2FeTY2bBOm6Nhq8PRhzM1JS12fyMe1gWKiZkCKxHdYg-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CDUbOmcsxmenBQMf51gjd4TVyg-2BYdpw8kJcU9segSE0bMAvnuZ2XwltlPOPdk-2FfZu2sCjOTINw6bVpbqEzq2lDun9uFxeRfRGUJ-2FxQmbwh-2BxvGHlIjJcyE1A8lZdC-2B0hRCpyc0Ao4J5pNVwL5Qv12izJLNUCckNo7tHWcXwAOC0s-3D
 [1]

Also, have you actually set up the back end for https?


I'm not suggesting you change the working of nginx, I'm suggesting you 
work through the symptoms to locate what's incompatible with your setup.


Thus, this question (for which I haven't seen an answer):

https://u2182357.ct.sendgrid.net/wf/click?upn=gHC-2FQ4aFf0eQjtLJSQQ-2BdZtT-2B1GrVoFn98d8hTvP5qh8-2FeTY2bBOm6Nhq8PRhzM1CYbZiJw-2BhyFQbv2HpLjVSQ-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CDUbOmcsxmenBQMf51gjd4XCUdNUvmSlOnuuXGa0pZHUnNjtL8IS1y2MtAlfH8qCvrZaXmNScjTxbuXfWRA1ba-2BInmooiAhkrtUa-2BfivcNooCsoQVs1WiOPRCxOGBgaHhd1NYribvpT84KhDeS9n-2FsTVMCoxei783GTq56Kxml04-3D

Call it from http or https as appropriate, sure.  I'm not impugning your 
decisions, I'm trying to understand them so that I can offer more than 
broad advice.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on another host

2019-04-18 Thread Michael Stowe

On 2019-04-18 01:40, Mathew Perry wrote:

hmmm... no!

 The Requested URL /BackupPC_Admin was not found on this server

 FYI: the url which "comes back" from nginx looks like:

 
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7T8nrzTGGnwx9UNS1WVjQxnVSBGmhdBbJoi5j5f7UDEJ_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CxFeNcoCh25-2Fubo8rY9bYNs3Ku7bT-2BL9gcEFJWKWFiFj-2FhL7Fzg3rWmP-2Fk5Uy1RNQ89pK4L-2Fu2H-2F-2B7NxpdnZ0l0Dub299FjAYAOx6GUR2LllNFGgvOqt9L27Qt1VXZCZGnvbjETQXz-2F8xmQPTEUt-2FU74FU9JC-2Bn0gbduBJh22nTM-3D
 [1]

 nginx testet internally (LAN) and externally. Same result


And what happens when you navigate to 
https://u2182357.ct.sendgrid.net/wf/click?upn=gHC-2FQ4aFf0eQjtLJSQQ-2BdZtT-2B1GrVoFn98d8hTvP5qh8-2FeTY2bBOm6Nhq8PRhzM1JS12fyMe1gWKiZkCKxHdYg-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CxFeNcoCh25-2Fubo8rY9bYNoD1xiEbyugVWedRTENls6PrQovtqBJcmIRDketR-2FO2dL3iezRi1me9fjiJvSVkJkoEXFd2Ps2DKvOTVOSCzJJe1nPR-2BmP3ODdOoCtYmoVEM-2F-2BqK0-2FMk0LXUMBvRUTg6j5RN1vmmSawurlmdXej8e-2BQ-3D


Also, have you actually set up the back end for https?
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on another host

2019-04-18 Thread Michael Stowe

On 2019-04-17 21:27, Mathew Perry wrote:

you explained it exactly. But i've posted my directive already. If i
would know what to do know with my nginx config, i wouldn't ask here.



Ah.  Let's fix it.

This should suffice:

location /backuppc/ {
  proxy_pass 
https://u2182357.ct.sendgrid.net/wf/click?upn=OIcehkuoae0CzVKCEzanyrAnO8bh02MVVnzKtEAq6iQ-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91COiLZEGT-2Bli7V3p-2F2jKE-2FjCkVx6E-2BPxPP4oLsyhDPgbYUVy64Fki9hMO6LYlb3CArUXWLH2Qrs-2FCVFaE-2F7TH7-2FLNAbyUrWh-2FtGBC4Rf2QmHNKYixVy-2FrOIvuFuIhfT79l35Sq085ChbOvMyc1oRLbsU7y6saFfiW1J4VeOYE7qtM-3D
  }

And you posted:

location /backuppc/ {
  proxy_set_header X-Real-IP $remote_addr;
  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
  proxy_set_header X-Forwarded-Proto https;
  proxy_set_header Host $http_host;
  proxy_max_temp_file_size 0;
  proxy_pass 
https://u2182357.ct.sendgrid.net/wf/click?upn=OIcehkuoae0CzVKCEzanyrAnO8bh02MVVnzKtEAq6iQ-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91COiLZEGT-2Bli7V3p-2F2jKE-2FjDig91GlK9N0KqgtqdyVo8zz9eXBRFvw84nUj8xGVBZCN1vG0ChpN9Rr7pEoEe6kr6AdJ8G1fws-2BPC6f6GH6IJjmOYhNLieCw0yAsIPqYOM62bAESC-2B1j-2F13MOcfax2zgHNZPXl0psSZ4VTW7eg-2F7xw-3D
  proxy_redirect 
https://u2182357.ct.sendgrid.net/wf/click?upn=q9oknVuBMf-2FjuIJ06YpcAg-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91COiLZEGT-2Bli7V3p-2F2jKE-2FjFM0tYwnIO18E7YkiE9-2F7qvQOMt6QcsDSnYsHfl0QPP8jyCRf6987sm6GimwV36WL23xP5Hr8FF9uPBdNRbC4umXMjiGE-2FbFVeIGOB6Ec9aa8iT-2FNKZLtQgC2CAQnQ8RqXaSHJ1luc-2BHUXuvOME83rs-3D
 
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7SmCwE3HeriTlvqQE-2FYLnDc-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91COiLZEGT-2Bli7V3p-2F2jKE-2FjKnM-2F9M6G4WPOJN7rHvzf04JkR0YG9osUFHUrD5p-2Bntv6sRRLFah1hs-2BYU7itjIVT2TccezF5XNasBs-2F84xRJTB-2F17l3qB8IJ6v8gvTN8IwN5YkJiL573t0Vjia6kXWyLdfeSiq3XSFg9uho11tntqI-3D

  proxy_redirect off;
  }

The headers are superfluous and perhaps innocuous, but where the 
breakage occurs here is in the proxy_redirect directives, where the 
first one breaks the default redirect by telling nginx that instead of 
properly replacing the back end server's url with the front end server's 
url, you'd like to force the front end to use https on the wrong url.  
The second proxy_redirect directive turns that off, so that instead of 
properly replacing the back end server's url with the front end server's 
url, you'd like to just use the wrong url.  I'd recommend removing all 
this nonsense.



leads to:
"The Requested URL /BackupPC_Admin was not found on this server"


Yes, you've specifically told it not to redirect to the proper server


i wouldn't ask here if only a proxy pass is needed


That is literally all you need, although it's possible to break the 
proxy pass with additional directives.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on another host

2019-04-16 Thread Michael Stowe

On 2019-04-16 01:23, Mathew Perry wrote:

Hi Michael

 the example was from local LAN. backuppc is running and working and
nginx runs with a lot other services behind it.
 I'm not that new to nginx, more a "new" user to backuppc.
 normally the directive /pathto/ needs a change in the base url of
some web services behind nginx. This "could" help sometimes.

 I do not know where to change the base url in the backuppc server

 Regards

 Matt


Hang on -- if the local UI doesn't work, it won't work through a reverse 
proxy.  So if BackupPC is on Server "A" and nginx is on Server "B", 
Server "A" really needs to be serving up the UI first.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on another host

2019-04-15 Thread Michael Stowe

On 2019-04-15 12:13, Mathew Perry wrote:

it's not that easy 


Did you ensure that it was working locally first?


 with this directive:

 location /backuppc/ {

  proxy_set_header X-Real-IP $remote_addr;

  proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

  proxy_set_header X-Forwarded-Proto https;

  proxy_set_header Host $http_host;

  proxy_max_temp_file_size 0;

  proxy_pass 
https://u2182357.ct.sendgrid.net/wf/click?upn=OIcehkuoae0CzVKCEzanyrAnO8bh02MVVnzKtEAq6iQ-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91ClCxQ1Qsn89oEtKqFdiYBtl93CUqCAPaeb2zgXoldnEbTgpCzAUVKZxXb-2B4JzRy1cK1m0NXJBgFz5JQ-2FM0LzSw1eqHjeHCpf42fU-2FTxXB2fFB16f4P-2F-2FkcjwBabI1woU40tZq8bWR4TZXTRGzhMSNJXoH-2Ffrt-2B1LtI-2Bh2OQKz0Uc-3D

  proxy_redirect 
https://u2182357.ct.sendgrid.net/wf/click?upn=q9oknVuBMf-2FjuIJ06YpcAg-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91ClCxQ1Qsn89oEtKqFdiYBtt2iAyFgLJ6VjTyV3i13b8yYL9INm7q2JrvP8lpPl8pq8izVHGLF5UBqDxf7VWnjbD9ThQ7XSg8CpkGNLRbXYpYhMaPeU66DdpxdmhAyOWWhs5LLg2dgczwUg3G0ClLiNhkSgWxrBUUbJfFrd0Z4ILw-3D
 
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7SmCwE3HeriTlvqQE-2FYLnDc-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91ClCxQ1Qsn89oEtKqFdiYBthz3Oq3LUsj9ziTBIFHGqG8g-2BLR2rvBx-2BNIhUPDGo72MjNFqZURIBLNO1JEQc7OI3ZDH1jCV9bBryhqms7A8RXyhjKz0dGsKHx3Vg-2FYFOBcfscujazm4UQG1HjDAQdatSgMryiwPMRc9HnMz0Flmafc-3D

  proxy_redirect off;
   }

 leads to:
 "The Requested URL /BackupPC_Admin was not found on this server"

 i wouldn't ask here if only a proxy pass is needed

-

FROM: Johan Ehnberg 
SENT: Monday, April 15, 2019 10:08 AM
TO: backuppc-users@lists.sourceforge.net
SUBJECT: [BackupPC-users] Fwd: Re: backuppc behind nginx which is on
another host

I accidentally sent my reply just to Matthew. Here is a copy for
future reference.

 Forwarded Message 

SUBJECT:
Re: [BackupPC-users] backuppc behind nginx which is on another 
host

DATE:
Mon, 15 Apr 2019 12:25:51 +0300

FROM:
Johan Ehnberg 

TO:
Mathew Perry 

Hi Matthew,

I would recommend using the nginx server as a reverse proxy.

So, set up the backuppc server normally (i.e. with its own web
server), and on nginx, use something like the following if you will
connect remotely over TLS and Let's Encrypt certificate:

server {
  listen 80;
  server_name backuppc.example.com;
  return 301 
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7TC34CkM-2FTKiyIQzRymkOx1xdqYC5E3Bp3F7k5LtuoC1_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91ClCxQ1Qsn89oEtKqFdiYBtoa1ObICDPceySbFC-2F3ih5oq3hg-2FkhYIFcNLmvtIBTmGJJe-2F964lNoHAGDnQds-2Fxmlmr9hfr8PAnq56dnLpCgZL5RBs5scEWSsAWx7xIQObJ5hyDJUkTJjeuRTt1isxmmyWe7fiR7n4YUy3CEuFPPHU-3D
 [1];
}

server {
  listen 443 ssl http2;
  server_name backuppc.example.com;
  ssl_prefer_server_ciphers on;
  ssl_ciphers
EECDH+CHACHA20:EECDH+AES128:RSA+AES128:EECDH+AES256:RSA+AES256:EECDH+3DES:RSA+3DES:!MD5;
  ssl_certificate
/etc/letsencrypt/live/backuppc.example.com/fullchain.pem;
  ssl_certificate_key
/etc/letsencrypt/live/backuppc.example.com/privkey.pem;
  ssl_session_cache shared:SSL:5m;
  ssl_session_timeout 1h;
  add_header Strict-Transport-Security max-age=15768000 always;
  location / {
include /etc/nginx/proxy_params;
proxy_pass 
https://u2182357.ct.sendgrid.net/wf/click?upn=phT9o-2FIE-2FsNYgBy7FoncRJHlffvAv5mRzpSKVBf1eks-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91ClCxQ1Qsn89oEtKqFdiYBtvnoP96JWsBx1dwm70lKh1zzUcqxeAQNaLSoPvQ9dfIrrZnNcRr05ee0t-2FR3FdRd8lxdgqUdATk9ipwA8i7gPy7-2FUUrBB-2BZSCAYtk1f9Dv-2BEHGLuJs4Gjckc0Ep65-2BGe-2B1LT8mFDiq171-2BtGkJjqLJg-3D
  }
}

Change domains and IP to taste.

Best regards,

Johan Ehnberg

On 4/15/19 11:19 AM, Mathew Perry wrote:


Hi

i'm using latest backuppc and want to connect emotely to it. I have
nginx on another host, not on same host where backuppc is running
on.

So, i'm struggling to get the nginx config working to connect to
the backuppc host. The configs on the internet are all for nginx
configs if backuppc and nginx are on the same host.


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] backuppc behind nginx which is on another host

2019-04-15 Thread Michael Stowe

On 2019-04-15 01:19, Mathew Perry wrote:

Hi

 i'm using latest backuppc and want to connect emotely to it. I have
nginx on another host, not on same host where backuppc is running on.

 So, i'm struggling to get the nginx config working to connect to the
backuppc host. The configs on the internet are all for nginx configs
if backuppc and nginx are on the same host.


In the scenario you describe, BackupPC is not particularly relevant.  
This is what you're looking for:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7Qbq8wyTVtPdoL6cG1NoY5QSxro8-2BGDy3hYNwXdYZWDRo-2BJs-2F4ma49gq77H68OBQg26JWHq5oY7rWfRC0kzPudk-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CUtUIap9o-2F-2FLY8mJujHuhGHmdt7vMXOLqsr5JZevMfKN-2Fud-2BhwIAlPNbR1T0dysWh5kZbDo7XI80cVKBlkIVGFKeD4J5Vdo0v8CjCkdfM67Gk8TsPzIjoJ-2BL7MwpI6zVLY-2Fvyvq6lq-2FJn-2FjnBV9tkkoCn-2FFNNjDNRubEmMkQnUOaHAGGWaCAYQ24422bNijYa

You'll still need BackupPC's web interface working on its own server, on 
any port, and you can lock it down to the reverse proxy server (how one 
accomplishes this differs depending on how one configures the web 
interface.)
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Xfer errors (especially bpc_sysCall_checkFileMatch(/path/to/file): file doesn't exist)

2019-04-05 Thread Michael Stowe

On 2019-04-05 01:11, Daniel Berteaud wrote:

Thanks for your response Michael, some comments inline

Le 2019-04-04 23:22, Michael Stowe a écrit :


On 2019-04-03 05:02, Daniel Berteaud wrote:

First, as soon as there's a single error (most common are "file has
vanished"), BackupPC reports 3 errors.

Often, the backup reports an error count of 2, with these kind of
errors :

[ 213 lignes sautées ]
rsync_bpc: stat "/.config/google-chrome/CertificateRevocation/5022"
failed: No such file or directory (2)
rsync_bpc: stat
"/.config/google-chrome/CertificateRevocation/5022/_metadata"
failed:
No such file or directory (2)
rsync_bpc: stat "/.config/google-chrome/CertificateRevocation/5023"
failed: No such file or directory (2)
rsync_bpc: stat
"/.config/google-chrome/CertificateRevocation/5023/_metadata"
failed:
No such file or directory (2)


These files seem likely to have been updated or removed during the
backup run. That directory contains bookmarks, cookies, extensions,
and all kinds of things one might expect to change.

My log snippets are just examples. I know this directory contains
moving parts, I just try to understand why I do get this error while
the mentionned dirs exist both on the source, and in the backup. To me
this makes no sense. Either the dir doesn't exist during the scan, and
rsync won't try to transfert it, or the dir exists and rsync transfert
it, or the dir exists during the scan, but disapears before rsync
transfert it. In this last case, I expect this kind of errors in the
logs, but the directory should not exists in the backup.


There are other possibilities:  the directory is moved, or files are 
deleted and recreated (the latter is likely the case for Chrome's 
transient files.)


Do you have any examples of files or directories which are not volatile?


The symptoms here and the directory in question matches the behavior
that is expected when backing up a Samba share used by Windows
systems with access to a recycle bin. Windows will rename a file
into the recycle bin, so you'll see this when you back up the
original file -and- the recycle bin. I can't imagine why one would
normally want to back up a recycle bin, since presumably one would
have already backed up the file before it was deleted, so it's
probably simplest just to exclude the bin from the backups.
Alternatively, one can remove the bin (or access to it) so that
Windows clients delete the original files, though that also makes
backup of the recycle bin somewhat pointless.


This was also just the first occurence of the error I found in my
logs, I have the same errors for other files. Besides that, in this
case, from BackupPC's POV, it doesn't matter if the directory is used
a Recycle Bin for samba. It's just a directory where files which were
previously elswhere have been moved. It could have been done by a
human intervention in a FooBar directory.

I'm just trying to understand what the error means


You're correct; the error and BackupPC are telling you that the files 
moved during the backup.  Perhaps I'm not understanding -- if the file 
has moved, and the error reflects that fact ... what are you trying to 
understand?
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Xfer errors (especially bpc_sysCall_checkFileMatch(/path/to/file): file doesn't exist)

2019-04-04 Thread Michael Stowe

On 2019-04-03 05:02, Daniel Berteaud wrote:

Am I the only one having this kind of errors, or the only one caring
about the errors ?


Cheers

Daniel

Le 12/03/2019 à 08:56, Daniel Berteaud a écrit :


Hi.

I've been using BackupPC since the past 13 years (starting with 2.1.2
I think), with great success. I'm now running a few installations with
v4, and one of the things I don't understand is the Xfer errors. I
regularily check Xfer error to be sure my backups are OK. Before v3, I
could reliabily check the number of Xfer errors, but in v4, I don't
understand.

First, as soon as there's a single error (most common are "file has
vanished"), BackupPC reports 3 errors.

Often, the backup reports an error count of 2, with these kind of 
errors :


[ 213 lignes sautées ]
rsync_bpc: stat "/.config/google-chrome/CertificateRevocation/5022"
failed: No such file or directory (2)
rsync_bpc: stat
"/.config/google-chrome/CertificateRevocation/5022/_metadata" failed:
No such file or directory (2)
rsync_bpc: stat "/.config/google-chrome/CertificateRevocation/5023"
failed: No such file or directory (2)
rsync_bpc: stat
"/.config/google-chrome/CertificateRevocation/5023/_metadata" failed:
No such file or directory (2)


These files seem likely to have been updated or removed during the 
backup run.  That directory contains bookmarks, cookies, extensions, and 
all kinds of things one might expect to change.



[ 106 lignes sautées ]

[...]

rsync error: some files/attrs were not transferred (see previous
errors) (code 23) at main.c(1676) [generator=3.1.2.0]
rsync_bpc exited with benign status 23 (5888)


This essentially summarizes that files weren't found during the rsync 
transfer that were identified before the transfer, which matches the 
symptom above, which is expected when trying to back up transient files.



This is quite confusing because, in this example, the directories does
exist, both on the source, and in the backup accounting these errors.


For files which are frequently deleted and recreated (such as cache 
files) this is expected behavior.



So, what does this error mean ? It looks like it happens when those
directories did not exist in the previous backup (the one aginst which
rsync is transfering deltas), but, why would this be an error ?


rsync doesn't know why files would disappear or be created during a run 
-- as has been pointed out elsewhere, you may want to quiesce chrome, or 
(more sensibly) not try to back up transient files.



Last, and the most important to me, I have some hosts with hundreds of
Xfer errors like these :


[ 53777 lignes sautées ]
R bpc_sysCall_checkFileMatch(shares/assistantedirection/files/Recycle
Bin/BIBLIOTHEQUE/DOCUMENTATION/AMIANTE/Courrier FEDEREC-CRAMIF-Demande
de conventionnement.pdf): file doesn't exist
R bpc_sysCall_checkFileMatch(shares/assistantedirection/files/Recycle
Bin/BIBLIOTHEQUE/DOCUMENTATION/BTP/Courrier FEDEREC-CRAMIF-Demande de
conventionnement.pdf): file doesn't exist
R bpc_sysCall_checkFileMatch(shares/assistantedirection/files/Recycle
Bin/DOSSIERS COMMUNS GROUPE PENA/ACHATS/DEMANDES D'ACHAT/2019/2 FEV
19/110319CB - GESTECO Château de France.docx): file doesn't exist
[ 161 lignes sautées ]

Those errors are not accounted in the Xfer error count, but displayed
if we open the XferErr log. My C foo is not good enough to understand
exactly what's going on in
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7dCyrjIWd3LVbeqZFg4ygUGvSfi41qKNj5ycIIg4GkJF3ZYo8-2Fl9JPHIyP-2FT2T5NlSfZvMVW9z8xIQ8r-2FhomH40-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CBnNjwF56OFtfIV34pOjuQRpX-2B3dRh-2FjPvm7nBC7QxnAw7MCkPTmIk5XYe1BLmUKuA6iTm8N0OT4B0OgafrFblGeYsQ8kd01o5RmDhP2n6-2B8W41NzlSMQcHJX7hUnGeIeAdm5W7NTHAd2gXpp7P0Ej3e-2BOUWZ1nTVQpIsrBRs8alPGkONXI1q-2Be9bHyoBqXWW

Looks like this error can be logged when a file is moved (so it exists
in the pool, but not at the same path in the previous backup agsint
which we delta ?)

Can someone confirm this is harmless ? I so, wouldn't it make sens to
remove this from the XferErr (or maybe just include it if XferLogLevel
> 3 or something like that ?)


The symptoms here and the directory in question matches the behavior 
that is expected when backing up a Samba share used by Windows systems 
with access to a recycle bin.  Windows will rename a file into the 
recycle bin, so you'll see this when you back up the original file -and- 
the recycle bin.  I can't imagine why one would normally want to back up 
a recycle bin, since presumably one would have already backed up the 
file before it was deleted, so it's probably simplest just to exclude 
the bin from the backups.  Alternatively, one can remove the bin (or 
access to it) so that Windows clients delete the original files, though 
that also makes backup of the recycle bin somewhat pointless.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:

Re: [BackupPC-users] What's the latest on migrating V3 to V4To:

2019-02-13 Thread Michael Stowe

On 2019-02-13 05:57, backu...@kosowsky.org wrote:

Looking to finally move to V4.

Would love to eat my cake-and-have-it-too by eliminating all those
nasty linked pool files while preserving all my old V3 backups.

In the past I have done quite a bit of utility hacking into the V3
pool and tree structures so I am aware of the complexities at least in
V3 but I have no experience or knowledge of how trees and metadata are
stored in V4.

Has anyone written a clean migration utility that copies all backups
over (not just the latest)?

In a sense I have written half of it in the past in my
BackupPC_copyPcPool.pl.jjk that allows one to copy over BackupPC V3
archives by creating a text file download of the backup trees and then
a simple copy of the pool.

But I am not sure I have the time to write a new utility that would
dump this back into V4 format while converting the pool structure to
md5sum titling and organization.


Would you not just use this?

usage: BackupPC_migrateV3toV4 -a [-m] [-p] [-v]
   BackupPC_migrateV3toV4 -h host [-n V3backupNum] [-m] [-p] [-v]
  Options:
 -a  migrate all hosts and all backups
 -h host migrate just a specific host
 -n V3backupNum  migrate specific host backup; does all V3 backups
 for that host if not specified
 -m  don't migrate anything; just print what would be 
done

 -p  don't print progress information
 -v  verbose
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Global vs. local exclude

2019-01-17 Thread Michael Stowe

On 2019-01-17 09:37, Jan Stransky wrote:

Hi,
sorry for rising another exclude/include thread, but...
Does globally defined exclusion rules apply also when there is
host-specific one?
For example:
in configure.pl, for share '*' I have excluded '*.tmp'
in host.pl I have share '/home' which has excluded '/*/.ssh'
However, the exclusion of the *.tmp does not apply. Feature, or bug?
Cheers,
Jan


Short answer:  no, it's a complete override

Longer answer:  it's possible to put together some perl to make this 
work, but it does not do so out-of-the box.  I suspect the reason would 
be the baffling experience of trying to figure out where combined sets 
of exclusions are coming from, or because this is how every other 
setting works, it would be a bit weird to diverge for one or two 
variables.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Configuration for Win10 clients

2019-01-17 Thread Michael Stowe

On 2019-01-17 02:45, Mike Bosschaert wrote:

Hi,

I'm struggling with the configuration of a Win10 client. I'm running
4.2.1 and have several linux
clients which have backups daily for years now without any problem,
with various different
schedules of incremental backups.

For some time I'm also backing up a win10 laptop and I found out that
in certain circumstances
backups, which should have a volume of about 900G are nearly empty.
The entire directory structure
is built correctly but most directories are empty, whereas if you look
in the history, they
contain some backupped files, sometimes as old as a couple of months
(indicated with v0). I do not
see errormessages in the logs. The backups are mostly made during
nightly hours so it's unlikely
that there are massive file-locks blocking the backups. The backups
are from one smb share and all
share the same owner.


File permissions on W10 are a frothy mix of local permissions and 
SMB/CIFS permissions.  If files are simply not visible in a full backup, 
these are the first things to check.  smbclient is a good diagnostic 
tool for poking around and ensuring that you actually can remotely 
access files you should be able to access.



I've done some tests with different scenarios. It appears that if a
directory is copied to the
smb-share, in the incremental backups, only files are backed up with a
datetime later than the
reference file. The files which are newly copied upon the share but
have a date before the
reference date are ignored. These files are only backed up in full 
backups.


This is expected behavior of smb.  You'll need to switch to rsync if you 
want rsync-like behavior.



My question is, is there a way to tell smbclient which runs an
incremental backup ( tarmode\full
-TcN) to also backup files which are NOT already present in the delta
of the directory. So the
backup would copy files which are newer than the reference AND files
which are older but not yet
backed up in the past.


Short answer:  no

Longer answer:  your choices are either to switch to rsync or to run a 
lot more full backups.  On the plus side, there are at least two 
well-known setups that implement rsync and shadow files, which would 
take care of any file locking issues.  (These would show up in the 
logs.)



Thx
Mike
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrade from 3 to 4 or start afresh

2019-01-15 Thread Michael Stowe

On 2019-01-15 05:34, Paul Littlefield wrote:

Hello,

I look after a 3.3.1-2ubuntu3.3 installation and now all clients do
not backup and have tar_process done, err = 0.

I have been told that version 4 of BackupPC will fix the errors.


I would not expect this, but it's *possible* if you have, say, upgraded 
smbclient and use samba for backups.  You can also downgrade smbclient 
if this is the case.  You can validate this by executing the backup 
command line and reviewing the results.



Should I try to upgrade the Ubuntu v3 apt-get installation or start
afresh with a TAR install of v4?


No, I don't recommend upgrading without at least a rudimentary diagnosis 
of the problem.



There are 6 clients running a mix of Windows 7, Windows 10 and all
using Microsoft Outlook.


Samba can't back up Outlook unless it's shut down, obviously.  Is this 
what you're trying to back up?  What method are you using?



Thanks in advance.

Regards,

Paul
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Queued jobs not running automatically

2019-01-07 Thread Michael Stowe

On 2019-01-07 08:29, W Forum W wrote:

Hi,

Ẃe have more than 50 jobs in 'Background Queue Summary' but no job
is running.
Manually we can start these Jobs.

Why are the Jobs not running automatically as they are queued?

In the status overview we see only 2 Jobs running fo days now

 Host
 Type
 User
 Start Time
 Command
 PID
 Xfer PID

 admin

 5/1 01:00
 BackupPC_nightly -m 0 63
 5001

 trashClean

 4/1 12:59
 BackupPC_trashClean
 22313

Any ideas why?

Many thanks!!


When I have seen this behavior, it has usually been due to upgrading one 
or more perl modules or the perl installation, or BackupPC, while 
leaving the backuppc instance running.  Or, damage to the running 
instance (e.g., I/O failure or OOM error.)  YMMV, but restarting 
BackupPC is where I'd start.
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Performance - backing up to USB drives

2018-12-20 Thread Michael Stowe

On 2018-12-20 00:12, Gordon Marinovich wrote:

I just read about the slow speed when backing up to USB drives:

"The backuppc on usb drive is slow to write backups - about 1mb/s
which makes backups of 40-80gb take 10-12 hours"  Ref
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7YQ8BYSHbrWZS1jKQzKBLmHDvD-2BcaXOPRSW8DuNIHKGhPi3D3DpEFmm6RSN-2Bd74o3As619-2BLojesx9YjS6tiM-2BA-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CPsoq48UqGeORl7oNwBXWzTr5I9j3KZndImK3RJ97l-2B6hYs0HkxvA5yWGw0b-2B0tYmNFTvXptX0BEx9e0WiTXdhQXeAwyHOUDX3E1SI4vWOUKdBYRGEmdOTtlfxWT4FQcdMDM3oInjqKwcefqb9R4BBsuOUAWv1zF4wBCiyGQDzI3nFPVr2GucTeUSWtxSujmx

What sort of performance have others found when backing up to USB2/3?


This is more a statement to pay attention to your I/O speed than 
anything.  If you have a blazin' fast USB 3.1 array, you may not see a 
lot of difference between that and, say, SATA or iSCSI.  If you have a 
consumer-grade USB 2 "backup" drive and think it's up to the task of 
supporting a pool-based backup system for a dozen systems, you're likely 
to be sorely disappointed, and are better off using whatever software 
came with it.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] nmblookup

2018-12-13 Thread Michael Stowe

On 2018-12-13 20:22, Alan Taylor wrote:

Hello,

I am running BackupPC 4.3 for a network of Linux and MacOS clients
i.e. at present no samba servers anywhere.

Not surprisingly, nmblookup fails. I presume this is normal in this
circumstance ? If so, are there any modifications I should make to
config.pl  ?

BRgds/Alan


Why would you need nmblookup to succeed?  It's only needed for the smb 
transport ... which you're not using if you're not using Windows or 
Samba.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BackupPC writes to disk very slow

2018-12-03 Thread Michael Stowe

On 2018-12-03 04:22, Tapio Lehtonen wrote:
Out of curiosity, have you experimented with other filesystems? I have 
couple relatively large setups (pool at ~ 9 terabytes) with ext4 and 
those still crunch backups happily.




Previously I have used EXT4, only problem was once running out of
inodes. But I find it hard to believe XFS with default options would
be so much worse than any other file system with default options.


I performed a number of benchmarks, specifically with BackupPC and a 
handful of file systems.  My informal results were that XFS was slightly 
faster *unless* there was subtle corruption in the filesystem, in which 
case, the only remedy seemed to be a complete rebuild of XFS.  (The fsck 
tools were capable of detecting it, but not repairing it.)  I did not 
pursue this further, but my point is that it's probably worth forcing a 
check of XFS as well as examining your drives for low-level retries if 
your write speed is slower than you expect.




___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Weird backup commands

2018-10-30 Thread Michael Stowe

On 2018-10-30 08:04, Holger Parplies wrote:

Hi,

Jaime Fenton wrote on 2018-10-24 16:17:24 + [Re: [BackupPC-users]
Weird backup commands]:
Also, my apologies if my tone came across as snippy or rude [...] 
I???m just

frustrated with the issues I???m having [...]


well ...


[...]
If I thought samba was not going to work, then I would dump it.


Believe it or not, Samba *can* never work properly, presuming you want 
both
incremental and exact backups. As Craig pointed out, Samba *can* never 
give
you the in-/exclude options rsync does. If you're fine with all of 
that, then
go ahead. The only benefit Samba gives you is presumably the ease of 
setup,
and you're apparently not getting much out of that, because it doesn't 
"just
work". You're going to have to put in some work one way or the other. 
There

is simply no way around that.


Let me amplify this:  samba can also easily miss incremental files whose 
timestamp is not as expected, something rsync is not subject to.  
There's a set of issues surrounding open and locked files, which I won't 
attempt to cover in detail here, except to point out that it's suited to 
a use case where the files you care about are expected to be static 
during the backup period.



[...]
Out of curiosity, has anyone else had a problem with samba 4.7.1? I 
did see
one mention in a forum somewhere that there issues were resolved with 
samba

4.7.3. Thoughts?


Between 4.7.1 and 4.7.3 of Samba, there were a number of server changes 
around vulnerabilities, particularly using the SMB1 protocol.  smbclient 
(the only relevant portion) is largely unchanged.  So, my main thoughts 
are:

1) No, this isn't relevant
2) Reading the release notes is a good way to find out if a minor 
version change would be important or not


It seems highly unlikely that one Samba version changes the command 
line

BackupPC passes to it. Much more likely is an issue with your BackupPC
configuration. That is why Craig asked:


[...]
What are your settings for $Conf{SmbClientFullCmd}, 
$Conf{SmbShareName},
$Conf{BackupFilesOnly} and $Conf{BackupFilesExclude} on the two 
machines?


So, my thought is: answer the question. Without that information, the 
only

help we can give you is, "there's probably something wrong with your
configuration".

I might add that each of these settings individually might be global or
host specific. How do you manipulate the settings? Do you edit the 
config

file or use the web interface?


[...]
??? Good example: Running: /usr/bin/smbclient 
computername.domainname\\C\$ 
-U AL\\backuppc -E -d 1 -c tarmode\ full -TcN /data/backuppc/pc/ 
computername.domainname /timeStamp.level0 - Users\\username\\AppData 
Users\\ username \\Documents Users\\ username 
\\Downloads Users\\ username \\My\ 
Documents Users\\ username \\Desktop


??? Error example: Running: /usr/bin/smbclient  
computername.domainname \\C\$ -U AL\\backuppc -E -d 1 
-c tarmode\ full -Tc - Users\\\*\\AppData\\\* Users\\\*\\Documents\\\* 
Users\\\*\\Downloads\\\* Users\\\*\\Desktop\\\* Users\\\*\\My\ 
Documents\\\*


This seems to be badly mangled by both your MUA as well as yourself - 
I'm
guessing the spaces around "computername.domainname" as well as 
"username"
are probably not there (they're not escaped, so they would separate 
arguments,
basically meaning the good example command would be nonsense, even with 
the

file tags removed).
In any case, the confusion Windoze path separators introduce is a pain. 
I'm
not sure this is better with rsync, but it probably is - at least you 
*should
be* able to consistently use "/", which doesn't need escaping. Oh, and 
you
can use both in- and excludes with rsync. You can't with Samba. You 
know that,

right?

My reaction, at least, is, I'm not taking a closer look at any of that,
because it's bogus, confusing, and irrelevant. We need the 
configuration

details.

Hope that helps.

Also, my apologies if my tone came across as snippy or rude, I'm just
frustrated with questions (and followups) that prevent the possibility
of meaningful help.

Regards,
Holger


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7azDepoKsO2pWuEqaZQXziclOk1C3J1HK5ryLm0UkwauF8WGAbPQPulkw6jGlj-2BNdMKzyS4-2BY7RcC51XaueEjwE-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CEbovT0uHKS5g6KcNPIwMvz-2FNHTng-2BY-2Fax7k3U0jEfJ4khOUNpv4xYynMLaHyxmTerj3jFw-2Bui1DjreafkyZNbF13q0FCpXMO-2BkU30rYL4BFUhpQdloIJolYBx98kSKV6Si5OG65P5DVrUHgPc6LaNrpk8YuZ2xROf0KMXCdIK-2BrOIG-2FnKM7dAc2-2BcgK5YDnQ
Wiki:

Re: [BackupPC-users] backuppc and selinux context on restore

2018-10-16 Thread Michael Stowe

On 2018-10-15 23:05, Eero Volotinen wrote:

Hi All,

Looks like backuppc destroys selinux context on restore? is this
feature or bug? any workaroud available?

Eero


Neither one -- it's a feature of selinux that files that are touched (or 
replaced) lose their context.  The labels are still there, but are 
unconfirmed.


There are a few workarounds.
1) Don't use selinux
2) Even if contexts were copied using extended attributes, they would be 
unconfirmed, so you'd have to use restorecon.  (Related:  you need to 
use xattr for backup and restore)

3) You can use backuppc's hooks to automate #2


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-10-04 Thread Michael Stowe

On 2018-10-04 11:28, Jaime Fenton wrote:

Ok, so that breakdown with suggestions seems to have worked, I'm now
getting successful backups. I suspect some of this was it was hitting
local links that weren't going anywhere and getting caught in a loop.

Is there any way to set a variable that can point to the username
that's set in HOSTS? Ie if I wanted to backup files:
- Users\usernamegoeshere\Appdata\
- Users\usernamegoeshere\Documents\
- Users\usernamegoeshere\Downloads\
- Users\usernamegoeshere\Desktop\

Could it be possible to make something like $username to replace
"usernamegoeshere" so that it pulls that name automatically? (this
would make templating a lot easier IMO but I may not have thought of
downsides)

Thanks,
Jaime


For what it's worth, I use wildcards for that particular issue, e.g.:

Users\*\Appdata\

That should work for either exclusions or inclusions -- however, it 
doesn't help much for multiuser systems, since it will (naturally) try 
to grab every user's directories.  In practice, it means I get some 
mostly empty directories for built-in accounts or old accounts, but at 
least get what I'm looking for.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-10-04 Thread Michael Stowe

On 2018-10-04 09:14, Michael Stowe wrote:

On 2018-10-03 16:17, Jaime Fenton wrote:

I thought I had a breakthrough as I found that for whatever reason,
the backuppc was using the local server account and not the service
account that has access to the smb share for the computer. I have
confirmed that  the BackupPC domain account does have access to the
files and folders that we list..

I've now cleared that up and raised the verbosity level.

However, I'm still seeing errors and having backups fail. I do need
some assistance getting to the bottom of it because there does not
appear to be a clear reason it's failing.

I've included the xfer log and the xfer error log in case that shines
some light on what's going on.

Thanks,
Jaime
-Original Message-
From: Michael Stowe [mailto:michael.st...@member.mensa.org]
Sent: Monday, September 17, 2018 8:49 PM
To: General list for user discussion, questions and support

Cc: Jaime Fenton 
Subject: Re: [BackupPC-users] Troubleshooting assistance for Bad 
Transfer


On 2018-09-17 16:26, Jaime Fenton wrote:

Here's my Error log example.

Some of the file locks are legitimate and I completely understand why
they happen, but the NT_STATUS_SHARING_VIOLATIONs and
NT_STATUS_CONNECTION_DISCONNECTEDs don't make a lot of sense to me.
The Backup aborted ( ) also happens for not reason (that I can note)
and would love if someone could explain that.

Jaime


I'll explain what I can:
NT_STATUS_ACCESS_DENIED listing \Users\\AppData\Local\Application Data\*

In a nutshell, the "AL\backuppc" user doesn't have access to  files by default, even if it is granted administrative
privileges.

NT_STATUS_SHARING_VIOLATION opening remote file \Users\\AppData\Local\Microsoft\Windows\UsrClass.dat

This is a registry file -- you won't be able to back it up *at all*
without using shadow copies.

Backup aborted ()

I can't really explain this one, but I do recall that certain newer
versions of backuppc and older versions of smbclient would combine so
that BackupPC would have trouble interpreting the return messages from
smbclient.  Might need to check your versions, or bump up the log
verbosity for more info.
--
Jaime Fenton
Support Engineer

T: +1 604 398 4800 (main)
D: +1 604 398 4813 (direct)
E: jaime.fen...@animallogic.ca

840 Cambie Street
Vancouver, BC V6B 2P6



I'll try to break it down:
NT_STATUS_ACCESS_DENIED listing \Users\\AppData\Local\Application Data\*

In brief, the smbclient uses the "execute" bit to determine whether it
can traverse a directory, in addition to windows and smb permissions.
Windows doesn't have an execute bit (of course) which can lead to some
weird symptoms.  At any rate, this is typically inherited permission
issues, so start there.  This is testable by using smbclient to
navigate to the directory and typing "ls," then correcting permissions
on the Windows side, then typing "ls" in smbclient again until it
works.

NT_STATUS_SHARING_VIOLATION opening remote file \Users\\AppData\Local\Comms\UnistoreDB\store.jfm

This is an open file.  Some files in Windows are *always* open, and
you'll have to either exclude them, or shut down the program that has
them open.

NT_STATUS_UNSUCCESSFUL listing \Users\\AppData\Local\Google\Chrome\User
Data\Default\data_reduction_proxy_leveldb\*

This is a different level of error than NT_STATUS_ACCESS_DENIED --
basically, it means that the directory refuses to be browseable by
SMB, which generally indicates locks or transience rather than
permission errors.  Exclude this directory.

Error reading file \Users\\AppData\Local\Google\Chrome\User
Data\Default\data_reduction_proxy_leveldb\LOCK :
NT_STATUS_FILE_LOCK_CONFLICT

This means that this file is not just open, but locked.  You'll have
to shut down whatever has it open (in this case, Chrome, obviously) or
exclude it.

NT_STATUS_DELETE_PENDING opening remote file \Users\\AppData\Local\Microsoft\Windows\Explorer\ThumbCacheToDelete\thmC27C.tmp

This one might be obvious, but you're trying to back up files that are
in the process of being deleted.  This usually means temp, cache, or
other transient files.  Exclude the directory.

--

A lot of these look like cache files or transient files, which I can't
imagine are worth backing up.  The registry also appears, which
*cannot* be backed up without using shadow copies.  (In the interest
of full disclosure, I use shadow copies, so I usually back up the
registry, cache files and everything because a) I'm lazy and b) I
figure I'm less likely to miss something I actually want.)


Here's my list of exludes:
$Conf{BackupFilesExclude} = {
  '*' => [
  'BackupPC/diagnostic.txt',
  'BackupPC/file.out',
  'BackupPC/vss-setvar.cmd',
  'Documents and Settings/*/Application Data/Microsoft/Search/Data',
  'Documents and Settings/*/Local Settings/Application 
Data/Google/Chrome/User Data/Default/Cache',
  'Documents and Settings/*/Local Settings/Application 
Data/Mozilla/Firefox/P

Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-10-04 Thread Michael Stowe

On 2018-10-03 16:17, Jaime Fenton wrote:

I thought I had a breakthrough as I found that for whatever reason,
the backuppc was using the local server account and not the service
account that has access to the smb share for the computer. I have
confirmed that  the BackupPC domain account does have access to the
files and folders that we list..

I've now cleared that up and raised the verbosity level.

However, I'm still seeing errors and having backups fail. I do need
some assistance getting to the bottom of it because there does not
appear to be a clear reason it's failing.

I've included the xfer log and the xfer error log in case that shines
some light on what's going on.

Thanks,
Jaime
-Original Message-
From: Michael Stowe [mailto:michael.st...@member.mensa.org]
Sent: Monday, September 17, 2018 8:49 PM
To: General list for user discussion, questions and support

Cc: Jaime Fenton 
Subject: Re: [BackupPC-users] Troubleshooting assistance for Bad 
Transfer


On 2018-09-17 16:26, Jaime Fenton wrote:

Here's my Error log example.

Some of the file locks are legitimate and I completely understand why
they happen, but the NT_STATUS_SHARING_VIOLATIONs and
NT_STATUS_CONNECTION_DISCONNECTEDs don't make a lot of sense to me.
The Backup aborted ( ) also happens for not reason (that I can note)
and would love if someone could explain that.

Jaime


I'll explain what I can:
NT_STATUS_ACCESS_DENIED listing \Users\\AppData\Local\Application Data\*

In a nutshell, the "AL\backuppc" user doesn't have access to  files by default, even if it is granted administrative
privileges.

NT_STATUS_SHARING_VIOLATION opening remote file \Users\\AppData\Local\Microsoft\Windows\UsrClass.dat

This is a registry file -- you won't be able to back it up *at all*
without using shadow copies.

Backup aborted ()

I can't really explain this one, but I do recall that certain newer
versions of backuppc and older versions of smbclient would combine so
that BackupPC would have trouble interpreting the return messages from
smbclient.  Might need to check your versions, or bump up the log
verbosity for more info.
--
Jaime Fenton
Support Engineer

T: +1 604 398 4800 (main)
D: +1 604 398 4813 (direct)
E: jaime.fen...@animallogic.ca

840 Cambie Street
Vancouver, BC V6B 2P6



I'll try to break it down:
NT_STATUS_ACCESS_DENIED listing \Users\username>\AppData\Local\Application Data\*


In brief, the smbclient uses the "execute" bit to determine whether it 
can traverse a directory, in addition to windows and smb permissions.  
Windows doesn't have an execute bit (of course) which can lead to some 
weird symptoms.  At any rate, this is typically inherited permission 
issues, so start there.  This is testable by using smbclient to navigate 
to the directory and typing "ls," then correcting permissions on the 
Windows side, then typing "ls" in smbclient again until it works.


NT_STATUS_SHARING_VIOLATION opening remote file \Users\username>\AppData\Local\Comms\UnistoreDB\store.jfm


This is an open file.  Some files in Windows are *always* open, and 
you'll have to either exclude them, or shut down the program that has 
them open.


NT_STATUS_UNSUCCESSFUL listing \Users\username>\AppData\Local\Google\Chrome\User 
Data\Default\data_reduction_proxy_leveldb\*


This is a different level of error than NT_STATUS_ACCESS_DENIED -- 
basically, it means that the directory refuses to be browseable by SMB, 
which generally indicates locks or transience rather than permission 
errors.  Exclude this directory.


Error reading file \Users\\AppData\Local\Google\Chrome\User 
Data\Default\data_reduction_proxy_leveldb\LOCK : 
NT_STATUS_FILE_LOCK_CONFLICT


This means that this file is not just open, but locked.  You'll have to 
shut down whatever has it open (in this case, Chrome, obviously) or 
exclude it.


NT_STATUS_DELETE_PENDING opening remote file \Users\username>\AppData\Local\Microsoft\Windows\Explorer\ThumbCacheToDelete\thmC27C.tmp


This one might be obvious, but you're trying to back up files that are 
in the process of being deleted.  This usually means temp, cache, or 
other transient files.  Exclude the directory.


--

A lot of these look like cache files or transient files, which I can't 
imagine are worth backing up.  The registry also appears, which *cannot* 
be backed up without using shadow copies.  (In the interest of full 
disclosure, I use shadow copies, so I usually back up the registry, 
cache files and everything because a) I'm lazy and b) I figure I'm less 
likely to miss something I actually want.)



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BackupPC 4.2.1 apparently in an infinite loop.

2018-09-19 Thread Michael Stowe

On 2018-09-19 03:52, G.W. Haywood via BackupPC-users wrote:

Hello again all,

On Tue, 18 Sep 2018, Michael Stowe wrote:

On Mon, 17 Sep 2018, G.W. Haywood wrote:


I suspect that this is a fault in BackupPC_backupDelete ...
RmTreeQuietInner seems to be seeing some sort of bowdlerization
...
RmTreeQuietInner: 
/mnt/3TLV/backuppc/pc/kestrel/268//var/lib/backuppc/pc/kestrel/268/X 
isn't a directory (while removing 
/var/lib/backuppc/pc/kestrel/268/X/f.cache)

...


What is indicating to you that a directory path is built incorrectly?


The part that says:

RmTreeQuietInner:
/mnt/3TLV/backuppc/pc/kestrel/268//var/lib/backuppc/pc/kestrel/268/X
isn't a directory (while removing
/var/lib/backuppc/pc/kestrel/268/X/f.cache)

BackupPC has somehow concatenated the mount path and the symlink path
and come up with balderdash.  I've started on tracking it down by
means of log statements, but what with all these alligators time has
been a bit short.


This is curious, as it would potentially represent a problem for anybody 
whose repository is sym-linked.  It's possible that the delete process 
is leaving unexpected debris, and this is reponsible for the unexpected 
space that BackupPC is consuming.


We've all got alligators; if nobody gets to it first, I'll pursue this.


Does it make sense to focus on rsync first?


Well it's rsync_bpc not rsync, and I have no idea, but vide infra.


Fair enough -- though presumably it's rsync_bpc on one side and rsync on 
the other.  If you've already killed the process and it doesn't get 
stuck again, it may not be much to worry about.  If it hangs again, it's 
probably worth tracing both sides to see what it's up to.



Is [the correct way to reduce the storage used by V4] documented?


I guess I'll just leave this hanging and hope for the hive to come up
with something.

On Tue, 18 Sep 2018, Guillermo Rozas wrote:

On Mon, 17 Sep 2018, G.W. Haywood wrote:


... BackupPC appears to think that it has now used 5TB of a 3TB
disc and the claimed usage is growing. ...


... and growing. ...


Regarding this point: have you tried reducing the parameter
PoolSizeNightlyUpdatePeriod? ... forcing it to check the size in a
single night ... for a couple of days solved it.


Oh, dammit, thank you!  I remember seeing that when I trawled through
the new configuration options and then forgot all about it.  I set it
to 1 last night, before the nightly backup run, then stopped BackupPC,
killed the ten-day-old rsync_bpc, restarted BackupPC and hit the sack.
This morning normality seems to be restored, at least the crazy usage
figures have disappeared and the V4 pool size looks sane again:

8<--
Pool is 940.29+987.81GiB comprising 1757074+1459146 files and
16512+4369 directories (as of 2018-09-19 01:29),
Pool hashing gives 1678+1909 repeated files with longest chain 7+4,
Nightly cleanup removed 6436+10275 files of size 135.04+0.26GiB
(around 2018-09-19 01:29),
Pool file system was recently at 75% (2018-09-19 08:49), today's max
is 80% (2018-09-19 01:00) and yesterday's max was 80%.
8<--

There were log messages for over 17,000 missing pool files but I'm
guessing that this is probably the result of my manually running
BackupPC_backupDelete.  I haven't had a chance to investigate that
but I will do later.

Thanks again!



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-09-17 Thread Michael Stowe

On 2018-09-17 16:26, Jaime Fenton wrote:

Here's my Error log example.

Some of the file locks are legitimate and I completely understand why
they happen, but the NT_STATUS_SHARING_VIOLATIONs and
NT_STATUS_CONNECTION_DISCONNECTEDs don't make a lot of sense to me.
The Backup aborted ( ) also happens for not reason (that I can note)
and would love if someone could explain that.

Jaime


I'll explain what I can:
NT_STATUS_ACCESS_DENIED listing \Users\username>\AppData\Local\Application Data\*


In a nutshell, the "AL\backuppc" user doesn't have access to username> files by default, even if it is granted administrative 
privileges.


NT_STATUS_SHARING_VIOLATION opening remote file \Users\username>\AppData\Local\Microsoft\Windows\UsrClass.dat


This is a registry file -- you won't be able to back it up *at all* 
without using shadow copies.


Backup aborted ()

I can't really explain this one, but I do recall that certain newer 
versions of backuppc and older versions of smbclient would combine so 
that BackupPC would have trouble interpreting the return messages from 
smbclient.  Might need to check your versions, or bump up the log 
verbosity for more info.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BackupPC 4.2.1 apparently in an infinite loop.

2018-09-17 Thread Michael Stowe

On 2018-09-17 12:15, G.W. Haywood via BackupPC-users wrote:

Hello again all,

[Replying to myself here to try to get the thread back on topic.]


Thank you!


On Mon, 17 Sep 2018, G.W. Haywood wrote:


... I suspect that this is a fault in BackupPC_backupDelete ...



8<--
BackupPC_backupDelete debug output: userid is redacted, NO OTHER 
CHANGES.
The pool is on a 3TB LUKS encrypted volume which is mounted on 
/mnt/3TLV
and /var/lib/backuppc is a symlink to /mnt/3TLV/backuppc.  The 
function
RmTreeQuietInner seems to be seeing some sort of bowdlerization of 
these.

8<--
tornado:/var/lib/backuppc/pc/kestrel# >>> 
/usr/share/backuppc/bin/BackupPC_backupDelete -h kestrel -n 268 -l -s 
Homes X/.cache 'X/.moonchild productions'

__bpc_pidStart__ 13276
BackupPC_backupDelete: $fullDelPath=[/Homes/X/.cache]; 
$delPathM=[/X/f.cache]

BackupPC_backupDelete: removing #268/Homes/X/.cache
__bpc_progress_state__ merge #268/Homes/X/.cache -> 
#267/Homes/X/.cache

BackupPC_backupDelete: Merge into backup 267/Homes/X/.cache
bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, 
KeepOldAttribFiles = 0
RmTreeQuietInner: 
/mnt/3TLV/backuppc/pc/kestrel/268//var/lib/backuppc/pc/kestrel/268/X 
isn't a directory (while removing 
/var/lib/backuppc/pc/kestrel/268/X/f.cache)
BackupPC_backupDelete: $fullDelPath=[/Homes/X/.moonchild 
productions]; $delPathM=[/X/f.moonchild productions]
BackupPC_backupDelete: removing #268/Homes/X/.moonchild 
productions
__bpc_progress_state__ merge #268/Homes/X/.moonchild productions 
-> #267/Homes/X/.moonchild productions
BackupPC_backupDelete: Merge into backup 267/Homes/X/.moonchild 
productions
RmTreeQuietInner: 
/mnt/3TLV/backuppc/pc/kestrel/268//var/lib/backuppc/pc/kestrel/268/X 
isn't a directory (while removing 
/var/lib/backuppc/pc/kestrel/268/X/f.moonchild productions)
bpc_attrib_dirWrite: old attrib has same digest; no changes to ref 
counts

__bpc_pidStart__ 13279
BackupPC_refCountUpdate: computing totals for host kestrel
__bpc_progress_state__ sumUpdate 0/128
bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, 
KeepOldAttribFiles = 0

__bpc_progress_state__ sumUpdate 8/128
__bpc_progress_state__ sumUpdate 16/128
__bpc_progress_state__ sumUpdate 24/128
__bpc_progress_state__ sumUpdate 32/128
__bpc_progress_state__ sumUpdate 40/128
__bpc_progress_state__ sumUpdate 48/128
__bpc_progress_state__ sumUpdate 56/128
__bpc_progress_state__ sumUpdate 64/128
__bpc_progress_state__ sumUpdate 72/128
__bpc_progress_state__ sumUpdate 80/128
__bpc_progress_state__ sumUpdate 88/128
__bpc_progress_state__ sumUpdate 96/128
__bpc_progress_state__ sumUpdate 104/128
__bpc_progress_state__ sumUpdate 112/128
__bpc_progress_state__ sumUpdate 120/128
__bpc_progress_state__ rename total
BackupPC_refCountUpdate: host kestrel got 0 errors (took 8 secs)
BackupPC_refCountUpdate total errors: 0
__bpc_pidEnd__ 13279
BackupPC_backupDelete: got 0 errors
__bpc_pidEnd__ 13276
8<--


It looks very much to me as if a directory path was built incorrectly.
Observations anyone?  For the avoidance of unfairly cast doubt, the
command line that I've given is verbatim except as noted carefully in
my intro.  In other words nothing has been changed which would make
the slightest difference to someone who wanted to establish that the
command I used was correct - or otherwise.  If it was incorrect, I'd
love to know in what way.  If the tool is not to be used in this way
to do what the documentation says it does, please explain why.  The
documentation says "You don't need to manually run these utilities."
which is a long way from "You must not manually run these utilities."
and in fact it goes on later to discuss manual use of some of them.
(And one of them can, apparently, _only_ be run manually.)


What is indicating to you that a directory path is built incorrectly?


Is [the correct way to reduce the storage used by V4] documented?


Pointers would still be most welcome.  I'd like to delete some junk
from the pool.  Yes, it would have been nice if it hadn't got in there
in the first place.  But it's there now and I'd like to get rid of it.


From your description of symptoms, it seems like something else is going 
on that might be interfering with what you're attempting to do.  Does it 
make sense to focus on rsync first?



... BackupPC appears to think that it has now used 5TB of a 3TB
disc and the claimed usage is growing. ...


... and growing.  It's now claiming 8.5TB used by the pool.  Pretty
soon the claimed pool usage will exceed the total storage used by all
the hosts that are being backed up.  Real partition usage is ~80%, of
course it's is using nothing like the storage it claims to be using.


This can be a side effect of running the 

Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-09-17 Thread Michael Stowe

On 2018-09-17 10:43, Jaime Fenton wrote:

Thanks Guillermo, that would be good to see for reference.

However, SMB works for all my other machines, it’s just these two
machines that won’t work completely with SMB. Rsync was suggested as
an alternative and it didn’t work.

Ultimately I just need help figuring out why SMB isn’t working for
two machines only.

Jaime


Unless you set up rsync to coordinate with shadow copies, it doesn't 
avoid file semantic problems with SMB.  (I don't see a description of 
this in the thread, but it's possible I missed it.)  If you need it, I 
have an installer that will help coordinate rsync with shadow copying, 
but there's at least one alternative implementation.


Do you have the relevant detailed log from either rsync or SMB?  (I also 
didn't see this, but again, it's possible I lost the thread.)



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BackupPC 4.2.1 apparently in an infinite loop.

2018-09-16 Thread Michael Stowe

On 2018-09-16 11:59, Mike Hughes wrote:

Michael,

Condescending and belittling treatment of others in this list is not
the norm. Your personal attacks are unwarranted and unhelpful.


I certainly didn't mean anything that way, and let me apologize should 
anybody have taken anything that way.  My general comments and generic 
example were meant to be taken as that, and not a commentary on the OP.



Since you reached out and asked for help in relating to others I will
share with you this piece from Dr. Phil Agre which helped me immensely
when I was learning the ropes at the helpdesk and I still refer to it
regularly to help keep me on track:


I appreciate the reference (and am familiar with it) and anybody who has 
spent time doing general tech support knows that a lot of finding the 
right level of help involves properly gauging the level of expertise, 
which is not always a simple task, especially on this list:  we've had 
people ask for help who have needed basic background on what a web 
server is, and people who have been mortally insulted by any suggestion 
that they are not foremost experts in Linux, perl, and the field of 
backup software.  Sometimes, these are the same people.


I've gotten it wrong quite a bit.  I've written long passages on how to 
diagnose a [presumably] complex problem only to find the questioner 
simply didn't know how to invoke an executable that wasn't in the path, 
and I've inquired about simple, in-the-manual, obvious steps and it 
became clear later that the questioner was well into debugging a tough 
corner case.  With this group in particular, some questioners only need 
a tiny nudge, and some need handholding through the most basic 
operations, and some uncover weird corner cases that need addressing.  
Hopefully I get it right more often than not, but it's something I can 
always improve upon.



___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BackupPC 4.2.1 apparently in an infinite loop.

2018-09-16 Thread Michael Stowe

On 2018-09-16 05:27, G.W. Haywood via BackupPC-users wrote:


Once again, thank you for your input.  Do consider some restraint.


That's fair.  Consider the context:  you've already admitted to breaking 
BackupPC after an upgrade and instead of methodically going through the 
steps of getting it working again, you upgrade to a completely new 
version, misunderstand how it works, and start mucking about with 
internal tools from the command line, while demonstrating that you don't 
know the side effects.


Is there a quick fix for this?  Not if you're trying to solve one 
problem with a completely unrelated tool:  if you want to solve taking 
too much space, there are better ways to do it.  If you want to solve 
rsync problems, there are better ways to do it.


You do have my apologies for any confusion I have caused -- so yes, 
BackupPC_backupDelete is capable of removing directories from a share 
that were backed up.  Will this solve your problem, or is it a good 
idea?  No.  Are you interpreting what it's doing and its side effects 
properly?  No.  Did you get the syntax right?  Hard to say, as you've 
edited the command line and its responses -- and as I've already pointed 
out, most people wouldn't run it from the command line, so if there are 
quirks in how it handles directory input, a test case would be 
warranted.


In many years of helping users who have wide variety in degrees of 
expertise, I do struggle with people who demonstrate an incomplete 
understanding of the tool they're trying to use, while insisting on 
doing things that are, at best, orthogonal to their problem.  It is 
usually coupled with a healthy dose of failing to follow reasonable 
steps up until that point, and unwinding that is difficult, if not 
impossible.  It is often unclear whether it is a problem of attention to 
detail, ability to comprehend, missing fundamentals, poor 
troubleshooting techniques, or an outsized ego.


Tech Support: "Okay, let's see what's in the directory.  Type 'ls -l' "
Customer: [typing sounds continue for several minutes]
Tech Support: "Uh.  Do you have a directory listing yet?"
Customer: "I read that hard drives need fsck'ing every now and then"
Tech Support: "It would help to know what's in that directory"
Customer: "It's rebooting now"

I'm open to suggestions as to how to be helpful in such situations.  If 
obliquely pointing toward the documentation doesn't work, would it be 
better to be more direct?  Would you prefer no response?  What is the 
best way to gently suggest that leaping to a solution before 
understanding the problem is unlikely to yield satisfying results?


Thanks


___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BackupPC 4.2.1 apparently in an infinite loop.

2018-09-14 Thread Michael Stowe

On 2018-09-14 04:37, G.W. Haywood via BackupPC-users wrote:

Hi there,

About a month ago I made the elementary mistake of upgrading my backup
server from Debian Jessie to Debian Stretch.  Just about everything 
broke.


The biggest concern was the backups themselves.  The upgrade changed
BackupPC from version 3.3.0, which seemed to have been working fine for
years, to version 3.3.1, which didn't work at all.  Wouldn't even 
start.


Naturally, it would be incorrect to ascribe this breakage to BackupPC 
3.3.1, which works well.  Debian made a number of changes which would 
directly affect BackupPC, all of which require adjustments.  
(Alternatively, of course, you can start over.)


After a couple of days trying to get it to start I decided the best 
thing
to do would be to cut my losses and install a version 4.  So I 
installed

version 4.2.1.

A couple of days later I had what I thought was a working 4.2.1 and 
things
started to settle down, but 4.2.1 seemed to be eating a LOT more disc 
space
than V3.x ever did and I spent another couple of days moving everything 
I
could off the 3TB drive which holds the pool partition.  V3 used just 
over
1TB to back up 12TB of data on our various machines.  V4 wanted more 
like
2.6TB and it was still rising slowly four days after the initial 
backups
had completed.  Well they'd stopped, if not exactly completed - there 
were

some 'partial' backups.

Still fearing "no space left on device" I used BackupPC_ls (it's 
lovely!)
to look around in the backups, and then took my courage in both hands 
and
tried used BackupPC_backupDelete to delete some cruft from home 
directories
which really doesn't need to be backed up; entire directories like 
'.cache'
and '.moonchild productions' from several home directories on several 
hosts.


And I used "rm -r *" to list the contents of a directory.

In all seriousness, BackupPC_backupDelete is not a tool which removes 
directories you did not mean to back up.  It deletes entire backups or 
shares.


That's when I think things really started to unravel.  AFAICT when I 
gave a
command to BackupPC_backupDelete to delete a directory from a backup, 
it in
fact deleted the entire backup.  More than once.  Here's some debug 
output
for an attempt to delete data from a backup for a remote host.  I've 
made a
couple of edits to shorten the bash prompt, split the long command line 
with

a backslash escaped newline, and redact the user's login name:

8<--
tornado:[...]/kestrel# >>> 
/usr/share/backuppc/bin/BackupPC_backupDelete \
-h kestrel -n 271 -l -s Homes /xx/.cache '/xx/.moonchild 
productions'

__bpc_pidStart__ 20851
BackupPC_backupDelete: removing #271/Homes
__bpc_progress_state__ delete #271/Homes
BackupPC_backupDelete: No prior backup for merge
__bpc_progress_fileCnt__ 17116
__bpc_progress_fileCnt__ 17918
__bpc_progress_fileCnt__ 28734
bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, 
KeepOldAttribFiles = 0

BackupPC_backupDelete: removing #271/Homes
__bpc_progress_state__ delete #271/Homes
BackupPC_backupDelete: No prior backup for merge
bpc_attrib_dirWrite: old attrib has same digest; no changes to ref 
counts

__bpc_pidStart__ 20861
BackupPC_refCountUpdate: computing totals for host kestrel
__bpc_progress_state__ sumUpdate 0/128
bpc_attrib_backwardCompat: WriteOldStyleAttribFile = 0, 
KeepOldAttribFiles = 0

__bpc_progress_state__ sumUpdate 8/128
__bpc_progress_state__ sumUpdate 16/128
__bpc_progress_state__ sumUpdate 24/128
__bpc_progress_state__ sumUpdate 32/128
__bpc_progress_state__ sumUpdate 40/128
__bpc_progress_state__ sumUpdate 48/128
__bpc_progress_state__ sumUpdate 56/128
__bpc_progress_state__ sumUpdate 64/128
__bpc_progress_state__ sumUpdate 72/128
__bpc_progress_state__ sumUpdate 80/128
__bpc_progress_state__ sumUpdate 88/128
__bpc_progress_state__ sumUpdate 96/128
__bpc_progress_state__ sumUpdate 104/128
__bpc_progress_state__ sumUpdate 112/128
__bpc_progress_state__ sumUpdate 120/128
__bpc_progress_state__ rename total
BackupPC_refCountUpdate: host kestrel got 0 errors (took 6 secs)
BackupPC_refCountUpdate total errors: 0
__bpc_pidEnd__ 20861
BackupPC_backupDelete: got 0 errors
__bpc_pidEnd__ 20851
tornado:/var/lib/backuppc/pc/kestrel# >>>
8<--

After a couple of attempts to delete these directories, the next backup 
of
home directories on the _local_ machine seems to have started from 
scratch
five days ago and is still in progress.  The actual disc usage as 
reported
by the OS (df) isn't changing, but BackupPC appears to think that it 
has
now used 5TB of a 3TB disc and the claimed usage is growing.  If the 
graph
on the status page were to be believed (and clearly is it not) the V3 
pool
is growing by hundreds of gigabytes per day, and the V4 pool is 
shrinking.


Output from top:
8<--

Re: [BackupPC-users] Deltacopy - Cant' backup subdir

2018-08-31 Thread Michael Stowe

On 2018-08-31 09:18, Omar Speranza wrote:

Il 31/08/2018 18:01, Michael Stowe ha scritto:
Of course, the usual way of doing this is to specify which directories 
you want to back up, in the web interface.



Thank you
Sorry for my english

In the web interface I've set the "BackupFilesOnly" parameter writing
"/photos" as directory of "MyShareTest"

But BackupPc backups *all* sharename "MyShareTest" and not only 
"photos"



Here is a screenshot with my parameters
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7TU5RmA9p08N47F1T88ruyg-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CuTHF4XsCSatjFvPgFh8pn2d0ej55pwyN0jwQQREJPU0Vp1LdkNqTJC-2F9DLunLq1JqV7CAmS9xmXwmhqpUMWYqijTBvX2GDlqbiew9IZnRFNJ9AaKO7QvUehDkrxSLycjlKHT4UQPWlUVXgAiIxsW6RzQgxD5f6xWLYO8ZikcZdE-3D

And the result of backup :-(
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7R5IMfVmhbxOK525J6S9Odk-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CuTHF4XsCSatjFvPgFh8pn60I5DO3PQzFu6VN1idhBeg0VO50oek-2B4nkh-2FdlLZirGZBg9DlBcQsXntF0X-2B-2FmVfSXUjzVQH7jzNTnAPGxJTj-2FZO5iy93oiyGiOcBhh4YmNvcwy1x1iINWlE-2B-2B2uw43-2FIFaL5wyWr3RUOmseo2L3Pg-3D


Excellent -- that helps quite a bit, though it would also help to see 
the "BackupFilesOnly" sections of your config.pl (if any) and the 
machine-specific config.  One possibility is that there are updates to 
those files that don't show up on the web interface.


In this case, however, your picture illustrates fairly well what 
happened -- the "BackupFilesOnly" directory isn't associated with the 
rsyncsharename.  You either need to add '*' (to have it apply to all 
shares) or "MyShareTest" to have it apply to that one share.


N.B.:  this *should* also be clear in the machine-specific config
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Deltacopy - Cant' backup subdir

2018-08-31 Thread Michael Stowe

On 2018-08-31 07:24, Omar Speranza wrote:

Hi,
I'd like to use backuppc to backup some subdir of C drive of my Windows 
PC.


I've installed DeltaCopy on Windows machine and I've created a
sharename called "DriveC" that points to C:\ path

On backuppc of my Linux server I've created RsyncD xfer profile with
user/pass of deltacopy share.

But seems that is not possibile to backup only certains subdir of
"DriveC" (ex: DriveC/Doc, DriveC/Photos)

I can only get *entire" share DriveC


Can you help me?


Of course, the usual way of doing this is to specify which directories 
you want to back up, in the web interface.  Since this is *really* 
straightforward, it's hard to provide reasonable advice without either 
asking questions that may seem insulting, or additional information.  
I'll try to avoid that by pointing out common questions you should 
answer when asking for help:


- What are you trying to accomplish?
- What have you tried?
- What was the result?
Also useful:
- What versions and operating systems are involved on each end?
- What are the relevant parts of your config.pl and [machine].pl?

This will avoid a barrage of questions which may not be helpful, such as 
"did you know there's a web interface?" and "do you know the difference 
between a directory and a module?"
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Troubleshooting assistance for Bad Transfer

2018-08-29 Thread Michael Stowe

On 2018-08-28 14:09, Jaime Fenton wrote:

Hi there,

On advice from craigbarrett, I'm posting this query here.

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7dCyrjIWd3LVbeqZFg4ygUFVElutyD6tczisbTOCLT1CDLPzWtJttd9gBFNYNE3XXg-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CONFFacxtX9Hc57GMbLmaIWQkGF-2FgKLul2It-2FOulNL1-2B2XznC8u6Kz8YxmGXpNj7cHDEAHOkAb4j3gp7JYmOK-2FkPqe801yFuHJhXQsvYL9D02EI4RgW7HtqrAnEmF9zCbuE2oNydwWMObSElhRJz6ycjLWkwsbkOD38G1VxgXXdM5SddCxp-2Fb7khWGW8gEGwQ

I've gone to the XferLog file and it's gibberish. Is there a way to
compile that file or advice on troubleshooting that file that can give
me some insight as to why my backups are having fatal errors?

Thanks in advance,

Jaime Microsoft Office Outlook Signature
--
Jaime Fenton
Support Engineer


From what you posted in the bug report, you're trying to back up an 
Administrative share via smb.


Once you sort out how to read logs, I'll note that various Microsoft 
updates have disabled Administrative shares, and smb is notoriously 
picky, and cannot handle things like open files.


As a general troubleshooting step, it's a good idea to ensure you can 
actually access the system via smbclient, especially for something 
dubious like an Administrative share.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] backup in a special scenario

2018-08-04 Thread Michael Stowe

On 2018-08-04 10:55, daggs wrote:

Greetings,

I have a windows installation running inside a kvm vm, for past
experience, backup using the samba protocol is unreliable (mostly
permissions issue which cause the backup to fail).


This is probably why rsync is the protocol of choice for Windows rather 
than smb.



so I'v decided to try a different approach, I want to run a script on
the vm's host prior to the backup, preform the backup only if it
doesn't fails, run the backup and run another script on the vm's host
in the same manner as the pre.


Sure, this is the kind of thing DumpPreUserCmd is designed for.  I'm not 
sure how this is a different approach, or how it relates to solving your 
protocol issue, though.



the main idea is to mount the image remotely, use rsync to backup and
unmount the image.


Naturally, the image will not be able to be pooled.


does backuppc supports the able features?


What is an "able feature?"


Thanks,

Dagg.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Tricks for Win 10 SMB Backup?

2018-07-30 Thread Michael Stowe

On 2018-07-30 11:24, Tim Evans wrote:

On 07/30/2018 01:44 AM, Craig Barratt wrote:
What is the value of $Conf{SmbClientFullCmd} and what is the smb 
command that is being run (see the XferLOG file)?


$Conf{SmbClientFullCmd} = '$smbClientPath $host\\$shareName
$I_option -U $userName -E -d 1 -c tarmode\\ full -Tc$X_option -
$fileList';

XferLOG file /raptor/pc-backups//pc/new-pelican/XferLOG.0.z created
2018-07-30 14:11:20
Backup prep: type = full, case = 1, inPlace = 1, doDuplicate = 0,
newBkupNum = 0, newBkupIdx = 0, lastBkupNum = , lastBkupIdx =
(FillCycle = 0, noFillCnt = )
Running: /usr/bin/smbclient new-pelican\\C\$ -U backup -E -d 1 -c
tarmode\ full -Tc -
full backup started for share C$
Xfer PIDs are now 31549,31548
tarExtract: /usr/share/BackupPC/bin/BackupPC_tarExtract: got Full = 1
tarExtract: /usr/share/BackupPC/bin/BackupPC_tarExtract starting...
(XferLogLevel = 1)
XferErr tree connect failed: NT_STATUS_ACCESS_DENIED
XferErr tree connect failed: NT_STATUS_ACCESS_DENIED

The C drive share on the PC is manually accessible:

# smbclient //new-pelican/c -Ubackup%XXX
Try "help" to get a list of possible commands.
smb: \>

Same password used for the user 'backup.' Can 'get' files here.

Adding the dollar sign (i.e., referencing the default 'c$' share) to 
the above:


smbclient //new-pelican/c\$ -Ubackup%XXX
tree connect failed: NT_STATUS_ACCESS_DENIED


This is normal for Administrative Shares since approximately Vista.

Did you override the Administrative Shares policy that disables remote 
access to Administrative Shares?
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] New Client; Same Hostname

2018-07-24 Thread Michael Stowe

On 2018-07-24 07:28, G.W. Haywood via BackupPC-users wrote:

Hi there,

On Tue, 24 Jul 2018, Tim Evans wrote:


I'm retiring the oldest PC among my BackupPC clients, and replacing it
with a new one.  Would like to use the same hostname for the new
machine, but think this might confuse things with BackupPC.


I'd recommend that you don't do it.  BackupPC is not the only thing
that will get confused.  You will too.  Examples below.

On Tue, 24 Jul 2018, Michael Stowe wrote:


2. Login to the configuration folder

cd /etc/BackupPC

2. Change the host file to the corresponding one :

vi hosts

*change old_host to new_host*


Examples:

1: There are two '2.'s in there.  Has this been tested?


No, the email was not tested before being sent.  Short of QA resources 
to test emails, it was decided to send it without committing to a full 
QA cycle because the risk was low.  As it turns out, the HTML codes for 
"enumerate" aren't consistent among email clients, and different people 
may have experienced different numbering.


However, the steps to renaming a host are sound.


2: In some installations the directory structure is different and the
configuration isn't in /etc/BackupPC.  Debian packages for example use
/etc/backuppc/ for configuration and /var/lib/backuppc/ for data.


I generally assume that people are capable of replacing directory 
examples with real directories.  I also assumed that the Debian was not 
involved because it or apt-get had not been mentioned half a dozen 
times.



3: You need to *keep* old_host and *add* new_host, otherwise the
confusingly-named old_host (it's really old_NAME) won't get backed up
any more.


What would you say the difference between the name of a host and a named 
host are, exactly?  It might be helpful to elaborate on why "NAME" is 
better than "host," to help explain the distinction you're drawing.  (Do 
you name your hosts "host?"  Or "NAME?")



4: Some things on old_host will probably still think they're on
old_host even after you've renamed it to new_(host|name|whatever).


This probably deserves more explanation as well, because without any 
specific examples, I'm going to call nonsense on this.  This isn't 
magic.



5: DHCP servers and things that rely on MAC addresses can get confused.


In what way?


6: Nameservers and things that rely on IP addresses can get confused.


Again, in what way?  Anthropomorphizing DNS is pretty weird to begin 
with, but imagining that changing some hosts around will send one into a 
tailspin of dementia seems beyond the pale.  You do know that DNS 
servers are meant to associate IP addresses with names, right?  This is 
what they do?


--

Sure, it's simplest just to give every host its own name for the life of 
the host and leave it at that, and sure, I recommend it, but there are 
many use cases where this is desirable, and spreading FUD isn't 
particularly helpful.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] New Client; Same Hostname

2018-07-23 Thread Michael Stowe

On 2018-07-23 12:36, Tim Evans wrote:

I'm retiring the oldest PC among my BackupPC clients, and replacing it
with a new one.  Would like to use the same hostname for the new
machine, but think this might confuse things with BackupPC.

Can anyone suggest a process for making this work right?  Thanks.


Rsync is the only transport which will care.  This can be fixed by 
editing ~backuppc/.ssh/knownhosts and deleting the lines that reference 
that system, then using ssh as the backuppc user to log in again and 
store the fingerprint.


If you're not concerned with backups from the new host being incremental 
with (and combined with) the old one, do nothing!  You're all set.  It's 
simplest to do a full backup right away.  Your old host will gracefully 
age out of the pool.


If you are, first rename the old host:

1. Stop BackupPC service :

service backuppc stop

2. Login to the configuration folder

cd /etc/BackupPC

2. Change the host file to the corresponding one :

vi hosts

*change old_host to new_host*

3. Rename the pl file in pc folder :

cd pc

mv old_host.pl new_host.pl

4. Make the changes in the backup location (I assume your backup is 
storing in the location /backup):


cd /backup/filesystem/pc/

mv old_host new_host

5.Start Backuppc service:

service backuppc start

6. Login to the backuppc console and make sure that the changes are 
replicated there.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Windows backup using SMB will not complete

2018-07-18 Thread Michael Stowe

On 2018-07-17 15:06, Sal Ghani wrote:

Hi All,

Newbie to BackupPC, running v 3.3.1 on CentOS 7. I am attempting to
backup 1 shared folder on a Windows 2008 R2 server using SMB.

On running a full backup, I see files get copied to the backuppc
server from the 2008R2 server but the backup job will fail with the
following error:

Backup failed on  (tar:717  Total bytes received:
15010160766).

I haven’t been able to figure out what is going on. Since about 14G
of files get copied, it doesn’t seem to be a permissions issue.

Config.pl for the client is as follows:

$Conf{XferMethod} = 'smb';

$Conf{SmbShareName} = [

  'Public-Share'

];

$Conf{XferLogLevel} = '5';

$Conf{CompressLevel} = '9';

$Conf{BackupZeroFilesIsFatal} = '0';

Smbclientfullcmd is set with the default: $smbClientPath
\\$host\$shareName $I_option -U $userName -E -d 1 -c tarmode\ full
-Tc$X_option - $fileList

Any help would be greatly appreciated.

Thanks,

Sal Ghani.


This advice is pretty generic, since there's not a lot to go on:
1) Test the command line outside of BackupPC to ensure the underlying 
transport is behaving as expected

2) Increase the log verbosity to see what's going on
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Large directory

2018-07-09 Thread Michael Stowe

On 2018-07-09 09:21, Bowie Bailey wrote:

On 7/9/2018 12:09 PM, Les Mikesell wrote:
On Mon, Jul 9, 2018 at 9:42 AM, Bowie Bailey  
wrote:

There was still plenty of free RAM and no swap usage.  I know it was
still doing something because the pool filesystem was slowly growing. 
 I

could try an strace, but I'll have to research that.  I've never used
strace before.


Be sure to check RAM/swap at both ends.   You give strace a process id
number and it shows every system call and its arguments that the
process is making.  It's usually an overwhelming stream of text but
you should be able to see the open() and read()s happening as long as
it is still sending files.


Both ends are on the same machine for this backup.


rsync still spawns two independent processes
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Are Files Re-Copied In Pool On Reinstall of Backup Client Operating System

2018-06-19 Thread Michael Stowe

On 2018-06-19 08:29, Carl W. Soderstrom wrote:

On 06/19 03:18 , Akibu Flash wrote:
So, does the BackupPC pool keep the original backed-up files 
indefinitely, or is there a period of time after which it deletes the 
files, the theory being it doesn't see that Windows client anymore and 
thus the pool files aren't needed anymore.  For example, if I my 
Windows machine is off line for a couple of weeks while I am 
reinstalling the operating system, will BackupPC delete the files in 
the interim?  Thanks.


It keeps old files until the backup containing them gets expired by
newer backups.



My experience with Windows and reinstalling the OS is that many files 
change during a reinstall (notably, almost all of the dot-net libraries 
will change) and the rsync client will need to read them all, but 
there's usually between a 70-90% match (assuming the same version of 
Windows.)
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Slow local backup

2018-06-19 Thread Michael Stowe

On 2018-06-19 06:55, Bowie Bailey wrote:

On 6/19/2018 1:35 AM, Michael Schumacher wrote:

Hi there

Friday, June 15, 2018, 3:06:30 PM, you wrote:

BB> It finally finished after 24 hours.  That gives about 13G/hour or 
about

BB> 3.8M/s.

BB> The CPUs were not busy.  That's what I was confused about.  I 
would have
BB> expected to see a bottleneck at some point, but nothing seemed to 
be
BB> busy.  The CPUs were all at or below 20% and iowait was close to 0 
most
BB> of the time.  I'm not sure how I would determine if the loopback 
was

BB> saturated.

what file system are you using? I am operating a mailserver on a
dedicated machine. There are some millions of small files on that
machine, so the situation is comparable to yours. When I set up that
machine some years ago, I compared various file systems. Surprisingly,
ext4 came out as winner to handle these small files. I am just using
ext4 on top of the hardware, no lvm on that machine. And of course
having a lot of RAM may also speed up your machine, because the server
can read ahead data. I think it may be a matter of tuning that server
to get better performance.
https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7QZGjYW6utjZlDnU1Z3QOebvPgcj6rwg-2BwN0ANYujoLploi4MuWYkssMD-2BgaM20V4XxCXAwbqwVkDg6Hcr3r3gg-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CnT16ypaenVxeoDy3w0AsF8dqMUfQGKK-2BfYZDSqtYmGoclGDxESF14nOfj-2Bkqdl2WSEBw4z6PQevIFSOd4lsL3u-2FnZF4LbeZQhQ3zOdQ5LeHhvcRCOai30995dwwynkAYO-2BeXJ5DtQi2U-2FrVdTRMnTworB7cRhz2qnlPweNM9S1xcdyZkUEmauz4qql9a4VA0
 may
give some hints on read ahead tuning.


The backup server is xfs.  I haven't looked into performance tuning,
since most of my servers are not doing enough IO to need it.


It's probably worth mentioning that I experienced a slowness problem on 
xfs, specifically.


Repeated (forced, offline) fscks sometimes turned up subtle filesystem 
issues, and sometimes reported everything was ok.  The symptoms were 
slowness, and occasional hangs in rsync.  My test results:


* Moving all files to a new jfs filesystem.  Performance improved.  No 
hangs.
* Moving all files to a new xfs filesystem.  Performance improved.  No 
hangs.
* Moving all files to a new ext4 filesystem.  Performance improved.  No 
hangs.
* Moving all files back to the same physical drives, btrfs.  Performance 
improved.  No hangs.


My conclusion at the time was that some kind of corruption or decay had 
crept into the xfs filesystem over time.  The physical drives were 
perfectly fine.  I did not diagnose further, but I did abandon xfs and 
have not had further issues.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Slow local backup

2018-06-14 Thread Michael Stowe

On 2018-06-14 10:05, Bowie Bailey wrote:

I just installed BackupPC v4 on a CentOS 7 server with 4G ram.  I am
trying to back up a local 318G filesystem.  I am using rsyncd over the
loopback connection.  It has been running for 17 hours so far and has
backed up less than half of the directory (based on the size of the
backup filesystem).  Running top does not show any excessive cpu or
iowait.  "free" shows no swap usage and 1.5G available memory.


Why are you using rsyncd over the loopback instead of ... rsync?


Is it normal for the backup to take this long?


While that's hard to guess without knowing the particulars of your 
system, I'm going to go out on a limb and say, no.  No it is not.



Is there a better way to back up a local filesystem?


Personally, I use rsync (not rsyncd) and in the cases where I have 
experienced slowness, it was due to poorly chosen rsync parameters (I 
note that this would not differ between rsync and rsyncd), a broken 
filesystem, or a specific bug in a specific version of rsync.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Successful Backup ?

2018-04-26 Thread Michael Stowe

On 2018-04-26 00:53, Stefan Denzinger wrote:

Hey, thanks for all your replies,

...

tarExtract: Done: 0 ERRORS, 85547 filesExist, 164644946917 sizeExist,
9958971618 5
sizeExistComp, 1019494 filesTotal, 870658911095 sizeTotal

GOT FATAL ERROR DURING XFER (TAR:712  TOTAL BYTES RECEIVED:
870649101872)

...

Saving this as a partial backup, replacing the prior one (got 1019494
and 0 file s versus 0)

...

0 errors, so why does backuppc tell me an fatal error, it is not
understandable for me.


This is because the version of smbclient is newer than the version of 
backuppc you're using, so backuppc is not properly interpreting the 
output.


You can upgrade backuppc, downgrade smbclient, or fix it yourself:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7QRKGSZWvjj3-2Bcv-2F1NJOK6kq-2FHTCkjUYqJpD0siBzoSwR74iYxTZAlw1GRHA0e2X9avBB3vCLA4twONy1UlclRc-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CAJOuA23dWwhjgik2j5wCWzS-2FWclvClq-2FqBW4PURzE2miJNF6s8Nv8AWavbkatzK2ZByHphiMgvFw5WBbB43VBtFYj2pPH7kcwMP-2FuxvXXzKE-2BQkFx8krMalXUA5H8Bnp7yIsmFeVUjIpa9RjoCL7S2l2Ydhnm8bMUmDnBmejHVo-3D

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Successful Backup ?

2018-04-25 Thread Michael Stowe

On 2018-04-24 02:03, Stefan Denzinger wrote:

Hi Guys,

i started working with backuppc since a few weeks, and i now ran inteo
some problems i do not get resolved myself. So maybe somone can help
me.

...


tar:957  NT_STATUS_SHARING_VIOLATION opening remote file
\JEx\Marketing\Events\NX CAMDay\NX CAMDay
2018\Agenda\~nx_cam_day_agenda_~9-79bd.idlk

...

So what does this means to me ? is the full backup now successful or
not ?

When I try to start an incremental backup, it always starts a new full
and after 8 hours the same error came up. But I want to have new
increments..

Does anybody know my mistake ?


It means, that file it open or locked, so you cannot back it up this 
way.  You can try again, and hope it isn't locked, or try a method that 
uses Shadow Copies in Windows to deal with the "open files" issue.


Whether or not you consider this "successful" is whether or not you care 
about this file, but BackupPC does not, since there was a fatal error.  
So incrementals are kind of a non-starter.


If this is a permanent situation and you don't care about the file, you 
may exclude it.  If it's a regular occurrence that files will be opened, 
consider using Shadow Copies instead.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] considerations on i-nodes

2018-04-19 Thread Michael Stowe

On 2018-04-19 06:31, G.W. Haywood via BackupPC-users wrote:

Hi there,

On Thu, 19 Apr 2018, Michael Stowe wrote:


... Those who do reach back a decade or more to select a filesystem,


That's like saying the Linux kernel is a hangover from the end of the
20th century.  It's misleading, and more than a bit unfair especially
considering the numbers of posts to the ext4 mailing list:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7RuEFgozAWvnlNmELy4oKuiA3Uh2zVaVksANvOXpuehpkneX0E-2FfKBOjzEo25YDMGw-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CasKnxVRTUBW0lnnPUiBFDbnzrPzFGfYmk0Iwn1duJneKWemz0bfm83-2Fl8P8pIa0YGeA8QAnhxRLigz6DuEWH0WyS6jHH3rHg5QJhsnUJVs1DRyBbhwrZ-2FLQ9SNo7ZD6ANz0-2BbsyrFGuo-2FUHmLHgg9NSil8n8wpRXy0GiOoHi4bIQ3Lhx4MOCnpVaarfVSQNm


While there's nothing inherently wrong with selecting an older 
filesystem, ext4's design decision of backward compatibility has 
essentially set some of its limitations in stone.  (Your article below 
elaborates on this point; it's not a next generation filesystem, it's 
just something that works.)



I generally expect to have good reason ...


Like, er, it works, it's currently under active development, and it's
supported by every Linux distribution you're likely to meet? :)

Yes, even the developers will agree that it's a stop-gap measure.

But it's a big gap, and the rest of them haven't quite plugged it yet:

https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7UY4nHmLotbnt-2B5EKj0ng5UcSekH-2BHZpU1dS98SsvUShSx4-2BUbCV8Vb8B6dBgin8IA-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CasKnxVRTUBW0lnnPUiBFDZN1h9k4ItyZ8gGNYlU-2B0dRBvelHnc-2BeUkG1G6dU7PDf-2FElfMd2R-2FZAdFHXec7SgsItHvPTxrnMLjr3JiKAKszqOf3OTw4zwTUilcgLC8kMmaSdKlN7-2BtdgplPjuEfgnssjx7gIlgRgctYxuXZesbWmzum-2FxNj24SzlNV-2FxfC2p7


I don't think we're disagreeing, but I note that I'm specifically not 
talking about what filesystem one might select for a disposable Linux 
system and whatever came with the distro, but presumably a backup system 
from which one wants to store and recover files, into which one 
presumably would wish to select something most appropriate to the task, 
and "I want to futz around with inode allocations" is rarely at the top 
of people's to do lists.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] considerations on i-nodes

2018-04-19 Thread Michael Stowe

On 2018-04-19 07:37, Gerald Brandt wrote:

I've had data loss on XFS, so I'm very wary of it. It used to be my go
to filesystem.

Lately, I've been looking at bcachefs2. Anyone experience using it?

Gerald


I've also had data loss on XFS, for which I have no satisfactory 
explanation, but I'm equally wary.  Somewhere around here there's a 
limited comparison I did where I recommend btrfs and JFS.  bcachefs 
seems intriguing, but I haven't used it in anger and I haven't seen 
anything practical to distinguish it from btrfs, but I haven't dived 
into it in much depth, either.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] considerations on i-nodes

2018-04-18 Thread Michael Stowe

On 2018-04-18 04:25, f...@igh.de wrote:

Dear List,

running BackupPC v4 I sometimes ran out of i-nodes.

I have about twelve servers to backup with a total amount of about 2
TB and aboz 20 million files. I keep about 50 backups.

On an ext4 file system it is likely to run out of i-nodes rather than
to run out of disk space (this holds if the file system was created
using the default parameters). In my situation I have only used half
the space but almost all i-nodes available.

Because I prefer ext4 for it's robustness and reliability I have to
consider i-nodes.

We have the pool (or cpool) storing the actual files. And we have the
pc directory storing the structure. The most files in pc are small or
even empty, but they are many (in my case ten times more than the
actual files in pool). They waste an enourmous amount of disk
space.

So I created a separate (logical) partition of only 200 GB in size and
formatted with 1024 bytes per inode and per block (mkfs.ext4 -b 1024
-i 1024 ...). I later use it for pc.

Then I simply copied the pc entries (note: only the entries, not the
complete directory) to the new partition using cp -a. This took about
50 hours but worked correctly (this also shows, that BackupPC v4 can be
replicated). Afterwards I removed the entries from the original pc
directory and mounted the new partition there.

I started BackupPC once again and it works like a charm.

In this constellation remarkable space for the future backup space was
released. If I ever would run out of space or i-nodes again I will
simply increase either of the partitions.

As a suggestion I would recommend an information about i-nodes
consumption in the server status page.


Finally I am looking for a formula that could be used to estimate the
required disk space and i-nodes depending on the number  and size of
files and the number of backups kept.


Best regards


Torsten


I note that many people simply use modern file systems which employ 
dynamic inode allocation, and therefore, neither this feature, nor this 
type of analysis nor maintenance is necessary.  Those who do reach back 
a decade or more to select a filesystem, I generally expect to have good 
reason and spend time performing analysis rather than using the 
defaults, though your mileage may vary.  I'm not impugning your choice 
of ext4 as a matter of taste (nor your excellent work in solving the 
issue) I'm pointing out that information about inode consumption in the 
server status page would be pretty useless to a lot of people.


It's difficult to provide an exacting formula, naturally, but here's the 
general form:


backup size x ( number retained x rate of change ) / compression + 
overhead


Backup size and number retained are relatively easy to measure, 
typically, both in terms of number of files and data size.  "Rate of 
change" is the tricky one, since it's the measure of how quickly your 
files churn.  If this ratio is zero (i.e., your files don't change) then 
backups don't add much needed space.  If they change a lot, you'll need 
a lot more space.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Serious error: last backup ... directory doesn't exist!!! - reason found

2018-03-13 Thread Michael Stowe

On 2018-03-13 16:33, Holger Parplies wrote:
My first thought is to avoid the issue altogether by using a file 
system
that doesn't statically allocate inodes (e.g. XFS or reiserfs, the 
latter
I wouldn't recommend for other reasons, though; I don't know about 
ext4,
btrfs and ZFS, but my guess would be that ext4 has static allocation 
and
the others dynamic). Why worry about a problem modern file systems 
simply

don't have?

Regards,
Holger


I happen to have used BackupPC with ext4, xfs, and btrfs.  ext4 (unlike 
ext2 and ext3) does indeed have a way to extend the number of inodes 
beyond the static allocation.  XFS, elegant and well designed, gave me 
no end of trouble, frankly.  Your mileage may vary, but on more than one 
occasion, I found subtle corruption for unknown reasons had caused weird 
ripple effects throughout the file system.  btrfs has, perhaps 
ironically, proven the most reliable of the three.  Again, your mileage 
may vary, and while I can go into considerable detail about my own 
experiences and testing, they are limited in scope and nature.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-03-01 Thread Michael Stowe

On 2018-03-01 07:22, Marc Gilliatt wrote:

So I've since removed every instance of BackupPC from my server, all
my backups and config files have gone.

I have installed BackupPC 4 from a script I've seen from Github, and I
cannot connect to the GUI whatsoever, it mentions on the webpage...

_Not Found_
_The requested URL /BackupPC_Admin was not found on this server._

_Apache/2.4.18 (Ubuntu) Server at 10.16.0.163 Port 80_

I have also seen that the BackupPC service is still not running...

_● backuppc.service_
_   Loaded: not-found (Reason: No such file or directory)_
_   Active: failed (Result: exit-code) since Tue 2018-02-20 12:44:10
GMT; 1 weeks 2 days ago_

_Feb 20 12:44:10 backuppc01 systemd[1]: Starting LSB: Launch backuppc
server..._
_Feb 20 12:44:10 backuppc01 backuppc[45980]: Starting backuppc:
2018-02-20 12:44:10 Another BackupPC is running (pid 45589);
quitting..._
_Feb 20 12:44:10 backuppc01 systemd[1]: backuppc.service: Control
process exited, code=exited status=1_
_Feb 20 12:44:10 backuppc01 systemd[1]: Failed to start LSB: Launch
backuppc server._
_Feb 20 12:44:10 backuppc01 systemd[1]: backuppc.service: Unit entered
failed state._
_Feb 20 12:44:10 backuppc01 systemd[1]: backuppc.service: Failed with
result 'exit-code'._
_Mar 01 13:26:37 backuppc01 systemd[1]: Stopped LSB: Launch backuppc
server._

 I feel like I've tried everything to resolve this issue regarding a
simple upgrade of this software. I feel my last resort now is to
completely re-raid my server and start fresh with a new install of
Ubuntu and BackupPC. Is there anything I can do to get BackupPC to
work, please?

-


In a nutshell, what's going on is an aborted or crashed BackupPC 
instance left behind its PID file, which will need to be cleaned up 
before BackupPC will start.  (As you can imagine, having more than a 
single instance running at a time is a bad idea.)  If you're certain 
it's not running, you'll find the relevant files in RunDir, set at 
installation time and defined in config.pl.  In my installation, this is 
'/var/run/BackupPC' but different installers/distros make different 
assumptions.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-19 Thread Michael Stowe

On 2018-02-16 01:05, Marc Gilliatt wrote:

I've just tried the _/configure.pl --batch --cgi-dir
/var/www/cgi-bin/BackupPC --data-dir /var/lib/backuppc --backuppc01
backuppc  --html-dir /var/www/html/BackupPC' _command, and it hasn't
worked I'm afraid, I tried reinstalling the Bakcuppc-41.3 tar file
again, and gave it the full correct paths, and when I try to restart
the service I'm still getting _Job for backuppc.service failed because
the control process exited with error code. See "systemctl status
backuppc.service" and "_journalctl_ -_xe_" for details. _With regards
to the SCGI port number, do I set it to -1 or give it a port number?

__And I'm still getting the internal error in the GUI. I don't
understand why this has happened, all I did was upgraded the software,
it looks like I'm going to have to re-raid my server and start
fresh!!!

Is there any way to roll back to my previous version of BackupPC
please?
-


Moving from an installation package to a source-based install can be a 
bit of a trial, depending on how much the norms of that distro and 
packaging system deviate from the tarball.  Version 4 is a fairly 
dramatic leap as well, which complicates things further.


First, though, I'll address your question and give you the good news:  
since you never actually launched BackupPC 4, your repository is instact 
and in the expected location for your distro.  Therefore, to revert to 
the previous version, you need only delete the tarball's files, and 
install the distro's version.


N.B. preserve your config.pl and return it to its original location, as 
your distro may assume it is a new installation and do god-knows-what to 
your config.pl.  Some are kinder than others.


--

The SCGI question is a bit harder to answer without knowing what other 
choices you've made.  Since I have set up SCGI, I match the port to what 
I have set up within Apache... which can be anything, as long as the 
number in Apache and in config.pl are the same.  If I had not set up 
SCGI, I'd set it to -1.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-19 Thread Michael Stowe

On 2018-02-19 01:22, Marc Gilliatt wrote:

I removed the BackupPC 4.1.3 tarball's file, and I've installed the
backuppc_3.3.1-2ubuntu3_amd64.deb(my previous version) I got from
online. When I ran it I still receive the following message...

update-rc.d: warning: start and stop actions are no longer supported;
falling back to defaults
Job for backuppc.service failed because the control process exited
with error code. See "systemctl status backuppc.service" and
"journalctl -xe" for details.
invoke-rc.d: initscript backuppc, action "start" failed.
● backuppc.service - LSB: Launch backuppc server
   Loaded: loaded (/etc/init.d/backuppc; bad; vendor preset: enabled)
   Active: failed (Result: exit-code) since Mon 2018-02-19 09:20:56
GMT; 12ms ago
 Docs: man:systemd-sysv-generator(8)
  Process: 6728 ExecStart=/etc/init.d/backuppc start (code=exited,
status=1/FAILURE)

Feb 19 09:20:56 backuppc01 systemd[1]: Starting LSB: Launch backuppc
server...
Feb 19 09:20:56 backuppc01 backuppc[6728]:  * Starting backuppc...
Feb 19 09:20:56 backuppc01 backuppc[6728]: BackupPC cannot be started
because important parameters are missing from config.pl.
Feb 19 09:20:56 backuppc01 backuppc[6728]: If you just upgraded
BackupPC, please update /etc/backuppc/config.pl.
Feb 19 09:20:56 backuppc01 systemd[1]: backuppc.service: Control
process exited, code=exited status=1
Feb 19 09:20:56 backuppc01 systemd[1]: Failed to start LSB: Launch
backuppc server.
Feb 19 09:20:56 backuppc01 systemd[1]: backuppc.service: Unit entered
failed state.
Feb 19 09:20:56 backuppc01 systemd[1]: backuppc.service: Failed with
result 'exit-code'.
dpkg: error processing package backuppc (--configure):
 subprocess installed post-installation script returned error exit
status 1
Errors were encountered while processing:
 backuppc
E: Sub-process /usr/bin/dpkg returned an error code (1)

 I'm not too sure about what I have done wrong?




It's worth looking at the config.pl to see what happened.  It may be 
zero length, or missing, or the 3->4 upgrade may have partially worked 
after all.  In any of these cases, you need only restore the config.pl 
from wherever you saved it, and you should be all set.


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-19 Thread Michael Stowe

On 2018-02-19 02:14, Marc Gilliatt wrote:

I apologise for the bombardment of email/replies. However, I am able
to get into the GUI now after playing about with backuppc.

I went to back up my Linux servers and they are failing every time I
try to do a full backup, I'm getting the same error for all my Linux
servers:

Backup aborted (fileListReceive failed)

However my Windows share is backing up just fine, I haven't received
any errors whatsoever and I've checked the top command and its running
a BackupPC_tarExt dump.

Surprisingly however, I’m still receiving the same failed error
message when I run _service backuppc status_?

-


The error you're receiving is fairly generic, you should be able to 
glean more from the details logs.  The big things which make rsync not 
work are usually authentication and configuration related, but I can't 
think of a good reason any of that would have changed during an up or 
downgrade.


The service definition is a different matter -- if I were to venture a 
guess, I'd suggest that the upgrade installed v4 service definitions, 
and the v3 reinstall did not touch them, for whatever reason (there are 
a number of possibilities:  neglecting to run systemctl daemon-reload, 
v3 doesn't bother if it sees they're already installed, v3 uses the sys5 
style instead of systemctl, or v3 tried but failed.)  If I were sorting 
this out on my systems, I'd probably extract the relevant scripts from 
the .deb file and install them by hand.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-18 Thread Michael Stowe

On 2018-02-16 01:05, Marc Gilliatt wrote:

I've just tried the _/configure.pl --batch --cgi-dir
/var/www/cgi-bin/BackupPC --data-dir /var/lib/backuppc --backuppc01
backuppc  --html-dir /var/www/html/BackupPC' _command, and it hasn't
worked I'm afraid, I tried reinstalling the Bakcuppc-41.3 tar file
again, and gave it the full correct paths, and when I try to restart
the service I'm still getting _Job for backuppc.service failed 
because

the control process exited with error code. See "systemctl status
backuppc.service" and "_journalctl_ -_xe_" for details. _With 
regards

to the SCGI port number, do I set it to -1 or give it a port number?

__And I'm still getting the internal error in the GUI. I don't
understand why this has happened, all I did was upgraded the 
software,

it looks like I'm going to have to re-raid my server and start
fresh!!!

Is there any way to roll back to my previous version of BackupPC
please?
-


Moving from an installation package to a source-based install can be a 
bit of a trial, depending on how much the norms of that distro and 
packaging system deviate from the tarball.  Version 4 is a fairly 
dramatic leap as well, which complicates things further.


First, though, I'll address your question and give you the good news:  
since you never actually launched BackupPC 4, your repository is instact 
and in the expected location for your distro.  Therefore, to revert to 
the previous version, you need only delete the tarball's files, and 
install the distro's version.


N.B. preserve your config.pl and return it to its original location, as 
your distro may assume it is a new installation and do god-knows-what to 
your config.pl.  Some are kinder than others.


--

The SCGI question is a bit harder to answer without knowing what other 
choices you've made.  Since I have set up SCGI, I match the port to what 
I have set up within Apache... which can be anything, as long as the 
number in Apache and in config.pl are the same.  If I had not set up 
SCGI, I'd set it to -1.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-15 Thread Michael Stowe

FROM: Michael Stowe <michael.st...@member.mensa.org>
SENT: 13 February 2018 09:01
TO: Marc Gilliatt
CC: General list for user discussion, questions and support
SUBJECT: Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

On 2018-02-13 00:34, Marc Gilliatt wrote:


So can I just install the tar file again then, and when it asks me
for the _Full path to existing main config.pl []? _should I put what
you had mentioned?


My suggestion is yes, give it a whirl.

More accurately, I'd suggest saving a copy of your current config.pl,
so you can try again if anything goes awry. Also, depending on what
procedure or script you're using, you may want to examine it to see if
it has options to update your config.pl or assume you need a new one,
as the one from which that command line came. It doesn't need to be
done at the same time as the tar installation, but it won't hurt to go
through the steps again, either.



On 2018-02-14 04:18, Marc Gilliatt wrote:

When I try and give it the full path to the config.pl path and I
press enter. It's just repeating the path I have given it, it's not
moving on to the next step of the installation?

 Example:

--> Full path to existing main config.pl []? /configure.pl --batch
--cgi-dir /var/www/cgi-bin/BackupPC --data-dir /var/lib/backuppc
--backuppc01 backuppc  --html-dir /var/www/html/BackupPC
The file '/configure.pl --batch --cgi-dir /var/www/cgi-bin/BackupPC
--data-dir /var/lib/backuppc --backuppc01 backuppc  --html-dir
/var/www/html/BackupPC' doesn't exist.

Is this a new installation or upgrade for BackupPC?  If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl).  Otherwise, just
hit return.

--> Full path to existing main config.pl [/configure.pl --batch
--cgi-dir /var/www/cgi-bin/BackupPC --data-dir /var/lib/backuppc
--backuppc01 backuppc  --html-dir /var/www/html/BackupPC]?
The file '/configure.pl --batch --cgi-dir /var/www/cgi-bin/BackupPC
--data-dir /var/lib/backuppc --backuppc01 backuppc  --html-dir
/var/www/html/BackupPC' doesn't exist.

Is this a new installation or upgrade for BackupPC?  If this is
an upgrade please tell me the full path of the existing BackupPC
configuration file (eg: /etc/BackupPC/config.pl).  Otherwise, just
hit return.

 It just keeps repeating itself?


I believe I misunderstood what you were asking.  When it asks for the 
full path, just give it the full path to your config.pl, which is likely 
to be something like /etc/BackupPC/config.pl -- I'm not sure what the 
Ubuntu convention is, so you might need to locate it first.  What I 
provided was a *command line* that would process a config.pl in the 
*current* directory, that you may need to tailor to your current paths.  
Sorry about the confusion.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-13 Thread Michael Stowe

On 2018-02-13 00:34, Marc Gilliatt wrote:

So can I just install the tar file again then, and when it asks me
for the _Full path to existing main config.pl []? _should I put what
you had mentioned?


My suggestion is yes, give it a whirl.

More accurately, I'd suggest saving a copy of your current config.pl, so 
you can try again if anything goes awry.  Also, depending on what 
procedure or script you're using, you may want to examine it to see if 
it has options to update your config.pl or assume you need a new one, as 
the one from which that command line came.  It doesn't need to be done 
at the same time as the tar installation, but it won't hurt to go 
through the steps again, either.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-12 Thread Michael Stowe

On 2018-02-12 12:18, Marc Gilliatt wrote:

Thanks for the quick response.

How do I update the config.pl file, please?


I'm not sure what upgrade instructions you followed, which may have more 
specific notes, but the usual course of action when installing from the 
tarball is this:


# When upgrading, use this instead:
# ./configure.pl --batch --config-path /etc/BackupPC/config.pl

I note that in one of the popular Ubuntu upgrade scripts, it's commented 
just like that, so it may have attempted to install as new.  I'm not 
sure what happens when that's attempted and a config.pl already exists, 
but if the above doesn't work (remove the # at the beginning of the 
second line, obviously) then you might want to move your config.pl to a 
backup and try this:


# When installing, use this
../configure.pl --batch --cgi-dir /var/www/cgi-bin/BackupPC --data-dir 
/var/lib/backuppc --hostname backuppc --html-dir /var/www/html/BackupPC 
--html-dir-url /BackupPC --install-dir /usr/local/BackupPC


Adjust the paths for your actual paths, of course.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Upgrading to backuppc-4.1.3 FAILED!

2018-02-12 Thread Michael Stowe

On 2018-02-12 01:00, Marc Gilliatt wrote:

I tried to download backuppc-4.1.3 tar file from the backuppc
website.

 The upgrade went fine, I went through the upgrade steps and no error
was shown, however, the backuppc.server won't start now?

 In the GUI, I'm getting.

ERROR: UNABLE TO CONNECT TO BACKUPPC SERVER
This CGI script (/backuppc/index.cgi) is unable to connect to the
BackupPC server on backuppc01 port -1.
The error was: unix connect: No such file or directory.
Perhaps the BackupPC server is not running or there is a configuration
error. Please report this to your Sys Admin.

 And when I try to run "service backuppc restart", Im getting.

 Job for backuppc.service failed because the control process exited
with error code. See "systemctl status backuppc.service" and
"journalctl -xe" for details.

 And when I run "systemctl status backuppc.service", Im getting.

backuppc.service - LSB: Launch backuppc server
Loaded: loaded (/etc/init.d/backuppc; bad; vendor preset: enabled)

Active: failed (Result: exit-code) since Wed 2018-01-31 09:33:33 GMT;
20s ago
Docs: man:systemd-sysv-generator(8)
 Process: 29163 ExecStop=/etc/init.d/backuppc stop (code=exited,
status=0/SUCCESS)

Process: 29425 ExecStart=/etc/init.d/backuppc start (code=exited,
status=1/FAILURE)
Jan 31 09:33:33 backuppc01 systemd[1]: Starting LSB: Launch backuppc
server...
 Jan 31 09:33:33 backuppc01 backuppc[29425]:  * Starting backuppc...

 Jan 31 09:33:33 backuppc01 backuppc[29425]: BackupPC cannot be
started because important parameters are missing from config.pl.

Jan 31 09:33:33 backuppc01 backuppc[29425]: If you just upgraded
BackupPC, please update /etc/backuppc/config.pl.

Jan 31 09:33:33 backuppc01 systemd[1]: backuppc.service: Control
process exited, code=exited status=1

Jan 31 09:33:33 backuppc01 systemd[1]: Failed to start LSB: Launch
backuppc server.

Jan 31 09:33:33 backuppc01 systemd[1]: backuppc.service: Unit entered
failed state.

Jan 31 09:33:33 backuppc01 systemd[1]: backuppc.service: Failed with
result 'exit-code'.

 I followed the upgrade instructions correctly.

 And when I go into the GUI

INTERNAL SERVER ERROR
The server encountered an internal error or misconfiguration and was
unable to complete your request.
Please contact the server administrator at webmaster@localhost to
inform them of the time this error occurred, and the actions you
performed just before this error.
More information about this error may be available in the server error
log.
-

_Apache/2.4.18 (Ubuntu) Server at backuppc01.dsdb.int Port 80_

 Before the upgrade, I was running smbd 4.3.11-Ubuntu and backuppc
3.3.1-2ubuntu3.3

 Any ideas or suggestions on how to fix this, please? This is our only
source of backing up our servers.

I have no working backuppc now...any ideas, please?


If I may draw your attention to these lines from your logs:

Jan 31 09:33:33 backuppc01 backuppc[29425]: BackupPC cannot be started 
because important parameters are missing from config.pl.
Jan 31 09:33:33 backuppc01 backuppc[29425]: If you just upgraded 
BackupPC, please update /etc/backuppc/config.pl.


The settings in config.pl have changed.  These will need to be updated 
before you will be able to start 4.x.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] [BackupPC-devel] SMB Windows share not backing up - NT_STATUS_ACCESS_DENIED listing \System Volume Information\SRM\*

2018-02-02 Thread Michael Stowe

On 2018-02-02 01:20, Marc Gilliatt wrote:

Can I please get an update on my issue, please?

 This is our only source of backing up our servers, we are not backing
up at the current moment because of this issue.

 As mentioned, I upgraded backuppc from 3.3.1 to 4.1.3 from which I
download the tar file from the backuppc website.

  However, when I see what version of backuppc I'm currently running
right now, it's still 3.3.1?

 I'm not sure what I've done wrong?

-


This will be somewhat backwards due to top-posting, but I'll start here 
-- the first thing you've done wrong is persisted in using the developer 
list, but it is a user issue.  You're much more likely to get faster and 
better responses on the correct list.  I've taken the liberty of 
crossposting this reply to the users list (see above) in the hopes that 
we can terminate this thread in the developer discussion, where it does 
not belong.


FROM: Marc Gilliatt <m.gilli...@live.co.uk>
SENT: 31 January 2018 10:17
TO: Michael Stowe; Developers discussion
SUBJECT: Re: [BackupPC-devel] SMB Windows share not backing up -
NT_STATUS_ACCESS_DENIED listing \System Volume Information\SRM\*

 I apologize for the multiple emails.

 I am now getting the following message when I go into the GUI

INTERNAL SERVER ERROR

The server encountered an internal error or misconfiguration and was
unable to complete your request.

Please contact the server administrator at webmaster@localhost to
inform them of the time this error occurred, and the actions you
performed just before this error.

More information about this error may be available in the server error
log.
-
 Apache/2.4.18 (Ubuntu) Server at backuppc01.dsdb.int Port 80

 Any ideas or suggestions on how to fix this, please? This is our only
source of backing up our servers.
-


That part about more information available in the server error log is 
usually a good clue -- what does the server error log say?



FROM: Marc Gilliatt <m.gilli...@live.co.uk>
SENT: 31 January 2018 09:39
TO: Michael Stowe; Developers discussion
SUBJECT: Re: [BackupPC-devel] SMB Windows share not backing up -
NT_STATUS_ACCESS_DENIED listing \System Volume Information\SRM\*

 I tried to download backuppc-4.1.3 from the backuppc website.

 The upgrade went fine, however, I backuppc.server won't start now?

 In the GUI, I'm getting.

 ERROR: UNABLE TO CONNECT TO BACKUPPC SERVER

 This CGI script (/backuppc/index.cgi) is unable to connect to the
BackupPC server on backuppc01 port -1.
The error was: unix connect: No such file or directory.
Perhaps the BackupPC server is not running or there is a configuration
error. Please report this to your Sys Admin.

 And when I try to run "service backuppc restart", Im getting.

 Job for backuppc.service failed because the control process exited
with error code. See "systemctl status backuppc.service" and
"journalctl -xe" for details.

 And when I run "systemctl status backuppc.service", Im getting.

backuppc.service - LSB: Launch backuppc server
Loaded: loaded (/etc/init.d/backuppc; bad; vendor preset: enabled)

Active: failed (Result: exit-code) since Wed 2018-01-31 09:33:33 GMT;
20s ago
Docs: man:systemd-sysv-generator(8)
 Process: 29163 ExecStop=/etc/init.d/backuppc stop (code=exited,
status=0/SUCCESS)

Process: 29425 ExecStart=/etc/init.d/backuppc start (code=exited,
status=1/FAILURE)
Jan 31 09:33:33 backuppc01 systemd[1]: Starting LSB: Launch backuppc
server...
 Jan 31 09:33:33 backuppc01 backuppc[29425]:  * Starting backuppc...

 Jan 31 09:33:33 backuppc01 backuppc[29425]: BackupPC cannot be
started because important parameters are missing from config.pl.

Jan 31 09:33:33 backuppc01 backuppc[29425]: If you just upgraded
BackupPC, please update /etc/backuppc/config.pl.

Jan 31 09:33:33 backuppc01 systemd[1]: backuppc.service: Control
process exited, code=exited status=1

Jan 31 09:33:33 backuppc01 systemd[1]: Failed to start LSB: Launch
backuppc server.

Jan 31 09:33:33 backuppc01 systemd[1]: backuppc.service: Unit entered
failed state.

Jan 31 09:33:33 backuppc01 systemd[1]: backuppc.service: Failed with
result 'exit-code'.

 I followed the upgrade instructions correctly.

I have no working backuppc now...any ideas, please?
-


I can't speak to which upgrade instructions you followed and whether or 
not you did so correctly, but this is telling you that you're trying to 
start a version 4 server with a version 3 config.pl, which probably 
isn't going to work well.



FROM: Marc Gilliatt <m.gilli...@live.co.uk>
SENT: 31 January 2018 08:37
TO: Michael Stowe
CC: Developers discussion
SUBJECT: Re: [BackupPC-devel] SMB Windows share not backing up -
NT_STATUS_ACCESS_DENIED listing \System Volume Information\SRM\*

 My distro is Ubuntu 16.04.3

 I try to upgrade just backuppc, and it's telling me that "backuppc is
already the 

Re: [BackupPC-users] Fedora 27 and SCGI

2018-01-01 Thread Michael Stowe

On 2017-12-31 19:45, Akibu Flash wrote:

Thanks @Michael Stowe.

Concerning the 9 port number, I only put 9 in the email as a
stand in number as I did not know if there was any security
ramification to disclosing the actual port publicly.  The port is
actually 10268.


If there are security implications from disclosure of a port publicly 
(security through obscurity) then you have much larger problems; at any 
rate, this is not related to your issue.



Concerning mod_scgi not being enabled, that may be the case, but I
don't know how to enable it.  I know that on a debian based system, I
would enable the module by use of "a2enmod".  However, there is no
such module enabler on Fedora and the apache documentation doesn't
really say how it is done.  Any suggestions as to the command to use?


I'm no Fedora expert, but I'm not aware of a Fedora repository that 
offers mod_scgi.  So you have a few options:


-) Locate a repository that does offer this (good luck!)
-) Build Apache (and mod_scgi) from source
-) Switch distros to one that offers mod_scgi
-) Use something other than Apache for scgi (e.g. lighthttpd)
-) Use something other than scgi
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Fedora 27 and SCGI

2017-12-31 Thread Michael Stowe

On 2017-12-31 17:51, Akibu Flash wrote:

I am running a Fedora 27 server and am attempting to get backuppc
working.  It installs without event using the command “dnf install
backuppc”.  I have also installed the apache webserver.  BackupPC
starts as expected with the command “systemctl start backuppc”.
My issue is that I am unable to get the CGI interface working.  I am
attempting to use SCGI as described in the manual.  I have input
"$Conf{SCGIServerPort} = 9; in my config.pl file.  Also, I have
input the following lines in my /etc/httpd/conf.d/BackupPC.conf file:

LoadModule proxy_scgi_module modules/mod_proxy_scgi.so

SCGIMount /BackupPC_Admin 127.0.0.1:9

AuthUserFile /etc/httpd/conf/passwd
AuthType basic
AuthName "backuppc"
require all granted



Port numbers are 16-bit integers, so you might want to start by picking 
a valid port within that range.



However, when trying to restart the apache server, I get the following
error message:

Invalid command 'SCGIMount', perhaps misspelled or defined by a module
not included in the server configuration
Dec 31 20:44:43 localhost.localdomain systemd[1]: httpd.service: Main
process exited, code=exited, status=1/FAILURE
Dec 31 20:44:43 localhost.localdomain systemd[1]: Failed to start The
Apache HTTP Server.
Dec 31 20:44:43 localhost.localdomain systemd[1]: httpd.service: Unit
entered failed state.
Dec 31 20:44:43 localhost.localdomain systemd[1]: httpd.service:
Failed with result 'exit-code'.

I am unclear where to go from here.  Any help is appreciated.  Thanks.


Akibu


That error is telling you that mod_scgi is not enabled.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] Win10 rsync

2017-12-07 Thread Michael Stowe

On 2017-12-07 07:08, Greg Harris wrote:

With the release of WSL in Win10, I thought I’d utilize the
functionality of WSL to get rsync up and running for backing up the
client.  However, as I’ve been working on getting it going, it seems
like WSL is like using a jackhammer when you need a light ball peen
hammer.  From what I can tell, it’s a full Ubuntu install that then
requires separate updates and permissions maintenance.  Essentially,
it seems that WSL  is just a VM with some fancy integration for file
and resource sharing.  Anybody done this?  Am I missing something and
making it harder than it should be?

Thanks,

 Greg Harris


It's not really a VM, per se -- it's better to think of it as a sort of 
Cygwin but with the goal of running unmodified Linux binaries rather 
than providing GNU tools that work with Windows.  On the plus side, this 
abstraction and ABI compatibility means that with WSL, you can copy over 
a binary from Ubuntu and it will run.  On the minus side, this separates 
each WSL binary from Windows file semantics and baffles them with the 
simplest Windows filesystem interactions.


In short, it's probably a *terrible* choice for backing up Windows 
files, but it's probably a good choice if your goal is to back up WSL 
files.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] how to solve an error exteracting error

2017-11-09 Thread Michael Stowe

On 2017-11-09 06:59, Mohammad Goodarzi wrote:

Hi,

What do you think I must show so that you can help me fixing the
problem?

Thanks


By way of imperfect analogy, let's say you've telephoned a mechanic and 
said your car doesn't start.  He's told you the most likely reason cars 
in general don't start, and now you've followed up with, "what else do 
you need to know?"


You haven't even mentioned what kind of car is involved, much less any 
surrounding context.  Have you driven the car off a cliff lately?  Did 
it ever start?  This isn't a fun guessing game for the mechanic.


By a similar token, you haven't said what version you're running, what 
operating systems are involved, or provided any kind of diagnostics 
whatsoever, or insight into the symptoms.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] how to solve an error exteracting error

2017-11-08 Thread Michael Stowe

On 2017-11-08 13:40, Mohammad Goodarzi wrote:

hi,

I get this error when I do ssh -l root myservice service ssh reload

ssh: unrecognized service


ssh is [probably] telling you that "service" isn't what your distro uses 
to reload services.  Since you haven't mentioned your Linux flavor -- or 
even *whether* you're using Linux -- it's very difficult to steer you in 
the direction of what exact command will reload ssh on the remote 
server.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BPC/rsync fails on huge directory

2017-11-03 Thread Michael Stowe

On 2017-11-03 04:40, infos+backu...@ba-cst.net wrote:

On 31/10/17 14:44, Jenkins, Wade William wrote:

What kind of failure are you seeing? What method are you using?


In the client bpc log, we have

- incr backup started for directory racine
- Got fatal error during xfer (rsync error: error in rsync protocol 
data

stream (code 12) at io.c(1556) [receiver=3.0.9.3])
- Backup aborted (rsync error: error in rsync protocol data stream 
(code

12) at io.c(1556) [receiver=3.0.9.3])

In the server's syslog, it's

kernel: [566863.920163] rsync_bpc[6209]: segfault at 7f6b29599e28 ip
0044ad0d sp 7ffdb7b005c0 error 4 in rsync_bpc[40+74000]

That sounds similar to the problem that I’ve been having since March 
that I can’t seem to sort out.  I’m using rsync backups, but trying 
tar backups has run into the same issue.  My backups will hang, 
seemingly indefinitely—I’ve had a backup sit unmoving for 7 days 
before.  Sometimes, killing that backup and running a new full will 
give me a reprieve.  But I’ve not been able to find or fix the 
underlying cause to this point.

On 31/10/17 18:01, Michael Stowe wrote:
Well, yes. rsync 3.1.x has a bug which might be directly affecting 
your transfer, and sounds related to your use case:


https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7cLPoeBV4fOdLf9yHXzdtaUlQr1SMje3Vyc54M-2BaQOv1npeWL3R2yAXh23Gynj2D-2Fw-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CCseQKZkmdXms9wDpB15ATwvqwlBC3iXHybKz5CErzOHNFaLJkpmEuAiHmUWdQXLV1lhO9sKiMRHcAA9k-2BIindHGZ8wUAv9Dd392Mh8rbGJIoA4fInHAtQdmP8B4bxVZl5wbkRMsT3xG5tJvM6DrbLRxoY7IgoXGGV05ZDj5TLdpQtBrRabCMMcEKQWpa9ofU

If this is the case, either upgrading (or downgrading) should take 
care of the issue.


It's not a process hanging, just plain crashing (which may come from 
the

same bug or not). Since it's rsync_bpc that goes over the cliff, it may
not be related to an rsync bug.


A crash like that is probably not the same bug, sorry for 
misunderstanding.  For the record, that bug manifests itself only as an 
apparent hang (though rsync does seem to spend its time re-reading the 
same files.)
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


Re: [BackupPC-users] BPC/rsync fails on huge directory

2017-10-31 Thread Michael Stowe

On 2017-10-31 04:58, infos+backu...@ba-cst.net wrote:

Hello,

We are happily using BPC 4 since almost a year, without problems.

Except since a few days ago, where one system backup began to fail
consistently. After some tests, it seems the failure occurs when 
backing

a (recent and) huge directory containing 111000+ files (some tools are
just weird). Those files are not too big, amounting to a little bit 
less

than 550 Mb.

Does this ring a bell to anyone ? And is there a way out to get the
backup all right, knowing that reorganizing the directory content is 
not

possible ?

Tia,


Well, yes.  rsync 3.1.x has a bug which might be directly affecting your 
transfer, and sounds related to your use case:


https://u2182357.ct.sendgrid.net/wf/click?upn=rBK8reUlX8Sxr7Iz1fV-2F7cLPoeBV4fOdLf9yHXzdtaUlQr1SMje3Vyc54M-2BaQOv1npeWL3R2yAXh23Gynj2D-2Fw-3D-3D_OypFYCWzG5ApGW-2FFpGTxc4RCS9eud0Dl1htN5rYoUZ8To4zeNUFBkAGI3hzer91CWtMNj2C6eNI75Fh522o8EfUb1cKCkhlkRUkBO4B1AKoO57PhwhW9LrH6IrAsPD6sdn0jNphmZvh-2BxNiTx53MOuwZUX-2B7O9jMvFoKoIYlLoeuAsTI9gS6vr0pRbKiNQHANe1d8IiTnJzbkCyEvX-2FT-2FpYrSB5xuGXecLu67CD7qsA-3D

If this is the case, either upgrading (or downgrading) should take care 
of the issue.


I note that there are (of course) other things that can cause the same 
symptoms, from a poor network connection to disk corruption, but I'd 
start with the above.
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/


  1   2   3   4   5   >