Re: [BackupPC-users] full backup of windows client succeeds, incremental fails

2016-06-09 Thread Carl Wilhelm Soderstrom 
On 06/09 02:36 , Les Mikesell wrote:
> It might just be somewhat different timing for that host too - that
> is, there may be a large number of unchanging files or it has slow
> drives that make it take a longer time to find something that changed.

I don't think so.
At this point I'm starting to think it's a networking problem. Lots of
people have been plugging and unplugging networking cables, moving switches
and machines around, and otherwise fiddling with stuff - generally with very
little thought given to network diameter, bisectional bandwidth, etc. I
wouldn't be surprised if BackupPC is just exposing a deeper problem.

-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] full backup of windows client succeeds, incremental fails

2016-06-09 Thread Michael Stowe 
On 2016-06-09 12:43, Carl Wilhelm Soderstrom wrote:
> I have seen it happen on a couple of occasions where a Windows machine
> (backed up via Cygwin rsyncd, not the minimal rsyncd off the SF page) 
> will
> suddenly stop working for incremental backups. Full backups will 
> continue to
> work, but incrementals will start failing with a PIPE error. For 
> example,
> here's a run where I was trying to debug with
> '/usr/share/backuppc/bin/BackupPC_dump -i -v host.example.com':
> 
>   create d 770 544/1049089   0 Program Files (x86)/Common
> Files/Adobe/AAMUpdaterInventory/1.0/AdobeIDCC2015AppLangen_US-11.0
>   create d 770 544/1049089   0 Program Files (x86)/Common
> Files/Adobe/AAMUpdaterInventory/1.0/AdobeIDCC2015AppLangen_US-11.0/11.2.0.100
> Done: 0 files, 0 bytes
> Got fatal error during xfer (aborted by signal=PIPE)
> Backup aborted by user signal
> dump failed: aborted by signal=PIPE
> 
> On the rsyncd side, the rsyncd.log says:
> 2016/06/09 11:26:23 [1728] connect from backuppc.example.com
> (10.77.87.121)
> 2016/06/09 11:26:23 [1728] rsync on . from
> rsyncba...@backuppc.example.com (10.77.87.121)
> 2016/06/09 11:26:23 [1728] building file list
> 2016/06/09 11:30:25 [1728] rsync: [sender] write error: Broken pipe 
> (32)
> 2016/06/09 11:30:25 [1728] rsync error: error in socket IO (code 10) at
> io.c(820) [sender=3.1.2]
> 
> A run with '/usr/share/backuppc/bin/BackupPC_dump -f -v 
> host.example.com'
> will run to completion, but the incremental fails, even if it's run 
> soon
> after a full is run.

This implies that there's something in the comparison of files that's 
breaking the transfer that isn't breaking on a full copy.  I note that 
this can happen if the BackupPC side has performance, timing, or 
corruption issues; the clue is that it happens to incrementals and not 
fulls, so it's not likely to be on the Windows' side.

> I've tried updating cygwin, and the rsyncd package was updated. The 
> fact
> that I was able to upgrade a very large number of packages in the 
> cygwin
> installation, indicates to me that it wasn't a cygwin upgrade which 
> broke
> things.

I've had problems with certain versions of cygwin before; the "broken 
pipe" error is very generic and unhelpful, and can happen when it chokes 
on umlauts or when it times out due to fragmentation or due to build 
issues.  If you'd like to check it against a known good version, I do 
maintain one here:

https://github.com/mwstowe/BackupPC-Client/tree/master/backuppc

Note that rsync and its libraries are only a grand total of five files 
(or fewer.)  While you're at it, you might want to pull the rsyncd.conf, 
which has a few options that correct the more sporadic rsync/Windows 
issues.

> This only happens to an occasional Windows machine, and it happens 
> without
> any obvious cause. Not all Windows machines will break, just one.
> 
> Any suggestions on how to debug/fix this? the obvious workaround is 
> just to
> only do full backups, and I've done that in the past, but if I can 
> solve
> this a better way I'd like to do so.
> 
> The next step would be to uninstall and reinstall cygwin. It's possible 
> the
> problem is some corruption in a file which hasn't been upgraded and 
> thus
> repaired.
> --
> Carl Soderstrom
> Systems Administrator
> Real-Time Enterprises
> www.real-time.com


--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] full backup of windows client succeeds, incremental fails

2016-06-09 Thread Les Mikesell 
On Thu, Jun 9, 2016 at 2:07 PM, Carl Wilhelm Soderstrom
 wrote:
> On 06/09 01:50 , Les Mikesell wrote:
>> Sometimes this is caused by a nat router or stateful firewall
>> (possibly even host firewall software) timing out and breaking a
>> connection due to too much idle time in the traffic.  If you are
>> running over ssh you can usually fix it by enabling keepalives - not
>> sure about the standalone rsyncd options.
>
> Good thought. However, it consistently happens to only one host, doesn't
> seem to correspond to having moved that host to another side of the
> firewall, and both the client and the backup server are on the same
> broadcast domain.
>
> That said, it's at a remote location and I can't trace the cabling myself,
> so it's possible there's a switch or something which is a bit more 'clever'
> than it should be, and is causing this.

It might just be somewhat different timing for that host too - that
is, there may be a large number of unchanging files or it has slow
drives that make it take a longer time to find something that changed.

-- 
   Les Mikesell
 lesmikes...@gmail.com

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] full backup of windows client succeeds, incremental fails

2016-06-09 Thread Carl Wilhelm Soderstrom 
On 06/09 01:50 , Les Mikesell wrote:
> Sometimes this is caused by a nat router or stateful firewall
> (possibly even host firewall software) timing out and breaking a
> connection due to too much idle time in the traffic.  If you are
> running over ssh you can usually fix it by enabling keepalives - not
> sure about the standalone rsyncd options.

Good thought. However, it consistently happens to only one host, doesn't
seem to correspond to having moved that host to another side of the
firewall, and both the client and the backup server are on the same
broadcast domain.

That said, it's at a remote location and I can't trace the cabling myself,
so it's possible there's a switch or something which is a bit more 'clever'
than it should be, and is causing this.

-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

Re: [BackupPC-users] full backup of windows client succeeds, incremental fails

2016-06-09 Thread Les Mikesell 
On Thu, Jun 9, 2016 at 12:43 PM, Carl Wilhelm Soderstrom
 wrote:
> I have seen it happen on a couple of occasions where a Windows machine
> (backed up via Cygwin rsyncd, not the minimal rsyncd off the SF page) will
> suddenly stop working for incremental backups. Full backups will continue to
> work, but incrementals will start failing with a PIPE error. For example,
> here's a run where I was trying to debug with
> '/usr/share/backuppc/bin/BackupPC_dump -i -v host.example.com':
>
>   create d 770 544/1049089   0 Program Files (x86)/Common
> Files/Adobe/AAMUpdaterInventory/1.0/AdobeIDCC2015AppLangen_US-11.0
>   create d 770 544/1049089   0 Program Files (x86)/Common
> Files/Adobe/AAMUpdaterInventory/1.0/AdobeIDCC2015AppLangen_US-11.0/11.2.0.100
> Done: 0 files, 0 bytes
> Got fatal error during xfer (aborted by signal=PIPE)
> Backup aborted by user signal
> dump failed: aborted by signal=PIPE

Sometimes this is caused by a nat router or stateful firewall
(possibly even host firewall software) timing out and breaking a
connection due to too much idle time in the traffic.  If you are
running over ssh you can usually fix it by enabling keepalives - not
sure about the standalone rsyncd options.

-- 
Les Mikesell
  lesmikes...@gmail.com

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/

[BackupPC-users] full backup of windows client succeeds, incremental fails

2016-06-09 Thread Carl Wilhelm Soderstrom 
I have seen it happen on a couple of occasions where a Windows machine
(backed up via Cygwin rsyncd, not the minimal rsyncd off the SF page) will
suddenly stop working for incremental backups. Full backups will continue to
work, but incrementals will start failing with a PIPE error. For example,
here's a run where I was trying to debug with
'/usr/share/backuppc/bin/BackupPC_dump -i -v host.example.com':

  create d 770 544/1049089   0 Program Files (x86)/Common
Files/Adobe/AAMUpdaterInventory/1.0/AdobeIDCC2015AppLangen_US-11.0
  create d 770 544/1049089   0 Program Files (x86)/Common
Files/Adobe/AAMUpdaterInventory/1.0/AdobeIDCC2015AppLangen_US-11.0/11.2.0.100
Done: 0 files, 0 bytes
Got fatal error during xfer (aborted by signal=PIPE)
Backup aborted by user signal
dump failed: aborted by signal=PIPE

On the rsyncd side, the rsyncd.log says:
2016/06/09 11:26:23 [1728] connect from backuppc.example.com
(10.77.87.121)
2016/06/09 11:26:23 [1728] rsync on . from
rsyncba...@backuppc.example.com (10.77.87.121)
2016/06/09 11:26:23 [1728] building file list
2016/06/09 11:30:25 [1728] rsync: [sender] write error: Broken pipe (32)
2016/06/09 11:30:25 [1728] rsync error: error in socket IO (code 10) at
io.c(820) [sender=3.1.2]

A run with '/usr/share/backuppc/bin/BackupPC_dump -f -v host.example.com'
will run to completion, but the incremental fails, even if it's run soon
after a full is run.

I've tried updating cygwin, and the rsyncd package was updated. The fact
that I was able to upgrade a very large number of packages in the cygwin
installation, indicates to me that it wasn't a cygwin upgrade which broke
things.

This only happens to an occasional Windows machine, and it happens without
any obvious cause. Not all Windows machines will break, just one. 

Any suggestions on how to debug/fix this? the obvious workaround is just to
only do full backups, and I've done that in the past, but if I can solve
this a better way I'd like to do so.

The next step would be to uninstall and reinstall cygwin. It's possible the
problem is some corruption in a file which hasn't been upgraded and thus
repaired.
-- 
Carl Soderstrom
Systems Administrator
Real-Time Enterprises
www.real-time.com

--
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are 
consuming the most bandwidth. Provides multi-vendor support for NetFlow, 
J-Flow, sFlow and other flows. Make informed decisions using capacity 
planning reports. https://ad.doubleclick.net/ddm/clk/305295220;132659582;e
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/