Re: [BackupPC-users] busybox tar problem?
Hi,
Ward... James Ward wrote on 2008-08-22 08:30:22 -0700 [Re: [BackupPC-users]
busybox tar problem?]:
> On Aug 21, 2008, at 7:04 PM, Holger Parplies wrote:
> >Ward... James Ward wrote on 2008-08-19 16:02:16 -0700 [Re: [BackupPC-
> >users] busybox tar problem?]:
> >>$Conf{PingCmd} = '/usr/local/bin/check_ssh $host';
> >>[...]
> >>$Conf{BackupFilesOnly} = '/etc';
> >>
> >>When the last line contains a '/', I get "no ping response".
> >>When I leave it out, I get "No files dumped for share /".
> >>[and sometimes] I get "lost network connection during backup"?
> >
> >That most certainly has nothing to do with the '/', because the ping
> >command
> >is run long before the tar command line is built.
actually, there is a pattern, and your error messages are incomplete (you
don't just get one message, you get more in your XferLOG, and more than the
one message is relevant). I am convinced that check_ssh is not working
properly for whatever reason (see below).
If it fails initially, you get "no ping response". If it works initially, you
apparently get "No files dumped for share /" (in *both* cases - '/etc' and
'etc', as they're both changed to './etc' by BackupPC::Xfer::Tar, and there's
no reason why an identical command line should behave differently depending on
the contents of an unrelated configuration file on a remote host), which in
turn triggers another "ping" check. If that fails, you get "lost network
connection ..." (and if it doesn't fail, you don't). So, we're back to the
question I forgot to ask initially: what's in /etc/config/users/backup/tarSend?
What shell is it run with, busybox ash? And while we're at it:
$Conf{TarFullArgs}, $Conf{TarIncrArgs} use the default values? What command is
actually executed (XferLOG)? What happens if you pipe that command into
'tar tvf -' on the command line?
> >What does /usr/local/bin/check_ssh contain?
>
> /usr/local/bin/check_ssh is a nagios (binary) plugin that checks for
> ssh connectivity. I use it on all my linux boxes as we have ping
> blocked by the firewall.
>
> [...]
>
> Well, I'd like to think it was that simple, but the network is 1G
> ethernet hard wired.
That's a contradiction. Either you have a firewall or a simple hard wired
ethernet. A firewall can and will limit what goes across the wire - that's its
purpose. It's not uncommon to limit more than you really want to (and that's
better than limitting less than you really want to).
First of all: I don't know check_ssh. Does it reliably return exit code 0 for
success and nonzero for failure? Can you run it repeatedly from a shell
without getting a failure? Try something like
% for i in `seq 1 100`; do
> /usr/local/bin/check_ssh $host || echo "failed on iteration $i"
> done
Yes, that's a tight loop on purpose.
> And these results are 100% repeatable.
If that is true, you must have put considerable effort into verifying it.
There are up to 4 invocations of the ping command if I counted correctly. It
is obviously failing sometimes - the occurrences of "no ping response" and
"lost network connection ..." prove that. You can get more information on
which ones are failing by running BackupPC_dump with a -v option.
> I gave up and am just running a tar via cron.
Presuming you are not satisfied with that solution.
Regards,
Holger
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] busybox tar problem?
On Aug 21, 2008, at 7:04 PM, Holger Parplies wrote:
Hi,
Ward... James Ward wrote on 2008-08-19 16:02:16 -0700 [Re: [BackupPC-
users] busybox tar problem?]:
But here's the problem I'm currently struggling with, here's the
config file for this client:
$Conf{TarClientCmd} = '$sshPath -q -x -n -l backup $host /usr/bin/env
LC_ALL=C /etc/config/users/backup/tarSend -c -v -f - -C $shareName+';
$Conf{PingCmd} = '/usr/local/bin/check_ssh $host';
$Conf{XferMethod} = "tar";
$Conf{BackupFilesOnly} = '/etc';
When the last line contains a '/', I get "no ping response".
That most certainly has nothing to do with the '/', because the ping
command
is run long before the tar command line is built. What does
/usr/local/bin/check_ssh contain?
/usr/local/bin/check_ssh is a nagios (binary) plugin that checks for
ssh connectivity. I use it on all my linux boxes as we have ping
blocked by the firewall.
When I leave it out, I get "No files dumped for share /". What
gives?
All of it sounds like intermittent network failures. What did you
say about
the network? WLAN with 20 m of concrete between machines? Broken
cable/switch/multiport repeater? Arp poisoning device? When did this
problem
start? What piece of hardware (or software) did you change shortly
before?
What does a simple 'ping' say (when run for a while)? A flood ping?
A command
line rsync (or tar ... "ssh -l backup $host tar cf - / > /dev/null")?
Well, I'd like to think it was that simple, but the network is 1G
ethernet hard wired. And these results are 100% repeatable. I gave
up and am just running a tar via cron.
Regards,
Holger
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] busybox tar problem?
Hi,
Ward... James Ward wrote on 2008-08-19 16:02:16 -0700 [Re: [BackupPC-users]
busybox tar problem?]:
> Okay, here's more information:
>
> Here's the script I'm running on the BusyBox Arm system:
>
> #!/bin/sh -f
> /bin/tar $*
> exec 1>&2
> echo 'Total bytes written: 1863680 (1.8MiB, 12MiB/s)'
>
> I believe this sends the output of the echo to STDERR?
yes, much the same as (but more complicated to understand than)
echo '...' >&2
> But here's the problem I'm currently struggling with, here's the
> config file for this client:
>
> $Conf{TarClientCmd} = '$sshPath -q -x -n -l backup $host /usr/bin/env
> LC_ALL=C /etc/config/users/backup/tarSend -c -v -f - -C $shareName+';
> $Conf{PingCmd} = '/usr/local/bin/check_ssh $host';
> $Conf{XferMethod} = "tar";
> $Conf{BackupFilesOnly} = '/etc';
>
> When the last line contains a '/', I get "no ping response".
That most certainly has nothing to do with the '/', because the ping command
is run long before the tar command line is built. What does
/usr/local/bin/check_ssh contain?
> When I leave it out, I get "No files dumped for share /". What gives?
All of it sounds like intermittent network failures. What did you say about
the network? WLAN with 20 m of concrete between machines? Broken
cable/switch/multiport repeater? Arp poisoning device? When did this problem
start? What piece of hardware (or software) did you change shortly before?
What does a simple 'ping' say (when run for a while)? A flood ping? A command
line rsync (or tar ... "ssh -l backup $host tar cf - / > /dev/null")?
Regards,
Holger
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] busybox tar problem?
And with this config file:
$Conf{TarClientCmd} = '$sshPath -q -x -n -l backup $host /usr/bin/env
LC_ALL=C /etc/config/users/backup/tarSend -c -v -f - -C /';
$Conf{PingCmd} = '/usr/local/bin/check_ssh $host';
$Conf{XferMethod} = "tar";
$Conf{BackupFilesOnly} = 'etc';
I get "lost network connection during backup"?
Ward... James Ward
[EMAIL PROTECTED]
Linux Sysadmin
(520) 290-0910x268
ICQ: 201663408
On Aug 19, 2008, at 4:02 PM, Ward... James Ward wrote:
Okay, here's more information:
Here's the script I'm running on the BusyBox Arm system:
#!/bin/sh -f
/bin/tar $*
exec 1>&2
echo 'Total bytes written: 1863680 (1.8MiB, 12MiB/s)'
I believe this sends the output of the echo to STDERR?
But here's the problem I'm currently struggling with, here's the
config file for this client:
$Conf{TarClientCmd} = '$sshPath -q -x -n -l backup $host /usr/bin/
env LC_ALL=C /etc/config/users/backup/tarSend -c -v -f - -C
$shareName+';
$Conf{PingCmd} = '/usr/local/bin/check_ssh $host';
$Conf{XferMethod} = "tar";
$Conf{BackupFilesOnly} = '/etc';
When the last line contains a '/', I get "no ping response". When I
leave it out, I get "No files dumped for share /". What gives?
Ward... James Ward
[EMAIL PROTECTED]
Linux Sysadmin
(520) 290-0910x268
ICQ: 201663408
On Aug 19, 2008, at 12:41 PM, Craig Barratt wrote:
James writes:
I was writing the bogus line to stdout. Does it need to go to
stderr?
Yes.
Craig
-
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
Build the coolest Linux based applications with Moblin SDK & win
great prizes
Grand prize is a trip for two to an Open Source event anywhere in
the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] busybox tar problem?
Okay, here's more information:
Here's the script I'm running on the BusyBox Arm system:
#!/bin/sh -f
/bin/tar $*
exec 1>&2
echo 'Total bytes written: 1863680 (1.8MiB, 12MiB/s)'
I believe this sends the output of the echo to STDERR?
But here's the problem I'm currently struggling with, here's the
config file for this client:
$Conf{TarClientCmd} = '$sshPath -q -x -n -l backup $host /usr/bin/env
LC_ALL=C /etc/config/users/backup/tarSend -c -v -f - -C $shareName+';
$Conf{PingCmd} = '/usr/local/bin/check_ssh $host';
$Conf{XferMethod} = "tar";
$Conf{BackupFilesOnly} = '/etc';
When the last line contains a '/', I get "no ping response". When I
leave it out, I get "No files dumped for share /". What gives?
Ward... James Ward
[EMAIL PROTECTED]
Linux Sysadmin
(520) 290-0910x268
ICQ: 201663408
On Aug 19, 2008, at 12:41 PM, Craig Barratt wrote:
James writes:
I was writing the bogus line to stdout. Does it need to go to
stderr?
Yes.
Craig
-
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/___
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
List:https://lists.sourceforge.net/lists/listinfo/backuppc-users
Wiki:http://backuppc.wiki.sourceforge.net
Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] busybox tar problem?
gnu tar can be compiled for arm cpus. do you have a space issue that you cant use full gnu tar? I suggest you ditch busybox tar and move to gnu. this would be your simplest solution. unless you lack a machine with gcc and make to build it on but any linux machine with those tools should be able to cross-compile for arm. On Mon, Aug 18, 2008 at 11:51 PM, Craig Barratt < [EMAIL PROTECTED]> wrote: > James writes: > > > I'm trying to back up some Arm processor console servers which only > > have busybox tar available. Busybox tar does not support --totals > > and I THINK this is why the backups are failing. I tried writing a > > wrapper script that spits out a bogus --totals line, but so far, no > > luck. Any ideas? > > Yes, most likely that is it. Does the script write the totals > output to stdout or stderr? > > You can also modify the code to not expect the totals line. > > Craig > > - > This SF.Net email is sponsored by the Moblin Your Move Developer's > challenge > Build the coolest Linux based applications with Moblin SDK & win great > prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > ___ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List:https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki:http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] busybox tar problem?
James writes: > I'm trying to back up some Arm processor console servers which only > have busybox tar available. Busybox tar does not support --totals > and I THINK this is why the backups are failing. I tried writing a > wrapper script that spits out a bogus --totals line, but so far, no > luck. Any ideas? Yes, most likely that is it. Does the script write the totals output to stdout or stderr? You can also modify the code to not expect the totals line. Craig - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/ ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] busybox tar problem?
Hi, I'm trying to back up some Arm processor console servers which only have busybox tar available. Busybox tar does not support --totals and I THINK this is why the backups are failing. I tried writing a wrapper script that spits out a bogus --totals line, but so far, no luck. Any ideas? Thanks in advance, James Ward... James Ward [EMAIL PROTECTED] Linux Sysadmin (520) 290-0910x268 ICQ: 201663408 - This SF.Net email is sponsored by the Moblin Your Move Developer's challenge Build the coolest Linux based applications with Moblin SDK & win great prizes Grand prize is a trip for two to an Open Source event anywhere in the world http://moblin-contest.org/redirect.php?banner_id=100&url=/___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
