The estimate took more than 6 hours!
After how much time do you get the timeout error?
What is your etimeout setting?
You could tried faster estimate method: calcsize or server.
Jean-Louis
On 01/21/2015 03:52 PM, Chris Hoogendyk wrote:
First note that regardless of the timing for any particular DLE,
*every* single DLE for this one host is failing, while all other hosts
are getting fully backed up without any trouble.
On the host that is failing, there are no senbackup debug files since
it started failing.
On that same host, the sendsize debug file from last night includes
(this is just a segment):
"/tmp/amanda/client/daily/sendsize.20150120233005.debug" 1165 lines,
145734 characters
Tue Jan 20 23:30:05 2015: thd-32a58: sendsize: pid 11633 ruid 555 euid
555 version 3.3.2: start at Tue Jan 20 23:30:05 2015
Tue Jan 20 23:30:05 2015: thd-32a58: sendsize: version 3.3.2
Tue Jan 20 23:30:05 2015: thd-32a58: sendsize: pid 11633 ruid 555 euid
555 version 3.3.2: rename at Tue Jan 20 23:30:05 2015
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: waiting for any
estimate child: 2 running
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: calculating for amname
/, dirname /, spindle 100 DUMP
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: getting size via dump
for / level 0
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: calculating for amname
/export/baja, dirname /export/baja, spindle 45010 GNUTAR
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: getting size via gnutar
for /export/baja level 0
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: calculating for device
/dev/rdsk/c1t0d0s0 with ufs
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: running
"/usr/local/etc/amanda/tools/ufsdump 0Ssf 1048576 - /dev/rdsk/c1t0d0s0"
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: running
/usr/local/libexec/amanda/killpgrp
Tue Jan 20 23:30:08 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /export/baja --one-file-system --listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_export_baja_0.new --sparse
--ignore-failed-read
--totals ." in pipeline
Tue Jan 20 23:30:19 2015: thd-32a58: sendsize: 13847108608
Tue Jan 20 23:30:19 2015: thd-32a58: sendsize: .....
Tue Jan 20 23:30:19 2015: thd-32a58: sendsize: estimate time for /
level 0: 11.540
Tue Jan 20 23:30:19 2015: thd-32a58: sendsize: estimate size for /
level 0: 13522567 KB
Tue Jan 20 23:30:19 2015: thd-32a58: sendsize: asking killpgrp to
terminate
Tue Jan 20 23:30:20 2015: thd-32a58: sendsize: getting size via dump
for / level 1
Tue Jan 20 23:30:20 2015: thd-32a58: sendsize: calculating for device
/dev/rdsk/c1t0d0s0 with ufs
Tue Jan 20 23:30:20 2015: thd-32a58: sendsize: running
"/usr/local/etc/amanda/tools/ufsdump 1Ssf 1048576 - /dev/rdsk/c1t0d0s0"
Tue Jan 20 23:30:20 2015: thd-32a58: sendsize: running
/usr/local/libexec/amanda/killpgrp
Tue Jan 20 23:32:33 2015: thd-32a58: sendsize: 1461860352
Tue Jan 20 23:32:33 2015: thd-32a58: sendsize: .....
Tue Jan 20 23:32:33 2015: thd-32a58: sendsize: estimate time for /
level 1: 133.065
Tue Jan 20 23:32:33 2015: thd-32a58: sendsize: estimate size for /
level 1: 1427598 KB
Tue Jan 20 23:32:33 2015: thd-32a58: sendsize: asking killpgrp to
terminate
Tue Jan 20 23:32:34 2015: thd-32a58: sendsize: done with amname /
dirname / spindle 100
Tue Jan 20 23:32:34 2015: thd-32a58: sendsize: waiting for any
estimate child: 2 running
Tue Jan 20 23:32:34 2015: thd-32a58: sendsize: calculating for amname
/archive, dirname /archive, spindle 100 GNUTAR
Tue Jan 20 23:32:34 2015: thd-32a58: sendsize: getting size via gnutar
for /archive level 0
Tue Jan 20 23:32:34 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /archive --one-file-system --listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_archive_0.new
--sparse --ignore-failed-read --totals
." in pipeline
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: Total bytes written:
10917795840 (11GiB, 63MiB/s)
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: .....
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: estimate time for
/export/baja level 0: 169.954
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: estimate size for
/export/baja level 0: 10661910 KB
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: waiting for runtar
"/export/baja" child
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: after runtar
/export/baja wait
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: getting size via gnutar
for /export/baja level 1
Tue Jan 20 23:32:58 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /export/baja --one-file-system --listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_export_baja_1.new --sparse
--ignore-failed-read
--totals ." in pipeline
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: Total bytes written:
133120 (130KiB, 39KiB/s)
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: .....
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: estimate time for
/export/baja level 1: 3.525
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: estimate size for
/export/baja level 1: 130 KB
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: waiting for runtar
"/export/baja" child
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: after runtar
/export/baja wait
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: done with amname
/export/baja dirname /export/baja spindle 45010
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: waiting for any
estimate child: 2 running
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: calculating for amname
/export/barbados, dirname /export/barbados, spindle 45010 GNUTAR
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: getting size via gnutar
for /export/barbados level 0
Tue Jan 20 23:33:01 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /export/barbados --one-file-system
--listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_export_barbados_0.new
--sparse --ignore-fail
ed-read --totals ." in pipeline
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: Total bytes written:
11691386880 (11GiB, 51MiB/s)
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: .....
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: estimate time for
/export/barbados level 0: 222.591
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: estimate size for
/export/barbados level 0: 11417370 KB
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: waiting for runtar
"/export/barbados" child
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: after runtar
/export/barbados wait
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: getting size via gnutar
for /export/barbados level 1
Tue Jan 20 23:36:44 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /export/barbados --one-file-system
--listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_export_barbados_1.new
--sparse --ignore-fail
ed-read --totals ." in pipeline
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: Total bytes written:
153600 (150KiB, 125KiB/s)
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: .....
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: estimate time for
/export/barbados level 1: 1.378
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: estimate size for
/export/barbados level 1: 150 KB
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: waiting for runtar
"/export/barbados" child
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: after runtar
/export/barbados wait
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: done with amname
/export/barbados dirname /export/barbados spindle 45010
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: waiting for any
estimate child: 2 running
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: calculating for amname
/export/bermuda, dirname /export/bermuda, spindle 45010 GNUTAR
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: getting size via gnutar
for /export/bermuda level 0
Tue Jan 20 23:36:45 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /export/bermuda --one-file-system
--listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_export_bermuda_0.new
--sparse --ignore-failed
-read --totals ." in pipeline
SKIPPING A THOUSAND OR SO LINES . . . .
Wed Jan 21 04:40:50 2015: thd-32a58: sendsize: .....
Wed Jan 21 04:40:50 2015: thd-32a58: sendsize: estimate time for
/u1/home/micro/./k level 0: 3245.840
Wed Jan 21 04:40:50 2015: thd-32a58: sendsize: estimate size for
/u1/home/micro/./k level 0: 45555590 KB
Wed Jan 21 04:40:50 2015: thd-32a58: sendsize: waiting for runtar
"/u1/home/micro/./k" child
Wed Jan 21 04:40:50 2015: thd-32a58: sendsize: after runtar
/u1/home/micro/./k wait
Wed Jan 21 04:40:50 2015: thd-32a58: sendsize: getting size via gnutar
for /u1/home/micro/./k level 1
Wed Jan 21 04:40:51 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /u1/home/micro --one-file-system
--listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_u1_home_micro_._k_1.new
--sparse --ignore-fail
ed-read --totals --files-from
/tmp/amanda/sendsize._u1_home_micro_._k.20150121044050.include" in
pipeline
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: Total bytes written:
6410967040 (6.0GiB, 9.6MiB/s)
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: .....
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: estimate time for
/u1/home/micro/./k level 1: 640.501
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: estimate size for
/u1/home/micro/./k level 1: 6260710 KB
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: waiting for runtar
"/u1/home/micro/./k" child
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: after runtar
/u1/home/micro/./k wait
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: done with amname
/u1/home/micro/./k dirname /u1/home/micro spindle 506
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: waiting for any
estimate child: 1 running
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: calculating for amname
/u1/home/micro/./l-z, dirname /u1/home/micro, spindle 506 GNUTAR
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: getting size via gnutar
for /u1/home/micro/./l-z level 0
Wed Jan 21 04:51:31 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /u1/home/micro --one-file-system
--listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_u1_home_micro_._l-z_0.new
--sparse --ignore-fa
iled-read --totals --files-from
/tmp/amanda/sendsize._u1_home_micro_._l-z.20150121045131.include" in
pipeline
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: Total bytes written:
18660341760 (18GiB, 13MiB/s)
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: .....
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: estimate time for
/u1/home/micro/./l-z level 0: 1475.354
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: estimate size for
/u1/home/micro/./l-z level 0: 18222990 KB
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: waiting for runtar
"/u1/home/micro/./l-z" child
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: after runtar
/u1/home/micro/./l-z wait
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: getting size via gnutar
for /u1/home/micro/./l-z level 1
Wed Jan 21 05:16:06 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /u1/home/micro --one-file-system
--listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_u1_home_micro_._l-z_1.new
--sparse --ignore-fa
iled-read --totals --files-from
/tmp/amanda/sendsize._u1_home_micro_._l-z.20150121051606.include" in
pipeline
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: Total bytes written:
8945582080 (8.4GiB, 11MiB/s)
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: .....
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: estimate time for
/u1/home/micro/./l-z level 1: 782.213
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: estimate size for
/u1/home/micro/./l-z level 1: 8735920 KB
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: waiting for runtar
"/u1/home/micro/./l-z" child
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: after runtar
/u1/home/micro/./l-z wait
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: getting size via gnutar
for /u1/home/micro/./l-z level 2
Wed Jan 21 05:29:09 2015: thd-32a58: sendsize: Spawning
"/usr/local/libexec/amanda/runtar runtar daily
/usr/local/etc/amanda/tools/gtar --create --file /dev/null --numeric-
owner --directory /u1/home/micro --one-file-system
--listed-incremental
/usr/local/var/amanda/gnutar-lists/marlin.bio.mor.nsm_u1_home_micro_._l-z_2.new
--sparse --ignore-fa
iled-read --totals --files-from
/tmp/amanda/sendsize._u1_home_micro_._l-z.20150121052909.include" in
pipeline
Wed Jan 21 05:42:15 2015: thd-32a58: sendsize: Total bytes written:
8937635840 (8.4GiB, 11MiB/s)
Wed Jan 21 05:42:15 2015: thd-32a58: sendsize: .....
Wed Jan 21 05:42:15 2015: thd-32a58: sendsize: estimate time for
/u1/home/micro/./l-z level 2: 786.054
Wed Jan 21 05:42:15 2015: thd-32a58: sendsize: estimate size for
/u1/home/micro/./l-z level 2: 8728160 KB
Wed Jan 21 05:42:15 2015: thd-32a58: sendsize: waiting for runtar
"/u1/home/micro/./l-z" child
Wed Jan 21 05:42:15 2015: thd-32a58: sendsize: after runtar
/u1/home/micro/./l-z wait
Wed Jan 21 05:42:15 2015: thd-32a58: sendsize: done with amname
/u1/home/micro/./l-z dirname /u1/home/micro spindle 506
AND THAT IS THE END OF THE SENDSIZE DEBUG FILE
On 1/21/15 3:02 PM, Jean-Louis Martineau wrote:
You get error at estimate or at backup time?
Look at the time in the sendsize and sendbackup debug file to find
which one is slow.
On 01/21/2015 02:40 PM, Chris Hoogendyk wrote:
Folks,
I have an Ubuntu 14.04 LTS system running Amanda 3.3.6 server
backing up a Solaris 10 system with Amanda 3.3.2.
I had it working and I was getting backups of that particular
Solaris system. Then I suddenly started getting the "timeout on
reply pipe" on every single dle on that system, but not on any other
systems. There is also another virtually identical Solaris system
(except with Amanda 2.5.1p3) that has continued getting backed up as
well as a number of Ubuntu systems with various versions of Ubuntu
(10.04LTS, 12.04LTS, or 14.04LTS) and Amanda (either 2.5.1p3, 3.3.2,
or 3.3.6).
If I run `amcheck -c daily`, I get 0 problems.
How do I troubleshoot this? Why would it have suddenly come up (last
Friday) and then been consistently non functional? (Whereas before
it was consistently functional). I've poked through the /tmp/amanda
debug logs, but haven't been able to identify any errors that would
tell me what was wrong.
I should note that most of these servers are in the same two
adjacent racks and have GigE connections to the same switch.
The server that is not getting backed up at present is our main
departmental server that is running mail services, web, file shares,
printing, anonymous ftp, mysql, etc. for a fairly active department.
