Re: if to fork Amanda

[Chris Nighswonger]

+1 ...

On Mon, May 6, 2019 at 1:25 PM Dustin J. Mitchell  wrote:

> On Mon, May 6, 2019 at 12:25 PM Chris Hassell 
> wrote:
>
>> I'm a major advocate for OSS in general here at Zmanda/BETSOL. I'm
>> interested in giving a full-voiced support to the community of those who
>> know Amanda works well for them.   Opinions may vary and projects fork
>> for all reasons or no reason (as everyone has a right to) but I hope to
>> help keep contributions and development all in one if we can.
>>
>
> I'm *really* happy to hear this and happy you're here :)
>
> Dustin
>

Re: amstatus lapsed time

[Chris Nighswonger]

On Tue, Dec 4, 2018 at 6:37 PM Uwe Menges  wrote:

> On 12/4/18 5:41 PM, Chris Nighswonger wrote:
> > So it appears that the
> > 11:29:10 part is nearly correct, but the 1+ part is clearly not.
>
> The last column is the current time for running dumps, or the last time
> it was dumping for DLEs that are already dumped (== finish time).
>
> There is special coding in place to put N+ in front of the current time
> if it was started on N day(s) before now, see
>
> https://github.com/zmanda/amanda/blob/b2fd140efb54e1ebca464f0a9ff407460d8c350b/perl/Amanda/Status.pm#L2301
>
>
The approach there is confusing at best for the unsuspecting. For example:
if the backup job were to start on one day at 23:59 and then finish up the
next at 0:59 the results would show it running for 1+ 0:59 which seems to
imply it has run for 24 hours plus 59 minutes. Reading the code (along with
your explanation) clears this up showing that what is being stated is that
the dump finished up at 12:59a on the day after it was started.

I notice that the man page for amstatus does not cover the output format. A
header line like that given in amreport might be helpful in avoiding such
confusion. Something like this:
https://github.com/zmanda/amanda/blob/b2fd140efb54e1ebca464f0a9ff407460d8c350b/perl/Amanda/Report/human.pm#L386

Kind regards,
Chris

Re: amstatus lapsed time

[Chris Nighswonger]

On Tue, Dec 4, 2018 at 1:51 PM Debra S Baddorf  wrote:

> > On Dec 4, 2018, at 12:49 PM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
> >
> > On Tue, Dec 4, 2018 at 1:44 PM Debra S Baddorf  wrote:
> > Well, for starters,  2200 to 1129  is 13:29  so that doesn’t match up
> either.
> >
> > Talk about bad math
> >
> > Not to loud... I'll loose my job teaching math. ;-)
>
> LOL!   Well,  I was your A++ pupil,  so don’t feel too bad.  I had already
> read the book,
> and so could pay attention to the derivations the prof was doing,  and
> correct his errors.
> They appreciated it,  so the end of the derivation would work right.


I wonder if it is only coincidental that I was grading Physics tests
today

Way too many numbers in any case. Better go back and check over the grades.

Chris

Re: amstatus lapsed time

[Chris Nighswonger]

On Tue, Dec 4, 2018 at 1:44 PM Debra S Baddorf  wrote:

> Well, for starters,  2200 to 1129  is 13:29  so that doesn’t match up
> either.
>

Talk about bad math

Not to loud... I'll loose my job teaching math. ;-)

amstatus lapsed time

[Chris Nighswonger]

So when I run amstatus  I see stuff like this:

>From Mon Dec  3 22:00:01 EST 2018



1359952k dumping   533792k (148.30%) (1+11:29:10)

For reference:

backup@scriptor:~ date
Tue Dec  4 11:37:07 EST 2018

I'm assuming that the last field is the time the dumper has been running.
But that is impossible as this dump began at 2200 yesterday and amstatus
was run at 1129 today (12:29 later). So it appears that the 11:29:10 part
is nearly correct, but the 1+ part is clearly not. I've noted in amreport
 that the time lapsed time estimates appear grossly incorrect as
well. ie:

  Total   Full  Incr.   Level:#
        
Estimate Time (hrs:min) 0:07
Run Time (hrs:min)  1:57
Dump Time (hrs:min)28:13  26:27   1:46

Perhaps "Dump Time" is cumulative across all dumpers? That still does not
seem to explain the amstatus discrepancy.

What am I missing?

Kind regards,
Chris

Re: Dumping and taping in parallel

[Chris Nighswonger]

On Wed, Nov 28, 2018 at 11:17 AM Austin S. Hemmelgarn 
wrote:

> Based on your configuration, your tapes are configured to store just
> short of 800GB of data.
>
> The relevant lines then are these two:
>
> flush-threshold-scheduled 50
> flush-threshold-dumped 50
>
>
I misunderstood the man pages there and for some reason thought that volume
referred to the holding disk. Probably because I was reading way too
fast


> In your case, I'd suggest figuring out the average amount of data you
> dump each run, and then configuring things to start flushing when about
> half that much data has been dumped.  That will still have the taping
> run in parallel with the dumping, but will give you enough of a buffer
> that the taper should never have to wait for dumps to finish.
>

So over the last 13 runs, the:

-- smallest volume size has been 152G (19% tape capacity)
-- average volume size has been 254G (32% tape capacity)
-- largest volume size has been 612G (76% tape capacity)

So do the following values look "reasonable" based on those numbers:

flush-threshold-scheduled 25
flush-threshold-dumped 0

That should target the larger sizes which are the ones which tend to lap
into the next business day.

Kind regards,
Chris

Re: Dumping and taping in parallel

[Chris Nighswonger]

On Wed, Nov 28, 2018 at 10:49 AM Austin S. Hemmelgarn 
wrote:

> On 2018-11-28 09:53, Stefan G. Weichinger wrote:
> > Am 28.11.18 um 15:47 schrieb Chris Nighswonger:
> >> So why won't amanda dump and tape at the same time?
> >
> > It does normally, that is what the holding disk is for.
> Really?  I was under the impression that it was for making sure you can
> finish dumps if something goes wrong with taping, and cache dumps so
> they can be written to tape in one pass.  Without a holding disk, Amanda
> dumps straight to tape, which is technically dumping and taping in
> parallel.
> >
> > More details might lead to better suggestions.
> >
> > Show your amanda.conf etc
> Indeed, though I suspect it's something regarding the flushing
> configuration.
>

inparallel 10
maxdumps 1
# (equivalent to one Tbit/s).
netusage 1073741824
dumporder "STSTStstst"
dumpcycle 5 days
runspercycle 5
tapecycle 13 tapes
runtapes 1
flush-threshold-scheduled 50
flush-threshold-dumped 50
bumpsize 10 Mbytes
bumppercent 0
bumpmult 1.5
bumpdays 2
ctimeout 60
dtimeout 1800
etimeout 300
dumpuser "backup"
tapedev "Quantum-Superloader3-LTO-V4"
autolabel "$c-$b" EMPTY
labelstr "campus-.*"
tapetype LTO-4
logdir "/var/backups/campus/log"
infofile "/var/backups/campus/curinfo"
indexdir "/var/backups/campus/index"
tapelist "/var/backups/campus/tapelist"
autoflush all

holdingdisk hd1 {
comment "Local striped raid array"
directory "/storage/campus"
use 0 Gb
chunksize 1 Gb
}

define changer Quantum-Superloader3-LTO-V4 {
tapedev "chg-robot:/dev/sg3"
property "use-slots" "1-13"
property "tape-device" "0=tape:/dev/nst0"
device-property "LEOM" "TRUE"
}

define tapetype LTO-4 {
comment "Created by amtapetype; compression enabled"
length 794405408 kbytes
filemark 1385 kbytes
speed 77291 kps
blocksize 512 kbytes
}

Dumping and taping in parallel

[Chris Nighswonger]

So why won't amanda dump and tape at the same time?

Kind regards,
Chris

Re: Another dumper question

[Chris Nighswonger]

On Mon, Nov 26, 2018 at 3:28 PM Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

>
>
> So I've set these params back as they were before (ie. inparallel 10,
> maxdumps 1) and opened the throttle on the bandwidth. We'll see if
> things get better or worse tonight. I have a few more DLEs I need to
> split which may help the over all problem.
>

The results are in:

 dumper0 busy   :  5:25:03  ( 98.03%)
 dumper1 busy   :  4:26:25  ( 80.35%)
 dumper2 busy   :  4:05:57  ( 74.18%)
 dumper3 busy   :  1:55:55  ( 34.96%)
 dumper4 busy   :  3:21:39  ( 60.82%)
 dumper5 busy   :  2:12:41  ( 40.02%)
 dumper6 busy   :  1:55:50  ( 34.93%)
 dumper7 busy   :  2:18:20  ( 41.72%)
 dumper8 busy   :  1:57:28  ( 35.43%)
 dumper9 busy   :  2:10:02  ( 39.22%)
 0 dumpers busy :  0:06:32  (  1.97%)   0:  0:06:32
(100.00%)
 1 dumper busy  :  0:58:37  ( 17.68%)   0:  0:58:37
(100.00%)
 2 dumpers busy :  0:20:26  (  6.17%)   0:  0:20:26
(100.00%)
 3 dumpers busy :  0:43:21  ( 13.08%)   0:  0:43:21
(100.00%)
 4 dumpers busy :  1:04:15  ( 19.38%)   0:  1:04:15
(100.00%)
 5 dumpers busy :  0:05:38  (  1.70%)   0:  0:05:38
(100.00%)
 6 dumpers busy :  0:02:38  (  0.79%)   0:  0:02:38  (
99.99%)
 7 dumpers busy :  0:12:34  (  3.79%)   0:  0:12:34
(100.00%)
 8 dumpers busy :  0:01:38  (  0.49%)   0:  0:01:38  (
99.99%)
 9 dumpers busy :  0:00:52  (  0.27%)   0:  0:00:52  (
99.65%)
10 dumpers busy :  1:54:57  ( 34.67%)   0:  1:53:34  (
98.79%)

Here's a graph of the bandwidth on the trunking interface on the switch the
backup server is on:

[image: 192.168.x.x_1_1-day.png]

As you can see, the network load is negligible even during business hours.
So the giving amanda the reigns is not a problem and it looks like the
dumps are running more efficiently.

Comments appreciated.

Kind regards,
Chris

Re: Another dumper question

[Chris Nighswonger]

On Mon, Nov 26, 2018 at 10:13 PM Nathan Stratton Treadway
 wrote:
>
> On Mon, Nov 26, 2018 at 17:07:54 -0500, Chris Hoogendyk wrote:
> > My understanding was that Amanda cannot throttle the throughput, but
> > it can see what it is. If it is at or over the limit, it won't start
> > another dump. So, you can end up over the limit that you have
> > specified for periods of time, but it won't just continue increasing
> > without respect for the limit.
>
> To expand on this slightly (and tie it back to the begining of this
> thread): what Amanda actually looks at is the dump-size/time figure from
> the "info" database it keeps on each DLE.  So, it's not looking at
> dynamic real-time network bandwidth usage, but rather calculating what
> the average throughput turned out to be the last time this DLE was
> dumped.
>
> As you say, the important thing to note is that Amanda uses this info to
> do throttling at the dump/dumper level: when there is a free dumper, it
> will send a request for a new DLE to that dumper only if the bandwidth
> usage calculated for currently-in-progress dumps doesn't exceed the
> configured limit.
>
> If the configured limit is too high, the possiblity is that too many
> simultaneous dumps, especially of very fast client systems, will
> saturate the network interface (or some part of the network
> infrastructure) -- while if it's too low, the symptom will be that some
> of the configured "inparallel" dumpers will be left idle.  (But the nice
> thing about having Amanda do this calculation is that it can go ahead
> and kick off more dumpers when working on slow client systems and fewer
> when processing the fast ones.)
>
> Anyway, in the case of the original email in this thread, the problem
> seems to be that the calculated bandwidth for the single DLE in processs
> of being dumped already exceeds the configured bandwidth limit (as
> represented by the amstatus report line "network free kps: 0") -- and
> thus the other 9 dumpers are all left idle even though there are many
> DLEs out there waiting to be dumped.

Nathan: Thank-you for the very clear explanation of how amanda handles
this. That's good wiki material.

So last night's run borked due to my having two netusage statements in
the config. For some reason I missed that yesterday. Maybe its
age-related, I don't know... ;-)

That's fixed and we'll see what sort of throughput the network can
handle tonight.

Re: Another dumper question

[Chris Nighswonger]

On Mon, Nov 26, 2018 at 3:22 PM Nathan Stratton Treadway
 wrote:
>
> On Mon, Nov 26, 2018 at 14:15:41 -0500, Chris Nighswonger wrote:
> > That makes more sense. I've set it to 10 and inparallel to 2 and we'll
> > see how it goes tonight.
> >
> > On Mon, Nov 26, 2018 at 2:01 PM Cuttler, Brian R (HEALTH)
> >  wrote:
> > >
> > > I believe maxdumps is max concurrent dumps across all clients.
> > >
> > > You might have 10 clients each with an inparallel of 2, giving 20
> > > possible concurrent dumps, but because of server limitations you
> > > might set maxdumps to something between 2 and 20.
>
> Actually, it is the other way around: when you kick off amdump, the
> driver retrieves the "inparallel" configuration option and then starts
> up that number of dumper subprocesses -- so "inparallel" controls the
> total number of dumpers available for the whole run.
>
> In contrast, the amanda.conf man page section on maxdumps says:  "The
> maximum number of backups from a single host that Amanda will attempt to
> run in parallel." So that controls how many (out of the "inparallel"
> total dumpers available) might simultaneously run on one host at the
> same time.
>
> (Maxdumps defaults to 1 to avoid two dumpers on the same client
> competing with each other for disk I/O [or worse, thrashing the disk
> heads by interweaving reads from different places on the device], CPU
> [e.g. for compression], and/or network bandwidth.  This seems likely to
> be a reasonable default, though I guess if you had multiple physical
> disk devices each of which was relatively slow compared to CPU and
> network bandwidth then it would make sense to run more than one dump at
> a time.  But I'd do careful testing to make sure that's not actually
> slowing down the overall dump.)

So I've set these params back as they were before (ie. inparallel 10,
maxdumps 1) and opened the throttle on the bandwidth. We'll see if
things get better or worse tonight. I have a few more DLEs I need to
split which may help the over all problem.

Re: Another dumper question

[Chris Nighswonger]

On Mon, Nov 26, 2018 at 2:32 PM Nathan Stratton Treadway
 wrote:
>
> On Mon, Nov 26, 2018 at 13:56:52 -0500, Austin S. Hemmelgarn wrote:
> > On 2018-11-26 13:34, Chris Nighswonger wrote:
> > The other possibility that comes to mind is that your bandwidth
> > settings are making Amanda decide to limit to one dumper at a time.
>
> Chris, this is certainly the first thing to look at: note in your
> amstatus output the line "network free kps: 0":
>
>
> > >9 dumpers idle  : 0
> > >taper status: Idle
> > >taper qlen: 1
> > >network free kps: 0
> > >holding space   : 436635431k ( 50.26%)

Hmm... I missed that completely. I'll set it arbitrarily high as
Austin suggested and test it overnight.

Re: Another dumper question

[Chris Nighswonger]

That makes more sense. I've set it to 10 and inparallel to 2 and we'll
see how it goes tonight.

On Mon, Nov 26, 2018 at 2:01 PM Cuttler, Brian R (HEALTH)
 wrote:
>
> I believe maxdumps is max concurrent dumps across all clients.
>
> You might have 10 clients each with an inparallel of 2, giving 20 possible 
> concurrent dumps, but because of server limitations you might set maxdumps to 
> something between 2 and 20.
>
> -Original Message-
> From: Chris Nighswonger 
> Sent: Monday, November 26, 2018 1:57 PM
> To: Cuttler, Brian R (HEALTH) 
> Cc: amanda-users@amanda.org
> Subject: Re: Another dumper question
>
> ATTENTION: This email came from an external source. Do not open attachments 
> or click on links from unknown senders or unexpected emails.
>
>
> inparallel 10
>
> maxdumps not listed, so I'm assuming the default of 1 is being observed.
>
> I'm not sure that the maxdumps parameter would affect dumping DLEs from 
> multiple clients in parallel, though. The manpage states, "The maximum number 
> of backups from a single host that Amanda will attempt to run in parallel." 
> That seems to indicate that this parameter controls parallel dumps of DLEs on 
> a single client.
>
> Kind regards,
> Chris
> On Mon, Nov 26, 2018 at 1:50 PM Cuttler, Brian R (HEALTH) 
>  wrote:
> >
> > Did you check your maxdumps and inparallel parameters?
> >
> > -Original Message-
> > From: owner-amanda-us...@amanda.org  On
> > Behalf Of Chris Nighswonger
> > Sent: Monday, November 26, 2018 1:34 PM
> > To: amanda-users@amanda.org
> > Subject: Another dumper question
> >
> > ATTENTION: This email came from an external source. Do not open attachments 
> > or click on links from unknown senders or unexpected emails.
> >
> >
> > So in one particular configuration I have the following lines:
> >
> > inparallel 10
> > dumporder "STSTSTSTST"
> >
> > I would assume that that amanda would spawn 10 dumpers in parallel and 
> > execute them giving priority to largest size and largest time alternating. 
> > I would assume that amanda would do some sort of sorting of the DLEs based 
> > on size and time, set them in descending order, and the run the first 10 
> > based on the list thereby utilizing all 10 permitted dumpers in parallel.
> >
> > However, based on the amstatus excerpt below, it looks like amanda simply 
> > starts with the largest size and runs the DLEs one at a time, not making 
> > efficient use of parallel dumpers at all. This has the unhappy results at 
> > times of causing amdump to be running when the next backup is executed.
> >
> > I have changed the dumporder to STSTStstst for tonight's run to see if that 
> > makes any  difference. But I don't have much hope it will.
> >
> > Any thoughts?
> >
> > Kind regards,
> > Chris
> >
> >
> >
> >
> > From Mon Nov 26 01:00:01 EST 2018
> >
> > 1   4054117k waiting for dumping
> > 1  6671k waiting for dumping
> > 1   222k waiting for dumping
> > 1  2568k waiting for dumping
> > 1  6846k waiting for dumping
> > 1125447k waiting for dumping
> > 1 91372k waiting for dumping
> > 192k waiting for dumping
> > 132k waiting for dumping
> > 132k waiting for dumping
> > 132k waiting for dumping
> > 132k waiting for dumping
> > 1290840k waiting for dumping
> > 1 76601k waiting for dumping
> > 186k waiting for dumping
> > 1 71414k waiting for dumping
> > 0  44184811k waiting for dumping
> > 1   281k waiting for dumping
> > 1  6981k waiting for dumping
> > 150k waiting for dumping
> > 1 86968k waiting for dumping
> > 1 81649k waiting for dumping
> > 1359952k waiting for dumping
> > 0 198961004k dumping 159842848k ( 80.34%) (7:23:39)
> > 1 73966k waiting for dumping
> > 1821398k waiting for dumping
> > 1674198k waiting for dumping
> > 0 233106841k dump done (7:23:37), waiting for writing to tape
> > 132k waiting for dumping
> > 132k waiting for dumping
> > 1166876k waiting for dumping
> > 132k waiting for dumping
> > 1170895k waiting for dumping
> > 1162817k waiting for dumping
> > 0 failed: planner: [Request to client failed: Connection timed out]
> > 132k waiting for dumping
> > 132k waiting for dumping
> > 053k waiting for dumping
> > 0  77134628k waiting for dumping
> > 1  2911k waiting 

Re: Another dumper question

[Chris Nighswonger]

inparallel 10

maxdumps not listed, so I'm assuming the default of 1 is being observed.

I'm not sure that the maxdumps parameter would affect dumping DLEs
from multiple clients in parallel, though. The manpage states, "The
maximum number of backups from a single host that Amanda will attempt
to run in parallel." That seems to indicate that this parameter
controls parallel dumps of DLEs on a single client.

Kind regards,
Chris
On Mon, Nov 26, 2018 at 1:50 PM Cuttler, Brian R (HEALTH)
 wrote:
>
> Did you check your maxdumps and inparallel parameters?
>
> -Original Message-
> From: owner-amanda-us...@amanda.org  On Behalf 
> Of Chris Nighswonger
> Sent: Monday, November 26, 2018 1:34 PM
> To: amanda-users@amanda.org
> Subject: Another dumper question
>
> ATTENTION: This email came from an external source. Do not open attachments 
> or click on links from unknown senders or unexpected emails.
>
>
> So in one particular configuration I have the following lines:
>
> inparallel 10
> dumporder "STSTSTSTST"
>
> I would assume that that amanda would spawn 10 dumpers in parallel and 
> execute them giving priority to largest size and largest time alternating. I 
> would assume that amanda would do some sort of sorting of the DLEs based on 
> size and time, set them in descending order, and the run the first 10 based 
> on the list thereby utilizing all 10 permitted dumpers in parallel.
>
> However, based on the amstatus excerpt below, it looks like amanda simply 
> starts with the largest size and runs the DLEs one at a time, not making 
> efficient use of parallel dumpers at all. This has the unhappy results at 
> times of causing amdump to be running when the next backup is executed.
>
> I have changed the dumporder to STSTStstst for tonight's run to see if that 
> makes any  difference. But I don't have much hope it will.
>
> Any thoughts?
>
> Kind regards,
> Chris
>
>
>
>
> From Mon Nov 26 01:00:01 EST 2018
>
> 1   4054117k waiting for dumping
> 1  6671k waiting for dumping
> 1   222k waiting for dumping
> 1  2568k waiting for dumping
> 1  6846k waiting for dumping
> 1125447k waiting for dumping
> 1 91372k waiting for dumping
> 192k waiting for dumping
> 132k waiting for dumping
> 132k waiting for dumping
> 132k waiting for dumping
> 132k waiting for dumping
> 1290840k waiting for dumping
> 1 76601k waiting for dumping
> 186k waiting for dumping
> 1 71414k waiting for dumping
> 0  44184811k waiting for dumping
> 1   281k waiting for dumping
> 1  6981k waiting for dumping
> 150k waiting for dumping
> 1 86968k waiting for dumping
> 1 81649k waiting for dumping
> 1359952k waiting for dumping
> 0 198961004k dumping 159842848k ( 80.34%) (7:23:39)
> 1 73966k waiting for dumping
> 1821398k waiting for dumping
> 1674198k waiting for dumping
> 0 233106841k dump done (7:23:37), waiting for writing to tape
> 132k waiting for dumping
> 132k waiting for dumping
> 1166876k waiting for dumping
> 132k waiting for dumping
> 1170895k waiting for dumping
> 1162817k waiting for dumping
> 0 failed: planner: [Request to client failed: Connection timed out]
> 132k waiting for dumping
> 132k waiting for dumping
> 053k waiting for dumping
> 0  77134628k waiting for dumping
> 1  2911k waiting for dumping
> 136k waiting for dumping
> 132k waiting for dumping
> 1 84935k waiting for dumping
>
> SUMMARY  part  real  estimated
>size   size
> partition   :  43
> estimated   :  42559069311k
> flush   :   0 0k
> failed  :   10k   (  0.00%)
> wait for dumping:  40128740001k   ( 23.03%)
> dumping to tape :   00k   (  0.00%)
> dumping :   1 159842848k 198961004k ( 80.34%) ( 28.59%)
> dumped  :   1 233106841k 231368306k (100.75%) ( 41.70%)
> wait for writing:   1 233106841k 231368306k (100.75%) ( 41.70%)
> wait to flush   :   0 0k 0k (100.00%) (  0.00%)
> writing to tape :   0 0k 0k (  0.00%) (  0.00%)
> failed to tape  :   0 0k 0k (  0.00%) (  0.00%)
> taped   :   0 0k 0k (  0.00%) (  0.00%)
> 9 dumpers idle  : 0
> taper status: Idle
> taper qlen: 1
> network free kps: 0
> holding space   : 436635431k ( 50.26%)
> chunker0 busy   :  6:17:03  ( 98.28%)
>  dumper0 busy   :  6:17:03  ( 98.28%)
>  0 dumpers busy :  0:06:34  (  1.72%)   0:  0:06:34  (100.00%)
>  1 dumper busy  :  6:17:03  ( 98.28%)   0:  6:17:03  (100.00%)

Another dumper question

[Chris Nighswonger]

So in one particular configuration I have the following lines:

inparallel 10
dumporder "STSTSTSTST"

I would assume that that amanda would spawn 10 dumpers in parallel and
execute them giving priority to largest size and largest time
alternating. I would assume that amanda would do some sort of sorting
of the DLEs based on size and time, set them in descending order, and
the run the first 10 based on the list thereby utilizing all 10
permitted dumpers in parallel.

However, based on the amstatus excerpt below, it looks like amanda
simply starts with the largest size and runs the DLEs one at a time,
not making efficient use of parallel dumpers at all. This has the
unhappy results at times of causing amdump to be running when the next
backup is executed.

I have changed the dumporder to STSTStstst for tonight's run to see if
that makes any  difference. But I don't have much hope it will.

Any thoughts?

Kind regards,
Chris




>From Mon Nov 26 01:00:01 EST 2018

1   4054117k waiting for dumping
1  6671k waiting for dumping
1   222k waiting for dumping
1  2568k waiting for dumping
1  6846k waiting for dumping
1125447k waiting for dumping
1 91372k waiting for dumping
192k waiting for dumping
132k waiting for dumping
132k waiting for dumping
132k waiting for dumping
132k waiting for dumping
1290840k waiting for dumping
1 76601k waiting for dumping
186k waiting for dumping
1 71414k waiting for dumping
0  44184811k waiting for dumping
1   281k waiting for dumping
1  6981k waiting for dumping
150k waiting for dumping
1 86968k waiting for dumping
1 81649k waiting for dumping
1359952k waiting for dumping
0 198961004k dumping 159842848k ( 80.34%) (7:23:39)
1 73966k waiting for dumping
1821398k waiting for dumping
1674198k waiting for dumping
0 233106841k dump done (7:23:37), waiting for writing to tape
132k waiting for dumping
132k waiting for dumping
1166876k waiting for dumping
132k waiting for dumping
1170895k waiting for dumping
1162817k waiting for dumping
0 failed: planner: [Request to client failed: Connection timed out]
132k waiting for dumping
132k waiting for dumping
053k waiting for dumping
0  77134628k waiting for dumping
1  2911k waiting for dumping
136k waiting for dumping
132k waiting for dumping
1 84935k waiting for dumping

SUMMARY  part  real  estimated
   size   size
partition   :  43
estimated   :  42559069311k
flush   :   0 0k
failed  :   10k   (  0.00%)
wait for dumping:  40128740001k   ( 23.03%)
dumping to tape :   00k   (  0.00%)
dumping :   1 159842848k 198961004k ( 80.34%) ( 28.59%)
dumped  :   1 233106841k 231368306k (100.75%) ( 41.70%)
wait for writing:   1 233106841k 231368306k (100.75%) ( 41.70%)
wait to flush   :   0 0k 0k (100.00%) (  0.00%)
writing to tape :   0 0k 0k (  0.00%) (  0.00%)
failed to tape  :   0 0k 0k (  0.00%) (  0.00%)
taped   :   0 0k 0k (  0.00%) (  0.00%)
9 dumpers idle  : 0
taper status: Idle
taper qlen: 1
network free kps: 0
holding space   : 436635431k ( 50.26%)
chunker0 busy   :  6:17:03  ( 98.28%)
 dumper0 busy   :  6:17:03  ( 98.28%)
 0 dumpers busy :  0:06:34  (  1.72%)   0:  0:06:34  (100.00%)
 1 dumper busy  :  6:17:03  ( 98.28%)   0:  6:17:03  (100.00%)

Configuration Rollback [Was: Reusable]

[Chris Nighswonger]

On Fri, Nov 23, 2018 at 6:47 PM Jon LaBadie  wrote:

> On Wed, Nov 21, 2018 at 11:55:21AM -0800, Chris Miller wrote:
> > Hi Folks,
> >
> > I have written some very small DLEs so I can rip through weeks
> > of backups in minutes. I've learned some things.
>
> When I'm experimenting with amanda after I create the
> test configuration, before the first amdump run, I
> copy or tar the entire configuration.
>
> Then I write a little shell script so that after I've
> run a few test dumps, I can delete the entire config
> and restore it to the original state from the copy or
> tarball.
>

I use Git and create a repo at the root of each configuration. Every change
gets a commit. Rolling back to any point in time is super easy in that way.
It has saved my skin on several occasions.

Kind regards,
Chris

Re: Taper scan algorithm did not find an acceptable volume.

[Chris Nighswonger]

On Tue, Nov 20, 2018 at 1:51 PM Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> On Tue, Nov 20, 2018 at 1:45 PM Chris Miller  wrote:
>
>> Hi Chris,
>>
>> *From: *"Chris Nighswonger" 
>> *To: *"Chris Miller" 
>> *Cc: *"amanda-users" 
>> *Sent: *Tuesday, November 20, 2018 9:56:12 AM
>> *Subject: *Re: Taper scan algorithm did not find an acceptable volume.
>>
>> On Tue, Nov 20, 2018 at 10:58 AM Chris Miller  wrote:
>>
>>> Hi Jon,
>>> 
>>> This is confirmed. From my AMANDA server, I can see each of my NAS as a
>>> SaMBa share. I can ping each of my clients from the AMANDA server, and
>>> smbclient -L let's me browse their filesystems.
>>>
>>
>> And amcheck  says?
>>
>> amcheck reports the following for each of the three clients in my test
>> config:
>>
>> Amanda Tape Server Host Check
>> -
>> NOTE: Holding disk '/var/amanda/hold': 2720485376 KB disk space
>> available, using 2720485376 KB
>> slot 0: contains an empty volume
>> 
>> slot 28: contains an empty volume
>> all slots have been loaded
>> Taper scan algorithm did not find an acceptable volume.
>> (expecting a new volume)
>> ERROR: No acceptable volumes found
>> Server check took 1.036 seconds
>>
>
> amcheck also checks the clients. I see no output here to indicate that
> amanda is able to connect to your clients. Was there no client related
> output or did you snip it out?
>

For example (limited to one host just to illustrate):

backup@scriptor:/amcheck -c campus host.fqdn

Amanda Backup Client Hosts Check

Client check: 1 host checked in 2.119 seconds.  0 problems found.

(brought to you by Amanda 3.3.6)
backup@scriptor:/

Re: Taper scan algorithm did not find an acceptable volume.

[Chris Nighswonger]

On Tue, Nov 20, 2018 at 1:45 PM Chris Miller  wrote:

> Hi Chris,
>
> *From: *"Chris Nighswonger" 
> *To: *"Chris Miller" 
> *Cc: *"amanda-users" 
> *Sent: *Tuesday, November 20, 2018 9:56:12 AM
> *Subject: *Re: Taper scan algorithm did not find an acceptable volume.
>
> On Tue, Nov 20, 2018 at 10:58 AM Chris Miller  wrote:
>
>> Hi Jon,
>> 
>> This is confirmed. From my AMANDA server, I can see each of my NAS as a
>> SaMBa share. I can ping each of my clients from the AMANDA server, and
>> smbclient -L let's me browse their filesystems.
>>
>
> And amcheck  says?
>
> amcheck reports the following for each of the three clients in my test
> config:
>
> Amanda Tape Server Host Check
> -
> NOTE: Holding disk '/var/amanda/hold': 2720485376 KB disk space available,
> using 2720485376 KB
> slot 0: contains an empty volume
> 
> slot 28: contains an empty volume
> all slots have been loaded
> Taper scan algorithm did not find an acceptable volume.
> (expecting a new volume)
> ERROR: No acceptable volumes found
> Server check took 1.036 seconds
>

amcheck also checks the clients. I see no output here to indicate that
amanda is able to connect to your clients. Was there no client related
output or did you snip it out?

Re: Windows Domains

[Chris Nighswonger]

http://wiki.zmanda.com/index.php/Zmanda_Windows_Client


On Tue, Nov 20, 2018 at 12:59 PM Chris Miller  wrote:

> Hi Folks,
>
> Two of my "clients" are Windows Server boxes. One is a domain controller
> and the other is a member of the domain. ZWC needs a username and suggests
> "amandabackup", which is fine and the installation procedure creates such a
> local user. I need this to be a domain user, meaning \amandabackup,
> not \amandabackup. Since \amandabackup would be the
> default if AMANDA didn't know the value for , I need to understand
> how I configure this. Does anybody have any advice?
>
> Thanks for the help,
> --
> Chris.
>
> V:916.974.0424
> F:916.974.0428
>

Re: Taper scan algorithm did not find an acceptable volume.

[Chris Nighswonger]

On Tue, Nov 20, 2018 at 10:58 AM Chris Miller  wrote:

> Hi Jon,
>
> - Original Message -
> > From: "Jon LaBadie"
> > To: "amanda-users" 
> > Sent: Monday, November 19, 2018 1:44:35 PM
> > Subject: Re: Taper scan algorithm did not find an acceptable volume.
>
> > As a general approach, an admin should not run new software systems
> > until its dependices are independently verified.  For example
> > client backups depand on a working network.  You should confirm
> > the clients can reach the server and visa-versa.
>
> This is confirmed. From my AMANDA server, I can see each of my NAS as a
> SaMBa share. I can ping each of my clients from the AMANDA server, and
> smbclient -L let's me browse their filesystems.
>

And amcheck  says?

Chris

Re: Amanda Planner Logic

[Chris Nighswonger]

On Sat, Nov 17, 2018 at 12:56 PM Nathan Stratton Treadway <
natha...@ontko.com> wrote:

> On Sat, Nov 17, 2018 at 12:00:49 -0500, Gene Heskett wrote:
> > But without the logdir on the cli as --logdir=/path/to, it still doesn't
> > work because planner, although it does read amanda.conf, does not pickup
> > the logdir entry in amanda.conf.  And I just found that the configure
> > disallows the setting of logdir as its a deprecated method.
> >
> > That damns us to specifying the --logdir on the cli, and I believe we
> > should not use the same path as in amanda.conf,
>
> Yes, in v3.5 the planner --log-filename option tells the program where
> to write some output, and it required.  For these manual runs you don't
> care about the info it writes there, so you just need to specify some
> out-of-the-way place for it to write to.
>
> Since it's creating the file in question, and normally is passed that
> argument by amdump which has already gone through the process of
> building a log file name to use, the planner executable itself does not
> look at the "logdir" config setting.
>
> (Note that the config option is "logdir", i.e. a directory, while the
> planner command-line argument is a full path-to-a-specific-file-name --
> two different beasts.)
>
>
> > my balance report this
> > morning is blank for this mornings run.
> > amanda@coyote:~/amanda-3.5.1$ /usr/local/sbin/amadmin Daily balance
> >
> >  due-date  #fsorig MB out MB   balance
> > --
> > 11/17 Sat0  0  0  ---
> > 11/18 Sun3  17741  15322 -9.7%
> > 11/19 Mon5  27503  15341 -9.6%
> > 11/20 Tue1  49240  25295+49.1%
> > 11/21 Wed6  17357  14821-12.6%
> > 11/22 Thu7   1086360-97.9%
> > 11/23 Fri3  40754  15666 -7.6%
> > 11/24 Sat2  22109  22109+30.4%
> > 11/25 Sun4  68490  43131   +154.3%
> > 11/26 Mon   38  31911  17565 +3.6%
> > --
> > TOTAL   69 276191 169610 16961
> >   (estimated 10 runs per dumpcycle)
> >
> >
> >  So I am assuming I nuked one too many planner files when cleaning up
> > from the aborted run. Planner does not leave its log files there in that
> > directory anyway. Is it supposed to? All that left when its done is the
> > amdump-datetime files.
>
> It's hard to know for sure without having seen the balance output before
> your most recent run, but "amadmin ... balance" uses data from the info
> files, not any sort of log files, so I am guessing the output above is
> not related to the deletion of any "planner files".
>
> Rather, I think it just reflects the fact that you are running it after
> today's dump has already finished, so nothing is due for full dumps
> today.  (Before we fixed the small-DLE bug you wouldn't see
> completely-empty days because those small DLEs were always showing as
> needing an immedate full dump... but now that that's fixed you will see
> today as "empty" after today's dump has finished.)
>
>
>

So in the midst of all of this, be careful not to leave a stray log file in
your 'logdir' or the planner will bork on the next run like so:

NOTES:
  planner: WARNING: Argument '--start-time' matches neither a host nor a
disk.

The job still ran. I'm just not sure if there will be any issues to arise.
If there is, I'll post back.

Chris

Re: Amanda Planner Logic

[Chris Nighswonger]

On Fri, Nov 16, 2018 at 7:36 PM Gene Heskett  wrote:
>
> On Friday 16 November 2018 15:38:27 Debra S Baddorf wrote:
>
> > > On Nov 16, 2018, at 2:09 PM, Gene Heskett 
> > > wrote:
> > >
> > > On Friday 16 November 2018 14:38:58 Chris Nighswonger wrote:
> > >> On Fri, Nov 16, 2018 at 2:30 PM Gene Heskett 
> > >
> > > wrote:
> > >>> from amanda as usr, find /usr/local -name 'planner'
> > >>> returns that its in /usr/local/libexec/amanda/planner
> > >>> Now I'll find out (maybe) what it thinks of my changes. Ignore
> > >>> that faint knocking sound. :) But I assumed you meant backup_job
> > >>> was your config dir, in my case /usr/local/etc/amanda/Daily so it
> > >>> failed.
> > >>
> > >> backup_job is the name of the backup rather than the directory. So
> > >> in my case it looks like this:
> > >>
> > >> /usr/lib/amanda/planner campus
> > >
> > > ooohhkkaaay. Can I use wild cards like "Daily  PublicA*" ?
> > >
> > > No, the logfile error already posted prevents it from running so it
> > > never gets to PublicA in any form.
> > >
> > >> Be sure to run it as your backup user.
> > >
> > > amanda in this case.
> > >
> > >> Sorry for any confusion there.
> > >>
> > >> Kind regards,
> > >> Chris
> > >
> > > Thanks Chris
> >
> > I’m not familiar with the syntax,  but I would suspect it wants
> > planner node  
> >
> > which in my case, is the  “homeu” word in the following:
> > node   homeu   /home   {
> >include “./u*”
> > }
> >
> > You have to give it a nickname,  because ALL the DLEs for this
> > disk  (“/home”)   are based on that same disk location   /home
> > So you nickname the  A parts,  or the  AF  (a-f) parts, etc.
> >
> > If it WASN’T a  tar  subset  (using includes or excludes)  then you
> > would skip the nickname and just put the disk area/home
> > But for subsets,  you need a name for each one.
> >
> > Deb Baddorf
> The dle's in question:
>
> coyote PublicAar /home/gene/PublicA/ {
> comp-coyote-tar
> include "./[a-r]*" "./[-]*"  "./_*"
> } 1 local
>
> coyote PublicAsz /home/gene/PublicA/ {
> comp-coyote-tar
> include "./[s-z]*"
> } 1 local
>
> coyote PublicAAZ /home/gene/PublicA {
> comp-coyote-tar
> include "./[A-Z]*"
> } 1 local
>
> /usr/local/libexec/amanda/planner Daily coyote PublicA*
> gets me:
> planner: pid 4411 executable /usr/local/libexec/amanda/planner version
> 3.5.1
> planner: build: VERSION="Amanda-3.5.1"
> planner:BUILT_DATE="Fri Nov 9 20:28:49 EST 2018" BUILT_MACH=""
> planner:BUILT_REV="7557" BUILT_BRANCH="tags" CC="gcc"
> planner: paths: bindir="/usr/local/bin" sbindir="/usr/local/sbin"
> planner:libexecdir="/usr/local/libexec"
> planner:amlibexecdir="/usr/local/libexec/amanda"
> planner:mandir="/usr/local/share/man" AMANDA_TMPDIR="/tmp/amanda"
> planner:AMANDA_DBGDIR="/tmp/amanda-dbg/"
> planner:CONFIG_DIR="/usr/local/etc/amanda" DEV_PREFIX="/dev/"
> planner:DUMP=UNDEF RESTORE=UNDEF VDUMP=UNDEF VRESTORE=UNDEF
> planner:XFSDUMP=UNDEF XFSRESTORE=UNDEF VXDUMP=UNDEF
> VXRESTORE=UNDEF
> planner:SAMBA_CLIENT="/usr/bin/smbclient" GNUTAR="/bin/tar"
> planner:COMPRESS_PATH="/bin/gzip" UNCOMPRESS_PATH="/bin/gzip"
> planner:LPR="/usr/bin/lpr" MAILER="/usr/bin/Mail"
> planner:listed_incr_dir="/usr/local/var/amanda/gnutar-lists"
> planner: defs:  DEFAULT_SERVER="coyote" DEFAULT_CONFIG="DailySet1"
> planner:DEFAULT_TAPE_SERVER="coyote" DEFAULT_TAPE_DEVICE=""
> planner:NEED_STRSTR AMFLOCK_POSIX AMFLOCK_FLOCK AMFLOCK_LOCKF
> planner:AMFLOCK_LNLOCK SETPGRP_VOID AMANDA_DEBUG_DAYS=4
> BSD_SECURITY
> planner:USE_AMANDAHOSTS CLIENT_LOGIN="amanda" CHECK_USERID
> HAVE_GZIP
> planner:COMPRESS_SUFFIX=".gz" COMPRESS_FAST_OPT="--fast"
> planner:COMPRESS_BEST_OPT="--best" UNCOMPRESS_OPT="-dc"
> planner:
> CONFIGURE_ARGS=" '--with-user=amanda' '--with-group=disk' 
> '--with-ow

Re: Amanda Planner Logic

[Chris Nighswonger]

On Fri, Nov 16, 2018 at 2:30 PM Gene Heskett  wrote:

>
> from amanda as usr, find /usr/local -name 'planner'
> returns that its in /usr/local/libexec/amanda/planner
> Now I'll find out (maybe) what it thinks of my changes. Ignore that faint
> knocking sound. :) But I assumed you meant backup_job was your config
> dir, in my case /usr/local/etc/amanda/Daily so it failed.
>

backup_job is the name of the backup rather than the directory. So in my
case it looks like this:

/usr/lib/amanda/planner campus

Be sure to run it as your backup user.

Sorry for any confusion there.

Kind regards,
Chris

Re: Amanda Planner Logic

[Chris Nighswonger]

On Fri, Nov 16, 2018 at 11:36 AM Gene Heskett  wrote:
>
> On Friday 16 November 2018 08:43:05 Chris Nighswonger wrote:
>
> > Just to get this into the archives:
> >
> > Running '/usr/lib/amanda/planner backup_job' provides immediate
> > insight into how the planner is thinking at that point-in-time.
>
> For me, that ought to be /usr/local/lib/amanda/planner, but its not
> there?

Maybe use some variation of

find /usr -name 'planner'

to dig it up?

Not sure where else to tell you to look.

Kind regards,
Chris

Re: Amanda Planner Logic

[Chris Nighswonger]

Just to get this into the archives:

Running '/usr/lib/amanda/planner backup_job' provides immediate insight
into how the planner is thinking at that point-in-time.

On Thu, Nov 15, 2018 at 12:00 PM Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> I've extracted the comments from server-src/planner.c which describe
> Amanda's logic in planning dumps. I found it quite informative and pretty
> much fitting the behavior I've observed on my systems. I thought it might
> be helpful for others.
>
> I've included it in the body of this email as well as an attached PDF.
>
> Kind regards,
> Chris
>
> *Amanda Planner Comments as Found in server-src/planner.c**Note: This is a 
> raw extract of the comments with very minimal post-extraction processing.*
>
> *1. Networking Setup*
> *2. Read in Configuration Information*
>
>   All the Amanda configuration files are loaded before we begin.
>*3. Send autoflush dumps left on the holding disks*
>
>   This should give us something to do while we generate the new dump 
> schedule.
> *4. Calculate Preliminary Dump Levels*
>
>   Before we can get estimates from the remote slave hosts, we make a 
> first attempt at guessing what dump levels we will be dumping at based on the 
> *curinfo* database.
>*5. Get Dump Size Estimates from Remote Client Hosts*
>
>   Each host is queried (in parallel) for dump size information on all of 
> its disks, and the results gathered as they come in.
>
>   Go out and get the dump estimates At this point, all disks with 
> estimates are in estq, and all the disks on hosts that didn't respond to our 
> inquiry are in failq.
> *6. Analyze Dump Estimates*
>
>   Each disk's estimates are looked at to determine what level it should 
> dump at, and to calculate the expected size and time taking historical dump 
> rates and compression ratios into account. The total expected size is 
> accumulated as well.
>
>   At this point, all the disks are on schedq sorted by priority. The 
> total estimated size of the backups is in total_size.
> *7. Delay Dumps if Schedule Too Big*
>
>   If the generated schedule is too big to fit on the tape, we need to 
> delay some full dumps to make room. Incrementals will be done instead (except 
> for new or forced disks).
>
>   In extreme cases, delaying all the full dumps is not even enough. If 
> so, some low-priority incrementals will be skipped completely until the dumps 
> fit on the tape.
> *8. Promote Dumps if Schedule Too Small*
>
>   Amanda attempts to balance the full dumps over the length of the dump 
> cycle. If this night's full dumps are too small relative to the other nights, 
> promote some high-priority full dumps that will be due for the next run, to 
> full dumps for tonight, taking care not to overflow the tape size.
>
>   This doesn't work too well for small sites. For these we scan ahead 
> looking for nights that have an excessive number of dumps and promote one of 
> them.  Amanda never delays full dumps just for the sake of balancing the 
> schedule, so it can take a full cycle to balance the schedule after a big 
> bump.
>*9. Output Schedule*
>
>   The schedule goes to stdout, presumably to driver. A copy is written on 
> stderr for the debug file.
>
>

Re: Does anyone know how to make an amadmin $config estimate work for new dle's?

[Chris Nighswonger]

On Fri, Nov 16, 2018 at 7:24 AM Austin S. Hemmelgarn 
wrote:

>
> Except that it actually runs on the client systems.  I've actually
> looked at this, the calcsize program is running on the clients and not
> the server.
>

For my own understanding: If estimate is set to client, the client runs
calcsize and returns a most accurate point-in-time size of the DLE?

Re: Does anyone know how to make an amadmin $config estimate work for new dle's?

[Chris Nighswonger]

On Thu, Nov 15, 2018 at 7:40 AM Austin S. Hemmelgarn 
wrote:

> On 2018-11-15 06:16, Gene Heskett wrote:
> > I ask because after last nights run it showed one huge and 3 teeny level
> > 0's for the 4 new dle's.  So I just re-adjusted the locations of some
> > categories and broke the big one up into 2 pieces. "./[A-P]*"
> > and ./[Q-Z]*", so the next run will have 5 new dle's.
> >
> > But an estimate does not show the new names that results in. I've even
> > took the estimate assignment calcsize back out of the global dumptype,
> > which ack the manpage, forces the estimates to be derived from a dummy
> > run of tar, didn't help.
> >
> > Clues? Having this info from an estimate query might take a couple hours,
> > but it sure would be helpfull when redesigning ones dle's.



> I'm fairly certain you can't, because it specifically shows server-side
> estimates, which have no data to work from if there has never been a
> dump run for the DLE.
>

What would be the downside to having the amanda client execute ' du -s' or
some such on the DLE and return the results when amcheck and friends
realize there is no reliable size estimate? This would seem to be a much
more accurate estimate than a non-existent server estimate.

Chris

Re: Amanda's DB

[Chris Nighswonger]

On Wed, Nov 14, 2018 at 7:08 PM Nathan Stratton Treadway 
wrote:

> On Wed, Nov 14, 2018 at 15:35:50 -0500, Chris Nighswonger wrote:
> > Can anyone point me to "good" documentation that describes Amanda's DB?
> > Things like schema, field descriptions and/or uses, etc.
>
>
[snip]


>
> > I notice that the later versions of ZWC use SQL.
>
> I saw that the Amanda source repository master branch includes the
> following commit:
>
> commit ea4dd29d265b44a914e794d5150e53bce9d9435d
> Author: Jean-Louis Martineau 
> Date:   Thu Jan 4 12:52:28 2018 +
>
> Use a database (SQLite, MySQL or ProstgreSQL) to replace the tapelist
> and log.* file
> Read the Amanda-Catalog file
> * perl/*, server-src/*, common-src/conffile.c: Major changes
> * installcheck/*: Fix tests
> * Amanda-Catalog: New documentation document
>
> Jan 2018 is after Amanda 3.5 was released so this is still
> work-in-progress, and I don't know how "mature" this transition got
> before Jean-Louis left
>

Digging a bit in the new Catalog code

The style makes for difficult reading. Clearly what docs there are show
that work was being done on SQLite. The DB schema is not readily apparent.

It seems to me that Amanda is heavily dependent upon its logs to coordinate
and keep it running. It would be very nice to have a solid set of docs
outlining the log syntax for each log and describing the various roles of
each log in the overall scheme of Amanda's operation.

Without some sort of understanding of the larger picture of how things
operate, its going to be mighty difficult picking up and moving forward
with development.

I have been involved in development (thousands of lines of code) and
project management of another large OSS package written primarily in Perl.
I am certainly willing to contribute to Amanda, but the "activation energy"
(aka learning curve) appears way too high at the moment.

I think when Dustin left, the end of true open development in Amanda left
with him.

My $0.02 worth.

Kind regards,
Chris

Re: Now I'm confuzzled about disklist entry's for "./[a-g]*" types

[Chris Nighswonger]

On Wed, Nov 14, 2018 at 3:34 PM Debra S Baddorf  wrote:

>
> And I’m not certain that amcheck  cares, I don’t think it’ll tell you if
> the set is empty.
>
>
It doesn't and it won't.

Kind regards,
Chris

Amanda's DB

[Chris Nighswonger]

Can anyone point me to "good" documentation that describes Amanda's DB?
Things like schema, field descriptions and/or uses, etc.

It looks like maybe they are located in
'/var/backups/jobname/index|curinfo' ? So flat files?

I notice that the later versions of ZWC use SQL.

Kind regards,
Chris

Re: Now I'm confuzzled about disklist entry's for "./[a-g]*" types

[Chris Nighswonger]

On Wed, Nov 14, 2018 at 1:10 PM Gene Heskett  wrote:

>
> First a snip/paste of my 3 new del's as I try to split a big one up:
> line entry
> 28 coyote /home/gene/PublicA/ Publicag {
> 29  comp-coyote-tar
> 30  include "./[a-g]*"
> 31  exclude "./[h-z]*"
> 32  } 1 local
>
> Try this variation:

coyote Publicag /home/gene/PublicA/ {
 comp-coyote-tar
 include "./[a-g]*"
 exclude "./[h-z]*"
 } 1 local

Syntax is:

hostname diskname diskdevice {
}

Kind regards,
Chris

Balance after DLE splitting

[Chris Nighswonger]

Was "All level 0 on the same run?"

The old thread was getting a bit stranded...

On Sat, Nov 10, 2018 at 5:03 PM Nathan Stratton Treadway 
wrote:

> On Sat, Nov 10, 2018 at 11:39:58 -0500, Chris Nighswonger wrote:
> > I think that is output from one of Gene's systems, but here is the latest
> > from mine after DLE balancing has been through one successful run. (The
> > next run will take place Monday morning @ 0200).
> >
> > backup@scriptor:~ amadmin campus balance
> >
> >  due-date  #fsorig kB out kB   balance
> > --
> > 11/10 Sat4  676705513  447515666   +178.0%
> > 11/11 Sun889438595250400-96.7%
> > 11/12 Mon   12  127984592   84074623-47.8%
> > 11/13 Tue   19  304110025  267932333+66.5%
> > 11/14 Wed0  0  0  ---
> > --
> > TOTAL   43 1117743989  804773022 160954604
> >   (estimated 5 runs per dumpcycle)
> >  (3 filesystems overdue. The most being overdue 17841 days.)
> >
> > Not sure what's up with the overdues. There were none prior to breaking
> up
> > the DLEs. It may just be an artifact.
>
>
>
This afternoon's balance report looks much nicer:

backup@scriptor:~ amadmin campus balance

 due-date  #fsorig kB out kB   balance
--
11/12 Mon   23  421824968  306613160+90.3%
11/13 Tue   19  304110025  267932333+66.3%

11/16 Fri1  393088305  230949229+43.4%
--
TOTAL   43 1119023298  805494722 161098944
  (estimated 5 runs per dumpcycle)
 (11 filesystems overdue. The most being overdue 17843 days.)

Kind Regards,
Chris

Re: All level 0 on the same run?

[Chris Nighswonger]

On Mon, Nov 12, 2018 at 1:50 PM Nathan Stratton Treadway 
wrote:

> On Mon, Nov 12, 2018 at 13:28:15 -0500, Chris Nighswonger wrote:
> > I found the long overdue culprit... It was a windows client using the ZWC
> > community client. The ZWC service had hung (not an uncommon problem) and
> > Amanda was missing backups from it. If I had been paying attention to the
> > nightly reports, I would have investigated it sooner. However, it does
> beg
> > the question: That DLE is NOT 49 years overdue... So where in the world
> did
> > the planner get that idea from?
>
> Again, that 1969 date is the "no data saved" timestamp.  You can see
> this mostly-directly by doing "amadmin campus info [HOST] [DEV]" on that
> DLE... or completely-directly by looking at the
>  .../campus/curinfo/[HOST]/[DEV]/info
> file (where the "seconds since the epoch" timestamp will show up as "-1"
> instead of a number in the range around 1542048289).
>
>
backup@scriptor:~ amadmin campus info host dev

Current info for host dev:
  Stats: dump rates (kps), Full:  6015.9, 5898.2, 5771.4
Incremental:  1658.2, 1456.6, 343.4
  compressed size, Full:  66.9%, 67.4%, 67.8%
Incremental:  71.3%, 68.0%, 67.4%
  Dumps: lev datestmp  tape file   origK   compK secs
  0  19691231   0 -1 -1 -1

 That timestamp is a bit odd, but...

(I don't know off hand why Amanada would have saved a "no data recorded"
> status rather than still having a record of the last time ZWC ran
> sucessfully -- perhaps the last successful dump went to a volume that
> has since been overwritten, and so the entry had to be deleted from the
> info database without anything new replacing it?)
>
>
This makes the most sense. This client has been offline for two cycles.

Chris

Re: All level 0 on the same run?

[Chris Nighswonger]

On Sat, Nov 10, 2018 at 5:03 PM Nathan Stratton Treadway 
wrote:

> On Sat, Nov 10, 2018 at 11:39:58 -0500, Chris Nighswonger wrote:
> >  (3 filesystems overdue. The most being overdue 17841 days.)
> >
> > Not sure what's up with the overdues. There were none prior to breaking
> up
> > the DLEs. It may just be an artifact.
>
> With a 5-day dumpcycle that would mean Amanda thinks the last dump took
> place 17846-ish days ago:
>   $ date --date="17846 days ago"
>   Wed Dec 31 14:24:39 EST 1969
> ... and the date of 1969/12/13 is the "no data saved" placeholder date
> within Amanda's info database.
>
> Anyway, you should be able to identify the three DLEs in question with
>   amadmin campus due | grep "Overdue"
> and then use "amadmin campus info [...]" to see what amanda has recorded
> about them.
>
>
I found the long overdue culprit... It was a windows client using the ZWC
community client. The ZWC service had hung (not an uncommon problem) and
Amanda was missing backups from it. If I had been paying attention to the
nightly reports, I would have investigated it sooner. However, it does beg
the question: That DLE is NOT 49 years overdue... So where in the world did
the planner get that idea from?

I'll check the balance again after tonight's run.

Kind regards,
Chris

Re: All level 0 on the same run?

[Chris Nighswonger]

On Sat, Nov 10, 2018 at 10:57 AM Nathan Stratton Treadway <
natha...@ontko.com> wrote:

> On Tue, Oct 30, 2018 at 15:51:36 -0400, Gene Heskett wrote:
> > I just changed the length of the dumpcycle and runs percycle up to 10,
> > about last friday while I was makeing the bump* stuff more attractive,
> > but the above command returns that the are 5 filesystens out of date:
> > su amanda -c "/usr/local/sbin/amadmin Daily balance"
> >
> >  due-date  #fsorig MB out MB   balance
> > --
> > 10/30 Tue5  0  0  ---
> > 10/31 Wed1  17355   8958-45.3%
> > 11/01 Thu2  10896  10887-33.5%
> > 11/02 Fri4  35944   9298-43.2%
> > 11/03 Sat4  14122  10835-33.8%
> > 11/04 Sun3  57736  57736   +252.7%
> > 11/05 Mon2  39947  30635+87.1%
> > 11/06 Tue8   4235   4215-74.3%
> > 11/07 Wed4  19503  14732-10.0%
> > 11/08 Thu   32  31783  16408 +0.2%
> > --
> > TOTAL   65 231521 163704 16370
>
> Okay, now that the small-DLE distraction is out of the way, we can get
> back to the original question regarding the scheduling of dumps over
> your dumpcycle.
>
> What does your "balance" output show now?
>

I think that is output from one of Gene's systems, but here is the latest
from mine after DLE balancing has been through one successful run. (The
next run will take place Monday morning @ 0200).

backup@scriptor:~ amadmin campus balance

 due-date  #fsorig kB out kB   balance
--
11/10 Sat4  676705513  447515666   +178.0%
11/11 Sun889438595250400-96.7%
11/12 Mon   12  127984592   84074623-47.8%
11/13 Tue   19  304110025  267932333+66.5%
11/14 Wed0  0  0  ---
--
TOTAL   43 1117743989  804773022 160954604
  (estimated 5 runs per dumpcycle)
 (3 filesystems overdue. The most being overdue 17841 days.)

Not sure what's up with the overdues. There were none prior to breaking up
the DLEs. It may just be an artifact.

Kind regards,
Chris

>
> (In particular, I'm curious if there is still one day with a huge surge
> like shown for 11/04 in the listing above.)
>
>
> Nathan
>
>
> 
> Nathan Stratton Treadway  -  natha...@ontko.com  -  Mid-Atlantic region
> Ray Ontko & Co.  -  Software consulting services  -
> http://www.ontko.com/
>  GPG Key: http://www.ontko.com/~nathanst/gpg_key.txt   ID: 1023D/ECFB6239
>  Key fingerprint = 6AD8 485E 20B9 5C71 231C  0C32 15F3 ADCD ECFB 6239
>

Re: Breaking DLEs up

[Chris Nighswonger]

On Thu, Nov 8, 2018 at 10:57 PM Nathan Stratton Treadway 
wrote:

> On Thu, Nov 08, 2018 at 21:24:04 -0500, Chris Nighswonger wrote:
> > Dump program is GNUTAR. I can switch to amgtar.
> >
>
> Amgtar is more flexible to use (and easier to maintain, should
> development on Amanda ever resume), so while you're rolling this out
> it's probably worth switching.  (And it should let you work around this
> permission problem, too.)
>

This makes the most sense as it will hopefully require less maintenance.

Thanks to everyone for the help. It has been quite educational in many ways.

Kind regards,
Chris

Re: Breaking DLEs up

[Chris Nighswonger]

On Thu, Nov 8, 2018 at 5:35 PM Nathan Stratton Treadway
 wrote:
>
>
> I haven't quite followed this whole thread, either, but have you taken a
> look at
>   http://wiki.zmanda.com/index.php/How_To:Split_DLEs_With_Exclude_Lists
> ? It may help explain some of the nuances in how things work.
>

That information is very similar to the information in the example
disklist file included with Amanda.

> As Stefan mentioned, it would be helpeful to know what amanda version
> you are using, and which dump program.  If your client supports it,
> using APPLICATION amgtar (rather than GNUTAR) is definitely a good idea.

This client is running 3.3.1.

The server is running 3.3.6.

Dump program is GNUTAR. I can switch to amgtar.

>
> But, especially if you are using GNUTAR, an important thing to keep in
> mind is that the exclude list is processed by passing the whole list
> using an --exclude option to GNU tar, which then processes that list as
> it does the dump -- while in contrast the "include" options are
> processed ahead of time by Amanda to build a list of directories to pass
> on the command line (thus making up tar's list of "what should I be
> backing up"?).  The tricky thing is that the tar process runs with root
> priviledge, so it can see any directory out there... but the "include"
> work is done by an unprivileged process and so the directory *above* the
> include pattern must be readable by the amanda user...
>

That explains a lot.

> So, what are the permissions on the /netdrives/CAMPUS/ directory itself?

root@fileserver:~# stat -c "%a %n" /netdrives
755 /netdrives
root@fileserver:~# stat -c "%a %n" /netdrives/CAMPUS
2771 /netdrives/CAMPUS

Which explains the problem if I understand correctly. 771 would
prohibit the amanda user from reading everything below CAMPUS.

Chris

Re: Breaking DLEs up

[Chris Nighswonger]

On Thu, Nov 8, 2018 at 1:56 PM Cuttler, Brian R (HEALTH) <
brian.cutt...@health.ny.gov> wrote:

>
>
> Your syntax
>
>
>
> fileserver "/netdrives/CAMPUS/af" "/netdrives/CAMPUS" {
>   comp-tar
>   include "./[a-f]*"
>   estimate server
> }
>
>
>
> my syntax
>
>
>
> finsen  /export/home-A /export/home   {
>
> user-tar2
>
> include "./[a]*"
>
> }
>
>
>
> finsen  /export/home-AZ /export/home   {
>
> user-tar2
>
> include "./[A-Z]*"
>
> }
>
>
>

Well, this fixes my problem, though why I do not know.

fileserver CAMPUS_a-f /netdrives/CAMPUS {
  comp-tar
  exclude file "./[g-z]*"
  estimate server
} 1

It seems a bit of work compared to the include directive. I tried "include
file" to no avail.

I'll see how the backup runs tonight, but amcheck likes it.

Kind regards,
Chris

Re: Breaking DLEs up

[Chris Nighswonger]

On Thu, Nov 8, 2018 at 2:57 PM Alan Hodgson 
wrote:

>
> I looked at your original post and it looked like it should work, and
> the logs show it mostly did work, it just didn't see anything to back
> up. I'd suspect permission problems or a bug in your version of tar
> maybe. The only thing the logs don't show are the contents of
> /tmp/amanda/sendbackup._netdrives_CAMPUS_sz.20181108052711.include,
> which presumable has the regex in it. You should be able to recreate
> that and test the tar command manually.
>
>
I finally looked in the right place and located the sendbackup...include
files: sendbackup._netdrives_CAMPUS_af.20181108052816.include, etc.

They are empty.

sendbackup...exclude files related to a completely different DLE on the
same server DO contain the correct exclude lists for that DLE.

That doesn't sound right.

Re: Breaking DLEs up

[Chris Nighswonger]

On Thu, Nov 8, 2018 at 2:34 PM Jon LaBadie  wrote:

> On Thu, Nov 08, 2018 at 01:24:46PM -0500, Chris Nighswonger wrote:
> > > Here is the relevant portion of my DLEs:
> > >
> > > fileserver "/netdrives/CAMPUS/af" "/netdrives/CAMPUS" {
> > >   comp-tar
> > >   include "./[a-f]*"
> > >   estimate server
> > > }
>
> One difference I noticed.  Deb, Brian, and myself all use
> "disknames" that are not pathnames.  I've not heard of pathnanmes
> causing a problem, but you might consider changine /netdrives/CAMPUS/af
> to something like CAMPUSa-f.
>
>
You may be on to something there. I can easily conceive of Amanda not
discerning the difference given that the default behavior is to use the
diskname as the diskdevice. [1]

Kind regards,
Chris


[1] https://wiki.zmanda.com/man/disklist.5.html

Re: Breaking DLEs up

[Chris Nighswonger]

No question is stupid. I learned that beating my head against the wall for
long hours. :-)

/netdrives/CAMPUS/ is a path which contains users' network drives. The
level below CAMPUS contains folders which follow the naming convention of
the username of each account. ie. Chris Nighswonger would be cnighswonger
and thus /netdrives/CAMPUS/cnighswonger/ would be my related network drive.

There are somewhat less than 100 user directories. Prior to this I have
been backing them all up with a DLE which looks like this:

host "/netdrives/CAMPUS" {
  comp-tar
  estimate server
}

This works fine with the caveat that it results in a huge level 0 backup.

In the supplied disklist file example in Amanda's documentation
(/usr/share/doc/amanda-server/examples/disklist), I discovered the DLE form
I am currently attempting. According to the example this should limit each
DLE to backing up subdirectories of /netdrives/CAMPUS/ based on the regexp
supplied in the "include" directive.

It appears to me that something may have changed with the way Amanda
handles this since that document was written.

As Stefan points out, Amanda seems to thing that there is "nothing" to be
backed up. Furthermore, it appears that the log excerpts I posted also show
that the regexp is not being applied but that Amanda is actually looking
for specific subdirectories like /netdrives/CAMPUS/af and the like.

Maybe the DLE syntax is incorrect?

On Thu, Nov 8, 2018 at 12:31 PM Cuttler, Brian R (HEALTH) <
brian.cutt...@health.ny.gov> wrote:

>
>
>
>
> Stupid question, host fileserver, directories  /netdrives/CAMPUS/s* to
> /netdrives/CAMPUS/z* exist and have some files in them?
>
>
>
> *From:* Chris Nighswonger 
> *Sent:* Thursday, November 8, 2018 12:11 PM
> *To:* Cuttler, Brian R (HEALTH) 
> *Cc:* amanda-users@amanda.org
> *Subject:* Re: Breaking DLEs up
>
>
>
> *ATTENTION: This email came from an external source. Do not open
> attachments or click on links from unknown senders or unexpected emails.*
>
> From the client:
>
>
>
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23692 ruid 10195
> euid 10195 version 3.3.1: start at Thu Nov  8 05:27:11 2018
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Version 3.3.1
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23692 ruid 10195
> euid 10195 version 3.3.1: rename at Thu Nov  8 05:27:11 2018
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:   Parsed request as:
> program `GNUTAR'
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
> disk `/netdrives/CAMPUS/sz'
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
> device `/netdrives/CAMPUS'
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
> level 0
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
> since NODATE
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
> options `'
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
> datapath `AMANDA'
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: start:
> host:/netdrives/CAMPUS/sz lev 0
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Spawning "/bin/gzip
> /bin/gzip --fast" in pipeline
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: gnutar: pid 23694:
> /bin/gzipThu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23694:
> /bin/gzip --fast
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: doing level 0 dump as
> listed-incremental to
> '/var/lib/amanda/gnutar-lists/host_netdrives_CAMPUS_sz_0.new'
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Nothing found to
> include for disk /netdrives/CAMPUS/sz
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Spawning
> "/usr/libexec/amanda/runtar runtar campus /bin/tar --create --file -
> --directory /netdrives/CAMPUS --one-file-system --listed-incremental
> /var/lib/amanda/gnutar-lists/host_netdrives_CAMPUS_sz_0.new --sparse
> --ignore-failed-read --totals --files-from
> /tmp/amanda/sendbackup._netdrives_CAMPUS_sz.20181108052711.include" in
> pipeline
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: gnutar:
> /usr/libexec/amanda/runtar: pid 23696
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Started backup
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Started index
> creator: "/bin/tar -tf - 2>/dev/null | sed -e 's/^\.//'"
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:  46:size(|):
> Total bytes written: 10240 (10KiB, 78MiB/s)
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Index created
> successfully
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Parsed backup messages
> Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23692 finish time
> Thu Nov  8 05:27:11 2018
>
>
>
> From the server:
>
>
>
> /v

Re: Breaking DLEs up

[Chris Nighswonger]

>From the client:

Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23692 ruid 10195
euid 10195 version 3.3.1: start at Thu Nov  8 05:27:11 2018
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Version 3.3.1
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23692 ruid 10195
euid 10195 version 3.3.1: rename at Thu Nov  8 05:27:11 2018
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:   Parsed request as:
program `GNUTAR'
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
disk `/netdrives/CAMPUS/sz'
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
device `/netdrives/CAMPUS'
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
level 0
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
since NODATE
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
options `'
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:
datapath `AMANDA'
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: start:
host:/netdrives/CAMPUS/sz lev 0
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Spawning "/bin/gzip
/bin/gzip --fast" in pipeline
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: gnutar: pid 23694:
/bin/gzipThu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23694:
/bin/gzip --fast
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: doing level 0 dump as
listed-incremental to
'/var/lib/amanda/gnutar-lists/host_netdrives_CAMPUS_sz_0.new'
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Nothing found to
include for disk /netdrives/CAMPUS/sz
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Spawning
"/usr/libexec/amanda/runtar runtar campus /bin/tar --create --file -
--directory /netdrives/CAMPUS --one-file-system --listed-incremental
/var/lib/amanda/gnutar-lists/host_netdrives_CAMPUS_sz_0.new --sparse
--ignore-failed-read --totals --files-from
/tmp/amanda/sendbackup._netdrives_CAMPUS_sz.20181108052711.include" in
pipeline
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: gnutar:
/usr/libexec/amanda/runtar: pid 23696
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Started backup
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Started index creator:
"/bin/tar -tf - 2>/dev/null | sed -e 's/^\.//'"
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup:  46:size(|): Total
bytes written: 10240 (10KiB, 78MiB/s)
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Index created
successfully
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: Parsed backup messages
Thu Nov  8 05:27:11 2018: thd-0x1c8f200: sendbackup: pid 23692 finish time
Thu Nov  8 05:27:11 2018

>From the server:

/var/log/amanda/server/campus/dumper.20181108020002007.debug:165691:Thu
Nov  8 05:27:11 2018: thd-0x5579048e2400: dumper: getcmd: PORT-DUMP
03-00023 50013 1 host 9efefbff1f /netdrives/CAMPUS/sz
/netdrives/CAMPUS 0 1970:1:1:0:0:0 GNUTAR "" "" "" "" bsdtcp AMANDA
127.0.0.1:50014 20 |"  bsdtcp\n  FAST\n
YES\n  YES\n
AMANDA\n  \n./[s-z]*\n
\n"
/var/log/amanda/server/campus/dumper.20181108020002007.debug:165705:
/netdrives/CAMPUS/sz
/var/log/amanda/server/campus/dumper.20181108020002007.debug:165706:
/netdrives/CAMPUS
/var/log/amanda/server/campus/dumper.20181108020002007.debug:165740:
/netdrives/CAMPUS/sz
/var/log/amanda/server/campus/dumper.20181108020002007.debug:165741:
/netdrives/CAMPUS
/var/log/amanda/server/campus/dumper.20181108020002007.debug:165857:Thu
Nov  8 05:27:11 2018: thd-0x5579048e2400: dumper: Building type FILE header
of 32768-32768 bytes with name='host' disk='/netdrives/CAMPUS/sz'
dumplevel=0 and blocksize=32768
/var/log/amanda/server/campus/dumper.20181108020002007.debug:165944:Thu
Nov  8 05:27:11 2018: thd-0x5579048e2400: dumper: Building type FILE header
of 32768-32768 bytes with name='host' disk='/netdrives/CAMPUS/sz'
dumplevel=0 and blocksize=32768


On Thu, Nov 8, 2018 at 12:00 PM Cuttler, Brian R (HEALTH) <
brian.cutt...@health.ny.gov> wrote:

> Client and server side?
>
> /var/log/amanda/ ?
>
>
>
>
>
> *From:* Chris Nighswonger 
> *Sent:* Thursday, November 8, 2018 11:43 AM
> *To:* Cuttler, Brian R (HEALTH) 
> *Cc:* amanda-users@amanda.org
> *Subject:* Re: Breaking DLEs up
>
>
>
> *ATTENTION: This email came from an external source. Do not open
> attachments or click on links from unknown senders or unexpected emails.*
>
> Oddly enough, /tmp/amanda is empty.
>
>
>
> On Thu, Nov 8, 2018 at 11:33 AM Cuttler, Brian R (HEALTH) <
> brian.cutt...@health.ny.gov> wrote:
>
> I have been using a very similar setup for years, though I did not have
> any quotes in the first line of each DLE. I do NOT believe the quotes are
> an issue.
>
>
>
> What do the /tmp/amanda files show for these attempted dumps?
>
>
>
> *From:* owner-amanda-us...@amanda.org  *On
> Behalf Of *Chris Nighswonger
> *Sent:* Thursday, November 8, 2018 11:12 AM
> *To

Re: Breaking DLEs up

[Chris Nighswonger]

Oddly enough, /tmp/amanda is empty.

On Thu, Nov 8, 2018 at 11:33 AM Cuttler, Brian R (HEALTH) <
brian.cutt...@health.ny.gov> wrote:

> I have been using a very similar setup for years, though I did not have
> any quotes in the first line of each DLE. I do NOT believe the quotes are
> an issue.
>
>
>
> What do the /tmp/amanda files show for these attempted dumps?
>
>
>
> *From:* owner-amanda-us...@amanda.org  *On
> Behalf Of *Chris Nighswonger
> *Sent:* Thursday, November 8, 2018 11:12 AM
> *To:* amanda-users@amanda.org
> *Subject:* Breaking DLEs up
>
>
>
> *ATTENTION: This email came from an external source. Do not open
> attachments or click on links from unknown senders or unexpected emails.*
>
> I attempted this and it appears to not have worked. I'm not sure why.
>
>
>
> Here is the relevant portion of my DLEs:
>
>
>
> fileserver "/netdrives/CAMPUS/af" "/netdrives/CAMPUS" {
>   comp-tar
>   include "./[a-f]*"
>   estimate server
> }
> fileserver "/netdrives/CAMPUS/gl" "/netdrives/CAMPUS" {
>   comp-tar
>   include "./[g-l]*"
>   estimate server
> }
> fileserver "/netdrives/CAMPUS/mr" "/netdrives/CAMPUS" {
>   comp-tar
>   include "./[m-r]*"
>   estimate server
> }
> fileserver "/netdrives/CAMPUS/sz" "/netdrives/CAMPUS" {
>   comp-tar
>   include "./[s-z]*"
>   estimate server
> }
>
>
>
> Here are the corresponding lines from amreport for the last backup run:
>
>
>
> fileserver:/netdrives/CAMPUS/af
> 0 1k dump done (5:28:16), waiting for writing to tape
> fileserver:/netdrives/CAMPUS/gl
> 0 1k dump done (5:28:11), waiting for writing to tape
> fileserver:/netdrives/CAMPUS/mr
> 0 1k dump done (5:28:06), waiting for writing to tape
> fileserver:/netdrives/CAMPUS/sz
> 0 1k dump done (5:27:11), waiting for writing to tape
>
>
>
> Kind regards,
>
> Chris
>

Breaking DLEs up

[Chris Nighswonger]

I attempted this and it appears to not have worked. I'm not sure why.

Here is the relevant portion of my DLEs:

fileserver "/netdrives/CAMPUS/af" "/netdrives/CAMPUS" {
  comp-tar
  include "./[a-f]*"
  estimate server
}
fileserver "/netdrives/CAMPUS/gl" "/netdrives/CAMPUS" {
  comp-tar
  include "./[g-l]*"
  estimate server
}
fileserver "/netdrives/CAMPUS/mr" "/netdrives/CAMPUS" {
  comp-tar
  include "./[m-r]*"
  estimate server
}
fileserver "/netdrives/CAMPUS/sz" "/netdrives/CAMPUS" {
  comp-tar
  include "./[s-z]*"
  estimate server
}

Here are the corresponding lines from amreport for the last backup run:

fileserver:/netdrives/CAMPUS/af
0 1k dump done (5:28:16), waiting for writing to tape
fileserver:/netdrives/CAMPUS/gl
0 1k dump done (5:28:11), waiting for writing to tape
fileserver:/netdrives/CAMPUS/mr
0 1k dump done (5:28:06), waiting for writing to tape
fileserver:/netdrives/CAMPUS/sz
0 1k dump done (5:27:11), waiting for writing to tape

Kind regards,
Chris

Re: sharing scripts (was: Can AMANDA be configured "per client"?)

[Chris Nighswonger]

Why not form a document git repo to house just Amanda docs?

Kind regards,
Chris
On Thu, Nov 8, 2018 at 7:45 AM Stefan G. Weichinger  wrote:
>
> Am 08.11.18 um 12:46 schrieb Gene Heskett:
> > On Thursday 08 November 2018 05:02:30 Stefan G. Weichinger wrote:
> >
> > For those who might be interested, the link on my web page that leads
> > to "Genes-os9-stf", now has a new readme for GenesAmandaHelper, and a
> > fresh backup of that whole thing as 00029etcetc. Adjust to suit your
> > system.  Its been in use here since 2002. Currently working with
> > amanda-3.5.1.
>
>
> A small suggestion here:
>
> instead of posting the script to the list over and over again (thanks
> for doing so! no offense intended!) we could think of collecting
> knowledge and tools like this and maybe provide stuff via a github-repo
> or so (as done by other oss-projects ...).
>
> Currently 57 forks of the zmanda/amanda repo exist, let's choose one and
> commit our tricks etc there.
>
> We can't wait for betsol afai see ...
>
> Feedback welcome here.
>
> (to be clear: I don't say "fork amanda" here, I just say "let's collect
> the spread hints and tips shared via the mailing list and build up some
> knowledge base there")
>
> -
>
> Additionally I think of adding and maintaining updated versions of the
> vtapes-howto there etc
>
> Stefan

Re: dumporder

[Chris Nighswonger]

This seems to work:

amplot /var/backups/campus/log/amdump.1

Running under the amanda user.

However, the issue now is the attempt to write the output to the X11 terminal:


gnuplot: unable to open display ''
gnuplot: X11 aborted.

Not sure what all that's about. So I'm doing a bit of hacking on the
gnuplot script to have it write the results out to a png file.

Chris
On Tue, Nov 6, 2018 at 12:29 PM Ned Danieley  wrote:
>
> On Tue, Nov 06, 2018 at 11:50:57AM -0500, Chris Nighswonger wrote:
> > Digging around a bit, it appears that it might be a reference to a
> > file which is missing. From amplot.g, line 62 we see:
> >
> > # file title has the parameters that this program needs
> > load 'title'
> > plot"run_queue" title "Run Queue" with lines,\
> > "tape_queue" title "Tape Queue" with lines,\
> > "finished"  title "Dumps Finished" with lines,\
> > "bandw_free" title "Bandwidth Allocated" with lines, \
> > "disk_alloc" title "%Disk Allocated" with lines, \
> > "tape_wait" title "%Tape Wait" with lines,\
> > "tape_idle" title "Taper Idle" with lines,\
> > "dump_idle" title "Dumpers Idle" with lines
> >
> > Where is a developer when you need one? :-P
>
> looks like the awk script is supposed to generate 'title'. on my system, I
> have to run amplot as user 'amanda'. that means that I have to be in a
> directory where amanda has write permission, otherwise title can't be
> generated. my home directory doesn't work, but a temp dir that's chmod 777
> does.
>
> --
> Ned Danieley (ned.danie...@duke.edu)
> Department of Biomedical Engineering
> Box 90281, Duke University
> Durham, NC  27708   (919) 660-5111
>
> http://dilbert.com/strips/comic/2012-02-11/

Re: dumporder

[Chris Nighswonger]

My bad here. I should have inferred this from the man page for amplot,
but I've been so wound up in the debug logs that I forgot there are
others. This seems to work:

amplot /var/backups/campus/log/amdump.1
On Tue, Nov 6, 2018 at 11:54 AM Cuttler, Brian R (HEALTH)
 wrote:
>
> I think you need to provide the name of the amconfig. I believe it reads 
> amanda.conf.
>
> -Original Message-
> From: Chris Nighswonger 
> Sent: Tuesday, November 6, 2018 11:51 AM
> To: Cuttler, Brian R (HEALTH) 
> Cc: amanda-users@amanda.org
> Subject: Re: dumporder
>
> ATTENTION: This email came from an external source. Do not open attachments 
> or click on links from unknown senders or unexpected emails.
>
>
> Digging around a bit, it appears that it might be a reference to a file which 
> is missing. From amplot.g, line 62 we see:
>
> # file title has the parameters that this program needs load 'title'
> plot"run_queue" title "Run Queue" with lines,\
> "tape_queue" title "Tape Queue" with lines,\
> "finished"  title "Dumps Finished" with lines,\
> "bandw_free" title "Bandwidth Allocated" with lines, \
> "disk_alloc" title "%Disk Allocated" with lines, \
> "tape_wait" title "%Tape Wait" with lines,\
> "tape_idle" title "Taper Idle" with lines,\
> "dump_idle" title "Dumpers Idle" with lines
>
> Where is a developer when you need one? :-P
>
> On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH) 
>  wrote:
> >
> > It has been a while, title might be the org string from amanda.conf?
> > I'm sorry, it has been years since I ran it regularly.
> >
> > -Original Message-
> > From: Chris Nighswonger 
> > Sent: Tuesday, November 6, 2018 11:42 AM
> > To: Cuttler, Brian R (HEALTH) 
> > Cc: amanda-users@amanda.org
> > Subject: Re: dumporder
> >
> > ATTENTION: This email came from an external source. Do not open attachments 
> > or click on links from unknown senders or unexpected emails.
> >
> >
> > The amplot utility is new to me. Here is what it says when I attempt to run 
> > it:
> >
> > root@scriptor:~# amplot
> > /var/log/amanda/server/campus/amdump.20181106020001.debug
> > Displaying graph on the screen,  for next graph : MISSING SPACE 
> > DECLARATION "title", line 62: Cannot open script file 'title'
> >
> > I have both gnuplot and gawk installed on the backup server.
> >
> > On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) 
> >  wrote:
> > >
> > > Chris,
> > >
> > > There is an amplot utility that I used to run a lot, it would show me 
> > > what I was constrained on, often holding space, which if you are running 
> > > a bunch of large dumps to start off with could constrain dumping later on.
> > >
> > >
> > > -Original Message-
> > > From: owner-amanda-us...@amanda.org 
> > > On Behalf Of Chris Nighswonger
> > > Sent: Tuesday, November 6, 2018 11:02 AM
> > > To: amanda-users@amanda.org
> > > Subject: Re: dumporder
> > >
> > > ATTENTION: This email came from an external source. Do not open 
> > > attachments or click on links from unknown senders or unexpected emails.
> > >
> > >
> > > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger 
> > >  wrote:
> > > >
> > > > Is there any wisdom available on optimization of dumporder?
> > > >
> > >
> > > After looking over the feedback from Brian and Austin and reviewing the 
> > > actual sizes of the DLEs, I ended up with this sort of thing:
> > >
> > > inparallel 15
> > > dumporder "STs"
> > >
> > > Which resulted in a reduced time over what I have been seeing. Here are 
> > > some stats from amstatus for this config. It looks like I could drop 
> > > about half of the dumpers and still be fine. Any idea what causes the 
> > > dumpers over the first five to be utilized less?
> > >
> > >  dumper0 busy   :  5:13:42  ( 97.98%)
> > >  dumper1 busy   :  1:12:09  ( 22.54%)
> > >  dumper2 busy   :  0:40:30  ( 12.65%)
> > >  dumper3 busy   :  0:33:44  ( 10.54%)
> > >  dumper4 busy   :  0:03:47  (  1.19%)
> > >  dumper5 busy   :  0:37:32  ( 11.73%)
> > >  dumper6 busy   :  0:02:00  (  0.62%)
> > >  dumper7 busy   :  0:00:57  (  0.30%)
> > >  dumper8 busy 

Re: dumporder

[Chris Nighswonger]

Digging around a bit, it appears that it might be a reference to a
file which is missing. From amplot.g, line 62 we see:

# file title has the parameters that this program needs
load 'title'
plot"run_queue" title "Run Queue" with lines,\
"tape_queue" title "Tape Queue" with lines,\
"finished"  title "Dumps Finished" with lines,\
"bandw_free" title "Bandwidth Allocated" with lines, \
"disk_alloc" title "%Disk Allocated" with lines, \
"tape_wait" title "%Tape Wait" with lines,\
"tape_idle" title "Taper Idle" with lines,\
"dump_idle" title "Dumpers Idle" with lines

Where is a developer when you need one? :-P

On Tue, Nov 6, 2018 at 11:45 AM Cuttler, Brian R (HEALTH)
 wrote:
>
> It has been a while, title might be the org string from amanda.conf?
> I'm sorry, it has been years since I ran it regularly.
>
> -Original Message-
> From: Chris Nighswonger 
> Sent: Tuesday, November 6, 2018 11:42 AM
> To: Cuttler, Brian R (HEALTH) 
> Cc: amanda-users@amanda.org
> Subject: Re: dumporder
>
> ATTENTION: This email came from an external source. Do not open attachments 
> or click on links from unknown senders or unexpected emails.
>
>
> The amplot utility is new to me. Here is what it says when I attempt to run 
> it:
>
> root@scriptor:~# amplot
> /var/log/amanda/server/campus/amdump.20181106020001.debug
> Displaying graph on the screen,  for next graph : MISSING SPACE 
> DECLARATION "title", line 62: Cannot open script file 'title'
>
> I have both gnuplot and gawk installed on the backup server.
>
> On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH) 
>  wrote:
> >
> > Chris,
> >
> > There is an amplot utility that I used to run a lot, it would show me what 
> > I was constrained on, often holding space, which if you are running a bunch 
> > of large dumps to start off with could constrain dumping later on.
> >
> >
> > -Original Message-
> > From: owner-amanda-us...@amanda.org  On
> > Behalf Of Chris Nighswonger
> > Sent: Tuesday, November 6, 2018 11:02 AM
> > To: amanda-users@amanda.org
> > Subject: Re: dumporder
> >
> > ATTENTION: This email came from an external source. Do not open attachments 
> > or click on links from unknown senders or unexpected emails.
> >
> >
> > On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger 
> >  wrote:
> > >
> > > Is there any wisdom available on optimization of dumporder?
> > >
> >
> > After looking over the feedback from Brian and Austin and reviewing the 
> > actual sizes of the DLEs, I ended up with this sort of thing:
> >
> > inparallel 15
> > dumporder "STs"
> >
> > Which resulted in a reduced time over what I have been seeing. Here are 
> > some stats from amstatus for this config. It looks like I could drop about 
> > half of the dumpers and still be fine. Any idea what causes the dumpers 
> > over the first five to be utilized less?
> >
> >  dumper0 busy   :  5:13:42  ( 97.98%)
> >  dumper1 busy   :  1:12:09  ( 22.54%)
> >  dumper2 busy   :  0:40:30  ( 12.65%)
> >  dumper3 busy   :  0:33:44  ( 10.54%)
> >  dumper4 busy   :  0:03:47  (  1.19%)
> >  dumper5 busy   :  0:37:32  ( 11.73%)
> >  dumper6 busy   :  0:02:00  (  0.62%)
> >  dumper7 busy   :  0:00:57  (  0.30%)
> >  dumper8 busy   :  0:05:54  (  1.85%)
> >  dumper9 busy   :  0:04:38  (  1.45%)
> > dumper10 busy   :  0:00:16  (  0.08%)
> > dumper11 busy   :  0:01:39  (  0.52%)
> > dumper12 busy   :  0:00:01  (  0.01%)
> >  0 dumpers busy :  0:02:43  (  0.85%)   0:  0:02:43  
> > (100.00%)
> >  1 dumper busy  :  3:39:13  ( 68.47%)   0:  3:39:13  
> > (100.00%)
> >  2 dumpers busy :  0:23:10  (  7.24%)   0:  0:23:10  
> > (100.00%)
> >  3 dumpers busy :  1:07:57  ( 21.22%)   0:  1:07:57  
> > (100.00%)
> >  4 dumpers busy :  0:02:09  (  0.67%)   0:  0:02:09  ( 
> > 99.99%)
> >  5 dumpers busy :  0:01:07  (  0.35%)   0:  0:01:07  ( 
> > 99.98%)
> >  6 dumpers busy :  0:00:03  (  0.02%)   0:  0:00:03  ( 
> > 99.59%)
> >  7 dumpers busy :  0:00:43  (  0.22%)   0:  0:00:24  ( 
> > 56.35%)
> > 5:  0:00:09  ( 
> > 22.36%)
> > 4:  0

Re: dumporder

[Chris Nighswonger]

The amplot utility is new to me. Here is what it says when I attempt to run it:

root@scriptor:~# amplot
/var/log/amanda/server/campus/amdump.20181106020001.debug
Displaying graph on the screen,  for next graph : MISSING SPACE DECLARATION
"title", line 62: Cannot open script file 'title'

I have both gnuplot and gawk installed on the backup server.

On Tue, Nov 6, 2018 at 11:10 AM Cuttler, Brian R (HEALTH)
 wrote:
>
> Chris,
>
> There is an amplot utility that I used to run a lot, it would show me what I 
> was constrained on, often holding space, which if you are running a bunch of 
> large dumps to start off with could constrain dumping later on.
>
>
> -Original Message-
> From: owner-amanda-us...@amanda.org  On Behalf 
> Of Chris Nighswonger
> Sent: Tuesday, November 6, 2018 11:02 AM
> To: amanda-users@amanda.org
> Subject: Re: dumporder
>
> ATTENTION: This email came from an external source. Do not open attachments 
> or click on links from unknown senders or unexpected emails.
>
>
> On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger 
>  wrote:
> >
> > Is there any wisdom available on optimization of dumporder?
> >
>
> After looking over the feedback from Brian and Austin and reviewing the 
> actual sizes of the DLEs, I ended up with this sort of thing:
>
> inparallel 15
> dumporder "STs"
>
> Which resulted in a reduced time over what I have been seeing. Here are some 
> stats from amstatus for this config. It looks like I could drop about half of 
> the dumpers and still be fine. Any idea what causes the dumpers over the 
> first five to be utilized less?
>
>  dumper0 busy   :  5:13:42  ( 97.98%)
>  dumper1 busy   :  1:12:09  ( 22.54%)
>  dumper2 busy   :  0:40:30  ( 12.65%)
>  dumper3 busy   :  0:33:44  ( 10.54%)
>  dumper4 busy   :  0:03:47  (  1.19%)
>  dumper5 busy   :  0:37:32  ( 11.73%)
>  dumper6 busy   :  0:02:00  (  0.62%)
>  dumper7 busy   :  0:00:57  (  0.30%)
>  dumper8 busy   :  0:05:54  (  1.85%)
>  dumper9 busy   :  0:04:38  (  1.45%)
> dumper10 busy   :  0:00:16  (  0.08%)
> dumper11 busy   :  0:01:39  (  0.52%)
> dumper12 busy   :  0:00:01  (  0.01%)
>  0 dumpers busy :  0:02:43  (  0.85%)   0:  0:02:43  (100.00%)
>  1 dumper busy  :  3:39:13  ( 68.47%)   0:  3:39:13  (100.00%)
>  2 dumpers busy :  0:23:10  (  7.24%)   0:  0:23:10  (100.00%)
>  3 dumpers busy :  1:07:57  ( 21.22%)   0:  1:07:57  (100.00%)
>  4 dumpers busy :  0:02:09  (  0.67%)   0:  0:02:09  ( 99.99%)
>  5 dumpers busy :  0:01:07  (  0.35%)   0:  0:01:07  ( 99.98%)
>  6 dumpers busy :  0:00:03  (  0.02%)   0:  0:00:03  ( 99.59%)
>  7 dumpers busy :  0:00:43  (  0.22%)   0:  0:00:24  ( 56.35%)
> 5:  0:00:09  ( 22.36%)
> 4:  0:00:08  ( 18.89%)
> 1:  0:00:01  (  2.37%)
>  8 dumpers busy :  0:00:20  (  0.10%)   0:  0:00:19  ( 96.50%)
>  9 dumpers busy :  0:01:51  (  0.58%)   0:  0:01:51  ( 99.98%)
> 10 dumpers busy :  0:00:41  (  0.22%)   0:  0:00:37  ( 89.71%)
> 5:  0:00:02  (  7.13%)
> 4:  0:00:01  (  3.09%)
> 11 dumpers busy :  0:00:07  (  0.04%)   5:  0:00:07  ( 99.54%)
> 12 dumpers busy :  0:00:00  (  0.00%)
> 13 dumpers busy :  0:00:00  (  0.00%)
>
> I am going to give things another week or so and then try breaking up some of 
> the very large DLEs in the config.

Re: dumporder

[Chris Nighswonger]

On Mon, Nov 5, 2018 at 1:31 PM Chris Nighswonger
 wrote:
>
> Is there any wisdom available on optimization of dumporder?
>

After looking over the feedback from Brian and Austin and reviewing
the actual sizes of the DLEs, I ended up with this sort of thing:

inparallel 15
dumporder "STs"

Which resulted in a reduced time over what I have been seeing. Here
are some stats from amstatus for this config. It looks like I could
drop about half of the dumpers and still be fine. Any idea what causes
the dumpers over the first five to be utilized less?

 dumper0 busy   :  5:13:42  ( 97.98%)
 dumper1 busy   :  1:12:09  ( 22.54%)
 dumper2 busy   :  0:40:30  ( 12.65%)
 dumper3 busy   :  0:33:44  ( 10.54%)
 dumper4 busy   :  0:03:47  (  1.19%)
 dumper5 busy   :  0:37:32  ( 11.73%)
 dumper6 busy   :  0:02:00  (  0.62%)
 dumper7 busy   :  0:00:57  (  0.30%)
 dumper8 busy   :  0:05:54  (  1.85%)
 dumper9 busy   :  0:04:38  (  1.45%)
dumper10 busy   :  0:00:16  (  0.08%)
dumper11 busy   :  0:01:39  (  0.52%)
dumper12 busy   :  0:00:01  (  0.01%)
 0 dumpers busy :  0:02:43  (  0.85%)   0:  0:02:43  (100.00%)
 1 dumper busy  :  3:39:13  ( 68.47%)   0:  3:39:13  (100.00%)
 2 dumpers busy :  0:23:10  (  7.24%)   0:  0:23:10  (100.00%)
 3 dumpers busy :  1:07:57  ( 21.22%)   0:  1:07:57  (100.00%)
 4 dumpers busy :  0:02:09  (  0.67%)   0:  0:02:09  ( 99.99%)
 5 dumpers busy :  0:01:07  (  0.35%)   0:  0:01:07  ( 99.98%)
 6 dumpers busy :  0:00:03  (  0.02%)   0:  0:00:03  ( 99.59%)
 7 dumpers busy :  0:00:43  (  0.22%)   0:  0:00:24  ( 56.35%)
5:  0:00:09  ( 22.36%)
4:  0:00:08  ( 18.89%)
1:  0:00:01  (  2.37%)
 8 dumpers busy :  0:00:20  (  0.10%)   0:  0:00:19  ( 96.50%)
 9 dumpers busy :  0:01:51  (  0.58%)   0:  0:01:51  ( 99.98%)
10 dumpers busy :  0:00:41  (  0.22%)   0:  0:00:37  ( 89.71%)
5:  0:00:02  (  7.13%)
4:  0:00:01  (  3.09%)
11 dumpers busy :  0:00:07  (  0.04%)   5:  0:00:07  ( 99.54%)
12 dumpers busy :  0:00:00  (  0.00%)
13 dumpers busy :  0:00:00  (  0.00%)

I am going to give things another week or so and then try breaking up
some of the very large DLEs in the config.

dumporder

[Chris Nighswonger]

Is there any wisdom available on optimization of dumporder?

Kind regards,
Chris

Re: All level 0 on the same run?

[Chris Nighswonger]

On Wed, Oct 31, 2018 at 6:30 PM Debra S Baddorf  wrote:

> We may have found an answer for Gene’s problem.
> Has the original poster, Chris Nighswonger  found an answer?
>
> Deb
>
>
Indeed! Enough to get me set off in what seems to be a right direction.

Thanks to everyone for the kind assistance.

Kind regards,
Chris

Re: All level 0 on the same run?

[Chris Nighswonger]

On Wed, Oct 31, 2018 at 5:32 PM Nathan Stratton Treadway 
wrote:

>
> Am I correct that you actually ran two separate amdump runs within the
> calendar day of 10/30 (with the first "balance" command executed between
> the runs)?  That would explain why all 39 DLEs are now showing as due on
> the same day.
>
>
No. The first balance was run on 10/29. The second on 10/30.

Here is the balance after last night's run (10/31):

root@scriptor:/var/log/amanda/server/campus# su backup -c
"/usr/sbin/amadmin campus balance"

 due-date  #fsorig kB out kB   balance
--
11/01 Thu1  0  0  ---
11/02 Fri0  0  0  ---
11/03 Sat   21  899936964  623033612   +301.4%
11/04 Sun0  0  0  ---
11/05 Mon   18  179898725  153104174 -1.4%
--
TOTAL   40 1079835689  776137786 155227557
  (estimated 5 runs per dumpcycle)

Sounds to me like I need to give it another week or so to settle out. Then
revisit breaking up the two or three large DLEs.

Kind regards,
Chris

Re: All level 0 on the same run?

[Chris Nighswonger]

FWIW, here is the output of amadmin balance before last nights run and
again this morning. No overdues, so I guess that's good. I'm not
experienced enough to make much of the balance percentages, but am now
wondering if I should work at breaking up the large DLEs into smaller
subsets as several have suggested.

root@scriptor:/home/manager# su backup -c "/usr/sbin/amadmin campus balance"

 due-date  #fsorig kB out kB   balance
--
10/30 Tue   25  359009166  284972243   +102.1%

11/03 Sat   15  632526057  420083122   +197.9%
--
TOTAL   40  991535223  705055365 141011073
  (estimated 5 runs per dumpcycle)
 (13 filesystems overdue. The most being overdue 1 day.)
root@scriptor:/home/manager# su backup -c "/usr/sbin/amadmin campus balance"

 due-date  #fsorig kB out kB   balance
--
10/31 Wed1  0  0  ---

11/03 Sat   39 1079730080  776153947   +400.0%
11/04 Sun0  0  0  ---
--
TOTAL   40 1079730080  776153947 155230789
  (estimated 5 runs per dumpcycle)


On Tue, Oct 30, 2018 at 3:56 PM Gene Heskett  wrote:

> On Tuesday 30 October 2018 15:29:37 Nathan Stratton Treadway wrote:
>
> > On Tue, Oct 30, 2018 at 14:20:55 -0400, Chris Nighswonger wrote:
> > > Why in the world does Amanda plan level 0 backups for all entries in
> > > a DLE for the same run This causes all sorts of problems.
> > >
> > > Is there any solution for this? I've read some of the creative
> > > suggestions, but it seems a bunch of trouble.
> >
> > The operation of Amanda's planner depends on many inputs, both "fixed"
> > (e.g. configuration options) and constantly-varying (e.g. estimate
> > sizes and dump history), and I suspect there are only a few people in
> > the world who really understand it fully -- and I don't know how many
> > of them still read this mailing list :(.  But even one of those people
> > would probably need to look at a lot of information in order to know
> > what exactly was going on.
> >
> >
> > The good news is that I have noticed that the planner records a bunch
> > of interesting information in the amdump.DATETIMESTAMP log file, so at
> > least that seems like the place to start investigated.  Look in
> > particular for the following sections: DONE QUEUE, ANALYZING
> > ESTIMATES, INITIAL SCHEDULE, DELAYING DUMPS IF NEEDED, PROMOTING DUMPS
> > IF NEEDED, and finally GENERATING SCHEDULE.
> >
> > In your case, it seems likely that the  PROMOTING DUMPS section should
> > have a bunch of activity listed; if so, that might explain what it's
> > "thinking".
> >
> > If that doesn't give a clear answer, does the INITIAL SCHEDULE section
> > show all the dumps are already scheduled for level 0?  If not, pick a
> > DLE that is not shown at level 0 there and follow it down the log to
> > see if you can figure out what stage bumps it back to level 0...
> >
> >
> > On a different track of investigation, the  output of "amadmin CONFIG
> > balance" might show something useful (though off hand it seems
> > unlikely to explain why _all_ DLEs would be switched to level 0).
> >
> >
> > Let us know what you find out :)
> >   Nathan
> >
> I just changed the length of the dumpcycle and runs percycle up to 10,
> about last friday while I was makeing the bump* stuff more attractive,
> but the above command returns that the are 5 filesystens out of date:
> su amanda -c "/usr/local/sbin/amadmin Daily balance"
>
>  due-date  #fsorig MB out MB   balance
> --
> 10/30 Tue5  0  0  ---
> 10/31 Wed1  17355   8958-45.3%
> 11/01 Thu2  10896  10887-33.5%
> 11/02 Fri4  35944   9298-43.2%
> 11/03 Sat4  14122  10835-33.8%
> 11/04 Sun3  57736  57736   +252.7%
> 11/05 Mon2  39947  30635+87.1%
> 11/06 Tue8   4235   4215-74.3%
> 11/07 Wed4  19503  14732-10.0%
> 11/08 Thu   32  31783  16408 +0.2%
> --
> TOTAL   65 231521 163704 16370
>   (estimated 10 runs per dumpcycle)
>  (5 filesystems overdue. The most being overdue 20 days.)
>
> That last line is disturbing. Ideas anyone? I'll certainly keep an eye on
> it.
>
> Cheers & thanks, Gene Heskett
> --
> "There are four boxes to be used in defense of liberty:
>  soap, ballot, jury, and ammo. Please use in that order."
> -Ed Howdershelt (Author)
> Genes Web page <http://geneslinuxbox.net:6309/gene>
>

Re: All level 0 on the same run?

[Chris Nighswonger]

So, looking at this more, it may be self-inflicted. Last week I changed
blocksize to 512k, and began amrmtape and amlabel with the oldest tape
first and working backward day by day. I run backups 5 nights per week with
a cycle of 13 tapes (see below). I would have thought that this would have
allowed the change in blocksize to run seamlessly. Maybe not. I'm now
suspecting that by amrmtape --cleanup, this caused Amanda to bork and fall
back to level 0 backups. She did this two nights in a row!!!

Anyway, I'm going to hold off any further concerns until I finish a
complete tapecycle. If the problem continues after that point, I'll pick
back up.

Relevant lines from amanda.conf:

dumpcycle 5 days
runspercycle 5
tapecycle 13 tapes
runtapes 1
flush-threshold-dumped 50
bumpsize 10 Mbytes
bumppercent 0
bumpmult 1.5
bumpdays 2

Kind regards,
Chris

On Tue, Oct 30, 2018 at 2:32 PM Debra S Baddorf  wrote:

> Is this the first backup run for a long while?  If so, then they are all
> DUE, so amanda feels it has to schedule them all, now.
>
> Is this the first backup ever?   Ditto above.
>
> Did you perhaps run  “amadminforce  *”  which forces a level 0
> on all disks.
> Did you specify  “strategy noinc”which does the same?
> Or  "skip-incr yes”  ?  Ditto.
>
> Did you replace a whole disk, making all the files look like they’ve never
> been backed up?
>
> Okay,  failing all the above obvious reasons,  I’ll leave others to
> discuss “planner” reasons.  Sorry!
> Deb Baddorf
> Fermilab
>
> > On Oct 30, 2018, at 1:20 PM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
> >
> > Why in the world does Amanda plan level 0 backups for all entries in a
> DLE for the same run This causes all sorts of problems.
> > Is there any solution for this? I've read some of the creative
> suggestions, but it seems a bunch of trouble.
> > Kind regards,
> > Chris
> >
> > 0 19098649k waiting for dumping
> > 0 9214891k waiting for dumping
> > 0 718824k waiting for dumping
> > 0 365207k waiting for dumping
> > 0 2083027k waiting for dumping
> > 0 3886869k waiting for dumping
> > 0 84910k waiting for dumping
> > 0 22489k dump done (7:23:34), waiting for writing to tape
> > 0 304k dump done (7:22:30), waiting for writing to tape
> > 0 2613k waiting for dumping
> > 0 30k dump done (7:23:07), waiting for writing to tape
> > 0 39642k dump done (7:23:07), waiting for writing to tape
> > 0 8513409k waiting for dumping
> > 0 39519558k waiting for dumping
> > 0 47954k waiting for dumping
> > 0 149877984k dumping 145307840k ( 96.95%) (7:22:15)
> > 0 742804k waiting for dumping
> > p" 0 88758k waiting for dumping
> > 0 12463k dump done (7:24:19), waiting for writing to tape
> > 0 5544352k waiting for dumping
> > 0 191676480k waiting for dumping
> > 0 3799277k waiting for dumping
> > 0 3177171k waiting for dumping
> > 0 11058544k waiting for dumping
> > 0 230026440k dump done (7:22:13), waiting for writing to tape
> > 0 8k dump done (7:24:24), waiting for writing to tape
> > 0 184k dump done (7:24:19), waiting for writing to tape
> > 0 1292009k waiting for dumping
> > 0 2870k dump done (7:23:23), waiting for writing to tape
> > 0 13893263k waiting for dumping
> > 0 6025026k waiting for dumping
> > 0 6k dump done (7:22:15), waiting for writing to tape
> > 0 42k dump done (7:24:24), waiting for writing to tape
> > 0 53k dump done (7:24:19), waiting for writing to tape
> > 0 74462169k waiting for dumping
> > 0 205032k waiting for dumping
> > 0 32914k waiting for dumping
> > 0 1k dump done (7:24:02), waiting for writing to tape
> > 0 854272k waiting for dumping
> >
>
>

All level 0 on the same run?

[Chris Nighswonger]

Why in the world does Amanda plan level 0 backups for all entries in a DLE
for the same run This causes all sorts of problems.

Is there any solution for this? I've read some of the creative suggestions,
but it seems a bunch of trouble.

Kind regards,

Chris


0 19098649k waiting for dumping

0 9214891k waiting for dumping

0 718824k waiting for dumping

0 365207k waiting for dumping

0 2083027k waiting for dumping

0 3886869k waiting for dumping

0 84910k waiting for dumping

0 22489k dump done (7:23:34), waiting for writing to tape

0 304k dump done (7:22:30), waiting for writing to tape

0 2613k waiting for dumping

0 30k dump done (7:23:07), waiting for writing to tape

0 39642k dump done (7:23:07), waiting for writing to tape

0 8513409k waiting for dumping

0 39519558k waiting for dumping

0 47954k waiting for dumping

0 149877984k dumping 145307840k ( 96.95%) (7:22:15)

0 742804k waiting for dumping

p" 0 88758k waiting for dumping

0 12463k dump done (7:24:19), waiting for writing to tape

0 5544352k waiting for dumping

0 191676480k waiting for dumping

0 3799277k waiting for dumping

0 3177171k waiting for dumping

0 11058544k waiting for dumping

0 230026440k dump done (7:22:13), waiting for writing to tape

0 8k dump done (7:24:24), waiting for writing to tape

0 184k dump done (7:24:19), waiting for writing to tape

0 1292009k waiting for dumping

0 2870k dump done (7:23:23), waiting for writing to tape

0 13893263k waiting for dumping

0 6025026k waiting for dumping

0 6k dump done (7:22:15), waiting for writing to tape

0 42k dump done (7:24:24), waiting for writing to tape

0 53k dump done (7:24:19), waiting for writing to tape

0 74462169k waiting for dumping

0 205032k waiting for dumping

0 32914k waiting for dumping

0 1k dump done (7:24:02), waiting for writing to tape

0 854272k waiting for dumping

Re: Amanda calling tapes full which are not

[Chris Nighswonger]

On Wed, Oct 24, 2018 at 2:05 PM Debra S Baddorf  wrote:
>
> “waits till 6 tapes are full before a problem”  and  “errors on amrmtape”
>
> While I didn’t think amrmtape actually touched the tape  (merely the files 
> containing amanda’s memory),
> it kinda sounds like your tape drive can write,  but not over-write.Dunno 
> if that’s a useful point to
> tell your repair team?

Several reboots of both the library and server cleared the CRC errors
on the bus. So this is probably a hardware issue. At least I'll
proceed with that assumption for now and see how far it gets me.

Kind regards,
Chris

Re: Amanda calling tapes full which are not

[Chris Nighswonger]

On Wed, Oct 24, 2018 at 1:41 PM Debra S Baddorf  wrote:
>
> I agree with the blocksize comment below.   Also:  doing   amrmtape  (remove 
> tape)
> on that label, before relabelling it,  can’t hurt.   If you’re having a 
> hardware issue, it won’t help.
> But it does clear out amanda’s recollection of the contents of that tape.  
> Worth a try.

I failed to mention in my original post that I had amrmtape'd each of
the 7 tapes in question prior to labeling them.

Kind regards,
Chris

Re: Amanda calling tapes full which are not

[Chris Nighswonger]

On Wed, Oct 24, 2018 at 12:04 PM Jean-Francois Malouin
 wrote:
>
> * Chris Nighswonger  [20181024 10:25]:
> > Any thoughts on what's going on here? I thought running amlabel -f
> > forced deletion of any amanda volumes on that tape?
>
> The '-f' flag forces amlabel to write a label even if something is detected at
> the beginnning of the tape, and does nothing else to the tape.
>

I see that in the man page now. I was reading way too fast. :P

> Are you sure your hardware is healthy?
>

Brand new hardware. It was just replaced by Quantum two months ago.
FWIW, this is the third replacement during our four-year support
contract with them. S I headed over to xTalk and got the
following results which are really no surprise given the track record
of Quantum:

===
xTalk Management Console Wizard: User selections
===
Device entered->/dev/sg3<-
Script entered->SCSI_Interconnect.xcs<-
===

Press 'Enter' when ready to continue or Ctrl-C to cancel:

Phase 1 : Collection...
Loading: ./Scripts//SCSI_Interconnect.xcs
Complete...
Phase 1 Completed.
Phase 2 : Validation...
Phase 2 Complete.
Phase 3 : Linking...
   Symbols Link Complete...
   Uninitialized Variable Scan Complete...
   Vector Linking Complete...
Phase 3 Complete.


SCSI Interconnect Test
Sense: CC: 0B  ASC: 47  ASCQ: 03  = "Informational unit CRC failure."
VS Sense: 0
  Catagory: N/A
  Details: No error
Writing Problem Detected
Problem Detected
===

So it looks like its off to open yet another rapid replacement request.

Of course, this begs the question: Why do the backups run fine until
the first 6 tapes are full and then start borking?

> Also, a blocksize of 32k (default value) might not give you the best
> performance. My experiments over the years suggest to err on much bigger
> values, 512k and more for LTO5 and 6 seem reasonable in my local environment.

Thanks. I have adjusted accordingly. We'll see how it goes. Any
performance improvement is very much welcomed.

Interestingly enough, running amrmtape --erase results in an error
stating the device does not support this feature.

Amanda calling tapes full which are not

[Chris Nighswonger]

Any thoughts on what's going on here? I thought running amlabel -f
forced deletion of any amanda volumes on that tape?

After:

amlabel -f campus campus-NGH864L4 slot 12
Reading label...
Found Amanda volume 'campus-NGH864L4'.
Writing label 'campus-NGH864L4'...
Checking label...
Success!

taper debug log:

Wed Oct 24 09:47:35 2018: thd-0x20a1c00: taper: Slot 12 with label
campus-NGH864L4 is usable
Wed Oct 24 09:47:35 2018: thd-0x20a1c00: taper:
Amanda::Taper::Scan::traditional result: 'campus-NGH864L4' on
tape:/dev/nst0 slot 12, mode 2
Wed Oct 24 09:47:35 2018: thd-0x20a1c00: taper: Amanda::Taper::Scribe
preparing to write, part size 0, using LEOM (falling back to holding
disk as cache) (splitter)  (LEOM supported)
Wed Oct 24 09:47:35 2018: thd-0x20a1c00: taper: Starting
 ->
)>
Wed Oct 24 09:47:35 2018: thd-0x20a1c00: taper: Final linkage:
 -(PULL_BUFFER)->
 -(PUSH_BUFFER)->

Wed Oct 24 09:47:41 2018: thd-0x20a1c00: taper: Building type
TAPESTART header of 32768-32768 bytes with name='campus-NGH864L4'
disk='' dumplevel=0 and blocksize=32768

. . .

Wed Oct 24 09:47:49 2018: thd-0x23ab050: taper: Building type
SPLIT_FILE header of 32768-32768 bytes with name='host.dn.tld'
disk='/' dumplevel=2 and blocksize=32768
Wed Oct 24 09:47:49 2018: thd-0x23ab050: taper: warning: Got EIO on
/dev/nst0, assuming end of tape
Wed Oct 24 09:47:49 2018: thd-0x23ab050: taper: Device tape:/dev/nst0
error = 'No space left on device'
Wed Oct 24 09:47:49 2018: thd-0x23ab050: taper: Device tape:/dev/nst0
setting status flag(s): DEVICE_STATUS_VOLUME_ERROR

Kind regards,
Chris

Re: Zmanda acquired from Carbonite by BETSOL -- future of Amanda development?

[Chris Nighswonger]

Yes.

On Tue, Oct 16, 2018 at 12:06 PM J Chapman Flack 
wrote:

> On 10/16/2018 05:38 AM, Stefan G. Weichinger wrote:
> > Am 15.10.18 um 17:17 schrieb Ashwin Krishna:
> >> Let me go back and check with the team.  This has to be sent out.
> >
> > I am curious already, and I assume other amanda-users as well ... ?
> Yes.
>

Re: Zmanda acquired from Carbonite by BETSOL -- future of Amanda development?

[Chris Nighswonger]

On Wed, Sep 26, 2018 at 1:31 PM Gene Heskett  wrote:

>
> Fork it if someone has a long term interest in seeing a good, long term
> backup solution keep sucking air regularly.
>
> Said by a 20 year user of amanda.
>

+1 to forking.

Chris

Re: Oddity with the disklist

[Chris Nighswonger]

Based on the example disklist file, it looks like when using the multi-line
format, you need a leading slash before your diskname entry:

/tank/jail/webhost091

or some such.

Kind regards,
Chris


On Fri, Jun 1, 2018 at 12:09 PM, Dirk-Willem van Gulik  wrote:

> I’ve got a bit of an oddity with the disklist - suspect it is really
> trivial; but am not seeing it.
>
> This works splendidly and as expected:
>
> ...
> .webweaving.orgtank/jail/webhost
> openssl-client-encrypt-best-zfssnapshot-jail
>
> But this (with whitespace reduced to just space and a \n straight after
> every line, no space in between):
>
> .webweaving.org  tank/jail/webhost091 {
>
> openssl-client-encrypt-best-zfssnapshot-jail
> exclude append "./usr/local/www/logs"
> }
>
> Gives me the error:
>
> ERROR: .webweaving.org: [can not stat tank/jail/webhost091:
> No such file or directory]
>
> And bringing it back to just:
>
> .webweaving.org  tank/jail/webhost091 {
>
> openssl-client-encrypt-best-zfssnapshot-jail
> }
>
> gives me the same error. This is on Amanada 3.3.9, stock FreeBSD install
> against a stock FreeBSD-11.1 remote host with zfs #28.
>
> What am I missing. Must be trivial !
>
> Thanks,
>
> Dw.
>
>
>

Fwd: Zmanda acquired from Carbonite by BETSOL

[Chris Nighswonger]

This probably also explains the lack of attention to various ZWC issues as
well.

Christopher Nighswonger
Faculty Member
Network & Systems Director
Foundations Bible College & Seminary
www.foundations.edu
www.fbcradio.org
-
NOTICE: The information contained in this electronic mail message is
intended only for the use of the intended recipient, and may also be
protected by the Electronic Communications Privacy Act, 18 USC Sections
2510-2521. If the reader of this message is not the intended recipient, you
are hereby notified that any dissemination, distribution or copying of this
communication is strictly prohibited. If you have received this
communication in error, please reply to the sender, and delete the original
message. Thank you.

On Tue, May 8, 2018 at 12:14 PM, Chris Hoogendyk 
wrote:

> My Google alert on Zmanda woke up last night for the first time in a long
> time and sent me this:
>
> BETSOL Completes the Acquisition of *Zmanda* Enterprise Backup
> 
>
> Chicago Evening Post
> “We are excited to add *Zmanda* to the BETSOL product family,” said Ashok
> Reddy, CEO of BETSOL. “With the Rebit acquisition, we are gaining good
> traction in consumer and SMB backup market. Now, with *Zmanda*, our
> enterprise customers have award-winning reliable product lines with
> best-in-class ...
>
>
> I assume we take them at their word that it will be a seamless transition.
> There is supposed to be a new Zmanda website in the works. My sense was
> that it had become stale under Carbonite. For example, the blogs show 2013
> as the latest entry. I've also had some trouble editing the wiki at
> multiple times over the last couple of years.
>
> Who is still with Zmanda in the core team and management? Are they going
> to be adding anyone? Or getting help from BETSOL?
>
> Just wondering if anyone has any inside information they want to share –
> realizing it is still early in the game.
>
> --
> ---
>
> Chris Hoogendyk
>
> -
>O__   Systems Administrator
>   c/ /'_ --- Biology & Geosciences Departments
>  (*) \(*) -- 315 Morrill Science Center
> ~~ - University of Massachusetts, Amherst
>  
>
> ---
>
> Erdös 4
>
>
>

Re: Community ZWC Issue

[Chris Nighswonger]

Well, at this point, given the near zero "community" support for the
"community" edition of ZWC, I'm going to move away from it for my M$
systems and utilize Windows 10's backup utility to push client backups to
my file server and let Amanda pick them up from there. I'm really not
concerned about bare metal capabilities with the M$ clients. Since ZWC and
the OS backup utility both rely on VSS, I'm pretty much getting the same
thing AFAICT.

On Tue, Mar 27, 2018 at 3:43 PM, Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> Yes. Here is a DLE from my dislist:
>
> admin-asst-1.foo.bar "C:/Documents and Settings" {
>   zwc-compress
>   estimate server
>   exclude "C:\\Documents and Settings\\*\\Local Settings\\Temp"
>
> }
>
>
> On Tue, Mar 27, 2018 at 1:20 PM, Paddy Sreenivasan <
> paddy.sreeniva...@gmail.com> wrote:
>
>> Have you configured the DLE the same way i.e, using scriptor.foo.bar as
>> hostname in both cases?
>>
>> On Tue, Mar 27, 2018 at 3:05 AM, Chris Nighswonger <
>> cnighswon...@foundations.edu> wrote:
>>
>>> Has Zamanda abandoned ZWC?
>>>
>>> On Thu, Mar 22, 2018, 9:09 AM Chris Nighswonger <
>>> cnighswon...@foundations.edu> wrote:
>>>
>>>> Ping.
>>>>
>>>>
>>>> On Sun, Mar 18, 2018 at 1:34 PM, Chris Nighswonger <
>>>> cnighswon...@foundations.edu> wrote:
>>>>
>>>>> No takers? To be sure someone who works on the ZWC can comment on what
>>>>> sorts of issues might cause this change in behavior.
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> Chris
>>>>>
>>>>> On Mar 14, 2018 3:24 PM, "Chris Nighswonger" <
>>>>> cnighswon...@foundations.edu> wrote:
>>>>>
>>>>>> Thanks Paddy. I also had to restart the ZWC service to make the log
>>>>>> level change effective.
>>>>>>
>>>>>> So maybe someone who works on the ZWC code can explain what operation
>>>>>> might account for the following differences in log entries. Both are from
>>>>>> the same machine. The backup began failing with the 18/1/2018 run. I 
>>>>>> wonder
>>>>>> if ZWC is doing some sort of name resolution here which suddenly began to
>>>>>> fail? On the client, nslookup results are correct both ways (see below).
>>>>>> This same failure occurs on all of my ZWC installs.
>>>>>>
>>>>>> ZWC working fine:
>>>>>>
>>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Entering
>>>>>> ExecuteValidateServerJob
>>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server address to
>>>>>> be validated from the list of authentic server = 192.168.x.x
>>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server name =
>>>>>> scriptor.foo.bar
>>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Authentic Server
>>>>>> name = scriptor.foo.bar
>>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Leaving
>>>>>> ExecuteValidateServerJob
>>>>>>
>>>>>> ZWC borking:
>>>>>>
>>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Entering
>>>>>> ExecuteValidateServerJob
>>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server address to
>>>>>> be validated from the list of authentic server = 192.168.x.x
>>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server name =
>>>>>> 192.168.x.x
>>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Authentic Server
>>>>>> name = scriptor.foo.bar
>>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server Validation
>>>>>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Leaving
>>>>>> ExecuteValidateServerJob
>>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler:: Exiting
>>>>>> ExecuteJob with status = 65535
>>>>>>
>>>>>> Client-side nslookup:
>>>>>>
>>>>>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>>>>>> Edition(x64)\bin> nslookup 192.168.x.x
>>&g

Re: Community ZWC Issue

[Chris Nighswonger]

Yes. Here is a DLE from my dislist:

admin-asst-1.foo.bar "C:/Documents and Settings" {
  zwc-compress
  estimate server
  exclude "C:\\Documents and Settings\\*\\Local Settings\\Temp"
}


On Tue, Mar 27, 2018 at 1:20 PM, Paddy Sreenivasan <
paddy.sreeniva...@gmail.com> wrote:

> Have you configured the DLE the same way i.e, using scriptor.foo.bar as
> hostname in both cases?
>
> On Tue, Mar 27, 2018 at 3:05 AM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
>
>> Has Zamanda abandoned ZWC?
>>
>> On Thu, Mar 22, 2018, 9:09 AM Chris Nighswonger <
>> cnighswon...@foundations.edu> wrote:
>>
>>> Ping.
>>>
>>>
>>> On Sun, Mar 18, 2018 at 1:34 PM, Chris Nighswonger <
>>> cnighswon...@foundations.edu> wrote:
>>>
>>>> No takers? To be sure someone who works on the ZWC can comment on what
>>>> sorts of issues might cause this change in behavior.
>>>>
>>>> Kind regards,
>>>>
>>>> Chris
>>>>
>>>> On Mar 14, 2018 3:24 PM, "Chris Nighswonger" <
>>>> cnighswon...@foundations.edu> wrote:
>>>>
>>>>> Thanks Paddy. I also had to restart the ZWC service to make the log
>>>>> level change effective.
>>>>>
>>>>> So maybe someone who works on the ZWC code can explain what operation
>>>>> might account for the following differences in log entries. Both are from
>>>>> the same machine. The backup began failing with the 18/1/2018 run. I 
>>>>> wonder
>>>>> if ZWC is doing some sort of name resolution here which suddenly began to
>>>>> fail? On the client, nslookup results are correct both ways (see below).
>>>>> This same failure occurs on all of my ZWC installs.
>>>>>
>>>>> ZWC working fine:
>>>>>
>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Entering
>>>>> ExecuteValidateServerJob
>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server address to
>>>>> be validated from the list of authentic server = 192.168.x.x
>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server name =
>>>>> scriptor.foo.bar
>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Authentic Server
>>>>> name = scriptor.foo.bar
>>>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Leaving
>>>>> ExecuteValidateServerJob
>>>>>
>>>>> ZWC borking:
>>>>>
>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Entering
>>>>> ExecuteValidateServerJob
>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server address to
>>>>> be validated from the list of authentic server = 192.168.x.x
>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server name =
>>>>> 192.168.x.x
>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Authentic Server
>>>>> name = scriptor.foo.bar
>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server Validation
>>>>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Leaving
>>>>> ExecuteValidateServerJob
>>>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler:: Exiting ExecuteJob
>>>>> with status = 65535
>>>>>
>>>>> Client-side nslookup:
>>>>>
>>>>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>>>>> Edition(x64)\bin> nslookup 192.168.x.x
>>>>> Server:  UnKnown
>>>>> Address:  192.168.y.y
>>>>>
>>>>> Name:scriptor.foo.bar
>>>>> Address:  192.168.x.x
>>>>>
>>>>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>>>>> Edition(x64)\bin> nslookup scriptor.foo.bar
>>>>> Server:  UnKnown
>>>>> Address:  192.168.x.x
>>>>>
>>>>> Name:scriptor.foo.bar
>>>>> Address:  192.168.x.x
>>>>>
>>>>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>>>>> Edition(x64)\bin>
>>>>>
>>>>> Kind regards,
>>>>> Chris
>>>>>
>>>>>
>>>>> On Fri, Mar 9, 2018 at 10:46 AM, Paddy Sreenivasan <
>>>>> paddy

Re: Community ZWC Issue

[Chris Nighswonger]

Has Zamanda abandoned ZWC?

On Thu, Mar 22, 2018, 9:09 AM Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> Ping.
>
>
> On Sun, Mar 18, 2018 at 1:34 PM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
>
>> No takers? To be sure someone who works on the ZWC can comment on what
>> sorts of issues might cause this change in behavior.
>>
>> Kind regards,
>>
>> Chris
>>
>> On Mar 14, 2018 3:24 PM, "Chris Nighswonger" <
>> cnighswon...@foundations.edu> wrote:
>>
>>> Thanks Paddy. I also had to restart the ZWC service to make the log
>>> level change effective.
>>>
>>> So maybe someone who works on the ZWC code can explain what operation
>>> might account for the following differences in log entries. Both are from
>>> the same machine. The backup began failing with the 18/1/2018 run. I wonder
>>> if ZWC is doing some sort of name resolution here which suddenly began to
>>> fail? On the client, nslookup results are correct both ways (see below).
>>> This same failure occurs on all of my ZWC installs.
>>>
>>> ZWC working fine:
>>>
>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Entering
>>> ExecuteValidateServerJob
>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server address to be
>>> validated from the list of authentic server = 192.168.x.x
>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server name =
>>> scriptor.foo.bar
>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Authentic Server name
>>> = scriptor.foo.bar
>>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Leaving
>>> ExecuteValidateServerJob
>>>
>>> ZWC borking:
>>>
>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Entering
>>> ExecuteValidateServerJob
>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server address to be
>>> validated from the list of authentic server = 192.168.x.x
>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server name =
>>> 192.168.x.x
>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Authentic Server name
>>> = scriptor.foo.bar
>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server Validation
>>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Leaving
>>> ExecuteValidateServerJob
>>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler:: Exiting ExecuteJob
>>> with status = 65535
>>>
>>> Client-side nslookup:
>>>
>>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>>> Edition(x64)\bin> nslookup 192.168.x.x
>>> Server:  UnKnown
>>> Address:  192.168.y.y
>>>
>>> Name:scriptor.foo.bar
>>> Address:  192.168.x.x
>>>
>>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>>> Edition(x64)\bin> nslookup scriptor.foo.bar
>>> Server:  UnKnown
>>> Address:  192.168.x.x
>>>
>>> Name:scriptor.foo.bar
>>> Address:  192.168.x.x
>>>
>>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>>> Edition(x64)\bin>
>>>
>>> Kind regards,
>>> Chris
>>>
>>>
>>> On Fri, Mar 9, 2018 at 10:46 AM, Paddy Sreenivasan <
>>> paddy.sreeniva...@gmail.com> wrote:
>>>
>>>>
>>>>1. Open the ZWC Config Utility: Start Menu → All Programs → Zmanda →
>>>>     Zmanda Client for Windows → ZWC Config Utility
>>>>2. Select the Logging tab
>>>>3. Set Log Level to 5
>>>>4. Click Save
>>>>5. Click Exit
>>>>
>>>>
>>>>
>>>> On Fri, Mar 9, 2018 at 5:59 AM, Chris Nighswonger <
>>>> cnighswon...@foundations.edu> wrote:
>>>>
>>>>> Jean-Louis can you help with this? Is there a way to up the verbosity
>>>>> level of the ZWC log?
>>>>>
>>>>>
>>>>> On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
>>>>> cnighswon...@foundations.edu> wrote:
>>>>>
>>>>>> No takers?
>>>>>>
>>>>>> The ZWC is always a hard one to gin up help for.
>>>>>>
>>>>>> An additional piece of information: DNS resolution seems to be
>>>>>> working fine both server and client side.
>>>>>>
>&g

Re: Community ZWC Issue

[Chris Nighswonger]

Ping.


On Sun, Mar 18, 2018 at 1:34 PM, Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> No takers? To be sure someone who works on the ZWC can comment on what
> sorts of issues might cause this change in behavior.
>
> Kind regards,
>
> Chris
>
> On Mar 14, 2018 3:24 PM, "Chris Nighswonger" <cnighswon...@foundations.edu>
> wrote:
>
>> Thanks Paddy. I also had to restart the ZWC service to make the log level
>> change effective.
>>
>> So maybe someone who works on the ZWC code can explain what operation
>> might account for the following differences in log entries. Both are from
>> the same machine. The backup began failing with the 18/1/2018 run. I wonder
>> if ZWC is doing some sort of name resolution here which suddenly began to
>> fail? On the client, nslookup results are correct both ways (see below).
>> This same failure occurs on all of my ZWC installs.
>>
>> ZWC working fine:
>>
>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Entering
>> ExecuteValidateServerJob
>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server address to be
>> validated from the list of authentic server = 192.168.x.x
>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server name =
>> scriptor.foo.bar
>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Authentic Server name
>> = scriptor.foo.bar
>> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Leaving
>> ExecuteValidateServerJob
>>
>> ZWC borking:
>>
>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Entering
>> ExecuteValidateServerJob
>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server address to be
>> validated from the list of authentic server = 192.168.x.x
>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server name =
>> 192.168.x.x
>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Authentic Server name
>> = scriptor.foo.bar
>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server Validation
>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Leaving
>> ExecuteValidateServerJob
>> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler:: Exiting ExecuteJob
>> with status = 65535
>>
>> Client-side nslookup:
>>
>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>> Edition(x64)\bin> nslookup 192.168.x.x
>> Server:  UnKnown
>> Address:  192.168.y.y
>>
>> Name:scriptor.foo.bar
>> Address:  192.168.x.x
>>
>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>> Edition(x64)\bin> nslookup scriptor.foo.bar
>> Server:  UnKnown
>> Address:  192.168.x.x
>>
>> Name:scriptor.foo.bar
>> Address:  192.168.x.x
>>
>> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
>> Edition(x64)\bin>
>>
>> Kind regards,
>> Chris
>>
>>
>> On Fri, Mar 9, 2018 at 10:46 AM, Paddy Sreenivasan <
>> paddy.sreeniva...@gmail.com> wrote:
>>
>>>
>>>1. Open the ZWC Config Utility: Start Menu → All Programs → Zmanda → 
>>> Zmanda
>>>Client for Windows → ZWC Config Utility
>>>2. Select the Logging tab
>>>3. Set Log Level to 5
>>>4. Click Save
>>>5. Click Exit
>>>
>>>
>>>
>>> On Fri, Mar 9, 2018 at 5:59 AM, Chris Nighswonger <
>>> cnighswon...@foundations.edu> wrote:
>>>
>>>> Jean-Louis can you help with this? Is there a way to up the verbosity
>>>> level of the ZWC log?
>>>>
>>>>
>>>> On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
>>>> cnighswon...@foundations.edu> wrote:
>>>>
>>>>> No takers?
>>>>>
>>>>> The ZWC is always a hard one to gin up help for.
>>>>>
>>>>> An additional piece of information: DNS resolution seems to be working
>>>>> fine both server and client side.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
>>>>> cnighswon...@foundations.edu> wrote:
>>>>>
>>>>>> Any thoughts on what might be going on here?
>>>>>>
>>>>>> Last week I began to have numerous Win10 clients failing in this
>>>>>> fashion:
>>>>>>
>>>>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>>&

Re: Community ZWC Issue

[Chris Nighswonger]

No takers? To be sure someone who works on the ZWC can comment on what
sorts of issues might cause this change in behavior.

Kind regards,

Chris

On Mar 14, 2018 3:24 PM, "Chris Nighswonger" <cnighswon...@foundations.edu>
wrote:

> Thanks Paddy. I also had to restart the ZWC service to make the log level
> change effective.
>
> So maybe someone who works on the ZWC code can explain what operation
> might account for the following differences in log entries. Both are from
> the same machine. The backup began failing with the 18/1/2018 run. I wonder
> if ZWC is doing some sort of name resolution here which suddenly began to
> fail? On the client, nslookup results are correct both ways (see below).
> This same failure occurs on all of my ZWC installs.
>
> ZWC working fine:
>
> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Entering
> ExecuteValidateServerJob
> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server address to be
> validated from the list of authentic server = 192.168.x.x
> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server name =
> scriptor.foo.bar
> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Authentic Server name =
> scriptor.foo.bar
> 4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Leaving
> ExecuteValidateServerJob
>
> ZWC borking:
>
> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Entering
> ExecuteValidateServerJob
> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server address to be
> validated from the list of authentic server = 192.168.x.x
> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server name =
> 192.168.x.x
> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Authentic Server name
> = scriptor.foo.bar
> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server Validation
> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Leaving
> ExecuteValidateServerJob
> 5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler:: Exiting ExecuteJob
> with status = 65535
>
> Client-side nslookup:
>
> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
> Edition(x64)\bin> nslookup 192.168.x.x
> Server:  UnKnown
> Address:  192.168.y.y
>
> Name:scriptor.foo.bar
> Address:  192.168.x.x
>
> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
> Edition(x64)\bin> nslookup scriptor.foo.bar
> Server:  UnKnown
> Address:  192.168.x.x
>
> Name:scriptor.foo.bar
> Address:  192.168.x.x
>
> PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
> Edition(x64)\bin>
>
> Kind regards,
> Chris
>
>
> On Fri, Mar 9, 2018 at 10:46 AM, Paddy Sreenivasan <
> paddy.sreeniva...@gmail.com> wrote:
>
>>
>>1. Open the ZWC Config Utility: Start Menu → All Programs → Zmanda → 
>> Zmanda
>>Client for Windows → ZWC Config Utility
>>2. Select the Logging tab
>>3. Set Log Level to 5
>>4. Click Save
>>5. Click Exit
>>
>>
>>
>> On Fri, Mar 9, 2018 at 5:59 AM, Chris Nighswonger <
>> cnighswon...@foundations.edu> wrote:
>>
>>> Jean-Louis can you help with this? Is there a way to up the verbosity
>>> level of the ZWC log?
>>>
>>>
>>> On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
>>> cnighswon...@foundations.edu> wrote:
>>>
>>>> No takers?
>>>>
>>>> The ZWC is always a hard one to gin up help for.
>>>>
>>>> An additional piece of information: DNS resolution seems to be working
>>>> fine both server and client side.
>>>>
>>>>
>>>>
>>>> On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
>>>> cnighswon...@foundations.edu> wrote:
>>>>
>>>>> Any thoughts on what might be going on here?
>>>>>
>>>>> Last week I began to have numerous Win10 clients failing in this
>>>>> fashion:
>>>>>
>>>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>>>
>>>>> Amanda Backup Client Hosts Check
>>>>> 
>>>>> ERROR: shipping.foo.bar: Server validation Failed. Please register
>>>>> server with client.
>>>>> Client check: 1 host checked in 0.072 seconds.  1 problem found.
>>>>>
>>>>> Taking a look at the ZWC debug log on that client, I see the following:
>>>>>
>>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
>>>>> ExecuteValidateServerJo

Re: Community ZWC Issue

[Chris Nighswonger]

Thanks Paddy. I also had to restart the ZWC service to make the log level
change effective.

So maybe someone who works on the ZWC code can explain what operation might
account for the following differences in log entries. Both are from the
same machine. The backup began failing with the 18/1/2018 run. I wonder if
ZWC is doing some sort of name resolution here which suddenly began to
fail? On the client, nslookup results are correct both ways (see below).
This same failure occurs on all of my ZWC installs.

ZWC working fine:

4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Entering
ExecuteValidateServerJob
4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server address to be
validated from the list of authentic server = 192.168.x.x
4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Server name =
scriptor.foo.bar
4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Authentic Server name =
scriptor.foo.bar
4424:384:17/1/2018:12:59:36:548::CZWCJobHandler : Leaving
ExecuteValidateServerJob

ZWC borking:

5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Entering
ExecuteValidateServerJob
5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server address to be
validated from the list of authentic server = 192.168.x.x
5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server name = 192.168.x.x
5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Authentic Server name =
scriptor.foo.bar
5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Server Validation Failed
- Server 192.168.x.x With IP 192.168.x.x is not registered
5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler : Leaving
ExecuteValidateServerJob
5692:12004:14/3/2018:14:54:9:706::CZWCJobHandler:: Exiting ExecuteJob with
status = 65535

Client-side nslookup:

PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
Edition(x64)\bin> nslookup 192.168.x.x
Server:  UnKnown
Address:  192.168.y.y

Name:scriptor.foo.bar
Address:  192.168.x.x

PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
Edition(x64)\bin> nslookup scriptor.foo.bar
Server:  UnKnown
Address:  192.168.x.x

Name:scriptor.foo.bar
Address:  192.168.x.x

PS C:\Program Files\Zmanda\Zmanda Client for Windows Community
Edition(x64)\bin>

Kind regards,
Chris


On Fri, Mar 9, 2018 at 10:46 AM, Paddy Sreenivasan <
paddy.sreeniva...@gmail.com> wrote:

>
>1. Open the ZWC Config Utility: Start Menu → All Programs → Zmanda → Zmanda
>Client for Windows → ZWC Config Utility
>2. Select the Logging tab
>3. Set Log Level to 5
>4. Click Save
>5. Click Exit
>
>
>
> On Fri, Mar 9, 2018 at 5:59 AM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
>
>> Jean-Louis can you help with this? Is there a way to up the verbosity
>> level of the ZWC log?
>>
>>
>> On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
>> cnighswon...@foundations.edu> wrote:
>>
>>> No takers?
>>>
>>> The ZWC is always a hard one to gin up help for.
>>>
>>> An additional piece of information: DNS resolution seems to be working
>>> fine both server and client side.
>>>
>>>
>>>
>>> On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
>>> cnighswon...@foundations.edu> wrote:
>>>
>>>> Any thoughts on what might be going on here?
>>>>
>>>> Last week I began to have numerous Win10 clients failing in this
>>>> fashion:
>>>>
>>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>>
>>>> Amanda Backup Client Hosts Check
>>>> 
>>>> ERROR: shipping.foo.bar: Server validation Failed. Please register
>>>> server with client.
>>>> Client check: 1 host checked in 0.072 seconds.  1 problem found.
>>>>
>>>> Taking a look at the ZWC debug log on that client, I see the following:
>>>>
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
>>>> ExecuteValidateServerJob
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server address to be
>>>> validated from the list of authentic server = 192.168.x.x
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server name =
>>>> 192.168.x.x
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Authentic Server
>>>> name = scriptor.foo.bar
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server Validation
>>>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Leaving
>>>> ExecuteValidateServerJob
>>>>
>>>> However, the server is registered with the client:
>&g

Re: Community ZWC Issue

[Chris Nighswonger]

amservice must be run as root

root@scriptor:/home/manager# amservice shipping.foo.bar
<http://shipping.campus.foundations.edu> bsdtcp noop  wrote:

> More (apparently conflicting) information. A noop works in amcheck, but
> not directly from amservice.
>
> Packet Dump server-side:
>
> ...Q.SECURITY USER backup
>
> SERVICE noop
>
> OPTIONS features=9efefbff3f;
>
> ..,.OPTIONS features=ff7f9cfed3cf1300;
>
> ...SECURITY USER backup
>
> SERVICE selfcheck
>
> OPTIONS features=9efefbff3f;maxdumps=1;hostname=
> shipping.foo.bar;config=campus;
>
> 
>
> DUMP
>
> SERVER
>
> C:/Program_
> Files_(x86)/Stamps.com_Internet_Postage
>
> bsdtcp
>
> FAST
>
> YES
>
> YES
>
> 
>
> 
>
> DUMP
>
> SERVER
>
> C:/Users/auser
>
> bsdtcp
>
> FAST
>
> YES
>
> YES
>
> 
>
> C:_Users_auser_AppData_Local_Temp
>
> 
>
> 
>
> ..E.ERROR Server validation Failed. Please register
> server with client.
>
> ......
>
> The results of an amservice noop server-side:
>
> backup@scriptor:~ amservice shipping.foo.bar bsdtcp noop  Request failed: Permission denied
>
>
> On Fri, Mar 9, 2018 at 9:20 AM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
>
>> One additional note: The wiki seems to think that name resolution has not
>> been working since sometime in 2010. However, it has worked fine for me for
>> years up until about a month ago.
>>
>>
>>
>> On Fri, Mar 9, 2018 at 8:59 AM, Chris Nighswonger <
>> cnighswon...@foundations.edu> wrote:
>>
>>> Jean-Louis can you help with this? Is there a way to up the verbosity
>>> level of the ZWC log?
>>>
>>>
>>>
>>> On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
>>> cnighswon...@foundations.edu> wrote:
>>>
>>>> No takers?
>>>>
>>>> The ZWC is always a hard one to gin up help for.
>>>>
>>>> An additional piece of information: DNS resolution seems to be working
>>>> fine both server and client side.
>>>>
>>>>
>>>>
>>>> On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
>>>> cnighswon...@foundations.edu> wrote:
>>>>
>>>>> Any thoughts on what might be going on here?
>>>>>
>>>>> Last week I began to have numerous Win10 clients failing in this
>>>>> fashion:
>>>>>
>>>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>>>
>>>>> Amanda Backup Client Hosts Check
>>>>> 
>>>>> ERROR: shipping.foo.bar: Server validation Failed. Please register
>>>>> server with client.
>>>>> Client check: 1 host checked in 0.072 seconds.  1 problem found.
>>>>>
>>>>> Taking a look at the ZWC debug log on that client, I see the following:
>>>>>
>>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
>>>>> ExecuteValidateServerJob
>>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server address to
>>>>> be validated from the list of authentic server = 192.168.x.x
>>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server name =
>>>>> 192.168.x.x
>>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Authentic Server
>>>>> name = scriptor.foo.bar
>>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server Validation
>>>>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Leaving
>>>>> ExecuteValidateServerJob
>>>>>
>>>>> However, the server is registered with the client:
>>>>>
>>>>> Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Zmanda\ZWC\1.0\Install\Servers
>>>>> value is 'scriptor.foo.bar'
>>>>>
>>>>> If I add the server IP (192.168.x.x) to the list of registered
>>>>> Servers, everything checks out fine:
>>>>>
>>>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>>>
>>>>> Amanda Backup Client Hosts Check
>>>>> 
>>>>> Client check: 1 host checked in 0.071 seconds.  0 problems found.
>>>>>
>>>>> (brought to you by Amanda 3.3.6)
>>>>>
>>>>> And the ZWC debug log says:
>>>>>
>>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Entering
>>>>> ExecuteValidateServerJob
>>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server address to
>>>>> be validated from the list of authentic server = 192.168.2.203
>>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server name =
>>>>> 192.168.x.x
>>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server
>>>>> name = scriptor.foo.bar
>>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server
>>>>> name = 192.168.x.x
>>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Leaving
>>>>> ExecuteValidateServerJob
>>>>>
>>>>> Thanks,
>>>>> Chris
>>>>>
>>>>
>>>>
>>>
>>
>

Re: Community ZWC Issue

[Chris Nighswonger]

More (apparently conflicting) information. A noop works in amcheck, but not
directly from amservice.

Packet Dump server-side:

...Q.SECURITY USER backup

SERVICE noop

OPTIONS features=9efefbff3f;

..,.OPTIONS features=ff7f9cfed3cf1300;

...SECURITY USER backup

SERVICE selfcheck

OPTIONS
features=9efefbff3f;maxdumps=1;hostname=shipping.foo.bar;config=campus;



DUMP

SERVER

C:/Program_Files_(x86)/Stamps.com_Internet_Postage

bsdtcp

FAST

YES

YES





DUMP

SERVER

C:/Users/auser

bsdtcp

FAST

YES

YES



C:_Users_auser_AppData_Local_Temp





..E.ERROR Server validation Failed. Please register server
with client.

..

The results of an amservice noop server-side:

backup@scriptor:~ amservice shipping.foo.bar bsdtcp noop  wrote:

> One additional note: The wiki seems to think that name resolution has not
> been working since sometime in 2010. However, it has worked fine for me for
> years up until about a month ago.
>
>
>
> On Fri, Mar 9, 2018 at 8:59 AM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
>
>> Jean-Louis can you help with this? Is there a way to up the verbosity
>> level of the ZWC log?
>>
>>
>>
>> On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
>> cnighswon...@foundations.edu> wrote:
>>
>>> No takers?
>>>
>>> The ZWC is always a hard one to gin up help for.
>>>
>>> An additional piece of information: DNS resolution seems to be working
>>> fine both server and client side.
>>>
>>>
>>>
>>> On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
>>> cnighswon...@foundations.edu> wrote:
>>>
>>>> Any thoughts on what might be going on here?
>>>>
>>>> Last week I began to have numerous Win10 clients failing in this
>>>> fashion:
>>>>
>>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>>
>>>> Amanda Backup Client Hosts Check
>>>> 
>>>> ERROR: shipping.foo.bar: Server validation Failed. Please register
>>>> server with client.
>>>> Client check: 1 host checked in 0.072 seconds.  1 problem found.
>>>>
>>>> Taking a look at the ZWC debug log on that client, I see the following:
>>>>
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
>>>> ExecuteValidateServerJob
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server address to be
>>>> validated from the list of authentic server = 192.168.x.x
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server name =
>>>> 192.168.x.x
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Authentic Server
>>>> name = scriptor.foo.bar
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server Validation
>>>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Leaving
>>>> ExecuteValidateServerJob
>>>>
>>>> However, the server is registered with the client:
>>>>
>>>> Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Zmanda\ZWC\1.0\Install\Servers
>>>> value is 'scriptor.foo.bar'
>>>>
>>>> If I add the server IP (192.168.x.x) to the list of registered Servers,
>>>> everything checks out fine:
>>>>
>>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>>
>>>> Amanda Backup Client Hosts Check
>>>> 
>>>> Client check: 1 host checked in 0.071 seconds.  0 problems found.
>>>>
>>>> (brought to you by Amanda 3.3.6)
>>>>
>>>> And the ZWC debug log says:
>>>>
>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Entering
>>>> ExecuteValidateServerJob
>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server address to
>>>> be validated from the list of authentic server = 192.168.2.203
>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server name =
>>>> 192.168.x.x
>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server
>>>> name = scriptor.foo.bar
>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server
>>>> name = 192.168.x.x
>>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Leaving
>>>> ExecuteValidateServerJob
>>>>
>>>> Thanks,
>>>> Chris
>>>>
>>>
>>>
>>
>

Re: Community ZWC Issue

[Chris Nighswonger]

One additional note: The wiki seems to think that name resolution has not
been working since sometime in 2010. However, it has worked fine for me for
years up until about a month ago.


On Fri, Mar 9, 2018 at 8:59 AM, Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> Jean-Louis can you help with this? Is there a way to up the verbosity
> level of the ZWC log?
>
>
>
> On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
>
>> No takers?
>>
>> The ZWC is always a hard one to gin up help for.
>>
>> An additional piece of information: DNS resolution seems to be working
>> fine both server and client side.
>>
>>
>>
>> On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
>> cnighswon...@foundations.edu> wrote:
>>
>>> Any thoughts on what might be going on here?
>>>
>>> Last week I began to have numerous Win10 clients failing in this fashion:
>>>
>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>
>>> Amanda Backup Client Hosts Check
>>> 
>>> ERROR: shipping.foo.bar: Server validation Failed. Please register
>>> server with client.
>>> Client check: 1 host checked in 0.072 seconds.  1 problem found.
>>>
>>> Taking a look at the ZWC debug log on that client, I see the following:
>>>
>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
>>> ExecuteValidateServerJob
>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server address to be
>>> validated from the list of authentic server = 192.168.x.x
>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server name =
>>> 192.168.x.x
>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Authentic Server name
>>> = scriptor.foo.bar
>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server Validation
>>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Leaving
>>> ExecuteValidateServerJob
>>>
>>> However, the server is registered with the client:
>>>
>>> Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Zmanda\ZWC\1.0\Install\Servers
>>> value is 'scriptor.foo.bar'
>>>
>>> If I add the server IP (192.168.x.x) to the list of registered Servers,
>>> everything checks out fine:
>>>
>>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>>
>>> Amanda Backup Client Hosts Check
>>> 
>>> Client check: 1 host checked in 0.071 seconds.  0 problems found.
>>>
>>> (brought to you by Amanda 3.3.6)
>>>
>>> And the ZWC debug log says:
>>>
>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Entering
>>> ExecuteValidateServerJob
>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server address to be
>>> validated from the list of authentic server = 192.168.2.203
>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server name =
>>> 192.168.x.x
>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server
>>> name = scriptor.foo.bar
>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server
>>> name = 192.168.x.x
>>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Leaving
>>> ExecuteValidateServerJob
>>>
>>> Thanks,
>>> Chris
>>>
>>
>>
>

Re: Community ZWC Issue

[Chris Nighswonger]

Jean-Louis can you help with this? Is there a way to up the verbosity level
of the ZWC log?


On Tue, Feb 27, 2018 at 10:43 AM, Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> No takers?
>
> The ZWC is always a hard one to gin up help for.
>
> An additional piece of information: DNS resolution seems to be working
> fine both server and client side.
>
>
>
> On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
> cnighswon...@foundations.edu> wrote:
>
>> Any thoughts on what might be going on here?
>>
>> Last week I began to have numerous Win10 clients failing in this fashion:
>>
>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>
>> Amanda Backup Client Hosts Check
>> 
>> ERROR: shipping.foo.bar: Server validation Failed. Please register server
>> with client.
>> Client check: 1 host checked in 0.072 seconds.  1 problem found.
>>
>> Taking a look at the ZWC debug log on that client, I see the following:
>>
>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
>> ExecuteValidateServerJob
>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server address to be
>> validated from the list of authentic server = 192.168.x.x
>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server name =
>> 192.168.x.x
>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Authentic Server name
>> = scriptor.foo.bar
>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server Validation
>> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
>> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Leaving
>> ExecuteValidateServerJob
>>
>> However, the server is registered with the client:
>>
>> Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Zmanda\ZWC\1.0\Install\Servers
>> value is 'scriptor.foo.bar'
>>
>> If I add the server IP (192.168.x.x) to the list of registered Servers,
>> everything checks out fine:
>>
>> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>>
>> Amanda Backup Client Hosts Check
>> 
>> Client check: 1 host checked in 0.071 seconds.  0 problems found.
>>
>> (brought to you by Amanda 3.3.6)
>>
>> And the ZWC debug log says:
>>
>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Entering
>> ExecuteValidateServerJob
>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server address to be
>> validated from the list of authentic server = 192.168.2.203
>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server name =
>> 192.168.x.x
>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server name
>> = scriptor.foo.bar
>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server name
>> = 192.168.x.x
>> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Leaving
>> ExecuteValidateServerJob
>>
>> Thanks,
>> Chris
>>
>
>

Re: Community ZWC Issue

[Chris Nighswonger]

No takers?

The ZWC is always a hard one to gin up help for.

An additional piece of information: DNS resolution seems to be working fine
both server and client side.


On Fri, Feb 23, 2018 at 8:34 AM, Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> Any thoughts on what might be going on here?
>
> Last week I began to have numerous Win10 clients failing in this fashion:
>
> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>
> Amanda Backup Client Hosts Check
> 
> ERROR: shipping.foo.bar: Server validation Failed. Please register server
> with client.
> Client check: 1 host checked in 0.072 seconds.  1 problem found.
>
> Taking a look at the ZWC debug log on that client, I see the following:
>
> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
> ExecuteValidateServerJob
> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server address to be
> validated from the list of authentic server = 192.168.x.x
> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server name =
> 192.168.x.x
> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Authentic Server name =
> scriptor.foo.bar
> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server Validation
> Failed - Server 192.168.x.x With IP 192.168.x.x is not registered
> 11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Leaving
> ExecuteValidateServerJob
>
> However, the server is registered with the client:
>
> Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Zmanda\ZWC\1.0\Install\Servers value
> is 'scriptor.foo.bar'
>
> If I add the server IP (192.168.x.x) to the list of registered Servers,
> everything checks out fine:
>
> backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar
>
> Amanda Backup Client Hosts Check
> 
> Client check: 1 host checked in 0.071 seconds.  0 problems found.
>
> (brought to you by Amanda 3.3.6)
>
> And the ZWC debug log says:
>
> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Entering
> ExecuteValidateServerJob
> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server address to be
> validated from the list of authentic server = 192.168.2.203
> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server name =
> 192.168.x.x
> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server name
> = scriptor.foo.bar
> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server name
> = 192.168.x.x
> 11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Leaving
> ExecuteValidateServerJob
>
> Thanks,
> Chris
>

Community ZWC Issue

[Chris Nighswonger]

Any thoughts on what might be going on here?

Last week I began to have numerous Win10 clients failing in this fashion:

backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar

Amanda Backup Client Hosts Check

ERROR: shipping.foo.bar: Server validation Failed. Please register server
with client.
Client check: 1 host checked in 0.072 seconds.  1 problem found.

Taking a look at the ZWC debug log on that client, I see the following:

11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Entering
ExecuteValidateServerJob
11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server address to be
validated from the list of authentic server = 192.168.x.x
11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server name = 192.168.x.x
11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Authentic Server name =
scriptor.foo.bar
11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Server Validation Failed
- Server 192.168.x.x With IP 192.168.x.x is not registered
11020:9144:23/2/2018:08:16:7:22::CZWCJobHandler : Leaving
ExecuteValidateServerJob

However, the server is registered with the client:

Computer\HKEY_LOCAL_MACHINE\SOFTWARE\Zmanda\ZWC\1.0\Install\Servers value
is 'scriptor.foo.bar'

If I add the server IP (192.168.x.x) to the list of registered Servers,
everything checks out fine:

backup@scriptor:/home/manager amcheck -c campus shipping.foo.bar

Amanda Backup Client Hosts Check

Client check: 1 host checked in 0.071 seconds.  0 problems found.

(brought to you by Amanda 3.3.6)

And the ZWC debug log says:

11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Entering
ExecuteValidateServerJob
11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server address to be
validated from the list of authentic server = 192.168.2.203
11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Server name = 192.168.x.x
11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server name =
scriptor.foo.bar
11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Authentic Server name =
192.168.x.x
11020:8956:23/2/2018:08:21:17:29::CZWCJobHandler : Leaving
ExecuteValidateServerJob

Thanks,
Chris

Re: amcheck failing when checking multiple clients

[Chris Nighswonger]

On Sat, Jul 30, 2016 at 2:05 AM, Uwe Menges <uwe.men...@web.de> wrote:

> On 07/29/16 16:59, Chris Nighswonger wrote:
> > Yes, but they are compiled with user 'amandabackup' whereas packages in
> > the distro repo are compiled with 'backup.' The site here is large
> > enough that the pain of switching the backup username would be quite
> large.
>
> To get around this issue, you could duplicate the existing user with the
> other name:
>
> # grep back /etc/passwd
> amandabackup:x:33:6:Amanda user:/var/lib/amanda:/bin/bash
> backup:x:33:6:Amanda user:/var/lib/amanda:/bin/bash
>
>
If most of the clients were *nix, that would be great; however, the
majority are WinXX clients which require much more work to change the
account.

Kind regards,
Chris

Re: amcheck failing when checking multiple clients

[Chris Nighswonger]

Yes, but they are compiled with user 'amandabackup' whereas packages in the
distro repo are compiled with 'backup.' The site here is large enough that
the pain of switching the backup username would be quite large.

What would be nice would be for whomever builds the packages published by
Zmanda to coordinate with those maintaining the distro repos...

Still, John Louis clearly states that this bug should be fixed in 3.3.6.



On Fri, Jul 29, 2016 at 10:43 AM, Ochressandro Rettinger <
orettin...@salud.unm.edu> wrote:

> Oh, there are Ubuntu .deb packages of 3.3.9 on the
> downloads page on the website.  http://www.zmanda.com/download-amanda.php
>
>
>
> It looks like they might only go through Ubuntu 14,
> though.  I dunno what the difference between system version packages would
> be.
>
>
>
>     -Sandro
>
>
>
> *From:* Chris Nighswonger [mailto:cnighswon...@foundations.edu]
> *Sent:* Friday, July 29, 2016 8:31 AM
> *To:* Ochressandro Rettinger <orettin...@salud.unm.edu>
> *Cc:* amanda-users@amanda.org; Jean-Louis Martineau <
> jmartin...@carbonite.com>
>
> *Subject:* Re: amcheck failing when checking multiple clients
>
>
>
> Good question.
>
> The amanda server on this site is running Ubuntu and so 3.3.6 is the
> latest available amanda in the distro repos. Since most other *nix boxes
> are running Ubuntu, doing my own build is a bit of a chore. Not to mention
> I am quite unfamiliar with cooking up deb packages. So the curve is a bit
> steep which is what keeps me from going to the latest amanda version.
>
> That said, I did upgrade the amanda server to Ubuntu 16 in order to move
> from amanda 3.3.3 to 3.3.6 thinking that would correct this problem based
> on the post I mentioned previously.
>
>
>
> On Fri, Jul 29, 2016 at 10:21 AM, Ochressandro Rettinger <
> orettin...@salud.unm.edu> wrote:
>
> Asking as a newb who genuinely doesn't know: How hard is it to
> upgrade Amanda?
>
> If this is a 3.3.6 era bug, maybe upgrading all of your nodes to
> 3.3.9 would solve your problem?
>
> -Sandro
>
> -Original Message-
> From: owner-amanda-us...@amanda.org [mailto:owner-amanda-us...@amanda.org]
> On Behalf Of Chris Nighswonger
>
> Sent: Thursday, July 28, 2016 12:08 PM
> To: amanda-users@amanda.org
> Cc: Jean-Louis Martineau <jmartin...@carbonite.com>
> Subject: Re: amcheck failing when checking multiple clients
>
> Maybe John-Louis can jump in here since these are symptoms of a bug which
> was supposed to have been fixed in 3.3.6
>
>
> http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/amanda-20/amdumps-failing-and-some-amchecks-as-well-124340/
>
> Summary of where things stand on this install:
>
> Server version 3.3.6
> Client versions 3.3.3, 3.3.6
>
> auth = bsdtcp
>
> The issue:
>
> 1. amcheck fails on a subset of clients when run against multiple clients
> (regardless of client version) with "selfcheck request failed:
> error sending REQ: write error to: Broken pipe"
> 2. amcheck does not fail on all clients, but fails on the same subset of
> clients each time.
> 3. amcheck does not fail when run against any single client, including
> those in the subset which does fail when run against multiple clients.
> 4. amservice fails when run against any member of that subset of clients
> which fails amcheck with "Request failed: Permission denied"
> 5. dumps run fine during the nightly backups for all clients, including
> those members of that amcheck failing subset.
>
> Misc. notes:
>
> 1. When amcheck fails on a client, the client contains no debug log.
> 2. The server amcheck debug log reflects the same issue mentioned above
> "Broken pipe"
> 3. From the prospective of a failing client: tcpdump shows an exchange of
> about 40 packets with the client when amcheck is run against a single
> client and succeeds 4. From the same prospective, tcpdump shows an exchange
> of about 68 packets with the client when amcheck is run against multiple
> clients and fails 5. All file perms have been checked and are correct on
> both server and client 6. backup user settings are correct on server and
> client 7. amandahosts is properly configured 8. xinetd configuration is
> correct on both server and client 9. forward and reverse DNS works
> correctly for the client
>
> I wonder if this might be a variation of the bug mentioned by JLM in the
> above referenced thread?
>
> Thanks for everyone's help here.
>
>
> On Thu, Jul 28, 2016 at 10:15 AM, Ochressandro Rettinger <
> orettin...@salud.unm.edu> wrote:
> > In the seco

Re: amcheck failing when checking multiple clients

[Chris Nighswonger]

Good question.

The amanda server on this site is running Ubuntu and so 3.3.6 is the latest
available amanda in the distro repos. Since most other *nix boxes are
running Ubuntu, doing my own build is a bit of a chore. Not to mention I am
quite unfamiliar with cooking up deb packages. So the curve is a bit steep
which is what keeps me from going to the latest amanda version.

That said, I did upgrade the amanda server to Ubuntu 16 in order to move
from amanda 3.3.3 to 3.3.6 thinking that would correct this problem based
on the post I mentioned previously.


On Fri, Jul 29, 2016 at 10:21 AM, Ochressandro Rettinger <
orettin...@salud.unm.edu> wrote:

> Asking as a newb who genuinely doesn't know: How hard is it to
> upgrade Amanda?
>
> If this is a 3.3.6 era bug, maybe upgrading all of your nodes to
> 3.3.9 would solve your problem?
>
> -Sandro
>
> -Original Message-
> From: owner-amanda-us...@amanda.org [mailto:owner-amanda-us...@amanda.org]
> On Behalf Of Chris Nighswonger
> Sent: Thursday, July 28, 2016 12:08 PM
> To: amanda-users@amanda.org
> Cc: Jean-Louis Martineau <jmartin...@carbonite.com>
> Subject: Re: amcheck failing when checking multiple clients
>
> Maybe John-Louis can jump in here since these are symptoms of a bug which
> was supposed to have been fixed in 3.3.6
>
>
> http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/amanda-20/amdumps-failing-and-some-amchecks-as-well-124340/
>
> Summary of where things stand on this install:
>
> Server version 3.3.6
> Client versions 3.3.3, 3.3.6
>
> auth = bsdtcp
>
> The issue:
>
> 1. amcheck fails on a subset of clients when run against multiple clients
> (regardless of client version) with "selfcheck request failed:
> error sending REQ: write error to: Broken pipe"
> 2. amcheck does not fail on all clients, but fails on the same subset of
> clients each time.
> 3. amcheck does not fail when run against any single client, including
> those in the subset which does fail when run against multiple clients.
> 4. amservice fails when run against any member of that subset of clients
> which fails amcheck with "Request failed: Permission denied"
> 5. dumps run fine during the nightly backups for all clients, including
> those members of that amcheck failing subset.
>
> Misc. notes:
>
> 1. When amcheck fails on a client, the client contains no debug log.
> 2. The server amcheck debug log reflects the same issue mentioned above
> "Broken pipe"
> 3. From the prospective of a failing client: tcpdump shows an exchange of
> about 40 packets with the client when amcheck is run against a single
> client and succeeds 4. From the same prospective, tcpdump shows an exchange
> of about 68 packets with the client when amcheck is run against multiple
> clients and fails 5. All file perms have been checked and are correct on
> both server and client 6. backup user settings are correct on server and
> client 7. amandahosts is properly configured 8. xinetd configuration is
> correct on both server and client 9. forward and reverse DNS works
> correctly for the client
>
> I wonder if this might be a variation of the bug mentioned by JLM in the
> above referenced thread?
>
> Thanks for everyone's help here.
>
>
> On Thu, Jul 28, 2016 at 10:15 AM, Ochressandro Rettinger <
> orettin...@salud.unm.edu> wrote:
> > In the second chunk of log messages, it looks like for some
> reason the BSDTCP connection is closing early.  Unfortunately, I can't help
> you out with that, as I'm set up to use ssh.
> >
> > Good luck!
> >
> > -Sandro
> >
> > -Original Message-
> > From: Chris Nighswonger [mailto:cnighswon...@foundations.edu]
> > Sent: Thursday, July 28, 2016 7:10 AM
> > To: amanda-users@amanda.org
> > Cc: Ochressandro Rettinger <orettin...@salud.unm.edu>
> > Subject: Re: amcheck failing when checking multiple clients
> >
> > Here are some potentially relevant lines form the log:
> >
> > Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> > conn_read_callback 1 3
> > Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients:
> > tcpm_recv_token: read 47 bytes from 1
> > Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> > conn_read_callback: tcpm_recv_token returned 47 Wed Jul 27 13:37:41
> 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> > stream_read_callback: handle 1
> > Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> > stream_read_callback: it was for us
> > Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:

Re: amcheck failing when checking multiple clients

[Chris Nighswonger]

Maybe John-Louis can jump in here since these are symptoms of a bug
which was supposed to have been fixed in 3.3.6

http://www.backupcentral.com/phpBB2/two-way-mirrors-of-external-mailing-lists-3/amanda-20/amdumps-failing-and-some-amchecks-as-well-124340/

Summary of where things stand on this install:

Server version 3.3.6
Client versions 3.3.3, 3.3.6

auth = bsdtcp

The issue:

1. amcheck fails on a subset of clients when run against multiple
clients (regardless of client version) with "selfcheck request failed:
error sending REQ: write error to: Broken pipe"
2. amcheck does not fail on all clients, but fails on the same subset
of clients each time.
3. amcheck does not fail when run against any single client, including
those in the subset which does fail when run against multiple clients.
4. amservice fails when run against any member of that subset of
clients which fails amcheck with "Request failed: Permission denied"
5. dumps run fine during the nightly backups for all clients,
including those members of that amcheck failing subset.

Misc. notes:

1. When amcheck fails on a client, the client contains no debug log.
2. The server amcheck debug log reflects the same issue mentioned
above "Broken pipe"
3. From the prospective of a failing client: tcpdump shows an exchange
of about 40 packets with the client when amcheck is run against a
single client and succeeds
4. From the same prospective, tcpdump shows an exchange of about 68
packets with the client when amcheck is run against multiple clients
and fails
5. All file perms have been checked and are correct on both server and client
6. backup user settings are correct on server and client
7. amandahosts is properly configured
8. xinetd configuration is correct on both server and client
9. forward and reverse DNS works correctly for the client

I wonder if this might be a variation of the bug mentioned by JLM in
the above referenced thread?

Thanks for everyone's help here.


On Thu, Jul 28, 2016 at 10:15 AM, Ochressandro Rettinger
<orettin...@salud.unm.edu> wrote:
> In the second chunk of log messages, it looks like for some reason 
> the BSDTCP connection is closing early.  Unfortunately, I can't help you out 
> with that, as I'm set up to use ssh.
>
> Good luck!
>
> -Sandro
>
> -Original Message-
> From: Chris Nighswonger [mailto:cnighswon...@foundations.edu]
> Sent: Thursday, July 28, 2016 7:10 AM
> To: amanda-users@amanda.org
> Cc: Ochressandro Rettinger <orettin...@salud.unm.edu>
> Subject: Re: amcheck failing when checking multiple clients
>
> Here are some potentially relevant lines form the log:
>
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> conn_read_callback 1 3
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients:
> tcpm_recv_token: read 47 bytes from 1
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> conn_read_callback: tcpm_recv_token returned 47 Wed Jul 27 13:37:41 2016: 
> thd-0x55f48f90bc00: amcheck-clients: sec:
> stream_read_callback: handle 1
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> stream_read_callback: it was for us
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> stream_read_callback: read 47 bytes from scriptor:1 Wed Jul 27 13:37:41 2016: 
> thd-0x55f48f90bc00: amcheck-clients: sec:
> recvpkt_callback: 47
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> cancelling recvpkt for scriptor
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> conn_read_cancel: decremented ev_read_refcnt to 0 for scriptor Wed Jul 27 
> 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> conn_read_cancel: releasing event handler for scriptor Wed Jul 27 13:37:41 
> 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> parse_pkt: parsing buffer of 47 bytes
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> parse_pkt: REP (1): "OPTIONS features=9efefbff3f;
> "
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> received REP packet (1) from scriptor, contains:
>
> "OPTIONS features=9efefbff3f;
> "
>
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> stream_sendpkt: enter
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> stream_sendpkt: ACK (3) pkt_t (len 0) contains:
>
> ""
>
> Wed Jul 27 13:37:41 2016: thd-0x55f48f90bc00: amcheck-clients: sec:
> stream_write: writing 2 bytes to scriptor:1 3 Wed Jul 27 13:37:41 2016: 
> thd-0x55f48f90bc00: amcheck-clients:
> tcpm_send_token: write 2 bytes to handle 1 Wed Jul 27 13:37:41 2016: 
> thd-0x55f48f90bc00: amcheck-clients:
> security_getdriver(name=bsdtcp) 

Re: amcheck failing when checking multiple clients

[Chris Nighswonger]

And for your further diagnostic pleasure:

backup@scriptor:/home/manager amservice scriptor bsdtcp noop  wrote:
> That's a good question. Only various routine updates as far as I know.
>
> Strangely enough, the clients failing amcheck seemed to have backed up
> ok last evening
>
> Here is the amcheck output from before the amdump run:
>
> Amanda Tape Server Host Check
> -
> Holding disk /storage/campus: 912248832 kB disk space available, using
> 912248832 kB
> Searching for label 'campus-NGH861L4':found in slot 9: volume 
> 'campus-NGH861L4'
> Will write to volume 'campus-NGH861L4' in slot 9.
> NOTE: skipping tape-writable test
> Server check took 158.373 seconds
>
> Amanda Backup Client Hosts Check
> 
> WARNING: scriptor: selfcheck request failed: error sending REQ: write
> error to: Broken pipe
> WARNING: masada: selfcheck request failed: error sending REQ: write
> error to: Broken pipe
>
> Yet amreport says:
>
> masada  /etc/dansguardian1  180
> 73.90:00  122.6   0:05 1.4
> masada  /etc/network 0   50
> 7   14.00:00  286.7   0:04 1.8
> masada  /etc/squid3  1   10
> 1   10.00:00   24.6   0:05 0.0
> masada  /home/backups/database/postgresql1   20
> 15.00:00   24.7   0:06 0.0
> scriptor/etc/amanda  0   40
> 4   10.00:00   29.2   0:05 0.8
> scriptor/var/backups 1 5670
>  5069   89.40:01 5896.7   0:05  1013.8
>
> Go figure... I think I'm going crazy(er)...
>
> Yet amreport also says stuff like:
>
> NOTES:
>   planner: scriptor /etc/amanda 20160727221502 0 "Request to scriptor
> failed: error sending REQ: write error to: Broken
>  pipe, using server estimate"
>   planner: scriptor /var/backups 20160727221502 0 "Request to scriptor
> failed: error sending REQ: write error to: Broke
> n pipe, using server estimate"
>
>
> On Wed, Jul 27, 2016 at 3:02 PM, Debra S Baddorf <badd...@fnal.gov> wrote:
>> What did you change, “about a month ago” ?
>>
>> I have TCP problems when a node is DOWN,  wherein amchecking it will cause 
>> problems
>> with several other nodes (due to TCP timeout values)  BUT your nodes are up.
>>
>> “Broken pipe”  plus  no logs on the client   sounds like the system level  
>> (outside of amanda?)
>> TCP  is having troubles;  the client is never getting an amanda started at 
>> all.
>> But I can’t think why that would happen.
>>
>> Deb Baddorf
>>
>>
>>> On Jul 27, 2016, at 1:20 PM, Chris Nighswonger 
>>> <cnighswon...@foundations.edu> wrote:
>>>
>>> No takers?
>>>
>>> Additionally, on the failing client side an amcheck log is created
>>> *only* if amcheck is run against that client. When amcheck is run
>>> against the entire job (all clients) no amcheck log is created on the
>>> clients that fail.
>>>
>>>
>>>
>>> On Tue, Jul 26, 2016 at 9:09 AM, Chris Nighswonger
>>> <cnighswon...@foundations.edu> wrote:
>>>> tcpdump shows that there is no (zero) data flow between the server and
>>>> failed clients when running amcheck on multiple clients at once. Data
>>>> does flow when checking a single client.
>>>>
>>>> This only started about a month ago. Maybe this is a bug?
>>>>
>>>> On Mon, Jul 25, 2016 at 4:16 PM, Chris Nighswonger
>>>> <cnighswon...@foundations.edu> wrote:
>>>>> Here is some snips of output illustrating the problem:
>>>>>
>>>>>
>>>>> backup@scriptor: amcheck -c campus scriptor
>>>>>
>>>>> Amanda Backup Client Hosts Check
>>>>> 
>>>>> Client check: 1 host checked in 2.372 seconds.  0 problems found.
>>>>>
>>>>> (brought to you by Amanda 3.3.6)
>>>>> backup@scriptor: amcheck -c campus masada
>>>>>
>>>>> Amanda Backup Client Hosts Check
>>>>> 
>>>>> Client check: 1 host checked in 2.074 seconds.  0 problems found.
>>>>>
>>>>> (brought to you by Amanda 3.3.6)
>>>>> backup@scriptor: amcheck -c campus
>>>>>
>>>>> Amanda Backup Client Hosts Check
>>>>> 
>>>>> WARNING: scriptor: selfcheck request failed: error sending REQ: write
>>>>> error to: Broken pipe
>>>>> WARNING: masada: selfcheck request failed: error sending REQ: write
>>>>> error to: Broken pipe
>>>>>
>>>>>
>>>>> Server version is 3.3.6
>>>>> Client version on scriptor is 3.3.6 and on masada is 3.3.3
>>>>>
>>>>> When the backup runs over night, the same error appears on these clients.
>>>>>
>>>>> Any thoughts on what might be going on here?
>>>>>
>>>>> Kind regards,
>>>>> Chris
>>

Re: amcheck failing when checking multiple clients

[Chris Nighswonger]

No takers?

Additionally, on the failing client side an amcheck log is created
*only* if amcheck is run against that client. When amcheck is run
against the entire job (all clients) no amcheck log is created on the
clients that fail.



On Tue, Jul 26, 2016 at 9:09 AM, Chris Nighswonger
<cnighswon...@foundations.edu> wrote:
> tcpdump shows that there is no (zero) data flow between the server and
> failed clients when running amcheck on multiple clients at once. Data
> does flow when checking a single client.
>
> This only started about a month ago. Maybe this is a bug?
>
> On Mon, Jul 25, 2016 at 4:16 PM, Chris Nighswonger
> <cnighswon...@foundations.edu> wrote:
>> Here is some snips of output illustrating the problem:
>>
>>
>> backup@scriptor: amcheck -c campus scriptor
>>
>> Amanda Backup Client Hosts Check
>> 
>> Client check: 1 host checked in 2.372 seconds.  0 problems found.
>>
>> (brought to you by Amanda 3.3.6)
>> backup@scriptor: amcheck -c campus masada
>>
>> Amanda Backup Client Hosts Check
>> 
>> Client check: 1 host checked in 2.074 seconds.  0 problems found.
>>
>> (brought to you by Amanda 3.3.6)
>> backup@scriptor: amcheck -c campus
>>
>> Amanda Backup Client Hosts Check
>> 
>> WARNING: scriptor: selfcheck request failed: error sending REQ: write
>> error to: Broken pipe
>> WARNING: masada: selfcheck request failed: error sending REQ: write
>> error to: Broken pipe
>>
>>
>> Server version is 3.3.6
>> Client version on scriptor is 3.3.6 and on masada is 3.3.3
>>
>> When the backup runs over night, the same error appears on these clients.
>>
>> Any thoughts on what might be going on here?
>>
>> Kind regards,
>> Chris

Re: amcheck failing when checking multiple clients

[Chris Nighswonger]

tcpdump shows that there is no (zero) data flow between the server and
failed clients when running amcheck on multiple clients at once. Data
does flow when checking a single client.

This only started about a month ago. Maybe this is a bug?

On Mon, Jul 25, 2016 at 4:16 PM, Chris Nighswonger
<cnighswon...@foundations.edu> wrote:
> Here is some snips of output illustrating the problem:
>
>
> backup@scriptor: amcheck -c campus scriptor
>
> Amanda Backup Client Hosts Check
> 
> Client check: 1 host checked in 2.372 seconds.  0 problems found.
>
> (brought to you by Amanda 3.3.6)
> backup@scriptor: amcheck -c campus masada
>
> Amanda Backup Client Hosts Check
> 
> Client check: 1 host checked in 2.074 seconds.  0 problems found.
>
> (brought to you by Amanda 3.3.6)
> backup@scriptor: amcheck -c campus
>
> Amanda Backup Client Hosts Check
> 
> WARNING: scriptor: selfcheck request failed: error sending REQ: write
> error to: Broken pipe
> WARNING: masada: selfcheck request failed: error sending REQ: write
> error to: Broken pipe
>
>
> Server version is 3.3.6
> Client version on scriptor is 3.3.6 and on masada is 3.3.3
>
> When the backup runs over night, the same error appears on these clients.
>
> Any thoughts on what might be going on here?
>
> Kind regards,
> Chris

amcheck failing when checking multiple clients

[Chris Nighswonger]

Here is some snips of output illustrating the problem:


backup@scriptor: amcheck -c campus scriptor

Amanda Backup Client Hosts Check

Client check: 1 host checked in 2.372 seconds.  0 problems found.

(brought to you by Amanda 3.3.6)
backup@scriptor: amcheck -c campus masada

Amanda Backup Client Hosts Check

Client check: 1 host checked in 2.074 seconds.  0 problems found.

(brought to you by Amanda 3.3.6)
backup@scriptor: amcheck -c campus

Amanda Backup Client Hosts Check

WARNING: scriptor: selfcheck request failed: error sending REQ: write
error to: Broken pipe
WARNING: masada: selfcheck request failed: error sending REQ: write
error to: Broken pipe


Server version is 3.3.6
Client version on scriptor is 3.3.6 and on masada is 3.3.3

When the backup runs over night, the same error appears on these clients.

Any thoughts on what might be going on here?

Kind regards,
Chris

Re: amreport claims tape is full but it is not

[Chris Nighswonger]

On Tue, Apr 26, 2016 at 12:48 PM, Chris Nighswonger <
cnighswon...@foundations.edu> wrote:

> So the Quantum tech had me run a device health test on the drive using
> their xTalk utility. The drive failed with some sort of comm error similar
> to what Amanda was seeing. So he then had me clean the drive twice and
> re-run the health test. The drive passed this time around.
>
>
> We will see what happens during the nightly backup run tonight.
>

The taper failed again on an assumed EOT. I reran xTalk and the health test
failed again. Sent off the new logs to Quantum. They are shipping a new
library to me overnight. Should be on line in time for tomorrow night's
backup run.

Kudos to Quantum (and their Rapid Replacement Program)!

Re: amreport claims tape is full but it is not

[Chris Nighswonger]

On Tue, Apr 19, 2016 at 11:35 AM, Alan Hodgson <ahodg...@lists.simkin.ca>
wrote:

> On Tuesday, April 19, 2016 09:57:32 AM Chris Nighswonger wrote:
> > I attempted Alan's suggestion and dd'd to the tape. /dev/md0 is > 130G
> and
> > dd borked almost immediately:
> >
> > root@scriptor:/home/manager dd if=/dev/md0 of=/dev/st0
> > dd: writing to ‘/dev/st0’: Input/output error
> > 3+0 records in
> > 2+0 records out
> > 1024 bytes (1.0 kB) copied, 15.2882 s, 0.1 kB/s
> >
> > Funny thing is that the Superloader passes all of its internal
> diagnostics
> > as far as I can tell.
> >
> > Suggestions on how to verify which piece of hardware is causing the
> problem
> > would be greatly appreciated.
> >
>
> That's kind of tough unless you have either a separate tape drive, or a
> separate server/SAS controller/cable combination to test.
>
> It seems to be talking OK to the loader, though, so odds are it's the
> drive.
>
> Quantum's pretty good about taking RMA's on the Superloaders without too
> much
> fuss, as long as you have a maintenance contract in place.
>
>
So the Quantum tech had me run a device health test on the drive using
their xTalk utility. The drive failed with some sort of comm error similar
to what Amanda was seeing. So he then had me clean the drive twice and
re-run the health test. The drive passed this time around.

So if it was dirty heads, why did the drive not indicate cleaning needed?
Or does the drive just indicate cleaning based on tape motion hours?

Here are the stats retrieved from the drive. Only 35 hrs in motion. Seems a
bit early for a cleaning...


Drive Information Report
Drive Type  : Ultrium 4 HH SCSI
Drive Serial Number --- : HU
Media Changer Present - : No
Vendor  ID  : QUANTUM
Product ID  : ULTRIUM 4
Product Revision Level  : W61T
Product Family  : 800.0 / 1600.0 GB
Servo FW Revision - : N/A
Power On Hours  : 4681
Tape Motion Hours - : 35
Load Count  : 308
Cleaning Tape Count --- : 2


We will see what happens during the nightly backup run tonight.

Re: amreport claims tape is full but it is not

[Chris Nighswonger]

Well, perhaps it is a bad device. A run of amflush CONFIG resulted in
another failure:

Fri Apr 15 11:52:31 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: updating state
Fri Apr 15 11:52:31 2016: thd-0x18bae00: taper:
Amanda::Taper::Scan::traditional stage 2: scan for any reusable volume
Fri Apr 15 11:52:31 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: too early for another 'status' invocation
Fri Apr 15 11:52:31 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: loading next relative to 5: 6
Fri Apr 15 11:52:31 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: using drive 0
Fri Apr 15 11:52:31 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: unloading drive 0
Fri Apr 15 11:52:31 2016: thd-0x18bae00: taper: invoking /usr/sbin/mtx -f
/dev/sg4 unload 5 0
Fri Apr 15 11:53:41 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: unload complete
Fri Apr 15 11:53:41 2016: thd-0x18bae00: taper: invoking /usr/sbin/mtx -f
/dev/sg4 load 6 0
Fri Apr 15 11:55:01 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: polling 'tape:/dev/nst0' to see if it's ready
Fri Apr 15 11:55:04 2016: thd-0x18bae00: taper:
Quantum-Superloader3-LTO-V4: setting current slot to 6
Fri Apr 15 11:55:04 2016: thd-0x18bae00: taper: Slot 6 with label
campus-NGH874L4 is usable
Fri Apr 15 11:55:04 2016: thd-0x18bae00: taper:
Amanda::Taper::Scan::traditional result: 'campus-NGH874L4' on
tape:/dev/nst0 slot 6, mode 2
Fri Apr 15 11:55:04 2016: thd-0x18bae00: taper: Amanda::Taper::Scribe
preparing to write, part size 0, using LEOM (falling back to holding disk
as cache) (splitter)  (LEOM supported)
Fri Apr 15 11:55:04 2016: thd-0x18bae00: taper: Starting  -> )>
Fri Apr 15 11:55:04 2016: thd-0x18bae00: taper: Final linkage:
 -(PULL_BUFFER)-> 
-(PUSH_BUFFER)-> 
Fri Apr 15 11:55:09 2016: thd-0x18bae00: taper: Building type TAPESTART
header of 32768-32768 bytes with name='campus-NGH874L4' disk='' dumplevel=0
and blocksize=32768
Fri Apr 15 11:55:11 2016: thd-0x18bae00: taper: Device tape:/dev/nst0 error
= 'Error writing filemark: Input/output error'
Fri Apr 15 11:55:11 2016: thd-0x18bae00: taper: Device tape:/dev/nst0
setting status flag(s): DEVICE_STATUS_DEVICE_ERROR, and
DEVICE_STATUS_VOLUME_ERROR
Fri Apr 15 11:55:27 2016: thd-0x18bae00: taper: tape campus-NGH874L4 kb 0
fm -1 [OK]

Re: amreport claims tape is full but it is not

[Chris Nighswonger]

On Fri, Apr 15, 2016 at 11:17 AM, Alan Hodgson 
wrote:

> On Friday, April 15, 2016 09:31:26 AM you wrote:
> > I'm trying to figure out how a LTO4 tape was filled up by 32k of data
> > or am I missing something here?
> >
>
> Any time I've started getting premature end-of-tape it has been because the
> drive is malfunctioning. Try a few times with just dd on a new tape and see
> how much it can write before it gets EOT.
>
>
Ouch! This drive is only a couple of months old...

I'll give this a try if switching the device type does not work.

Kind regards,
Chris

Re: amreport claims tape is full but it is not

[Chris Nighswonger]

On Fri, Apr 15, 2016 at 11:24 AM, Jean-Louis Martineau <
jmartin...@carbonite.com> wrote:

> On 15/04/16 11:15 AM, Chris Nighswonger wrote:
>
>> warning: Got EIO
>>
> What can amanda do when it get an EIO?
> Check system log for error.
> Amanda require a non-rewing device, you must use /dev/nst0 instead of
> /dev/st0
>

Interesting. I ran the last Quantum drive as /dev/st0 and never had issues.
I've changed over to /dev/nst0 and will see how it runs now.


> Do the tape drive is in variable block size?
>   mt -f /dev/nst0 status


backup@scriptor:~/campus$ mt -f /dev/nst0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 0 bytes. Density code 0x46 (LTO-4).
Soft error count since last status=0
General status bits on (4101):
 BOT ONLINE IM_REP_EN

Thanks for your help.

Kind regards,
Chris

Re: amreport claims tape is full but it is not

[Chris Nighswonger]

On Fri, Apr 15, 2016 at 10:21 AM, Jean-Louis Martineau <
jmartin...@carbonite.com> wrote:

> Which version of amanda are you using?
>

3.3.3 (on Ubuntu 14.04.4 LTS)


> Any other error message in the report?
>

none

What is the setting of max-dle-by-volume?
>
>
   amgetconf CONFIG max-dle-by-volume
>

backup@scriptor:~/campus$ amgetconf campus max-dle-by-volume
10



> post the taper debug file and the amdump.[x] file
>


There's a lot of stuff to sanitize in those files before posting here, but
here are most likely the relevant lines from the taper debug file:

Thu Apr 14 22:15:16 2016: thd-0x21e5050: taper: Building type SPLIT_FILE
header of 32768-32768 bytes with name='some_client' disk='/path/to/data'
dumplevel=1 and blocksize=32768
Thu Apr 14 22:15:16 2016: thd-0x21e5050: taper: warning: Got EIO on
/dev/st0, assuming end of tape
Thu Apr 14 22:15:16 2016: thd-0x21e5050: taper: Device tape:/dev/st0 error
= 'No space left on device'
Thu Apr 14 22:15:16 2016: thd-0x21e5050: taper: Device tape:/dev/st0
setting status flag(s): DEVICE_STATUS_VOLUME_ERROR
Thu Apr 14 22:15:27 2016: thd-0xf98c00: taper: Will request retry of failed
split part.
Thu Apr 14 22:15:27 2016: thd-0xf98c00: taper: tape campus-NGH873L4 kb 0 fm
1 [OK]


So this is a new Quantum Superloader, I can't imagine the heads are dirty
already.

Here is the tapetype as suggested by amtapetype:

define tapetype LTO-4 {
comment "Created by amtapetype; compression enabled"
length 794405408 kbytes
filemark 1385 kbytes
speed 77291 kps
blocksize 32 kbytes
}



The size of this backup job does not exceed 400GB at its largest size.

Kind regards,
Chris


---

These dumps were to tape campus-NGH873L4.
Not using all tapes because 1 tapes filled; runtapes=1 does not allow
additional tapes.
There are 112130560k of dumps left in the holding disk.
They will be flushed on the next run.

...


Tape Time (hrs:min) 0:00   0:00   0:00
Tape Size (meg)  0.00.00.0
Tape Used (%)0.00.00.0
DLEs Taped 1  0  1  1:1
Parts Taped1  0  1  1:1
Avg Tp Write Rate (k/s)  0.0-- 0.0

USAGE BY TAPE:
  Label   Time Size  %  DLEs Parts
  campus-NGH873L4 0:00  32k0.0 1 1

-


I'm trying to figure out how a LTO4 tape was filled up by 32k of data
> or am I missing something here?
>
> Amanda was doing this last week, so I loaded blank tapes and started the
> cycle fresh.
>
> Here is the contents of tapelist for this backup:
>
> 20160414221501 campus-NGH873L4 reuse BARCODE:NGH873L4 BLOCKSIZE:32
> 20160413221501 campus-NGH872L4 reuse BARCODE:NGH872L4 BLOCKSIZE:32
> 20160412221501 campus-NGH871L4 reuse BARCODE:NGH871L4 BLOCKSIZE:32
> 20160411221501 campus-NGH870L4 reuse BARCODE:NGH870L4 BLOCKSIZE:32
> 20160408221501 campus-NGH869L4 reuse BARCODE:NGH869L4 BLOCKSIZE:32
> 20160406222549 campus-NGH863L4 reuse BARCODE:NGH863L4 BLOCKSIZE:32
> 0 campus-NGH862L4 reuse BARCODE:NGH862L4 BLOCKSIZE:32
> 0 campus-NGH861L4 reuse BARCODE:NGH861L4 BLOCKSIZE:32
> 0 campus-NGH876L4 reuse BARCODE:NGH876L4 BLOCKSIZE:32
> 0 campus-NGH875L4 reuse BARCODE:NGH875L4 BLOCKSIZE:32
> 0 campus-NGH874L4 reuse BARCODE:NGH874L4 BLOCKSIZE:32
> 0 campus-NGH864L4 reuse BARCODE:NGH864L4 BLOCKSIZE:32
> 0 campus-NGH865L4 reuse BARCODE:NGH865L4 BLOCKSIZE:32
>
>
> Kind regards,
> Chris
>
>
>

amreport claims tape is full but it is not

[Chris Nighswonger]

Strange goings on here. amreport says:

---

These dumps were to tape campus-NGH873L4.
Not using all tapes because 1 tapes filled; runtapes=1 does not allow
additional tapes.
There are 112130560k of dumps left in the holding disk.
They will be flushed on the next run.

...


Tape Time (hrs:min) 0:00   0:00   0:00
Tape Size (meg)  0.00.00.0
Tape Used (%)0.00.00.0
DLEs Taped 1  0  1  1:1
Parts Taped1  0  1  1:1
Avg Tp Write Rate (k/s)  0.0-- 0.0

USAGE BY TAPE:
  Label   Time Size  %  DLEs Parts
  campus-NGH873L4 0:00  32k0.0 1 1

-


I'm trying to figure out how a LTO4 tape was filled up by 32k of data
or am I missing something here?

Amanda was doing this last week, so I loaded blank tapes and started the
cycle fresh.

Here is the contents of tapelist for this backup:

20160414221501 campus-NGH873L4 reuse BARCODE:NGH873L4 BLOCKSIZE:32
20160413221501 campus-NGH872L4 reuse BARCODE:NGH872L4 BLOCKSIZE:32
20160412221501 campus-NGH871L4 reuse BARCODE:NGH871L4 BLOCKSIZE:32
20160411221501 campus-NGH870L4 reuse BARCODE:NGH870L4 BLOCKSIZE:32
20160408221501 campus-NGH869L4 reuse BARCODE:NGH869L4 BLOCKSIZE:32
20160406222549 campus-NGH863L4 reuse BARCODE:NGH863L4 BLOCKSIZE:32
0 campus-NGH862L4 reuse BARCODE:NGH862L4 BLOCKSIZE:32
0 campus-NGH861L4 reuse BARCODE:NGH861L4 BLOCKSIZE:32
0 campus-NGH876L4 reuse BARCODE:NGH876L4 BLOCKSIZE:32
0 campus-NGH875L4 reuse BARCODE:NGH875L4 BLOCKSIZE:32
0 campus-NGH874L4 reuse BARCODE:NGH874L4 BLOCKSIZE:32
0 campus-NGH864L4 reuse BARCODE:NGH864L4 BLOCKSIZE:32
0 campus-NGH865L4 reuse BARCODE:NGH865L4 BLOCKSIZE:32


Kind regards,
Chris

Re: Quantum Superloader 3 - configuration

[Chris Nighswonger]

On Tue, Nov 10, 2015 at 9:39 AM, David Simpson
<david.simp...@eng.ox.ac.uk> wrote:
> I notice use-slots is set to 1-12... is that because you have both LHS and 
> RHS tape magazines?

Yes. I actually run 16 tapes, but 4 are for a different job.

>
> I'll probably try to leave the profiler going for the HP LTO6 on Friday 
> before going home.

That's probably the best idea, since I'm sure the specs for LTO6 will
differ some.

Kind regards,
Chris


>
> -Original Message-
> From: Chris Nighswonger [mailto:cnighswon...@foundations.edu]
> Sent: 10 November 2015 14:03
> To: David Simpson <david.simp...@eng.ox.ac.uk>
> Cc: Amanda Users <amanda-users@amanda.org>
> Subject: Re: Quantum Superloader 3 - configuration
>
> Hi David,
>
> We run a Superloader 3 with an LTO4 drive. Here are some snips from my
> amanda.conf:
>
> define tapetype LTO-4 {
> length 794405408 kbytes
> filemark 1385 kbytes
> speed 77291 kps
> blocksize 32 kbytes
> }
>
> define changer Quantum-Superloader3-LTO-V4 {
> tapedev "chg-robot:/dev/sg3"
> property "use-slots" "1-12"
> property "tape-device" "0=tape:/dev/st0"
> device-property "LEOM" "TRUE"
> }
>
> This is running on Ubuntu 13.04.
>
> I ended up using amtapetype due to not being able to locate a tape 
> definition. Someday I'll add it to the wiki.
>
> Kind regards,
> Chris
>
>
> Christopher Nighswonger
> Faculty Member
> Network & Systems Director
> Foundations Bible College & Seminary
> www.foundations.edu
> www.fbcradio.org
> -
> NOTICE: The information contained in this electronic mail message is intended 
> only for the use of the intended recipient, and may also be protected by the 
> Electronic Communications Privacy Act, 18 USC Sections 2510-2521. If the 
> reader of this message is not the intended recipient, you are hereby notified 
> that any dissemination, distribution or copying of this communication is 
> strictly prohibited.
> If you have received this communication in error, please reply to the sender, 
> and delete the original message. Thank you.
>
>
> On Tue, Nov 10, 2015 at 8:41 AM, David Simpson <david.simp...@eng.ox.ac.uk> 
> wrote:
>> Hi,
>>
>> Does anyone have configuration for a Quantum Superloader 3 they would
>> be willing to share?
>>
>> I’m currently trying to use it with PERC5e and Centos 7. This is
>> proving a crash course for me in tape and Amanda! (first time with
>> both)
>>
>> In Amanda.conf I have the following plumbed in:
>>
>> changerdev "/dev/sg3"
>>
>> tapedev "/dev/nst0"
>>
>> I can write to a tape, but I’m not sure I have defined changer dev
>> correctly though. I’ve only got two HP LTO6 tapes to play with right
>> now, I’m looking to verify that Amanda can change to the next tape
>> (with no human
>> intervention) when it sees fit.
>>
>> I’m also going to try getting the profile of this HP LTO6 tape as I’ve
>> not found a definition for it.
>>
>> thanks
>> David
>>
>> 
>> David Simpson - Computing Support Officer IBME - Institute of
>> Biomedical Engineering Old Road Campus Research Building Oxford, OX3
>> 7DQ
>> Tel: 01865 617697 ext: 17697
>>
>>

Re: Quantum Superloader 3 - configuration

[Chris Nighswonger]

Hi David,

We run a Superloader 3 with an LTO4 drive. Here are some snips from my
amanda.conf:

define tapetype LTO-4 {
length 794405408 kbytes
filemark 1385 kbytes
speed 77291 kps
blocksize 32 kbytes
}

define changer Quantum-Superloader3-LTO-V4 {
tapedev "chg-robot:/dev/sg3"
property "use-slots" "1-12"
property "tape-device" "0=tape:/dev/st0"
device-property "LEOM" "TRUE"
}

This is running on Ubuntu 13.04.

I ended up using amtapetype due to not being able to locate a tape
definition. Someday I'll add it to the wiki.

Kind regards,
Chris


Christopher Nighswonger
Faculty Member
Network & Systems Director
Foundations Bible College & Seminary
www.foundations.edu
www.fbcradio.org
-
NOTICE: The information contained in this electronic mail message is
intended only for the use of the intended recipient, and may also be
protected by the Electronic Communications Privacy Act, 18 USC
Sections 2510-2521. If the reader of this message is not the intended
recipient, you are hereby notified that any dissemination,
distribution or copying of this communication is strictly prohibited.
If you have received this communication in error, please reply to the
sender, and delete the original message. Thank you.


On Tue, Nov 10, 2015 at 8:41 AM, David Simpson
 wrote:
> Hi,
>
> Does anyone have configuration for a Quantum Superloader 3 they would be
> willing to share?
>
> I’m currently trying to use it with PERC5e and Centos 7. This is proving a
> crash course for me in tape and Amanda! (first time with both)
>
> In Amanda.conf I have the following plumbed in:
>
> changerdev "/dev/sg3"
>
> tapedev "/dev/nst0"
>
> I can write to a tape, but I’m not sure I have defined changer dev correctly
> though. I’ve only got two HP LTO6 tapes to play with right now, I’m looking
> to verify that Amanda can change to the next tape (with no human
> intervention) when it sees fit.
>
> I’m also going to try getting the profile of this HP LTO6 tape as I’ve not
> found a definition for it.
>
> thanks
> David
>
> 
> David Simpson - Computing Support Officer
> IBME - Institute of Biomedical Engineering
> Old Road Campus Research Building
> Oxford, OX3 7DQ
> Tel: 01865 617697 ext: 17697
>
>

Re: To flush or not to flush, that is the question....

[Chris Nighswonger]

No takers?

On Tue, Oct 27, 2015 at 3:48 PM, Chris Nighswonger
<cnighswon...@foundations.edu> wrote:
> Here is one for the list:
>
> backup@scriptor:~/campus$ amreport campus
> Hostname: scriptor
> Org : Daily CAMPUS Backup
> Config  : campus
> Date: October 27, 2015
>
> These dumps were to tape campus-NGH862L4.
> There are 59843032k of dumps left in the holding disk.
> They will be flushed on the next run.
>
>
> And yet:
>
> backup@scriptor:~/campus$ amflush campus
> Could not find any Amanda directories to flush.
>
>
> Any idea what's up with all of that? And how to fix it?
>
> Kind regards,
> Chris

Re: To flush or not to flush, that is the question....

[Chris Nighswonger]

Hi Jean-Louis,

On Fri, Oct 30, 2015 at 1:24 PM, Jean-Louis Martineau
 wrote:
> Chris,
>
> You can use the following command to list all amanda dump in holding disk:
>  $ amadmin CONFIG holding list -l
>
> What's in the holding (with the 'ls' command), it looks you have a lot of
> files not recognized by amanda.

backup@scriptor:/storage/campus$ amadmin campus holding list -l
size (kB)  lv outd dump specification

So it looks like Amanda thinks there is nothing there. However,
between your suggestion and Debra's, I took a look at my holding disk
(I should have done this already) and found quite a large collection
of (very) old dumps. After cleaning up the holding disk, the flush
notice in the amreport is gone.

I'm not sure what all may have transpired to result in these orphaned
dumps, but it probably had to do with the failing tape library we
replaced last month.

Thanks to you and Debra for pointing me in the right direction!

Kind regards,
Chris

Logfile permissions

[Chris Nighswonger]

So why does amanda write backup job logfiles with 600 perms? And is there a
way to make it do otherwise?

Kind Regards,
Chris

Re: Zmanda Windows Client ZIP files

[Chris Nighswonger]

After some further testing this morning and this afternoon, here is what I
find:

1. I am able to unzip the files on our Amanda server using the unix 'unzip'
utility, but there are random  error:  invalid compressed data to
inflate errors. These do not appear to affect all files, but that is
little consolation. Testing the archive with 'unzip -t foo.zip' shows that
probably a majority of the files are OK, but many throw the same error just
mentioned.

2. Moving the archive to a Win32 box I find that I can browse the archive
using the built-in Windows zip functionality. I am also able to selectively
extract various files and their contents seem to be completely intact as
expected. However, attempting a full extract of the file results in an
Unexpected end of file error.

3. Attempting to use 7zip to open as an archive results in Cannot open
foo as an archive error. Testing the archive with 7zip results in the same
error.

It seems that there is a definite issue with ZWC. Its a bit much trouble to
setup a test bed and hunt through prior versions of ZWC to discover if this
is a regression and when it occurred, although I think it is as I have not
noticed this problem when restoring files in the past from Win32 backups.

Maybe Paddy or someone from Zmanda can jump in here and help scare up what
is going on here.

Kind Regards,
Chris

PS. I did find an old post regarding a similar issue[1]. However, the ZWC
does compression client-side, so I doubt the server hardware has much to do
with it unless one is unzipping on the server and it happens to have some
hardware problem.

[1] https://forums.zmanda.com/showthread.php?2139-Recovery-failed!

On Fri, Nov 15, 2013 at 12:41 PM, flooding Controlled 
flooding2...@gmail.com wrote:

 I have the  same issue on the pkzip files.  The latest version amanda
 3.3.4 uses this as default on windows client.

 So it made me difficult to unzip the pkzip files... any suggestions ?
  Thanks.


 Regards.

 flooding2012

Re: Zmanda Windows Client ZIP files

[Chris Nighswonger]

Adding some additional info which I should have included to start with:

On Fri, Nov 15, 2013 at 2:00 PM, Markus Iturriaga Woelfel 
mitur...@eecs.utk.edu wrote:

 Here are my findings in relation to what Chris posted:

 On Nov 15, 2013, at 1:55 PM, Chris Nighswonger 
 cnighswon...@foundations.edu wrote:

 After some further testing this morning and this afternoon, here is what I
 find:

 1. I am able to unzip the files on our Amanda server using the unix
 'unzip' utility, but there are random  error:  invalid compressed data
 to inflate errors. These do not appear to affect all files, but that is
 little consolation. Testing the archive with 'unzip -t foo.zip' shows that
 probably a majority of the files are OK, but many throw the same error just
 mentioned.


 I believe it may be related to file size. I can not even list the contents
 of a rather large (~16G) ZIP file. Unizp on my system is compiled with both
 large file and ZIP64 support.


The archive I used was a little over 4GB.




 2. Moving the archive to a Win32 box I find that I can browse the archive
 using the built-in Windows zip functionality. I am also able to selectively
 extract various files and their contents seem to be completely intact as
 expected. However, attempting a full extract of the file results in an
 Unexpected end of file error.


 My Windows 7 and 8 systems both hang when trying to open the file in
 Windows Explorer. It's possible it would eventually succeed but I gave up
 after about 20 minutes.


My tests were done on a Windows XP SP3 box.

Re: Zmanda Windows Client ZIP files

[Chris Nighswonger]

On Fri, Nov 15, 2013 at 2:20 PM, Schlacta, Christ aarc...@aarcane.orgwrote:

 This subject segues to a feature request fairly well. Can zwc or another
 client be made to integrate well with the existing Windows 7 and 8 system
 backup and restore interface? That'd be a dream.


ZWC uses the MS volume shadow service to take a snapshot of the volume.
Then they use the PKzip library[1] to zip it up (which is probably why the
source is not GPL).

It should be a relatively simple thing to do a similar thing using an OSS
zip library I would think.

Kind Regards,
Chris


[1] C:\Program Files\Zmanda\Zmanda Client for Windows Community
Edition\bin\pkzip.dll

Re: Zmanda Windows Client ZIP files

[Chris Nighswonger]

On Fri, Nov 15, 2013 at 2:37 PM, Paddy Sreenivasan 
psreeniva...@carbonite.com wrote:




 On Fri, Nov 15, 2013 at 11:31 AM, Chris Nighswonger 
 cnighswon...@foundations.edu wrote:


 On Fri, Nov 15, 2013 at 2:20 PM, Schlacta, Christ aarc...@aarcane.orgwrote:

 This subject segues to a feature request fairly well. Can zwc or another
 client be made to integrate well with the existing Windows 7 and 8 system
 backup and restore interface? That'd be a dream.


 ZWC uses the MS volume shadow service to take a snapshot of the volume.
 Then they use the PKzip library[1] to zip it up (which is probably why the
 source is not GPL).


 We don't use Pkzip command to zip it up. We use ZIP64 archive format.
 PKzip is only tool that we have seen that uses ZIP64 format
 well. Others zip programs also attempt to do so but may not work in all
 cases.


So what's wrong with 7zip's[1] library?

Kind Regards,
Chris


[1] http://7-zip.org/7z.html