Re: Odd non-fatal errors in amdump reports.

[Jean-Louis Martineau]

On 13/11/17 02:53 PM, Austin S. Hemmelgarn wrote:

driver: send-cmd time 9300.326 to taper1: VAULT-WRITE worker1-0 00-00120 local-vtl local-vtl Home-0001 client0 /home/1D 0 20171113073255 
"" "" "" "" 1073741824 memory "" "" 0


FAIL taper "ST:cloud" "POOL:cloud" client0 /home/1D 20171113073255 0 error "File 0 
not found"


Do that dump still exists on tape Home-0001? Find it with amfetchdump.

If yes, send me the taper debug file.


Jean-Louis
This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: power down hard drives

[Austin S. Hemmelgarn]

On 2017-11-13 14:51, Jon LaBadie wrote:

On Mon, Nov 13, 2017 at 02:04:42PM -0500, Gene Heskett wrote:

On Monday 13 November 2017 13:42:13 Jon LaBadie wrote:


On Mon, Nov 13, 2017 at 11:40:17AM -0500, Austin S. Hemmelgarn wrote:

On 2017-11-13 11:11, Gene Heskett wrote:

On Monday 13 November 2017 10:12:47 Austin S. Hemmelgarn wrote:

On 2017-11-13 09:56, Gene Heskett wrote:

On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:

On 2017-11-11 01:49, Jon LaBadie wrote:

Just a thought.  My amanda server has seven hard drives
dedicated to saving amanda data.  Only 2 are typically
used (holding and one vtape drive) during an amdump run.
Even then, the usage is only for about 3 hours.

So there is a lot of electricity and disk drive wear for
inactive drives.

Can todays drives be unmounted and powered down then
when needed, powered up and mounted again?

I'm not talking about system hibernation, the system
and its other drives still need to be active.

Back when 300GB was a big drive I had 2 of them in
external USB housings.  They shut themselves down
on inactivity.  When later accessed, there would
be about 5-10 seconds delay while the drive spun
up and things proceeded normally.

That would be a fine arrangement now if it could
be mimiced.


Aside from what Stefan mentioned (using hdparam to set the
standby timeout, check the man page for hdparam as the
numbers are not exactly sensible), you may consider looking
into auto-mounting each of the drives, as that can help
eliminate things that would keep the drives on-line (or make
it more obvious that something is still using them).


...


But if I allow the 2TB to be  unmounted and self-powered down,
once daily, what shortening of its life would I be subjected to?
In other words, how many start-stop cycles can it survive?


It's hard to be certain.  For what it's worth though, you might want
to test this to be certain that it's actually going to save you
energy.  It takes a lot of power to get the platters up to speed,
but it doesn't take much to keep them running at that speed.  It
might be more advantageous to just configure the device to idle
(that is, park the heads) after some time out and leave the platters
spinning instead of spinning down completely (and it should result
in less wear on the spindle motor).


In my situation, each of the six data drives is only
needed for a 2 week period out of each 12 weeks.  Once
shutdown, it could be down for 10 weeks.

Jon


Which is more than enough time for stiction to appear if the heads are
not parked off disk.


Don't today's drives automatically park heads?
I don't think there were ever any (at least, not ATA or SAS) that didn't 
when they went into standby.  In fact, I've never seen a modern style 
hard disk with 'voice coil' style actuators that didn't automatically 
park the heads (and part of my job is tearing apart old hard drives 
prior to physical media destruction, so I've seen my fair share of them).

Re: power down hard drives

[Gene Heskett]

On Monday 13 November 2017 14:51:59 Jon LaBadie wrote:

> On Mon, Nov 13, 2017 at 02:04:42PM -0500, Gene Heskett wrote:
> > On Monday 13 November 2017 13:42:13 Jon LaBadie wrote:
> > > On Mon, Nov 13, 2017 at 11:40:17AM -0500, Austin S. Hemmelgarn 
wrote:
> > > > On 2017-11-13 11:11, Gene Heskett wrote:
> > > > > On Monday 13 November 2017 10:12:47 Austin S. Hemmelgarn wrote:
> > > > > > On 2017-11-13 09:56, Gene Heskett wrote:
> > > > > > > On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn 
wrote:
> > > > > > > > On 2017-11-11 01:49, Jon LaBadie wrote:
> > > > > > > > > Just a thought.  My amanda server has seven hard
> > > > > > > > > drives dedicated to saving amanda data.  Only 2 are
> > > > > > > > > typically used (holding and one vtape drive) during an
> > > > > > > > > amdump run. Even then, the usage is only for about 3
> > > > > > > > > hours.
> > > > > > > > >
> > > > > > > > > So there is a lot of electricity and disk drive wear
> > > > > > > > > for inactive drives.
> > > > > > > > >
> > > > > > > > > Can todays drives be unmounted and powered down then
> > > > > > > > > when needed, powered up and mounted again?
> > > > > > > > >
> > > > > > > > > I'm not talking about system hibernation, the system
> > > > > > > > > and its other drives still need to be active.
> > > > > > > > >
> > > > > > > > > Back when 300GB was a big drive I had 2 of them in
> > > > > > > > > external USB housings.  They shut themselves down
> > > > > > > > > on inactivity.  When later accessed, there would
> > > > > > > > > be about 5-10 seconds delay while the drive spun
> > > > > > > > > up and things proceeded normally.
> > > > > > > > >
> > > > > > > > > That would be a fine arrangement now if it could
> > > > > > > > > be mimiced.
> > > > > > > >
> > > > > > > > Aside from what Stefan mentioned (using hdparam to set
> > > > > > > > the standby timeout, check the man page for hdparam as
> > > > > > > > the numbers are not exactly sensible), you may consider
> > > > > > > > looking into auto-mounting each of the drives, as that
> > > > > > > > can help eliminate things that would keep the drives
> > > > > > > > on-line (or make it more obvious that something is still
> > > > > > > > using them).
> > >
> > > ...
> > >
> > > > > But if I allow the 2TB to be  unmounted and self-powered down,
> > > > > once daily, what shortening of its life would I be subjected
> > > > > to? In other words, how many start-stop cycles can it survive?
> > > >
> > > > It's hard to be certain.  For what it's worth though, you might
> > > > want to test this to be certain that it's actually going to save
> > > > you energy.  It takes a lot of power to get the platters up to
> > > > speed, but it doesn't take much to keep them running at that
> > > > speed.  It might be more advantageous to just configure the
> > > > device to idle (that is, park the heads) after some time out and
> > > > leave the platters spinning instead of spinning down completely
> > > > (and it should result in less wear on the spindle motor).
> > >
> > > In my situation, each of the six data drives is only
> > > needed for a 2 week period out of each 12 weeks.  Once
> > > shutdown, it could be down for 10 weeks.
> > >
> > > Jon
> >
> > Which is more than enough time for stiction to appear if the heads
> > are not parked off disk.
>
> Don't today's drives automatically park heads?
>
> jl
Some may, but with improved control over surface smoothness, I suspect 
few are. In those I've taken apart, I have yet to find a parking ramp 
that lifted the heads clear of the disk when brought to the edge of the 
disk.


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: power down hard drives

[Jon LaBadie]

On Mon, Nov 13, 2017 at 02:04:42PM -0500, Gene Heskett wrote:
> On Monday 13 November 2017 13:42:13 Jon LaBadie wrote:
> 
> > On Mon, Nov 13, 2017 at 11:40:17AM -0500, Austin S. Hemmelgarn wrote:
> > > On 2017-11-13 11:11, Gene Heskett wrote:
> > > > On Monday 13 November 2017 10:12:47 Austin S. Hemmelgarn wrote:
> > > > > On 2017-11-13 09:56, Gene Heskett wrote:
> > > > > > On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:
> > > > > > > On 2017-11-11 01:49, Jon LaBadie wrote:
> > > > > > > > Just a thought.  My amanda server has seven hard drives
> > > > > > > > dedicated to saving amanda data.  Only 2 are typically
> > > > > > > > used (holding and one vtape drive) during an amdump run.
> > > > > > > > Even then, the usage is only for about 3 hours.
> > > > > > > >
> > > > > > > > So there is a lot of electricity and disk drive wear for
> > > > > > > > inactive drives.
> > > > > > > >
> > > > > > > > Can todays drives be unmounted and powered down then
> > > > > > > > when needed, powered up and mounted again?
> > > > > > > >
> > > > > > > > I'm not talking about system hibernation, the system
> > > > > > > > and its other drives still need to be active.
> > > > > > > >
> > > > > > > > Back when 300GB was a big drive I had 2 of them in
> > > > > > > > external USB housings.  They shut themselves down
> > > > > > > > on inactivity.  When later accessed, there would
> > > > > > > > be about 5-10 seconds delay while the drive spun
> > > > > > > > up and things proceeded normally.
> > > > > > > >
> > > > > > > > That would be a fine arrangement now if it could
> > > > > > > > be mimiced.
> > > > > > >
> > > > > > > Aside from what Stefan mentioned (using hdparam to set the
> > > > > > > standby timeout, check the man page for hdparam as the
> > > > > > > numbers are not exactly sensible), you may consider looking
> > > > > > > into auto-mounting each of the drives, as that can help
> > > > > > > eliminate things that would keep the drives on-line (or make
> > > > > > > it more obvious that something is still using them).
> >
> > ...
> >
> > > > But if I allow the 2TB to be  unmounted and self-powered down,
> > > > once daily, what shortening of its life would I be subjected to? 
> > > > In other words, how many start-stop cycles can it survive?
> > >
> > > It's hard to be certain.  For what it's worth though, you might want
> > > to test this to be certain that it's actually going to save you
> > > energy.  It takes a lot of power to get the platters up to speed,
> > > but it doesn't take much to keep them running at that speed.  It
> > > might be more advantageous to just configure the device to idle
> > > (that is, park the heads) after some time out and leave the platters
> > > spinning instead of spinning down completely (and it should result
> > > in less wear on the spindle motor).
> >
> > In my situation, each of the six data drives is only
> > needed for a 2 week period out of each 12 weeks.  Once
> > shutdown, it could be down for 10 weeks.
> >
> > Jon
> 
> Which is more than enough time for stiction to appear if the heads are 
> not parked off disk.
> 
Don't today's drives automatically park heads?

jl
-- 
Jon H. LaBadie j...@jgcomp.com
 11226 South Shore Rd.  (703) 787-0688 (H)
 Reston, VA  20190  (703) 935-6720 (C)

Re: power down hard drives

[Gene Heskett]

On Monday 13 November 2017 13:42:13 Jon LaBadie wrote:

> On Mon, Nov 13, 2017 at 11:40:17AM -0500, Austin S. Hemmelgarn wrote:
> > On 2017-11-13 11:11, Gene Heskett wrote:
> > > On Monday 13 November 2017 10:12:47 Austin S. Hemmelgarn wrote:
> > > > On 2017-11-13 09:56, Gene Heskett wrote:
> > > > > On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:
> > > > > > On 2017-11-11 01:49, Jon LaBadie wrote:
> > > > > > > Just a thought.  My amanda server has seven hard drives
> > > > > > > dedicated to saving amanda data.  Only 2 are typically
> > > > > > > used (holding and one vtape drive) during an amdump run.
> > > > > > > Even then, the usage is only for about 3 hours.
> > > > > > >
> > > > > > > So there is a lot of electricity and disk drive wear for
> > > > > > > inactive drives.
> > > > > > >
> > > > > > > Can todays drives be unmounted and powered down then
> > > > > > > when needed, powered up and mounted again?
> > > > > > >
> > > > > > > I'm not talking about system hibernation, the system
> > > > > > > and its other drives still need to be active.
> > > > > > >
> > > > > > > Back when 300GB was a big drive I had 2 of them in
> > > > > > > external USB housings.  They shut themselves down
> > > > > > > on inactivity.  When later accessed, there would
> > > > > > > be about 5-10 seconds delay while the drive spun
> > > > > > > up and things proceeded normally.
> > > > > > >
> > > > > > > That would be a fine arrangement now if it could
> > > > > > > be mimiced.
> > > > > >
> > > > > > Aside from what Stefan mentioned (using hdparam to set the
> > > > > > standby timeout, check the man page for hdparam as the
> > > > > > numbers are not exactly sensible), you may consider looking
> > > > > > into auto-mounting each of the drives, as that can help
> > > > > > eliminate things that would keep the drives on-line (or make
> > > > > > it more obvious that something is still using them).
>
> ...
>
> > > But if I allow the 2TB to be  unmounted and self-powered down,
> > > once daily, what shortening of its life would I be subjected to? 
> > > In other words, how many start-stop cycles can it survive?
> >
> > It's hard to be certain.  For what it's worth though, you might want
> > to test this to be certain that it's actually going to save you
> > energy.  It takes a lot of power to get the platters up to speed,
> > but it doesn't take much to keep them running at that speed.  It
> > might be more advantageous to just configure the device to idle
> > (that is, park the heads) after some time out and leave the platters
> > spinning instead of spinning down completely (and it should result
> > in less wear on the spindle motor).
>
> In my situation, each of the six data drives is only
> needed for a 2 week period out of each 12 weeks.  Once
> shutdown, it could be down for 10 weeks.
>
> Jon

Which is more than enough time for stiction to appear if the heads are 
not parked off disk.


Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: power down hard drives

[Jon LaBadie]

On Mon, Nov 13, 2017 at 11:40:17AM -0500, Austin S. Hemmelgarn wrote:
> On 2017-11-13 11:11, Gene Heskett wrote:
> > On Monday 13 November 2017 10:12:47 Austin S. Hemmelgarn wrote:
> > 
> > > On 2017-11-13 09:56, Gene Heskett wrote:
> > > > On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:
> > > > > On 2017-11-11 01:49, Jon LaBadie wrote:
> > > > > > Just a thought.  My amanda server has seven hard drives
> > > > > > dedicated to saving amanda data.  Only 2 are typically
> > > > > > used (holding and one vtape drive) during an amdump run.
> > > > > > Even then, the usage is only for about 3 hours.
> > > > > > 
> > > > > > So there is a lot of electricity and disk drive wear for
> > > > > > inactive drives.
> > > > > > 
> > > > > > Can todays drives be unmounted and powered down then
> > > > > > when needed, powered up and mounted again?
> > > > > > 
> > > > > > I'm not talking about system hibernation, the system
> > > > > > and its other drives still need to be active.
> > > > > > 
> > > > > > Back when 300GB was a big drive I had 2 of them in
> > > > > > external USB housings.  They shut themselves down
> > > > > > on inactivity.  When later accessed, there would
> > > > > > be about 5-10 seconds delay while the drive spun
> > > > > > up and things proceeded normally.
> > > > > > 
> > > > > > That would be a fine arrangement now if it could
> > > > > > be mimiced.
> > > > > 
> > > > > Aside from what Stefan mentioned (using hdparam to set the standby
> > > > > timeout, check the man page for hdparam as the numbers are not
> > > > > exactly sensible), you may consider looking into auto-mounting each
> > > > > of the drives, as that can help eliminate things that would keep
> > > > > the drives on-line (or make it more obvious that something is still
> > > > > using them).
> > > > 
...
> > 
> > But if I allow the 2TB to be  unmounted and self-powered down, once
> > daily, what shortening of its life would I be subjected to?  In other
> > words, how many start-stop cycles can it survive?
> >
> It's hard to be certain.  For what it's worth though, you might want to test
> this to be certain that it's actually going to save you energy.  It takes a
> lot of power to get the platters up to speed, but it doesn't take much to
> keep them running at that speed.  It might be more advantageous to just
> configure the device to idle (that is, park the heads) after some time out
> and leave the platters spinning instead of spinning down completely (and it
> should result in less wear on the spindle motor).
> > 
In my situation, each of the six data drives is only
needed for a 2 week period out of each 12 weeks.  Once
shutdown, it could be down for 10 weeks.

Jon
-- 
Jon H. LaBadie j...@jgcomp.com
 11226 South Shore Rd.  (703) 787-0688 (H)
 Reston, VA  20190  (703) 935-6720 (C)

Re: Odd non-fatal errors in amdump reports.

[Austin S. Hemmelgarn]

On 2017-11-10 12:52, Jean-Louis Martineau wrote:

The previous patch broke something.
Try this new set2-r2.diff patch


Unfortunately, that doesn't appear to have fixed it, though the errors 
look different now.  I'll try and get the log scrubbed by the end of the 
day and post it here.


On 10/11/17 10:40 AM, Austin S. Hemmelgarn wrote:
 > On 2017-11-10 08:27, Jean-Louis Martineau wrote:
 >> On 10/11/17 07:57 AM, Austin S. Hemmelgarn wrote:
 >>> On 2017-11-08 08:03, Jean-Louis Martineau wrote:
  On 07/11/17 02:58 PM, Austin S. Hemmelgarn wrote:
  > On 2017-11-07 10:22, Jean-Louis Martineau wrote:
  >> Austin,
  >>
  >> It's hard to say something with only the error message.
  >>
  >> Can you post the amdump. and log..0 for
  the 2
  >> backup set that fail.
  >>
  > I've attached the files (I would put them inline, but one of the
  sets
  > has over 100 DLE's, so the amdump file is huge, and the others are
  > still over 100k each, and I figured nobody want's to try and wad
  > through those in-line).
  >
  > The set1 and set2 files are for the two backup sets that show the
  > header mismatch error, and the set3 files are for the one that
  claims
  > failures in the dump summary.
 
 
  I looked at set3, the error in the 'DUMP SUMMARY' are related to the
  error in the 'FAILURE DUMP SUMMARY'
 
  client2 /boot lev 0 FLUSH [File 0 not found]
  client3 /boot lev 0 FLUSH [File 0 not found]
  client7 /boot lev 0 FLUSH [File 0 not found]
  client8 /boot lev 0 FLUSH [File 0 not found]
  client0 /boot lev 0 FLUSH [File 0 not found]
  client9 /boot lev 0 FLUSH [File 0 not found]
  client9 /srv lev 0 FLUSH [File 0 not found]
  client9 /var lev 0 FLUSH [File 0 not found]
  server0 /boot lev 0 FLUSH [File 0 not found]
  client10 /boot lev 0 FLUSH [File 0 not found]
  client11 /boot lev 0 FLUSH [File 0 not found]
  client12 /boot lev 0 FLUSH [File 0 not found]
 
  They are VAULT attemp, not FLUSH, looking only at the first entry, it
  try to vault 'client2 /boot 0 20171024084159' which it expect to
  find on
  tape Server-01. It is an older dump.
 
  Do Server-01 is still there? Did it still contains the dump?
 
 >>> OK, I've done some further investigation by tweaking the labeling a
 >>> bit (which actually fixed a purely cosmetic issue we were having),
 >>> but I'm still seeing the same problem that prompted this thread, and
 >>> I can confirm that the dumps are where Amanda is trying to look for
 >>> them, it's just not seeing them for some reason. I hadn't thought
 >>> of this before, but could it have something to do with the virtual
 >>> tape library being auto-mounted over NFS on the backup server?
 >>>
 >> Austin,
 >>
 >> Can you try to see if amfetchdump can restore it?
 >>
 >> * amfetchdump CONFIG client2 /boot 20171024084159
 >>
 > amfetchdump doesn't see it, and neither does amrecover, but the files
 > for the given parts are definitely there (I know for a fact that the
 > dump in question has exactly one part, and the file for that does
 > exist on the virtual tape mentioned in the log file).
 >
 > I'm probably not going to be able to check more on this today, but
 > I'll likely be checking if amrestore and amadmin find can see them.
 >

Re: power down hard drives

[Gene Heskett]

On Monday 13 November 2017 11:40:17 Austin S. Hemmelgarn wrote:

> On 2017-11-13 11:11, Gene Heskett wrote:
> > On Monday 13 November 2017 10:12:47 Austin S. Hemmelgarn wrote:
> >> On 2017-11-13 09:56, Gene Heskett wrote:
> >>> On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:
>  On 2017-11-11 01:49, Jon LaBadie wrote:
> > Just a thought.  My amanda server has seven hard drives
> > dedicated to saving amanda data.  Only 2 are typically
> > used (holding and one vtape drive) during an amdump run.
> > Even then, the usage is only for about 3 hours.
> >
> > So there is a lot of electricity and disk drive wear for
> > inactive drives.
> >
> > Can todays drives be unmounted and powered down then
> > when needed, powered up and mounted again?
> >
> > I'm not talking about system hibernation, the system
> > and its other drives still need to be active.
> >
> > Back when 300GB was a big drive I had 2 of them in
> > external USB housings.  They shut themselves down
> > on inactivity.  When later accessed, there would
> > be about 5-10 seconds delay while the drive spun
> > up and things proceeded normally.
> >
> > That would be a fine arrangement now if it could
> > be mimiced.
> 
>  Aside from what Stefan mentioned (using hdparam to set the
>  standby timeout, check the man page for hdparam as the numbers
>  are not exactly sensible), you may consider looking into
>  auto-mounting each of the drives, as that can help eliminate
>  things that would keep the drives on-line (or make it more
>  obvious that something is still using them).
> >>>
> >>> I've investigated that, and I have amanda wrapped up in a script
> >>> that could do that, but ran into a showstopper I've long since
> >>> forgotten about.  Al this was back in the time I was writing that
> >>> wrapper, years ago now. One of the show stoppers AIR was the fact
> >>> that only root can mount and unmount a drive, and my script runs
> >>> as amanda.
> >>
> >> While such a wrapper might work if you use sudo inside it (you can
> >> configure sudo to allow root to run things as the amanda user
> >> without needing a password, then run the wrapper as root), what I
> >> was trying to refer to in a system-agnostic manner (since the exact
> >> mechanism is different between different UNIX derivatives) was
> >> on-demand auto-mounting, as provided by autofs on Linux or the
> >> auto-mount daemon (amd) on BSD.  When doing on-demand
> >> auto-mounting, you don't need a wrapper at all, as the access
> >> attempt will trigger the mount, and then the mount will time out
> >> after some period of inactivity and be unmounted again.  It's
> >> mostly used for network resources (possibly with special
> >> auto-lookup mechanisms), as certain protocols (NFS in particular)
> >> tend to have issues if the server goes down while a share is
> >> mounted remotely, even if nothing is happening on that share, but
> >> it works just as well for auto-mounting of local fixed or removable
> >> volumes that aren't needed all the time (I use it for a handful of
> >> things on my personal systems to minimize idle resource usage).
> >
> > Sounds good perhaps. I am currently up to my eyeballs in an
> > unrelated problem, and I won't get to this again until that project
> > is completed and I have brought the 2TB drive in and configured it
> > for amanda's usage. That will tend to enforce my one thing at a time
> > but do it right bent. :)  What I have is working for a loose
> > definition of working...
>
> Yeah, I know what that's like.  Prior to switching to amanda where I
> worked, we had a home-grown backup system that had all kinds of odd
> edge cases I had to make sure never happened.  I'm extremely glad we
> decided to stop using that, since it means I can now focus on more
> interesting problems (in theory at least, we're having an issue with
> our Amanda config right now too, but thankfully it's not a huge one).
>
> > But if I allow the 2TB to be  unmounted and self-powered down, once
> > daily, what shortening of its life would I be subjected to?  In
> > other words, how many start-stop cycles can it survive?
>
> It's hard to be certain.  For what it's worth though, you might want
> to test this to be certain that it's actually going to save you
> energy.  It takes a lot of power to get the platters up to speed, but
> it doesn't take much to keep them running at that speed.  It might be
> more advantageous to just configure the device to idle (that is, park
> the heads) after some time out and leave the platters spinning instead
> of spinning down completely (and it should result in less wear on the
> spindle motor).
>
> > Interesting, I had started a long time test yesterday, and the
> > reported hours has wrapped in the report, apparently at 65636 hours.
> > Somebody apparently didn't expect a drive to last that long? ;-) 
> > The drive? Healthy as 

Re: power down hard drives

[Austin S. Hemmelgarn]

On 2017-11-13 11:11, Gene Heskett wrote:

On Monday 13 November 2017 10:12:47 Austin S. Hemmelgarn wrote:


On 2017-11-13 09:56, Gene Heskett wrote:

On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:

On 2017-11-11 01:49, Jon LaBadie wrote:

Just a thought.  My amanda server has seven hard drives
dedicated to saving amanda data.  Only 2 are typically
used (holding and one vtape drive) during an amdump run.
Even then, the usage is only for about 3 hours.

So there is a lot of electricity and disk drive wear for
inactive drives.

Can todays drives be unmounted and powered down then
when needed, powered up and mounted again?

I'm not talking about system hibernation, the system
and its other drives still need to be active.

Back when 300GB was a big drive I had 2 of them in
external USB housings.  They shut themselves down
on inactivity.  When later accessed, there would
be about 5-10 seconds delay while the drive spun
up and things proceeded normally.

That would be a fine arrangement now if it could
be mimiced.


Aside from what Stefan mentioned (using hdparam to set the standby
timeout, check the man page for hdparam as the numbers are not
exactly sensible), you may consider looking into auto-mounting each
of the drives, as that can help eliminate things that would keep
the drives on-line (or make it more obvious that something is still
using them).


I've investigated that, and I have amanda wrapped up in a script
that could do that, but ran into a showstopper I've long since
forgotten about.  Al this was back in the time I was writing that
wrapper, years ago now. One of the show stoppers AIR was the fact
that only root can mount and unmount a drive, and my script runs as
amanda.


While such a wrapper might work if you use sudo inside it (you can
configure sudo to allow root to run things as the amanda user without
needing a password, then run the wrapper as root), what I was trying
to refer to in a system-agnostic manner (since the exact mechanism is
different between different UNIX derivatives) was on-demand
auto-mounting, as provided by autofs on Linux or the auto-mount daemon
(amd) on BSD.  When doing on-demand auto-mounting, you don't need a
wrapper at all, as the access attempt will trigger the mount, and then
the mount will time out after some period of inactivity and be
unmounted again.  It's mostly used for network resources (possibly
with special auto-lookup mechanisms), as certain protocols (NFS in
particular) tend to have issues if the server goes down while a share
is mounted remotely, even if nothing is happening on that share, but
it works just as well for auto-mounting of local fixed or removable
volumes that aren't needed all the time (I use it for a handful of
things on my personal systems to minimize idle resource usage).


Sounds good perhaps. I am currently up to my eyeballs in an unrelated
problem, and I won't get to this again until that project is completed
and I have brought the 2TB drive in and configured it for amanda's
usage. That will tend to enforce my one thing at a time but do it right
bent. :)  What I have is working for a loose definition of working...
Yeah, I know what that's like.  Prior to switching to amanda where I 
worked, we had a home-grown backup system that had all kinds of odd edge 
cases I had to make sure never happened.  I'm extremely glad we decided 
to stop using that, since it means I can now focus on more interesting 
problems (in theory at least, we're having an issue with our Amanda 
config right now too, but thankfully it's not a huge one).


But if I allow the 2TB to be  unmounted and self-powered down, once
daily, what shortening of its life would I be subjected to?  In other
words, how many start-stop cycles can it survive?
It's hard to be certain.  For what it's worth though, you might want to 
test this to be certain that it's actually going to save you energy.  It 
takes a lot of power to get the platters up to speed, but it doesn't 
take much to keep them running at that speed.  It might be more 
advantageous to just configure the device to idle (that is, park the 
heads) after some time out and leave the platters spinning instead of 
spinning down completely (and it should result in less wear on the 
spindle motor).


Interesting, I had started a long time test yesterday, and the reported
hours has wrapped in the report, apparently at 65636 hours. Somebody
apparently didn't expect a drive to last that long? ;-)  The drive?
Healthy as can be.
That's about 7.48 years, so I can actually somewhat understand not going 
past 16-bits for that since most people don't use a given disk for more 
than about 5 years worth of power-on time before replacing it.  However, 
what matters is really not how long the device has been powered on, but 
how much abuse the drive has taken.  Running 24/7 for 5 years with no 
movement of the system (including nothing like earthquakes), in a 
temperature, humidity, and pressure controlled room will get 

Re: power down hard drives

[Austin S. Hemmelgarn]

On 2017-11-13 09:56, Gene Heskett wrote:

On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:


On 2017-11-11 01:49, Jon LaBadie wrote:

Just a thought.  My amanda server has seven hard drives
dedicated to saving amanda data.  Only 2 are typically
used (holding and one vtape drive) during an amdump run.
Even then, the usage is only for about 3 hours.

So there is a lot of electricity and disk drive wear for
inactive drives.

Can todays drives be unmounted and powered down then
when needed, powered up and mounted again?

I'm not talking about system hibernation, the system
and its other drives still need to be active.

Back when 300GB was a big drive I had 2 of them in
external USB housings.  They shut themselves down
on inactivity.  When later accessed, there would
be about 5-10 seconds delay while the drive spun
up and things proceeded normally.

That would be a fine arrangement now if it could
be mimiced.


Aside from what Stefan mentioned (using hdparam to set the standby
timeout, check the man page for hdparam as the numbers are not exactly
sensible), you may consider looking into auto-mounting each of the
drives, as that can help eliminate things that would keep the drives
on-line (or make it more obvious that something is still using them).


I've investigated that, and I have amanda wrapped up in a script that
could do that, but ran into a showstopper I've long since forgotten
about.  Al this was back in the time I was writing that wrapper, years
ago now. One of the show stoppers AIR was the fact that only root can
mount and unmount a drive, and my script runs as amanda.

While such a wrapper might work if you use sudo inside it (you can 
configure sudo to allow root to run things as the amanda user without 
needing a password, then run the wrapper as root), what I was trying to 
refer to in a system-agnostic manner (since the exact mechanism is 
different between different UNIX derivatives) was on-demand 
auto-mounting, as provided by autofs on Linux or the auto-mount daemon 
(amd) on BSD.  When doing on-demand auto-mounting, you don't need a 
wrapper at all, as the access attempt will trigger the mount, and then 
the mount will time out after some period of inactivity and be unmounted 
again.  It's mostly used for network resources (possibly with special 
auto-lookup mechanisms), as certain protocols (NFS in particular) tend 
to have issues if the server goes down while a share is mounted 
remotely, even if nothing is happening on that share, but it works just 
as well for auto-mounting of local fixed or removable volumes that 
aren't needed all the time (I use it for a handful of things on my 
personal systems to minimize idle resource usage).

Re: power down hard drives

[Gene Heskett]

On Monday 13 November 2017 07:19:45 Austin S. Hemmelgarn wrote:

> On 2017-11-11 01:49, Jon LaBadie wrote:
> > Just a thought.  My amanda server has seven hard drives
> > dedicated to saving amanda data.  Only 2 are typically
> > used (holding and one vtape drive) during an amdump run.
> > Even then, the usage is only for about 3 hours.
> >
> > So there is a lot of electricity and disk drive wear for
> > inactive drives.
> >
> > Can todays drives be unmounted and powered down then
> > when needed, powered up and mounted again?
> >
> > I'm not talking about system hibernation, the system
> > and its other drives still need to be active.
> >
> > Back when 300GB was a big drive I had 2 of them in
> > external USB housings.  They shut themselves down
> > on inactivity.  When later accessed, there would
> > be about 5-10 seconds delay while the drive spun
> > up and things proceeded normally.
> >
> > That would be a fine arrangement now if it could
> > be mimiced.
>
> Aside from what Stefan mentioned (using hdparam to set the standby
> timeout, check the man page for hdparam as the numbers are not exactly
> sensible), you may consider looking into auto-mounting each of the
> drives, as that can help eliminate things that would keep the drives
> on-line (or make it more obvious that something is still using them).

I've investigated that, and I have amanda wrapped up in a script that 
could do that, but ran into a showstopper I've long since forgotten 
about.  Al this was back in the time I was writing that wrapper, years 
ago now. One of the show stoppers AIR was the fact that only root can 
mount and unmount a drive, and my script runs as amanda.

Cheers, Gene Heskett
-- 
"There are four boxes to be used in defense of liberty:
 soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
Genes Web page 

Re: amcheck -m DailySet does not display, sends mail only

[Jean-Louis Martineau]

The -m flag should print nothing on stdout
Do not use the -m flag if you want something displayed.

Jean-Louis

On 12/11/17 11:19 PM, Jobst Schmalenbach wrote:
> Hi
>
> running amcheck-3.4.1, server and clients all run same software (3.4.1) and 
> only linux stations.
>
> On the tape server when running and logged in as amanda user
>
>/usr/local/sbin/amcheck -m DailySet1
>
> or (just to be sure)
>
>/usr/local/sbin/amcheck -cs -m DailySet1
>
> nothing is displayed, only mail is send.
>
> This used to work, but has stopped.
>
>
> Jobst
>
>
>
>
>
This message is the property of CARBONITE, INC. and may contain confidential or 
privileged information.
If this message has been delivered to you by mistake, then do not copy or 
deliver this message to anyone.  Instead, destroy it and notify me by reply 
e-mail

Re: Odd non-fatal errors in amdump reports.

[Austin S. Hemmelgarn]

On 2017-11-10 12:52, Jean-Louis Martineau wrote:

The previous patch broke something.
Try this new set2-r2.diff patch
Given that the switch to NFSv4 combined with a change to the labeling 
scheme fixed the other issue, I'm going to re-test these two sets with 
the same changes before I test the patch just so I've got something 
current to compare against.  I should have results from that later 
today, and will likely be testing this patch tomorrow if things aren't 
resolved by the other changes (and based on what you've said and what 
I've seen, I don't think the switch to NFSv4 or the labeling change will 
fix this one).


Jean-Louis

On 10/11/17 10:40 AM, Austin S. Hemmelgarn wrote:
 > On 2017-11-10 08:27, Jean-Louis Martineau wrote:
 >> On 10/11/17 07:57 AM, Austin S. Hemmelgarn wrote:
 >>> On 2017-11-08 08:03, Jean-Louis Martineau wrote:
  On 07/11/17 02:58 PM, Austin S. Hemmelgarn wrote:
  > On 2017-11-07 10:22, Jean-Louis Martineau wrote:
  >> Austin,
  >>
  >> It's hard to say something with only the error message.
  >>
  >> Can you post the amdump. and log..0 for
  the 2
  >> backup set that fail.
  >>
  > I've attached the files (I would put them inline, but one of the
  sets
  > has over 100 DLE's, so the amdump file is huge, and the others are
  > still over 100k each, and I figured nobody want's to try and wad
  > through those in-line).
  >
  > The set1 and set2 files are for the two backup sets that show the
  > header mismatch error, and the set3 files are for the one that
  claims
  > failures in the dump summary.
 
 
  I looked at set3, the error in the 'DUMP SUMMARY' are related to the
  error in the 'FAILURE DUMP SUMMARY'
 
  client2 /boot lev 0 FLUSH [File 0 not found]
  client3 /boot lev 0 FLUSH [File 0 not found]
  client7 /boot lev 0 FLUSH [File 0 not found]
  client8 /boot lev 0 FLUSH [File 0 not found]
  client0 /boot lev 0 FLUSH [File 0 not found]
  client9 /boot lev 0 FLUSH [File 0 not found]
  client9 /srv lev 0 FLUSH [File 0 not found]
  client9 /var lev 0 FLUSH [File 0 not found]
  server0 /boot lev 0 FLUSH [File 0 not found]
  client10 /boot lev 0 FLUSH [File 0 not found]
  client11 /boot lev 0 FLUSH [File 0 not found]
  client12 /boot lev 0 FLUSH [File 0 not found]
 
  They are VAULT attemp, not FLUSH, looking only at the first entry, it
  try to vault 'client2 /boot 0 20171024084159' which it expect to
  find on
  tape Server-01. It is an older dump.
 
  Do Server-01 is still there? Did it still contains the dump?
 
 >>> OK, I've done some further investigation by tweaking the labeling a
 >>> bit (which actually fixed a purely cosmetic issue we were having),
 >>> but I'm still seeing the same problem that prompted this thread, and
 >>> I can confirm that the dumps are where Amanda is trying to look for
 >>> them, it's just not seeing them for some reason. I hadn't thought
 >>> of this before, but could it have something to do with the virtual
 >>> tape library being auto-mounted over NFS on the backup server?
 >>>
 >> Austin,
 >>
 >> Can you try to see if amfetchdump can restore it?
 >>
 >> * amfetchdump CONFIG client2 /boot 20171024084159
 >>
 > amfetchdump doesn't see it, and neither does amrecover, but the files
 > for the given parts are definitely there (I know for a fact that the
 > dump in question has exactly one part, and the file for that does
 > exist on the virtual tape mentioned in the log file).
 >
 > I'm probably not going to be able to check more on this today, but
 > I'll likely be checking if amrestore and amadmin find can see them.
 >

Re: Odd non-fatal errors in amdump reports.

[Austin S. Hemmelgarn]

On 2017-11-10 08:45, Austin S. Hemmelgarn wrote:

On 2017-11-10 08:27, Jean-Louis Martineau wrote:

On 10/11/17 07:57 AM, Austin S. Hemmelgarn wrote:

On 2017-11-08 08:03, Jean-Louis Martineau wrote:

On 07/11/17 02:58 PM, Austin S. Hemmelgarn wrote:
 > On 2017-11-07 10:22, Jean-Louis Martineau wrote:
 >> Austin,
 >>
 >> It's hard to say something with only the error message.
 >>
 >> Can you post the amdump. and log..0 for the 2
 >> backup set that fail.
 >>
 > I've attached the files (I would put them inline, but one of the 
sets

 > has over 100 DLE's, so the amdump file is huge, and the others are
 > still over 100k each, and I figured nobody want's to try and wad
 > through those in-line).
 >
 > The set1 and set2 files are for the two backup sets that show the
 > header mismatch error, and the set3 files are for the one that 
claims

 > failures in the dump summary.


I looked at set3, the error in the 'DUMP SUMMARY' are related to the
error in the 'FAILURE DUMP SUMMARY'

client2 /boot lev 0 FLUSH [File 0 not found]
client3 /boot lev 0 FLUSH [File 0 not found]
client7 /boot lev 0 FLUSH [File 0 not found]
client8 /boot lev 0 FLUSH [File 0 not found]
client0 /boot lev 0 FLUSH [File 0 not found]
client9 /boot lev 0 FLUSH [File 0 not found]
client9 /srv lev 0 FLUSH [File 0 not found]
client9 /var lev 0 FLUSH [File 0 not found]
server0 /boot lev 0 FLUSH [File 0 not found]
client10 /boot lev 0 FLUSH [File 0 not found]
client11 /boot lev 0 FLUSH [File 0 not found]
client12 /boot lev 0 FLUSH [File 0 not found]

They are VAULT attemp, not FLUSH, looking only at the first entry, it
try to vault 'client2 /boot 0 20171024084159' which it expect to 
find on

tape Server-01. It is an older dump.

Do Server-01 is still there? Did it still contains the dump?

OK, I've done some further investigation by tweaking the labeling a 
bit (which actually fixed a purely cosmetic issue we were having), 
but I'm still seeing the same problem that prompted this thread, and 
I can confirm that the dumps are where Amanda is trying to look for 
them, it's just not seeing them for some reason.  I hadn't thought of 
this before, but could it have something to do with the virtual tape 
library being auto-mounted over NFS on the backup server?



Austin,

Can you try to see if amfetchdump can restore it?

  * amfetchdump CONFIG client2 /boot 20171024084159
At the moment, I'm re-testing things after tweaking some NFS parameters 
for the virtual tape library (apparently the FreeNAS server that's 
actually storing the data didn't have NFSv4 turned on, so it was mounted 
with NFSv3, which we've had issues with before on our network), so I 
can't exactly check immediately, but assuming the problem repeats, I'll 
do that first thing once the test dump is done.


It looks like the combination of fixing the incorrect labeling in the 
config and switching to NFSv4 fixed this particular case.

Re: power down hard drives

[Austin S. Hemmelgarn]

On 2017-11-11 01:49, Jon LaBadie wrote:

Just a thought.  My amanda server has seven hard drives
dedicated to saving amanda data.  Only 2 are typically
used (holding and one vtape drive) during an amdump run.
Even then, the usage is only for about 3 hours.

So there is a lot of electricity and disk drive wear for
inactive drives.

Can todays drives be unmounted and powered down then
when needed, powered up and mounted again?

I'm not talking about system hibernation, the system
and its other drives still need to be active.

Back when 300GB was a big drive I had 2 of them in
external USB housings.  They shut themselves down
on inactivity.  When later accessed, there would
be about 5-10 seconds delay while the drive spun
up and things proceeded normally.

That would be a fine arrangement now if it could
be mimiced.
Aside from what Stefan mentioned (using hdparam to set the standby 
timeout, check the man page for hdparam as the numbers are not exactly 
sensible), you may consider looking into auto-mounting each of the 
drives, as that can help eliminate things that would keep the drives 
on-line (or make it more obvious that something is still using them).