subject:"\[gentoo\-user\] Want to seriously test a NEW hard drive"

Yohan Pereira wrote:
 On 08/08/12 at 10:22pm, Dale wrote:
 I used something called Gparted to partition this monster.  This is
 temporary tho.  What should I be using to partition this thing?  What is
 the best, and easiest, tool for me to use?  I have been using cfdisk in
 the past but it doesn't seem to work on this one.  I do want one very
 large partition and plan to use LVM on it too. 

 Oh, it would be nice if the tool is on LiveCDs, SystemRescue in my
 case.  I use that when the stuff hits the fan and I am covered up pretty
 deep.  ;-) 
  
 GParted is on systemrescuecd :D. Its that goldenish disk icon on the
 panel. GParted is a frontend to parted you can use that directly as
 well.



It turned out that my problem was getting it to align properly.  It
seems that GParted does that pretty well.  I guess if I am without a GUI
thingy, I can use parted. 

I couldn't get cgdisk to work right tho.  Heck, it took me a bit to get
GParted to sort it out and it is supposed to be the easy way.  lol 
Sometimes new toys cause me grief.  ;-) 

Since I like cfdisk, are there any tricks to using cgdisk and getting it
to align it correctly?  The drive is a Seagate ST3000DM001 3Tb drive. 
I think this is the key bit:

Logical  Sector size:   512 bytes
Physical Sector size:  4096 bytes

Just not sure where cgdisk wants me to put that info. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-09 Thread Neil Bothwick

On Wed, 08 Aug 2012 22:22:25 -0500, Dale wrote:

 I used something called Gparted to partition this monster.  This is
 temporary tho.  What should I be using to partition this thing?  What is
 the best, and easiest, tool for me to use?  I have been using cfdisk in
 the past but it doesn't seem to work on this one.

Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
cgdisk with GPT partition tables. Create a 1MB partition at the start of
type EF02, which keeps backward compatibility, then partition as normal.

Note that GPT has none of the primary/logical crap, just create the
partitions you want.


-- 
Neil Bothwick

Downloading - A quick way of catching a virus from anywhere in the world.


signature.asc
Description: PGP signature

Re: [gentoo-user] Want to seriously test a NEW hard drive

Neil Bothwick wrote:
 On Wed, 08 Aug 2012 22:22:25 -0500, Dale wrote:

 I used something called Gparted to partition this monster.  This is
 temporary tho.  What should I be using to partition this thing?  What is
 the best, and easiest, tool for me to use?  I have been using cfdisk in
 the past but it doesn't seem to work on this one.
 Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
 cgdisk with GPT partition tables. Create a 1MB partition at the start of
 type EF02, which keeps backward compatibility, then partition as normal.

 Note that GPT has none of the primary/logical crap, just create the
 partitions you want.



A, so that 1Mb partition is what does the alignment thingy?  Is it
the same on all large drives or does it vary a bit based on size?

Thanks.

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-09 Thread Neil Bothwick

On Thu, 09 Aug 2012 04:35:15 -0500, Dale wrote:

  Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
  cgdisk with GPT partition tables. Create a 1MB partition at the start
  of type EF02, which keeps backward compatibility, then partition as
  normal.
 
  Note that GPT has none of the primary/logical crap, just create the
  partitions you want.
 
   
 
 A, so that 1Mb partition is what does the alignment thingy?  Is it
 the same on all large drives or does it vary a bit based on size?

No, gdisk/cgdisk does the alignment. The 1MB GPT boot partition give
backward compatibility with DOS booting. The size doesn't vary, I've used
the same on drives from 120GB to 3TB.


-- 
Neil Bothwick

the sum of all human intelligence is constant, only the number of humans
increases.


signature.asc
Description: PGP signature

Re: [gentoo-user] Want to seriously test a NEW hard drive

Neil Bothwick wrote:
 On Thu, 09 Aug 2012 04:35:15 -0500, Dale wrote:

 Do not use cfdisk on 2TB+ drives, it gives bad alignment. Use gdisk or
 cgdisk with GPT partition tables. Create a 1MB partition at the start
 of type EF02, which keeps backward compatibility, then partition as
 normal.

 Note that GPT has none of the primary/logical crap, just create the
 partitions you want.

  
 A, so that 1Mb partition is what does the alignment thingy?  Is it
 the same on all large drives or does it vary a bit based on size?
 No, gdisk/cgdisk does the alignment. The 1MB GPT boot partition give
 backward compatibility with DOS booting. The size doesn't vary, I've used
 the same on drives from 120GB to 3TB.




Hmmm, then I guess I didn't do something right the first time I tried
cgdisk then.  It said it wasn't aligned right but I couldn't figure out
how to get it to do it.  Then I used Gparted and it seemed to do it
without me telling it anything.  The only difference I saw was the
little 1Mb partition. 

Well, I plan to dd the thing and start over with LVM and all so I'll see
what it does next time.  Maybe I just didn't do something right.  Since
I have nothing windoze on here, do I really need to worry about DOS
booting?  All I have is Gentoo here. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-09 Thread Alex Schuster

Dale writes:

 I have seen where people use dd to do this sort of thing to.  I read
 somewhere that if you do a dd and put in all 1's, then all 0's then back
 again that it is very hard to get any data back off the drive.  I think
 if you do it like over a dozen times, it is deemed impossible to get
 anything back.  I think that is the Government standard of it's gone. 

There's no need for multiple passes of dd with different values.

http://www.h-online.com/security/news/item/Secure-deletion-a-single-overwrite-will-do-it-739699.html

Wonko

Re: [gentoo-user] Want to seriously test a NEW hard drive

Alex Schuster wrote:
 Dale writes:

 I have seen where people use dd to do this sort of thing to.  I read
 somewhere that if you do a dd and put in all 1's, then all 0's then back
 again that it is very hard to get any data back off the drive.  I think
 if you do it like over a dozen times, it is deemed impossible to get
 anything back.  I think that is the Government standard of it's gone. 
 There's no need for multiple passes of dd with different values.

 http://www.h-online.com/security/news/item/Secure-deletion-a-single-overwrite-will-do-it-739699.html

   Wonko




I wonder what some Government org like NSA would think about this?  Then
again, they may want us to believe this so they can get stuff back. 
;-)  ;-) 

That said, I always wondered how something can be there when it is
erased.  On paper, I can see that because it made a physical change to
the paper but on magnetic media, it is magnetic not physical. 

Anyway. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-09 Thread Alan McKinnon

On Thu, 09 Aug 2012 08:53:27 -0500
Dale rdalek1...@gmail.com wrote:

Alex Schuster wrote:
Dale writes:

I have seen where people use dd to do this sort of thing to. I
read somewhere that if you do a dd and put in all 1's, then all
0's then back again that it is very hard to get any data back off
the drive. I think if you do it like over a dozen times, it is
deemed impossible to get anything back. I think that is the
Government standard of it's gone.
There's no need for multiple passes of dd with different values.

http://www.h-online.com/security/news/item/Secure-deletion-a-single-overwrite-will-do-it-739699.html

Wonko

I wonder what some Government org like NSA would think about this?
Then again, they may want us to believe this so they can get stuff
back. ;-) ;-)

That said, I always wondered how something can be there when it is
erased. On paper, I can see that because it made a physical change to
the paper but on magnetic media, it is magnetic not physical.

It's quite simple once you understand how disks work.

In textbooks we need to keep things simple, so we say things like the
magnetic particles are all aligned this way for a 0 and that way for a
1. This gets the concept across but it also let's people believe that
bits on a disk are very much binary - like a light switch or a
transistor they are either on or off.

In practice, nothing could be further from the truth.

With disk magnetic media, you aren't dealing with a single isolated
thing (such as a chunk of disk that can only be one way), you are
dealing with a very very large number of magnetic particles that go to
make up one bit. It's how they average out that makes the drive think
it's a 0 or a 1. It all works much like tape cassettes - the head has a
little coil of wire in it and current flows through the coil. It passes
near a magnetic field on the tape, and the current in the coil changes.
Read the amount of change using fancy electronics, and voila! you have
audio. In the case of disks, it's voila! you have a stream of bits.

Disk drives can't afford to be almost right like audio tape though,
they have to be exactly right so the drive has some amazing maths built
into it for error-correction and redundancy. I believe something like
40% of the space containing raw data is pure error checking (so your 3T
drive is actually 4.1T, but you will only ever get to use 3T)

The trick is, when you overwrite an area of the disk, you don't erase
everything, there are some traces left behind.

Pencil and paper is a good analogy. Write something in pencil. Erase
wit with a rubber eraser, and write something else in the same space.
Now hold the paper up to the light and if you know how to look you can
see the indents in the paper from the first thing you wrote. Train your
eyes to ignore what's written there now and only look at dented paper
with no lead marks, and you can read things quite clearly. James Bond
was especially good at this but that's a movie so real life isn't
*that* quick.

Sekrit magic disk software does a very similar thing - it ignores the
current data and looks for traces left behind from the last write, and
the ones before that.

This trick isn't universal of course. As drive technology advances and
IBM figures out new ways to do it, they come up with ideas like using
the _depth_ of the magnetic material too. Neat trick - you can double
the data stored in the same surface area. With each technology advance,
things change a lot, so the amount of reading backwards you can do is
always changing and depends very much on exactly what drive you have.

--
Alan McKinnon
alan.mckin...@gmail.com

Re: [gentoo-user] Want to seriously test a NEW hard drive

William Kenworthy wrote:
 On Tue, 2012-08-07 at 23:18 -0500, Dale wrote:
 William Kenworthy wrote:
 On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
 Paul Hartman wrote:
 On Tue, Aug 7, 2012 at 5:26 PM, Dale rdalek1...@gmail.com wrote:
 ...
 Goggle have a well known document
 (http://research.google.com/archive/disk_failures.pdf) where they
 analysed hard drive failures for a very large number of drives ... the
 basic upshot is that a very large portion of failures happen with no
 pre-warning, so testing a drive like you are proposing not going to
 prove a thing.

 They also found that smart (is quite dumb) and its tests were of little
 use.

 And high temperatures and work loads were also not a reliable guide to
 trends in failure rates, both of which which surprised me.

 Some of those bathtub curves that I was trained on when setting
 maintenance schedules dont hold water here!

 This anaysis of the paper looks quite good if you want the lite view:
 http://storagemojo.com/2007/02/19/googles-disk-failure-experience/

 BillK


 Well, I am going by actual real experiences from other users of this
 model of drive.  I don't know what google was testing but I would bet it
 is not the drive model I just bought.  The users who bought this exact
 model drive report that most failures are either out of the box or
 within a few weeks to a month.  I'm just going to try to increase my
 odds even if it is just a little bit. 

 Smart may not always predict a failure but it is better than nothing at
 all.  Would you rather have a tool that may predict a failure or no tool
 at all?  Me, I'd rather have something that at least tries too.  The one
 drive I had to go bad, Smart predicted it very well.  It said I had
 about 24 hrs to get my stuff off.  Sure enough, the next day, it
 wouldn't do anything but spin.  Without Smart and its prediction, I'd
 have lost the data on the drive with no warning at all. 

 A couple questions.  What if while I am testing this drive, it dies? 
 Does that prove that my testing benefited me then? 

 Dale

 :-)  :-) 

 Read the paper - its written by someone who buys drives in batches of
 100's+, not by a few guys posting on a forum somewhere who bought one
 random drive, who probably didn't use anti-static techniques handling
 the drive, and thumped it around in the boot of the car or got it via
 the courier who was famous for delivering TV's by throwing them over the
 fence.  It is a bit of an eye opener - read it.

 My impression of models is that it is not really the model that has a
 run of failures, but the batch so a different run of the same model will
 have a different failure pattern.  There are exceptions such as those
 IBM 60G deathstar drives, but then again they fixed it and following
 drives of the same model were fine.

 My own experience of smart is it tells you something, but what it seems
 to say is not right (notice I am not saying it tells lies, but that the
 data and interpretation don't make sense on the drives Ive had)

 Drive failure does seem to be a semi-random lottery, but I am seriously
 doubtful that testing will do anything ... it has as much chance of
 precipitating failure that wouldn't occur otherwise because you are
 seriously hammering it, or weakening the drive so it will fail at some
 random time, but perhaps weeks away rather than the years it otherwise
 would, or nothing will happen except for wasted electrons.  

 Then again, I am of the view that modern electronics is
 designed/programmed to fail a few seconds past warranty expiry (why else
 do most devices have timekeeping built in :)

 BillK


Actually, I read the paper a long time ago.  May give it another look
but I'm still going by what people have posted about this specific
model.  If they make them all the same, then testing to at least see if
it is going to get past the initial stages is a good idea.  I do think
some failures were because of the BIOS and I stated that in my original
post.  Getting a DOA drive can happen but when there are mobos around
that can't see large drives, then one has to consider it.  The ones I
worry about are the ones that worked for a few weeks or a month then
died.  They obviously don't have BIOS issues but some other problem. 

Still, all things considered, I'm going to test the drive.  If it can
pass the test then I will feel better about putting my data on it.  As
for the paper:

root@fireball / # ls -al /home/dale/Desktop/disk_failures.pdf
-rw-r--r-- 1 dale users 247492 May 21 17:58
/home/dale/Desktop/disk_failures.pdf
root@fireball / #


I read it back in May.  It's still sitting on my desktop.  I might also
add, it is about 5 years old.  Drives have changed since then.  For one,
they have gotten larger.  We don't know what else may have changed either. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-08 Thread Peter Humphrey

On Wednesday 08 August 2012 01:32:55 Mark Knecht wrote:
 On Tue, Aug 7, 2012 at 4:06 PM, Peter Humphrey 
pe...@humphrey.ukfsn.org wrote:
  On Tuesday 07 August 2012 23:39:05 Mark Knecht wrote:
  Dead out of the box is dead. However a drive failing in a couple
  of months _might_ have showed up in the smartctl output ...
  
  I wonder. Does anyone here know what most often causes HD failure
  after a couple of months? I imagine it's some component whose
  material is substandard, in which case it might show up with
  smartclt or it might not. I dunno.
  
  --
  Rgds
  Peter
 
 This has come up before on this list I think.

I must have missed it then.

 This is the overarching sort of thinking about failure over
 lifetime...
 
 http://en.wikipedia.org/wiki/Bathtub_curve

Yes, I'm familiar with that idea thanks. I was wondering what specific 
causes were most implicated.

-- 
Rgds
Peter

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-08 Thread Mick

On Wednesday 08 Aug 2012 05:21:31 Adam Carter wrote:
  To wipe a drive use dban. - live CD which uses (US) gov approved
  standards of wipe methods/patterns.
 
 Or shred, which comes with coreutils.
 
  dd is only going to show sectors on a failed drive - too late!
  
  To explain, modern drives have a store of locations they can use to
  transparently replace any failed locations (apparently similar to the
  way SSD's do it)  - the internal drive electronics handle this and its
  not visible externally though smart data seems to show it, but as google
  says, smart is a bit suspect.  The problem of a bad sector will only
  show once all the reserved locations are used up, by which time the
  drive is usually in rampant failure.
  
  I do suspect this is one reason for googles results - actual failures of
  the media (as against the motors/electronics are much as they always
  have been, but the drives are not reporting them until its too late.
 
 Ahh - go to know. My reasoning assumed that smart reports all remaps.

May be it does, but I understand that dd or shred won't overwrite them, or any 
bad blocks.  You'll need the hdparm ATA secure erase (or enhanced secure 
erase) feature for that.

BTW, Dale make sure that you plug the drive in a SATA controller for running 
the hdparm erase function.  It has been reported that doing this using a USB 
port will brick the drive!
-- 
Regards,
Mick


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Want to seriously test a NEW hard drive

Mick wrote:
 On Wednesday 08 Aug 2012 05:21:31 Adam Carter wrote:
 To wipe a drive use dban. - live CD which uses (US) gov approved
 standards of wipe methods/patterns.
 Or shred, which comes with coreutils.

 dd is only going to show sectors on a failed drive - too late!

 To explain, modern drives have a store of locations they can use to
 transparently replace any failed locations (apparently similar to the
 way SSD's do it)  - the internal drive electronics handle this and its
 not visible externally though smart data seems to show it, but as google
 says, smart is a bit suspect.  The problem of a bad sector will only
 show once all the reserved locations are used up, by which time the
 drive is usually in rampant failure.

 I do suspect this is one reason for googles results - actual failures of
 the media (as against the motors/electronics are much as they always
 have been, but the drives are not reporting them until its too late.
 Ahh - go to know. My reasoning assumed that smart reports all remaps.
 May be it does, but I understand that dd or shred won't overwrite them, or 
 any 
 bad blocks.  You'll need the hdparm ATA secure erase (or enhanced secure 
 erase) feature for that.

 BTW, Dale make sure that you plug the drive in a SATA controller for running 
 the hdparm erase function.  It has been reported that doing this using a USB 
 port will brick the drive!


I don't use USB for drives, except the USB stick thingys. 

I have a question sort of related to this.  Anyone can share info.  I
see boxes that hold drives in them and connect via ethernet or something
like that.  Do those work really well?  I thought about getting one
someday but I don't know what they do and how they do it EXACTLY.  Is it
like a small puter in there or some other means of getting the data
across? 

Right now, I like having my drives in my Cooler Master case.  The fan
blows right on the drives so they stay nice and cool.  But, I have given
thought to having the non OS drives in one of those little boxes, maybe
using RAID and mirroring the data on two drives. 

Just what magic is in those things? 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-08 Thread Ricardo Jesus

Sent from my Android device.
On Aug 6, 2012 4:22 PM, Paul Hartman paul.hartman+gen...@gmail.com
wrote:

 On Sun, Aug 5, 2012 at 7:45 AM, Dale rdalek1...@gmail.com wrote:
  Howdy,
 
  I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
  have seen some reviews where it would not work right.  I think some of
  it may be BIOS related since some BIOS's don't like drives that large.
  Anyway, I want to test this thing real good to really make sure it is up
  to the task before putting my data on it.  It's going to be so much
  data, there is really no way to do back-ups at this point.  Come on, 2
  to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
  for now, well.
 
  I have heard of bonnie and friends.  I also think dd could do some
  testing too.  Is there any other way to give this a good work and see if
  it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
  never used it before.  Maybe someone has some test that is really brutal.

 I wouldn't want to torture the drive, but just make sure it works.

 What I do: boot parted magic liveCD, run ATA SECURE ERASE on the drive
 (you probably need to suspend and awaken your machine to unlock the
 drive, the GUI tool in the liveCD does this for you automatically),
 that will do factory reformat/low-level reformat, then run SMART full
 test which can take many hours. If it survives both of those, and
 doesn't make audible clicking noises in the process, I feel confident
 that it is in working order.

 Those steps are more important if I'm testing a used or refurbished
 drive, for a new drive you may want to skip the secure erase and only
 do the smart test. If SMART test passes then I don't think there's any
 reason to run badblocks, but you can have it run during mkfs if you
 want reassurance.

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-08 Thread Paul Hartman

On Tue, Aug 7, 2012 at 9:19 PM, Dale rdalek1...@gmail.com wrote:
 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
 late 80's or early 90's, it took that long for those whimpy little 100Mb
 drives.  Ooops, my ages is showing again.  lol

I recently found a box of hard drives in my house and have been using
ddrescue to pull the data off of them. These drives were not that old,
around 1gb each. Desktop drives. I was amazed at how slow they were
relative to today's drives... 2MB/sec? 5MB/sec? My internet connection
is faster than that now.

Also, found a dead 5.25 Quantum hard drive... forgot how huge those
are. Weighed a ton and it was built like a tank.

Your new drive will probably go around 100-150MB/sec or so on
sequential writes. So you can do the math and figure out how many
hours that will take.

Re: [gentoo-user] Want to seriously test a NEW hard drive

Paul Hartman wrote:
 On Tue, Aug 7, 2012 at 9:19 PM, Dale rdalek1...@gmail.com wrote:
 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
 late 80's or early 90's, it took that long for those whimpy little 100Mb
 drives.  Ooops, my ages is showing again.  lol
 I recently found a box of hard drives in my house and have been using
 ddrescue to pull the data off of them. These drives were not that old,
 around 1gb each. Desktop drives. I was amazed at how slow they were
 relative to today's drives... 2MB/sec? 5MB/sec? My internet connection
 is faster than that now.

 Also, found a dead 5.25 Quantum hard drive... forgot how huge those
 are. Weighed a ton and it was built like a tank.

 Your new drive will probably go around 100-150MB/sec or so on
 sequential writes. So you can do the math and figure out how many
 hours that will take.




It won't be that fast.  The drive supports 6Gbs/sec but my mobo is only
3Gbs/sec.  I hope to upgrade my mobo at some point.  Then maybe the ram
and CPU.  I been looking at those 8 core CPUs a bit.  The prices are
coming down slowly. 

Anyway, I'll only get the 3Gbs/sec for now. 

I used to have a couple of those really old 14 inch hard drives.  I
think I sold them for scrap a few years ago.  They were mostly aluminium
if I recall correctly.  They were only a few megabytes but they sure was
big. 

Our age is showing.  lol  I bet folks know I am not a teenager now.  ;-) 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

Dale wrote:
 Howdy,

 I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
 have seen some reviews where it would not work right.  I think some of
 it may be BIOS related since some BIOS's don't like drives that large. 
 Anyway, I want to test this thing real good to really make sure it is up
 to the task before putting my data on it.  It's going to be so much
 data, there is really no way to do back-ups at this point.  Come on, 2
 to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
 for now, well. 

 I have heard of bonnie and friends.  I also think dd could do some
 testing too.  Is there any other way to give this a good work and see if
 it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
 never used it before.  Maybe someone has some test that is really brutal. 

 Thanks

 Dale

 :-)  :-) 



Update.  I went to the mailbox and there was a nice pretty brown box. 
Bad thing is, drive is OEM with just a blister pack around it.  The box
had almost no packing, just a small amount of brown paper.  Eww!  It
was just flopping around in the box. 

Anyway, I stole a SATA cable from another rig and got this:

root@fireball / # smartctl -t long /dev/sdd
smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.5.0-gentoo] (local build)
Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

=== START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
Sending command: Execute SMART Extended self-test routine immediately
in off-line mode.
Drive command Execute SMART Extended self-test routine immediately in
off-line mode successful.
Testing has begun.
Please wait 255 minutes for test to complete.
Test will complete after Wed Aug  8 17:10:46 2012

Use smartctl -X to abort test.
root@fireball / #

255 minutes.  Wow.  My plan, let this test finish, do a little bit of
copy and rm, then test again.  I figure copying 800Gbs to it should give
it a little bit of a work out since I will be doing it twice. 

Yeppie. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-08 Thread Mark Knecht

On Wed, Aug 8, 2012 at 11:02 AM, Dale rdalek1...@gmail.com wrote:
 Dale wrote:
 Howdy,

 I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
 have seen some reviews where it would not work right.  I think some of
 it may be BIOS related since some BIOS's don't like drives that large.
 Anyway, I want to test this thing real good to really make sure it is up
 to the task before putting my data on it.  It's going to be so much
 data, there is really no way to do back-ups at this point.  Come on, 2
 to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
 for now, well.

 I have heard of bonnie and friends.  I also think dd could do some
 testing too.  Is there any other way to give this a good work and see if
 it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
 never used it before.  Maybe someone has some test that is really brutal.

 Thanks

 Dale

 :-)  :-)



 Update.  I went to the mailbox and there was a nice pretty brown box.
 Bad thing is, drive is OEM with just a blister pack around it.  The box
 had almost no packing, just a small amount of brown paper.  Eww!  It
 was just flopping around in the box.

 Anyway, I stole a SATA cable from another rig and got this:

 root@fireball / # smartctl -t long /dev/sdd
 smartctl 5.42 2011-10-20 r3458 [x86_64-linux-3.5.0-gentoo] (local build)
 Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net

 === START OF OFFLINE IMMEDIATE AND SELF-TEST SECTION ===
 Sending command: Execute SMART Extended self-test routine immediately
 in off-line mode.
 Drive command Execute SMART Extended self-test routine immediately in
 off-line mode successful.
 Testing has begun.
 Please wait 255 minutes for test to complete.
 Test will complete after Wed Aug  8 17:10:46 2012

 Use smartctl -X to abort test.
 root@fireball / #

 255 minutes.  Wow.  My plan, let this test finish, do a little bit of
 copy and rm, then test again.  I figure copying 800Gbs to it should give
 it a little bit of a work out since I will be doing it twice.

 Yeppie.

 Dale

 :-)  :-)

 --
 I am only responsible for what I said ... Not for what you understood or how 
 you interpreted my words!



Now, don't you wear it out, you hear?!?!? ;-)

If your machine runs all the time you can add smartctl runs as chron
jobs that run at midnight and have the results emailed to you. Look at
them once a week and other than an unpredictable catastrophic failure
you'll likely know a long time before it fails.

Cheers,
Mark

Re: [gentoo-user] Want to seriously test a NEW hard drive

Mark Knecht wrote:
 Now, don't you wear it out, you hear?!?!? ;-) If your machine runs all
 the time you can add smartctl runs as chron jobs that run at midnight
 and have the results emailed to you. Look at them once a week and
 other than an unpredictable catastrophic failure you'll likely know a
 long time before it fails. Cheers, Mark 

Not going to wear it out but going to make sure it survived the trip
across the country while bouncing around in the box.  :/  Check this out:

Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd, type changed
from 'scsi' to 'sat'
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], opened
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT],
ST3000DM001-9YN166, S/N:Z1F0PKT5, WWN:5-000c50-04d79e15c, FW:CC4C, 3.00 TB
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], not
found in smartd database.
Aug  8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], is SMART
capable. Adding to monitor list.


And:

Aug  8 13:46:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 80% remaining
Aug  8 14:16:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 70% remaining
Aug  8 14:46:48 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 60% remaining
Aug  8 15:16:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 50% remaining
Aug  8 15:46:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 40% remaining
Aug  8 16:16:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 30% remaining
Aug  8 16:46:47 localhost smartd[2085]: Device: /dev/sdd [SAT],
self-test in progress, 20% remaining

It's working on it.  ;-) 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

Dale wrote:
 Not going to wear it out but going to make sure it survived the trip
 across the country while bouncing around in the box. :/ Check this
 out: Aug 8 12:46:47 localhost smartd[2083]: Device: /dev/sdd, type
 changed from 'scsi' to 'sat' Aug 8 12:46:47 localhost smartd[2083]:
 Device: /dev/sdd [SAT], opened Aug 8 12:46:47 localhost smartd[2083]:
 Device: /dev/sdd [SAT], ST3000DM001-9YN166, S/N:Z1F0PKT5,
 WWN:5-000c50-04d79e15c, FW:CC4C, 3.00 TB Aug 8 12:46:47 localhost
 smartd[2083]: Device: /dev/sdd [SAT], not found in smartd database.
 Aug 8 12:46:47 localhost smartd[2083]: Device: /dev/sdd [SAT], is
 SMART capable. Adding to monitor list. And: Aug 8 13:46:47 localhost
 smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 80%
 remaining Aug 8 14:16:47 localhost smartd[2085]: Device: /dev/sdd
 [SAT], self-test in progress, 70% remaining Aug 8 14:46:48 localhost
 smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 60%
 remaining Aug 8 15:16:47 localhost smartd[2085]: Device: /dev/sdd
 [SAT], self-test in progress, 50% remaining Aug 8 15:46:47 localhost
 smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 40%
 remaining Aug 8 16:16:47 localhost smartd[2085]: Device: /dev/sdd
 [SAT], self-test in progress, 30% remaining Aug 8 16:46:47 localhost
 smartd[2085]: Device: /dev/sdd [SAT], self-test in progress, 20%
 remaining It's working on it. ;-) Dale :-) :-) 

OK.  Finished a selftest.  So far, so good.  Now, why didn't someone
remind me that I had to use some special tool to partition this
monster?  It took some doing but I'm not sure I am doing this the right
way.  It works for testing tho. 

I used something called Gparted to partition this monster.  This is
temporary tho.  What should I be using to partition this thing?  What is
the best, and easiest, tool for me to use?  I have been using cfdisk in
the past but it doesn't seem to work on this one.  I do want one very
large partition and plan to use LVM on it too. 

Oh, it would be nice if the tool is on LiveCDs, SystemRescue in my
case.  I use that when the stuff hits the fan and I am covered up pretty
deep.  ;-) 

Thoughts?  Ideas?  Suggestions? 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

Paul Hartman wrote:
 On Sun, Aug 5, 2012 at 7:45 AM, Dale rdalek1...@gmail.com wrote:
 Howdy,

 I finally got me a 3Tb drive on the way.  Should be here Wednesday.  I
 have seen some reviews where it would not work right.  I think some of
 it may be BIOS related since some BIOS's don't like drives that large.
 Anyway, I want to test this thing real good to really make sure it is up
 to the task before putting my data on it.  It's going to be so much
 data, there is really no way to do back-ups at this point.  Come on, 2
 to 3Tbs on 4Gb DVDs.  Really?  lol  Maybe a external drive later on but
 for now, well.

 I have heard of bonnie and friends.  I also think dd could do some
 testing too.  Is there any other way to give this a good work and see if
 it holds up?  Oh, helpful hints with Bonnie would be great too.  I have
 never used it before.  Maybe someone has some test that is really brutal.
 I wouldn't want to torture the drive, but just make sure it works.

 What I do: boot parted magic liveCD, run ATA SECURE ERASE on the drive
 (you probably need to suspend and awaken your machine to unlock the
 drive, the GUI tool in the liveCD does this for you automatically),
 that will do factory reformat/low-level reformat, then run SMART full
 test which can take many hours. If it survives both of those, and
 doesn't make audible clicking noises in the process, I feel confident
 that it is in working order.

 Those steps are more important if I'm testing a used or refurbished
 drive, for a new drive you may want to skip the secure erase and only
 do the smart test. If SMART test passes then I don't think there's any
 reason to run badblocks, but you can have it run during mkfs if you
 want reassurance.



Well, I don't want to force it to fail just for the heck of it.  I just
want to make sure it is not going to fail in the first few months of
use.  The reviews that people have wrote that had problems were
generally either out of the box dead or failed after a few weeks or a
month or so.  I figure if I can run tests that would be about a month or
so of use, then maybe I am OK.  That is my goal.  I just don't want to
get all my data on there and such and then have it go belly up. 

I didn't know you could do low level formats anymore.  Really?  What
package provides that?  Hmmm, I'm thinking about those HOURS spent
formatting a 100Mb drive and then thinking about how long it will take
to do a 3Tb drive.  O_O  I mean really O_O.  LOL 

Dale

:-)  :-) 

P. S.  It should be here tomorrow. UPS dropped it off at the post office
today. 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread Mark Knecht

On Tue, Aug 7, 2012 at 3:26 PM, Dale rdalek1...@gmail.com wrote:
SNIP
 Well, I don't want to force it to fail just for the heck of it.

Of course...

 I just
 want to make sure it is not going to fail in the first few months of
 use.

But what you don't know about those failed drives is whether the user
could have predicted it by watching the smartctl data more closely.

Dead out of the box is dead. However a drive failing in a couple of
months _might_ have showed up in the smartctl output in which case the
user could have transferred data to a new drive.

Keep in mind, that's all you're talking about here. You don't want to
spend the money for a second drive, but you know you'll have to do
_something_ if the drive starts failing. Not having backups isn't a
good plan...

Take care,
Mark

Re: [gentoo-user] Want to seriously test a NEW hard drive

Mark Knecht wrote:
 On Tue, Aug 7, 2012 at 3:26 PM, Dale rdalek1...@gmail.com wrote:
 SNIP
 Well, I don't want to force it to fail just for the heck of it.
 Of course...

 I just
 want to make sure it is not going to fail in the first few months of
 use.
 But what you don't know about those failed drives is whether the user
 could have predicted it by watching the smartctl data more closely.

 Dead out of the box is dead. However a drive failing in a couple of
 months _might_ have showed up in the smartctl output in which case the
 user could have transferred data to a new drive.

 Keep in mind, that's all you're talking about here. You don't want to
 spend the money for a second drive, but you know you'll have to do
 _something_ if the drive starts failing. Not having backups isn't a
 good plan...

 Take care,
 Mark



I realize that not having backups is not good.  Thing is, I really don't
have a way to back up that much data.  Heck, I got about 1Tb that will
be transfered to the new drive when I feel it is ready.  Other than a
second drive, I have no means of backing up that much data.  I thought
about a blue ray burner but even that is a lot of media.  I think I
figured up over a 100.  Plain DVDs are even worse. 

My plan is to watch the logs and buy another drive as soon as I can.  In
the meantime, it is what it is.  I don't like it but I can't change it
either.  If it helps any, I have only had one drive to ever fail on me. 
It was a WD and was pretty old.  It gave me enough warning to get the
data off tho.  We gotta love that. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread Peter Humphrey

On Tuesday 07 August 2012 23:39:05 Mark Knecht wrote:

 Dead out of the box is dead. However a drive failing in a couple of
 months _might_ have showed up in the smartctl output ...

I wonder. Does anyone here know what most often causes HD failure after 
a couple of months? I imagine it's some component whose material is 
substandard, in which case it might show up with smartclt or it might 
not. I dunno.

-- 
Rgds
Peter

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread Paul Hartman

On Tue, Aug 7, 2012 at 5:26 PM, Dale rdalek1...@gmail.com wrote:

 I didn't know you could do low level formats anymore.  Really?  What
 package provides that?  Hmmm, I'm thinking about those HOURS spent
 formatting a 100Mb drive and then thinking about how long it will take
 to do a 3Tb drive.  O_O  I mean really O_O.  LOL

hdparm provides it. Do a search for ATA secure erase or enhanced
secure erase. It is as close as there is to a low-level format in
modern drives. It is basically a erase/format within the drive's
firmware, that resets it all back to factory, including bad sectors,
with the same pattern of 1's and 0's and everything. You can do it
with hdparm but it's tricky and contains many warnings about killing
your drive. It is considered the only true way to properly erase a
hard drive as anything else is just overwriting and does not
necessarily touch all the areas that the firmware can touch. I think
actual implementation of what the secure erases do varies from drive
to drive, but they'll all format the whole disk for sure.

The parted magic live CD contains a GUI tool to automate it and it is
extremely simple to use. Choose your drive and go. On a 2tb drive I
think it took 4 or 5 hours when I ran it. There is absolutely no
feedback while it is running, so you're just waiting with no progress
indicator or anything. You can also do SMART tests from within the
parted magic live CD environment. And of course partitioning. :)

That all being said, when performing this kind of operation I usually
like to use a live CD and unplug ALL OTHER HARD DRIVES except for the
one I'm going to destroy. I don't want to accidentally erase the wrong
drive. (In fact I have an old Pentium 4 computer with no HDDs that I
use solely for the purpose of testing live CDs, testing and formatting
drives, partitioning new drives before i put them into a production
machine)

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread Mark Knecht

On Tue, Aug 7, 2012 at 4:05 PM, Dale rdalek1...@gmail.com wrote:
 In
 the meantime, it is what it is.

I completely understand Dale, and I'm not picking on you. Just get
that second drive as soon as you can.

Cheers,
Mark

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread Mark Knecht

On Tue, Aug 7, 2012 at 4:06 PM, Peter Humphrey pe...@humphrey.ukfsn.org wrote:
 On Tuesday 07 August 2012 23:39:05 Mark Knecht wrote:

 Dead out of the box is dead. However a drive failing in a couple of
 months _might_ have showed up in the smartctl output ...

 I wonder. Does anyone here know what most often causes HD failure after
 a couple of months? I imagine it's some component whose material is
 substandard, in which case it might show up with smartclt or it might
 not. I dunno.

 --
 Rgds
 Peter


This has come up before on this list I think. This is the overarching
sort of thinking about failure over lifetime...

http://en.wikipedia.org/wiki/Bathtub_curve

Re: [gentoo-user] Want to seriously test a NEW hard drive

Paul Hartman wrote:
 On Tue, Aug 7, 2012 at 5:26 PM, Dale rdalek1...@gmail.com wrote:
 I didn't know you could do low level formats anymore.  Really?  What
 package provides that?  Hmmm, I'm thinking about those HOURS spent
 formatting a 100Mb drive and then thinking about how long it will take
 to do a 3Tb drive.  O_O  I mean really O_O.  LOL
 hdparm provides it. Do a search for ATA secure erase or enhanced
 secure erase. It is as close as there is to a low-level format in
 modern drives. It is basically a erase/format within the drive's
 firmware, that resets it all back to factory, including bad sectors,
 with the same pattern of 1's and 0's and everything. You can do it
 with hdparm but it's tricky and contains many warnings about killing
 your drive. It is considered the only true way to properly erase a
 hard drive as anything else is just overwriting and does not
 necessarily touch all the areas that the firmware can touch. I think
 actual implementation of what the secure erases do varies from drive
 to drive, but they'll all format the whole disk for sure.

 The parted magic live CD contains a GUI tool to automate it and it is
 extremely simple to use. Choose your drive and go. On a 2tb drive I
 think it took 4 or 5 hours when I ran it. There is absolutely no
 feedback while it is running, so you're just waiting with no progress
 indicator or anything. You can also do SMART tests from within the
 parted magic live CD environment. And of course partitioning. :)

 That all being said, when performing this kind of operation I usually
 like to use a live CD and unplug ALL OTHER HARD DRIVES except for the
 one I'm going to destroy. I don't want to accidentally erase the wrong
 drive. (In fact I have an old Pentium 4 computer with no HDDs that I
 use solely for the purpose of testing live CDs, testing and formatting
 drives, partitioning new drives before i put them into a production
 machine)




I have seen where people use dd to do this sort of thing to.  I read
somewhere that if you do a dd and put in all 1's, then all 0's then back
again that it is very hard to get any data back off the drive.  I think
if you do it like over a dozen times, it is deemed impossible to get
anything back.  I think that is the Government standard of it's gone. 

4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
late 80's or early 90's, it took that long for those whimpy little 100Mb
drives.  Ooops, my ages is showing again.  lol 

I got to go read up on hdparm.  I already have it installed here.  I'm
not planning to use this part but do want to read up on this. 

Thanks.

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread William Kenworthy

On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
 Paul Hartman wrote:
  On Tue, Aug 7, 2012 at 5:26 PM, Dale rdalek1...@gmail.com wrote:
  I didn't know you could do low level formats anymore.  Really?  What
  package provides that?  Hmmm, I'm thinking about those HOURS spent
  formatting a 100Mb drive and then thinking about how long it will take
  to do a 3Tb drive.  O_O  I mean really O_O.  LOL
  hdparm provides it. Do a search for ATA secure erase or enhanced
  secure erase. It is as close as there is to a low-level format in
...
 
 I have seen where people use dd to do this sort of thing to.  I read
 somewhere that if you do a dd and put in all 1's, then all 0's then back
 again that it is very hard to get any data back off the drive.  I think
 if you do it like over a dozen times, it is deemed impossible to get
 anything back.  I think that is the Government standard of it's gone. 
 
 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
 late 80's or early 90's, it took that long for those whimpy little 100Mb
 drives.  Ooops, my ages is showing again.  lol 
 
 I got to go read up on hdparm.  I already have it installed here.  I'm
 not planning to use this part but do want to read up on this. 
 
 Thanks.
 
 Dale
 
 :-)  :-) 
 


Goggle have a well known document
(http://research.google.com/archive/disk_failures.pdf) where they
analysed hard drive failures for a very large number of drives ... the
basic upshot is that a very large portion of failures happen with no
pre-warning, so testing a drive like you are proposing not going to
prove a thing.

They also found that smart (is quite dumb) and its tests were of little
use.

And high temperatures and work loads were also not a reliable guide to
trends in failure rates, both of which which surprised me.

Some of those bathtub curves that I was trained on when setting
maintenance schedules dont hold water here!

This anaysis of the paper looks quite good if you want the lite view:
http://storagemojo.com/2007/02/19/googles-disk-failure-experience/

BillK



BillK

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread Adam Carter

 I have seen where people use dd to do this sort of thing to.  I read
 somewhere that if you do a dd and put in all 1's, then all 0's then back
 again that it is very hard to get any data back off the drive.  I think
 if you do it like over a dozen times, it is deemed impossible to get
 anything back.  I think that is the Government standard of it's gone.

I've heard the old attacks to recover data from a zerod drive are no
longer viable for disks of greater capacity than about 10G. I haven't
seen the information myself, however.

A single pass using dd would probably a good way of detecting any
existing bad blocks, so a smartctl then dd then smartctl again and a
diff of the results may be interesting.

I just use a 1TB software mirror for my backups.

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread William Kenworthy

On Wed, 2012-08-08 at 14:02 +1000, Adam Carter wrote:
  I have seen where people use dd to do this sort of thing to.  I read
  somewhere that if you do a dd and put in all 1's, then all 0's then back
  again that it is very hard to get any data back off the drive.  I think
  if you do it like over a dozen times, it is deemed impossible to get
  anything back.  I think that is the Government standard of it's gone.
 
 I've heard the old attacks to recover data from a zerod drive are no
 longer viable for disks of greater capacity than about 10G. I haven't
 seen the information myself, however.
 
 A single pass using dd would probably a good way of detecting any
 existing bad blocks, so a smartctl then dd then smartctl again and a
 diff of the results may be interesting.
 
 I just use a 1TB software mirror for my backups.
 

To wipe a drive use dban. - live CD which uses (US) gov approved
standards of wipe methods/patterns.

dd is only going to show sectors on a failed drive - too late!

To explain, modern drives have a store of locations they can use to
transparently replace any failed locations (apparently similar to the
way SSD's do it)  - the internal drive electronics handle this and its
not visible externally though smart data seems to show it, but as google
says, smart is a bit suspect.  The problem of a bad sector will only
show once all the reserved locations are used up, by which time the
drive is usually in rampant failure.

I do suspect this is one reason for googles results - actual failures of
the media (as against the motors/electronics are much as they always
have been, but the drives are not reporting them until its too late.

BillK

Re: [gentoo-user] Want to seriously test a NEW hard drive

William Kenworthy wrote:
 On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
 Paul Hartman wrote:
 On Tue, Aug 7, 2012 at 5:26 PM, Dale rdalek1...@gmail.com wrote:
 I didn't know you could do low level formats anymore.  Really?  What
 package provides that?  Hmmm, I'm thinking about those HOURS spent
 formatting a 100Mb drive and then thinking about how long it will take
 to do a 3Tb drive.  O_O  I mean really O_O.  LOL
 hdparm provides it. Do a search for ATA secure erase or enhanced
 secure erase. It is as close as there is to a low-level format in
 ...
 I have seen where people use dd to do this sort of thing to.  I read
 somewhere that if you do a dd and put in all 1's, then all 0's then back
 again that it is very hard to get any data back off the drive.  I think
 if you do it like over a dozen times, it is deemed impossible to get
 anything back.  I think that is the Government standard of it's gone. 

 4 or 5 hours huh.  I guess drives are a lot faster now.  Back in the
 late 80's or early 90's, it took that long for those whimpy little 100Mb
 drives.  Ooops, my ages is showing again.  lol 

 I got to go read up on hdparm.  I already have it installed here.  I'm
 not planning to use this part but do want to read up on this. 

 Thanks.

 Dale

 :-)  :-) 


 Goggle have a well known document
 (http://research.google.com/archive/disk_failures.pdf) where they
 analysed hard drive failures for a very large number of drives ... the
 basic upshot is that a very large portion of failures happen with no
 pre-warning, so testing a drive like you are proposing not going to
 prove a thing.

 They also found that smart (is quite dumb) and its tests were of little
 use.

 And high temperatures and work loads were also not a reliable guide to
 trends in failure rates, both of which which surprised me.

 Some of those bathtub curves that I was trained on when setting
 maintenance schedules dont hold water here!

 This anaysis of the paper looks quite good if you want the lite view:
 http://storagemojo.com/2007/02/19/googles-disk-failure-experience/

 BillK



 BillK



Well, I am going by actual real experiences from other users of this
model of drive.  I don't know what google was testing but I would bet it
is not the drive model I just bought.  The users who bought this exact
model drive report that most failures are either out of the box or
within a few weeks to a month.  I'm just going to try to increase my
odds even if it is just a little bit. 

Smart may not always predict a failure but it is better than nothing at
all.  Would you rather have a tool that may predict a failure or no tool
at all?  Me, I'd rather have something that at least tries too.  The one
drive I had to go bad, Smart predicted it very well.  It said I had
about 24 hrs to get my stuff off.  Sure enough, the next day, it
wouldn't do anything but spin.  Without Smart and its prediction, I'd
have lost the data on the drive with no warning at all. 

A couple questions.  What if while I am testing this drive, it dies? 
Does that prove that my testing benefited me then? 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread Adam Carter

 To wipe a drive use dban. - live CD which uses (US) gov approved
 standards of wipe methods/patterns.

Or shred, which comes with coreutils.

 dd is only going to show sectors on a failed drive - too late!

 To explain, modern drives have a store of locations they can use to
 transparently replace any failed locations (apparently similar to the
 way SSD's do it)  - the internal drive electronics handle this and its
 not visible externally though smart data seems to show it, but as google
 says, smart is a bit suspect.  The problem of a bad sector will only
 show once all the reserved locations are used up, by which time the
 drive is usually in rampant failure.

 I do suspect this is one reason for googles results - actual failures of
 the media (as against the motors/electronics are much as they always
 have been, but the drives are not reporting them until its too late.

Ahh - go to know. My reasoning assumed that smart reports all remaps.

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-07 Thread William Kenworthy

On Tue, 2012-08-07 at 23:18 -0500, Dale wrote:
 William Kenworthy wrote:
  On Tue, 2012-08-07 at 21:19 -0500, Dale wrote:
  Paul Hartman wrote:
  On Tue, Aug 7, 2012 at 5:26 PM, Dale rdalek1...@gmail.com wrote:
...
  Goggle have a well known document
  (http://research.google.com/archive/disk_failures.pdf) where they
  analysed hard drive failures for a very large number of drives ... the
  basic upshot is that a very large portion of failures happen with no
  pre-warning, so testing a drive like you are proposing not going to
  prove a thing.
 
  They also found that smart (is quite dumb) and its tests were of little
  use.
 
  And high temperatures and work loads were also not a reliable guide to
  trends in failure rates, both of which which surprised me.
 
  Some of those bathtub curves that I was trained on when setting
  maintenance schedules dont hold water here!
 
  This anaysis of the paper looks quite good if you want the lite view:
  http://storagemojo.com/2007/02/19/googles-disk-failure-experience/
 
  BillK
 
 
 
 Well, I am going by actual real experiences from other users of this
 model of drive.  I don't know what google was testing but I would bet it
 is not the drive model I just bought.  The users who bought this exact
 model drive report that most failures are either out of the box or
 within a few weeks to a month.  I'm just going to try to increase my
 odds even if it is just a little bit. 
 
 Smart may not always predict a failure but it is better than nothing at
 all.  Would you rather have a tool that may predict a failure or no tool
 at all?  Me, I'd rather have something that at least tries too.  The one
 drive I had to go bad, Smart predicted it very well.  It said I had
 about 24 hrs to get my stuff off.  Sure enough, the next day, it
 wouldn't do anything but spin.  Without Smart and its prediction, I'd
 have lost the data on the drive with no warning at all. 
 
 A couple questions.  What if while I am testing this drive, it dies? 
 Does that prove that my testing benefited me then? 
 
 Dale
 
 :-)  :-) 
 

Read the paper - its written by someone who buys drives in batches of
100's+, not by a few guys posting on a forum somewhere who bought one
random drive, who probably didn't use anti-static techniques handling
the drive, and thumped it around in the boot of the car or got it via
the courier who was famous for delivering TV's by throwing them over the
fence.  It is a bit of an eye opener - read it.

My impression of models is that it is not really the model that has a
run of failures, but the batch so a different run of the same model will
have a different failure pattern.  There are exceptions such as those
IBM 60G deathstar drives, but then again they fixed it and following
drives of the same model were fine.

My own experience of smart is it tells you something, but what it seems
to say is not right (notice I am not saying it tells lies, but that the
data and interpretation don't make sense on the drives Ive had)

Drive failure does seem to be a semi-random lottery, but I am seriously
doubtful that testing will do anything ... it has as much chance of
precipitating failure that wouldn't occur otherwise because you are
seriously hammering it, or weakening the drive so it will fail at some
random time, but perhaps weeks away rather than the years it otherwise
would, or nothing will happen except for wasted electrons.  

Then again, I am of the view that modern electronics is
designed/programmed to fail a few seconds past warranty expiry (why else
do most devices have timekeeping built in :)

BillK

Re: [gentoo-user] Want to seriously test a NEW hard drive

Dale wrote:
Howdy,

I finally got me a 3Tb drive on the way. Should be here Wednesday. I
have seen some reviews where it would not work right. I think some of
it may be BIOS related since some BIOS's don't like drives that large.
Anyway, I want to test this thing real good to really make sure it is up
to the task before putting my data on it. It's going to be so much
data, there is really no way to do back-ups at this point. Come on, 2
to 3Tbs on 4Gb DVDs. Really? lol Maybe a external drive later on but
for now, well.

I have heard of bonnie and friends. I also think dd could do some
testing too. Is there any other way to give this a good work and see if
it holds up? Oh, helpful hints with Bonnie would be great too. I have
never used it before. Maybe someone has some test that is really brutal.

Thanks

Dale

:-) :-)

I seem to have got a few really good responses to my question. Since I
really want to test this drive really hard, I'll try them all. lol If
it survives a few runs of all this, maybe it is going to live. Like the
one with the script too. May have to edit for my needs but certainly a
GREAT start.

While at it, I found a lot of references to the flood in Japan or I
assume it was Japan. Anyone think this could still be a problem?
Should they all be flushed out of the system by now? Pardon the flush
term there. ;-)

Linky:

http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=1310560CatId=4357

Please, no hard drive brand flame wars. Seagates fail, WD fails, they
all fail at some point. We all know that already. This is what I got
so it is what it is. :-D That is why I want to test this thing like a
wild man BEFORE putting anything on it. If it has a issue, hopefully
the test will bring that to the front now instead of later.

Waiting on Wednesday to get here now.

Dale

:-) :-)

P. S. What is a good way to back up something this large BESIDES
another drive the same size or larger? I thought about getting a blue
ray burner but even that will take a lot of media. Another drive is all
I can think of myself.

--
I am only responsible for what I said ... Not for what you understood or how
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-06 Thread Volker Armin Hemmann

What do you gain if you abuse your drive so hard that its lifetime is
severly impacted?
Am 06.08.2012 11:47 schrieb Dale rdalek1...@gmail.com:

Dale wrote:
Howdy,

Thanks

Dale

:-) :-)

Linky:

http://www.tigerdirect.com/applications/SearchTools/item-details.asp?EdpNo=1310560CatId=4357

Waiting on Wednesday to get here now.

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or
how you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-06 Thread Mick

On Monday 06 Aug 2012 10:42:17 Dale wrote:

 If it has a issue, hopefully
 the test will bring that to the front now instead of later.

Or it is just going to knacker your drive and make it fail earlier that it 
would otherwise (esp if you overheat it)?  ha, ha!


 P. S.  What is a good way to back up something this large BESIDES
 another drive the same size or larger?  I thought about getting a blue
 ray burner but even that will take a lot of media.  Another drive is all
 I can think of myself.

Splitting it?

Partimage would do that and so would star/tar for fs level backups.
-- 
Regards,
Mick


signature.asc
Description: This is a digitally signed message part.

Re: [gentoo-user] Want to seriously test a NEW hard drive

Volker Armin Hemmann wrote:

 What do you gain if you abuse your drive so hard that its lifetime is
 severly impacted?


That if it has a problem that will cause it to fail soon in it's life,
then I can find it soon.  Remember that curve about failures?  I would
like to get past that first part of the curve.  Maybe by the time I get
to the later part, I'll have another drive or some backup scheme.  Most
the failures I have read about in reviews for this drive were early or
was just plain old DOA.  Testing it will get me past that.  I'd rather
it fail before I get my data on it instead of after. 

I thought I posted why I wanted to do this in my first post. 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

Mick wrote:
 On Monday 06 Aug 2012 10:42:17 Dale wrote:

 If it has a issue, hopefully
 the test will bring that to the front now instead of later.
 Or it is just going to knacker your drive and make it fail earlier that it 
 would otherwise (esp if you overheat it)?  ha, ha!

Well, if it is going to fail because of anything, including heat, I
would rather it do so BEFORE I put my stuff on it.  Right now, a backup
is not possible other than a blue ray or something.  Also, I have a
Cooler Master case with the fan blowing right on the drives.  If it gets
hot and blows a fuse, it has a problem anyway.  It may as well die early
while under warranty. 


 P. S.  What is a good way to back up something this large BESIDES
 another drive the same size or larger?  I thought about getting a blue
 ray burner but even that will take a lot of media.  Another drive is all
 I can think of myself.
 Splitting it?

 Partimage would do that and so would star/tar for fs level backups.

Split what?  The drive into two partitions or something?  If that is
what you mean, if it dies, I'd still loose it all.  If you meant two
drives, well, I only have one on the way right now.  It will be a while
before I can get another.  In the past, I just backed up stuff on DVDs. 
3Tbs is a lot of DVDs tho.  It won't take to long to fill that up
either.  I love my DSL.  lol

I hope to get another drive in the future tho.  Just going to be a
little while.  ;-) 

Dale

:-)  :-) 

-- 
I am only responsible for what I said ... Not for what you understood or how 
you interpreted my words!

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-06 Thread Hinnerk van Bruinehsen

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 06.08.2012 12:14, Dale wrote:
 Volker Armin Hemmann wrote:
 
 What do you gain if you abuse your drive so hard that its
 lifetime is severly impacted?
 
 
 That if it has a problem that will cause it to fail soon in it's
 life, then I can find it soon.  Remember that curve about failures?
 I would like to get past that first part of the curve.  Maybe by
 the time I get to the later part, I'll have another drive or some
 backup scheme.  Most the failures I have read about in reviews for
 this drive were early or was just plain old DOA.  Testing it will
 get me past that.  I'd rather it fail before I get my data on it
 instead of after.
 
 I thought I posted why I wanted to do this in my first post.
 
 Dale
 
 :-)  :-)
 

Why not simply get you data on it and use it for about 2 weeks? Maybe
you should mirror important stuff to the old drive for that time.

After about 2 weeks of normal usage you should be well out of the
beginnig of that bathtub curve (I always had problems when copying
data to the new drive when I had a bad one, except DOA of course).


-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.19 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJQH6ESAAoJEJwwOFaNFkYc1f0IAK/UpzNp/r4KBYczYIc9jKmJ
3JBvtCWPZkJ1qRXfZtErTk/oamPa+/iapJSoaoj4p45Uvjq5Qh5WDXiAlZs3U805
X1nCc9OMfKc3KGK3bcSKLhSIA1oV1imdgq1OqH/NeioEqOuVy56ZOdvwq6ml2fwM
1ZjBjsp1uqirork5eyZcJGGoNcsJpX3DvTZ7tBwM0gDFHcu4Yj3alEy7PgiYny4W
z0/K91jnyBouX7+CwtWKvGNgjGqMxXXp4SUS4MuOKs06p4LFKi/gHBRCAYjuTRK1
O3cKfz4nHMI+F2rH48sCAUgT5K88cdiX40lZpqTWyqADUQO9F1ZzTcg4fZW447s=
=s524
-END PGP SIGNATURE-

Re: [gentoo-user] Want to seriously test a NEW hard drive

2012-08-06 Thread Neil Bothwick

On Mon, 06 Aug 2012 04:42:17 -0500, Dale wrote:

 P. S.  What is a good way to back up something this large BESIDES
 another drive the same size or larger?

A smaller drive, backups can be compressed. Also, you don't need to
backup everything, for example rips of DVDs and CDs you own can be
recreated. If you organise your data to separate stuff that needs to be
backed up from that which doesn't you can reduce the amount of space
needed.

Factor in that you won't be filling this drive for a while and that you
don't need maximum performance from a backup drive and you'll almost
certainly get by with using your old drive(s).

Really important stuff should be stored offsite, Amazon S3 gives the best
value for me.


-- 
Neil Bothwick

Ask a silly person, get a silly answer


signature.asc
Description: PGP signature

Re: [gentoo-user] Want to seriously test a NEW hard drive