TidBITS#707/24-Nov-03
=====================
What should you use to fix a corrupted hard disk? Disk expert
David Shayer ran detailed tests comparing DiskWarrior, Norton
Utilities, Drive 10, DiskGuardian, and Disk Utility - which do you
think won? Adam also chimes in with the latest Take Control ebook
free updates and system changes. In the news, Apple released a
20-inch iMac and speedier Power Mac G5s, and posted a new security
update, and Creo released Tokens, a solution to file transfer
problems. TidBITS is off next week for the Thanksgiving holiday
in the U.S. - see you in December!
Topics:
MailBITS/24-Nov-03
More Take Control Updates
Shootout at the Disk Repair Corral
Hot Topics in TidBITS Talk/24-Nov-03
<http://www.tidbits.com/tb-issues/TidBITS-707.html>
<ftp://ftp.tidbits.com/issues/2003/TidBITS#707_24-Nov-03.etx>
Copyright 2003 TidBITS: Reuse governed by Creative Commons license
<http://www.tidbits.com/terms/> Contact: <[EMAIL PROTECTED]>
---------------------------------------------------------------
This issue of TidBITS sponsored in part by:
* READERS LIKE YOU! Help keep TidBITS great via our voluntary <------ NEW!
contribution program. Our thanks this week to Ray Thompson,
Irma Johnson, and David Blanchard for their generous support!
<http://www.tidbits.com/about/support/contributors.html>
* SMALL DOG ELECTRONICS: Power On Sale! <---------------------------- NEW!
Selected Power Macs & PowerBooks on sale including G5s:
G5/1.8GHz: $2,049; G4/1GHz: $1479; 17" PowerBook G4: $2,299!
Visit: <http://www.smalldog.com/tb/> 800-511-MACS
* FETCH SOFTWORKS: Want to copy files between FTP servers? <--------- NEW!
Use Fetch, the FTP client for Mac OS 9 and X, and you can
drag and drop files for fast and easy file copying.
Download your free trial version! <http://fetchsoftworks.com/>
* Aladdin Systems: NEW & IMPROVED STUFFIT DELUXE 8.0 NOW SHIPPING!
Faster compression, more file formats, Archive via Rename, burn
to CD/DVD, and more! Manages StuffIt, Zip, and TAR archives!
$30 OFF AT: <http://www.aladdinsys.com/store/tidbitsoffer.html>
* ConceptDraw V: Special! ONLY $79 for owners of competing titles.
Network, UML, database, business, and project diagrams are easy
with ready-made shapes and wizards! More power than MS Visio
and compatible with Mac OS X! <http://www.conceptdraw.com/>
* DR. BOTT, LLC: Explore our vast assortment of iPod accessories!
NaviPod wireless remote; PocketDock for using standard FireWire
cables with your 3G iPod, and other startlingly cool gear for
enhancing your digital lifestyle. Visit <http://www.drbott.com>
* Bare Bones Software TextWrangler 1.5 -- General-purpose tool for <- NEW!
composing, modifying, and transforming text. Now with full
AppleScript support! US$49. For more info, to download a demo,
or to purchase a copy, visit <http://www.barebones.com/>.
* CREO TOKENS - Send big files or folders, around the office or <---- NEW!
the world. Combines compression, encryption, and cross-firewall
technology into a "one-click" package. Download the free Token
Redeemer or Token Creator trial at <http://www.creo.com/tokens>
---------------------------------------------------------------
MailBITS/24-Nov-03
------------------
**Next Issue 08-Dec-03** -- We're taking next week off from our
regular publication schedule so we can all spend some relaxing
time with families and friends over the Thanksgiving holiday here
in the U.S. I plan to continue moderating TidBITS Talk many of
these days other than a few around Thanksgiving itself, and we'll
be using some of the extra time to work on assembling our annual
gift issue. Look for the next issue of TidBITS on 08-Dec-03, and
for those partaking in the holiday festivities, enjoy! [ACE]
**Submit Your Holiday 2003 Gift Ideas** -- We may not approve of
ads for Christmas sales appearing the day after Halloween, but it
is now once again a decent time to start thinking about the year's
annual spectacle of consumer confidence. For us at TidBITS, that
translates into sharing the best holiday gift ideas from readers
in our traditional holiday gift issue, slated for publication
during the second week in December. So tell us what gifts you're
planning to bestow on your friends or family, or what gifts you're
hoping to receive yourself. As in previous years, we'll be
collecting ideas in TidBITS Talk, so please send your suggestions
to <[EMAIL PROTECTED]>. We've already started threads for
specific categories, and there's a "Respond (via email)" link for
you to use at the bottom of each message in the TidBITS Talk Web
archive (check out the Hot Topics in TidBITS Talk article at the
end of this issue for direct links). As always, please suggest
only one product or idea per message, give the reason why you're
recommending it, make sure to include a URL or other necessary
contact information, and recommend only others' products. Also
try to concentrate on items that haven't appeared in past years.
Thanks! [ACE]
<http://db.tidbits.com/getbits.acgi?tlkthrd=active>
<http://db.tidbits.com/getbits.acgi?tbiss=560>
<http://db.tidbits.com/getbits.acgi?tbiss=609>
<http://db.tidbits.com/getbits.acgi?tbiss=659>
**Apple Introduces 20-inch iMac** -- Just in time for the holiday
shopping season, Apple has released a new version of the flat-
panel iMac that offers a 20-inch LCD screen with a resolution
of 1680 by 1050 pixels. The screen is also brighter then the
smaller models and offers a higher contrast ratio. As far as other
functional specs go, the 20-inch iMac is identical to the 17-inch
iMac, with a 1.25 GHz PowerPC G4 processor, 256 MB of RAM,
80 GB hard drive, SuperDrive, and all the usual ports. The
one interesting deviation from the 17-inch iMac is with the
weight - the 20-inch iMac weighs 40.1 pounds (18.2 kg), 17.3
pounds (7.8 kg) heavier than the 22.8 pound (10.4 kg) 17-inch
iMac. The new 20-inch iMac is available immediately for $2,200
in the stock configuration, with all the usual build-to-order
options. The 15-inch iMac remains priced at $1,300 and the 17-inch
iMac still costs $1,800. [ACE]
<http://www.apple.com/imac/>
<http://www.apple.com/imac/graphics.html>
**Apple Releases 1.8 GHz Dual Processor G5** -- Apple has released
a new model of the Power Mac G5, its 64-bit minitower computer
aimed at professional and high-end Macintosh users. The new model
offers dual 1.8 GHz PowerPC G5 processors (each with a 900 MHz
frontside bus) with prices starting at $2,500. The new systems
slip into the middle of Apple's G5 lineup, replacing a $2,400
configuration with a single 1.8 GHz processor; otherwise, the
specifications for the Power Mac G5s remain unchanged, save
that they all now ship with Mac OS X 10.3 Panther. (For details
on the changes in the Power Mac G5 architecture compared to
previous systems, see "Apple Announces 64-Bit Power Mac G5s"
in TidBITS-685_.) However, Apple has reduced the starting price
of the 1.6 GHz single processor system by $200 to $1,800. The new
dual-processor model serves to highlight benefits of Mac OS X's
multiprocessor capabilities, particularly in computationally
intensive areas like video and media production. [GD]
<http://www.apple.com/powermac/>
<http://db.tidbits.com/getbits.acgi?tbart=07241>
**Security Update 2003-11-19 Helps Panther, Jaguar** -- Apple
released a pair of security updates last week via Software Update,
bringing both Mac OS X 10.2.8 Jaguar and Mac OS X 10.3.1 Panther
up to the same security level. Security Update 2003-11-19 improves
OpenSSL security and fixes a potential problem with third-party
applications using a print library. On Macs running Jaguar, the
security update also fixes issues with Mail, QuickTime for Java,
Personal File Sharing, and a few other vulnerabilities that were
implemented in the first release of Mac OS X 10.3 Panther (see
"Security Update 2003-10-28 Released" in TidBITS-704_). The update
for Jaguar is a 4.6 MB download; the update for Panther is a
1.3 MB download. [JLC]
<http://docs.info.apple.com/article.html?artnum=120278>
<http://docs.info.apple.com/article.html?artnum=120277>
<http://db.tidbits.com/getbits.acgi?tbart=07418>
**Creo Eases File Sharing with Tokens** -- Have you found sharing
files via email frustrating? Email, as we've been predicting,
is becoming increasingly unreliable, thanks to spam-overloaded
servers and inaccurate filters, and, when we're talking about file
sharing, encoding format troubles and attachment size limitations.
Creo, makers of the Six Degrees program for managing email-based
workgroups, has come up with an alternative called Tokens, which
works with both the Mac and Windows. Rather than attach your huge
Keynote presentation to an email message to multiple people, you
use the $50 Tokens Creator to create a several-kilobyte "token"
that points back to a compressed and encrypted version of the
Keynote file on your hard disk. When your recipients receive the
token you sent, they double-click it to open it in the free Token
Redeemer, which retrieves the presentation from your computer.
Your computer must be left on and connected to the Internet
to serve the file; if there are other network obstructions
(such as firewalls) between the recipient and your computer,
Token Redeemer automatically retrieves the file via Creo's Tokens
relay service (which allows up to 5 GB per month of transfer).
After a basic installation on both sides, no setup is necessary,
and no one has to worry about user accounts or passwords. Creo
also offers a $600 Tokens Server, which comes with 10 licenses
for Token Creator and handles the file serving duties for
all of them. Tokens is definitely a 1.0 product, but it's
interesting, and could serve an important role in simplifying
file transfer. [ACE]
<http://www.creo.com/tokens/>
More Take Control Updates
-------------------------
by Adam C. Engst <[EMAIL PROTECTED]>
Silly us. We thought things would ease up after our initial launch
and release of the 1.1 update to Joe Kissell's "Take Control of
Upgrading to Panther" two weeks after Panther shipped. On the
contrary, we're working as hard as ever, and orders continue to
flow in as more and more people hear about Take Control from
places like Apple eNews and a number of user group newsletters.
**"Take Control of Customizing Panther" Update** -- Most notably,
we've now released the 1.1 update to Matt Neuburg's "Take Control
of Customizing Panther." We're really pleased with it, since we've
made significant changes and additions that place it squarely
where we want the Take Control series to evolve. It's more than
twice as long, with 47 content pages, up from 22, and we focused
the writing to provide a more concrete set of tasks to perform
to customize Panther to your liking.
<http://www.tidbits.com/takecontrol/panther/customizing.html>
New sections include a discussion of using some of our favorite
third-party utilities to customize Panther, coverage of how you
can manage Services better, three ways of setting your Internet
helper applications, and small tweaks you can apply to Safari and
Terminal. Also, since we discovered that many readers didn't have
much, if any, experience with Mac OS X, we added a large section
listing the basic customizations you can perform using System
Preferences and Finder Preferences.
In accordance with the feedback we received after publishing the
update to Joe's book, we added internal links to the Changes in
Version 1.1 section, so you can jump quickly to the new bits. We
also added a tip about which pages largely stayed the same for
people who want to print only the changed pages; however, Matt
added and changed so much that most pages are different. One tip
regarding printing: in the Print dialog, choose Layout from the
Copies & Pages pop-up menu, choose 2 from the Pages Per Sheet
pop-up menu, and choose Single Hairline from the Border pop-up
menu. The large font we used prints quite nicely when reduced,
so that two pages fit on each piece of paper.
Since we believe early adopters should be rewarded, and since our
goal is to provide the best experience possible, we've sent email
to almost everyone who purchased the 1.0 version, telling them how
they can get the 1.1 version for free.
Why almost everyone? When you order through eSellerate (see
below), there's a checkbox titled "Send me information about
product updates and offers." We can't currently change the wording
of that checkbox, so if you deselect it, we won't send you email
about the free update - you've opted out. If that applies to you,
send us email at <[EMAIL PROTECTED]> and we'll let you know
how to get the update and add you to the list.
Similarly, if you didn't receive email notification because you
have changed email addresses or because your ISP is employing
stupid spam filters, send us email and we'll fix you up. That
said, dealing with these delivery problems generates a lot of
unnecessary work for us, so please try to ensure that you can
receive mail reliably at the address you give when ordering.
**Ordering System Changes** -- We're now trying eSellerate's
service for fulfilling our orders. Although we had no general
complaints with Kagi, we found that dealing with our own merchant
account was draining vast amounts of time and energy away from
editorial work, and we can't afford that right now. Overall,
eSellerate appears to be working well for us, and one very real
advantage is that you can download directly from the final order
screen, rather than waiting for your email receipt to arrive.
eSellerate also hosts our downloads now, which lets us eliminate
the encrypted archives that required entering passwords into
StuffIt Expander, a process that proved surprisingly troublesome.
We still compress our PDF files because so many Web browsers are
set to display PDF files in a window rather than download them
for later reference. To make things easier for people who want to
download at work on a Windows machine and print there, we used
StuffIt Deluxe to create a Zip archive that any recent version
of StuffIt Expander should be able to expand. People who download
using Safari in Panther will find the Zip archive expanded
automatically without launching StuffIt Expander.
**New Ebooks Coming Soon** -- Although most of our efforts have
been aimed toward our existing two titles so far, we have two
more books in the works right now: "Take Control of Sharing Files
in Panther," by Glenn Fleishman, and "Take Control of Users &
Accounts in Panther," by Kirk McElhearn. You can read more about
them on the Take Control home page, and stay tuned for even more
titles that are still in the outline stage.
<http://www.tidbits.com/takecontrol/>
Shootout at the Disk Repair Corral
----------------------------------
by David Shayer <[EMAIL PROTECTED]>
Some things are inevitable. Death, taxes, and disk crashes. One
day you will try to open an important file, only to receive a dire
error message. Or perhaps you'll discover that an entire folder
has vanished. Worse yet, maybe your Mac won't even boot, thanks
to some sort of disk corruption.
Fortunately, you have a full backup of all your data, so you just
restore the missing data from your backup, and you're back in
business. What's that you say? The last time you backed up was
during the Reagan administration?
If an ounce of prevention is worth a pound of cure, then surely
the most effective disk repair program is actually a reliable
backup utility. My favorite is Dantz's Retrospect. But whatever
backup program you choose, you must use it regularly, so you have
a current backup when your hard disk is called to that great clean
room in the sky. (See the TidBITS article series, "Have You Backed
Up Today?" for more details on setting up a good backup strategy.)
<http://www.dantz.com/>
<http://db.tidbits.com/getbits.acgi?tbser=1041>
Yet just as so many of us would prefer to lose weight by taking
a magic pill rather than through diet and exercise, we'd rather fix
a corrupted disk with a disk utility, rather than restoring from
a backup, even when a recent backup is available. It can take many
hours to do a full restore from a backup, whereas a good disk
utility can often fix minor disk errors in minutes.
Some of the Macintosh world's favorite disk repair programs have
recent upgrades, and here I'll compare the Norton Disk Doctor tool
in Symantec's Norton Utilities 8.0 ($100), Alsoft's DiskWarrior
3.0 ($80), Micromat's Drive 10 1.1.4 ($70), SubRosaSoft's
DiskGuardian 2.2 ($70), and Apple's Disk Utility (free). Although
these programs contain a wide variety of disparate features, I
concentrate on their disk repair functions in Mac OS X. I chose
not to include Prosoft Engineering's Data Rescue X, because it
recovers files onto another disk and does not repair the damaged
disk itself.
<http://www.symantec.com/nu/nu_mac/>
<http://www.alsoft.com/DiskWarrior/>
<http://www.micromat.com/drive_10/drive_10_introduction.html>
<http://www.subrosasoft.com/>
<http://www.prosoftengineering.com/products/data_rescue.php>
The user interface and ancillary features of a disk repair program
are secondary, because in the event of disaster you care about
only one thing: will it get my data back? So let's concentrate on
the heart of the issue: what are the most common disk errors you
may experience, and which disk repair programs can save your bacon
when you're unfortunate enough to suffer disk corruption?
In my experience, most people run into three general categories
of disk problems: hardware failure, bad sectors, and damaged
directories. After a brief examination of how you get started with
these programs, given that you can't repair an active startup disk
in Mac OS X, I look at the worst type of problem - hardware
failures, after which I examine the soft errors and compare
the performance of the disk repair programs.
**Booty Call** -- One disadvantage of Mac OS X is that a disk
repair program can't safely check the startup disk. Despite this,
and the fact that Apple's official line is that checking startup
disks is not supported, Norton Disk Doctor and DiskGuardian both
allow checking of the startup disk, although they warn against
doing so. I consider messing about with startup disks under
Mac OS X dangerous, and I advise you not to do it.
Fortunately, there is a simple solution. Restart in "single user
mode" by holding down Command-S while the Mac is starting up. In
single user mode, you're dropped into a command line version of
Mac OS X, without windows or a mouse pointer. Type "fsck -y" to
check (and repair, if necessary) the boot disk, after which you
restart the Mac by typing "reboot" (sans quotes for both
commands). Disk Utility and fsck rely on the same engine, so
running fsck in single user mode is exactly the same as running
Disk Utility.
If you're uncomfortable with the command line for even two
commands, you have an alternative. If you boot your Mac using the
Mac OS X Installer CD, you can run Disk Utility from there. At the
first screen in the Installer, choose Open Disk Utility from the
Installer menu.
What about the other disk repair programs? All except DiskGuardian
(for now, but a new version is expected soon) come on bootable
CDs. Insert the CD, turn on your Mac, and hold down the C key to
force the computer to boot from the CD. Bootable CDs are essential
in the event that your hard disk is so badly damaged that your
computer won't even start up.
**It's Dead, Jim** -- Let's look at what can go wrong now.
Hardware failure can result from the electronics on the drive's
controller board burning out, or the heads or the arm developing
mechanical problems. Sometimes a problem with the lubricating
grease prevents the disk from spinning or the read arm from
moving, causing a problem known as "sticktion."
These problems are caused by dropping the disk, by defective
components, by static discharge, or even by sheer age. Usually
the disk won't even show up on the Desktop. As far as the disk
utilities go, Drive 10 can detect hardware errors with a "Unit
Ready" test, which is just what it sounds like. Drive 10 asks
the drive "Are you ready?" and the drive replies "No." This
test is mostly helpful for confirming hardware errors you
probably already suspect.
Hopefully there's no vital data locked in the dead drive. No
software program can repair a disk with hardware problems. If
you desperately need to retrieve the data, your only option is
a data recovery company such as DriveSavers, who I've found to be
professional, competent, and expensive. (Also see Jeff Carlson's
report in "DriveSavers to the Rescue" in TidBITS-495_.)
<http://www.drivesavers.com/>
<http://db.tidbits.com/getbits.acgi?tbart=05530>
**Extended Warranties** -- If your drive has just suffered a
hardware failure, you're probably checking the warranty and
hunting for your receipt. Don't despair, even if your computer
or external hard drive is out of warranty from the manufacturer.
Many hard drives, even those installed in computers, are also
covered by longer warranties provided by the OEM (original
equipment manufacturer), the company that actually made the
drive. If you're willing and able to open a case and remove
the drive mechanism, you can take advantage of this warranty.
Here's the trick. Although you may have bought the computer from
Apple, or the drive from VST, those companies don't make drive
mechanisms. Instead, your drive was probably made by IBM, Seagate,
Maxtor, HP, Western Digital, or another hard drive manufacturer.
These companies often offer an independent warranty on their
drives, and it's often longer than the warranty on your Mac or
external hard drive. Drive mechanism warranties may be two years,
and some run as long as five years. Better yet, if you've lost the
receipt, the company can sometimes look up a drive's serial number
to verify that it's still under warranty.
Both HP and IBM have replaced dead drives for me, without a
receipt, simply by checking my serial numbers. Even better, IBM
didn't make 14 GB drives anymore, so they replaced my dead one
with a 20 GB unit.
**SMART Stuff** -- Some modern hard drives have a feature called
SMART, which stands for Self-Monitoring, Analysis, and Reporting
Technology. Originally developed by IBM, SMART-compliant drives
constantly perform diagnostics to monitor variables like drive
temperature, spin up time, and how well the heads stay on track.
By noting when these physical parameters slip out of spec, SMART
can predict some types of drive failure before they happen, giving
you time to back up your data and buy a new drive.
Unfortunately, most hard drives have no way to tell you about an
impending disaster; you need a program to query the drive's SMART
statistics. Both DiskWarrior and DiskGuardian can check the drive
at regular intervals and alert you if they find trouble.
Using SMART works well if your disk has a factory defect that's
slowly getting worse, because it gives you time to move your data
to a new disk. But SMART doesn't help you deal with the occasional
bad sector or corrupted directory, much less catastrophic hardware
failures. Since I don't have a drive with impending hardware
failure, I was unable to test the SMART capabilities of
DiskWarrior and DiskGuardian.
**Unsavory Sectors** -- Now it's time to move from pure hardware
problems to problems that could be either physical (hardware) or
logical (software). Disks are broken up into sectors. Each sector
usually holds 512 bytes of data. There are two types of problems
that lead to bad sectors: hard errors and soft errors.
Hard errors are caused by physical damage to the disk surface.
Dropping the disk and manufacturing defects are the most common
causes of hard errors. Although there's no way to repair hard
errors, they can be "fixed" by "sparing" the bad sector. Disks
maintain a small number of spare sectors in reserve; when a hard
error occurs, the drive controller maps out the bad sector and
uses one of the spare sectors in its place.
How are these bad sectors spared? SCSI hard disks provide a SCSI
command - "reassign" - to spare a bad sector. A low-level format
also spares any bad sectors. On pre-SCSI disks, a low-level format
was the only way to fix bad sectors.
Many modern disks, including many internal ATA, FireWire, and USB
drives, automatically spare sectors with hard errors the next time
the sector is written. That's helpful, but if data is stored on
that sector, programs may be unable to read it successfully,
causing problems and making it difficult to spare. Erasing the
disk with Apple's Disk Utility spares any bad sectors if you
select the "Zero all data" option.
What about soft errors? In addition to the 512 bytes of data
stored in each sector, a few additional bytes hold an error
correction code (ECC). When the sector is written, the drive's
controller computes and records the ECC. When the file system
later reads that sector, it checks the ECC to make sure the data
hasn't been corrupted. If the ECC doesn't match the data, it's
called a soft error. The disk surface is fine, but the data on
that sector has become scrambled.
Soft errors can be caused if the disk is jarred while it's writing
or if power is lost while writing, either of which can leave
a sector half written. Large magnets (such as can be found in
electric motors) next to hard disks also tend to have bad effects
on the data. As with hard errors, most modern disks repair soft
errors automatically the next time the sector is written.
**Bad Sector Detector** - Norton Disk Doctor, DiskGuardian and
Drive 10 (but not DiskWarrior or Disk Utility) claim to detect bad
sectors using a test called either a "defective media check" or a
"surface scan." Using a proprietary tool that creates soft errors
on disks, I tested each program.
Although Norton Disk Doctor claims to be able to find and repair
bad sectors, its defective media check didn't detect the bad
sectors on my test disk, erroneously giving it gave a clean
bill of health.
DiskGuardian detected the bad sectors, although it took several
hours to run a full check. Unfortunately, it didn't tell me which
files used the bad sectors, so I had no way of finding out which
files were damaged and would thus need to be restored from backup.
DiskGuardian lacks the capability to repair bad sectors.
Like DiskGuardian, Drive 10 detected the bad sectors, but didn't
identify which files were damaged. Confusingly, Drive 10's report
describing the damage claimed it could fix the bad sectors, but
I couldn't find a command to fix them. Micromat tech support
confirmed the report was wrong; Drive 10 can't fix the bad sectors
it finds. It's too bad, since Drive 10 could fix the bad sectors
merely by writing zeros to them.
Although Disk Utility cannot scan for bad sectors, it can fix bad
sectors on modern disks if you erase the disk with the "Zero all
data" option selected.
I must rate all these products unacceptable in dealing with bad
sectors. Even though two could detect bad sectors, none of them
could tell you which files contain bad sectors, making it
impossible to learn which files you should restore from your
backup. Only Disk Utility successfully fixed the bad sectors,
but at the price of erasing the entire disk.
**Ripping the Yellow Pages** -- We've now looked at pure hardware
failures, and bad sectors, which can be either hard errors or soft
errors, and so far, our disk repair utilities don't help much at
all. Now it's time to move on to problems that exist entirely in
software, the most common type of which are errors in the
directory, which tracks the files and folders on the disk. In
the case of directory errors, there is nothing wrong with the
drive mechanism or the disk surface; instead, the directory
information that's necessary to locate your data on the disk
has simply become scrambled. Often your data is intact, if it
could just be located.
As an aside, people with important data sometimes use mirrored
disks or RAID arrays, which faithfully duplicate each byte on the
main disk to a backup disk. If the main disk suffers a hardware
failure or develops a bad sector, the backup disk can save the
day. However, it's worth noting that this strategy provides
absolutely no protection against directory damage. That's because
the RAID faithfully records all data to the backup disk, whether
or not that data is good, which results in both the main disk and
the backup disk containing corrupt data. I may sound like your
mother telling you to eat your vegetables, but the best protection
really is regular backups.
The most common cause of directory damage is crashing. If the
computer crashes while a file is being created or saved, causing
only part of the change to be written to disk, the directory will
contain inconsistent information. Mac OS X crashes far less often
than Mac OS 9, but directory-corrupting crashes can and do still
occur. Both Mac OS 9 and Mac OS X automatically check and repair
the startup disk after a crash, which reduces the incidence of
disk damage dramatically.
A new feature in Mac OS X 10.3 Panther that should reduce
directory errors even more is the journalled file system. You can
enable it in Disk Utility, and it's usually turned on for disks
onto which you install Panther. Here's how journalling works.
Before the file system changes the directory, it leaves a note on
the disk saying, "I'm going to make this change in the directory."
Then the file system makes the change, and once it finishes, it
clears the note. If the file system ever sees an incomplete change
note on the disk during startup, it knows something bad happened
and "rolls back" the directory to its previous state. You will
lose your last change, but the directory won't suffer any damage.
Other causes of damaged directories include buggy programs that
write bad data to disk, buggy programs that overwrite cached data
waiting to be written to disk, and even bugs in the file system
itself. The first two are much less likely in Mac OS X than in
Mac OS 9 because of its file privileges and memory protection,
respectively. Bugs in the file system are extremely rare but
have occurred at times in the past.
It's worth noting that directory damage is not always readily
apparent. A damaged disk may appear to operate perfectly, but
regular use can cause minor errors to grow into serious problems.
Most directory problems are easy to fix if they're caught early
but can be difficult, if not impossible, to fix later. That's why
checking and repairing startup disks automatically after a crash
is so important, and why it's essential to leave the Check Disk
option turned on in Mac OS 9's General Controls control panel
(the disk check isn't optional in Mac OS X).
To test how the disk repair utilities perform with different types
of directory errors, I created an HFS+ disk image, copied an
assortment of files and folders to it, and then used a low-level
disk editor to damage various directory data structures. I then
duplicated the damaged disk image, and let each utility try to
repair its own copy. Each utility repaired an identical disk
image, with identical damage.
**Errors Speak Volumes** -- For my first test, I started with
relatively simple damage in the volume bit map, which is also
known as the allocation file. The volume bit map tracks which
blocks on the disk contain files, and which are unused. All
five utilities fixed my damaged volume bit map easily.
Next up was damage to the volume header, which tracks vital
information about the disk, such as the amount of used and free
space, and the locations of the catalog and allocation file. The
volume header is stored at the beginning of the HFS+ partition.
I erased the volume header's signature, which makes the file
system assume the volume header is corrupt and refuse to use
the disk. Fortunately, the file system keeps a backup copy of
the volume header at the end of the disk; it's imaginatively
called the alternate volume header. All five of our utilities
successfully repaired the disk, although Drive 10 and DiskGuardian
couldn't figure out the name of the damaged disk.
**Catalog Catastrophe** -- The catalog b-tree tracks all the files
and folders on the disk. It's a vitally important part of the
directory, and many of my tests focus on it. The catalog is
divided into nodes, and each node is divided into records. Most
records track a file or folder on the disk, although some contain
threads or indexes, which are used internally by the file system
to look up files and folders.
The first node in the catalog is called the header node, which
points to other key nodes. I erased the header node. Norton Disk
Doctor, DiskWarrior, and Drive 10 recreated the header node
properly; Disk Utility and DiskGuardian failed to fix it. Once
again, Drive 10 couldn't figure out the name of the damaged disk.
The header node also contains a map which tracks which nodes are
used and which are free. I corrupted this map, but my corruption
didn't faze any of the utilities, all of which successfully fixed
the header node map.
The nodes in the catalog are linked together in a precise pattern
of connections. Horizontal links connect nodes on the same level,
and downward links connect the levels. The file system relies on
these links to look up files and folders. I damaged these links.
As happened when I erased the catalog header node, Norton Disk
Doctor, DiskWarrior, and Drive 10 fixed these links, but Disk
Utility and DiskGuardian weren't able to put the links back
together.
File and folder records are stored in alphabetical order in the
catalog. I rearranged these records, putting them in random order.
All the utilities restored the alphabetical order.
Certain characters, such as a colon, are illegal in file and
folder names. Normally, the operating system prevents you from
typing an illegal character when saving a file or creating a
folder, but it's not inconceivable that unusual circumstances
could cause one to appear. I renamed a folder with a colon by
inserting the colon directly into the folder record in the
catalog. With this test, the results start to become more
interesting. Disk Utility and DiskGuardian didn't detect any
problem. Drive 10 noticed the illegal character, but didn't
fix it. Norton Disk Doctor and DiskWarrior both fixed it
properly by replacing the colon with a legal character.
**More Catalog Corruption** -- Each catalog node ends with a map
that points back to the records in that node. I damaged the map
for one of the nodes, which sounds bad, but it's still possible
to find the records by calculating the size of each record to
find the next record. Disk Utility and Norton Disk Doctor
realized there was a problem, but they couldn't fix it. Drive 10
and DiskGuardian both identified and fixed the problem, but in
the process lost five and six files, respectively. Partial repair
isn't always better than complete failure, since you may believe
the disk was repaired successfully and only later - potentially
much later - realize that some files have been lost. DiskWarrior
fixed the catalog node map properly.
Next, I changed a thread record to be an unknown type of record,
which creates two problems. A thread record that the file system
relied upon was missing, and it was confronted by a record with
an illegal type. Disk Utility and DiskGuardian detected the
corruption, but couldn't fix it. DiskWarrior fixed the problem
but lost some of the data in one file. Only Drive 10 and Norton
Disk Doctor managed to repair my damage properly.
Note that DiskWarrior moves any files it suspects may have
problems into a folder called Rescued Items. In my tests, most of
these files turned out to be fine. This approach has the advantage
that it's clear which files may be damaged. But if the Rescued
Items folder contains many files, checking them and putting them
away can be tedious. Norton Disk Doctor can optionally put aliases
to damaged files in a folder, a potentially more helpful feature.
But in my tests it didn't work. Norton Disk Doctor also lists the
names of damaged files in its report.
The most important aspect of a file record is the location of
the file's data on the disk. One of the worst sorts of directory
damage that you may see happens when two files try to occupy the
same physical space at the same time. Different utilities refer to
this problem as "overlapping extents" or "cross-linked files." In
the best case, one file has entirely overwritten the other, since
then one file has valid data, while the other's data is completely
gone. In the worst case, the two files somehow manage to
interleave their data, which results in both being damaged beyond
repair. I cross-linked two files, thus damaging the files' catalog
records, as well as the volume header and volume bit map. Disk
Utility and DiskGuardian repaired the catalog records, the volume
header, and the volume bit map, but they didn't actually separate
the two files. In contrast, Drive 10, DiskWarrior and Norton Disk
Doctor fixed the damage and separated the files. It's important
to realize that the data in the overwritten file couldn't be
recovered, but not through any failing of these repair programs.
When one file overwrites another, the unlucky file has no chance
of surviving the encounter.
**I See Fragged People** -- Some months ago, I wrote an article
for TidBITS explaining why defragmenting disks generally
isn't worthwhile (see "Optimizing Disks Is a Waste of Time"
in TidBITS-686_). Although fragmentation is totally normal and
acceptable, serious fragmentation requires additional directory
structures, and they too can become corrupted and require repair.
<http://db.tidbits.com/getbits.acgi?tbart=07254>
Using another proprietary tool, I fragmented a disk very badly,
which breaks files into so many pieces that the file record in
the catalog b-tree can't track them all. The file system responds
by creating new records in the extents b-tree to help track all
the pieces. The extents b-tree is like the catalog b-tree, but
exists solely to help track highly fragmented files. Again, severe
fragmentation is not inherently a problem, but I had put the disk
into a precarious, if legal, state. I then damaged two extent
records so the file system couldn't find all the pieces to two
files, but I damaged each in a different way. Disk Utility,
DiskGuardian, and Drive 10 all detected my damaged extent records,
but weren't able to repair the damage. DiskWarrior and Norton Disk
Doctor performed better, fixing the problem but losing part of the
data in one file. Considering the type of damage I inflicted, they
did as well as could be expected.
HFS+ volumes are enclosed in a "wrapper," which is actually a
plain old HFS volume. The reason for the wrapper is historical.
Apple first released HFS+ with Mac OS 8.1. If you connected an
HFS+-formatted disk to a Mac running Mac OS 8.0 or earlier - in
other words, one that understood only the older HFS format, the
wrapper kept the older system from deciding the HFS+ disk was
damaged and offering to initialize it. For my next test, I damaged
the catalog b-tree header node in the wrapper. Disk Utility
and DiskGuardian didn't notice anything wrong. Drive 10 and
DiskWarrior detected the corrupt wrapper, but didn't fix it.
Norton Disk Doctor identified the damage and fixed it properly.
Disks can contain multiple partitions, which are listed in a
partition map at the beginning of the disk. Since disk images
don't have partition maps, I used an external FireWire hard disk
for this test, in which I damaged the partition map, making the
disk driver's partition overlap the HFS partition. Of all these
disk repair utilities, only Norton Disk Doctor claims to check
partition maps, and indeed it was the only one to detect the
problem, although even it proved incapable of fixing the
overlapping partition map. Luckily, damaged partition maps are
extremely rare, which may be why none of the other utilities
bother to check them.
**The Grand Finale** -- Finally, I decided to recreate the worst
damage I've ever seen on a Macintosh hard disk. Starting with the
badly fragmented disk above, I corrupted and overwrote various
parts of the catalog and extent b-trees. In some nodes I corrupted
the node header (not to be confused with the header node), in
some I munged the data records, and in others I zapped the record
offset map. A few lucky nodes suffered all three types of damage
at the hands of my disk editor. Only DiskWarrior was able to bring
the disk back to a usable state, although 35 files were either
lost or partly damaged. That 35 files were lost or damaged is
not an indictment of DiskWarrior; the program couldn't have
done any better, considering how much vital information had
been destroyed. None of the other utilities managed to repair
the disk successfully.
**And the Winner Is...** Of my 15 tests, DiskWarrior fixed 12
successfully, Norton Disk Doctor fixed 11, Drive 10 fixed 9,
DiskGuardian fixed 5, and Disk Utility fixed 4. "Fixed" includes
cases where recovery may not have been perfect, but was good
enough.
So what, in my professional opinion, should you do if your disk
starts acting up? First, try Apple's free Disk Utility. It may
fix only a limited set of problems, but when Disk Utility finds
a problem, it's invariably correct, and it applies fixes only when
it's absolutely certain it knows the correct fix. I've never seen
Disk Utility accidentally make a problem worse, something the
other utilities can do, even if only very occasionally.
If Disk Utility doesn't succeed, let DiskWarrior do battle with
your damaged directory. It was our overall winner, and it deserves
its excellent reputation. DiskWarrior can also show you a preview
of the repairs before you accept them, which lets you check that
a damaged file or folder really was fixed before DiskWarrior makes
the fix permanent.
If DiskWarrior fails, give Norton Disk Doctor a try, since it can
address some problems that DiskWarrior misses. After that, try
sacrificing chickens. Seriously, if the combination of Disk
Utility, DiskWarrior, and Norton Disk Doctor can't repair your
disk, you can either restore your data from backup, or, if that's
not possible, decide if the data is sufficiently important to pay
DriveSavers for recovery.
I still think the most important data protection utility you
should own is a backup program. But sometimes a good disk repair
program can save the day by repairing minor damage quickly so
you don't have to run through the time-consuming process of
reinitializing your hard disk and restoring from backup.
[David Shayer was a senior engineer on Norton Utilities for
Macintosh 3.0, 4.0, and 5.0. Before that he worked on Public
Utilities, a disk repair program that won the MacUser Magazine
Editor's Choice Award, and on Sedit, a low-level disk editor.]
PayBITS: If David's detailed and expert testing told you which
disk utility you should rely on, reward his efforts via PayBITS!
<http://www.amazon.com/paypage/P12NE4WQ7K8ODD>
Read more about PayBITS: <http://www.tidbits.com/paybits/>
Hot Topics in TidBITS Talk/24-Nov-03
------------------------------------
by TidBITS Staff <[EMAIL PROTECTED]>
**Jeff's video cropping odyssey** -- Jeff Porten follows up his
article about EyeTV with this description of how he stores
programs on video. (1 message)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2118>
**Faxing in Panther** -- TidBITS Talk readers report their
experiences using Panther's built-in fax software. (2 messages)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2117>
**Indie Digital Music** -- As MP3.com goes dark (including the
vaporization of its library), readers point out the quality music
that existed amongst the sludge. (2 messages)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2116>
**Stopping spam at the server** -- Charles Arthur's review of
PostArmor sparks discussion about other sever-level spam-fighting
tools. (8 messages)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2114>
**2003 Holiday Gift Ideas: Hardware** -- It's time for gizmos,
gadgets, and honkin' big Power Macs. What hardware gifts do
you suggest? (1 message)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2123>
**2003 Holiday Gift Ideas: Software** -- Hand your niece or nephew
some source code... or, be traditional and get some pre-compiled
software for the holidays. Which titles do you favor? (1 message)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2122>
**2003 Holiday Gift Ideas: Gamwa** -- Whether you prefer to save
the universe or puzzle through some good brain-benders, which
game titles do you recommend? (1 message)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2120>
**2003 Holiday Gift Ideas: Computer Miscellaneous** -- Have a gift
idea that doesn't fit into an easy category? Suggest it here!
(1 message)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2119>
**2003 Holiday Gift Ideas: For the Macintosh-minded** -- Mac users
do have other interests, you know. What would your favorite Apple
enthusiast appreciate? (1 message)
<http://db.tidbits.com/getbits.acgi?tlkthrd=2121>
$$
Non-profit, non-commercial publications may reprint articles if
full credit is given. Others please contact us. We don't guarantee
accuracy of articles. Caveat lector. Publication, product, and
company names may be registered trademarks of their companies.
This file is formatted as setext. For more information send email
to <[EMAIL PROTECTED]>. A file will be returned shortly.
For information: how to subscribe, where to find back issues,
and more, email <[EMAIL PROTECTED]>. TidBITS ISSN 1090-7017.
Send comments and editorial submissions to: <[EMAIL PROTECTED]>
Back issues available at: <http://www.tidbits.com/tb-issues/>
And: <ftp://ftp.tidbits.com/issues/>
Full text searching available at: <http://www.tidbits.com/search/>
-------------------------------------------------------------------
