On Thu, 4 May 2000, Tracy R Reed wrote:
Anyone have any tips on how to effeciently backup Maildir systems with millions
of files? I am pondering switching the company mail server over to Maildir.
It's a very large and busy system. We have had situations before where there
were millions of files to be backed up which took many days or perhaps even
weeks to fully back up. In this case we were backing up from a NetApp to a DLT
robot using NDMP. We never successfully finished a backup and ended up totally
rearchitecting that setup but this won't be possible with the mail system.
Short answer: Don't use tape.
Long answer: There are two different requirements you may be addressing;
archival copies so you can go back in time to retrieve files, and disaster
recovery such as having a lunatic empty a machine gun in to your disk
farm. The thing to worry about in the latter is time-to-restore.
We store about 7.9 million messages on a single NetApp. We don't make
archival copies as they aren't likely to capture much of the message
stream. Not having archival copies also keeps us out of messy legal
territory.
We do replicate the entire message store on a warm standby NetApp using
their snapmirror feature. We snapshot every 15 minutes, so we'll never be
more than that interval out of date if we have to go to the standby. The
snapmirror update is incremental, so we aren't transferring much data at
each interval. The snapshot mechanism is nice because it provides a clean
time delineation of what is snapped.
If you want to keep archival copies you can simply take daily/whatever
standard snapshots on the source filer and leave them online for whatever
your retention period may be. All it costs is disk space, which isn't
cheap with NetApp, but it's better than tape.
It takes about 10 minutes to switch to the standby - something we've only
done once in production (in a controlled manner for a hardware upgrade).
All that's involved is mounting file systems from the standby on our unix
boxes.
Our standby filer isn't co-located with the active one...
-- Jeff Hayward
