on Thu, Nov 11, 2010 "Jeremy Hughes" <[email protected]> may have said:
>>Just try to copy one 1 MB file versus 1000 files of 1 KB each
>
>How often do you copy or move your mail folder?
Every day, alternating between a firewire Flash drive and a network
share via XServer 19.6. Speed wise, here's the difference between one
large file and lots of small files:
Flash XServer
1 x 250MB large file: 35MB/s 65MB/s
1000 x 1K-4K files: 2MB/s 6MB/s
I just pruned my DB to keep it under the 2GB limit, so it only has a
little over 25,000 emails. If each of those were a separate file, it
would take me 40-140 seconds instead of 4-8 seconds to copy my file
database. Before the split my 1.8GB DB had over 200,000K emails in it,
which would take over 15 minutes on the flash drive instead of ~50
seconds, and that's assuming the flash drive could even handle that many
individual files.
And, yes, TM and other backup systems would be more efficient with the
individual files, but that is not something I really care about (I have
TM backing up my DB as well, but it happens in the background and how
much time it takes does not impact my work). These copies do server to
provide another backup of my email, but mostly they server as a way to
share that data between my desktop and laptop (which I switch back and
forth between fairly often). Maybe I could setup a rsync or something to
do that, but since simple drag copies work so well I don't have to bother.
>>So I'm all for the monolithic database approach. I don't care if a
>>backup needs to copy the whole file - copying a single file is fast.
>
>Not so fast over a network, and wasteful of backup space.
Try for Time Machine and other whole-file backup systems, but not
necessarily for a network (see timings above).
I'm all for having a DB per folder, as that would solve the size issue
while keeping the benefits of the DB. For me this would probably even be
preferable over increasing the capacity of the single DB to 4GB (or
higher), as it presents most of the same benefits (could see it
requiring more memory and slightly reduced search speeds) while reducing
the risk of a DB failure causing the loss of all email (But everyone
does have backups, right? Even individual files won't protect you from
data loss w/o a good backup strategy!).
PS: I've actually looked into switching away from PM several times as I
got closer to the 2GB limit, but when you get the # of emails I have
into a program like Mail or Entourage searches grind to a halt, taking
minutes where the same search in PM is typically 2-5 seconds. Since I do
lots of searching, I just haven't been able to justify switching. And
that's not even touching upon the absolutely horrible UIs most of the
"modern" email programs have, or their lousy handling of a dozen or more
email accounts, or...
Lots of things can be improved in PM, and it'd be great to see more
effort from CTM here. That said, I think CTM has concentrated on the
"right stuff" while other email apps have focused on becoming page
layout (ie, web page) editors, web browsers, contact managers, and all
kinds of other things instead of the items that make email usable for
people like me (not sure what that kind of person is, but it's mainly
about working with email at a level above simple getting and receiving).
Lane Roathe, CEO
Ideas From the Deep, llc <http://www.ideasfromthedeep.com>
___________________________________________________________________
Forget about world peace, visualize using your turn signals!
