powermail-discuss Digest #2894 - Friday, November 12, 2010
Re(3): 2 GB limit
by "Peter Lovell" <plov...@mac.com>
Re: 2 GB limit
by "Rene Merz" <r.m...@telquel.net>
Re(2): 2 GB limit
by "Bill Schjelderup" <b...@companioncorp.com>
Re(3): 2 GB limit
by "Peter Lovell" <plov...@mac.com>
Re: 2 GB limit
by "Lane Roathe" <l...@roathe.com>
Re: 2 GB limit
by "Jeremy Hughes" <jer...@softpress.com>
Re: 2 GB limit
by "Jim Pistrang" <j...@jpcr.com>
Re: 2 GB limit
by "Michael J. Hußmann" <mich...@michael-hussmann.de>
----------------------------------------------------------------------
Subject: Re(3): 2 GB limit
From: "Peter Lovell" <plov...@mac.com>
Date: Thu, 11 Nov 2010 15:55:16 -0500
On Thu, Nov 11, 2010, George Henne <g...@nsbasic.com> wrote:
>I just checked - PowerMail is responsible for 90% of of the backup
>activity on my system. (I use TimeMachine). The problem is that if just
>one email comes in an hour (which always happens), the complete database
>gets backed up again. One database per folder would reduce the backup
>requirements dramatically. There would be less load on the system, and
>my Time Machine would be able backups for a much longer time period.
This scenario is perfect for the sparse-bundle-disk-image setup.
----------------------------------------------------------------------
Subject: Re: 2 GB limit
From: "Rene Merz" <r.m...@telquel.net>
Date: Thu, 11 Nov 2010 23:16:54 +0100
>I just checked - PowerMail is responsible for 90% of of the backup
>activity on my system. (I use TimeMachine). The problem is that if just
>one email comes in an hour (which always happens), the complete database
>gets backed up again. One database per folder would reduce the backup
>requirements dramatically. There would be less load on the system, and
>my Time Machine would be able backups for a much longer time period.
TimeMachineEditor is a free software to stop TimeMachine's every-hour-backup.
With this software you can choose and program the starting times by your own.
(I'm very satisfied with and make only four TimeMachine-backups a day.)
<http://timesoftware.free.fr/timemachineeditor/>
----------------------------------------------------------------------
Subject: Re(2): 2 GB limit
From: "Bill Schjelderup" <b...@companioncorp.com>
Date: Thu, 11 Nov 2010 15:50:55 -0700
Time machine has some very nice characteristics, but don't use it over a
network.
Check out Crashplan.com -- the basic software is free. I moved my
company to it this summer, and it's working great. The free version does
a backup once a day, or you can buy a version that can do more frequent
backups, like hourly. I use the free version to backup all my family
machines to my big home machine.
As for powermail - there are a few things I don't like, but for the most
part it does what I need and I've not seen anything else that matches
it. I get 300-500 emails a day, much of it spam, or "routine" stuff. I
toss out about 300+ messages a day. I archive certain folders every
year, if I need to search the data, I can.
When Powermail 7 comes out, I'll upgrade without a thought. I've lived
with it's little bugs and weirdness for years...I've not lost data, and
I can find what I need. I'm just happy there are developers who want to
create products that so many people are giving away.
I hope CTM can keep it's customer base so they can invest in future
releases so I hope those who need 2gb+ mail databases can get what they
want and stick with the product.
+-----------------------------------------------------------+
Bill Schjelderup, President b...@companioncorp.com
COMPanion Corporation 801-365-0555 voice
1831 Fort Union Blvd. 801-943-7752 fax
Salt Lake City, Utah 84121-3041 www.companioncorp.com
+-----------------------------------------------------------+
Nusquam est qui ubique est. - He who is everywhere is nowhere.
This email message is for the sole use of the intended recipient(s) and
may contain confidential and privileged information. Any unauthorized
review, use, disclosure, or distribution is prohibited. If you are NOT
the intended recipient, I'm sorry to bother you and will attempt to
address my messages more carefully in the future.
>
>
>>I just checked - PowerMail is responsible for 90% of of the backup
>>activity on my system. (I use TimeMachine). The problem is that if just
>>one email comes in an hour (which always happens), the complete database
>>gets backed up again. One database per folder would reduce the backup
>>requirements dramatically. There would be less load on the system, and
>>my Time Machine would be able backups for a much longer time period.
>
>TimeMachineEditor is a free software to stop TimeMachine's every-hour-backup.
>With this software you can choose and program the starting times by your own.
>(I'm very satisfied with and make only four TimeMachine-backups a day.)
><http://timesoftware.free.fr/timemachineeditor/>
>
>
>
----------------------------------------------------------------------
Subject: Re(3): 2 GB limit
From: "Peter Lovell" <plov...@mac.com>
Date: Thu, 11 Nov 2010 18:31:57 -0500
On Thu, Nov 11, 2010, George Henne <g...@nsbasic.com> wrote:
>I just checked - PowerMail is responsible for 90% of of the backup
>activity on my system. (I use TimeMachine). The problem is that if just
>one email comes in an hour (which always happens), the complete database
>gets backed up again.
Further on this ... in my instance the database is 860 MB and database
index data (in "Message Database index" package) is 101 MB. Both of
these will change for every new message.
On the other hand, the "Message Database Spotlight cache" package is 571
MB and contains about 107 K items. Most of these are 4 KB which is the
minimum allocation size for my disk.
My TM backup each hour is about 1 GB and takes about one minute.
Admittedly, this is with an internal drive, rather than sending over
Ethernet or wireless so it's not surprising that it's fast.
Trixi's comments, from experience, are instructive ...
On Thu, Nov 11, 2010, Beatrix Willius <bwill...@gmx.de> wrote:
>
>Database:
>Pro: fast useage, fast copying
>Con: if it's hosed, then it's really kaputt. However, most issues should
>be fixable by deleting the index.
>Incremental backups are not so easy.
>
>Files:
>Pro: simple, problems are usually easy to fix.
>Con: slloooowww searches. Try using AppleMail with a mailbox of 100.000
>mails. This is just a pain.
The work to be done is much the same whether the data is in a database
as we think of it, or the database consisting of files + directory. The
"files + directory" is a database by a different name, after all, just
with different performance characteristics with regard to speed of
access, time to back up, data space allocation on disk, and so on.
There's always a "directory" of some kind, either in the database or the
one in the file system.
The file system has to deal with all kinds of usage patterns so it's
optimized for an average mixture of work. But if you have more
constrained usage then you can improve performance by tailoring for
those, and that's what PM database does. You can, for example, hold a
big chunk of the directory in memory for fast access; something that a
file system won't allow you to do directly. I could go on (a lot) but
you get the point - the PM database is great for storage and access
efficiency, but not so good for incremental backup.
I'd like to see an installation option for PM to place its database on a
sparse-bundle-disk-image (for those users on Leopard and later). All the
"plumbing" for this already exists (you could do it yourself if you
wished) so it's not a big development effort. But it would be non-
trivial for testing and QA (I'm by no means saying that the total effort
is trivial, just that Apple already provides the support tools).
Regards.....Peter
----------------------------------------------------------------------
Subject: Re: 2 GB limit
From: "Lane Roathe" <l...@roathe.com>
Date: Thu, 11 Nov 2010 23:33:40 -0600
on Thu, Nov 11, 2010 "Jeremy Hughes" <jer...@softpress.com> may have said:
>>Just try to copy one 1 MB file versus 1000 files of 1 KB each
>
>How often do you copy or move your mail folder?
Every day, alternating between a firewire Flash drive and a network
share via XServer 19.6. Speed wise, here's the difference between one
large file and lots of small files:
Flash XServer
1 x 250MB large file: 35MB/s 65MB/s
1000 x 1K-4K files: 2MB/s 6MB/s
I just pruned my DB to keep it under the 2GB limit, so it only has a
little over 25,000 emails. If each of those were a separate file, it
would take me 40-140 seconds instead of 4-8 seconds to copy my file
database. Before the split my 1.8GB DB had over 200,000K emails in it,
which would take over 15 minutes on the flash drive instead of ~50
seconds, and that's assuming the flash drive could even handle that many
individual files.
And, yes, TM and other backup systems would be more efficient with the
individual files, but that is not something I really care about (I have
TM backing up my DB as well, but it happens in the background and how
much time it takes does not impact my work). These copies do server to
provide another backup of my email, but mostly they server as a way to
share that data between my desktop and laptop (which I switch back and
forth between fairly often). Maybe I could setup a rsync or something to
do that, but since simple drag copies work so well I don't have to bother.
>>So I'm all for the monolithic database approach. I don't care if a
>>backup needs to copy the whole file - copying a single file is fast.
>
>Not so fast over a network, and wasteful of backup space.
Try for Time Machine and other whole-file backup systems, but not
necessarily for a network (see timings above).
I'm all for having a DB per folder, as that would solve the size issue
while keeping the benefits of the DB. For me this would probably even be
preferable over increasing the capacity of the single DB to 4GB (or
higher), as it presents most of the same benefits (could see it
requiring more memory and slightly reduced search speeds) while reducing
the risk of a DB failure causing the loss of all email (But everyone
does have backups, right? Even individual files won't protect you from
data loss w/o a good backup strategy!).
PS: I've actually looked into switching away from PM several times as I
got closer to the 2GB limit, but when you get the # of emails I have
into a program like Mail or Entourage searches grind to a halt, taking
minutes where the same search in PM is typically 2-5 seconds. Since I do
lots of searching, I just haven't been able to justify switching. And
that's not even touching upon the absolutely horrible UIs most of the
"modern" email programs have, or their lousy handling of a dozen or more
email accounts, or...
Lots of things can be improved in PM, and it'd be great to see more
effort from CTM here. That said, I think CTM has concentrated on the
"right stuff" while other email apps have focused on becoming page
layout (ie, web page) editors, web browsers, contact managers, and all
kinds of other things instead of the items that make email usable for
people like me (not sure what that kind of person is, but it's mainly
about working with email at a level above simple getting and receiving).
Lane Roathe, CEO
Ideas From the Deep, llc <http://www.ideasfromthedeep.com>
___________________________________________________________________
Forget about world peace, visualize using your turn signals!
----------------------------------------------------------------------
Subject: Re: 2 GB limit
From: "Jeremy Hughes" <jer...@softpress.com>
Date: Fri, 12 Nov 2010 10:59:39 +0000
Lane Roathe (11/11/10, 05:33) said:
>Speed wise, here's the difference between one large file and lots of
>small files:
>
> Flash XServer
>1 x 250MB large file: 35MB/s 65MB/s
>1000 x 1K-4K files: 2MB/s 6MB/s
If you need to copy lots of small files over a network, the fastest and
most efficient way to do it is to zip them first.
Jeremy
----------------------------------------------------------------------
Subject: Re: 2 GB limit
From: "Jim Pistrang" <j...@jpcr.com>
Date: Fri, 12 Nov 2010 06:13:54 -0500
Hi Lane,
>I just pruned my DB to keep it under the 2GB limit, so it only has a
>little over 25,000 emails.
Do you really mean 25,000?
My PM database is under 600 MB with 90,000 messages.
Jim
--
Jim Pistrang
JP Computer Resources
Certified Member, Apple Consultants Network
413-256-4569
<http://www.jpcr.com>
----------------------------------------------------------------------
Subject: Re: 2 GB limit
From: "Michael J. Hußmann" <mich...@michael-hussmann.de>
Date: Fri, 12 Nov 2010 13:45:20 +0200
Jim Pistrang (j...@jpcr.com) wrote:
>> I just pruned my DB to keep it under the 2GB limit, so it only has a
>> little over 25,000 emails.
>
> Do you really mean 25,000?
>
> My PM database is under 600 MB with 90,000 messages.
For what it's worth, I have found that on average my messages take up 10
KB in the message database. Should that trend continue the 2 GB limit
would be good for 200,000 messages which would be adequate for my needs.
- Michael
Michael J. Hußmann
E-mail: mich...@michael-hussmann.de
WWW (personal): http://michael-hussmann.de
WWW (professional): http://digicam-experts.de
----------------------------------------------------------------------
End of powermail-discuss Digest
