I still don't see why you want to put this type of "space compressing"
in the IMAP layer rather then to put it in the maildrop layer where it
would do the most good.
The maildrop is the one doing the symlink and managing email ID's.
Binc just acts like a normal IMAPD reading a NFS mounted Maildir..
BUT I just wanted Binc to have a text filter when it opens each email from
the Maildir.
-- This is just a feature request... for the filter.
%%full_name%% %%company%% tags are unlikely to change, so to the end
client they
won't see a message that appears different everytime they download it.
Having the MDA sorting out if the recent 100 droped mails could be the
same and then moving it to a shared location and linking it into the
users Maildir seems to me to be the effect you want.
Yes it is. And the MDA does it.
Having the space compressed right away insteed of the MDA duplicating the
mails and
waiting for binc to HOOKS it right and _then_ compressing.
That would mean that the 10MB mail gets duplicated 100 times until the
users checks its INBOX.
Binc has nothing to do with the Symlink system other than CAT (reading) the
symlink file.
Again, this should be done directly by the MDA.
It is.
If performance and reliability is important, then you must remove the NFS
mount and store the emails physically on disk. NFS is unreliable in its
very nature, as there is no guarantee that emails are committed to disk.
The NFS is 99% ONE-WAY [read], the MDA uses sockets to store the messages on
the NFS server.
Binc would only WRITE to the NFS when a person checks their email (to
uncheck it) or moves a message.
http://www.time-travellers.org/shane/papers/NFS_considered_harmful.html
I don't think caching would be a big issue, as the client is reading their
email from
which-ever IMAPD server they connected to. [caching could be turned off
also, to ensure no latency]
If you store emails on an NFS mount and your machines crash, then email
data will certainly be lost
Depends when it crashed and what it was doing. If it was unchecking an email
then
the user would see it as unread still (minor), if it was moving a message
then as long as the headers were
written to the NFS then the symlink system can recover. [Clients create a
new file and copy the
contents of the EMAIL in the new folder when a message is moved.] A cron job
on the NFS scans for
files that are NOT symlinks... reads the ID in the headers, deletes the
email, and makes a symlink.
The client would notice that an email was deleted and that a new email
(marked as read or unread) now exists and would download it.
I've moved files back and forth in on a shell to controll spam and never
had trouble with deleting or moving maildir files.
The imap client just updates it's email list and downloads/deletes messages
as they exist in the FS.
. Also performancewise, NFS is a killer. The load on your system will be
much higher than with a physical volume.
Thats expected. Though the Gigabit network should have enough bandwidth.
There are OTHER alternatives to NFS, like SHFS and
Unless we put 3 TB of space on EVERY imap server and use Rsync, I don't see
an alternative.
-- NFS is pretty widely used in the hosting world. I can have a dozen
servers load-balance a single site.
When the user FTP's they all get updated.
If saving space is important, then of course, symlinking emails will do
some good for you. But it's much better to do what the huge email stores
do, and use a file system that eliminates duplicates for you. If you want
to use the accessor, or IMAP server, to work with symlinks and so on, then
you will be breaking important features in the mailbox specification that
directly affect reliability.
Binc IS NOT the program doing the Symlinks! It just has to read them.
I'm just asking if a Text Filter could be added to Binc when it reads a
files in the Maildir
so that the 'single email' could be customized based on the users login ID.
(personalize the email if tags were embedded)
If I was to do something like what you are describing, then I'd define a
specialized mailbox format that stores IDs only, with message bodies
stored in a shared collection. Then I'd write a backend for the mailstore
accessors that allowed them to deal with this format..
Already done. It's now part of the MTA. We noticed that our Tagged Emails
now break as
there is nothing to filter them. The MTA can't filter them as a dozen or
more people read the same one.
In any case, if you start messing with message contents after they have
been delivered to a Maildir mailbox (or any other mailbox), then IMAP
clients are at risk to go haywire... :-)
The client should never notice the contents changing. The filter (added to
Binc)
would make the same replacements, time and again when the user RE-downloads
a message.
-- Clients usually cache known/read emails and don't redownload unless told
to. As the filename
is the ID, I'd doubt Binc would have to refilter the same message more than
once a month.
Notes that the Filter is actually a 3rd party perl script (yes, there is a
performance hit)
I just wanted to know or ask if I could get a HOOK in the fileopen section
of binc to call the script, if it exists.
