Re: procmail vs dovecote (was Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?)

2012-11-11 Thread Chris Green
On Sat, Nov 10, 2012 at 10:37:04PM +, Tony's unattended mail wrote:
> > However, I find dovecot deliver (which uses the sieve language 
> > for filtering) to be much more readable/writable than procmail.
> 
> Sieve does not include regular expressions -- I shit you not.
> 
> Dovecote needs regular expression capability to be shoe-horned in by
> some hokey plugin.  Regular expressions are quite fundamental to a
> mail filtering language that has an appropriate amount of expressive
> power.  It's bizarre that sieve is presented as a thought out
> successor to procmail complete with an RFC, and yet it excludes
> something as essential as regular expressions.
> 
> I'm resisting sieve because the C-style makes the code look cluttered,
> and it lacks expressive power.  The one aspect that may compel a
> switch to sieve is the fact that it is MIME-aware.  MIME predates
> procmail, and it's a shame that procmail has become unmaintained.
> 
> OTOH, I might rather have third party tools for MIME than third party
> tools for regular expressions.

I simply have a custom script written in Python, hence I have all the RE
and/or other technology I need without much effort.

My filter rules *aren't* written in Python or any language as such, they
are in a format that is as user-friendly (well, me friendly) as possible
and are thus a straight text file with very little special syntax.  The
*program* takes care of that, it's what computers are good at, why
should I faff about with funny characters, layout, etc. when the
computer can do it all for me.

My filter file has the following format:-

cheddar Li  cheddar-us...@lists.halon.org.uk
daboLi  dabo-us...@leafe.comdabo-users
dbacruising Li  dbacruis...@lists.shire.net
dbamain Li  dbam...@lists.shire.net
dbamatters  Li  dbaassociationmatt...@lists.shire.net
dbasocial   Li  dbasoc...@lists.shire.net
dia Li  dia-l...@gnome.org
digitempLi  digit...@googlegroups.com
dnsmasq Li  dnsmasq-disc...@lists.thekelleys.org.uk 
Dnsmasq-discuss

Where the first column is both the mutt alias for the list *and* the
directory (under Li) where the incoming list mail is stored.  The second
column is a destination directory (some things get directed to a Ju -
junk - hierarchy).  The third column is a string to match in either To:
or Cc:. The fourth (optional) column is a string to remove from the
subject line if found between [].

My Python script to implement all this is only a 100 lines or so of code
in total.

When I subscribe to a new mailing list I just add the appropriate line
to the above filter file and that's it, nothing else to do at all.  A
couple of very simple scripts get what's needed from the file to provide
(as I said) a mutt alias for the list and to add the list to provide
what is needed for mutt 'lists', 'subscribe' and 'mailboxes'.

... and as I was saying the filter file itself is in an incredibly
simple format, no XML, no indenting, no block structure, no funny
characters required.  About the only 'special' thing is that you can add
comments by having lines starting with a #.


-- 
Chris Green


Re: procmail vs dovecote (was Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?)

2012-11-10 Thread Jamie Paul Griffin
/ Tony's unattended mail wrote on Sat 10.Nov'12 at 22:37:04 + /

> > However, I find dovecot deliver (which uses the sieve language 
> > for filtering) to be much more readable/writable than procmail.
> 
> Sieve does not include regular expressions -- I shit you not.
> 
> Dovecote needs regular expression capability to be shoe-horned in by
> some hokey plugin.  Regular expressions are quite fundamental to a
> mail filtering language that has an appropriate amount of expressive
> power.  It's bizarre that sieve is presented as a thought out
> successor to procmail complete with an RFC, and yet it excludes
> something as essential as regular expressions.
> 
> I'm resisting sieve because the C-style makes the code look cluttered,
> and it lacks expressive power.  The one aspect that may compel a
> switch to sieve is the fact that it is MIME-aware.  MIME predates
> procmail, and it's a shame that procmail has become unmaintained.
> 
> OTOH, I might rather have third party tools for MIME than third party
> tools for regular expressions.

For what it's worth, I have used both of the above and now I only use procmail. 
Some experienced folk like Cameron have made very good points about its 
problems, but for me, it does the job nicely. I've been using it for about 3 
years, maybe more and i still am learning about its complexities but it's a 
good tool in my opinion. I once had a set up that used procmail recipies in 
conjunction with dovecot deliver and that worked great for me. But I work from 
home all the time now so have no need for dovecot or any other IMAP server, not 
yet anyway.


procmail vs dovecote (was Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?)

2012-11-10 Thread Tony's unattended mail
> However, I find dovecot deliver (which uses the sieve language 
> for filtering) to be much more readable/writable than procmail.

Sieve does not include regular expressions -- I shit you not.

Dovecote needs regular expression capability to be shoe-horned in by
some hokey plugin.  Regular expressions are quite fundamental to a
mail filtering language that has an appropriate amount of expressive
power.  It's bizarre that sieve is presented as a thought out
successor to procmail complete with an RFC, and yet it excludes
something as essential as regular expressions.

I'm resisting sieve because the C-style makes the code look cluttered,
and it lacks expressive power.  The one aspect that may compel a
switch to sieve is the fact that it is MIME-aware.  MIME predates
procmail, and it's a shame that procmail has become unmaintained.

OTOH, I might rather have third party tools for MIME than third party
tools for regular expressions.



Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-09 Thread Chris Bannister
On Fri, Nov 09, 2012 at 02:49:48PM +0200, Nikola Petrov wrote:
> On Fri, Nov 09, 2012 at 11:17:06PM +1300, Chris Bannister wrote:
> > On Thu, Nov 08, 2012 at 03:17:35AM +0200, Nikola Petrov wrote:
> > > No it doesn't deliver them to you. It sort of filters them online on the
> > > server. You can then use something like offlineimap to deliver them
> > > locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
> > > far from happy with my setup at the moment.
> > 
> > So what would you prefer?
> 
> Hi,
> 
> I am not sure I understand your question?

Ummm, Is "What would make you more happy?" a better question?

-- 
"If you're not careful, the newspapers will have you hating the people
who are being oppressed, and loving the people who are doing the 
oppressing." --- Malcolm X


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-09 Thread Chris Bannister
On Thu, Nov 08, 2012 at 03:17:35AM +0200, Nikola Petrov wrote:
> No it doesn't deliver them to you. It sort of filters them online on the
> server. You can then use something like offlineimap to deliver them
> locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
> far from happy with my setup at the moment.

So what would you prefer?

-- 
"If you're not careful, the newspapers will have you hating the people
who are being oppressed, and loving the people who are doing the 
oppressing." --- Malcolm X


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Jamie Paul Griffin
/ Chris Green wrote on Thu  8.Nov'12 at 18:13:10 + /

> On Thu, Nov 08, 2012 at 11:06:35AM -0600, Derek Martin wrote:
> > On Thu, Nov 08, 2012 at 01:03:07PM +, Jamie Paul Griffin wrote:
> > > Hi Chris, personally, i'd stick with what your current set-up. 
> > 
> > Ditto.  I don't currently do this but that's only because port 25 is
> > blocked by my ISP.  I've run my mail this way before and would do it
> > again if it were a practical option.
> > 
> Er, we're talking (or at least I'm talking) about mail delivery *from*
> your ISP to your computer at home (or wherever).  Your ISP blocking port
> 25 won't affect this at all, it's your receiving computer that needs to
> have port 25 open.
> 
> -- 
> Chris Green

Yes as far as I know, the ISP blocking port 25 thing is for sending mail out, 
in which case you can use your mta to send through a smarthost. Receiving I set 
up DNS to forward mail to my IP, and port forwarding on my router at home to 
the machine that has an smtp daemon listening on port 25 or as Patrick said, 
listening on a different port like the submission port for example. 


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Chris Green
On Thu, Nov 08, 2012 at 11:06:35AM -0600, Derek Martin wrote:
> On Thu, Nov 08, 2012 at 01:03:07PM +, Jamie Paul Griffin wrote:
> > Hi Chris, personally, i'd stick with what your current set-up. 
> 
> Ditto.  I don't currently do this but that's only because port 25 is
> blocked by my ISP.  I've run my mail this way before and would do it
> again if it were a practical option.
> 
Er, we're talking (or at least I'm talking) about mail delivery *from*
your ISP to your computer at home (or wherever).  Your ISP blocking port
25 won't affect this at all, it's your receiving computer that needs to
have port 25 open.

-- 
Chris Green


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Patrick Shanahan
* Derek Martin  [11-08-12 12:06]:
> On Thu, Nov 08, 2012 at 01:03:07PM +, Jamie Paul Griffin wrote:
> > Hi Chris, personally, i'd stick with what your current set-up. 
> 
> Ditto.  I don't currently do this but that's only because port 25 is
> blocked by my ISP.  I've run my mail this way before and would do it
> again if it were a practical option.

Why not change postfix master.cf to use another port > 1024?  Not a big
thing, one line to edit and restart postfix.


-- 
(paka)Patrick Shanahan   Plainfield, Indiana, USA  HOG # US1244711
http://wahoo.no-ip.orgPhoto Album: http://wahoo.no-ip.org/gallery2
http://en.opensuse.org   openSUSE Community Member
Registered Linux User #207535@ http://linuxcounter.net


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Derek Martin
On Thu, Nov 08, 2012 at 01:03:07PM +, Jamie Paul Griffin wrote:
> Hi Chris, personally, i'd stick with what your current set-up. 

Ditto.  I don't currently do this but that's only because port 25 is
blocked by my ISP.  I've run my mail this way before and would do it
again if it were a practical option.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpYNShC3vIom.pgp
Description: PGP signature


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Mark H. Wood
On Wed, Nov 07, 2012 at 10:48:45AM -0600, Derek Martin wrote:
> On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> > What does everyone else here do for collecting mail and filtering mail
> > with mutt?
> 
> Fetchmail and procmail.  Ugly, but ubiquitous and reliable.  A friend
> pointed me at something "better" for mail filtering, but I can't
> recall what it was... mainly because I haven't gotten around to
> looking into it, on account of the fact that my current solution works
> well and requires no learning curve.  Learning a new mail filter
> system is very low on the priority list.

I'm guessing 'sieve'.  Haven't tried it.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Asking whether markets are efficient is like asking whether people are smart.


pgpY9juePS2uk.pgp
Description: PGP signature


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Mark H. Wood
fetchmail + maildrop works for me.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Asking whether markets are efficient is like asking whether people are smart.


pgpGksnsN8kgQ.pgp
Description: PGP signature


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Jamie Paul Griffin
/ Chris Green wrote on Thu  8.Nov'12 at 10:51:59 + /

> On Wed, Nov 07, 2012 at 04:33:58PM -0600, David Champion wrote:
> > * On 07 Nov 2012, Derek Martin wrote: 
> > > On Wed, Nov 07, 2012 at 08:48:08PM +, Chris Green wrote:
> > > > server retrying if my SMTP server isn't running (or connected).  That's
> > > > one of the reasons I'd quite like to move away from SMTP.  It *should*
> > > > be OK but I'm relying on the other end to behave properly.
> > > 
> > > It will.  It has to.  If it didn't, e-mail on the internet would be
> > > horribly unreliable.
> > 
> > I hate to break it to you, but :)
> > 
> > I've used IMAP pickup in the past and it's OK for some IMAP servers.  A
> > year or two ago my employer moved my mailbox to MS Exchange.  Exchange
> > doesn't (necessarily?) hand you the exact e-mail it received.  It
> > parses incoming mail, stores the parsed components, and reconstructs
> > the message the best it can figure when you pick it up via IMAP or POP.
> > Along the way it might modify or remove some components for no good
> > reason; for example, multipart/alternative with text/plan and text/html
> > invisibly becomes just a text/html message.  I've also heard of its
> > breaking crypto, although I haven't seen that myself for a while.
> > 
> > So I forward my mail via SMTP away from my employer now.
> > 
> Yes, I realise there are good and bad places that deliver mail to you! :-)
> 
> If I move from SMTP delivery to collecting it myself (from POP3 or IMAP)
> it will be from exactly the same source, my Tsohost web hosting service
> which (in many people's opinions) is excellent.  
> 
> All I have to do is change the ultimate destination of all my E-Mail
> from zbmc.eu (whose MX record points at my home LAN) to a POP3 mailbox
> on the TsoHost servers and then collect from that POP3 mailbox.
> 
> I'm thinking in fact that I'm going to stay with much the same system as
> I already have but my Python filter script will collect E-Mail direct
> from the Tsohost POP3 server instead of having it fed into its standard
> input by the .forward.  I already have *another* Python script that
> collects mail from a Tsohost POP3 box so I know how to do that already,
> all I need to do is merge a few bits of existing code.
> 
> -- 
> Chris Green

Hi Chris, personally, i'd stick with what your current set-up. Since I set up 
my own mta and security software i've never been happier with it. I much prefer 
the flexibility of being able to control almost all aspects of my mail 
delivery, reading and sending; including the DNS configuration etc... 

What I did do, though, is look into OS's and software that provide the security 
we all need, whether collecting mail from remote IMAP storage or POP3, or 
having it routed directly using smtp. I stick with BSD systems and use a number 
of different security software on my end to deal with zombies, malware 
filtering, spam filtering, etc. They are all generally well documented as you 
no doubt know, and certainly great efforts have been made to make installation 
and configuration of these software easy(-ish). 

The only downfall I think is the time involved with setting up and upgrading 
and monitoring these things. So, do you have the time and will to invest in 
doing that? As you've written some nice tools already, it sounds like you know 
what you're doing so stick with it i'd say. Plus, for me, I find it great fun 
and interesting learning about these things. I've learned loads by getting 
stuck in and engrossed in it all. Some people, however, have been there and 
done that and simply just can't be bothered with it anymore and/or would rather 
spend their time on something else. 

Jamie


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Chris Green
On Wed, Nov 07, 2012 at 04:33:58PM -0600, David Champion wrote:
> * On 07 Nov 2012, Derek Martin wrote: 
> > On Wed, Nov 07, 2012 at 08:48:08PM +, Chris Green wrote:
> > > server retrying if my SMTP server isn't running (or connected).  That's
> > > one of the reasons I'd quite like to move away from SMTP.  It *should*
> > > be OK but I'm relying on the other end to behave properly.
> > 
> > It will.  It has to.  If it didn't, e-mail on the internet would be
> > horribly unreliable.
> 
> I hate to break it to you, but :)
> 
> I've used IMAP pickup in the past and it's OK for some IMAP servers.  A
> year or two ago my employer moved my mailbox to MS Exchange.  Exchange
> doesn't (necessarily?) hand you the exact e-mail it received.  It
> parses incoming mail, stores the parsed components, and reconstructs
> the message the best it can figure when you pick it up via IMAP or POP.
> Along the way it might modify or remove some components for no good
> reason; for example, multipart/alternative with text/plan and text/html
> invisibly becomes just a text/html message.  I've also heard of its
> breaking crypto, although I haven't seen that myself for a while.
> 
> So I forward my mail via SMTP away from my employer now.
> 
Yes, I realise there are good and bad places that deliver mail to you! :-)

If I move from SMTP delivery to collecting it myself (from POP3 or IMAP)
it will be from exactly the same source, my Tsohost web hosting service
which (in many people's opinions) is excellent.  

All I have to do is change the ultimate destination of all my E-Mail
from zbmc.eu (whose MX record points at my home LAN) to a POP3 mailbox
on the TsoHost servers and then collect from that POP3 mailbox.

I'm thinking in fact that I'm going to stay with much the same system as
I already have but my Python filter script will collect E-Mail direct
from the Tsohost POP3 server instead of having it fed into its standard
input by the .forward.  I already have *another* Python script that
collects mail from a Tsohost POP3 box so I know how to do that already,
all I need to do is merge a few bits of existing code.

-- 
Chris Green


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-08 Thread Chris Green
On Thu, Nov 08, 2012 at 03:17:35AM +0200, Nikola Petrov wrote:
> On Wed, Nov 07, 2012 at 05:35:45PM +, Chris Green wrote:
> > > I am using imapfilter with lua configuration file for my imap account.
> > > That does the job for me and I like the fact that I declare my filters
> > > with actual code(be it lua, python or what not)
> > > 
> > How does that actually work?  I've found its home at 
> > https://github.com/lefcha/imapfilter
> > but the documentation doesn't really tell me what it does (maybe my fault!).
> > 
> > Does it move E-Mails around on the IMAP server, or does it collect them
> > from the IMAP server and deliver them to you locally?  Or does it do
> > something else?
> 
> No it doesn't deliver them to you. It sort of filters them online on the
> server. You can then use something like offlineimap to deliver them
> locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
> far from happy with my setup at the moment.
> 
OK, thanks for the description, I don't think it's quite where I want to
be. 

-- 
Chris Green


Re: mutt + exchange woes (Was: Re: What are the current fetchmail/getmail and/or procmail/maildrop) utilities?

2012-11-07 Thread David Champion
* On 07 Nov 2012, Jeremy Kitchen wrote: 
> 
> I haven't had it break crypto, but I'm one of 2 people at the company
> doing pgp signatures and both of us send *only* text/plain.

My memory is fuzzy but I think it was more complex multipart signed
messages that it broke.


> I have had it give me text/plain only when there was an html part, which
> normally I wouldn't complain about, but if someone used an html link in
> their email, I *never* see the link or the url.

I may have it backwards.  OTOH it may have both problems.


> Otherwise, mutt seems to work just fine with exchange. I do need to set
> up lbdb to pull from our exchange server at some point, but fortunately
> I interact with only a very small subset of the company, so my aliases
> file suffices for this, and if I need to look up someone's address I can
> always open up OWA.

True, mutt does pretty well.  If you're compelled to use Exchange, mutt
remains a good option.

You can set a query_command that looks people up in AD, theoretically.
I've never done this but I suspect that I could.  Maybe that's
functionally what you mean to do with lbdb, though?

(Our campus uses both AD and LDAP, and LDAP is used for first-pass mail
routing, so I'm querying it alone for now.

-- 
David Champion • d...@bikeshed.us


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?]

2012-11-07 Thread Ken Moffat
Doh!  My reply went to Peter instead of list!  Bad mutt!

On Wed, Nov 07, 2012 at 04:00:19PM -0500, Peter Davis wrote:
> On Wed, Nov 07, 2012 at 10:48:45AM -0600, Derek Martin wrote:
> > On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> > > What does everyone else here do for collecting mail and filtering mail
> > > with mutt?
> > 
> > Fetchmail and procmail.  Ugly, but ubiquitous and reliable.  
> 
> Same here. I keep meaning to hook in an adaptive spam filter, but I
> haven't bothered so far. Maybe mutt just makes it so easy to quickly
> triage my mail that it hasn't seemed worth it.
> 
 /me too - the weirdnesses of procmail do my head in (too many
things which get routed to deviant mailboxes, and a general need to
allow multiple mailboxes for anything, such as lkml, which exceeds
about 51 MB), but I have a setup which works _adequately_ [ had to
add a 4th lkml mailbox at the end of last month : my mailboxes get
rolled-over at the start of the month ].

ĸen
-- 
das eine Mal als Tragödie, das andere Mal als Farce



Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Jeremy Kitchen
On Thu, Nov 08, 2012 at 01:06:54AM +0100, Andre Klärner wrote:
> On Wed, Nov 07, 2012 at 11:21:59PM +, Jamie Paul Griffin wrote:
> > Yes i think the benefits of using your own smtp delivery are worth it.
> 
> I can only agree. And to avoid issues when my landline is down I have a VM
> on a big hoster that on one side delivers all my locally generated mails to
> avoid the dialin IP address problem. And on the other side it acts as the
> backup MX that stores my mails until my landline is back online and it can
> be delivered at my home.

IMO, a better way to do this would be to have your current backup MX be
your primary (and only?) and set it to have a high retry time, possibly
even setting up something like ETRN[1] to trigger the remote MTA to flush
its queue to you.

You may also be able to configure your mail server to have a separate
retry time for incoming vs outgoing mail. I *think* postfix can do this?
Seems like something it would be able to do.

[1]: http://www.postfix.org/ETRN_README.html

I'm sure other MTAs support ETRN, but that was the first hit on google,
and I'm a postfix user (retired qmail user, and qmail does *not* support
ETRN) so it seemed prudent :)

-Jeremy


pgpAgBZsvGkf0.pgp
Description: PGP signature


mutt + exchange woes (Was: Re: What are the current fetchmail/getmail and/or procmail/maildrop) utilities?

2012-11-07 Thread Jeremy Kitchen
On Wed, Nov 07, 2012 at 04:33:58PM -0600, David Champion wrote:
> I've used IMAP pickup in the past and it's OK for some IMAP servers.  A
> year or two ago my employer moved my mailbox to MS Exchange.  Exchange
> doesn't (necessarily?) hand you the exact e-mail it received.  It
> parses incoming mail, stores the parsed components, and reconstructs
> the message the best it can figure when you pick it up via IMAP or POP.
> Along the way it might modify or remove some components for no good
> reason; for example, multipart/alternative with text/plan and text/html
> invisibly becomes just a text/html message.  I've also heard of its
> breaking crypto, although I haven't seen that myself for a while.

I haven't had it break crypto, but I'm one of 2 people at the company
doing pgp signatures and both of us send *only* text/plain.

I have had it give me text/plain only when there was an html part, which
normally I wouldn't complain about, but if someone used an html link in
their email, I *never* see the link or the url.

> So I forward my mail via SMTP away from my employer now.

I would love to do this, if for no other reason than I can have better
server-side filtering, but I very highly doubt the company would go for
it.

Otherwise, mutt seems to work just fine with exchange. I do need to set
up lbdb to pull from our exchange server at some point, but fortunately
I interact with only a very small subset of the company, so my aliases
file suffices for this, and if I need to look up someone's address I can
always open up OWA.

-Jeremy


pgpvBZQTthinp.pgp
Description: PGP signature


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Cameron Simpson
On 07Nov2012 14:15, Chris Green  wrote:
| I also have a fairly complex mail filtering script I wrote myself in
| Python which is fed mail via .forward.
| 
| What's the current "state of the art" way to collect mail and deliver it
| through a filtering system to mutt?  If I can do this all in one program
| than so much the better but I'm happy with two programs if that would
| work better.  I can stay with my existing filter system but, again, if I
| can consolidate things into one, easier to maintain, chunk then I'd be
| happy. 
| 
| I *don't* like procmail configuration files, they're one of the reasons
| I wrote my own.
| 
| What does everyone else here do for collecting mail and filtering mail
| with mutt?

I collect email with getmail and deliver it to my "spool" folder.

I file mail with "mailfiler", a python program of my own, to monitor the
"spool", "spool-in", "spool-out" and "spool-spam-subj" maildirs.  It
understands extremely easy to write and read filter rules, eg:

  =spam SPAM-SUBJ subject:/^You have 24 hours left to TRIPLE your deposits

  !=me,$EMAIL_IPHONE .  to,cc:(ME)
from:(FAMILY|FRIENDS|[...snipped: other mail group 
names])

  muttMutt-Usersmutt-users@mutt.org

The first rule uses a regexp on the subject header and diverts matching
messages to my "spam" mail folder/septic-tank.

The second rule does proper address parsing of the named headers and (in
the example above) checks addresses against sets of addresses in my
maildb. Almost instant, and very reliable. The "!" means issue an
"alert", for "important" messages.

The third rule also does a proper address parse of to/cc/bcc and if
"mitt-us...@mutt.org" is there, saves the message in the "mutt" folder
with the "Mutt-Users" X-Label header added.

I've actually described my setup at some length on the list just
recently, here and here:

  http://www.mail-archive.com/mutt-users@mutt.org/msg45217.html
  http://www.mail-archive.com/mutt-users@mutt.org/msg45215.html

outlining my setup, why I don't use procmail, what I used to do to beat
procmail into order in the past, etc.

In short, the rules for "spool" divert spam and copy other messages to
"spool-in". The rules for "spool-in" file messages into my main inbox
and multitudinous other folders for mailing lists. "spool-out" is for
cross filing copies of my outbound email.

Aside: "spool-spam-subj" is for recording the subject lines of all messages
  filed there as to be considered spam. So I've got a mutt macro to save a
  repetitious spam to that folder, and the rules there add a new rule to my
  spam filter:-) Mailfiler notices rule updates on the fly.

Because mailfiler runs in the background, polling very regularly (1Hz,
and the machine load for that is insignificant), if I modify my rules
all I need to do to refile a message via the rules is to save it into
"spool-in" again. And off it goes. Of course there is a log file to tail
to watch this stuff happen.

There's a man page for the mailfiler rule syntax here:

  https://bitbucket.org/cameron_simpson/css/src/tip/man/mailfiler.5.pod

and for mailfiler itself here:

  https://bitbucket.org/cameron_simpson/css/src/tip/man/mailfiler.1.pod

Source code:
  
  
https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/app/mailfiler.py

Cheers,
-- 
Cameron Simpson 

On 12/22/07, Brian Hansen  wrote:   
> 2. Rather than auditing a lot of code, correcting a lot of coding mistakes,
> like the OpenBSD security team has done, and still do, why not shift from C
> to something, just as fast and powerfull as C, but more secure? Again like
> Ada. (to completely avoid the possibilities of those errors).
why did you write your email in english?  esperanto is simpler and
less error-prone.
- Ted Unangst  in misc.openbsd.org


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Andre Klärner
On Wed, Nov 07, 2012 at 11:21:59PM +, Jamie Paul Griffin wrote:
> Yes i think the benefits of using your own smtp delivery are worth it.

I can only agree. And to avoid issues when my landline is down I have a VM
on a big hoster that on one side delivers all my locally generated mails to
avoid the dialin IP address problem. And on the other side it acts as the
backup MX that stores my mails until my landline is back online and it can
be delivered at my home.

regards, 
Andre

-- 
Andre Klärner


smime.p7s
Description: S/MIME cryptographic signature


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Jamie Paul Griffin
/ David Champion wrote on Wed  7.Nov'12 at 16:33:58 -0600 /

> * On 07 Nov 2012, Derek Martin wrote: 
> > On Wed, Nov 07, 2012 at 08:48:08PM +, Chris Green wrote:
> > > server retrying if my SMTP server isn't running (or connected).  That's
> > > one of the reasons I'd quite like to move away from SMTP.  It *should*
> > > be OK but I'm relying on the other end to behave properly.
> > 
> > It will.  It has to.  If it didn't, e-mail on the internet would be
> > horribly unreliable.
> 
> I hate to break it to you, but :)
> 
> I've used IMAP pickup in the past and it's OK for some IMAP servers.  A
> year or two ago my employer moved my mailbox to MS Exchange.  Exchange
> doesn't (necessarily?) hand you the exact e-mail it received.  It
> parses incoming mail, stores the parsed components, and reconstructs
> the message the best it can figure when you pick it up via IMAP or POP.
> Along the way it might modify or remove some components for no good
> reason; for example, multipart/alternative with text/plan and text/html
> invisibly becomes just a text/html message.  I've also heard of its
> breaking crypto, although I haven't seen that myself for a while.
> 
> So I forward my mail via SMTP away from my employer now.
> 
> -- 
> David Champion • d...@bikeshed.us

Yes i think the benefits of using your own smtp delivery are worth it.


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Derek Martin
On Wed, Nov 07, 2012 at 04:33:58PM -0600, David Champion wrote:
> * On 07 Nov 2012, Derek Martin wrote: 
> > On Wed, Nov 07, 2012 at 08:48:08PM +, Chris Green wrote:
> > > server retrying if my SMTP server isn't running (or connected).  That's
> > > one of the reasons I'd quite like to move away from SMTP.  It *should*
> > > be OK but I'm relying on the other end to behave properly.
> > 
> > It will.  It has to.  If it didn't, e-mail on the internet would be
> > horribly unreliable.
> 
> I hate to break it to you, but :)
> 
> I've used IMAP pickup in the past and it's OK for some IMAP servers.  

Right, we're talking about SMTP delivery to a machine that's sometimes
down or unreachable...  Very reliable.  :)  Presumably that's exactly
why you're using SMTP to deal with your exchange problem!

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpjzCLzrNrxL.pgp
Description: PGP signature


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread David Champion
* On 07 Nov 2012, Derek Martin wrote: 
> On Wed, Nov 07, 2012 at 08:48:08PM +, Chris Green wrote:
> > server retrying if my SMTP server isn't running (or connected).  That's
> > one of the reasons I'd quite like to move away from SMTP.  It *should*
> > be OK but I'm relying on the other end to behave properly.
> 
> It will.  It has to.  If it didn't, e-mail on the internet would be
> horribly unreliable.

I hate to break it to you, but :)

I've used IMAP pickup in the past and it's OK for some IMAP servers.  A
year or two ago my employer moved my mailbox to MS Exchange.  Exchange
doesn't (necessarily?) hand you the exact e-mail it received.  It
parses incoming mail, stores the parsed components, and reconstructs
the message the best it can figure when you pick it up via IMAP or POP.
Along the way it might modify or remove some components for no good
reason; for example, multipart/alternative with text/plan and text/html
invisibly becomes just a text/html message.  I've also heard of its
breaking crypto, although I haven't seen that myself for a while.

So I forward my mail via SMTP away from my employer now.

-- 
David Champion • d...@bikeshed.us


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Nikola Petrov
On Wed, Nov 07, 2012 at 05:35:45PM +, Chris Green wrote:
> On Wed, Nov 07, 2012 at 07:17:46PM +0200, Nikola Petrov wrote:
> > On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> > > I currently have my mail delivered to my desktop system using SMTP as
> > > the system is on all the time and has a static IP.
> > > 
> > > However I always get paranoid when I reconfigure it and/or do other
> > > maintenance so I'm considering moving back to a fetchmail/getmail based
> > > system.   
> > > 
> > > I also have a fairly complex mail filtering script I wrote myself in
> > > Python which is fed mail via .forward.
> > > 
> > > 
> > > What's the current "state of the art" way to collect mail and deliver it
> > > through a filtering system to mutt?  If I can do this all in one program
> > > than so much the better but I'm happy with two programs if that would
> > > work better.  I can stay with my existing filter system but, again, if I
> > > can consolidate things into one, easier to maintain, chunk then I'd be
> > > happy. 
> > > 
> > > 
> > > I *don't* like procmail configuration files, they're one of the reasons
> > > I wrote my own.
> > > 
> > > 
> > > What does everyone else here do for collecting mail and filtering mail
> > > with mutt?
> > 
> > I am using imapfilter with lua configuration file for my imap account.
> > That does the job for me and I like the fact that I declare my filters
> > with actual code(be it lua, python or what not)
> > 
> How does that actually work?  I've found its home at 
> https://github.com/lefcha/imapfilter
> but the documentation doesn't really tell me what it does (maybe my fault!).
> 
> Does it move E-Mails around on the IMAP server, or does it collect them
> from the IMAP server and deliver them to you locally?  Or does it do
> something else?

No it doesn't deliver them to you. It sort of filters them online on the
server. You can then use something like offlineimap to deliver them
locally to you. I use imapfilter + offlineimap + notmuch + mutt and I am
far from happy with my setup at the moment.

Best, Nikola


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Derek Martin
On Wed, Nov 07, 2012 at 08:48:08PM +, Chris Green wrote:
> No specific "protective measures" at all, it just relies on the sending
> server retrying if my SMTP server isn't running (or connected).  That's
> one of the reasons I'd quite like to move away from SMTP.  It *should*
> be OK but I'm relying on the other end to behave properly.

It will.  It has to.  If it didn't, e-mail on the internet would be
horribly unreliable.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpJffX3FQ2q2.pgp
Description: PGP signature


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Peter Davis
On Wed, Nov 07, 2012 at 10:48:45AM -0600, Derek Martin wrote:
> On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> > What does everyone else here do for collecting mail and filtering mail
> > with mutt?
> 
> Fetchmail and procmail.  Ugly, but ubiquitous and reliable.  

Same here. I keep meaning to hook in an adaptive spam filter, but I
haven't bothered so far. Maybe mutt just makes it so easy to quickly
triage my mail that it hasn't seemed worth it.

-pd




-- 

   Peter Davis
   The Tech Curmudgeon
  http://www.techcurmudgeon.com


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Chris Green
On Wed, Nov 07, 2012 at 08:16:42PM +, Jamie Paul Griffin wrote:
> / Nikola Petrov wrote on Wed  7.Nov'12 at 19:17:46 +0200 /
> 
> > On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> > > I currently have my mail delivered to my desktop system using SMTP as
> > > the system is on all the time and has a static IP.
> > > 
> > > However I always get paranoid when I reconfigure it and/or do other
> > > maintenance so I'm considering moving back to a fetchmail/getmail based
> > > system.   
> 
> May I ask what it is that you are worried about using smtp delivery - I take 
> it you have various protective measures in place with your configuration?

No specific "protective measures" at all, it just relies on the sending
server retrying if my SMTP server isn't running (or connected).  That's
one of the reasons I'd quite like to move away from SMTP.  It *should*
be OK but I'm relying on the other end to behave properly.

-- 
Chris Green


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Jamie Paul Griffin
/ Nikola Petrov wrote on Wed  7.Nov'12 at 19:17:46 +0200 /

> On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> > I currently have my mail delivered to my desktop system using SMTP as
> > the system is on all the time and has a static IP.
> > 
> > However I always get paranoid when I reconfigure it and/or do other
> > maintenance so I'm considering moving back to a fetchmail/getmail based
> > system.   

May I ask what it is that you are worried about using smtp delivery - I take it 
you have various protective measures in place with your configuration?


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Chris Green
On Wed, Nov 07, 2012 at 01:04:17PM -0500, Tim Gray wrote:
> On Nov 07, 2012 at 02:15 PM +, Chris Green wrote:
> >I *don't* like procmail configuration files, they're one of the reasons
> >I wrote my own.
> >
> >What does everyone else here do for collecting mail and filtering mail
> >with mutt?
> 
> I use getmail and dovecot deliver.  Getmail is great, fast, and

I had a look at dovecot deliver, I make take another look.


> flexible (and supports the OS X keychain, which I like).  Dovecot is
> a bit overkill for just a filtering solution since it's a full IMAP
> server.  However, I find dovecot deliver (which uses the sieve
> language for filtering) to be much more readable/writable than
> procmail.  An added bonus is that my main IMAP account has sieve on

I think anything is more readable than procmail!  :-)


> the server, so I can filter mail remotely there using the same
> syntax as I do with my other accounts using getmail.
> 
> I've looked into other solutions that are more compatible with
> offlineimap, like imapfilter (which I believe moves message server
> side, but is run from as a remote client) and a few other lesser
> known local solutions, like maildirproc, but in the end, sieve was
> the most straight forward for my setup.

There seem to be very few currently maintained MDA/filters.

Thanks.

-- 
Chris Green


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Tim Gray

On Nov 07, 2012 at 02:15 PM +, Chris Green wrote:

I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.

What does everyone else here do for collecting mail and filtering mail
with mutt?


I use getmail and dovecot deliver.  Getmail is great, fast, and 
flexible (and supports the OS X keychain, which I like).  Dovecot is a 
bit overkill for just a filtering solution since it's a full IMAP 
server.  However, I find dovecot deliver (which uses the sieve language 
for filtering) to be much more readable/writable than procmail.  An 
added bonus is that my main IMAP account has sieve on the server, so I 
can filter mail remotely there using the same syntax as I do with my 
other accounts using getmail.


I've looked into other solutions that are more compatible with 
offlineimap, like imapfilter (which I believe moves message server side, 
but is run from as a remote client) and a few other lesser known local 
solutions, like maildirproc, but in the end, sieve was the most straight 
forward for my setup.


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Chris Green
On Wed, Nov 07, 2012 at 07:17:46PM +0200, Nikola Petrov wrote:
> On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> > I currently have my mail delivered to my desktop system using SMTP as
> > the system is on all the time and has a static IP.
> > 
> > However I always get paranoid when I reconfigure it and/or do other
> > maintenance so I'm considering moving back to a fetchmail/getmail based
> > system.   
> > 
> > I also have a fairly complex mail filtering script I wrote myself in
> > Python which is fed mail via .forward.
> > 
> > 
> > What's the current "state of the art" way to collect mail and deliver it
> > through a filtering system to mutt?  If I can do this all in one program
> > than so much the better but I'm happy with two programs if that would
> > work better.  I can stay with my existing filter system but, again, if I
> > can consolidate things into one, easier to maintain, chunk then I'd be
> > happy. 
> > 
> > 
> > I *don't* like procmail configuration files, they're one of the reasons
> > I wrote my own.
> > 
> > 
> > What does everyone else here do for collecting mail and filtering mail
> > with mutt?
> 
> I am using imapfilter with lua configuration file for my imap account.
> That does the job for me and I like the fact that I declare my filters
> with actual code(be it lua, python or what not)
> 
How does that actually work?  I've found its home at 
https://github.com/lefcha/imapfilter
but the documentation doesn't really tell me what it does (maybe my fault!).

Does it move E-Mails around on the IMAP server, or does it collect them
from the IMAP server and deliver them to you locally?  Or does it do
something else?

-- 
Chris Green


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Nikola Petrov
On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> I currently have my mail delivered to my desktop system using SMTP as
> the system is on all the time and has a static IP.
> 
> However I always get paranoid when I reconfigure it and/or do other
> maintenance so I'm considering moving back to a fetchmail/getmail based
> system.   
> 
> I also have a fairly complex mail filtering script I wrote myself in
> Python which is fed mail via .forward.
> 
> 
> What's the current "state of the art" way to collect mail and deliver it
> through a filtering system to mutt?  If I can do this all in one program
> than so much the better but I'm happy with two programs if that would
> work better.  I can stay with my existing filter system but, again, if I
> can consolidate things into one, easier to maintain, chunk then I'd be
> happy. 
> 
> 
> I *don't* like procmail configuration files, they're one of the reasons
> I wrote my own.
> 
> 
> What does everyone else here do for collecting mail and filtering mail
> with mutt?

I am using imapfilter with lua configuration file for my imap account.
That does the job for me and I like the fact that I declare my filters
with actual code(be it lua, python or what not)


Best, Nikola


Re: What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Derek Martin
On Wed, Nov 07, 2012 at 02:15:41PM +, Chris Green wrote:
> What does everyone else here do for collecting mail and filtering mail
> with mutt?

Fetchmail and procmail.  Ugly, but ubiquitous and reliable.  A friend
pointed me at something "better" for mail filtering, but I can't
recall what it was... mainly because I haven't gotten around to
looking into it, on account of the fact that my current solution works
well and requires no learning curve.  Learning a new mail filter
system is very low on the priority list.

-- 
Derek D. Martinhttp://www.pizzashack.org/   GPG Key ID: 0xDFBEAD02
-=-=-=-=-
This message is posted from an invalid address.  Replying to it will result in
undeliverable mail due to spam prevention.  Sorry for the inconvenience.



pgpcLHQgiAmm9.pgp
Description: PGP signature


What are the current fetchmail/getmail and/or procmail/maildrop utilities?

2012-11-07 Thread Chris Green
I currently have my mail delivered to my desktop system using SMTP as
the system is on all the time and has a static IP.

However I always get paranoid when I reconfigure it and/or do other
maintenance so I'm considering moving back to a fetchmail/getmail based
system.   

I also have a fairly complex mail filtering script I wrote myself in
Python which is fed mail via .forward.


What's the current "state of the art" way to collect mail and deliver it
through a filtering system to mutt?  If I can do this all in one program
than so much the better but I'm happy with two programs if that would
work better.  I can stay with my existing filter system but, again, if I
can consolidate things into one, easier to maintain, chunk then I'd be
happy. 


I *don't* like procmail configuration files, they're one of the reasons
I wrote my own.


What does everyone else here do for collecting mail and filtering mail
with mutt?

-- 
Chris Green