Re: Killfiling, anyone?
Thanks a lot for your post, David. I'll look it over. Some good ideas in there! -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
Re: Killfiling, anyone?
* On 25 Jul 2012, John Long wrote: > Guys, what are you using for killfiling/mail filtering? This isn't exactly what you're looking for since you want it filtered pre-download, but it's perhaps something worth thinking about. I've been meaning to post it for years but never got around to it, so I finally made time for this topic. Background: I use procmail pretty heavily for delivery filtering, because I know of nothing else with its power. However I absolutely loathe Procmail's performance and configuration syntax. It's OK for the kind of programmatic filtering I use it for, but less OK for the very narrowly-scoped filtering that killfiling implies. I should be able to add a very simple expression to a file and go, without banging around in procmail's bizarre notation, external programs to extract content, etc. (I was never very interested in Sieve for my needs, which include a lot of content filtering, not just sorting. I don't want to get into details, but it may be worth another look now that it's been 12 years since I first looked at it.) Stage 1: I decided I would do my killfiling in mutt, using mutt's pattern expressions for simplicity and flexibility. First I wrote a simple shell script that generates mutt commands based on a 'killfile' that contains simple mutt patterns -- e.g.: ~s '^Output from .cron. command' ~y mutt ~s 'Design choices' ~t root@ ~s 'logwatch' ~y mailman ~s '^.rt \#[0-9]+. List ([^ ]+) creation request' I run this with a macro: macro index \;j "source 'mutt-killfile |'" "Delete junk mail." So whenever I want to run the killfile I press ;j, and that tags all such messages. I eyeball the results and tag-delete. Stage 2: I found that once I added enough patterns to killfile everything, execution got really slow because mutt was running dozens of regular expressions across my entire mailbox. It would be faster to adjust my script to aggregate these expressions into a single regexp. I did that, and performance improved dramatically. Stage 3: Before long I encountered mutt's command length limit of 1024 characters. My program needed to detect when its tagging commands were encroaching on that limit, and break the aggregate expression up into multiple commands. I wrote this initially in Perl years ago, but recently converted it to Python (as I eventually do with every Perl program I've written) and added features. I currently condense about 180 patterns in my killfile into 8 actual mutt commands with this approach. This is what I use today. With no arguments, this program reads three files, if they exist: ~/.killfile ~/.mutt-killfile ~/.mutt/killfile Alternatively it can read stdin or from a specific file name or names: usage: mutt-killfile mutt-killfile - mutt-killfile file [...] It generates muttrc commands based on the patterns contained within, as described above. For example, the patterns above create this result: push "(~s '\^Output from .cron. command')|(~y mutt ~s 'Design choices')|(~t root@ ~s 'logwatch')|(~y mailman ~s '\^.rt \#[0-9]+. List ([\^ ]+) creation request')" Additionally you can control the output template. This input: template push "%p" ~s '^Output from .cron. command' ~y mutt ~s 'Design choices' ~t root@ ~s 'logwatch' ~y mailman ~s '^.rt \#[0-9]+. List ([^ ]+) creation request' generates this: push "(~s '\^Output from .cron. command')|(~y mutt ~s 'Design choices')|(~t root@ ~s 'logwatch')|(~y mailman ~s '\^.rt \#[0-9]+. List ([\^ ]+) creation request')" ... which immediately deletes messages instead of tagging, like an actual killfile for the non-paranoid. A killfile also can source other killfiles. Here's my ~/.mutt/killfile: source ~/.mutt/killfile.$DOMAIN Since DOMAIN=uchicago.edu in my environment, that causes it to source ~/.mutt/killfile.uchicago.edu. The killfile may contain blank lines and comments for readability. Each sourced killfile may have its own template. If no template is explicit in the file, it inherits from the previous file or the default. When the file is done, the template reverts to the previous template. You can of course set a folder-hook to run this script upon entering a folder, too, if you prefer full automation. I suppose something like this would do it: folder-hook . 'source "mutt-killfile |"' Now that it's a Python program I may add the capability to automatically add killfile entries for author, subject, etc, but so far it doesn't bother me enough that I can't. The main thing to beware of with this program is that it's exactly as sensitive to escaping in patterns as mutt itself is. You may encounter problems using it that are hard to diagnose, since they boil down to a few characters in a generated muttrc command up to ~1020 characters long. -- David Champion • d...@uchicago.edu • IT Services • University of Chicago #!/usr/bin/env python impor
Re: Killfiling, anyone?
> > I am afraid I don't understand. What doesn't work? Following up to myself... It seems sourcing the score file works even when new emails are received. Sorry I didn't test this properly before asking. I was just sourcing manually. Adding it to the startup seems to work fine. Thanks for the help! > -- > ASCII ribbon campaign ( ) Powered by Lemote Fuloong > against HTML e-mail X Loongson MIPS and OpenBSD >and proprietary/ \http://www.mutt.org > attachmentsCode Blue or Go Home! -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
Re: Killfiling, anyone?
On Thu, Jul 26, 2012 at 11:23:14PM +0200, Christian Brabandt wrote: > Hi John! Hello Christian! > > Christian, your idea is working pretty good so far but I didn't figure out > > how to source the script by binding it to a key. Is that possible somehow? > > I am afraid I don't understand. What doesn't work? Maybe I missed something because ever since I started to work on this I have only gotten one spam email to test on :-( I thought I had to source the script each time I receive mail. If so, can I bind it to a key somehow? Or did I misunderstand, and sourcing it once when Mutt starts will cause the scoring to work on every email in the index whether I get new mail during that session of Mutt or not? Or something else...? Thank you! -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
Re: Killfiling, anyone?
Hi John! On Do, 26 Jul 2012, John Long wrote: > On Wed, Jul 25, 2012 at 08:53:03PM +0200, Christian Brabandt wrote: > > > Additionally, I needed to set up scoring for mutt like this: > > Christian, your idea is working pretty good so far but I didn't figure out > how to source the script by binding it to a key. Is that possible somehow? I am afraid I don't understand. What doesn't work? regards, Christian --
Re: Killfiling, anyone?
On Wed, Jul 25, 2012 at 08:53:03PM +0200, Christian Brabandt wrote: > Additionally, I needed to set up scoring for mutt like this: Christian, your idea is working pretty good so far but I didn't figure out how to source the script by binding it to a key. Is that possible somehow? -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
Re: Killfiling, anyone?
On Thu, Jul 26, 2012 at 11:55:25AM +0200, Christian Brabandt wrote:
> On Thu, July 26, 2012 11:38, John Long wrote:
[..]
> > Then couldn't you have filtered from sieve? Why did you choose to do it
> > from
> > Mutt itself?
>
> I could possibly discard mails using Sieve, but I didn't know how to
> automate it and this was easier for me and a nice litte task to see,
> if I can make use of scoring.
That's quite easy:
if header :contains ["from"] ["id...@example.edu"] {
discard;
}
With this no email from id...@example.edu will ever hit a mailbox.
Also see here: http://sieve.mozdev.org/reference.html and here:
http://sieve.info/ for more infos about sive.
HTH
Dennis
Re: Killfiling, anyone?
On Thu, Jul 26, 2012 at 11:55:25AM +0200, Christian Brabandt wrote: > On Thu, July 26, 2012 11:38, John Long wrote: > > On Wed, Jul 25, 2012 at 08:53:03PM +0200, Christian Brabandt wrote: > >> Hi John! > > > > Hello! :) > > > >> I used to have a little shell-script, that was killfiling within mutt > >> for me (attached). It simply generates a pattern, that can be used by > >> mutt to delete messages by scoring. > > > > I like what you did here and it is actually pretty close to what I wanted, > > but I guess it requires you to press the $ key to actually do the delete > > once the messages are scored and until then they actually show up in the > > index. What I would like to do is never see the messages at all. Is there > > any way to do this with your method or do I really need to take the advice > > of the (let me count...5 guys...did I miss anybody) who voted for > > procmail? > > folder-hook . 'push ' should apply the settings. Thanks, I'll give this a try! I would still prefer not to see the emails, but this may be a good solution until I learn/implement procmail. /john -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
Re: Killfiling, anyone?
On Thu, July 26, 2012 11:38, John Long wrote: > On Wed, Jul 25, 2012 at 08:53:03PM +0200, Christian Brabandt wrote: >> Hi John! > > Hello! :) > >> I used to have a little shell-script, that was killfiling within mutt >> for me (attached). It simply generates a pattern, that can be used by >> mutt to delete messages by scoring. > > I like what you did here and it is actually pretty close to what I wanted, > but I guess it requires you to press the $ key to actually do the delete > once the messages are scored and until then they actually show up in the > index. What I would like to do is never see the messages at all. Is there > any way to do this with your method or do I really need to take the advice > of the (let me count...5 guys...did I miss anybody) who voted for > procmail? folder-hook . 'push ' should apply the settings. >> PS: No, procmail was no option, since I used sieve scripts to deliver my >> mails. > > Then couldn't you have filtered from sieve? Why did you choose to do it > from > Mutt itself? I could possibly discard mails using Sieve, but I didn't know how to automate it and this was easier for me and a nice litte task to see, if I can make use of scoring. regards, Christian
Re: Killfiling, anyone?
On Wed, Jul 25, 2012 at 08:53:03PM +0200, Christian Brabandt wrote: > Hi John! Hello! :) > I used to have a little shell-script, that was killfiling within mutt > for me (attached). It simply generates a pattern, that can be used by > mutt to delete messages by scoring. I like what you did here and it is actually pretty close to what I wanted, but I guess it requires you to press the $ key to actually do the delete once the messages are scored and until then they actually show up in the index. What I would like to do is never see the messages at all. Is there any way to do this with your method or do I really need to take the advice of the (let me count...5 guys...did I miss anybody) who voted for procmail? > PS: No, procmail was no option, since I used sieve scripts to deliver my > mails. Then couldn't you have filtered from sieve? Why did you choose to do it from Mutt itself? Thanks alot for the help guys! -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
Re: Killfiling, anyone?
Hi John!
On Mi, 25 Jul 2012, John Long wrote:
> Guys, what are you using for killfiling/mail filtering?
>
> I am using Mutt's built in POP and SMTP at this point, is there any way to
> killfile emails based on header contents? Scoring won't be enough, I want to
> delete this crap as the email is being downloaded from my mail provider.
>
> I am not looking for bayesian or other high-falutin' anti spam measures. I
> want to create my own killfile like I do with my newsreader, basically just
> creating rules that match the idiots that spam the mailing lists I'm in.
I used to have a little shell-script, that was killfiling within mutt
for me (attached). It simply generates a pattern, that can be used by
mutt to delete messages by scoring.
Additionally, I needed to set up scoring for mutt like this:
#v+
~$ cat ~/.mutt/score
##
# Scoring Definitions for Mutt/Muttng
#
# Last update: Mi 2010-02-17 22:02
#
##
set score # enable Scoring
set score_threshold_delete=0# delete messages with score 0
set score_threshold_flag=65 # auto-flag messages w/ score >= 35
set score_threshold_read=5 # mark messages w/ score <= 5 as read
# Remove all scorings
unscore *
# Default - Scoring
score '~N' +10 # new mails have a higher score than old mail
score '~A' +10 # all messages start with score 10
score '~g|~G'+2 # PGP signed / encrypted messages
score ~F +20 # flagged mails are important
score ~D =0 # this is a deleted email
score ~S =0 # superseded messages
score ~V +3 # cryptographic verified message
score '~x @256bit.org' +5# Message references one of my message
# Subject dependent scoring
score '~s [Vv][Ii][Mm]' +10 # vim
score '~s [Mm][Uu][Tt][Tt]' +10 # mutt
score '~s [Ss][Hh][Ee][Ll][Ll]' +10 # shell
score '~s [Bb][Aa][Ss][Hh]' +10 # bash
score '~s [Zz][Ss][Hh]' +10 # zsh
# The good one ;)
score '~P' =20 # Message is from me
# SPAM
score '~f @aol.com' -2 # AOL is usually less important
score '~f webmaster@*'-2 # Webmasters kidnapped by evil cult :>
score '~s ^test$' -2 # all messages w/ subject "test" are killed
score '~s sex | ~s adult' -10 # STFU
score '~f anonymous' -10 # Yeah. Sure. Evil hackers from Serbia.
score '~=' - # all duplicates are killed
score '~s ^unsubscribe$ !(~p|~P|~Q|~F)' - # all msgs w/ subject
"unsubscribe" not by myself are killed
#score '!~f@' -10 # no mail address present?
score "~f '^([^ <>@]+ ){5,}'" -2 # a realname consisting of >=5 portions is
invalid, too
score "!~s .*"-2 # no Subject line
source ~/.mutt/score_gen
$~
#v-
That was basically it. It still works, although I don't use it anymore.
PS: No, procmail was no option, since I used sieve scripts to deliver my
mails.
regards,
Christian
mutt_score.sh
Description: Bourne shell script
Re: Killfiling, anyone?
>> That's three votes for procmail. I guess I will have to look into it! >4 :^) And me too is voting for procmail of course ☺! So it is 5.
Re: Killfiling, anyone?
On Jul 25, 2012 at 10:26 AM -0400, Mark H. Wood wrote: Well, here's one vote for maildrop (from a former procmail user). They're both good. And, since I use exim, I keep telling myself that someday I'm going to try its Sieve per-user filter support. I use Sieve for this kind of thing. Download my mail with getmail and send it to Dovecot's deliver sorts mail into maildirs based on the Sieve setup.
Re: Killfiling, anyone?
Well, here's one vote for maildrop (from a former procmail user). They're both good. And, since I use exim, I keep telling myself that someday I'm going to try its Sieve per-user filter support. -- Mark H. Wood, Lead System Programmer mw...@iupui.edu Asking whether markets are efficient is like asking whether people are smart. pgpykwq5kWVZ1.pgp Description: PGP signature
Re: Killfiling, anyone?
* John Long [07-25-12 07:52]: > That's three votes for procmail. I guess I will have to look into it! 4 :^) -- (paka)Patrick Shanahan Plainfield, Indiana, USA HOG # US1244711 http://wahoo.no-ip.orgPhoto Album: http://wahoo.no-ip.org/gallery2 http://en.opensuse.org openSUSE Community Member Registered Linux User #207535@ http://linuxcounter.net
Re: Killfiling, anyone?
That's three votes for procmail. I guess I will have to look into it! Thanks everybody. -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
Re: Killfiling, anyone?
>From the Debian FAQ:
muttrc:
macro index"|grep "^^From:" | sed -e 's/ *(.*)//; s/>.*//;
s/.*[:<] *//' \
>> $HOME/.spam && echo Add sender to killfile\n" "kill sender"
macro pager"|grep "^^From:" | sed -e 's/ *(.*)//; s/>.*//;
s/.*[:<] *//' \
>> $HOME/.spam && echo Add sender to killfile\n" "kill sender"
procmailrc:
FROM=`formail -xFrom: | sed -e 's/ *(.*)//; s/>.*//; s/.*[:<] *//'`
:0
* ? fgrep -qxis "$FROM" $HOME/.spam
{
LOG="Spam from $FROM"
:0:
spam
}
--
Ivo Engelhardt
Re: Killfiling, anyone?
* John Long schrieb am 25.07.2012 um 8:48 Uhr: > Guys, what are you using for killfiling/mail filtering? Procmail http://www.procmail.org/ http://www.ii.com/internet/robots/procmail/qs/ Andreas
Re: Killfiling, anyone?
On Wed, Jul 25, 2012 at 08:48:08AM +, John Long wrote: > Guys, what are you using for killfiling/mail filtering? procmail. There's no substitute, IMHO. Later, --jim -- THE SCORE: ME: 2 CANCER: 0 73 DE N5IAL (/4) | Peter da Silva: No, try "rm -rf /" spooky1...@gmail.com | Dave Aronson:As your life flashes before < Running Mac OS X Lion > | your eyes, in the unit of time known as an ICBM / Hurricane: | ohnosecond (alt.sysadmin.recovery) 30.44406N 86. 59909W | Android Apps Listing at http://www.jstrack.org/barcodes.html
Killfiling, anyone?
Guys, what are you using for killfiling/mail filtering? I am using Mutt's built in POP and SMTP at this point, is there any way to killfile emails based on header contents? Scoring won't be enough, I want to delete this crap as the email is being downloaded from my mail provider. I am not looking for bayesian or other high-falutin' anti spam measures. I want to create my own killfile like I do with my newsreader, basically just creating rules that match the idiots that spam the mailing lists I'm in. -- ASCII ribbon campaign ( ) Powered by Lemote Fuloong against HTML e-mail X Loongson MIPS and OpenBSD and proprietary/ \http://www.mutt.org attachmentsCode Blue or Go Home!
