Re: mairix search

2011-05-12 Thread Jason Helfman

On Sun, May 08, 2011 at 05:10:22PM -0400, Tim Gray thus spake:

On May 08, 2011 at 10:47 PM +0200, Christian Ebert wrote:

$ time mairix -v -p


I bet that was my problem.  I don't think I ever used -p, so there were
a lot of dead messages floating around in my db.

The times I'm getting now are pretty good.  Notmuch seems to be faster,
but the times are all low enough that I don't have a problem with any of
them.

mairix -v -p

real0m17.682s
user0m4.911s
sys 0m8.524s

notmuch new
---
real0m5.152s
user0m0.067s
sys 0m0.261s

Searches for the two showed a similar gap.  Again, neither was slow
enough for me to lose any sleep over.

mairix: 0m3.044s
notmuch: 0m0.410s

This is an interesting discussion though.  I might play around with
mairix a bit more again.  I still see mu and notmuch having a major
advantage of being built on proper database tools.  I get a lot of
errors about messages not being indexed by mairix, and that whole
recommended dance of removing the lock file before a search, etc. is
annoying as well.  Furthermore, the thing that excites me about notmuch
that the others don't have is the fact that it's built as a library.  An
enterprising developer could integrate it into a mail client (other than
the emacs thing they have going on) and it would be pretty great in my
mind.  Remember, notmuch isn't just an indexing tool - it also lets you
tag messages and search on tags, etc.



Does anyone use notmuch for FreeBSD? If so I have created a port, and would
like to try it out, but I don't have my mail locally sync'd, yet to verify
if this actually works.

Anyone willing to try out the attached shell archive? Just run /bin/sh
against the attachment, then:

cd notmuch; make install (as root, or use sudo)

Thanks,
Jason
# This is a shell archive.  Save it in a file, remove anything before
# this line, and then unpack it by entering sh file.  Note, it may
# create directories; files and directories will be owned by you and
# have default permissions.
#
# This archive contains:
#
#   notmuch/
#   notmuch/Makefile
#   notmuch/pkg-descr
#   notmuch/distinfo
#
echo c - notmuch/
mkdir -p notmuch/  /dev/null 21
echo x - notmuch/Makefile
sed 's/^X//' notmuch/Makefile  'daed98e951f4e6015a65d45ec4b2a91d'
X# New ports collection makefile for:   muttils
X# Date created:April 2 2011
X# Whom:Jason Helfman jhelf...@experts-exchange.com
X#
X# $FreeBSD$
X#
X
XPORTNAME=  notmuch
XPORTVERSION=   0.5
XCATEGORIES=mail python
XMASTER_SITES=  http://notmuchmail.org/releases/
X
XMAINTAINER=jhelf...@experts-exchange.com
XCOMMENT=   Mail indexing tool
X
XLIB_DEPENDS=   gmime-2.4:${PORTSDIR}/mail/gmime24 \
X   talloc.2:${PORTSDIR}/devel/talloc
XBUILD_DEPENDS+=xapian-config:${PORTSDIR}/databases/xapian-core
XRUN_DEPENDS+=  xapian-config:${PORTSDIR}/databases/xapian-core
X
XPLIST_FILES=   bin/notmuch \
X   man/man1/notmuch.1.gz
XUSE_GMAKE= yes
X
Xdo-install:
X   ${INSTALL_PROGRAM} ${WRKSRC}/notmuch ${PREFIX}/bin
X   ${INSTALL_DATA} ${WRKSRC}/notmuch.1.gz ${PREFIX}/man/man1
X
X.include bsd.port.mk
daed98e951f4e6015a65d45ec4b2a91d
echo x - notmuch/pkg-descr
sed 's/^X//' notmuch/pkg-descr  '96faee797c242a96d3f25e49bbbd52d1'
XNotmuch is a command-line based program for indexing, searching, reading, 
Xand tagging large collections of email messages.
X
XWWW: http://notmuchmail.org
96faee797c242a96d3f25e49bbbd52d1
echo x - notmuch/distinfo
sed 's/^X//' notmuch/distinfo  '38ecdd5c0c83491813eed1b00316c9dd'
XSHA256 (notmuch-0.5.tar.gz) = 
c7eeb95c89c5b9cb22cc0b90abce5f923c20c982d607bf32829c989e905ff1a9
XSIZE (notmuch-0.5.tar.gz) = 340156
38ecdd5c0c83491813eed1b00316c9dd
exit




pgp7NbcKDEP5v.pgp
Description: PGP signature


Re: mairix search

2011-05-08 Thread Richard
On Fri, May 06, 2011 at 12:12:38AM -0400, Tim Gray wrote:

 One of the other things I like about notmuch is that I feel it has a
 much more intuitive syntax.  I could never remember which switches I
 needed to use with mairix and mu.  With notmuch, if I want to find
 an email from j...@joe.com with a subject of 'work order', I can
 usually just search 'joe work order' and get the correct email.
 Correct me if I'm wrong, but don't mu and/or mairix require to use a
 from: of f: tag and only match on complete addresses?

indeed wrong; 
  mairix j...@joe.com
  mairix j...@joe.com work order
will do exactly what most people would think it would do. It has some
special treatment of email addresses in addition to that.

The substring wildcard = is rather unusual but makes sense once you
actually make use of the additional parameter (which does approximate
matching).

Is mairix the only option for mboxes?

Richard

---
Name and OpenPGP keys available from pgp key servers



Re: mairix search

2011-05-08 Thread Tim Gray

On May 08, 2011 at 11:09 AM +0200, Richard wrote:

indeed wrong;
 mairix j...@joe.com
 mairix j...@joe.com work order
will do exactly what most people would think it would do. It has some
special treatment of email addresses in addition to that.


Good to know.  Though I still do find the search syntax easier with 
notmuch (and mu).  Not saying everyone that that's true for everyone.


Again, the difference on the time it took to re-index was probably the 
real eye-opener for me.


Yes, notmuch and mu don't work on mbox.


Re: mairix search

2011-05-08 Thread Richard
On Sun, May 08, 2011 at 10:13:49AM -0400, Tim Gray wrote:

 Again, the difference on the time it took to re-index was probably
 the real eye-opener for me.

I have just done a re-index with mairix and have no reason to complain:)

As of search operators most people are happiest if they don't need them
at all.

One of the things thats solved nicely with google but hard to integrate
sensibly when using mutt+external search is giving search suggestions when
words are misspelled etc.


Richard

---
Name and OpenPGP keys available from pgp key servers



Re: mairix search

2011-05-08 Thread Tim Gray

On May 08, 2011 at 09:21 PM +0200, Richard wrote:

I have just done a re-index with mairix and have no reason to complain:)


In my experience, on my system, mairix was slower.  I seem to recall 
times of around 15-20 minutes to go through a couple hundred thousand 
messages looking for new ones.


I think my mu database is still kicking around, but as I don't have GTK 
installed on this system anymore, I can't build mu anymore.  It's 
probably not worth my effort to resurrect it all just to time things, 
which I had done previously in my switch from mu to notmuch.


As far as mairix goes, I'll run those tests now.  I just built 0.22 and 
will report back in a couple of hours.


On the other hand, if what you are using works, then by all means 
continue to use it.



As of search operators most people are happiest if they don't need them
at all.


Agreed.


Re: mairix search

2011-05-08 Thread Christian Ebert
* Tim Gray on Sunday, May 08, 2011 at 15:39:53 -0400
 On May 08, 2011 at 09:21 PM +0200, Richard wrote:
 I have just done a re-index with mairix and have no reason to complain:)
 
 In my experience, on my system, mairix was slower.  I seem to recall
 times of around 15-20 minutes to go through a couple hundred thousand
 messages looking for new ones.

$ time mairix -v -p
mairix DEVELOPMENT, Copyright (C) 2002-2010 Richard P. Curnow
mairix comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under certain conditions; see the GNU General Public License for details.

Finding all currently existing messages...
Reading existing database...
Checking message path integrity
Checking to
Checking cc
Checking from
Checking subject
Checking body
Checking attachment_name
Loaded 254115 existing messages
0 newly dead messages, 0 messages now dead in total
No new messages found
Checking message path integrity
Checking to
Checking cc
Checking from
Checking subject
Checking body
Checking attachment_name
Culling dead messages

real2m0.107s
user0m31.165s
sys 0m53.417s

c
-- 
theatre - books - texts - movies
Black Trash Productions at home: http://www.blacktrash.org
Black Trash Productions on Facebook:
http://www.facebook.com/blacktrashproductions


Re: mairix search

2011-05-08 Thread Tim Gray

On May 08, 2011 at 10:47 PM +0200, Christian Ebert wrote:

$ time mairix -v -p


I bet that was my problem.  I don't think I ever used -p, so there were 
a lot of dead messages floating around in my db.  

The times I'm getting now are pretty good.  Notmuch seems to be faster, 
but the times are all low enough that I don't have a problem with any of 
them.


mairix -v -p

real0m17.682s
user0m4.911s
sys 0m8.524s

notmuch new
---
real0m5.152s
user0m0.067s
sys 0m0.261s

Searches for the two showed a similar gap.  Again, neither was slow 
enough for me to lose any sleep over.


mairix: 0m3.044s
notmuch: 0m0.410s

This is an interesting discussion though.  I might play around with 
mairix a bit more again.  I still see mu and notmuch having a major 
advantage of being built on proper database tools.  I get a lot of 
errors about messages not being indexed by mairix, and that whole 
recommended dance of removing the lock file before a search, etc. is 
annoying as well.  Furthermore, the thing that excites me about notmuch 
that the others don't have is the fact that it's built as a library.  An 
enterprising developer could integrate it into a mail client (other than 
the emacs thing they have going on) and it would be pretty great in my 
mind.  Remember, notmuch isn't just an indexing tool - it also lets you 
tag messages and search on tags, etc.


Re: mairix search

2011-05-08 Thread Christian Ebert
* Tim Gray on Sunday, May 08, 2011 at 17:10:22 -0400
 On May 08, 2011 at 10:47 PM +0200, Christian Ebert wrote:
 $ time mairix -v -p
 
 I bet that was my problem.  I don't think I ever used -p, so there
 were a lot of dead messages floating around in my db.

I believe --purge makes it actually slower, as it compacts dead
messages away in the db.

~$ time mairix

real1m54.450s
user0m28.735s
sys 0m54.465s
~$ time mairix --fast-index

real0m37.570s
user0m28.725s
sys 0m1.503s
~$ time mairix --fast-index --no-integrity-checks

real0m28.202s
user0m26.776s
sys 0m0.772s

 The times I'm getting now are pretty good.  Notmuch seems to be
 faster, but the times are all low enough that I don't have a problem
 with any of them.
 
 mairix -v -p
 
 real  0m17.682s
 user  0m4.911s
 sys   0m8.524s
 
 notmuch new
 ---
 real  0m5.152s
 user  0m0.067s
 sys   0m0.261s
 
 Searches for the two showed a similar gap.  Again, neither was slow
 enough for me to lose any sleep over.
 
 mairix: 0m3.044s
 notmuch: 0m0.410s
 
 This is an interesting discussion though.  I might play around with
 mairix a bit more again.  I still see mu and notmuch having a major
 advantage of being built on proper database tools.  I get a lot of
 errors about messages not being indexed by mairix, and that whole
 recommended dance of removing the lock file before a search, etc. is
 annoying as well.  Furthermore, the thing that excites me about
 notmuch that the others don't have is the fact that it's built as a
 library.  An enterprising developer could integrate it into a mail
 client (other than the emacs thing they have going on) and it would be
 pretty great in my mind.  Remember, notmuch isn't just an indexing
 tool - it also lets you tag messages and search on tags, etc.

I would rather be interested whether notmuch correctly finds
strings containing non-ascii characters. In my script I work
around by appending the results of a second search, so when I
enter e.g. 'première' in utf-8, the first search is in utf-8 and
second, usually finding more matches converts the search string
to cp1252 - but that is obviously a hack.

I once tried with mu, and it wasn't any better in detecting
those, but perhaps I gave up to quickly.

c
-- 
\black\trash movie   _SAME  TIME  SAME  PLACE_
   New York, in the summer of 2001

--- http://www.blacktrash.org/underdogma/stsp.php


Re: mairix search

2011-05-05 Thread Jostein Berntsen
On 03.05.11,10:21, Tim Gray wrote:
 On May 03, 2011 at 08:39 AM +0200, Jostein Berntsen wrote:
 I use mairix, but it seems like mu is being quite actively developed:
 
 Yes, mu is quite actively developed.  I liked it a fair amount.  I
 just have a feeling that notmuch has a brighter future.
 

I tested the most recent version of mu v. 0.9.5, works great with many 
useful features. That cfind option is a nice one.

http://gitorious.org/mu/mu-ng/blobs/master/NEWS



Jostein




Re: mairix search

2011-05-05 Thread Sebastian Tramp
On Thu, May 05, 2011 at 09:08:36PM +0200, Jostein Berntsen wrote:

  I use mairix, but it seems like mu is being quite actively
  developed:
 
  Yes, mu is quite actively developed. I liked it a fair amount. I
  just have a feeling that notmuch has a brighter future.

 I tested the most recent version of mu v. 0.9.5, works great with many
 useful features. That cfind option is a nice one.

This is indeed an interesting feature. Do you use it instead of lbdbq?

Seebi

-- 
Sebastian Tramp
WebID: http://sebastian.tramp.name


Re: mairix search

2011-05-05 Thread Sebastian Tramp
On Tue, May 03, 2011 at 10:21:10AM -0400, Tim Gray wrote:

 I use mairix, but it seems like mu is being quite actively
 developed:

 Yes, mu is quite actively developed. I liked it a fair amount. I just
 have a feeling that notmuch has a brighter future.

Does notmuch have a similar feature to mu's cfind? I did not find
it in the docu -- but the project name is policy also in terms of
documentation :-)

Seebi

-- 
Sebastian Tramp
WebID: http://sebastian.tramp.name


Re: mairix search

2011-05-05 Thread Sebastian Tramp
On Tue, May 03, 2011 at 11:20:02AM -0400, Tim Gray wrote:

 Can you share your config parts regarding notmuch / mutt integration?
 (or do you use it via emacs?)

 I don't use it via emacs. The mutt integration is very similar to what
 you'd do with mairix or mu. A couple of bindings that just run the
 command line 'notmuch' program with your search terms following. You
 do need to do a bit of command line piping to turn the filenames that
 notmuch outputs as search results into symbolic links. Then you jump
 to a maildir with the results.

 Though you can run it directly like this, I wrote a little python
 script that I call instead which gives me readline search history and
 cleans out the search results automatically when I make a new search.
 I've attached the script 'notmuch-mutt.py'.

Hi Tim, thanks for this resource!

Seebi

-- 
Sebastian Tramp
WebID: http://sebastian.tramp.name


Re: mairix search

2011-05-05 Thread Tim Gray

On May 06, 2011 at 05:17 AM +0200, Sebastian Tramp wrote:

This is indeed an interesting feature. Do you use it instead of lbdbq?


Maybe I'm missing something.  Is it really that useful of a feature if 
you already use lbdb and feed it with your outgoing mail?


One of the other things I like about notmuch is that I feel it has a 
much more intuitive syntax.  I could never remember which switches I 
needed to use with mairix and mu.  With notmuch, if I want to find an 
email from j...@joe.com with a subject of 'work order', I can usually 
just search 'joe work order' and get the correct email.  Correct me if 
I'm wrong, but don't mu and/or mairix require to use a from: of f: tag 
and only match on complete addresses?


Re: mairix search

2011-05-05 Thread Tim Gray

On May 06, 2011 at 05:24 AM +0200, Sebastian Tramp wrote:

Does notmuch have a similar feature to mu's cfind? I did not find
it in the docu -- but the project name is policy also in terms of
documentation :-)


See my other email - I'm not exactly sure what cfind does.  It's pretty 
easy to search for an email from someone if you remember parts of their 
name and/or address.  Once you find the email, you could just use 
mail2muttalias.py to extract the address and stick it in your alias 
file.


I can't remember ever actually needing to do this though.  I feed lbdb 
with my outgoing mail to scrape addresses, in addition to being hooked 
into the OS X address book and my mutt alias files.


Re: mairix search

2011-05-05 Thread Omen Wild
Quoting Tim Gray lists+m...@protozoic.com on Fri, May 06 00:12:

 Correct me if I'm wrong, but don't mu and/or mairix require to use a
 from: of f: tag and only match on complete addresses?

By default all matches are exact, which is nice for subject or body
searches, but not so nice for addresses.  You can do a substring match
with =, i.e. `f:omen=' would find all message with the string omen in
the from header.  You can anchor to the start of a word with ^,
`f:^omen='.

-- 
As of next week, passwords will be entered in Morse code.


smime.p7s
Description: S/MIME cryptographic signature


Re: mairix search

2011-05-03 Thread Jostein Berntsen
On 03.05.11,00:26, Tim Gray wrote:
 On Apr 29, 2011 at 01:56 PM +0200, Sebastian Tramp wrote:
 Is searching / indexing with mairix state of the art or is there a
 better solution available? I am quite happy with that, just wanted to
 ask ;-).
 
 I found that mairix was a lot better for me than nmzmail.  I used
 mairix for quite a bit.  I then found mu (and figured out how to
 compile it on OS X).  I liked that even more.
 
 Now however, I'm running notmuch as just an indexer.  It's pretty
 fast.  I really like it.  It's also being actively developed.  And
 the best part in my mind is that it's developed as a library with a
 command line utility.  So my secret dream is that some enterprising
 developer hooks notmuch directly into mutt.  That would be awesome.
 

I use mairix, but it seems like mu is being quite actively developed:

http://www.djcbsoftware.nl/code/mu/


Jostein




Re: mairix search

2011-05-03 Thread Sebastian Tramp
On Tue, May 03, 2011 at 12:26:45AM -0400, Tim Gray wrote:

 On Apr 29, 2011 at 01:56 PM +0200, Sebastian Tramp wrote:

 Is searching / indexing with mairix state of the art or is there a
 better solution available? I am quite happy with that, just wanted to
 ask ;-).

 I found that mairix was a lot better for me than nmzmail. I used
 mairix for quite a bit. I then found mu (and figured out how to
 compile it on OS X). I liked that even more.

 Now however, I'm running notmuch as just an indexer. It's pretty fast.
 I really like it. It's also being actively developed. And the best
 part in my mind is that it's developed as a library with a command
 line utility. So my secret dream is that some enterprising developer
 hooks notmuch directly into mutt. That would be awesome.

Sounds interesting. do you think, notmuch is faster than mairix (not that
I have a problem with mairix speed, just for our information)

Can you share your config parts regarding notmuch / mutt integration?
(or do you use it via emacs?)

best regards

Sebastian Tramp

-- 
Sebastian Tramp
WebID: http://sebastian.tramp.name


Re: mairix search

2011-05-03 Thread Sebastian Tramp
On Mon, May 02, 2011 at 06:00:04PM -0400, Ed Blackman wrote:
 On Fri, Apr 29, 2011 at 02:08:33PM -0700, Omen Wild wrote:
 Quoting Ed Blackman e...@edgewood.to on Fri, Apr 29 17:03:
 
 In my case, mairixquery is a Perl script that prompts me for the mairix
 search string, gives me yes or no prompts for whether to search threads
 or augment previous results, and saves the most recent 100 searches so
 that I can make edits if the query returns too little or way too much.
 
 That sounds really interesting.  Any chance you would be willing to
 share it?
 
 Sure.  I've attached it.

great, it works for me. just a notice for the other: under
debian/ubuntu, there is a dependency for the libterm-readline-gnu-perl
package ...

best regards

Sebastian Tramp


Re: mairix search

2011-05-03 Thread Tim Gray

On May 03, 2011 at 08:39 AM +0200, Jostein Berntsen wrote:

I use mairix, but it seems like mu is being quite actively developed:


Yes, mu is quite actively developed.  I liked it a fair amount.  I just 
have a feeling that notmuch has a brighter future.


Re: mairix search

2011-05-03 Thread Tim Gray

On May 03, 2011 at 08:47 AM +0200, Sebastian Tramp wrote:

Sounds interesting. do you think, notmuch is faster than mairix (not that
I have a problem with mairix speed, just for our information)


It's been a while since I've used mairix.  I forget exactly how long it 
took to index things initially.  I do remember however that I would only 
reindex to look for new mail once or twice a day because it took a fair 
amount of time and cpu power.


I'm currently indexing about 260,000 messages across many maildirs.  For 
unimportant reasons, I just reindexed them all from scratch last night.  
It took 1 hour and 10 minutes.  I did this while I was video chatting a 
friend.  This was an 'initial' index.  Subsequent indexing, looking for 
newly delivered messages, takes between 20-40 seconds, so I can run it 
every time I check for mail.  Which I do - I have it in my getmail 
script.  You can run it any time you want with the command `notmuch 
new`.


The other thing that I liked about notmuch infinitely more than mairix, 
and a fair bit more than mu, is that the search syntax just feels more 
natural to me.  Your mileage may vary.



Can you share your config parts regarding notmuch / mutt integration?
(or do you use it via emacs?)


I don't use it via emacs.  The mutt integration is very similar to what 
you'd do with mairix or mu.  A couple of bindings that just run the 
command line 'notmuch' program with your search terms following.  You do 
need to do a bit of command line piping to turn the filenames that 
notmuch outputs as search results into symbolic links.  Then you jump to 
a maildir with the results.


Though you can run it directly like this, I wrote a little python script 
that I call instead which gives me readline search history and cleans 
out the search results automatically when I make a new search.  I've 
attached the script 'notmuch-mutt.py'.  I started building in support 
for notmuch's tagging features but kind of lost interest.  You could 
also write an option if you want to *not* clear out the previous search 
results; then you could build up results incrementally if you desired.


I have a couple bindings defined.  I'll try to list them below; 
hopefully they don't get too jumbled up since they are kind of long (I 
don't break up my bindings definitions into separate lines in my muttrc 
- hopefully the \'s I put in break things appropriately).


This one just changes to the search results folder:
macro index,pager .r change-folder-readonly \
  ~/.notmuchmutt/search enter 


This on runs a search:
macro index,pager .s enter-commandunset wait_keyenter \
  shell-escape~/bin/notmuch-mutt.py -penter \
  change-folder-readonly~/.notmuchmutt/searchenter \
  enter-commandset wait_keyenter

This on runs a search and includes the full thread in the results.  You 
can also run it on a search result to reconstruct the thread from the 
message of interest.

macro index,pager .t enter-commandunset wait_keyenter \
  pipe-message~/bin/notmuch-mutt.py - --threadenter \
  change-folder-readonly~/.notmuchmutt/searchenter \
  enter-commandset wait_keyenter

This runs muttjump on the current message (see below).  I don't do all 
the screen stuff that muttjump can do.

macro generic .o enter-commandpush pipe-messagemuttjump \
  enterenter jump to original message

notmuch also has python bindings, so you can access the library 
directly.  For some reason on OS X, they don't work right for me, so my 
python script just calls the command line utility directly.


I also use the awesome 'muttjump' script [1].  It takes you to the 
parent mailbox of a message in the search results.


Hopefully I didn't forget any important bits.  The actual notmuch config 
file is very simple as there aren't a lot of relevant options.  You just 
set the directory to your maildir, and define your user name and address 
(I think this is used only if you are running it in emacs or vim as a 
full mail client).  Lastly, you can set which tags get set for new 
messages and whether or not you want imap flags (read, flagged, etc.) 
that are set in notmuch synchronized back to the maildir files.  Again, 
the last option is mostly for the full notmuch client.


[1]: https://github.com/weisslj/muttjump
#!/usr/bin/env python

__author__ = Tim Gray
__version__ = 1.0

import sys
import os
import optparse
import subprocess as sb
import shlex
import email
import readline

cfgdir = '~/.notmuchmutt'
cfgdir = os.path.expanduser(cfgdir)
searchdir = os.path.join(cfgdir, search/)
searchhist = os.path.join(cfgdir, 'search-history')
taghist = os.path.join(cfgdir, 'tag-history')
if os.path.isdir(cfgdir):
cfgFlag = True
else:
print 'must run with --config first to set up directory'
sys.exit(10)

class notmuch():
def __init__(self):
self.nm = '/usr/local/bin/notmuch'

def runCmd(self, query):
Runs the given command.
cmd 

Re: mairix search

2011-05-02 Thread Ed Blackman

On Fri, Apr 29, 2011 at 02:08:33PM -0700, Omen Wild wrote:

Quoting Ed Blackman e...@edgewood.to on Fri, Apr 29 17:03:


In my case, mairixquery is a Perl script that prompts me for the mairix
search string, gives me yes or no prompts for whether to search threads
or augment previous results, and saves the most recent 100 searches so
that I can make edits if the query returns too little or way too much.


That sounds really interesting.  Any chance you would be willing to
share it?


Sure.  I've attached it.

It was originally the shell script that Christian Ebert posted a couple 
of years ago.  He reposted an enhanced version in this thread.  I got 
frustrated with having to retype the same queries with slight 
variations, so converted it to Perl to take advantage of the built-in 
history feature in Term::ReadLine, and have enhanced it from there.


Features:
- saves last 100 (configurable) queries in $HOME/.mairixquery (configurable)
- automatically replaces X:me in typed queries with 
   X:myem...@example.com (you'll want to edit $me in the script to 
   use your own adress, of course!).  I quite frequently want to find 
   emails that I sent or that were addresssed or copied to me, and this 
   cuts down on typing my own address quite a bit.
- saves the answers to include threads? and add to existing results? 
   in the history, and uses them to set the default for the questions.  
   If I didn't want to use threads the last time I used this query, I 
   most likely don't want to this time.
- expands %name% so that it matches all email addresses reurned from a 
   lbdbq search for name.  This is the one idiosyncratic piece that's 
   hardcoded.  Ithought about making lbdbq configurable, but the code 
   around it is very dependant on lbdbq's output structure.  If you don't 
   use %foo%, it will never call lbdbq, though.


Patches and suggestions welcome.  If there are people interested in 
hacking on it, I'd be happy to start a little github project.


--
Ed Blackman
#!/usr/bin/perl -w

# customizable variables

my $me = 'e...@edgewood.to';
my $histFile = $ENV{HOME}/.mairixquery;
my $histSaveLines = 100;

# code begins

use strict;
use Term::ReadLine;

$ENV{PATH} = $ENV{HOME}/bin:$ENV{PATH};

my ($Agent) = $0 =~ m@.*/([^/]*)@;

print Enter mairix query:\n;

my $term = new Term::ReadLine $Agent;

if($histFile) {
  $term-read_history($histFile);
  # don't automatically add to history, so lines can be changed before being 
saved
  $term-MinLine(undef);
}

my $query = $term-readline('query') or die No query\n;
my @args;

# mairix doesn't like multiple commas, so collapse runs of commas to one
$query =~ s/,,+/,/g;

# if there are words in percents (%bill%), expand it out using lbdb
$query = expandNames($query);

my ($threads) = $query =~ /--(no)?threads /;
$threads = 'yes' if(!defined($threads));

if(yorn(Retrieve thread(s)?, $threads)) {
  unshift @args, --threads;
  $threads = 'yes';
}
else {
  $threads = 'no';
}

$query =~ s/--(no)?threads //;

my ($augment) = $query =~ /--(no)?augment /;
$augment = 'no' if(!defined($augment));

if(yorn(Append message(s) to mfolder?, $augment)) {
  unshift @args, --augment;
  $augment = 'yes';
}
else {
  $augment = 'no';
}

$query =~ s/--(no)?augment //;

push @args, map {
  # make it easy to find my address without typing it out
  s/^(\w+):me$/$1:$me/;

  # limit message-id strings to 31 chars (mairix limit)
  if(/^(m:.{0,31}).*/) {
$_ = $1;
  }

  $_;
} split(' ', $query);

system(('mairix', @args));

# mairix doesn't understand --no arguments, so they need to be added to saved 
args after execution
unshift @args, '--nothreads' if($threads eq 'no');
unshift @args, '--noaugment' if($augment eq 'no');

if($histSaveLines  $histSaveLines  0) {
  $term-add_history(join(' ' ,  @args));
  $term-write_history($histFile);
  $term-history_truncate_file($histFile, $histSaveLines);
}

sub yorn {
  my $prompt = shift;
  my $default = shift;

  $| = 1;
  $prompt .=  ( . ( lc($default) eq 'yes' ? [yes]/no : yes/[no]) . ) ;

  my $ans = $term-readline($prompt);

  $ans = 'yes' if($ans eq ''  $default eq 'yes');

  return $ans =~ /^y.*/i;
}

sub expandNames {
  my $query = shift;

  while ($query =~ /[\W\b]%(\w+)%(?:[\W\b]|$)/) {
my $name = $1;
my @expanded;

foreach my $match (split(/\n/, qx(lbdbq '$name' | tail -n +2))) {
  my ($address, $name, $where) = split(/\t/, $match);

  push @expanded, $address;
  push @expanded, join(',', split(/ /, $name));
}

my $expanded = join('/', @expanded);

$expanded = $name if(!$expanded);   # fall back to replacing '%foo%' with 
'foo' if no matches

$query =~ s/%$name%/$expanded/;
  }

  return $query;
}


signature.txt
Description: Digital signature


Re: mairix search [SEC=UNCLASSIFIED]

2011-05-02 Thread Wilkinson, Alex

0n Fri, Apr 29, 2011 at 01:56:51PM +0200, Sebastian Tramp wrote: 

Is searching / indexing with mairix state of the art or is there a
better solution available? I am quite happy with that, just wanted to
ask ;-).

There is also nmzmail (only for Maildir though) - I actually thought many users
switched from mairix to nmzmail.

   nmzmail is a tool to use the namazu2 search engine from within a mail
   reader such as mutt to search mail stored in maildir folders. Based on
   the result of the namazu query, nmzmail generates a maildir folder
   containing symbolic links to the mails matching the query. Mutt and
   other mail readers can make it easy to use nmzmail with a simple
   macro.
   
   Features:
   * fast mail searching even with large number of mails
   * incremental index building
   * rich query language including regular expressions (see the namazu
 documentation)
   * integrates nicely into mutt
   * query history

   WWW: http://www.ecademix.com/JohannesHofmann/nmzmail.html


 -Alex

IMPORTANT: This email remains the property of the Department of Defence and is 
subject to the jurisdiction of section 70 of the Crimes Act 1914. If you have 
received this email in error, you are requested to contact the sender and 
delete the email.


Re: mairix search

2011-05-02 Thread Tim Gray

On Apr 29, 2011 at 01:56 PM +0200, Sebastian Tramp wrote:

Is searching / indexing with mairix state of the art or is there a
better solution available? I am quite happy with that, just wanted to
ask ;-).


I found that mairix was a lot better for me than nmzmail.  I used mairix 
for quite a bit.  I then found mu (and figured out how to compile it on 
OS X).  I liked that even more.


Now however, I'm running notmuch as just an indexer.  It's pretty fast.  
I really like it.  It's also being actively developed.  And the best 
part in my mind is that it's developed as a library with a command line 
utility.  So my secret dream is that some enterprising developer hooks 
notmuch directly into mutt.  That would be awesome.


Re: mairix search

2011-05-02 Thread Omen Wild
Quoting Ed Blackman e...@edgewood.to on Mon, May 02 18:00:

 That sounds really interesting.  Any chance you would be willing to
 share it?

 Sure.  I've attached it.

Thanks!

 Patches and suggestions welcome.  If there are people interested in  
 hacking on it, I'd be happy to start a little github project.

I'll start looking at it in the next couple days.

-- 
The soul would have no rainbow had the eyes no tears.


signature.asc
Description: Digital signature


Re: mairix search

2011-05-01 Thread Sebastian Tramp
On Fri, Apr 29, 2011 at 05:03:39PM -0400, Ed Blackman wrote:

Hi Ed,

this sounds really useful. Since I use tmux, I need to change this macro to
control tmux instead but this is not such a problem.

Can you provide a download link for your mairix wrappe?

thx

ST

 On Fri, Apr 29, 2011 at 01:56:51PM +0200, Sebastian Tramp wrote:
 I have a question regarding macros in mutt. Currently I use these two
 macros for searching:
 
 macro generic ,f shell-escapemairix  search via mairix
 macro generic ,,f change-folderkill-line=searchenter load the 
 search results mailbox
 
 Can I optimize it in the way that after finishing the search with ,f -
 the result mailbox is automatically loaded?
 
 I run mutt inside screen, and use the following macro to switch to a
 separate window to run a mairix query.  It will open mutt on the search
 folder if the query is successful.  Otherwise, it will exit, which returns
 me to the original screen session (mutt).
 
 # prompt for mairix search parameters
 macro index,pager,browser ,m \
 enter-command set my_wait_key=\$wait_key wait_key=noenter\
 shell-escapescreen -X screen -t 'mutt search' sh -c 'mairixquery  mutt -f 
 =search'enter\
 enter-commandset wait_key=\$my_wait_key my_wait_keyenter mairix query
 
 In my case, mairixquery is a Perl script that prompts me for the mairix
 search string, gives me yes or no prompts for whether to search threads or
 augment previous results, and saves the most recent 100 searches so that I
 can make edits if the query returns too little or way too much.
 
 It doesn't need to be that fancy for this strategy to work for you, assuming
 you read mutt inside screen.  All it has to do is let you enter mairix
 command line options after the new screen has already been launched.
 



-- 
Sebastian Tramp
WebID: http://sebastian.tramp.name


Re: mairix search

2011-05-01 Thread Sebastian Tramp
On Sat, Apr 30, 2011 at 02:12:51AM +0100, Christian Ebert wrote:

Hi Christian,

this is exactly what I wanted.
Thanks very much!

ST


  I have a question regarding macros in mutt. Currently I use these two
  macros for searching:
  
  macro generic ,f shell-escapemairix  search via mairix
  macro generic ,,f change-folderkill-line=searchenter load the 
  search results mailbox
  
  Can I optimize it in the way that after finishing the search with ,f -
  the result mailbox is automatically loaded?
  
  Is searching / indexing with mairix state of the art or is there a
  better solution available? I am quite happy with that, just wanted to
  ask ;-).
 
 I use the following shell script for interactive mairix search
 from within mutt; it also makes a rudimentary attempt to search
 using a fallback charset for non-ascii strings.
 
 
 #!/bin/sh
 
 threads=
 augment=
 charset=`printf '%s' $LANG | cut -d . -f 2`
 fallback=windows-1252
 
 yorn() {
 local ans
 printf '%s %s ' $1 (yes/[no])
 read -e ans
 case $ans in
 y*|Y*) return 0;;
 *) return 1;;
 esac
 }
 
 printf '%s\n' Enter mairix query:
 read -e query
 test -z $query  exit 0
 
 yorn Retrieve thread(s)?  threads=--threads
 yorn Append message(s) to mfolder?  augment=--augment
 
 query=`echo $query | iconv -f $charset -t UTF-8`
 mairix $threads $augment $query
 
 queryx=`echo $query | iconv -f $charset -t $fallback`
 if [ $queryx != $query ]; then
 echo trying $fallback
 mairix $threads --augment $queryx
 fi
 
 
 muttrc:
 
 set my_mairixfolder=`awk -F '=' '/^mfolder/ { print $2 }' ~/.mairixrc`
 
 macro index,pager ,m \
 enter-command set my_wait_key=\$wait_key wait_key=noenter\
 shell-escape mairixquery.shenter\
 enter-command set wait_key=\$my_wait_key my_wait_keyenter\
 change-folder+$my_mairixfolder mairix query
 
 macro browser ,m \
 enter-command set my_wait_key=\$wait_key wait_key=noenter\
 shell-escape mairixquery.shenter\
 enter-command set wait_key=\$my_wait_key my_wait_keyenter\
 exitchange-folder+$my_mairixfolder mairix query
 
 
 For reverse detection of the original mails I recommend muttjump:
 
 https://github.com/weisslj/muttjump
 
 c

-- 
Sebastian Tramp
WebID: http://sebastian.tramp.name


Re: mairix search

2011-04-29 Thread Christian Ebert
* Sebastian Tramp on Friday, April 29, 2011 at 13:56:51 +0200
 I have a question regarding macros in mutt. Currently I use these two
 macros for searching:
 
 macro generic ,f shell-escapemairix  search via mairix
 macro generic ,,f change-folderkill-line=searchenter load the search 
 results mailbox
 
 Can I optimize it in the way that after finishing the search with ,f -
 the result mailbox is automatically loaded?
 
 Is searching / indexing with mairix state of the art or is there a
 better solution available? I am quite happy with that, just wanted to
 ask ;-).

I use the following shell script for interactive mairix search
from within mutt; it also makes a rudimentary attempt to search
using a fallback charset for non-ascii strings.


#!/bin/sh

threads=
augment=
charset=`printf '%s' $LANG | cut -d . -f 2`
fallback=windows-1252

yorn() {
local ans
printf '%s %s ' $1 (yes/[no])
read -e ans
case $ans in
y*|Y*) return 0;;
*) return 1;;
esac
}

printf '%s\n' Enter mairix query:
read -e query
test -z $query  exit 0

yorn Retrieve thread(s)?  threads=--threads
yorn Append message(s) to mfolder?  augment=--augment

query=`echo $query | iconv -f $charset -t UTF-8`
mairix $threads $augment $query

queryx=`echo $query | iconv -f $charset -t $fallback`
if [ $queryx != $query ]; then
echo trying $fallback
mairix $threads --augment $queryx
fi


muttrc:

set my_mairixfolder=`awk -F '=' '/^mfolder/ { print $2 }' ~/.mairixrc`

macro index,pager ,m \
enter-command set my_wait_key=\$wait_key wait_key=noenter\
shell-escape mairixquery.shenter\
enter-command set wait_key=\$my_wait_key my_wait_keyenter\
change-folder+$my_mairixfolder mairix query

macro browser ,m \
enter-command set my_wait_key=\$wait_key wait_key=noenter\
shell-escape mairixquery.shenter\
enter-command set wait_key=\$my_wait_key my_wait_keyenter\
exitchange-folder+$my_mairixfolder mairix query


For reverse detection of the original mails I recommend muttjump:

https://github.com/weisslj/muttjump

c
-- 
theatre - books - texts - movies
Black Trash Productions at home: http://www.blacktrash.org
Black Trash Productions on Facebook:
http://www.facebook.com/blacktrashproductions


Re: Mairix search results

2007-10-20 Thread Patrick Shanahan
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

* Rem P Roberti [EMAIL PROTECTED] [10-20-07 20:55]:
 Mairix is installed and working.  I have tried to create a macro which
 would allow me to view the results of the search in the mailbox
 (mfolder) that I have created for Mairix finds.  Here it is:
 
 macro generic escf change-folder=mfolder  Search results
 
 When I invoke the macro I get this message: change-folder=mfolder is not
 a mailbox.  What am I doing wrong?

what is the result of:  grep set\ folder= ~/.muttrc

- -- 
Patrick Shanahan Plainfield, Indiana, USAHOG # US1244711
http://wahoo.no-ip.org Photo Album:  http://wahoo.no-ip.org/gallery2
Registered Linux User #207535@ http://counter.li.org
-BEGIN PGP SIGNATURE-
Version: GnuPG v2.0.4-svn4472 (GNU/Linux)

iD8DBQFHGrn8ClSjbQz1U5oRAhlsAKCfmMmd5GfeIwffinrZlqa+pl+ttwCcCJWG
ybhOguex6MYjaxfSsfRjkco=
=lHpt
-END PGP SIGNATURE-