[notmuch] [PATCH 2/2] notmuch list: A new command to produce various lists.

[Carl Worth]

On Thu, 19 Nov 2009 10:59:13 -0700, Bdale Garbee  wrote:
> I've been pondering approaches to prioritizing the pool of unread
> messages.  Most of my thinking so far is along the lines of the ability
> to automatically apply tags to new messages on various criteria combined
> with the ability to manipulate the order in which tags are presented in
> a view like what you're describing.

Yes. There's definitely a lot of room for experimenting here to figure
out how to make things work.

> For better or worse, with about 45k messages hitting my inbox per year
> *after* most of the list traffic gets peeled off and fed to a private
> NNTP server, it's not about reading all of my email any more... it's
> about finding and reading the stuff that actually matters *to me*.

I totally understand that, (even if I've got an order of magnitude less
of non-list mail coming to me). And I hope we can figure out how to support
that mode well.

> Can't tell you how excited I am about what's happening here!

Thanks so much. And it's a pleasure to have you as part of the community
here. I'll look forward to your input.

-Carl

[notmuch] [PATCH] Allow to redefine notmuch binary name and path in elisp mode

[Mikhail Gusarov]

Signed-off-by: Mikhail Gusarov 
---
 notmuch.el |   13 -
 1 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index 1fc54c3..f97950f 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -71,6 +71,9 @@ pattern can still test against the entire line).")
 (defvar notmuch-show-signature-lines-max 12
   "Maximum length of signature that will be hidden by default.")

+(defvar notmuch-command "notmuch"
+  "Notmuch binary location")
+
 (set 'notmuch-show-message-begin-regexp"message{")
 (set 'notmuch-show-message-end-regexp  "message}")
 (set 'notmuch-show-header-begin-regexp "header{")
@@ -251,7 +254,7 @@ buffer."

 (defun notmuch-reply (query-string)
   (switch-to-buffer (generate-new-buffer "notmuch-draft"))
-  (call-process "notmuch" nil t nil "reply" query-string)
+  (call-process notmuch-command nil t nil "reply" query-string)
   (goto-char (point-min))
   (if (re-search-forward "^$" nil t)
   (progn
@@ -690,7 +693,7 @@ thread from that buffer can be show when done with this 
one)."
   (erase-buffer)
   (goto-char (point-min))
   (save-excursion
-   (call-process "notmuch" nil t nil "show" thread-id)
+   (call-process notmuch-command nil t nil "show" thread-id)
(notmuch-show-markup-messages)
)
   (run-hooks 'notmuch-show-hook)
@@ -851,7 +854,7 @@ and will also appear in a buffer named \"*Notmuch 
errors*\"."
   (let ((error-buffer (get-buffer-create "*Notmuch errors*")))
 (with-current-buffer error-buffer
(erase-buffer))
-(if (eq (apply 'call-process "notmuch" nil error-buffer nil args) 0)
+(if (eq (apply 'call-process notmuch-command nil error-buffer nil args) 0)
(point)
   (progn
(with-current-buffer error-buffer
@@ -917,8 +920,8 @@ This function advances point to the next line when 
finished."
   (goto-char (point-min))
   (save-excursion
(if oldest-first
-   (call-process "notmuch" nil t nil "search" "--sort=oldest-first" 
query)
- (call-process "notmuch" nil t nil "search" "--sort=newest-first" 
query))
+   (call-process notmuch-command nil t nil "search" 
"--sort=oldest-first" query)
+ (call-process notmuch-command nil t nil "search" 
"--sort=newest-first" query))
(notmuch-search-markup-thread-ids)
))
 (run-hooks 'notmuch-search-hook)))
-- 
1.6.3.3

[notmuch] link error

[Carl Worth]

On Thu, 19 Nov 2009 19:13:35 +1100, Peter Wang  wrote:
> Linking fails on my system for some reason (undefined references to
> talloc functions).  Putting $(LDFLAGS) after the object list solves
> it.

Thanks Peter.

I've pushed this now.

-Carl

[notmuch] lots of emacs mode enhancements

[Alexander Botero-Lowry]

I've just been diving into the elisp, and have added a bunch of useful
features.

1) buttonized citation and signature expanders and made them locally
collapsable instead of globally (this could take some cleanup to remove
the global key-map binding or make it work again)
2) fixed an annoying warning about cons not being a face name
3) made header names bold to make it easier to distingush them from
their value

My next target is to carry the buttonization through to threads and
headers, and then I'm going to look into doing better mime-handling in
general.

also, RFP includes a -p argument that gives a patch :)

alex


The following changes since commit 9b560fb3eb87b2a4f9d092bc1b124ccb6d11c975:
  Alexander Botero-Lowry (1):
Checkin some command-only tcsh completions

are available in the git repository at:

  git://alexbl.net/notmuch.git master

Alexander Botero-Lowry (5):
  Buttonize citation expander.
  buttonize signatures as well
  fix the message about cons not being a valid face attribute
  Make expanding/collapsing signatures and citations local to them
  make header names bold in show-mode

 notmuch.el |   72 +++
 1 files changed, 57 insertions(+), 15 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index 1fc54c3..bb69aa3 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -440,6 +440,14 @@ which this thread was originally shown."
(if last
(notmuch-show-archive-thread))

+(defun notmuch-toggle-invisible-action (cite-button)
+  (let ((invis-spec (button-get button 'invisibility-spec)))
+(if (invisible-p invis-spec)
+(remove-from-invisibility-spec invis-spec)
+  (add-to-invisibility-spec invis-spec)
+  ))
+  (goto-char (button-end cite-button)))
+
 (defun notmuch-show-markup-citations-region (beg end depth)
   (goto-char beg)
   (beginning-of-line)
@@ -451,25 +459,51 @@ which this thread was originally shown."
  (progn
(while (looking-at citation)
  (forward-line))
-   (let ((overlay (make-overlay beg-sub (point
- (overlay-put overlay 'invisible 'notmuch-show-citation)
- (overlay-put overlay 'before-string
-  (concat indent
-  "[" (number-to-string (count-lines beg-sub 
(point)))
-  "-line citation. Press 'c' to show.]\n")
+   (let ((overlay (make-overlay beg-sub (point)))
+  (invis-spec (make-symbol "notmuch-citation-region")))
+  (add-to-invisibility-spec invis-spec)
+ (overlay-put overlay 'invisible invis-spec)
+  (let (
+(p (point))
+(cite-button-text
+ (concat "["  (number-to-string (count-lines beg-sub 
(point)))
+ "-line citation.]"))
+)
+(goto-char (- beg-sub 1))
+(insert (concat "\n" indent))
+(let ((cite-button (insert-button cite-button-text)))
+  (button-put cite-button 'invisibility-spec invis-spec)
+  (button-put cite-button 'action 
'notmuch-toggle-invisible-action)
+  (button-put cite-button 'help-echo
+  "mouse-2, RET: Show citation")
+
+  )
+(insert "\n")
+(goto-char (+ (length cite-button-text) p))
+  
   (move-to-column depth)
   (if (looking-at notmuch-show-signature-regexp)
  (let ((sig-lines (- (count-lines beg-sub end) 1)))
(if (<= sig-lines notmuch-show-signature-lines-max)
(progn
- (overlay-put (make-overlay beg-sub end)
-  'invisible 'notmuch-show-signature)
- (overlay-put (make-overlay beg (- beg-sub 1))
-  'after-string
-  (concat "\n" indent
-  "[" (number-to-string sig-lines)
-  "-line signature. Press 's' to show.]"))
- (goto-char end)
+  (let ((invis-spec (make-symbol "notmuch-signature-region")))
+(add-to-invisibility-spec invis-spec)
+(overlay-put (make-overlay beg-sub end)
+ 'invisible invis-spec)
+  
+(goto-char (- beg-sub 1))
+(insert (concat "\n" indent))
+(let ((sig-button (insert-button 
+   (concat "[" (number-to-string sig-lines)
+ "-line signature.]"
+  (button-put sig-button 'invisibility-spec invis-spec)
+  (button-put sig-button 'action
+  

[notmuch] notmuch new: Memory problem

[Dominik Epple]

Hi,

I am strongly interested in giving notmuch a try. But I fail setting
it up. The problem is that during "notmuch new", memory consumption
and system load increases to values that make my system unusable. I
then killed "notmuch new" at a memory consumption of 2.7G and at a
system load of 7.

After hitting Ctrl-C, it says "Stopping" but does not stop. I then
killed "notmuch new" after some minutes with signal KILL.

Is there a problem with the number of my mails? I currently have over
40.000 Mails... they live currently in mbox files, I created a Maildir
with mb2md-3.20.pl.

OS is SuSE Linux 11.1, kernel 2.6.27.29-0.1-default, notmuch pulled
today from git, compiled manually, dependencies also downloaded and
installed manually, in the following versions:

gmime-2.4.11.tar.bz2
talloc-2.0.0.tar.gz
xapian-core-1.0.17.tar.gz

Any help?

Thanks
Dominik

[notmuch] [PATCH 1/2] Improve installation of emacs mode.

[Carl Worth]

On Thu, 19 Nov 2009 15:20:01 -0600, "Jeffrey C. Ollie"  
wrote:
> 1) Add a separate targets to build and install emacs mode.
> 
> 2) Don't hardcode the installation directory, instead use emacs'
>pkg-config module.
> 
> 3) Install a byte compiled version of the emacs mode.
> 
> 4) Install the emacs mode in emacs' site-lisp directory.  Put
>"(require 'notmuch)" in your .emacs to load it automatically.
> 
> 5) Ignore byte-compiled emacs files.

Thanks, Jeffrey!

This is pushed now.

-Carl

[notmuch] RPM spec file

[Carl Worth]

On Thu, 19 Nov 2009 00:56:22 -0600, Jeffrey Ollie  wrote:
> I don't think that a separate Git branch makes sense, but maybe stuff
> like this should be in a subdirectory like "packaging/fedora", since I
> imagine that there will eventually be a need for "packaging/debian"
> etc. as well.

OK, Done.

Thanks again!

-Carl

[notmuch] Gentoo ebuild for notmuch

[Carl Worth]

On Thu, 19 Nov 2009 00:11:46 -0800 (PST), James Rowe  
wrote:
>   Just in case other Gentoo users are trying notmuch out I thought I'd post my
> ebuild(perhaps you'll make it better for me too :).  It is working well up to
> at least e5da2b70.

Hi James,

Thanks for your contribution.

>   I won't bother the list if it requires changes, as it is available from my
> main overlay[1] if you wish to check for updated versions.

If you think it makes sense, I can add this to the notmuch repository in
a packaging/gentoo directory. Just let me know if you'd like that.

-Carl

[notmuch] Segfault searching for tags

[Carl Worth]

On Thu, 19 Nov 2009 16:45:43 +0100, Adrian Perez de Castro  wrote:
> The thing is that in notmuch_message_get_in_reply_to(), line 288, a NULL
> instance of Xapian::TermIterator is dereferenced. In my particular case,
> the culpript is a cache file of Claws-Mail, as seen in the following GDB
> session:

Not quite NULL, (nor is it quite dereferencing---this is nasty C++
overloading), but yeah, the idea is the same. We need to protect all of
our "calls" to this overloaded operator to not call it when the iterator
is equal to the value returned by termlist_end ().

On Thu, 19 Nov 2009 20:23:15 -0600, Jeffrey Ollie  wrote:
> I straced some of the crashes, and the last file that was read before
> the crash was a malformed message.  I've attached one of the messages.
>  I've been using offlineimap to sync my gmail mailbox to my laptop so
> that I can use notmuch.  offlineimap isn't the most stable program,
> but I'm not sure yet if offlineimap is causing the problem or if
> that's the way the message is in gmail.

Thanks for the file. I never like to push code that I haven't tested, so
this was very helpful.

Below is the patch that I just pushed which seems to do the trick.

-Carl

commit 31b54bc78735c628035a046e526ac4c596d830cf
Author: Carl Worth 
Date:   Fri Nov 20 12:06:11 2009 +0100

Avoid access of a Xapian iterator's object when there's nothing
there.

This eliminates a crash when a message (either corrupted or a
non-mail
file that wasn't properly detected as not being mail) has no
In-Reply-To
header, (and so few terms that trying to skip to the prefix of the
In-Reply-To terms actually brings us to the end of the termlist).

diff --git a/lib/message.cc b/lib/message.cc
index 9488fb6..410 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -285,7 +285,8 @@ _notmuch_message_get_in_reply_to (notmuch_message_t
*message
 i = message->doc.termlist_begin ();
 i.skip_to (prefix);

-in_reply_to = *i;
+if (i != message->doc.termlist_end ())
+   in_reply_to = *i;

 /* It's perfectly valid for a message to have no In-Reply-To
  * header. For these cases, we return an empty string. */
@@ -332,10 +333,10 @@ notmuch_message_get_thread_id (notmuch_message_t
 *message)
return message->thread_id;

 i = message->doc.termlist_begin ();
-
 i.skip_to (prefix);

-id = *i;
+if (i != message->doc.termlist_end ())
+   id = *i;

 if (i == message->doc.termlist_end () || id[0] != *prefix)
INTERNAL_ERROR ("Message with document ID of %d has no thread
ID.\n",
@@ -466,7 +467,7 @@ notmuch_message_get_tags (notmuch_message_t
*message)

 i.skip_to (prefix);

-while (1) {
+while (i != end) {
tag = *i;

if (tag.empty () || tag[0] != *prefix)

[notmuch] [PATCH] Allow to redefine notmuch binary name and path in elisp mode

[Carl Worth]

On Fri, 20 Nov 2009 07:15:40 +0600, Mikhail Gusarov  wrote:
> Signed-off-by: Mikhail Gusarov 

Thanks for the patch, Mikhail!

This is pushed now.

-Carl

PS. Some people may have noticed that the original mail from Mikhail put
notmuch-show-mode into an infinite loop. The bug is triggered by the
mail message containing a notmuch message marker, "^lmessage{" in the
body, (but with a literal ^L).

I haven't fixed the infinite loop yet, and it would be good to come up
with a way to do ignore delimiters internal to the body of a message.
Until then, I've changed the notmuch.el code to use "\f" instead of
literal ^L so we at least won't hit this bug with patches to notmuch.el
anymore.

[notmuch] [PATCH] Makefile: Make object targets depend on Makefiles

[Carl Worth]

On Thu, 19 Nov 2009 01:37:25 +0100, Jan Janak  wrote:
> All objects need to be recompiled when any of the Makefiles changes, so
> we make them all depend on all the Makefiles.

An excellent fix, Jan. Pushed!

-Carl

[notmuch] [PATCH] Add notmuch.1.gz to files to be cleaned

[Mikhail Gusarov]

Signed-off-by: Mikhail Gusarov 
---
 Makefile.local |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/Makefile.local b/Makefile.local
index ecd4ceb..bf81c03 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -45,4 +45,4 @@ install-emacs: install emacs
install -m0644 notmuch.elc $(DESTDIR)$(emacs_lispdir)

 SRCS  := $(SRCS) $(notmuch_client_srcs)
-CLEAN := $(CLEAN) notmuch $(notmuch_client_modules) notmuch.elc
+CLEAN := $(CLEAN) notmuch $(notmuch_client_modules) notmuch.elc notmuch.1.gz
-- 
1.6.3.3

[notmuch] Segfault searching for tags

[Jeffrey Ollie]

On Fri, Nov 20, 2009 at 5:32 AM, Carl Worth  wrote:
> On Thu, 19 Nov 2009 16:45:43 +0100, Adrian Perez de Castro  igalia.com> wrote:
>> The thing is that in notmuch_message_get_in_reply_to(), line 288, a NULL
>> instance of Xapian::TermIterator is dereferenced. In my particular case,
>> the culpript is a cache file of Claws-Mail, as seen in the following GDB
>> session:
>
> Not quite NULL, (nor is it quite dereferencing---this is nasty C++
> overloading), but yeah, the idea is the same. We need to protect all of
> our "calls" to this overloaded operator to not call it when the iterator
> is equal to the value returned by termlist_end ().
>
> On Thu, 19 Nov 2009 20:23:15 -0600, Jeffrey Ollie  wrote:
>> I straced some of the crashes, and the last file that was read before
>> the crash was a malformed message. ?I've attached one of the messages.
>
> Thanks for the file. I never like to push code that I haven't tested, so
> this was very helpful.
>
> Below is the patch that I just pushed which seems to do the trick.

Ah, excellent!  This does indeed seem to prevent the crash.  Now I
just need to figure out how to get all my mail out of GMail.

-- 
Jeff Ollie

[notmuch] [PATCH] RFC: quiet make

[Mikhail Gusarov]

I don't entirely like duplicating every command line in makefile,
so this patch is RFC. Someone with bigger Make-fu than mine probably
knows a better way.

Signed-off-by: Mikhail Gusarov 
---
 Makefile   |   36 
 Makefile.local |   10 ++
 lib/Makefile.local |5 +
 3 files changed, 51 insertions(+), 0 deletions(-)

diff --git a/Makefile b/Makefile
index b6861e9..72a72ae 100644
--- a/Makefile
+++ b/Makefile
@@ -32,29 +32,65 @@ include lib/Makefile.local
 include Makefile.config

 %.o: %.cc $(all_deps)
+ifeq ($(V),1)
$(CXX) -c $(CFLAGS) $(CXXFLAGS) $< -o $@
+else
+   @echo CXX $<
+   @$(CXX) -c $(CFLAGS) $(CXXFLAGS) $< -o $@
+endif

 %.o: %.c $(all_deps)
+ifeq ($(V),1)
$(CC) -c $(CFLAGS) $< -o $@
+else
+   @echo CC $<
+   @$(CC) -c $(CFLAGS) $< -o $@
+endif

 %.elc: %.el
+ifeq ($(V),1)
emacs -batch -f batch-byte-compile $<
+else
+   @echo ELCOMPILE $<
+   @emacs -batch -f batch-byte-compile $<
+endif

 .deps/%.d: %.c $(all_deps)
+ifeq ($(V),1)
+   set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
+   $(CC) -M $(CPPFLAGS) $(CFLAGS) $< > $@.; \
+   sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@. > $@; \
+   rm -f $@.
+else
+   @echo DEPCXX $<
@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
$(CC) -M $(CPPFLAGS) $(CFLAGS) $< > $@.; \
sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@. > $@; \
rm -f $@.
+endif

 .deps/%.d: %.cc $(all_deps)
+ifeq ($(V),1)
+   set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
+   $(CXX) -M $(CPPFLAGS) $(CXXFLAGS) $< > $@.; \
+   sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@. > $@; \
+   rm -f $@.
+else
+   @echo DEPCC $<
@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
$(CXX) -M $(CPPFLAGS) $(CXXFLAGS) $< > $@.; \
sed 's,'$$(basename $*)'\.o[ :]*,$*.o $@ : ,g' < $@. > $@; \
rm -f $@.
+endif

 DEPS := $(SRCS:%.c=.deps/%.d)
 DEPS := $(DEPS:%.cc=.deps/%.d)
 -include $(DEPS)

 clean:
+ifeq ($(V),1)
rm -f $(CLEAN); rm -rf .deps
+else
+   @echo CLEAN
+   @rm -f $(CLEAN); rm -rf .deps
+endif
diff --git a/Makefile.local b/Makefile.local
index bf81c03..0addfed 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -20,10 +20,20 @@ notmuch_client_srcs =   \

 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
 notmuch: $(notmuch_client_modules) lib/notmuch.a
+ifeq ($(V),1)
$(CXX) $^ $(LDFLAGS) -o $@
+else
+   @echo LINK $^
+   @$(CXX) $^ $(LDFLAGS) -o $@
+endif

 notmuch.1.gz: notmuch.1
+ifeq ($(V),1)
gzip --stdout notmuch.1 > notmuch.1.gz
+else
+   @echo GZIP $<
+   @gzip --stdout notmuch.1 > notmuch.1.gz
+endif

 install: all notmuch.1.gz
for d in $(DESTDIR)$(prefix)/bin/ $(DESTDIR)$(prefix)/share/man/man1 \
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 79f7b0b..5a66716 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -18,7 +18,12 @@ libnotmuch_cxx_srcs =\

 libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 $(dir)/notmuch.a: $(libnotmuch_modules)
+ifeq ($(V),1)
$(AR) rcs $@ $^
+else
+   @echo AR $^
+   @$(AR) rcs $@ $^
+endif

 SRCS  := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs)
 CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a
-- 
1.6.3.3

[notmuch] Segfault searching for tags

[Jan Janak]

On Fri, Nov 20, 2009 at 2:10 PM, Jeffrey Ollie  wrote:
> On Fri, Nov 20, 2009 at 5:32 AM, Carl Worth  wrote:
>> On Thu, 19 Nov 2009 16:45:43 +0100, Adrian Perez de Castro > igalia.com> wrote:
>>> The thing is that in notmuch_message_get_in_reply_to(), line 288, a NULL
>>> instance of Xapian::TermIterator is dereferenced. In my particular case,
>>> the culpript is a cache file of Claws-Mail, as seen in the following GDB
>>> session:
>>
>> Not quite NULL, (nor is it quite dereferencing---this is nasty C++
>> overloading), but yeah, the idea is the same. We need to protect all of
>> our "calls" to this overloaded operator to not call it when the iterator
>> is equal to the value returned by termlist_end ().
>>
>> On Thu, 19 Nov 2009 20:23:15 -0600, Jeffrey Ollie  wrote:
>>> I straced some of the crashes, and the last file that was read before
>>> the crash was a malformed message. ?I've attached one of the messages.
>>
>> Thanks for the file. I never like to push code that I haven't tested, so
>> this was very helpful.
>>
>> Below is the patch that I just pushed which seems to do the trick.
>
> Ah, excellent! ?This does indeed seem to prevent the crash. ?Now I
> just need to figure out how to get all my mail out of GMail.

I did exactly that with offlineimap. It crashes from time to time, but
then you can just restart it and continue.

A few days ago I sent a patch which converts mail subdirectories to
tags and because Gmail IMAP server converts labels to subdirectories,
you can use that to convert gmail's labels to notmuch tags
automatically.

   -- Jan

[notmuch] Segfault with weird Message-ID

[Mike Hommey]

Hi,

I got a segfault when importing my maildir. It happened because of an
old weird email, where the message-id is the following:
Message-ID: <22b17a1f$4fbe$0550 at myrop (ew6.southwind.net 
[216.53.98.70]) by onyx.southwind.net from homepage.com (114.230.197.216) by 
newmail.spectraweb.ch from default (m202.2-25.warwick.net [
218.242.202.80]) by host.warwick.net (8.10.0.Beta10/8.10.0.Beta10) with SMTP id 
e9GKEKk19201>

I have absolutely no idea how it got this value, but the mail being
an archived 8 years old spam, I'm not exactly sure if anyone would 
still expect such message id to occur.

Anyways, the stack dump is the following:
#0  0x76d1e598 in Xapian::Document::add_term(std::string const&, 
unsigned int) () from /usr/lib/libxapian.so.15
#1  0x0040f5ff in _notmuch_message_add_term (message=0x0, 
prefix_name=0x41ad7f "tag", value=0x4191b0 "inbox") at lib/message.cc:587
#2  0x0040f827 in notmuch_message_add_tag (message=0x0, tag=0x4191b0 
"inbox") at lib/message.cc:668
#3  0x00407bc8 in tag_inbox_and_unread (message=0x0) at notmuch-new.c:44
#4  0x00407f63 in add_files_recursive (notmuch=0x62cc20, path=0x832e90 
"/home/mh/Maildir/saved-messages/cur", st=0x7fffe000, state=0x7fffe240) 
at notmuch-new.c:185
#5  0x00408036 in add_files_recursive (notmuch=0x62cc20, path=0x832de0 
"/home/mh/Maildir/saved-messages", st=0x7fffe000, state=0x7fffe240) at 
notmuch-new.c:223
#6  0x00408036 in add_files_recursive (notmuch=0x62cc20, path=0x62c920 
"/home/mh/Maildir", st=0x7fffe000, state=0x7fffe240) at 
notmuch-new.c:223
#7  0x00408245 in add_files (notmuch=0x62cc20, path=0x62c920 
"/home/mh/Maildir", state=0x7fffe240) at notmuch-new.c:287
#8  0x00408704 in notmuch_new_command (ctx=0x61f140, argc=0, 
argv=0x7fffe3e8) at notmuch-new.c:431
#9  0x00406ea8 in main (argc=2, argv=0x7fffe3d8) at notmuch.c:400

And the most likely problem is that message is NULL.

Now, looking at the code, there seems to me there actually 3 problems:
- _notmuch_message_create_for_message_id can return NULL, and while
  there is a test for it in notmuch_database_add_message, the function
  still returns a success code
- things are still going on even when message is NULL in
  add_files_recursive
- for some reason, xapian doesn't want to add the document corresponding
  to this old spam message: notmuch->xapian_db->add_document throws an
  exception.

I can provide the spam if necessary, or can continue debugging the issue
with some guidance.

Cheers,

Mike

[notmuch] [PATCH 1/2] zsh-completion: Initial zsh-completion for notmuch

[Ingmar Vanhassel]

Signed-off-by: Ingmar Vanhassel 
---
 contrib/notmuch-completion.zsh |   74 
 1 files changed, 74 insertions(+), 0 deletions(-)
 create mode 100644 contrib/notmuch-completion.zsh

Moved & renamed as you suggested.

diff --git a/contrib/notmuch-completion.zsh b/contrib/notmuch-completion.zsh
new file mode 100644
index 000..67a9aba
--- /dev/null
+++ b/contrib/notmuch-completion.zsh
@@ -0,0 +1,74 @@
+#compdef notmuch
+
+# ZSH completion for `notmuch`
+# Copyright ?? 2009 Ingmar Vanhassel 
+
+_notmuch_commands()
+{
+  local -a notmuch_commands
+  notmuch_commands=(
+'setup:interactively set up notmuch for first use'
+'new:find and import any new message to the database'
+'search:search for messages matching the search terms, display matching 
threads as results'
+'reply:constructs a reply template for a set of messages'
+'show:show all messages matching the search terms'
+'tag:add or remove tags for all messages matching the search terms'
+'dump:creates a plain-text dump of the tags of each message'
+'restore:restores the tags from the given file'
+'help:show details on a command'
+  )
+
+  _describe -t command 'command' notmuch_commands
+}
+
+_notmuch_dump()
+{
+  _files
+}
+
+_notmuch_help_topics()
+{
+  local -a notmuch_help_topics
+  notmuch_help_topics=(
+'search-terms:show common search-terms syntax'
+  )
+  _describe -t notmuch-help-topics 'topic' notmuch_help_topics
+}
+
+_notmuch_help()
+{
+  _alternative \
+_notmuch_commands \
+_notmuch_help_topics
+}
+
+_notmuch_restore()
+{
+  _files
+}
+
+_notmuch_search()
+{
+  _arguments -s : \
+'--max-threads=[display only the first x threads from the search 
results]:number of threads to show: ' \
+'--first=[omit the first x threads from the search results]:number of 
threads to omit: ' \
+'--sort=[sort results]:sorting:((newest-first\:"reverse chronological 
order" oldest-first\:"chronological order"))'
+}
+
+_notmuch()
+{
+  if (( CURRENT > 2 )) ; then
+local cmd=${words[2]}
+curcontext="${curcontext%:*:*}:notmuch-$cmd"
+(( CURRENT-- ))
+shift words
+_call_function ret _notmuch_$cmd
+return ret
+  else
+_notmuch_commands
+  fi
+}
+
+_notmuch "$@"
+
+# vim: set sw=2 sts=2 ts=2 et ft=zsh :
-- 
1.6.5.2.433.g23cdb

[notmuch] [PATCH 2/2] bash-completion: Move to contrib

[Ingmar Vanhassel]

Signed-off-by: Ingmar Vanhassel 
---
 Makefile.local |2 +-
 .../notmuch-completion.bash|0
 2 files changed, 1 insertions(+), 1 deletions(-)
 rename notmuch-completion.bash => contrib/notmuch-completion.bash (100%)

Move to contrib/ for consistency with zsh completion.

diff --git a/Makefile.local b/Makefile.local
index ecd4ceb..74a72aa 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -33,7 +33,7 @@ install: all notmuch.1.gz
done ;
install notmuch $(DESTDIR)$(prefix)/bin/
install -m0644 notmuch.1.gz $(DESTDIR)$(prefix)/share/man/man1/
-   install notmuch-completion.bash \
+   install contrib/notmuch-completion.bash \
$(DESTDIR)/etc/bash_completion.d/notmuch

 install-emacs: install emacs
diff --git a/notmuch-completion.bash b/contrib/notmuch-completion.bash
similarity index 100%
rename from notmuch-completion.bash
rename to contrib/notmuch-completion.bash
-- 
1.6.5.2.433.g23cdb

[notmuch] Recommended Coding Style?

[Jan Janak]

Hi Guys,

Is there any recommended coding style for the C/C++ code in notmuch?

  -- Jan

[notmuch] Recommended Coding Style?

[Chris Wilson]

Excerpts from Jan Janak's message of Fri Nov 20 15:08:31 + 2009:
> Is there any recommended coding style for the C/C++ code in notmuch?

My prediction is that Carl will go with something close to what he wrote
for Cairo:

  http://cgit.freedesktop.org/cairo/tree/CODING_STYLE

He may or may not change to a more kernel style, but he definitely will
not mandate 3-space indents. ;-)
-ickle
-- 
Chris Wilson, Intel Open Source Technology Centre

[notmuch] [PATCH] notmuch.el: Add face support to search and show mode

[Aneesh Kumar K.V]

This add two faces, notmuch-show-subject-face and
notmuch-tag-unread-face. The first face is used to show the subject
line in the notmuch-show-mode and the second one the unread tag in
the notmuch-search-mode.

The changes are done looking at message.el in emacs source

Signed-off-by: Aneesh Kumar K.V 
---
 notmuch.el |   55 ++-
 1 files changed, 50 insertions(+), 5 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index 4b2936a..71e9dea 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -603,9 +603,37 @@ which this thread was originally shown."
   (force-window-update)
   (redisplay t))

+(defun notmuch-font-lock-matcher (regexp)
+  (let ((form
+ `(lambda (limit)
+(let ((start (point)))
+  (save-restriction
+(widen)
+(goto-char (point-min))
+   (setq limit (min limit (point-max)))
+(goto-char start))
+  (and (< start limit)
+   (re-search-forward ,regexp limit t))
+(if (featurep 'bytecomp)
+(byte-compile form)
+  form)))
+
+(defface notmuch-show-subject-face
+ 'class color) (background light)) (:foreground "yellow" :bold t))
+(((class color) (background dark)) (:foreground "yellow" :bold t)))
+  "Notmuch show mode face used to highligh subject line."
+  :group 'notmuch)
+
+(defvar notmuch-show-font-lock-keywords
+  (let ((content ""))
+`((,(notmuch-font-lock-matcher
+(concat "\\(Subject:.*$\\)" content))
+(1 'notmuch-show-subject-face nil t
+  "Additonal expression to hightlight in notmuch-search-mode")
+
 ;;;###autoload
-(defun notmuch-show-mode ()
-  "Major mode for viewing a thread with notmuch.
+(define-derived-mode notmuch-show-mode text-mode "notmuch-show"
+"Major mode for viewing a thread with notmuch.

 This buffer contains the results of the \"notmuch show\" command
 for displaying a single thread of email from your email archives.
@@ -643,7 +671,9 @@ view, (remove the \"inbox\" tag from each), with
   (use-local-map notmuch-show-mode-map)
   (setq major-mode 'notmuch-show-mode
mode-name "notmuch-show")
-  (setq buffer-read-only t))
+  (setq buffer-read-only t)
+  (set (make-local-variable 'font-lock-defaults)
+ '(notmuch-show-font-lock-keywords t)))

 ;;;###autoload

@@ -769,8 +799,21 @@ thread from that buffer can be show when done with this 
one)."
   (end-of-buffer arg)
   (forward-line -1))

+(defface notmuch-tag-unread-face
+ 'class color) (background light)) (:foreground "goldenrod" :bold t))
+(((class color) (background dark)) (:foreground "goldenrod" :bold t)))
+  "Notmuch search mode face used to highligh inbox tags."
+  :group 'notmuch)
+
+(defvar notmuch-search-font-lock-keywords
+  (let ((content ""))
+`((,(notmuch-font-lock-matcher
+(concat "\\(unread\\)" content))
+(1 'notmuch-tag-unread-face nil t
+  "Additonal expression to hightlight in notmuch-search-mode")
+
 ;;;###autoload
-(defun notmuch-search-mode ()
+(define-derived-mode notmuch-search-mode text-mode "notmuch-search"
   "Major mode for searching mail with notmuch.

 This buffer contains the results of a \"notmuch search\" of your
@@ -799,7 +842,9 @@ global search.
   (setq truncate-lines t)
   (setq major-mode 'notmuch-search-mode
mode-name "notmuch-search")
-  (setq buffer-read-only t))
+  (setq buffer-read-only t)
+  (set (make-local-variable 'font-lock-defaults)
+ '(notmuch-search-font-lock-keywords t)))

 (defun notmuch-search-find-thread-id ()
   (save-excursion
-- 
1.6.5.2.74.g610f9

[notmuch] [PATCH] When a search query triggers a Xapian exception, log what the query was.

[Eric Anholt]

In my script containing a series of queries to be run on new mail for
setting up tags, it's nice to see which query I typed wrong.

Signed-off-by: Eric Anholt 
---
 lib/query.cc |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/lib/query.cc b/lib/query.cc
index 75f22b3..ea521dd 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -151,6 +151,7 @@ notmuch_query_search_messages (notmuch_query_t *query,
 } catch (const Xapian::Error ) {
fprintf (stderr, "A Xapian exception occurred: %s\n",
 error.get_msg().c_str());
+   fprintf (stderr, "Query string was: %s\n", query->query_string);
 }

 return _notmuch_messages_create (message_list);
-- 
1.6.4.3

[notmuch] [PATCH] Drop redundant CFLAGS, was already included in CXXFLAGS

[Jed Brown]

---
 Makefile |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
-- next part --
A non-text attachment was scrubbed...
Name: 0001-Drop-redundant-CFLAGS-was-already-included-in-CXXFLA.patch
Type: text/x-patch
Size: 330 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091120/b5d44dab/attachment-0001.bin>

[notmuch] notmuch new: Memory problem

[Carl Worth]

On Fri, 20 Nov 2009 09:56:50 +0100, Dominik Epple  wrote:
> I am strongly interested in giving notmuch a try.

Welcome to notmuch, Dominik! I'm sorry your initial attempt to use it
hasn't been quite as smooth as we might like.

>   But I fail setting
> it up. The problem is that during "notmuch new", memory consumption
> and system load increases to values that make my system unusable. I
> then killed "notmuch new" at a memory consumption of 2.7G and at a
> system load of 7.

Yikes. That really sounds like something ran out of control consuming
memory. I certainly haven't seen anything like that before.

> After hitting Ctrl-C, it says "Stopping" but does not stop. I then
> killed "notmuch new" after some minutes with signal KILL.

After "Stopping" gets printed, the notmuch code won't be doing any more
work. It is expected that it will take some time after that message is
printed before notmuch will actually exit. The extra time is to wait for
Xapian to flush out to disk data that notmuch has already provided to
it.

I'm curious how big your .notmuch directory ended up after this
operation. (And how that compares in size to the total size of your
collection of mail.)

> Is there a problem with the number of my mails? I currently have over
> 40.000 Mails... they live currently in mbox files, I created a Maildir
> with mb2md-3.20.pl.

That's definitely not too much mail. I think you should expect "notmuch
new" currently to index on the order of 10 - 100 messages/sec.

Your "notmuch new" process should have been reporting a count once per
second as it progressed, (at least until things went wrong). How far did
you see that go?

I'm wondering if there's a particular file (or files) that are
triggering the bad behavior. Maybe we need a debug option for "notmuch
new" to print the filenames of messages as they are being processed.

-Carl

[notmuch] Gentoo ebuild for notmuch

[James Rowe]

On Fri, 20 Nov 2009 10:39:35 +0100, Carl Worth  wrote:
> If you think it makes sense, I can add this to the notmuch repository in
> a packaging/gentoo directory. Just let me know if you'd like that.

  Hmm... The problem is the ebuild can't currently use the install
target because the file locations are incorrect for Gentoo[1]. Which
means it could require quite a bit of churn to keep it synced as the
build process changes.

  The patch that follows makes it easier to use the install target
within the ebuild. If this sort of change is acceptable then the ebuild
will be simpler, and it is more likely to stay working without me
kicking heaps of patches at you. Given that I'd say "yes, please include
it."

Thanks,

James
 1. fex. Bash completion files are installed in to
/usr/share/bash-completion, then enabled via a link in
/etc/bash_completion.d if desired

[notmuch] [PATCH] Make bash completion directory configurable.

[James Rowe]

Some systems install completion scripts in /usr/share/bash-completion, make the
location configurable from Makefile.config.
---
 Makefile.config |1 +
 Makefile.local  |4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/Makefile.config b/Makefile.config
index 63c90a8..d72a39e 100644
--- a/Makefile.config
+++ b/Makefile.config
@@ -1 +1,2 @@
 prefix = /usr/local
+bash_completion_dir = /etc/bash_completion.d
diff --git a/Makefile.local b/Makefile.local
index ecd4ceb..1017a8c 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -27,14 +27,14 @@ notmuch.1.gz: notmuch.1

 install: all notmuch.1.gz
for d in $(DESTDIR)$(prefix)/bin/ $(DESTDIR)$(prefix)/share/man/man1 \
-   $(DESTDIR)/etc/bash_completion.d/ ; \
+   $(DESTDIR)$(bash_completion_dir) ; \
do \
install -d $$d ; \
done ;
install notmuch $(DESTDIR)$(prefix)/bin/
install -m0644 notmuch.1.gz $(DESTDIR)$(prefix)/share/man/man1/
install notmuch-completion.bash \
-   $(DESTDIR)/etc/bash_completion.d/notmuch
+   $(DESTDIR)$(bash_completion_dir)/notmuch

 install-emacs: install emacs
for d in $(DESTDIR)/$(emacs_lispdir) ; \
-- 
1.6.4.4

[notmuch] Segfault searching for tags

[Carl Worth]

On Fri, 20 Nov 2009 14:20:45 +0100, Jan Janak  wrote:
> > Ah, excellent! ?This does indeed seem to prevent the crash. ?Now I
> > just need to figure out how to get all my mail out of GMail.
> 
> I did exactly that with offlineimap. It crashes from time to time, but
> then you can just restart it and continue.

It sounds like a lot of people are wanting to do that. So it probably
wouldn't hurt for someone who's done it to write up instructions for
that, (or post a link to existing instructions).

I'd say that it might make sense to put these up in a wiki, (and I'll
probably eventually setup an ikiwiki instance for notmuchmail.org just
like I do for cworth.org and cairographics.org). But then again, we're
*already* using email, and I've got this half-baked idea about being
able to just tag useful posts and have them appear on the webpage.

> A few days ago I sent a patch which converts mail subdirectories to
> tags and because Gmail IMAP server converts labels to subdirectories,
> you can use that to convert gmail's labels to notmuch tags
> automatically.

I haven't lost that. Sorry it's taking me a bit to get to it, though!

I think there were two different patches from two people for this
feature. So I've got those both tagged and will review them soon.

-Carl

[notmuch] Segfault with weird Message-ID

[Carl Worth]

On Fri, 20 Nov 2009 14:26:25 +0100, Mike Hommey  
wrote:
> I can provide the spam if necessary, or can continue debugging the issue
> with some guidance.

It sounds strange to say it, but yes, please send me that spam!

-Carl

[notmuch] [PATCH] Checkin some command-only tcsh completions

[Carl Worth]

On Wed, 18 Nov 2009 17:15:43 -0800, Alexander Botero-Lowry  wrote:
> diff --git a/notmuch-completion.tcsh b/notmuch-completion.tcsh

Thanks, Alexander.

I moved this down into contrib where our other completion scripts are
and pushed it out.

-Carl

[notmuch] Segfault with weird Message-ID

[Carl Worth]

On Fri, 20 Nov 2009 14:26:25 +0100, Mike Hommey <mh+notmuch at glandium.org> 
wrote:
> I got a segfault when importing my maildir. It happened because of an
> old weird email, where the message-id is the following:
> Message-ID: <22b17a1f$4fbe$0550 at myrop (ew6.southwind.net 
> [216.53.98.70]) by onyx.southwind.net from homepage.com (114.230.197.216) by 
> newmail.spectraweb.ch from default (m202.2-25.warwick.net [
> 218.242.202.80]) by host.warwick.net (8.10.0.Beta10/8.10.0.Beta10) with SMTP 
> id e9GKEKk19201>

Thanks for sharing this Mike, (and for sending me the original file).

> Anyways, the stack dump is the following:
> #0  0x76d1e598 in Xapian::Document::add_term(std::string const&, 
> unsigned int) () from /usr/lib/libxapian.so.15
> #1  0x0040f5ff in _notmuch_message_add_term (message=0x0, 
> prefix_name=0x41ad7f "tag", value=0x4191b0 "inbox") at lib/message.cc:587
> #2  0x0040f827 in notmuch_message_add_tag (message=0x0, tag=0x4191b0 
> "inbox") at lib/message.cc:668
> #3  0x00407bc8 in tag_inbox_and_unread (message=0x0) at 
> notmuch-new.c:44
> #4  0x00407f63 in add_files_recursive (notmuch=0x62cc20, 
> path=0x832e90 "/home/mh/Maildir/saved-messages/cur", st=0x7fffe000, 
> state=0x7fffe240) at notmuch-new.c:185
> #5  0x00408036 in add_files_recursive (notmuch=0x62cc20, 
> path=0x832de0 "/home/mh/Maildir/saved-messages", st=0x7fffe000, 
> state=0x7fffe240) at notmuch-new.c:223
> #6  0x00408036 in add_files_recursive (notmuch=0x62cc20, 
> path=0x62c920 "/home/mh/Maildir", st=0x7fffe000, state=0x7fffe240) at 
> notmuch-new.c:223
> #7  0x00408245 in add_files (notmuch=0x62cc20, path=0x62c920 
> "/home/mh/Maildir", state=0x7fffe240) at notmuch-new.c:287
> #8  0x00408704 in notmuch_new_command (ctx=0x61f140, argc=0, 
> argv=0x7fffe3e8) at notmuch-new.c:431
> #9  0x00406ea8 in main (argc=2, argv=0x7fffe3d8) at notmuch.c:400

I didn't get the same crash when importing the file. But I did get a
short document out of it (just a handful of terms indexed) and most
significantly, an empty message-ID term.

Xapian has a limit on the maximum length of a term, so one thing we'll
need to do here is to notice if the message ID exceeds that length and
then treat it as a we treat a missing Message-ID header, (that is,
generate our own message ID by computing a sha-1 hash over the message).

So, there was an obvious bug in the message-ID handling, (the code was
still looking for NULL for a missing header, but we now return "" for a
missing header instead). I've fixed this.

> Now, looking at the code, there seems to me there actually 3 problems:
> - _notmuch_message_create_for_message_id can return NULL, and while
>   there is a test for it in notmuch_database_add_message, the function
>   still returns a success code

Thanks. This is fixed now.

> - things are still going on even when message is NULL in
>   add_files_recursive

I didn't replicate this case, but it *should* be fixed now that
notmuch_database_add_message is returning a non-success value.

> - for some reason, xapian doesn't want to add the document corresponding
>   to this old spam message: notmuch->xapian_db->add_document throws an
>   exception.

I think things had just gone wrong long before then.

> I can provide the spam if necessary, or can continue debugging the issue
> with some guidance.

Thanks for providing it. It turns out that the giant Message-Id value
wasn't causing the problem. Instead the message was corrupt by having a
stray new line at the third line. (So GMime is seeing only the first two
lines of headers). We *used* to have working code to detect this kind of
file as "not an email" but again, this broke when we changed
notmuch_message_get_header to return "" instead of NULL for missing
headers.

See patches below (just pushed now as well) for the fixes.

-Carl

-- next part --
An embedded and charset-unspecified text was scrubbed...
Name: handle-corrupt-mail-and-non-mail.patchset
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091120/5619e634/attachment.txt>

[notmuch] notmuch new: Memory problem

[Carl Worth]

On Fri, 20 Nov 2009 09:56:50 +0100, Dominik Epple  wrote:
> Is there a problem with the number of my mails? I currently have over
> 40.000 Mails... they live currently in mbox files, I created a Maildir
> with mb2md-3.20.pl.

I'm suspecting that you have some big files in there, (such as indexes
from some other mail program). We had code in notmuch to detect and
ignore these, but a recent bug had broken that.

I just fixed this code as of the below commit. So please update and try
again and let us know if things work any better.

Thanks for your patience!

-Carl

commit 3ae12b1e286d1c0041a2e3957cb01daa2981dad9
Author: Carl Worth 
Date:   Fri Nov 20 21:46:37 2009 +0100

add_message: Re-fix handling of non-mail files.

More fallout from _get_header now returning "" for missing headers.

The bug here is that we would no longer detect that a file is not an
email message and give up on it like we should.

And this time, I actually audited all callers to
notmuch_message_get_header, so hopefully we're done fixing this
bug over and over.

[notmuch] Segfault with weird Message-ID

[Mike Hommey]

On Fri, Nov 20, 2009 at 09:53:37PM +0100, Carl Worth wrote:
> On Fri, 20 Nov 2009 14:26:25 +0100, Mike Hommey  
> wrote:
> > - for some reason, xapian doesn't want to add the document corresponding
> >   to this old spam message: notmuch->xapian_db->add_document throws an
> >   exception.
> 
> I think things had just gone wrong long before then.

I *did* see it throwing an exception from there. The sad thing is that I
can't reproduce the problem anymore :-/

> > I can provide the spam if necessary, or can continue debugging the issue
> > with some guidance.
> 
> Thanks for providing it. It turns out that the giant Message-Id value
> wasn't causing the problem. Instead the message was corrupt by having a
> stray new line at the third line. (So GMime is seeing only the first two
> lines of headers). We *used* to have working code to detect this kind of
> file as "not an email" but again, this broke when we changed
> notmuch_message_get_header to return "" instead of NULL for missing
> headers.

Interestingly, when I first traced on what message the crash was
happening, I did see notmuch having the message-id in the message_id
variable.

FWIW, that was using c05c3f1.

I'll see if I can reproduce my segfault again when starting from scratch
again, and will also give a try to your patches.

Cheers,

Mike

[notmuch] [PATCH] Make notmuch-show 'X' (and 'x') commands remove inbox (and unread) tags

[Keith Packard]

When closing a thread view, mark the thread as archived by removing
the "inbox" tag, and for the 'x' variant, the "unread" tag as well,
then kill the buffer and update the search window view as well.

This makes 'x' much the same as 'a', but instead of taking you to the
next message, it takes you back to the search window instead.

Signed-off-by: Keith Packard 
---
 notmuch.el |   86 ++-
 1 files changed, 67 insertions(+), 19 deletions(-)

This adds new functionality without removing any; without this patch,
the 'x' and 'q' commands do the same thing,  killing the buffer
without clearing any tags. With this patch, the 'x' command clears the
inbox and unread tags from the current message and returns to the
search index view, while the existing 'q' command continues to provide
old behaviour.

diff --git a/notmuch.el b/notmuch.el
index cd3780f..544f281 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -31,8 +31,8 @@
 ; Will be much preferable to switch to direct manipulation for
 ; toggling visibility of these components. Probably using
 ; overlays-at to query and manipulate the current overlay.
-(define-key map "a" 'notmuch-show-archive-thread)
-(define-key map "A" 'notmuch-show-mark-read-then-archive-thread)
+(define-key map "a" 'notmuch-show-mark-read-archive-thread-next-thread)
+(define-key map "A" 'notmuch-show-archive-thread-next-thread)
 (define-key map "b" 'notmuch-show-toggle-body-read-visible)
 (define-key map "c" 'notmuch-show-toggle-citations-visible)
 (define-key map "h" 'notmuch-show-toggle-headers-visible)
@@ -47,7 +47,8 @@
 (define-key map "s" 'notmuch-show-toggle-signatures-visible)
 (define-key map "v" 'notmuch-show-view-all-mime-parts)
 (define-key map "w" 'notmuch-show-view-raw-message)
-(define-key map "x" 'kill-this-buffer)
+(define-key map "x" 'notmuch-show-mark-read-archive-thread-kill-buffer)
+(define-key map "X" 'notmuch-show-archive-thread-kill-buffer)
 (define-key map "+" 'notmuch-show-add-tag)
 (define-key map "-" 'notmuch-show-remove-tag)
 (define-key map (kbd "DEL") 'notmuch-show-rewind)
@@ -193,7 +194,33 @@ Unlike builtin `next-line' this version accepts no 
arguments."
 (cons (notmuch-show-get-message-id) nil)))
  (notmuch-show-set-tags (sort (set-difference tags toremove :test 
'string=) 'string<))

-(defun notmuch-show-archive-thread-maybe-mark-read (markread)
+(defun notmuch-show-next-thread (markread)
+  (let ((parent-buffer notmuch-show-parent-buffer))
+(kill-this-buffer)
+(if parent-buffer
+   (progn
+ (switch-to-buffer parent-buffer)
+ (forward-line)
+ (notmuch-search-show-thread)
+  
+(defun notmuch-delete-tags (to-remove from)
+  (if to-remove
+  (delete (car to-remove) (notmuch-delete-tags (cdr to-remove) from))
+from))
+
+(defun notmuch-kill-message-buffer (markread)
+  (let ((parent-buffer notmuch-show-parent-buffer))
+(kill-this-buffer)
+(if parent-buffer
+   (progn
+ (switch-to-buffer parent-buffer)
+ (let ((tags (notmuch-search-get-tags)))
+   (setq tags (delete "inbox" tags))
+   (if markread (setq tags (delete "unread" tags)))
+   (notmuch-search-set-tags tags))
+ (forward-line)
+
+(defun notmuch-show-archive-thread-maybe-mark-read (markread shownext)
   (save-excursion
 (goto-char (point-min))
 (while (not (eobp))
@@ -204,15 +231,9 @@ Unlike builtin `next-line' this version accepts no 
arguments."
  (forward-char))
   (if (not (re-search-forward notmuch-show-message-begin-regexp nil t))
  (goto-char (point-max)
-  (let ((parent-buffer notmuch-show-parent-buffer))
-(kill-this-buffer)
-(if parent-buffer
-   (progn
- (switch-to-buffer parent-buffer)
- (forward-line)
- (notmuch-search-show-thread)
+  (if shownext (notmuch-show-next-thread markread) 
(notmuch-kill-message-buffer markread)))

-(defun notmuch-show-mark-read-then-archive-thread ()
+(defun notmuch-show-mark-read-archive-thread-next-thread ()
   "Remove \"unread\" tag from each message, then archive and show next thread.

 Archive each message currently shown by removing the \"unread\"
@@ -225,9 +246,22 @@ being delivered to the same thread. It does not archive the
 entire thread, but only the messages shown in the current
 buffer."
   (interactive)
-  (notmuch-show-archive-thread-maybe-mark-read t))
+  (notmuch-show-archive-thread-maybe-mark-read t t))
+
+(defun notmuch-show-mark-read-archive-thread-kill-buffer ()
+  "Remove \"unread\" tag from each message, then archive and kill the buffer.
+
+Archive each message currrently shown by removing the \"unread\"
+and \"inbox\" tag from each. Then kill this buffer.

-(defun notmuch-show-archive-thread ()
+Note: This command is safe from any race condition of new messages
+being delivered to the same thread. It does 

[notmuch] [PATCH] Insert signature into replies

[Keith Packard]

When you compose a new message, message mode carefully inserts your
mail signature at the bottom of the message; as notmuch constructs the
reply all by itself, this doesn't happen then. Use the message mode
function 'message-insert-signature' to add that to reply buffers.

Signed-off-by: Keith Packard 
---
 notmuch.el |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index bed6a25..feb84ad 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -308,6 +308,7 @@ buffer."
 (defun notmuch-reply (query-string)
   (switch-to-buffer (generate-new-buffer "notmuch-draft"))
   (call-process notmuch-command nil t nil "reply" query-string)
+  (message-insert-signature)
   (goto-char (point-min))
   (if (re-search-forward "^$" nil t)
   (progn
-- 
1.6.5.2

[notmuch] [PATCH 1/3] Make mouse-1 click in search view show thread

[Keith Packard]

Selecting text in the search view isn't all that useful, so instead,
make mouse-1 clicks actually show the thread you click on. It's almost
like direct manipulation or something.

Signed-off-by: Keith Packard 
---
 notmuch.el |1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index feb84ad..4f369de 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -783,6 +783,7 @@ thread from that buffer can be show when done with this 
one)."
 (define-key map "t" 'notmuch-search-filter-by-tag)
 (define-key map "x" 'kill-this-buffer)
 (define-key map (kbd "RET") 'notmuch-search-show-thread)
+(define-key map [mouse-1] 'notmuch-search-show-thread)
 (define-key map "+" 'notmuch-search-add-tag)
 (define-key map "-" 'notmuch-search-remove-tag)
 (define-key map "<" 'beginning-of-buffer)
-- 
1.6.5.2

[notmuch] [PATCH 2/3] Add 'notmuch count' command to show the count of matching messages

[Keith Packard]

Getting the count of matching threads or messages is a fairly
expensive operation. Xapian provides a very efficient mechanism that
returns an approximate value, so use that for this new command.

This returns the number of matching messages, not threads, as that is
cheap to compute.

Signed-off-by: Keith Packard 
---
 Makefile.local   |1 +
 lib/notmuch.h|8 
 lib/query.cc |   52 ++
 notmuch-client.h |3 +
 notmuch-count.c  |  109 ++
 notmuch.c|   11 +
 6 files changed, 184 insertions(+), 0 deletions(-)
 create mode 100644 notmuch-count.c

diff --git a/Makefile.local b/Makefile.local
index 3c99624..cbd75ce 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -5,6 +5,7 @@ emacs: notmuch.elc
 notmuch_client_srcs =  \
notmuch.c   \
notmuch-config.c\
+   notmuch-count.c \
notmuch-dump.c  \
notmuch-new.c   \
notmuch-reply.c \
diff --git a/lib/notmuch.h b/lib/notmuch.h
index cc713a3..937f3b6 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -459,6 +459,14 @@ notmuch_threads_advance (notmuch_threads_t *threads);
 void
 notmuch_threads_destroy (notmuch_threads_t *threads);

+/* Return an estimate of the number of messages matching a search
+ *
+ * This function performs a search and returns Xapian's best
+ * guess as to number of matching messages.
+ */
+unsigned
+notmuch_query_count_messages (notmuch_query_t *query);
+ 
 /* Get the thread ID of 'thread'.
  *
  * The returned string belongs to 'thread' and as such, should not be
diff --git a/lib/query.cc b/lib/query.cc
index ea521dd..dff9634 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -278,3 +278,55 @@ notmuch_threads_destroy (notmuch_threads_t *threads)
 {
 talloc_free (threads);
 }
+
+unsigned
+notmuch_query_count_messages (notmuch_query_t *query)
+{
+notmuch_database_t *notmuch = query->notmuch;
+const char *query_string = query->query_string;
+Xapian::doccount count;
+
+try {
+   Xapian::Enquire enquire (*notmuch->xapian_db);
+   Xapian::Query mail_query (talloc_asprintf (query, "%s%s",
+  _find_prefix ("type"),
+  "mail"));
+   Xapian::Query string_query, final_query;
+   Xapian::MSet mset;
+   unsigned int flags = (Xapian::QueryParser::FLAG_BOOLEAN |
+ Xapian::QueryParser::FLAG_PHRASE |
+ Xapian::QueryParser::FLAG_LOVEHATE |
+ Xapian::QueryParser::FLAG_BOOLEAN_ANY_CASE |
+ Xapian::QueryParser::FLAG_WILDCARD |
+ Xapian::QueryParser::FLAG_PURE_NOT);
+
+   if (strcmp (query_string, "") == 0) {
+   final_query = mail_query;
+   } else {
+   string_query = notmuch->query_parser->
+   parse_query (query_string, flags);
+   final_query = Xapian::Query (Xapian::Query::OP_AND,
+mail_query, string_query);
+   }
+
+   enquire.set_weighting_scheme(Xapian::BoolWeight());
+   enquire.set_docid_order(Xapian::Enquire::ASCENDING);
+
+#if DEBUG_QUERY
+   fprintf (stderr, "Final query is:\n%s\n", 
final_query.get_description().c_str());
+#endif
+
+   enquire.set_query (final_query);
+
+   mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount ());
+
+   count = mset.get_matches_estimated();
+
+} catch (const Xapian::Error ) {
+   fprintf (stderr, "A Xapian exception occurred: %s\n",
+error.get_msg().c_str());
+   fprintf (stderr, "Query string was: %s\n", query->query_string);
+}
+
+return count;
+}
diff --git a/notmuch-client.h b/notmuch-client.h
index b65aa77..9884497 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -91,6 +91,9 @@ chomp_newline (char *str)
 }

 int
+notmuch_count_command (void *ctx, int argc, char *argv[]);
+
+int
 notmuch_dump_command (void *ctx, int argc, char *argv[]);

 int
diff --git a/notmuch-count.c b/notmuch-count.c
new file mode 100644
index 000..68e428f
--- /dev/null
+++ b/notmuch-count.c
@@ -0,0 +1,109 @@
+/* notmuch - Not much of an email program, (just index and search)
+ *
+ * Copyright ?? 2009 Carl Worth
+ * Copyright ?? 2009 Keith Packard
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU 

[notmuch] [PATCH 3/3] Add notmuch-index mode to display message counts

[Keith Packard]

Index mode takes a (user-configurable) list of search patterns and
produces a list of those patterns and the count of messages that they
match. When an entry in this list is selected, a search window with
the defined search is opened.  The set of indexes is defined as a
list, each element contains the name of the index and the query string
to count.

This provides a view similar to a folder list in a more traditional
mail client.

Signed-off-by: Keith Packard 
---
 notmuch.el |   81 
 1 files changed, 81 insertions(+), 0 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index 4f369de..454320e 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -1046,4 +1046,85 @@ current search results AND that are tagged with the 
given tag."

 (setq mail-user-agent 'message-user-agent)

+(defvar notmuch-index-mode-map
+  (let ((map (make-sparse-keymap)))
+(define-key map "n" 'next-line)
+(define-key map "p" 'previous-line)
+(define-key map "x" 'kill-this-buffer)
+(define-key map "q" 'kill-this-buffer)
+(define-key map "s" 'notmuch-search)
+(define-key map (kbd "RET") 'notmuch-index-show-search)
+(define-key map "<" 'beginning-of-buffer)
+(define-key map "=" 'notmuch-index)
+(define-key map "?" 'describe-mode)
+(define-key map [mouse-1] 'notmuch-index-show-search)
+map)
+  "Keymap for \"notmuch index\" buffers.")
+
+(fset 'notmuch-index-mode-map notmuch-index-mode-map)
+
+(defcustom notmuch-indexes (quote (("inbox" . "tag:inbox") ("unread" . 
"tag:unread")))
+  "List of searches for the notmuch index view"
+  :type '(alist :key-type (string) :value-type (string))
+  :group 'notmuch)
+  
+(defun notmuch-index-mode ()
+  "Major mode for showing index of notmuch tags.
+
+This buffer contains a list of messages counts returned by a
+customizable set of searches of your email archives. Each line
+in the buffer shows the search terms and the resulting message count.
+
+Pressing RET on any line opens a search window containing the search
+results for the search terms in that line.
+
+\\{notmuch-index-mode-map}"
+  (interactive)
+  (kill-all-local-variables)
+  (use-local-map 'notmuch-index-mode-map)
+  (setq truncate-lines t)
+  (hl-line-mode 1)
+  (setq major-mode 'notmuch-index-mode
+   mode-name "notmuch-index")
+  (setq buffer-read-only t))
+
+(defun notmuch-index-add (indexes)
+  (if indexes
+  (let ((name (car (car indexes)))
+   (inhibit-read-only t)
+   (search (cdr (car indexes
+   (insert name)
+   (indent-to 16 1)
+   (call-process notmuch-command nil t nil "count" search)
+   (notmuch-index-add (cdr indexes)
+
+(defun notmuch-index-find-name ()
+  (save-excursion
+(beginning-of-line)
+(let ((beg (point)))
+  (forward-word)
+  (filter-buffer-substring beg (point)
+
+(defun notmuch-index-show-search ( index)
+  "Show a search window for the search related to the specified index."
+  (interactive)
+  (if (null index)
+  (setq index (notmuch-index-find-name)))
+  (let ((search (assoc index notmuch-indexes)))
+(if search
+   (notmuch-search (cdr search) t
+
+(defun notmuch-index ()
+  "Show the message index and update the displayed counts."
+  (interactive)
+  (let ((buffer (get-buffer-create "*notmuch-index*")))
+(switch-to-buffer buffer)
+(let ((inhibit-read-only t)
+ (n (line-number-at-pos)))
+  (erase-buffer)
+  (notmuch-index-mode)
+  (notmuch-index-add notmuch-indexes)
+  (goto-char (point-min))
+  (goto-line n
+
 (provide 'notmuch)
-- 
1.6.5.2

[notmuch] notmuch 'index' mode.

[Keith Packard]

I posted a patch adding an 'index' mode to notmuch and though I'd
explain my idea. Most mail systems provide a 'folder view' mode which
displays the set of folders and a count of messages in each folder. I
used this myself as the first sort of which messages I want to read.

Notmuch doesn't have folders, but it does have tags, so I thought I'd
take advantage of that, along with the quick searching ability in Xapian
to construct something much like the folder view.

In 'index' mode, notmuch shows a list of 'folders', each of which is
defined by a Xapian query. The new 'notmuch count' command is used to
figure out how many messages matching the query are in the
database. When you hit 'enter' (or click the mouse) on any line, it
executes the search for real using the notmuch-search command.

Here's the set of indexes I'm using right now (oddly, it looks a lot
like the set of tags I auto-configure in my notmuch-poll script).

(setq notmuch-indexes '(("inbox" . "tag:inbox")
("me" . "tag:inbox AND tag:me")
("announce" . "tag:inbox AND tag:announce")
("bugs" . "tag:inbox AND tag:bugs")
("cairo" . "tag:inbox AND tag:cairo")
("debian" . "tag:inbox AND tag:debian");
("hackers" . "tag:inbox AND tag:hackers")
("intel" . "tag:inbox AND tag:intel")
("mesa" . "tag:inbox AND tag:mesa")
("notmuch" . "tag:inbox AND tag:notmuch")
("oes" . "tag:inbox AND tag:oes")
("rockets" . "tag:inbox AND tag:rockets")
("x" . "tag:inbox AND tag:x")
("xorg-board" . "tag:inbox AND tag:xorg-board")
("xpatches" . "tag:inbox AND tag:x AND tag:patch")))

And here's what I see in the *notmuch-index* buffer:

inbox   3
me  3
announce0
bugs0
cairo   0
debian  0
hackers 0
intel   0
mesa0
notmuch 0
oes 0
rockets 0
x   0
xorg-board  0
xpatches0

This is all pretty simple, and it does much of what I want. However, one
could always get more :-)

One missing piece here is that the searches don't have a
total/unread count. Because the 'notmuch count' command is so fast
(notmuch count the returns 532891 on my machine, and takes 0.240
seconds), we really could count the number of messages matching a single
tag and then *also* count the number of messages matching that tag and
the inbox tag.

Another missing piece is that these should be shared with the
notmuch-poll script, probably via the .notmuch-config file. I do like
that I can have some entries here that aren't in my notmuch-poll script.

It might be nice to make this hierarchical as I used to do with
evolution's vfolder stuff -- an 'intel' folder with an 'intel/unread'
sub-folder. That would be done with simple conjunctions of tags, but
providing some structure on the screen might be nice.

I'm also wondering if we should elide, or at least change the appearance,
of lines which match zero messages.

I'd also like to be able to get the system to display my mail in three
panes -- one with the index, one with the current search and the last
with the current thread. Getting those laid out on the screen correctly
and letting you get them back to that layout easily would make notmuch
look a lot more like other email clients, a form I've found fairly
functional as I do not generally read mail in a linear fashion.

-- 
keith.packard at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091120/7c14d629/attachment.pgp>