Better Gmail handling by not using Notmuch tags

[Mark Anderson]

information about the relative
times of the actions.  If instead of trying to rationalize the two
"states" of my messages, I was trying to synchronize the changes, then I
just need to go down the list of changes and take the appropriate action.

You might have other actions, such as a true delete in Notmuch, which
should remove all copies of the email before doing a mail sync.

The benefit of using the mail sync is it uses a widely distributed mail
synchronization model, but it really tags expensive to synchronize.  It
gets better if you use the Gmail imap extensions that can list the tags
without your client requesting a copy of the entire email for each tag
the mail has.  However, Even when you have that, you don't have
bulletproof mail, because the actions need to be guaranteed to complete
before synchronization and after synchronization, and any user changes
need to be held off, as they _will_ be interpreted incorrectly if they
take place during the pre-sync, sync, post-sync window.

You can simplify this if you make guarantees in your usage model.  That
you will never do tagging operations during a pre-, sync, post- cycle,
or that you only do synchronization one way or the other, instead of
full bidirectional sync.

It's a difficult problem, I look forward to seeing other solutions
proposed.

Thanks,
-Mark Anderson

> upload local -> gmail
> 1) upload "All Mail folder
> 2) assign on gmail the labels corresponding to the notmuch tags.
> 
> The step 1 could be done by any sync tool available for this (offlineimap, 
> ...)
> 
> step 2 needs to be developed - no idea how, but it surely would be really 
> usefull, because then
> notmuch would even become a perfect tool for gmail backup as well.
> 
> Cheers,
> 
> Rainer
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
> 
> iEYEARECAAYFAlBS4akACgkQoYgNqgF2egoGhwCaAgfXQUAK4RK1v22JOhgYXfR1
> +C8AnRU892SrxK7IYN9xoxhM865Y+vTA
> =ma75
> -END PGP SIGNATURE-
> 
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
> 

Re: Better Gmail handling by not using Notmuch tags

[Mark Anderson]

 down the list of changes and take the appropriate action.

You might have other actions, such as a true delete in Notmuch, which
should remove all copies of the email before doing a mail sync.

The benefit of using the mail sync is it uses a widely distributed mail
synchronization model, but it really tags expensive to synchronize.  It
gets better if you use the Gmail imap extensions that can list the tags
without your client requesting a copy of the entire email for each tag
the mail has.  However, Even when you have that, you don't have
bulletproof mail, because the actions need to be guaranteed to complete
before synchronization and after synchronization, and any user changes
need to be held off, as they _will_ be interpreted incorrectly if they
take place during the pre-sync, sync, post-sync window.

You can simplify this if you make guarantees in your usage model.  That
you will never do tagging operations during a pre-, sync, post- cycle,
or that you only do synchronization one way or the other, instead of
full bidirectional sync.

It's a difficult problem, I look forward to seeing other solutions
proposed.

Thanks,
-Mark Anderson

 upload local - gmail
 1) upload All Mail folder
 2) assign on gmail the labels corresponding to the notmuch tags.
 
 The step 1 could be done by any sync tool available for this (offlineimap, 
 ...)
 
 step 2 needs to be developed - no idea how, but it surely would be really 
 usefull, because then
 notmuch would even become a perfect tool for gmail backup as well.
 
 Cheers,
 
 Rainer
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.11 (GNU/Linux)
 Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
 
 iEYEARECAAYFAlBS4akACgkQoYgNqgF2egoGhwCaAgfXQUAK4RK1v22JOhgYXfR1
 +C8AnRU892SrxK7IYN9xoxhM865Y+vTA
 =ma75
 -END PGP SIGNATURE-
 
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
 

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Message deletion wisdom

[Mark Anderson]

On Thu, 5 Apr 2012 12:20:43 -0400, Antoine Beaupr?  
wrote:
> Finally, I want to voice that I feel a "delete" key, even if it doesn't
> delete mails, seems like an important part of a mail user
> agent. Archiving mail is one thing, but for the love and respect of
> sysadmins and the infrastructure they maintain, please consider adding
> at least a way to *tag* those deleted emails.
> 
> Having the above keys being defined as standard in notmuch don't seem
> like much to ask.
> 
> This may be a dissenting view here, but your mail is not that
> important. :P

Hear! Hear!

I also would like to have some natural way to mark things as "I never
expect to look at this again".  Do I really need to keep track of every
vacation and calendar notice?  Or every logfile I have my infrastructure
email to me, that doesn't really need to be in email, but was easily
shoehorned into the existing notification/logging side effects of mail?
My infrastructure is building new models for unit-level testing daily, I
don't need to keep track of every model for years, and I have been using
notmuch for years. Hmm, December 2009, it has been a while. :)

I do set up a deleted tag for my own use, but it would be nice if that
were viewed as a little more natural use case by the software from the
./configure script.  Yes Virginia, notmuch comes with NotMuch of a box.

Thanks,
-Mark

Re: Message deletion wisdom

[Mark Anderson]

On Thu, 5 Apr 2012 12:20:43 -0400, Antoine Beaupré anar...@anarcat.ath.cx 
wrote:
 Finally, I want to voice that I feel a delete key, even if it doesn't
 delete mails, seems like an important part of a mail user
 agent. Archiving mail is one thing, but for the love and respect of
 sysadmins and the infrastructure they maintain, please consider adding
 at least a way to *tag* those deleted emails.
 
 Having the above keys being defined as standard in notmuch don't seem
 like much to ask.
 
 This may be a dissenting view here, but your mail is not that
 important. :P

Hear! Hear!

I also would like to have some natural way to mark things as I never
expect to look at this again.  Do I really need to keep track of every
vacation and calendar notice?  Or every logfile I have my infrastructure
email to me, that doesn't really need to be in email, but was easily
shoehorned into the existing notification/logging side effects of mail?
My infrastructure is building new models for unit-level testing daily, I
don't need to keep track of every model for years, and I have been using
notmuch for years. Hmm, December 2009, it has been a while. :)

I do set up a deleted tag for my own use, but it would be nice if that
were viewed as a little more natural use case by the software from the
./configure script.  Yes Virginia, notmuch comes with NotMuch of a box.

Thanks,
-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Handling of symlinked maildirs?

[Mark Anderson]

On Mon, 2 Apr 2012 10:54:48 -0700, Jameson Graef Rollins  wrote:
> On Mon, Apr 02 2012, Andrei POPESCU  wrote:
> > Im sorting my mailing lists with generic maildrop rules like this one:
> >
> > if (/^List-Id:.*/)
> > to Maildir/.debian.$MATCH1/
> >
> > However, I'm subscribed to a *lot* of mailing lists and in order to keep 
> > my folder view sane I use symlinks to conflate some of them, e.g. 
> >
> > .debian.devel-announce -> .debian.devel
> >
> > This works well since mutt simply ignores the symlink(s) so I don't even 
> > need to exclude them in the config, but it seems that notmuch does index 
> > each of the symlinks as a separate folder[1].
> >
> > Does it make sense to have this configurable or even completely exclude 
> > the symlinks?
> 
> Hey, Andrei.  I would say that if you have a compelling reason that
> notmuch should be able to ignore symlinked directories then it maybe
> makes sense to have it configurable.  However, I must say that I have
> completely dropped the whole folder concept myself since I find it
> totally redundant with notmuch's tagging and searching capabilities.

Please don't exclude the indexing of symlinks in notmuch.  I have a
fairly restricted IT environment at work, where I have chosen to symlink
all my maildirs to other disks so that they appear in my home directory
but do not fill my home disk quota every day.  Often my different
maildirs are to make my non-notmuch mail viewing slightly tolerable.

There was a configuration option (new.ignore) added to 0.12 that lets
one specify names of things not to index. That may be enough to do what
you want, although not as automatic as just pitching all symlinks while
indexing.

Thanks,

-Mark

Re: Handling of symlinked maildirs?

[Mark Anderson]

On Mon, 2 Apr 2012 10:54:48 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 On Mon, Apr 02 2012, Andrei POPESCU andreimpope...@gmail.com wrote:
  Im sorting my mailing lists with generic maildrop rules like this one:
 
  if (/^List-Id:.*debian-(.*)\.lists.debian.org/)
  to Maildir/.debian.$MATCH1/
 
  However, I'm subscribed to a *lot* of mailing lists and in order to keep 
  my folder view sane I use symlinks to conflate some of them, e.g. 
 
  .debian.devel-announce - .debian.devel
 
  This works well since mutt simply ignores the symlink(s) so I don't even 
  need to exclude them in the config, but it seems that notmuch does index 
  each of the symlinks as a separate folder[1].
 
  Does it make sense to have this configurable or even completely exclude 
  the symlinks?
 
 Hey, Andrei.  I would say that if you have a compelling reason that
 notmuch should be able to ignore symlinked directories then it maybe
 makes sense to have it configurable.  However, I must say that I have
 completely dropped the whole folder concept myself since I find it
 totally redundant with notmuch's tagging and searching capabilities.

Please don't exclude the indexing of symlinks in notmuch.  I have a
fairly restricted IT environment at work, where I have chosen to symlink
all my maildirs to other disks so that they appear in my home directory
but do not fill my home disk quota every day.  Often my different
maildirs are to make my non-notmuch mail viewing slightly tolerable.

There was a configuration option (new.ignore) added to 0.12 that lets
one specify names of things not to index. That may be enough to do what
you want, although not as automatic as just pitching all symlinks while
indexing.

Thanks,

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Goto command for existing search windows

[Mark Anderson]

On Tue, 27 Mar 2012 14:24:36 -0600, Mark Anderson  
wrote:
> I was looking for a function which would find a buffer based on one of
> my saved searches, and perform the search if it didn't exist.
> 
> I've gotten it a bit closer, if I perform the search that matches a
> saved search, then this routine will find it because of the magic in
> notmuch-search-buffer-title, but perhaps someone else feels up to
> searching through the saved searches directly?
> 
> 
> 
> (defun notmuch-goto-or-search ( query)
>   "Find a notmuch-search buffer with the given query, or run 
> \"notmuch search\" with the given `query' and display results.
> 
> If `query' is nil, it is read interactively from the minibuffer."
>   (interactive)
>   (if (null query)
>   (setq query (notmuch-read-query "Notmuch goto-or-search: ")))
>   (let ((buffer-name (notmuch-search-buffer-title query)))
> (setq buf (get-buffer buffer-name)))
>   
> (if (not buf)
> (notmuch-search query)
>   (switch-to-buffer buf)
>   )))

I have a slightly better-for-me version now:

(defun notmuch-goto-or-search ( query)
  "Find a notmuch saved-search query with the given name, or a
search with the given query, switching to an existing buffer
without changes in preference to automatically refreshing or
creating the search buffer.

If `query' is nil, it is read interactively from the minibuffer."
  (interactive)
  (if (null query)
  (setq query (notmuch-read-query "Notmuch goto-or-search: ")))
  (let ((saved-search-tuple (assoc query notmuch-saved-searches)))
(setq expanded-query
  (if (null saved-search-tuple)
  query
(cdr saved-search-tuple
  (let ((buffer-name (notmuch-search-buffer-title expanded-query)))
(setq buf (get-buffer buffer-name)))
(if (not buf)
(notmuch-search expanded-query)
  (switch-to-buffer buf)
  ))

This does search the saved searches to see if you entered a saved search
name.  With this I don't have to duplicate my query for saved searches
in key bindings.

(global-set-key [f8] (lambda () (interactive) (notmuch-goto-or-search "Inbox")))
(global-set-key [f9] (lambda () (interactive) (notmuch-goto-or-search "INBOX")))
(global-set-key [f10] (lambda () (interactive) (notmuch-goto-or-search "todo")))


> I then use it something like this:
> 
> (global-set-key [C-f1] (lambda () (interactive) (notmuch-goto-or-search 
> "tag:inbox and tag:unread and not tag:deleted")))
> (global-set-key [C-f2] (lambda () (interactive) (notmuch-goto-or-search 
> "tag:inbox and not tag:deleted")))
> (global-set-key [C-f3] 'notmuch)
> (global-set-key [C-f6] (lambda () (interactive) (notmuch-goto-or-search 
> "tag:todo and not tag:deleted")))
> 
> It would be better if I could use my Inbox, INBOX and todo names for the
> saved searches, but how to do that without breaking generality of
> searching the body of the email?  Do I have to define my own ss: (saved
> search) prefix or something, as I believe some others have?
> 
> This is what I'm willing to do today, and it works for me, I could patch
> notmuch.el, but I wondered about answering the other questions.
> 
> Also, some elisp master could hint about how to make the binding not so
> ugly. ;)
> 
> Another appreciated elisp hint would be how to get the buf variable to
> go inside the let, I keep getting complaints about buffer-name not being
> defined, thus the "ugly" setq, which works.
> 
> Enjoy,
> 
> -Mark

Re: Goto command for existing search windows

[Mark Anderson]

On Tue, 27 Mar 2012 14:24:36 -0600, Mark Anderson markr.ander...@amd.com 
wrote:
 I was looking for a function which would find a buffer based on one of
 my saved searches, and perform the search if it didn't exist.
 
 I've gotten it a bit closer, if I perform the search that matches a
 saved search, then this routine will find it because of the magic in
 notmuch-search-buffer-title, but perhaps someone else feels up to
 searching through the saved searches directly?
 
 
 
 (defun notmuch-goto-or-search (optional query)
   Find a notmuch-search buffer with the given query, or run 
 \notmuch search\ with the given `query' and display results.
 
 If `query' is nil, it is read interactively from the minibuffer.
   (interactive)
   (if (null query)
   (setq query (notmuch-read-query Notmuch goto-or-search: )))
   (let ((buffer-name (notmuch-search-buffer-title query)))
 (setq buf (get-buffer buffer-name)))
   
 (if (not buf)
 (notmuch-search query)
   (switch-to-buffer buf)
   )))

I have a slightly better-for-me version now:

(defun notmuch-goto-or-search (optional query)
  Find a notmuch saved-search query with the given name, or a
search with the given query, switching to an existing buffer
without changes in preference to automatically refreshing or
creating the search buffer.

If `query' is nil, it is read interactively from the minibuffer.
  (interactive)
  (if (null query)
  (setq query (notmuch-read-query Notmuch goto-or-search: )))
  (let ((saved-search-tuple (assoc query notmuch-saved-searches)))
(setq expanded-query
  (if (null saved-search-tuple)
  query
(cdr saved-search-tuple
  (let ((buffer-name (notmuch-search-buffer-title expanded-query)))
(setq buf (get-buffer buffer-name)))
(if (not buf)
(notmuch-search expanded-query)
  (switch-to-buffer buf)
  ))

This does search the saved searches to see if you entered a saved search
name.  With this I don't have to duplicate my query for saved searches
in key bindings.

(global-set-key [f8] (lambda () (interactive) (notmuch-goto-or-search Inbox)))
(global-set-key [f9] (lambda () (interactive) (notmuch-goto-or-search INBOX)))
(global-set-key [f10] (lambda () (interactive) (notmuch-goto-or-search todo)))


 I then use it something like this:
 
 (global-set-key [C-f1] (lambda () (interactive) (notmuch-goto-or-search 
 tag:inbox and tag:unread and not tag:deleted)))
 (global-set-key [C-f2] (lambda () (interactive) (notmuch-goto-or-search 
 tag:inbox and not tag:deleted)))
 (global-set-key [C-f3] 'notmuch)
 (global-set-key [C-f6] (lambda () (interactive) (notmuch-goto-or-search 
 tag:todo and not tag:deleted)))
 
 It would be better if I could use my Inbox, INBOX and todo names for the
 saved searches, but how to do that without breaking generality of
 searching the body of the email?  Do I have to define my own ss: (saved
 search) prefix or something, as I believe some others have?
 
 This is what I'm willing to do today, and it works for me, I could patch
 notmuch.el, but I wondered about answering the other questions.
 
 Also, some elisp master could hint about how to make the binding not so
 ugly. ;)
 
 Another appreciated elisp hint would be how to get the buf variable to
 go inside the let, I keep getting complaints about buffer-name not being
 defined, thus the ugly setq, which works.
 
 Enjoy,
 
 -Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Goto command for existing search windows

[Mark Anderson]

I was looking for a function which would find a buffer based on one of
my saved searches, and perform the search if it didn't exist.

I've gotten it a bit closer, if I perform the search that matches a
saved search, then this routine will find it because of the magic in
notmuch-search-buffer-title, but perhaps someone else feels up to
searching through the saved searches directly?



(defun notmuch-goto-or-search ( query)
  "Find a notmuch-search buffer with the given query, or run 
\"notmuch search\" with the given `query' and display results.

If `query' is nil, it is read interactively from the minibuffer."
  (interactive)
  (if (null query)
  (setq query (notmuch-read-query "Notmuch goto-or-search: ")))
  (let ((buffer-name (notmuch-search-buffer-title query)))
(setq buf (get-buffer buffer-name)))

(if (not buf)
(notmuch-search query)
  (switch-to-buffer buf)
  )))

I then use it something like this:

(global-set-key [C-f1] (lambda () (interactive) (notmuch-goto-or-search 
"tag:inbox and tag:unread and not tag:deleted")))
(global-set-key [C-f2] (lambda () (interactive) (notmuch-goto-or-search 
"tag:inbox and not tag:deleted")))
(global-set-key [C-f3] 'notmuch)
(global-set-key [C-f6] (lambda () (interactive) (notmuch-goto-or-search 
"tag:todo and not tag:deleted")))

It would be better if I could use my Inbox, INBOX and todo names for the
saved searches, but how to do that without breaking generality of
searching the body of the email?  Do I have to define my own ss: (saved
search) prefix or something, as I believe some others have?

This is what I'm willing to do today, and it works for me, I could patch
notmuch.el, but I wondered about answering the other questions.

Also, some elisp master could hint about how to make the binding not so
ugly. ;)

Another appreciated elisp hint would be how to get the buf variable to
go inside the let, I keep getting complaints about buffer-name not being
defined, thus the "ugly" setq, which works.

Enjoy,

-Mark

Goto command for existing search windows

[Mark Anderson]

I was looking for a function which would find a buffer based on one of
my saved searches, and perform the search if it didn't exist.

I've gotten it a bit closer, if I perform the search that matches a
saved search, then this routine will find it because of the magic in
notmuch-search-buffer-title, but perhaps someone else feels up to
searching through the saved searches directly?



(defun notmuch-goto-or-search (optional query)
  Find a notmuch-search buffer with the given query, or run 
\notmuch search\ with the given `query' and display results.

If `query' is nil, it is read interactively from the minibuffer.
  (interactive)
  (if (null query)
  (setq query (notmuch-read-query Notmuch goto-or-search: )))
  (let ((buffer-name (notmuch-search-buffer-title query)))
(setq buf (get-buffer buffer-name)))
  
(if (not buf)
(notmuch-search query)
  (switch-to-buffer buf)
  )))

I then use it something like this:

(global-set-key [C-f1] (lambda () (interactive) (notmuch-goto-or-search 
tag:inbox and tag:unread and not tag:deleted)))
(global-set-key [C-f2] (lambda () (interactive) (notmuch-goto-or-search 
tag:inbox and not tag:deleted)))
(global-set-key [C-f3] 'notmuch)
(global-set-key [C-f6] (lambda () (interactive) (notmuch-goto-or-search 
tag:todo and not tag:deleted)))

It would be better if I could use my Inbox, INBOX and todo names for the
saved searches, but how to do that without breaking generality of
searching the body of the email?  Do I have to define my own ss: (saved
search) prefix or something, as I believe some others have?

This is what I'm willing to do today, and it works for me, I could patch
notmuch.el, but I wondered about answering the other questions.

Also, some elisp master could hint about how to make the binding not so
ugly. ;)

Another appreciated elisp hint would be how to get the buf variable to
go inside the let, I keep getting complaints about buffer-name not being
defined, thus the ugly setq, which works.

Enjoy,

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] v2 emacs: colorize buttonized 'id:' links depending on the target message's state

[Mark Anderson]

On Wed, 18 Jan 2012 04:08:54 -0600, Pieter Praet  wrote:
> On Mon, 16 Jan 2012 16:43:06 -0500, Aaron Ecay  wrote:
> > On Mon, 16 Jan 2012 17:57:33 +0100, Pieter Praet  
> > wrote:
> > [...] Maybe you could change the regex that
> > matches id:?s to require a little more structure ? an at-sign, perhaps.
> > Or even requiring more than (say) 5 non-space characters after the
> > message id would cut down sharply on the false positive rate.
> > 
> 
> Not sure how that would pan out.  It's fairly common behaviour to put
> one or more spaces after a inline Message-Id, so I don't think such a
> limitation would be warmly recepted.

I thought this was a suggestion to have more than 5 non-space characters
after the id:, not the full id:LongIdThingHere 

It sounded to me like an attempt to prevent extra false positives and
the confusing buttonizing and notmuch queries that go with them.

-Mark

Improving notmuch query documentation [was: Re: Partial words on notmuch search?]

[Mark Anderson]

On Tue, 17 Jan 2012 16:29:23 -0600, Austin Clements  wrote:
> Quoth Andrei Popescu on Jan 18 at 12:14 am:
> > On Lu, 16 ian 12, 21:34:31, Austin Clements wrote:
> > > Quoth Andrei Popescu on Jan 16 at 10:21 pm:
> > > > Where can I read more about this? (except the source :)
> > > 
> > > Most of this is in the Xapian query syntax document you found.  Really
> > > we ought to beef-up Notmuch's query syntax documentation.
> > 
> > If I get around to write something myself where do you suggest I should 
> > start, the wiki or the manpage?
> 
> Probably expanding man/man7/notmuch-search-terms.7 would be the way to
> go.

I would appreciate it if the limitations of id: search were explained
there too.  I have some rules that I would love to make based on pattern
matching the message-id of the message, because I have a tool that
generates scads of email and I want to be able to delete a lot of it.

I think that id: is only matchable as an entire string, and a
confirmation of that would be nice to see.

For those who cringe when hearing the mention of _deletion_ of emails,
do you have a suggestion for how many copies of bugs in the bug database
I should store in my mail repository?  Note that IT only gives me a
couple gigabytes of home directory storage, and I don't have an SSD
Linux laptop, so the index does eventually slow down.

In other words, there are plenty of emails I love to forget ever having
received. :)


-Mark

Re: Improving notmuch query documentation [was: Re: Partial words on notmuch search?]

[Mark Anderson]

On Tue, 17 Jan 2012 16:29:23 -0600, Austin Clements amdra...@mit.edu wrote:
 Quoth Andrei Popescu on Jan 18 at 12:14 am:
  On Lu, 16 ian 12, 21:34:31, Austin Clements wrote:
   Quoth Andrei Popescu on Jan 16 at 10:21 pm:
Where can I read more about this? (except the source :)
   
   Most of this is in the Xapian query syntax document you found.  Really
   we ought to beef-up Notmuch's query syntax documentation.
  
  If I get around to write something myself where do you suggest I should 
  start, the wiki or the manpage?
 
 Probably expanding man/man7/notmuch-search-terms.7 would be the way to
 go.

I would appreciate it if the limitations of id: search were explained
there too.  I have some rules that I would love to make based on pattern
matching the message-id of the message, because I have a tool that
generates scads of email and I want to be able to delete a lot of it.

I think that id: is only matchable as an entire string, and a
confirmation of that would be nice to see.

For those who cringe when hearing the mention of _deletion_ of emails,
do you have a suggestion for how many copies of bugs in the bug database
I should store in my mail repository?  Note that IT only gives me a
couple gigabytes of home directory storage, and I don't have an SSD
Linux laptop, so the index does eventually slow down.

In other words, there are plenty of emails I love to forget ever having
received. :)


-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/5] lib: Add a MTIME value to every mail document

[Mark Anderson]

On Tue, 13 Dec 2011 11:11:42 -0600, Thomas Jost  
wrote:
> This is a time_t value, similar to the message date (TIMESTAMP). It is first 
> set
> when the message is added to the database, and is then updated every time a 
> tag
> is added or removed. It can thus be used for doing incremental dumps of the
> database or for synchronizing it between several computers.
> 
> This value can be read freely (with notmuch_message_get_mtime()) but for now 
> it
> can't be set to an arbitrary value: it can only be set to "now" when updated.
> There's no specific reason for this except that I don't really see a real use
> case for setting it to an arbitrary value.

I think it would be easier to write some testcases if the last modified
time could be touched directly.  Perhaps they aren't in the set of "must
have", but it's what comes to mind.

-Mark

> ---
>  lib/database.cc   |7 ++-
>  lib/message.cc|   32 
>  lib/notmuch-private.h |6 +-
>  lib/notmuch.h |4 
>  4 files changed, 47 insertions(+), 2 deletions(-)
> 
> diff --git a/lib/database.cc b/lib/database.cc
> index 2025189..6dc6f73 100644
> --- a/lib/database.cc
> +++ b/lib/database.cc
> @@ -81,7 +81,7 @@ typedef struct {
>   *   STRING is the name of a file within that
>   *   directory for this mail message.
>   *
> - *A mail document also has four values:
> + *A mail document also has five values:
>   *
>   *   TIMESTAMP:  The time_t value corresponding to the message's
>   *   Date header.
> @@ -92,6 +92,9 @@ typedef struct {
>   *
>   *   SUBJECT:The value of the "Subject" header
>   *
> + *   MTIME:  The time_t value corresponding to the last time
> + *   a tag was added or removed on the message.
> + *
>   * In addition, terms from the content of the message are added with
>   * "from", "to", "attachment", and "subject" prefixes for use by the
>   * user in searching. Similarly, terms from the path of the mail
> @@ -1735,6 +1738,8 @@ notmuch_database_add_message (notmuch_database_t 
> *notmuch,
>   date = notmuch_message_file_get_header (message_file, "date");
>   _notmuch_message_set_header_values (message, date, from, subject);
>  
> +_notmuch_message_update_mtime (message);
> +
>   _notmuch_message_index_file (message, filename);
>   } else {
>   ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
> diff --git a/lib/message.cc b/lib/message.cc
> index 0075425..0c98589 100644
> --- a/lib/message.cc
> +++ b/lib/message.cc
> @@ -830,6 +830,34 @@ _notmuch_message_set_header_values (notmuch_message_t 
> *message,
>  message->doc.add_value (NOTMUCH_VALUE_SUBJECT, subject);
>  }
>  
> +/* Get the message mtime, i.e. when it was added or the last time a tag was
> + * added/removed. */
> +time_t
> +notmuch_message_get_mtime (notmuch_message_t *message)
> +{
> +std::string value;
> +
> +try {
> + value = message->doc.get_value (NOTMUCH_VALUE_MTIME);
> +} catch (Xapian::Error ) {
> + INTERNAL_ERROR ("Failed to read mtime value from document.");
> + return 0;
> +}
> +
> +return Xapian::sortable_unserialise (value);
> +}
> +
> +/* Set the message mtime to "now". */
> +void
> +_notmuch_message_update_mtime (notmuch_message_t *message)
> +{
> +time_t time_value;
> +
> +time_value = time (NULL);
> +message->doc.add_value (NOTMUCH_VALUE_MTIME,
> +Xapian::sortable_serialise (time_value));
> +}
> +
>  /* Synchronize changes made to message->doc out into the database. */
>  void
>  _notmuch_message_sync (notmuch_message_t *message)
> @@ -994,6 +1022,8 @@ notmuch_message_add_tag (notmuch_message_t *message, 
> const char *tag)
>   private_status);
>  }
>  
> +_notmuch_message_update_mtime (message);
> +
>  if (! message->frozen)
>   _notmuch_message_sync (message);
>  
> @@ -1022,6 +1052,8 @@ notmuch_message_remove_tag (notmuch_message_t *message, 
> const char *tag)
>   private_status);
>  }
>  
> +_notmuch_message_update_mtime (message);
> +
>  if (! message->frozen)
>   _notmuch_message_sync (message);
>  
> diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
> index 60a932f..9859872 100644
> --- a/lib/notmuch-private.h
> +++ b/lib/notmuch-private.h
> @@ -95,7 +95,8 @@ typedef enum {
>  NOTMUCH_VALUE_TIMESTAMP = 0,
>  NOTMUCH_VALUE_MESSAGE_ID,
>  NOTMUCH_VALUE_FROM,
> -NOTMUCH_VALUE_SUBJECT
> +NOTMUCH_VALUE_SUBJECT,
> +NOTMUCH_VALUE_MTIME
>  } notmuch_value_t;
>  
>  /* Xapian (with flint backend) complains if we provide a term longer
> @@ -276,6 +277,9 @@ _notmuch_message_set_header_values (notmuch_message_t 
> *message,
>   const char *from,
>   const char *subject);
>  void
> +_notmuch_message_update_mtime (notmuch_message_t 

Re: [PATCH 2/5] lib: Add a MTIME value to every mail document

[Mark Anderson]

On Tue, 13 Dec 2011 11:11:42 -0600, Thomas Jost schno...@schnouki.net wrote:
 This is a time_t value, similar to the message date (TIMESTAMP). It is first 
 set
 when the message is added to the database, and is then updated every time a 
 tag
 is added or removed. It can thus be used for doing incremental dumps of the
 database or for synchronizing it between several computers.
 
 This value can be read freely (with notmuch_message_get_mtime()) but for now 
 it
 can't be set to an arbitrary value: it can only be set to now when updated.
 There's no specific reason for this except that I don't really see a real use
 case for setting it to an arbitrary value.

I think it would be easier to write some testcases if the last modified
time could be touched directly.  Perhaps they aren't in the set of must
have, but it's what comes to mind.

-Mark

 ---
  lib/database.cc   |7 ++-
  lib/message.cc|   32 
  lib/notmuch-private.h |6 +-
  lib/notmuch.h |4 
  4 files changed, 47 insertions(+), 2 deletions(-)
 
 diff --git a/lib/database.cc b/lib/database.cc
 index 2025189..6dc6f73 100644
 --- a/lib/database.cc
 +++ b/lib/database.cc
 @@ -81,7 +81,7 @@ typedef struct {
   *   STRING is the name of a file within that
   *   directory for this mail message.
   *
 - *A mail document also has four values:
 + *A mail document also has five values:
   *
   *   TIMESTAMP:  The time_t value corresponding to the message's
   *   Date header.
 @@ -92,6 +92,9 @@ typedef struct {
   *
   *   SUBJECT:The value of the Subject header
   *
 + *   MTIME:  The time_t value corresponding to the last time
 + *   a tag was added or removed on the message.
 + *
   * In addition, terms from the content of the message are added with
   * from, to, attachment, and subject prefixes for use by the
   * user in searching. Similarly, terms from the path of the mail
 @@ -1735,6 +1738,8 @@ notmuch_database_add_message (notmuch_database_t 
 *notmuch,
   date = notmuch_message_file_get_header (message_file, date);
   _notmuch_message_set_header_values (message, date, from, subject);
  
 +_notmuch_message_update_mtime (message);
 +
   _notmuch_message_index_file (message, filename);
   } else {
   ret = NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID;
 diff --git a/lib/message.cc b/lib/message.cc
 index 0075425..0c98589 100644
 --- a/lib/message.cc
 +++ b/lib/message.cc
 @@ -830,6 +830,34 @@ _notmuch_message_set_header_values (notmuch_message_t 
 *message,
  message-doc.add_value (NOTMUCH_VALUE_SUBJECT, subject);
  }
  
 +/* Get the message mtime, i.e. when it was added or the last time a tag was
 + * added/removed. */
 +time_t
 +notmuch_message_get_mtime (notmuch_message_t *message)
 +{
 +std::string value;
 +
 +try {
 + value = message-doc.get_value (NOTMUCH_VALUE_MTIME);
 +} catch (Xapian::Error error) {
 + INTERNAL_ERROR (Failed to read mtime value from document.);
 + return 0;
 +}
 +
 +return Xapian::sortable_unserialise (value);
 +}
 +
 +/* Set the message mtime to now. */
 +void
 +_notmuch_message_update_mtime (notmuch_message_t *message)
 +{
 +time_t time_value;
 +
 +time_value = time (NULL);
 +message-doc.add_value (NOTMUCH_VALUE_MTIME,
 +Xapian::sortable_serialise (time_value));
 +}
 +
  /* Synchronize changes made to message-doc out into the database. */
  void
  _notmuch_message_sync (notmuch_message_t *message)
 @@ -994,6 +1022,8 @@ notmuch_message_add_tag (notmuch_message_t *message, 
 const char *tag)
   private_status);
  }
  
 +_notmuch_message_update_mtime (message);
 +
  if (! message-frozen)
   _notmuch_message_sync (message);
  
 @@ -1022,6 +1052,8 @@ notmuch_message_remove_tag (notmuch_message_t *message, 
 const char *tag)
   private_status);
  }
  
 +_notmuch_message_update_mtime (message);
 +
  if (! message-frozen)
   _notmuch_message_sync (message);
  
 diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
 index 60a932f..9859872 100644
 --- a/lib/notmuch-private.h
 +++ b/lib/notmuch-private.h
 @@ -95,7 +95,8 @@ typedef enum {
  NOTMUCH_VALUE_TIMESTAMP = 0,
  NOTMUCH_VALUE_MESSAGE_ID,
  NOTMUCH_VALUE_FROM,
 -NOTMUCH_VALUE_SUBJECT
 +NOTMUCH_VALUE_SUBJECT,
 +NOTMUCH_VALUE_MTIME
  } notmuch_value_t;
  
  /* Xapian (with flint backend) complains if we provide a term longer
 @@ -276,6 +277,9 @@ _notmuch_message_set_header_values (notmuch_message_t 
 *message,
   const char *from,
   const char *subject);
  void
 +_notmuch_message_update_mtime (notmuch_message_t *message);
 +
 +void
  _notmuch_message_sync (notmuch_message_t *message);
  
  notmuch_status_t
 diff --git a/lib/notmuch.h 

notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Wed, 29 Jun 2011 13:54:40 -0700, Jameson Graef Rollins  wrote:
> On Wed, 29 Jun 2011 14:21:11 -0600, Mark Anderson  
> wrote:
> > I personally prefer --output=files remain as it was, with one file per
> > mail (even though I submitted the patch to change it).  I suggest that
> > we could add another format to supply all files (perhaps
> > --output=allfiles, or --output=dupfiles).  I don't like my original
> > suggestion of "filelists" because it implies a list of lists to me.  A
> > list of lists would correlate better to the number of messages which
> > match the search terms, but doesn't correlate well to xargs input.
> 
> What's wrong with just outputting all the files matching the search,
> including duplicates?  I can't think of any reason where one would want
> to not include all files matching the search.  I would be curious to
> hear a use case there.

For someone who is using gmail + offlineimap, labels in gmail become
folders in maildir.

The maildir structure can have a large number of copies of each email
corresponding to the labels/tags which have been applied.

To add a label/tag that is visible to the gmail interface, one should
copy a file representing the message to the folder representing the
gmail label, which will then sync to gmail.

Copying more than one file for each message being labeled is more
wasteful of time and storage.

With all files returned, a simple xargs script to add a label by copying
files will end up with many copies of the same file in the new
directory.

The consuming script could hunt for message-id's in files and uniquify,
but since notmuch was doing that implicitly before, and it's fairly
natural, it seems not a big deal to add.

> Since I'm on this kick anyway, I'm going to keep pushing against further
> customizations where there really isn't a need.

With a common use case for the biggest email userbase which makes
labels/tags natural, I think it is worth considering seriously.

There are certainly other namesets which could be used to reprecent the
two categories.  I'm happy to use names that makes the 'allfiles' output
the common case and the "one file/message" the longer string, but I
think we should provide the "one file/message" output category.

The 'allfiles' case is great for deleting all copies of an email, so I
definitely want it to continue being available.

-Mark

notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Tue, 28 Jun 2011 16:53:30 -0700, Carl Worth  wrote:
> On Tue, 28 Jun 2011 16:38:30 -0600, Mark Anderson  
> wrote:
> > I had briefly considered adding another output format "file", just to get a
> > single file for each message in the db, but the file/files distinction
> > feels a bit niggling.  Perhaps it should be changed to "files" and
> > "filelists" or something else more clear.
> 
> Another option that would be general to several commands would be:
> 
>   notmuch search --output=files --exclude-duplicates
> 
> Or alternately, --include-duplicates. That might be more useful for
> "notmuch show" which is a case where users have previously asked for the
> ability to ask for duplicate messages, (and where the duplicates are
> squelched by default).
> 
> Thoughts?

I personally prefer --output=files remain as it was, with one file per
mail (even though I submitted the patch to change it).  I suggest that
we could add another format to supply all files (perhaps
--output=allfiles, or --output=dupfiles).  I don't like my original
suggestion of "filelists" because it implies a list of lists to me.  A
list of lists would correlate better to the number of messages which
match the search terms, but doesn't correlate well to xargs input.

I understand that we could use --include-duplicates, but I don't think
there are currently any output specifers that actually have a plurality
for a single message.  If we had something like --output=from, or some
other message attribute, then I think we would achieve more useful
orthogonality from adding an argument similar to --include-duplicates.

As it stands, it looks better to me to have a different --output
specifier to represent the uncommon case of multiple outputs per search
match.

-Mark

[PATCH] Fix folder: coherence issue

[Mark Anderson]

Add removal of all ZXFOLDER terms to removal of all XFOLDER terms for
each message filename removal.

The existing filename-list reindexing will put all the needed terms
back in.  Test search-folder-coherence now passes.

Signed-off-by:Mark Anderson 
---

Once I fixed the removal instead of the addition side, things went
smoothly.

 lib/message.cc |   31 ---
 1 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 8b9c84f..d993cde 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -514,6 +514,8 @@ _notmuch_message_remove_filename (notmuch_message_t 
*message,
 const char *folder_prefix = _find_prefix ("folder");
 int folder_prefix_len = strlen (folder_prefix);
 void *local = talloc_new (message);
+char *zfolder_prefix = talloc_asprintf(local, "Z%s", folder_prefix);
+int zfolder_prefix_len = strlen (zfolder_prefix);
 char *direntry;
 notmuch_private_status_t private_status;
 notmuch_status_t status;
@@ -530,9 +532,12 @@ _notmuch_message_remove_filename (notmuch_message_t 
*message,
 status = COERCE_STATUS (private_status,
"Unexpected error from 
_notmuch_message_remove_term");

-/* Re-synchronize "folder:" terms for this message. This requires
- * first removing all "folder:" terms, then adding back terms for
- * all remaining filenames of the message. */
+/* Re-synchronize "folder:" terms for this message. This requires:
+ *  1. removing all "folder:" terms
+ *  2. removing all "folder:" stemmed terms
+ *  3. adding back terms for all remaining filenames of the message. */
+
+/* 1. removing all "folder:" terms */
 while (1) {
i = message->doc.termlist_begin ();
i.skip_to (folder_prefix);
@@ -551,6 +556,26 @@ _notmuch_message_remove_filename (notmuch_message_t 
*message,
}
 }

+/* 2. removing all "folder:" stemmed terms */
+while (1) {
+   i = message->doc.termlist_begin ();
+   i.skip_to (zfolder_prefix);
+
+   /* Terminate loop when no terms remain with desired prefix. */
+   if (i == message->doc.termlist_end () ||
+   strncmp ((*i).c_str (), zfolder_prefix, zfolder_prefix_len))
+   {
+   break;
+   }
+
+   try {
+   message->doc.remove_term ((*i));
+   } catch (const Xapian::InvalidArgumentError) {
+   /* Ignore failure to remove non-existent term. */
+   }
+}
+
+/* 3. adding back terms for all remaining filenames of the message. */
 i = message->doc.termlist_begin ();
 i.skip_to (direntry_prefix);

-- 
1.7.4.1

[PATCH] Fix folder: coherence issue

[Mark Anderson]

Add removal of all ZXFOLDER terms to removal of all XFOLDER terms for
each message filename removal.

The existing filename-list reindexing will put all the needed terms
back in.  Test search-folder-coherence now passes.

Signed-off-by:Mark Anderson ma.sk...@gmail.com
---

Once I fixed the removal instead of the addition side, things went
smoothly.

 lib/message.cc |   31 ---
 1 files changed, 28 insertions(+), 3 deletions(-)

diff --git a/lib/message.cc b/lib/message.cc
index 8b9c84f..d993cde 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -514,6 +514,8 @@ _notmuch_message_remove_filename (notmuch_message_t 
*message,
 const char *folder_prefix = _find_prefix (folder);
 int folder_prefix_len = strlen (folder_prefix);
 void *local = talloc_new (message);
+char *zfolder_prefix = talloc_asprintf(local, Z%s, folder_prefix);
+int zfolder_prefix_len = strlen (zfolder_prefix);
 char *direntry;
 notmuch_private_status_t private_status;
 notmuch_status_t status;
@@ -530,9 +532,12 @@ _notmuch_message_remove_filename (notmuch_message_t 
*message,
 status = COERCE_STATUS (private_status,
Unexpected error from 
_notmuch_message_remove_term);
 
-/* Re-synchronize folder: terms for this message. This requires
- * first removing all folder: terms, then adding back terms for
- * all remaining filenames of the message. */
+/* Re-synchronize folder: terms for this message. This requires:
+ *  1. removing all folder: terms
+ *  2. removing all folder: stemmed terms
+ *  3. adding back terms for all remaining filenames of the message. */
+
+/* 1. removing all folder: terms */
 while (1) {
i = message-doc.termlist_begin ();
i.skip_to (folder_prefix);
@@ -551,6 +556,26 @@ _notmuch_message_remove_filename (notmuch_message_t 
*message,
}
 }
 
+/* 2. removing all folder: stemmed terms */
+while (1) {
+   i = message-doc.termlist_begin ();
+   i.skip_to (zfolder_prefix);
+
+   /* Terminate loop when no terms remain with desired prefix. */
+   if (i == message-doc.termlist_end () ||
+   strncmp ((*i).c_str (), zfolder_prefix, zfolder_prefix_len))
+   {
+   break;
+   }
+
+   try {
+   message-doc.remove_term ((*i));
+   } catch (const Xapian::InvalidArgumentError) {
+   /* Ignore failure to remove non-existent term. */
+   }
+}
+
+/* 3. adding back terms for all remaining filenames of the message. */
 i = message-doc.termlist_begin ();
 i.skip_to (direntry_prefix);
 
-- 
1.7.4.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Tue, 28 Jun 2011 16:53:30 -0700, Carl Worth cwo...@cworth.org wrote:
 On Tue, 28 Jun 2011 16:38:30 -0600, Mark Anderson ma.sk...@gmail.com wrote:
  I had briefly considered adding another output format file, just to get a
  single file for each message in the db, but the file/files distinction
  feels a bit niggling.  Perhaps it should be changed to files and
  filelists or something else more clear.
 
 Another option that would be general to several commands would be:
 
   notmuch search --output=files --exclude-duplicates
 
 Or alternately, --include-duplicates. That might be more useful for
 notmuch show which is a case where users have previously asked for the
 ability to ask for duplicate messages, (and where the duplicates are
 squelched by default).
 
 Thoughts?

I personally prefer --output=files remain as it was, with one file per
mail (even though I submitted the patch to change it).  I suggest that
we could add another format to supply all files (perhaps
--output=allfiles, or --output=dupfiles).  I don't like my original
suggestion of filelists because it implies a list of lists to me.  A
list of lists would correlate better to the number of messages which
match the search terms, but doesn't correlate well to xargs input.

I understand that we could use --include-duplicates, but I don't think
there are currently any output specifers that actually have a plurality
for a single message.  If we had something like --output=from, or some
other message attribute, then I think we would achieve more useful
orthogonality from adding an argument similar to --include-duplicates.

As it stands, it looks better to me to have a different --output
specifier to represent the uncommon case of multiple outputs per search
match.

-Mark
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Wed, 29 Jun 2011 13:54:40 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 On Wed, 29 Jun 2011 14:21:11 -0600, Mark Anderson ma.sk...@gmail.com wrote:
  I personally prefer --output=files remain as it was, with one file per
  mail (even though I submitted the patch to change it).  I suggest that
  we could add another format to supply all files (perhaps
  --output=allfiles, or --output=dupfiles).  I don't like my original
  suggestion of filelists because it implies a list of lists to me.  A
  list of lists would correlate better to the number of messages which
  match the search terms, but doesn't correlate well to xargs input.
 
 What's wrong with just outputting all the files matching the search,
 including duplicates?  I can't think of any reason where one would want
 to not include all files matching the search.  I would be curious to
 hear a use case there.

For someone who is using gmail + offlineimap, labels in gmail become
folders in maildir.

The maildir structure can have a large number of copies of each email
corresponding to the labels/tags which have been applied.

To add a label/tag that is visible to the gmail interface, one should
copy a file representing the message to the folder representing the
gmail label, which will then sync to gmail.

Copying more than one file for each message being labeled is more
wasteful of time and storage.

With all files returned, a simple xargs script to add a label by copying
files will end up with many copies of the same file in the new
directory.

The consuming script could hunt for message-id's in files and uniquify,
but since notmuch was doing that implicitly before, and it's fairly
natural, it seems not a big deal to add.

 Since I'm on this kick anyway, I'm going to keep pushing against further
 customizations where there really isn't a need.

With a common use case for the biggest email userbase which makes
labels/tags natural, I think it is worth considering seriously.

There are certainly other namesets which could be used to reprecent the
two categories.  I'm happy to use names that makes the 'allfiles' output
the common case and the one file/message the longer string, but I
think we should provide the one file/message output category.

The 'allfiles' case is great for deleting all copies of an email, so I
definitely want it to continue being available.

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Tue, 28 Jun 2011 23:43:52 +0200, "Sander Boer"  wrote:
> 
> Carl Worth  writes:> 
> I was hoping that google somehow was able to expose the tags in the "All
> Mail" folder, like the headers that are gmail specific: X-pstn-nxpr and 
> X-pstn-nxp (which contains a
> hash) for instance.

I don't even see those headers in my email in All Mail.  How are you
fetching your mail?

Or are you hoping for something else to be there that isn't there?

-Mark

notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Tue, 28 Jun 2011 23:43:52 +0200, "Sander Boer"  wrote:
> Carl Worth  writes:> 
> >
> > Hopefully it's clear enough that you could do the above in a script that
> > loops over all of your existing tags.
> >
> > And if you were doing a one-time switch from Gmail to notmuch that would
> > be all you would need.
> >
> > I don't know if you're looking to also push tags added via some notmuch
> > interface back to Gmail, (does Gmail even provide a mechanism for doing
> > that?). If so, then you'd need something that took notmuch tags and made
> > copies of the message in the appropriate files. That would hopefully be
> > easy to script based on the output of:
> >
> > notmuch search --output=files tag:important

You'd probably actually want this:

 notmuch search --output=files tag:importand and not folder:important

Although until the folder: tag bug is fixed, it won't be as definitive
as you want, because once the message was in folder:important, it
doesn't really leave.

However, With my recent patch you'll also get more filenames than you
want for this behavior.  If you already have the mail in All Mail,
Inbox, my_special_tag, and not_that_tag, do you want 4 links or copies
of the message placed in the folder for Important?

I had thought of this, because I am a Gmail/notmuch user (well,
somewhat, I have some of the infra in place, but it's not polished, I
usually end up using phone or web)

I had briefly considered adding another output format "file", just to get a
single file for each message in the db, but the file/files distinction
feels a bit niggling.  Perhaps it should be changed to "files" and
"filelists" or something else more clear.

It was nagging me as I implemented the fix I submitted, which looks like
it has been pushed.

Any comments?  I don't think I have time to code any changes to this for
a couple days.

> > I think my short answer is that it's fairly easy to convert from Gmail
> > tags to notmuch tags as part of a one-time import. Doing this on a
> > continual basis might benefit from writing a few scripts, and I don't
> > know if anyone has written those scripts yet.

Yes, you need a label->folder as well as a folder->label part of the
script.  I think of it as:

1. label->folder sync
2. offlineimap (pushes label changes, pulls new mail)
3. folder->label sync

There is certainly room for some conflict, if you use multiple
interfaces.  So watch your head.

-Mark

> >
> > Would any Gmail+notmuch users care to add anything to the conversation?
> >
> > -Carl
> 
> 
> now using gnus still, b/c notmuch does not build for the N900 (bummer)
> 
> -- 
> Sander Boer
> 
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] test:Improve test behaviors when --root is used

[Mark Anderson]

Change add_email_corpus, emacs_deliver_message and tests to use
$TEST_DIRECTORY instead of '..'.

This improves the behavior of the usage of --root=, as the
assumption of what '..' means will usually be incorrect.

Document -root option in README and update valgrind to work with
-root.
---

This patch actually fixes what Austin pointed out.  

 test/README|9 +
 test/basic |   10 +-
 test/crypto|2 +-
 test/emacs |4 ++--
 test/symbol-hiding |4 ++--
 test/test-lib.sh   |   18 +-
 6 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/test/README b/test/README
index be75e0e..8fbf78d 100644
--- a/test/README
+++ b/test/README
@@ -41,6 +41,15 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.

+--root=::
+   This runs the testsuites specified under a seperate directory.
+   However, caution is advised, as not all tests are maintained
+   with this relocation in mind, so some tests may behave
+   differently.
+
+   Pointing this argument at a tmpfs filesystem can improve the
+   speed of the test suite for some users.
+
 When invoking the test suite via "make test" any of the above options
 can be specified as follows:

diff --git a/test/basic b/test/basic
index d6e8c10..33bf711 100755
--- a/test/basic
+++ b/test/basic
@@ -51,9 +51,9 @@ test_expect_code 2 'failure to clean up causes the test to 
fail' '

 # Ensure that all tests are being run
 test_begin_subtest 'Ensure that all available tests will be run by 
notmuch-test'
-eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test ../notmuch-test)
+eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test 
$TEST_DIRECTORY/notmuch-test)
 tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
-available=$(ls -1 ../ | \
+available=$(ls -1 $TEST_DIRECTORY/ | \
 sed -r -e 
"/^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test)/d" \
   -e 
"/^(README|test-lib.sh|test-lib.el|test-results|tmp.*|valgrind|corpus*)/d" \
   -e 
"/^(emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|symbol-test.cc)/d"
 \
@@ -63,19 +63,19 @@ available=$(ls -1 ../ | \
   | sort)
 test_expect_equal "$tests_in_suite" "$available"

-EXPECTED=../test.expected-output
+EXPECTED=$TEST_DIRECTORY/test.expected-output
 suppress_diff_date() {
 sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \
-e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'
 }

 test_begin_subtest "Ensure that test output is suppressed unless the test 
fails"
-output=$(cd ..; ./test-verbose 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)
 test_expect_equal "$output" "$expected"

 test_begin_subtest "Ensure that -v does not suppress test output"
-output=$(cd ..; ./test-verbose -v 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
 rm $TEST_DIRECTORY/test-results/test-verbose-*
diff --git a/test/crypto b/test/crypto
index 01daffe..7eb3559 100755
--- a/test/crypto
+++ b/test/crypto
@@ -12,7 +12,7 @@ add_gnupg_home ()
 local output
 [ -d ${GNUPGHOME} ] && return
 mkdir -m 0700 "$GNUPGHOME"
-gpg --no-tty --import <../gnupg-secret-key.asc >"$GNUPGHOME"/import.log 
2>&1
+gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc 
>"$GNUPGHOME"/import.log 2>&1
 test_debug "cat $GNUPGHOME/import.log"
 if (gpg --quick-random --version >/dev/null 2>&1) ; then
echo quick-random >> "$GNUPGHOME"/gpg.conf
diff --git a/test/emacs b/test/emacs
index 6f82b08..f91078e 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,7 +2,7 @@
 test_description="emacs interface"
 . test-lib.sh

-EXPECTED=../emacs.expected-output
+EXPECTED=$TEST_DIRECTORY/emacs.expected-output

 add_email_corpus

@@ -81,7 +81,7 @@ mkdir -p mail/sent/cur
 mkdir -p mail/sent/new
 mkdir -p mail/sent/tmp

-../smtp-dummy sent_message &
+$TEST_DIRECTORY/smtp-dummy sent_message &
 smtp_dummy_pid=$!
 test_emacs "(setq message-send-mail-function 'message-smtpmail-send-it) (setq 
smtpmail-smtp-server \"localhost\") (setq smtpmail-smtp-service \"25025\") 
(notmuch-hello) (notmuch-mua-mail) (message-goto-to) (insert \"user at 
example.com\nDate: Fri, 29 Mar 1974 10:00:00 -\") (message-goto-subject) 
(insert \"Testing message sent via SMTP\") (message-goto-body) (insert \"This 
is a test that messages are sent via SMTP\") (message-send-and-exit)" 
>/dev/null 2>&1
 wait ${smtp_dummy_pid}
diff --git a/test/symbol-hiding b/test/symbol-hiding
index bb55524..d0b31ae 100755
--- a/test/symbol-hiding
+++ b/test/symbol-hiding
@@ -12,13 +12,13 @@ test_description='exception symbol 

[PATCH v2] test:Improve test behaviors when --root is used

[Mark Anderson]

Change add_email_corpus, emacs_deliver_message and tests to use
$TEST_DIRECTORY instead of '..'.

This improves the behavior of the usage of --root=, as the
assumption of what '..' means will usually be incorrect.

Document -root option in README and update valgrind to work with
-root.

Signed-off-by: Mark Anderson 
---
> If you could follow up with an updated patch, (or an argument that the
> original patch is correct), that would be great.

Updated the patch with Austin's suggestion.

I'm not personally ready to forbid the use of '..', although I certainly
appreciate the motivation to keep our tests working in --root mode.

 test/README|9 +
 test/basic |   10 +-
 test/crypto|2 +-
 test/emacs |4 ++--
 test/symbol-hiding |4 ++--
 test/test-lib.sh   |   18 +-
 6 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/test/README b/test/README
index be75e0e..8fbf78d 100644
--- a/test/README
+++ b/test/README
@@ -41,6 +41,15 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.

+--root=::
+   This runs the testsuites specified under a seperate directory.
+   However, caution is advised, as not all tests are maintained
+   with this relocation in mind, so some tests may behave
+   differently.
+
+   Pointing this argument at a tmpfs filesystem can improve the
+   speed of the test suite for some users.
+
 When invoking the test suite via "make test" any of the above options
 can be specified as follows:

diff --git a/test/basic b/test/basic
index d6e8c10..33bf711 100755
--- a/test/basic
+++ b/test/basic
@@ -51,9 +51,9 @@ test_expect_code 2 'failure to clean up causes the test to 
fail' '

 # Ensure that all tests are being run
 test_begin_subtest 'Ensure that all available tests will be run by 
notmuch-test'
-eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test ../notmuch-test)
+eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test 
$TEST_DIRECTORY/notmuch-test)
 tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
-available=$(ls -1 ../ | \
+available=$(ls -1 $TEST_DIRECTORY/ | \
 sed -r -e 
"/^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test)/d" \
   -e 
"/^(README|test-lib.sh|test-lib.el|test-results|tmp.*|valgrind|corpus*)/d" \
   -e 
"/^(emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|symbol-test.cc)/d"
 \
@@ -63,19 +63,19 @@ available=$(ls -1 ../ | \
   | sort)
 test_expect_equal "$tests_in_suite" "$available"

-EXPECTED=../test.expected-output
+EXPECTED=$TEST_DIRECTORY/test.expected-output
 suppress_diff_date() {
 sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \
-e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'
 }

 test_begin_subtest "Ensure that test output is suppressed unless the test 
fails"
-output=$(cd ..; ./test-verbose 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)
 test_expect_equal "$output" "$expected"

 test_begin_subtest "Ensure that -v does not suppress test output"
-output=$(cd ..; ./test-verbose -v 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
 rm $TEST_DIRECTORY/test-results/test-verbose-*
diff --git a/test/crypto b/test/crypto
index 01daffe..7eb3559 100755
--- a/test/crypto
+++ b/test/crypto
@@ -12,7 +12,7 @@ add_gnupg_home ()
 local output
 [ -d ${GNUPGHOME} ] && return
 mkdir -m 0700 "$GNUPGHOME"
-gpg --no-tty --import <../gnupg-secret-key.asc >"$GNUPGHOME"/import.log 
2>&1
+gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc 
>"$GNUPGHOME"/import.log 2>&1
 test_debug "cat $GNUPGHOME/import.log"
 if (gpg --quick-random --version >/dev/null 2>&1) ; then
echo quick-random >> "$GNUPGHOME"/gpg.conf
diff --git a/test/emacs b/test/emacs
index 6f82b08..f91078e 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,7 +2,7 @@
 test_description="emacs interface"
 . test-lib.sh

-EXPECTED=../emacs.expected-output
+EXPECTED=$TEST_DIRECTORY/emacs.expected-output

 add_email_corpus

@@ -81,7 +81,7 @@ mkdir -p mail/sent/cur
 mkdir -p mail/sent/new
 mkdir -p mail/sent/tmp

-../smtp-dummy sent_message &
+$TEST_DIRECTORY/smtp-dummy sent_message &
 smtp_dummy_pid=$!
 test_emacs "(setq message-send-mail-function 'message-smtpmail-send-it) (setq 
smtpmail-smtp-server \"localhost

Race condition for '*' command

[Mark Anderson]

On Tue, 28 Jun 2011 08:49:06 +0200, Pieter Praet  wrote:
> On Sun, 26 Jun 2011 10:00:41 +0100, Robin Green  
> wrote:
> > On Sat, 25 Jun 2011 16:57:50 -0700, Jameson Graef Rollins  > finestructure.net> wrote:
> > > On Sat, 25 Jun 2011 23:18:52 +0100, Robin Green  
> > > wrote:
> > > > A race condition in the '*' command was noted when it was first
> > > > proposed. It looks to me like it still exists - has anything been done
> > > > about it?
> > > 
> > > Hi, Robin.  Can you explain what you mean by the "'*' command"?
> > 
> > Sorry - forgot to say I'm talking about the notmuch emacs mode. In that
> > mode '*' applies tags to all messages matching the current search query,
> > which means that (here's the race condition) new results that have
> > appeared since the last refresh will also be tagged.
> 
> This issue appears to stem from the fact that `notmuch-search-operate-all'
> runs (apply 'notmuch-tag notmuch-search-query-string action-split), in which
> `notmuch-search-query-string' points to a moving target.
> 
> Could be solved by doing it with `notmuch-search', `mark-whole-buffer'
> and `notmuch-search-{add,remove}-tag-region' instead, but I'm sure
> there's a better way (of which I'm as of yet unaware).

I don't think there's a better way, the results which the user is
viewing is the only visual reference for what the action they intend to
apply will use as the object of said action.  I can imagine some people
wanting to have a way to apply an action to a live query string, but I
distinguish between these people and normal humans reading their email.
I expect that when a normal human is reading their email, they do not
intend to apply actions to messages as yet unseen.

I can see some people using the '*' notmuch emacs command as a very
interactive notmuch-poll.sh script.  But I imagine the principle of
least surprise would be to have '*' only apply to visible messages, and
if the buffer is old, the buffer is the only record of what is being
viewed.

-Mark

> 
> > -- 
> > Robin
> > ___
> > notmuch mailing list
> > notmuch at notmuchmail.org
> > http://notmuchmail.org/mailman/listinfo/notmuch
> 
> Peace
> 
> -- 
> Pieter
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch

Re: Race condition for '*' command

[Mark Anderson]

On Tue, 28 Jun 2011 08:49:06 +0200, Pieter Praet pie...@praet.org wrote:
 On Sun, 26 Jun 2011 10:00:41 +0100, Robin Green gree...@greenrd.org wrote:
  On Sat, 25 Jun 2011 16:57:50 -0700, Jameson Graef Rollins 
  jroll...@finestructure.net wrote:
   On Sat, 25 Jun 2011 23:18:52 +0100, Robin Green gree...@greenrd.org 
   wrote:
A race condition in the '*' command was noted when it was first
proposed. It looks to me like it still exists - has anything been done
about it?
   
   Hi, Robin.  Can you explain what you mean by the '*' command?
  
  Sorry - forgot to say I'm talking about the notmuch emacs mode. In that
  mode '*' applies tags to all messages matching the current search query,
  which means that (here's the race condition) new results that have
  appeared since the last refresh will also be tagged.
 
 This issue appears to stem from the fact that `notmuch-search-operate-all'
 runs (apply 'notmuch-tag notmuch-search-query-string action-split), in which
 `notmuch-search-query-string' points to a moving target.
 
 Could be solved by doing it with `notmuch-search', `mark-whole-buffer'
 and `notmuch-search-{add,remove}-tag-region' instead, but I'm sure
 there's a better way (of which I'm as of yet unaware).

I don't think there's a better way, the results which the user is
viewing is the only visual reference for what the action they intend to
apply will use as the object of said action.  I can imagine some people
wanting to have a way to apply an action to a live query string, but I
distinguish between these people and normal humans reading their email.
I expect that when a normal human is reading their email, they do not
intend to apply actions to messages as yet unseen.

I can see some people using the '*' notmuch emacs command as a very
interactive notmuch-poll.sh script.  But I imagine the principle of
least surprise would be to have '*' only apply to visible messages, and
if the buffer is old, the buffer is the only record of what is being
viewed.

-Mark

 
  -- 
  Robin
  ___
  notmuch mailing list
  notmuch@notmuchmail.org
  http://notmuchmail.org/mailman/listinfo/notmuch
 
 Peace
 
 -- 
 Pieter
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] test:Improve test behaviors when --root is used

[Mark Anderson]

Change add_email_corpus, emacs_deliver_message and tests to use
$TEST_DIRECTORY instead of '..'.

This improves the behavior of the usage of --root=dir, as the
assumption of what '..' means will usually be incorrect.

Document -root option in README and update valgrind to work with
-root.

Signed-off-by: Mark Anderson ma.sk...@gmail.com
---
 If you could follow up with an updated patch, (or an argument that the
 original patch is correct), that would be great.

Updated the patch with Austin's suggestion.

I'm not personally ready to forbid the use of '..', although I certainly
appreciate the motivation to keep our tests working in --root mode.

 test/README|9 +
 test/basic |   10 +-
 test/crypto|2 +-
 test/emacs |4 ++--
 test/symbol-hiding |4 ++--
 test/test-lib.sh   |   18 +-
 6 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/test/README b/test/README
index be75e0e..8fbf78d 100644
--- a/test/README
+++ b/test/README
@@ -41,6 +41,15 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
 
+--root=dir::
+   This runs the testsuites specified under a seperate directory.
+   However, caution is advised, as not all tests are maintained
+   with this relocation in mind, so some tests may behave
+   differently.
+
+   Pointing this argument at a tmpfs filesystem can improve the
+   speed of the test suite for some users.
+
 When invoking the test suite via make test any of the above options
 can be specified as follows:
 
diff --git a/test/basic b/test/basic
index d6e8c10..33bf711 100755
--- a/test/basic
+++ b/test/basic
@@ -51,9 +51,9 @@ test_expect_code 2 'failure to clean up causes the test to 
fail' '
 
 # Ensure that all tests are being run
 test_begin_subtest 'Ensure that all available tests will be run by 
notmuch-test'
-eval $(sed -n -e '/^TESTS=$/,/^$/p' notmuch-test ../notmuch-test)
+eval $(sed -n -e '/^TESTS=$/,/^$/p' notmuch-test 
$TEST_DIRECTORY/notmuch-test)
 tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
-available=$(ls -1 ../ | \
+available=$(ls -1 $TEST_DIRECTORY/ | \
 sed -r -e 
/^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test)/d \
   -e 
/^(README|test-lib.sh|test-lib.el|test-results|tmp.*|valgrind|corpus*)/d \
   -e 
/^(emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|symbol-test.cc)/d
 \
@@ -63,19 +63,19 @@ available=$(ls -1 ../ | \
   | sort)
 test_expect_equal $tests_in_suite $available
 
-EXPECTED=../test.expected-output
+EXPECTED=$TEST_DIRECTORY/test.expected-output
 suppress_diff_date() {
 sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \
-e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'
 }
 
 test_begin_subtest Ensure that test output is suppressed unless the test 
fails
-output=$(cd ..; ./test-verbose 21 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose 21 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)
 test_expect_equal $output $expected
 
 test_begin_subtest Ensure that -v does not suppress test output
-output=$(cd ..; ./test-verbose -v 21 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose -v 21 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
 rm $TEST_DIRECTORY/test-results/test-verbose-*
diff --git a/test/crypto b/test/crypto
index 01daffe..7eb3559 100755
--- a/test/crypto
+++ b/test/crypto
@@ -12,7 +12,7 @@ add_gnupg_home ()
 local output
 [ -d ${GNUPGHOME} ]  return
 mkdir -m 0700 $GNUPGHOME
-gpg --no-tty --import ../gnupg-secret-key.asc $GNUPGHOME/import.log 
21
+gpg --no-tty --import $TEST_DIRECTORY/gnupg-secret-key.asc 
$GNUPGHOME/import.log 21
 test_debug cat $GNUPGHOME/import.log
 if (gpg --quick-random --version /dev/null 21) ; then
echo quick-random  $GNUPGHOME/gpg.conf
diff --git a/test/emacs b/test/emacs
index 6f82b08..f91078e 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,7 +2,7 @@
 test_description=emacs interface
 . test-lib.sh
 
-EXPECTED=../emacs.expected-output
+EXPECTED=$TEST_DIRECTORY/emacs.expected-output
 
 add_email_corpus
 
@@ -81,7 +81,7 @@ mkdir -p mail/sent/cur
 mkdir -p mail/sent/new
 mkdir -p mail/sent/tmp
 
-../smtp-dummy sent_message 
+$TEST_DIRECTORY/smtp-dummy sent_message 
 smtp_dummy_pid=$!
 test_emacs (setq message-send-mail-function 'message-smtpmail-send-it) (setq 
smtpmail-smtp-server \localhost\) (setq smtpmail-smtp-service \25025\) 
(notmuch-hello) (notmuch-mua-mail) (message-goto-to) (insert 
\u...@example.com\nDate: Fri, 29 Mar 1974 10:00:00 -\) 
(message-goto-subject) (insert \Testing message sent via SMTP\) 
(message-goto-body) (insert \This is a test that messages are sent via SMTP\) 
(message-send

[PATCH v3] test:Improve test behaviors when --root is used

[Mark Anderson]

Change add_email_corpus, emacs_deliver_message and tests to use
$TEST_DIRECTORY instead of '..'.

This improves the behavior of the usage of --root=dir, as the
assumption of what '..' means will usually be incorrect.

Document -root option in README and update valgrind to work with
-root.
---

This patch actually fixes what Austin pointed out.  

 test/README|9 +
 test/basic |   10 +-
 test/crypto|2 +-
 test/emacs |4 ++--
 test/symbol-hiding |4 ++--
 test/test-lib.sh   |   18 +-
 6 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/test/README b/test/README
index be75e0e..8fbf78d 100644
--- a/test/README
+++ b/test/README
@@ -41,6 +41,15 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
 
+--root=dir::
+   This runs the testsuites specified under a seperate directory.
+   However, caution is advised, as not all tests are maintained
+   with this relocation in mind, so some tests may behave
+   differently.
+
+   Pointing this argument at a tmpfs filesystem can improve the
+   speed of the test suite for some users.
+
 When invoking the test suite via make test any of the above options
 can be specified as follows:
 
diff --git a/test/basic b/test/basic
index d6e8c10..33bf711 100755
--- a/test/basic
+++ b/test/basic
@@ -51,9 +51,9 @@ test_expect_code 2 'failure to clean up causes the test to 
fail' '
 
 # Ensure that all tests are being run
 test_begin_subtest 'Ensure that all available tests will be run by 
notmuch-test'
-eval $(sed -n -e '/^TESTS=$/,/^$/p' notmuch-test ../notmuch-test)
+eval $(sed -n -e '/^TESTS=$/,/^$/p' notmuch-test 
$TEST_DIRECTORY/notmuch-test)
 tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
-available=$(ls -1 ../ | \
+available=$(ls -1 $TEST_DIRECTORY/ | \
 sed -r -e 
/^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test)/d \
   -e 
/^(README|test-lib.sh|test-lib.el|test-results|tmp.*|valgrind|corpus*)/d \
   -e 
/^(emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|symbol-test.cc)/d
 \
@@ -63,19 +63,19 @@ available=$(ls -1 ../ | \
   | sort)
 test_expect_equal $tests_in_suite $available
 
-EXPECTED=../test.expected-output
+EXPECTED=$TEST_DIRECTORY/test.expected-output
 suppress_diff_date() {
 sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \
-e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'
 }
 
 test_begin_subtest Ensure that test output is suppressed unless the test 
fails
-output=$(cd ..; ./test-verbose 21 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose 21 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)
 test_expect_equal $output $expected
 
 test_begin_subtest Ensure that -v does not suppress test output
-output=$(cd ..; ./test-verbose -v 21 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose -v 21 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
 rm $TEST_DIRECTORY/test-results/test-verbose-*
diff --git a/test/crypto b/test/crypto
index 01daffe..7eb3559 100755
--- a/test/crypto
+++ b/test/crypto
@@ -12,7 +12,7 @@ add_gnupg_home ()
 local output
 [ -d ${GNUPGHOME} ]  return
 mkdir -m 0700 $GNUPGHOME
-gpg --no-tty --import ../gnupg-secret-key.asc $GNUPGHOME/import.log 
21
+gpg --no-tty --import $TEST_DIRECTORY/gnupg-secret-key.asc 
$GNUPGHOME/import.log 21
 test_debug cat $GNUPGHOME/import.log
 if (gpg --quick-random --version /dev/null 21) ; then
echo quick-random  $GNUPGHOME/gpg.conf
diff --git a/test/emacs b/test/emacs
index 6f82b08..f91078e 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,7 +2,7 @@
 test_description=emacs interface
 . test-lib.sh
 
-EXPECTED=../emacs.expected-output
+EXPECTED=$TEST_DIRECTORY/emacs.expected-output
 
 add_email_corpus
 
@@ -81,7 +81,7 @@ mkdir -p mail/sent/cur
 mkdir -p mail/sent/new
 mkdir -p mail/sent/tmp
 
-../smtp-dummy sent_message 
+$TEST_DIRECTORY/smtp-dummy sent_message 
 smtp_dummy_pid=$!
 test_emacs (setq message-send-mail-function 'message-smtpmail-send-it) (setq 
smtpmail-smtp-server \localhost\) (setq smtpmail-smtp-service \25025\) 
(notmuch-hello) (notmuch-mua-mail) (message-goto-to) (insert 
\u...@example.com\nDate: Fri, 29 Mar 1974 10:00:00 -\) 
(message-goto-subject) (insert \Testing message sent via SMTP\) 
(message-goto-body) (insert \This is a test that messages are sent via SMTP\) 
(message-send-and-exit) /dev/null 21
 wait ${smtp_dummy_pid}
diff --git a/test/symbol-hiding b/test/symbol-hiding
index bb55524..d0b31ae 100755
--- a/test/symbol-hiding
+++ b/test/symbol-hiding
@@ -12,13 +12,13 @@ test_description='exception symbol hiding'
 . ./test-lib.sh
 
 run_test(){
-

Re: notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Tue, 28 Jun 2011 23:43:52 +0200, Sander Boer sanb...@gmail.com wrote:
 Carl Worth cwo...@cworth.org writes: 
 
  Hopefully it's clear enough that you could do the above in a script that
  loops over all of your existing tags.
 
  And if you were doing a one-time switch from Gmail to notmuch that would
  be all you would need.
 
  I don't know if you're looking to also push tags added via some notmuch
  interface back to Gmail, (does Gmail even provide a mechanism for doing
  that?). If so, then you'd need something that took notmuch tags and made
  copies of the message in the appropriate files. That would hopefully be
  easy to script based on the output of:
 
  notmuch search --output=files tag:important

You'd probably actually want this:

 notmuch search --output=files tag:importand and not folder:important

Although until the folder: tag bug is fixed, it won't be as definitive
as you want, because once the message was in folder:important, it
doesn't really leave.

However, With my recent patch you'll also get more filenames than you
want for this behavior.  If you already have the mail in All Mail,
Inbox, my_special_tag, and not_that_tag, do you want 4 links or copies
of the message placed in the folder for Important?

I had thought of this, because I am a Gmail/notmuch user (well,
somewhat, I have some of the infra in place, but it's not polished, I
usually end up using phone or web)

I had briefly considered adding another output format file, just to get a
single file for each message in the db, but the file/files distinction
feels a bit niggling.  Perhaps it should be changed to files and
filelists or something else more clear.

It was nagging me as I implemented the fix I submitted, which looks like
it has been pushed.

Any comments?  I don't think I have time to code any changes to this for
a couple days.

  I think my short answer is that it's fairly easy to convert from Gmail
  tags to notmuch tags as part of a one-time import. Doing this on a
  continual basis might benefit from writing a few scripts, and I don't
  know if anyone has written those scripts yet.

Yes, you need a label-folder as well as a folder-label part of the
script.  I think of it as:

1. label-folder sync
2. offlineimap (pushes label changes, pulls new mail)
3. folder-label sync

There is certainly room for some conflict, if you use multiple
interfaces.  So watch your head.

-Mark

 
  Would any Gmail+notmuch users care to add anything to the conversation?
 
  -Carl
 
 
 now using gnus still, b/c notmuch does not build for the N900 (bummer)
 
 -- 
 Sander Boer
 
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch Digest, Vol 20, Issue 57

[Mark Anderson]

On Tue, 28 Jun 2011 23:43:52 +0200, Sander Boer sanb...@gmail.com wrote:
 
 Carl Worth cwo...@cworth.org writes: 
 I was hoping that google somehow was able to expose the tags in the All
 Mail folder, like the headers that are gmail specific: X-pstn-nxpr and 
 X-pstn-nxp (which contains a
 hash) for instance.

I don't even see those headers in my email in All Mail.  How are you
fetching your mail?

Or are you hoping for something else to be there that isn't there?

-Mark
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] test:Improve test behaviors when --root is used

[Mark Anderson]

On Mon, 27 Jun 2011 15:50:47 -0500, Austin Clements  wrote:
> This looks great (modulo one bug, below).  I've wanted to run the
> tests on tmpfs before, but was too lazy to actually fix the tests.
> 
> Given how easy it is to accidentally use "..", I wonder if there's a
> way to force people to use $TEST_DIRECTORY?

I was thinking of a test of the test-suite running in this mode, but I
didn't quite feel comfortable, since it feels a bit slow sometimes
already.

So an instruction to run 'make test', which would include a 'make test
OPTIONS="--root=/dev/shm/notmuch_test.$$"' or some such if that was
acceptable.

I didn't think about it very hard, but I was intrigued by the idea.

-Mark

> On Mon, Jun 27, 2011 at 12:09 PM, Mark Anderson  wrote:
> > --- a/test/symbol-hiding
> > +++ b/test/symbol-hiding
> > @@ -12,13 +12,13 @@ test_description='exception symbol hiding'
> > ?. ./test-lib.sh
> >
> > ?run_test(){
> > - ? ?result=$(LD_LIBRARY_PATH=../../lib ./symbol-test 2>&1)
> > + ? ?result=$(LD_LIBRARY_PATH=$TEST_DIRECTORY/../../lib ./symbol-test 2>&1)
> 
> Did you mean $TEST_DIRECTORY/../lib?
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
> 

[PATCH] test:Folder tags shouldn't match after removal of file in given folder

[Mark Anderson]

On Mon, 27 Jun 2011 15:58:00 -0500, Austin Clements  wrote:
> On Mon, Jun 27, 2011 at 1:12 PM, Mark Anderson  wrote:
> > +test_begin_subtest "Test matches folder:spam"
> > +output=$(notmuch search folder:spam)
> > +test_expect_equal "$output" "thread:0001 ? 2001-01-05 [1/1] 
> > Notmuch Test Suite; Test message #1 (inbox unread)"
> > +
> > +sleep 1;
> 
> I assume this has to do with directory mtime's.  Is it sufficient to
> instead increment_mtime $dir/spam after the rm below?

I believe that would be sufficient.  I just had one of my runs fail to
even remove the filename, and I remember some earlier discussion along
these lines, but I just used the fastest way, I can certainly modify it
to use "increment_mtime"

> 
> > +
> > +test_begin_subtest "Remove folder:spam copy of email"
> > +rm $dir/spam/$(basename $file_x)
> > +output=$(NOTMUCH_NEW)
> > +test_expect_equal "$output" "No new mail. Detected 1 file rename."
> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
> 

[PATCH] test:Folder tags shouldn't match after removal of file in given folder

[Mark Anderson]

Test for bug.  Current stemming support for notmuch adds extra terms
to the DB which aren't removed when the file renames are detected.

When folder tags are added to a message, Xapian terms for both XFOLDER
and ZXFOLDER are generated.  When one of the filenames are
renamed/removed, only the XFOLDER tags are removed, leaving it possible
for a match on a folder: tag that was previously but is no longer a
match in the maildir.
---

I found this bug last week.  Essentially when the folder:spam tag is
added and puts the XFOLDERspam, it also inserts the ZXFOLDERspam.  Then
if the mail is removed from the folder, it neglects to remove
ZXFOLDERspam.

This was detected with my offlineimap usage with gmail, still haven't
polished my personal folder/label transition.  As I was looking into it,
I saw some unusual things flagged as spam, and investigated.

 test/notmuch-test|1 +
 test/search-folder-coherence |   48 ++
 2 files changed, 49 insertions(+), 0 deletions(-)
 create mode 100755 test/search-folder-coherence

diff --git a/test/notmuch-test b/test/notmuch-test
index fe85c6a..79e6267 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -41,6 +41,7 @@ TESTS="
   maildir-sync
   crypto
   symbol-hiding
+  search-folder-coherence
 "
 TESTS=${NOTMUCH_TESTS:=$TESTS}

diff --git a/test/search-folder-coherence b/test/search-folder-coherence
new file mode 100755
index 000..cf3ba40
--- /dev/null
+++ b/test/search-folder-coherence
@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+test_description='folder tags removed and added through file renames remain 
consistent'
+. ./test-lib.sh
+
+test_begin_subtest "No new messages"
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail."
+
+
+test_begin_subtest "Single new message"
+generate_message
+file_x=$gen_msg_filename
+id_x=$gen_msg_id
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "Added 1 new message to the database."
+
+test_begin_subtest "Add second folder for same message"
+dir=$(dirname $file_x)
+mkdir $dir/spam
+cp $file_x $dir/spam
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail."
+
+
+test_begin_subtest "Multiple files for same message"
+cat OUTPUT
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest "Test matches folder:spam"
+output=$(notmuch search folder:spam)
+test_expect_equal "$output" "thread:0001   2001-01-05 [1/1] 
Notmuch Test Suite; Test message #1 (inbox unread)"
+
+sleep 1;
+
+test_begin_subtest "Remove folder:spam copy of email"
+rm $dir/spam/$(basename $file_x)
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail. Detected 1 file rename."
+
+test_begin_subtest "No mails match the folder:spam search"
+output=$(notmuch search folder:spam)
+test_expect_equal "$output" ""
+
+test_done
-- 
1.7.4.1

[PATCH] test:Improve test behaviors when --root is used

[Mark Anderson]

Change add_email_corpus, emacs_deliver_message and tests to use
$TEST_DIRECTORY instead of '..'.

This improves the behavior of the usage of --root=, as the
assumption of '..' will be incorrect if the option is specified.

Document -root option in README and update valgrind to work with
-root.
---

I discovered the -root option when I wanted to run my test suite on
tmpfs, since my main drive is still spinning rust, and found test run
relocation is already implemented.

However, many tests were not well behaved when --root was specified,
which makes sense since it wasn't documented in the README, added in
this patch.

It seems like a good idea to try and run the test suite on a tmpfs
drive, but I don't know if there is a sufficiently generic standard
location or detection mechanism.  I use /dev/shm/notmuch_test, which
works for me.  I do wish I didn't have to keep specifying it, but I'm
not sure how well received putting this in notmuch's configure file
would be.


 test/README|9 +
 test/basic |   10 +-
 test/crypto|2 +-
 test/emacs |4 ++--
 test/symbol-hiding |4 ++--
 test/test-lib.sh   |   18 +-
 6 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/test/README b/test/README
index be75e0e..8fbf78d 100644
--- a/test/README
+++ b/test/README
@@ -41,6 +41,15 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.

+--root=::
+   This runs the testsuites specified under a seperate directory.
+   However, caution is advised, as not all tests are maintained
+   with this relocation in mind, so some tests may behave
+   differently.
+
+   Pointing this argument at a tmpfs filesystem can improve the
+   speed of the test suite for some users.
+
 When invoking the test suite via "make test" any of the above options
 can be specified as follows:

diff --git a/test/basic b/test/basic
index d6e8c10..33bf711 100755
--- a/test/basic
+++ b/test/basic
@@ -51,9 +51,9 @@ test_expect_code 2 'failure to clean up causes the test to 
fail' '

 # Ensure that all tests are being run
 test_begin_subtest 'Ensure that all available tests will be run by 
notmuch-test'
-eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test ../notmuch-test)
+eval $(sed -n -e '/^TESTS="$/,/^"$/p' notmuch-test 
$TEST_DIRECTORY/notmuch-test)
 tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
-available=$(ls -1 ../ | \
+available=$(ls -1 $TEST_DIRECTORY/ | \
 sed -r -e 
"/^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test)/d" \
   -e 
"/^(README|test-lib.sh|test-lib.el|test-results|tmp.*|valgrind|corpus*)/d" \
   -e 
"/^(emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|symbol-test.cc)/d"
 \
@@ -63,19 +63,19 @@ available=$(ls -1 ../ | \
   | sort)
 test_expect_equal "$tests_in_suite" "$available"

-EXPECTED=../test.expected-output
+EXPECTED=$TEST_DIRECTORY/test.expected-output
 suppress_diff_date() {
 sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \
-e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'
 }

 test_begin_subtest "Ensure that test output is suppressed unless the test 
fails"
-output=$(cd ..; ./test-verbose 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)
 test_expect_equal "$output" "$expected"

 test_begin_subtest "Ensure that -v does not suppress test output"
-output=$(cd ..; ./test-verbose -v 2>&1 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose -v 2>&1 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
 rm $TEST_DIRECTORY/test-results/test-verbose-*
diff --git a/test/crypto b/test/crypto
index 01daffe..7eb3559 100755
--- a/test/crypto
+++ b/test/crypto
@@ -12,7 +12,7 @@ add_gnupg_home ()
 local output
 [ -d ${GNUPGHOME} ] && return
 mkdir -m 0700 "$GNUPGHOME"
-gpg --no-tty --import <../gnupg-secret-key.asc >"$GNUPGHOME"/import.log 
2>&1
+gpg --no-tty --import <$TEST_DIRECTORY/gnupg-secret-key.asc 
>"$GNUPGHOME"/import.log 2>&1
 test_debug "cat $GNUPGHOME/import.log"
 if (gpg --quick-random --version >/dev/null 2>&1) ; then
echo quick-random >> "$GNUPGHOME"/gpg.conf
diff --git a/test/emacs b/test/emacs
index 6f82b08..f91078e 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,7 +2,7 @@
 test_description="emacs interface"
 . test-lib.sh

-EXPECTED=../emacs.expected-output
+EXPECTED=$TEST_DIRECTORY/emacs.expected-output

 add_email_corpus

@@ -81,7 +81,7 @@ mkdir -p mail/sent/cur
 mkdir -p mail/sent/new
 mkdir -p mail/sent/tmp

-../smtp-dummy sent_message &
+$TEST_DIRECTORY/smtp-dummy sent_message &
 smtp_dummy_pid=$!
 test_emacs "(setq message-send-mail-function 

[PATCH] test:Improve test behaviors when --root is used

[Mark Anderson]

Change add_email_corpus, emacs_deliver_message and tests to use
$TEST_DIRECTORY instead of '..'.

This improves the behavior of the usage of --root=dir, as the
assumption of '..' will be incorrect if the option is specified.

Document -root option in README and update valgrind to work with
-root.
---

I discovered the -root option when I wanted to run my test suite on
tmpfs, since my main drive is still spinning rust, and found test run
relocation is already implemented.

However, many tests were not well behaved when --root was specified,
which makes sense since it wasn't documented in the README, added in
this patch.

It seems like a good idea to try and run the test suite on a tmpfs
drive, but I don't know if there is a sufficiently generic standard
location or detection mechanism.  I use /dev/shm/notmuch_test, which
works for me.  I do wish I didn't have to keep specifying it, but I'm
not sure how well received putting this in notmuch's configure file
would be.


 test/README|9 +
 test/basic |   10 +-
 test/crypto|2 +-
 test/emacs |4 ++--
 test/symbol-hiding |4 ++--
 test/test-lib.sh   |   18 +-
 6 files changed, 28 insertions(+), 19 deletions(-)

diff --git a/test/README b/test/README
index be75e0e..8fbf78d 100644
--- a/test/README
+++ b/test/README
@@ -41,6 +41,15 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
 
+--root=dir::
+   This runs the testsuites specified under a seperate directory.
+   However, caution is advised, as not all tests are maintained
+   with this relocation in mind, so some tests may behave
+   differently.
+
+   Pointing this argument at a tmpfs filesystem can improve the
+   speed of the test suite for some users.
+
 When invoking the test suite via make test any of the above options
 can be specified as follows:
 
diff --git a/test/basic b/test/basic
index d6e8c10..33bf711 100755
--- a/test/basic
+++ b/test/basic
@@ -51,9 +51,9 @@ test_expect_code 2 'failure to clean up causes the test to 
fail' '
 
 # Ensure that all tests are being run
 test_begin_subtest 'Ensure that all available tests will be run by 
notmuch-test'
-eval $(sed -n -e '/^TESTS=$/,/^$/p' notmuch-test ../notmuch-test)
+eval $(sed -n -e '/^TESTS=$/,/^$/p' notmuch-test 
$TEST_DIRECTORY/notmuch-test)
 tests_in_suite=$(for i in $TESTS; do echo $i; done | sort)
-available=$(ls -1 ../ | \
+available=$(ls -1 $TEST_DIRECTORY/ | \
 sed -r -e 
/^(aggregate-results.sh|Makefile|Makefile.local|notmuch-test)/d \
   -e 
/^(README|test-lib.sh|test-lib.el|test-results|tmp.*|valgrind|corpus*)/d \
   -e 
/^(emacs.expected-output|smtp-dummy|smtp-dummy.c|test-verbose|symbol-test.cc)/d
 \
@@ -63,19 +63,19 @@ available=$(ls -1 ../ | \
   | sort)
 test_expect_equal $tests_in_suite $available
 
-EXPECTED=../test.expected-output
+EXPECTED=$TEST_DIRECTORY/test.expected-output
 suppress_diff_date() {
 sed -e 's/\(.*\-\-\- test-verbose\.4\.\expected\).*/\1/' \
-e 's/\(.*\+\+\+ test-verbose\.4\.\output\).*/\1/'
 }
 
 test_begin_subtest Ensure that test output is suppressed unless the test 
fails
-output=$(cd ..; ./test-verbose 21 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose 21 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-no | suppress_diff_date)
 test_expect_equal $output $expected
 
 test_begin_subtest Ensure that -v does not suppress test output
-output=$(cd ..; ./test-verbose -v 21 | suppress_diff_date)
+output=$(cd $TEST_DIRECTORY; ./test-verbose -v 21 | suppress_diff_date)
 expected=$(cat $EXPECTED/test-verbose-yes | suppress_diff_date)
 # Do not include the results of test-verbose in totals
 rm $TEST_DIRECTORY/test-results/test-verbose-*
diff --git a/test/crypto b/test/crypto
index 01daffe..7eb3559 100755
--- a/test/crypto
+++ b/test/crypto
@@ -12,7 +12,7 @@ add_gnupg_home ()
 local output
 [ -d ${GNUPGHOME} ]  return
 mkdir -m 0700 $GNUPGHOME
-gpg --no-tty --import ../gnupg-secret-key.asc $GNUPGHOME/import.log 
21
+gpg --no-tty --import $TEST_DIRECTORY/gnupg-secret-key.asc 
$GNUPGHOME/import.log 21
 test_debug cat $GNUPGHOME/import.log
 if (gpg --quick-random --version /dev/null 21) ; then
echo quick-random  $GNUPGHOME/gpg.conf
diff --git a/test/emacs b/test/emacs
index 6f82b08..f91078e 100755
--- a/test/emacs
+++ b/test/emacs
@@ -2,7 +2,7 @@
 test_description=emacs interface
 . test-lib.sh
 
-EXPECTED=../emacs.expected-output
+EXPECTED=$TEST_DIRECTORY/emacs.expected-output
 
 add_email_corpus
 
@@ -81,7 +81,7 @@ mkdir -p mail/sent/cur
 mkdir -p mail/sent/new
 mkdir -p mail/sent/tmp
 
-../smtp-dummy sent_message 
+$TEST_DIRECTORY/smtp-dummy sent_message 
 smtp_dummy_pid=$!
 test_emacs (setq message-send-mail-function 'message-smtpmail-send-it) (setq 
smtpmail-smtp-server 

[PATCH] test:Folder tags shouldn't match after removal of file in given folder

[Mark Anderson]

Test for bug.  Current stemming support for notmuch adds extra terms
to the DB which aren't removed when the file renames are detected.

When folder tags are added to a message, Xapian terms for both XFOLDER
and ZXFOLDER are generated.  When one of the filenames are
renamed/removed, only the XFOLDER tags are removed, leaving it possible
for a match on a folder: tag that was previously but is no longer a
match in the maildir.
---

I found this bug last week.  Essentially when the folder:spam tag is
added and puts the XFOLDERspam, it also inserts the ZXFOLDERspam.  Then
if the mail is removed from the folder, it neglects to remove
ZXFOLDERspam.

This was detected with my offlineimap usage with gmail, still haven't
polished my personal folder/label transition.  As I was looking into it,
I saw some unusual things flagged as spam, and investigated.

 test/notmuch-test|1 +
 test/search-folder-coherence |   48 ++
 2 files changed, 49 insertions(+), 0 deletions(-)
 create mode 100755 test/search-folder-coherence

diff --git a/test/notmuch-test b/test/notmuch-test
index fe85c6a..79e6267 100755
--- a/test/notmuch-test
+++ b/test/notmuch-test
@@ -41,6 +41,7 @@ TESTS=
   maildir-sync
   crypto
   symbol-hiding
+  search-folder-coherence
 
 TESTS=${NOTMUCH_TESTS:=$TESTS}
 
diff --git a/test/search-folder-coherence b/test/search-folder-coherence
new file mode 100755
index 000..cf3ba40
--- /dev/null
+++ b/test/search-folder-coherence
@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+test_description='folder tags removed and added through file renames remain 
consistent'
+. ./test-lib.sh
+
+test_begin_subtest No new messages
+output=$(NOTMUCH_NEW)
+test_expect_equal $output No new mail.
+
+
+test_begin_subtest Single new message
+generate_message
+file_x=$gen_msg_filename
+id_x=$gen_msg_id
+output=$(NOTMUCH_NEW)
+test_expect_equal $output Added 1 new message to the database.
+
+test_begin_subtest Add second folder for same message
+dir=$(dirname $file_x)
+mkdir $dir/spam
+cp $file_x $dir/spam
+output=$(NOTMUCH_NEW)
+test_expect_equal $output No new mail.
+
+
+test_begin_subtest Multiple files for same message
+cat EOF EXPECTED
+MAIL_DIR/msg-001
+MAIL_DIR/spam/msg-001
+EOF
+notmuch search --output=files id:$id_x | sed -e s,$MAIL_DIR,MAIL_DIR, OUTPUT
+test_expect_equal_file OUTPUT EXPECTED
+
+test_begin_subtest Test matches folder:spam
+output=$(notmuch search folder:spam)
+test_expect_equal $output thread:0001   2001-01-05 [1/1] 
Notmuch Test Suite; Test message #1 (inbox unread)
+
+sleep 1;
+
+test_begin_subtest Remove folder:spam copy of email
+rm $dir/spam/$(basename $file_x)
+output=$(NOTMUCH_NEW)
+test_expect_equal $output No new mail. Detected 1 file rename.
+
+test_begin_subtest No mails match the folder:spam search
+output=$(notmuch search folder:spam)
+test_expect_equal $output 
+
+test_done
-- 
1.7.4.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test:Folder tags shouldn't match after removal of file in given folder

[Mark Anderson]

On Mon, 27 Jun 2011 15:58:00 -0500, Austin Clements amdra...@mit.edu wrote:
 On Mon, Jun 27, 2011 at 1:12 PM, Mark Anderson ma.sk...@gmail.com wrote:
  +test_begin_subtest Test matches folder:spam
  +output=$(notmuch search folder:spam)
  +test_expect_equal $output thread:0001   2001-01-05 [1/1] 
  Notmuch Test Suite; Test message #1 (inbox unread)
  +
  +sleep 1;
 
 I assume this has to do with directory mtime's.  Is it sufficient to
 instead increment_mtime $dir/spam after the rm below?

I believe that would be sufficient.  I just had one of my runs fail to
even remove the filename, and I remember some earlier discussion along
these lines, but I just used the fastest way, I can certainly modify it
to use increment_mtime

 
  +
  +test_begin_subtest Remove folder:spam copy of email
  +rm $dir/spam/$(basename $file_x)
  +output=$(NOTMUCH_NEW)
  +test_expect_equal $output No new mail. Detected 1 file rename.
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
 

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] test:Improve test behaviors when --root is used

[Mark Anderson]

On Mon, 27 Jun 2011 15:50:47 -0500, Austin Clements amdra...@mit.edu wrote:
 This looks great (modulo one bug, below).  I've wanted to run the
 tests on tmpfs before, but was too lazy to actually fix the tests.
 
 Given how easy it is to accidentally use .., I wonder if there's a
 way to force people to use $TEST_DIRECTORY?

I was thinking of a test of the test-suite running in this mode, but I
didn't quite feel comfortable, since it feels a bit slow sometimes
already.

So an instruction to run 'make test', which would include a 'make test
OPTIONS=--root=/dev/shm/notmuch_test.$$' or some such if that was
acceptable.

I didn't think about it very hard, but I was intrigued by the idea.

-Mark

 On Mon, Jun 27, 2011 at 12:09 PM, Mark Anderson ma.sk...@gmail.com wrote:
  --- a/test/symbol-hiding
  +++ b/test/symbol-hiding
  @@ -12,13 +12,13 @@ test_description='exception symbol hiding'
   . ./test-lib.sh
 
   run_test(){
  -    result=$(LD_LIBRARY_PATH=../../lib ./symbol-test 21)
  +    result=$(LD_LIBRARY_PATH=$TEST_DIRECTORY/../../lib ./symbol-test 21)
 
 Did you mean $TEST_DIRECTORY/../lib?
 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
 

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] search --output=files: Output all filenames for each matching message

[Mark Anderson]

Messages in the database can have multiple files associated with a
single message-id, but until now only one filename for each message
has been reported by "notmuch search --output=files"

Signed-off-by: Mark Anderson 
---

Perhaps someone can offer a little help making the "separator" code
tighter, but this works.

 notmuch-search.c |   29 ++---
 1 files changed, 22 insertions(+), 7 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 616fe68..faccaf7 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -275,6 +275,7 @@ do_search_messages (const search_format_t *format,
 {
 notmuch_message_t *message;
 notmuch_messages_t *messages;
+notmuch_filenames_t *filenames;
 int first_message = 1;

 messages = notmuch_query_search_messages (query);
@@ -289,19 +290,33 @@ do_search_messages (const search_format_t *format,
 {
message = notmuch_messages_get (messages);

-   if (! first_message)
-   fputs (format->item_sep, stdout);
-
if (output == OUTPUT_FILES) {
-   format->item_id (message, "",
-notmuch_message_get_filename (message));
+   filenames = notmuch_message_get_filenames (message);
+
+   for (;
+notmuch_filenames_valid (filenames);
+notmuch_filenames_move_to_next (filenames))
+   {
+   if (! first_message)
+   fputs (format->item_sep, stdout);
+
+   format->item_id (message, "",
+notmuch_filenames_get (filenames));
+
+   first_message = 0;
+   }
+   
+   notmuch_filenames_destroy( filenames );
+
} else { /* output == OUTPUT_MESSAGES */
+   if (! first_message)
+   fputs (format->item_sep, stdout);
+
format->item_id (message, "id:",
 notmuch_message_get_message_id (message));
+   first_message = 0;
}

-   first_message = 0;
-
notmuch_message_destroy (message);
 }

-- 
1.7.4.1

[PATCH 1/2] test:Expect multiple filenames for message with multiple files

[Mark Anderson]

Update the test mail corpus to have two files with the same content to
expose the bug where a single message with multiple filenames only
reports a single filename.

Update expected results for search --output=files to match new
behavior for multiple files corresponding to a single message

Signed-off-by: Mark Anderson 
---

I just picked the smallest message and copied it to a new name.  This
 could certainly be done to a much larger degree.

 test/corpus/cur/51:2, |   12 
 test/search-output|2 ++
 2 files changed, 14 insertions(+), 0 deletions(-)
 create mode 100644 test/corpus/cur/51:2,

diff --git a/test/corpus/cur/51:2, b/test/corpus/cur/51:2,
new file mode 100644
index 000..f522f69
--- /dev/null
+++ b/test/corpus/cur/51:2,
@@ -0,0 +1,12 @@
+From: "Aron Griffis" 
+To: notmuch at notmuchmail.org
+Date: Tue, 17 Nov 2009 18:21:38 -0500
+Subject: [notmuch] archive
+Message-ID: <20091117232137.GA7669 at griffis1.net>
+
+Just subscribed, I'd like to catch up on the previous postings,
+but the archive link seems to be bogus?
+
+Thanks,
+Aron
+
diff --git a/test/search-output b/test/search-output
index 3c875cd..10291c3 100755
--- a/test/search-output
+++ b/test/search-output
@@ -207,6 +207,7 @@ MAIL_DIR/cur/22:2,
 MAIL_DIR/cur/21:2,
 MAIL_DIR/cur/19:2,
 MAIL_DIR/cur/18:2,
+MAIL_DIR/cur/51:2,
 MAIL_DIR/cur/20:2,
 MAIL_DIR/cur/17:2,
 MAIL_DIR/cur/16:2,
@@ -263,6 +264,7 @@ cat <EXPECTED
 "MAIL_DIR/cur/21:2,",
 "MAIL_DIR/cur/19:2,",
 "MAIL_DIR/cur/18:2,",
+"MAIL_DIR/cur/51:2,",
 "MAIL_DIR/cur/20:2,",
 "MAIL_DIR/cur/17:2,",
 "MAIL_DIR/cur/16:2,",
-- 
1.7.4.1

Debugging strangeness in To: field

[Mark Anderson]

On Wed, 06 Apr 2011 13:35:44 -0600, Mark Anderson  
wrote:
Hello All,

> It is rather painful that I can have a lot of recipients dropped
> silently by gmime.

Well, it's not this bad, I only lose the rest of the display name and
the true email address for the recipients where this matches.  Later
recipients are preserved.  That's better than I thought, but definitely
not good, since the same email list can have multiple display names
depending on the sender's preferences, and now I have no guarantee that
notmuch will have the true email address indexed.  Hopefully this poor
behavior is related to my exposure to Exchange and isn't contagious
without willful stupidity. :)

This behavior also breaks the idea that I can just copy and paste from
the To: field into a search, since some terms will be missing.

It looks like it would be better to stuff the entire string of the To:
field directly into Xapian.  GMime will give you a string output of what it
figured out from the message header, but that already has terms pruned
as shown below:

>From the raw file:

To: One Big Happy , 
dist.Happy Group ,
This Really Stinks ,
This.WillPrune ,
This Will Not Prune 

Trace output:

Email address list: One Big Happy , dist.Happy, This 
Really Stinks , This.WillPrune, This Will 
Not Prune 
Email address: One Big Happy 
Email address: dist.Happy
Email address: This Really Stinks 
Email address: This.WillPrune
Email address: This Will Not Prune 

Any suggestions for how to fix this?  Or is my mail broken irreparably?

-Mark

Debugging strangeness in To: field

[Mark Anderson]

On Wed, 6 Apr 2011 14:10:51 -0500, Aaron Williamson  wrote:
> Hi Mark,
> 
> On 04/06/2011 02:58 PM, Mark Anderson wrote:
> > Do you have any hints about how I could figure out why gmime doesn't
> > like this To: list?
> > 
> > To: One Big Happy , dist.Happy Group
> > 
> 
> I may be way off, but I wonder if it's seeing "dist.Happy" and confusing it 
> for
> an email address (or at least a domain) rather than the display name for the
> email address.  Maybe display names of the form xxx.yyy need quotes?

Hi Aaron,

H, if that's the case, then perhaps I need to process my headers to
"fix" Exchange email addresses with display names containing '.'

I'm fairly certain that Exchange "display names" for internal email
lists is the source, since instrumenting my notmuch, I see other
instances of the same craziness with incompletely indexed "To:" lists.

Arg, I'm reading RFC #822, and it seems pretty clear that originally
this would not be allowed.  '.' is a special character and wouldn't have
been allowed unquoted in the display name.

I could hope that some leeway has been added in more recent RFC's, but
with MSoft's traditional implement first, specs later methodology, it
seems unlikely.

What a bother.

It is rather painful that I can have a lot of recipients dropped
silently by gmime.

Thanks,
-Mark

> 
> Best,
> Aaron
> 

Debugging strangeness in To: field

[Mark Anderson]

Hello all,

Do you have any hints about how I could figure out why gmime doesn't
like this To: list?

To: One Big Happy , dist.Happy Group


I added printouts to lib/index.cc just so I could try to figure out what
was missing, and I see this:

Email address list: One Big Happy , dist.Happy
Email address: One Big Happy 
Email address: dist.Happy

Why did it do that?  I will try pulling a new gmime source to see if
this is a fixed bug, but I don't know the RFC's well enough to know if
there is any wrapping behavior that justifies gmime in considering the
email list complete.

Any hints are appreciated.

Thanks,
-Mark

Re: Debugging strangeness in To: field

[Mark Anderson]

On Wed, 6 Apr 2011 14:10:51 -0500, Aaron Williamson aa...@copiesofcopies.org 
wrote:
 Hi Mark,
 
 On 04/06/2011 02:58 PM, Mark Anderson wrote:
  Do you have any hints about how I could figure out why gmime doesn't
  like this To: list?
  
  To: One Big Happy onebigha...@amd.com, dist.Happy Group
  dist.happygr...@amd.com
 
 I may be way off, but I wonder if it's seeing dist.Happy and confusing it 
 for
 an email address (or at least a domain) rather than the display name for the
 email address.  Maybe display names of the form xxx.yyy need quotes?

Hi Aaron,

H, if that's the case, then perhaps I need to process my headers to
fix Exchange email addresses with display names containing '.'

I'm fairly certain that Exchange display names for internal email
lists is the source, since instrumenting my notmuch, I see other
instances of the same craziness with incompletely indexed To: lists.

Arg, I'm reading RFC #822, and it seems pretty clear that originally
this would not be allowed.  '.' is a special character and wouldn't have
been allowed unquoted in the display name.

I could hope that some leeway has been added in more recent RFC's, but
with MSoft's traditional implement first, specs later methodology, it
seems unlikely.

What a bother.

It is rather painful that I can have a lot of recipients dropped
silently by gmime.

Thanks,
-Mark

 
 Best,
 Aaron
 

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Debugging strangeness in To: field

[Mark Anderson]

On Wed, 06 Apr 2011 13:35:44 -0600, Mark Anderson markr.ander...@amd.com 
wrote:
Hello All,

 It is rather painful that I can have a lot of recipients dropped
 silently by gmime.

Well, it's not this bad, I only lose the rest of the display name and
the true email address for the recipients where this matches.  Later
recipients are preserved.  That's better than I thought, but definitely
not good, since the same email list can have multiple display names
depending on the sender's preferences, and now I have no guarantee that
notmuch will have the true email address indexed.  Hopefully this poor
behavior is related to my exposure to Exchange and isn't contagious
without willful stupidity. :)

This behavior also breaks the idea that I can just copy and paste from
the To: field into a search, since some terms will be missing.

It looks like it would be better to stuff the entire string of the To:
field directly into Xapian.  GMime will give you a string output of what it
figured out from the message header, but that already has terms pruned
as shown below:

From the raw file:

To: One Big Happy onebigha...@amd.com, 
dist.Happy Group dist.happygr...@amd.com,
This Really Stinks something_sme...@notfromhere.com,
This.WillPrune wrinkled_p...@plumfarm.com,
This Will Not Prune plump_pl...@plumsrus.com

Trace output:

Email address list: One Big Happy onebigha...@amd.com, dist.Happy, This 
Really Stinks something_sme...@notfromhere.com, This.WillPrune, This Will Not 
Prune plump_pl...@plumsrus.com
Email address: One Big Happy onebigha...@amd.com
Email address: dist.Happy
Email address: This Really Stinks something_sme...@notfromhere.com
Email address: This.WillPrune
Email address: This Will Not Prune plump_pl...@plumsrus.com

Any suggestions for how to fix this?  Or is my mail broken irreparably?

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Strange match to my query

[Mark Anderson]

On Tue, 1 Mar 2011 17:15:22 -0600, Jameson Rollins  wrote:
> On Tue, 1 Mar 2011 16:00:51 -0700, Mark Anderson  
> wrote:
>  
> > A simple rebuild when you go to bed can look like:
> 
> I think you're missing an important step:
> 
> notmuch dump >dump.txt
> mv $(notmuch config get database.path){,.bak}
> notmuch new
> notmuch restore dump.txt

True, that would be much better than my proposed flow.

-Mark

Strange match to my query

[Mark Anderson]

On Fri, 25 Feb 2011 15:29:05 -0600, Jameson Rollins  wrote:
> So I am in fact still seeing this bug, although I am ostensibly using a
> version that includes the patch to fix it (db70f3f0).  Does this fix
> require rebuilding the database?

Yes.

The termlist is constructed when the message is added to the database,
so the database must be reconstructed.

Newer messages will index email addresses so that they can't be matched
by overlapping term indexes.  However, the corpus of your database is
not going to change without manual intervention.

A simple rebuild when you go to bed can look like:

notmuch dump >dump.txt; notmuch new; notmuch restore dump.txt

-Mark

Re: Strange match to my query

[Mark Anderson]

On Tue, 1 Mar 2011 17:15:22 -0600, Jameson Rollins jroll...@finestructure.net 
wrote:
 On Tue, 1 Mar 2011 16:00:51 -0700, Mark Anderson markr.ander...@amd.com 
 wrote:
  
  A simple rebuild when you go to bed can look like:
 
 I think you're missing an important step:
 
 notmuch dump dump.txt
 mv $(notmuch config get database.path){,.bak}
 notmuch new
 notmuch restore dump.txt

True, that would be much better than my proposed flow.

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Strange match to my query

[Mark Anderson]

On Fri, 25 Feb 2011 12:19:21 -0600, Jameson Rollins  wrote:
> On Tue, 25 Jan 2011 16:29:22 -0700, Mark Anderson  
> wrote:
> > Apparently matching on email addresses doesn't work the way I hoped.
> > 
> > While debugging why my to:x at y.com search was matching far too many
> > entries, I whittled it down to this:
> > 
> > WORD1=hello
> > WORD2=goodbye
> > MSGID=junk$(date +%s)
> > TESTDIR=$(notmuch config get database.path)/.tmp/new
> > TESTMAIL=$TESTDIR/$MSGID:2,
> > 
> > mkdir -p $TESTDIR
> > 
> > echo Testcase for $WORD1@$WORD2, msgid: $MSGID at junk.com
> > 
> > echo "From: nobody at nobody.com
> > To: c@${WORD1}.com, K-R@${WORD2}.com
> > Date: Mon, 24 Jan 2011 23:41:34 -0600
> > Subject: Error
> > Message-ID: <$MSGID at junk.com>
> > 
> > Not empty body.=
> > 
> > " > $TESTMAIL
> > 
> > notmuch new
> > notmuch search --output=files to:$WORD1@$WORD2
> > notmuch search --output=files to:\"$WORD1@$WORD2\"
> > 
> > Why does that match, but this doesn't?
> > 
> > notmuch search --output=files to:\'$WORD1@$WORD2\'
> 
> Hey, guys.  Reopening an old thread here, found while trying to track
> down a similar problem.
> 
> I'm confused why any of these searches should return anything at all.
> "$WORD1@$WORD2" doesn't actually match either of the addresses in the
> test message, especially when quoted.  The expanded addresses should be:
> 
>   c at hello.com
>   K-R at goodbye.com
> 
> Why should
> 
>   hello at goodbye
> 
> match anything?  And in fact it doesn't for me if I recreate the same
> setup.  Am I missing something?

It shouldn't match anything, that's the value of finding this bug.

What happened is the term counter was reset for each email address, so
the term list for emails in "to:" looks something like this:

0 c  K
1 hello  R
2 comgoodbye
3com

So it matched a hello at 1 and a goodbye at 2.

I don't remember where the discussion on this went, but it was on the
list.

Perhaps you should search for it, it should take notmuch to
find... *duck*

-Mark

Strange match to my query

[Mark Anderson]

On Tue, 25 Jan 2011 23:59:50 -0600, Carl Worth  wrote:
> On Wed, 26 Jan 2011 12:19:17 +1000, Carl Worth  wrote:
> > And thanks, Mark for the bug report and the nice test case. I'll add
> > this to the test suite, and fix it. And that will give us yet one more
> > reason for all of us to rebuild our databases after the upcoming
> > release.
> 
> I've added a test case for this now, fixed the bug, and pushed out the
> new code.
> 
> Thanks again for the bug report.

That's great, apparently submitting the testcase was the best thing I
could do, because I didn't realize that I needed a 2-part name to align
the term lists, although I did start from one.  And now at least I know
that I can't construct the correct query without an updated notmuch.

It was very confusing trying to bend my head around the issue and tell
myself that I just didn't understand how notmuch worked at all on
searching through email addresses.

Glad to see such a quick response to my bug report.

> -Carl

Re: Strange match to my query

[Mark Anderson]

On Tue, 25 Jan 2011 23:59:50 -0600, Carl Worth cwo...@cworth.org wrote:
 On Wed, 26 Jan 2011 12:19:17 +1000, Carl Worth cwo...@cworth.org wrote:
  And thanks, Mark for the bug report and the nice test case. I'll add
  this to the test suite, and fix it. And that will give us yet one more
  reason for all of us to rebuild our databases after the upcoming
  release.
 
 I've added a test case for this now, fixed the bug, and pushed out the
 new code.
 
 Thanks again for the bug report.

That's great, apparently submitting the testcase was the best thing I
could do, because I didn't realize that I needed a 2-part name to align
the term lists, although I did start from one.  And now at least I know
that I can't construct the correct query without an updated notmuch.

It was very confusing trying to bend my head around the issue and tell
myself that I just didn't understand how notmuch worked at all on
searching through email addresses.

Glad to see such a quick response to my bug report.

 -Carl


___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Strange match to my query

[Mark Anderson]

Hi guys, What's up? ("Notmuch")

Apparently matching on email addresses doesn't work the way I hoped.

While debugging why my to:x at y.com search was matching far too many
entries, I whittled it down to this:

WORD1=hello
WORD2=goodbye
MSGID=junk$(date +%s)
TESTDIR=$(notmuch config get database.path)/.tmp/new
TESTMAIL=$TESTDIR/$MSGID:2,

mkdir -p $TESTDIR

echo Testcase for $WORD1@$WORD2, msgid: $MSGID at junk.com

echo "From: nobody at nobody.com
To: c@${WORD1}.com, K-R@${WORD2}.com
Date: Mon, 24 Jan 2011 23:41:34 -0600
Subject: Error
Message-ID: <$MSGID at junk.com>

Not empty body.=

" > $TESTMAIL

notmuch new
notmuch search --output=files to:$WORD1@$WORD2
notmuch search --output=files to:\"$WORD1@$WORD2\"

Why does that match, but this doesn't?

notmuch search --output=files to:\'$WORD1@$WORD2\'

Apparently single quotes are the only quote for Xapian's parser?

I guess this is a strong vote for the quick integration of the custom
parser with optimization passes that turn emails into phrases that can't
match across multiple emails.

This was just an egregious example of notmuch giving me notmuch of what
I wanted, or actually, far too much of what I didn't want.

Thanks,
-Mark

forwarded messages and threads

[Mark Anderson]

On Wed, 21 Jul 2010 14:57:42 -0500, Jameson Rollins  wrote:
> Hey, folks.  I was noticing that if I forward a message, the forwarded
> message and any responses to it don't show up as part of the thread of
> the original message.  It seems to me that this would be useful, to keep
> replies to forwarded messages as part of the original message thread.  I
> think this would be pretty simple to fix, my having
> notmuch-mua-forward-message just add the forwarded message-id to
> "References:" header when forwarding.

I have been wishing this worked, so I'd like it as an option, as to
whether it should ever be a default, that's an entirely different
question.

> What do folks think about this?  Is this suggestion wack, or is this
> something that commonly happens in other mailers but was just an
> oversight in notmuch?

Don't know about other mailers, if I'm not using a strongly
thread-oriented view, I'm not sure if it makes any difference.

-Mark

bug tracking

[Mark Anderson]

On Thu, 29 Apr 2010 12:48:38 -0500, Servilio Afre Puentes  wrote:
> On 28 April 2010 16:34, David Bremner  wrote:
> [...]
> > Meanwhile Jesse Rosenthal and I started chatting about, and Jesse
> > started implementing, some tools for grabbing remote collections of tags
> > and merging them into their own namespace. ?This may give us a
> > notmuch-oriented way of managing the mailing list a bit better as a kind
> > of bug tracker. I don't want to promise things on Jesse's behalf, but I
> > personally am holding off on further bug tracker experiments until I see
> > how this works out.
> 
> I have been playing in my head with the idea of using notmuch to track
> bugs, thinking of ways it could be done. Using tags and searches this
> should be fine, and even (if not right now, I am sure in the future we
> will) easily see what bugs have patches/pull requests proposed[1].
> 
> While reading your message, David, I think I've found and idea might
> enable this: enabling the dump command to accept a search term. With
> this in place, our bug database can be composed of the mail archive +
> a dump of the tags used for bug management.
> 
> There another wrinkle with this approach, but a solvable one I think:
> the current implementation of the restore command will wipe all local
> state for the related messages. This is really bad for people tracking
> individually bugs/features in their local notmuch databases, e.g.:
> using tags such as TODO, REVIEW, etc.
> 
> But with a way of non-destructively applying a set of tags to a
> messages we could have a distributed issue/feature/etc tracking system
> that is 100% notmuch-based and message driven (PR for the feature, we
> have to think forward!).
> 
> IMHO this will allow for totally awesome workflows.
> 
> And we should name it "notmuch issues" or "notmuch bugs".
> 
> Servilio

Wouldn't it be good to have a timestamp associated with the application
of a tag, especially if you're going to manage a bug workflow with tags?

You'll be going cross geography, so UTC sounds nice.

But what happens if a bug goes from OPEN->CLOSED->REOPENED->CLOSED->...,
can you track that state with a simple tag cloud?

How would you handle conflicts for the bug tracker?  Suppose two people
close the bug in different ways, and one fix goes through several state
switches before being committed to a globally visible repository.

Does this really work with distributed development?

Although a permutation of that question might be:

What does a distributed bug tracker look like?

You will have multiple bug DB's in different states, so do we default to
having a tie-breaker "master" DB designated by the community?

Distributed bug tracking - I'm certain it means different things to
different people, perhaps we ought to specify what it means with a bit
more clarity before jumping off and developing it. 

Nah, too much central control, just develop what you want, we'll bend it
and the conversation about what it means until they fit. ;}

Enjoy,
-Mark

[notmuch] Bulk message tagging

[Mark Anderson]

On Wed, 21 Apr 2010 18:02:59 -0500, Carl Worth  wrote:
> On Sat, 17 Apr 2010 20:32:27 +0200, Arian Kuschki  googlemail.com> wrote:
> > So one could query with sysconf and break things up into multiple
> > commands as needed.
> > 
> > Doesn't xargs do exactly this?
> 
> Almost.
> 
> The arguments being passed to the "notmuch tag" command in this case
> look like:
> 
>   notmuch tag -inbox thread:foo or thread:bar or ...
> 
> To break that up, we'd have to be careful to neither leave a trailing
> 'or' at the end of a command line nor to have an 'or' at the beginning
> of a command line.

Perhaps this hints at an opportunity to create a new operator, that you
can pass as part of the notmuch commandline.

Something like:
cat idlist.txt | xargs notmuch tag -inbox tag:inbox and oneof: 

Then the list of arguments can break anywhere it wants. It's not as
general as having notmuch take search terms from stdin or a file, but it
seems like a long list of ID's is going to be a common case.

Another problem with passing options to xarg is that any parentheses are
going to break easily.

Actually, looking at this random proposal, you can see the invented
operator has an implicit grouping semantic, which leads to all kinds of
confusion.

-Mark

RE: [notmuch] Bulk message tagging

[Mark Anderson]

On Wed, 21 Apr 2010 18:02:59 -0500, Carl Worth cwo...@cworth.org wrote:
 On Sat, 17 Apr 2010 20:32:27 +0200, Arian Kuschki 
 arian.kusc...@googlemail.com wrote:
  So one could query with sysconf and break things up into multiple
  commands as needed.
  
  Doesn't xargs do exactly this?
 
 Almost.
 
 The arguments being passed to the notmuch tag command in this case
 look like:
 
   notmuch tag -inbox thread:foo or thread:bar or ...
 
 To break that up, we'd have to be careful to neither leave a trailing
 'or' at the end of a command line nor to have an 'or' at the beginning
 of a command line.

Perhaps this hints at an opportunity to create a new operator, that you
can pass as part of the notmuch commandline.

Something like:
cat idlist.txt | xargs notmuch tag -inbox tag:inbox and oneof: 

Then the list of arguments can break anywhere it wants. It's not as
general as having notmuch take search terms from stdin or a file, but it
seems like a long list of ID's is going to be a common case.

Another problem with passing options to xarg is that any parentheses are
going to break easily.

Actually, looking at this random proposal, you can see the invented
operator has an implicit grouping semantic, which leads to all kinds of
confusion.

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] Fix bug, and clean up code duplication, in adding or removing tag by region.

[Mark Anderson]

On Wed, 14 Apr 2010 12:50:50 -0500, Jesse Rosenthal  
wrote:
> It occurs to me that the best way to do this would probably be to go to
> point-max, and then (forward-line -1) until we hit a thread-id. That way
> we wouldn't have to work all the way down long search indexes. I'll try
> to code that up for the next release, and then have
> notmuch-search-last-thread use it, as well as the region functions.

This sounds great, just be careful if this command is run before the
buffer has completed loading, as you could be in the middle of a search
instead of the end.

AFAIK, with the "asynchronous" buffer loading, there's no guarantee that
point-max is the end of the search until the other thread has exited.

Again, my lisp-fu is very poor, but just a concern I see.

-Mark

Re: [PATCH] Fix bug, and clean up code duplication, in adding or removing tag by region.

[Mark Anderson]

On Wed, 14 Apr 2010 12:50:50 -0500, Jesse Rosenthal jrosent...@jhu.edu wrote:
 It occurs to me that the best way to do this would probably be to go to
 point-max, and then (forward-line -1) until we hit a thread-id. That way
 we wouldn't have to work all the way down long search indexes. I'll try
 to code that up for the next release, and then have
 notmuch-search-last-thread use it, as well as the region functions.

This sounds great, just be careful if this command is run before the
buffer has completed loading, as you could be in the middle of a search
instead of the end.

AFAIK, with the asynchronous buffer loading, there's no guarantee that
point-max is the end of the search until the other thread has exited.

Again, my lisp-fu is very poor, but just a concern I see.

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Bulk message tagging

[Mark Anderson]

On Sat, 10 Apr 2010 08:56:48 -0500, Xavier Maillard  wrote:
> Hi,
> 
> On Tue, 6 Apr 2010 13:51:01 -0600, Mark Anderson  
> wrote:
> > 
> > I think that '*' is definitely an awesome command, but I wonder if we
> > shouldn't have another command for the notmuch-search buffer which means
> > 'tag all the threads that I can see in this buffer'.
> 
> This is exactly what my initial post asked for. '*' is not
> totally satisfying for me due to the "limitations" you
> exposed. Though It is a good and acceptable workaround for me.
> As said, I just have to pay attention to redo my search query
> before pressing the '*' key.

Another problem I have is that I often _don't_ want to refresh my
search.  Some of my mail processing, while not visible in the search
window, since we don't have a way to refresh tags yet, will remove some of
the current search results from matching the search query.

For example, I like to have notmuch folder definitions with "tag:unread"
in them.  For those search views, once I've looked at any of the mails,
they no longer match the query.  Sometimes I want to refresh the search
so that those mails are no longer visible, sometimed I want to apply an
action on all the visible messages which I've just processed.

When I visit a folder view, what intent do I have?  Am I returning to a
'moment-ago' processing view that was interrupted?  Or am I wanting to
do the search again on the current contents of the database?  I could
easily see mental models that match either way.

I think I read that Carl plans to update the tags in a search view at
some point, without removing threads automatically.

Perhaps there ought to be a way to colorize threads which are displayed
but no longer match the search criterion?

-Mark

Re: [notmuch] Bulk message tagging

[Mark Anderson]

On Sat, 10 Apr 2010 08:56:48 -0500, Xavier Maillard x...@gnu.org wrote:
 Hi,
 
 On Tue, 6 Apr 2010 13:51:01 -0600, Mark Anderson markr.ander...@amd.com 
 wrote:
  
  I think that '*' is definitely an awesome command, but I wonder if we
  shouldn't have another command for the notmuch-search buffer which means
  'tag all the threads that I can see in this buffer'.
 
 This is exactly what my initial post asked for. '*' is not
 totally satisfying for me due to the limitations you
 exposed. Though It is a good and acceptable workaround for me.
 As said, I just have to pay attention to redo my search query
 before pressing the '*' key.

Another problem I have is that I often _don't_ want to refresh my
search.  Some of my mail processing, while not visible in the search
window, since we don't have a way to refresh tags yet, will remove some of
the current search results from matching the search query.

For example, I like to have notmuch folder definitions with tag:unread
in them.  For those search views, once I've looked at any of the mails,
they no longer match the query.  Sometimes I want to refresh the search
so that those mails are no longer visible, sometimed I want to apply an
action on all the visible messages which I've just processed.

When I visit a folder view, what intent do I have?  Am I returning to a
'moment-ago' processing view that was interrupted?  Or am I wanting to
do the search again on the current contents of the database?  I could
easily see mental models that match either way.

I think I read that Carl plans to update the tags in a search view at
some point, without removing threads automatically.

Perhaps there ought to be a way to colorize threads which are displayed
but no longer match the search criterion?

-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Initial attempt at a "merge window" for notmuch

[Mark Anderson]

On Fri, 9 Apr 2010 11:29:20 -0500, Carl Worth  wrote:
> On Fri, 09 Apr 2010 09:23:27 -0700, Carl Worth  wrote:
> >  For my merge window, I also want something that can't be obtained
> >  today. I want to see all threads that contain at least one message
> >  that matches my date range and at least one message that doesn't
> >  have the "merged" or "postponed" tag.
> ...
> >  I'm not sure how to best provide the ability to express the result
> >  I want here.
> 
> Of course, it's the same set-theoretic operation I described earlier. I
> want the set of threads having messages matching:
> 
>   tag:notmuch and 
> 
> intersected with the set of threads having messages matching:
> 
>   tag:notmuch and not ("merged" or "postponed")
> 
> So I've got uses cases for set-difference and intersection already. Now
> we just need some search syntax to express that.
> 

Can we just start grouping with a set:( ) or { } on the command line?
I'm guessing the parentheses are slightly easier.

   set:( tag:notmuch and  ) 
 isect 
   set:( tag:notmuch and not (tag:merged or tag:postponed) )

Isn't that close to what you're asking for?

It seems easy enough, and making the set:( explicit keeps you from
inventing a new quoting syntax if you tried to say 

  set:"tag:notmuch and " 

then users get to play shell-quote-mambo to actually use this outside of
a client if they want (or think they want) quotes used in their search.

You can reuse the "and" and "or" term for set math, unless you are
averse to overloading, then isect, union are explicit enough about the
terms that you could infer the "set:" term for searches:

tag:notmuch and  isect not ( tag:postponed or tag:merged ) 

But at the cost of introducing a new order of ops hierarchy.  I'm not
sure if you want to go there either.

Just thinking about completeness, I wonder if there are some searches
for threads that aren't currently available.

You could introduce a search modifier that would allow you to treat a
tag on a message in a thread as a tag on the entire thread, so that if
you have tag:mute on on message and tag:merged on another message in the
same thread, currently that does match:

  tag:merged and not tag:mute

But maybe you wanted only the THREADS instead of THREAD CONTAINS
searching.

I guess we're hampered by the fact that a thread object isn't a separate
object from the messages which comprise it's conversation, so we can't
look at the tags on a thread separately from the tags on messages in the
thread.  And maybe we don't want to.

Actually, this is where we differ slightly from gmail.  They have tags
on threads, and unread tags on messages.

I don't know that there's value in distinguishing between tags on a
thread and tags on a message in a thread when there isn't an object in
the mailstore that actually corresponds to a thread.

Hmmm, this isn't clear enough, so I'll just stop the rambling, but let
it stand.

-Mark

P.S. Sometimes I write a long note and don't get to the last sentence
for an hour or two.  I can be funny that way.

[PATCH] Have notmuch count default to showing the total.

[Mark Anderson]

On Fri, 9 Apr 2010 14:28:32 -0500, Anthony Towns  wrote:
> [0] Not much, afaics! [1]
> [1] Man, what are the chances that will ever get old? [0]

Thanks AJ, I like it!

-Mark

[notmuch] Bulk message tagging

[Mark Anderson]

On Mon, 5 Apr 2010 01:15:39 -0500, Xavier Maillard  wrote:
> On Sun, 04 Apr 2010 07:38:03 -0400, Jesse Rosenthal  
> wrote:
> > On Sun, 04 Apr 2010 06:37:53 +0200, Xavier Maillard  wrote:
> > > Is there an easy way to mark a whole bunch of message (restricted
> > > in a region, result of a search, ...) ?
> > 
> > In addition to the "*" command that was mentioned, there is a patch I
> > wrote to tag messages in search view by region in emacs. You can find it
> 
> Sounds interesting. * is good when your search criteria is
> perfect but you proposed patch is a good companion for all the
> rest.
> 
> Sadly, git is not really something I know wll enough to play with
> all this stuff :(
> 
> Xavier

Also, you will want to be careful with '*' in emacs.

It runs notmuch tag + ,
but the notmuch-search buffer can be woefully out of date.

I like to go through and add tags, archive, etc, lots of email.  Then I
'q' back to the search buffer, and I may realize that I want to tag
everything I can see.  '*' isn't really that command, it applies a tag
to the results of a search that you are looking at.  But it will use a
fresh version of the search, not the version in the buffer, which could
be minutes, hours, days or weeks old, depending on your Emacs buffer
management habits (or lack of them).

I think that '*' is definitely an awesome command, but I wonder if we
shouldn't have another command for the notmuch-search buffer which means
'tag all the threads that I can see in this buffer'.

If you have a crontab running 'notmuch new', you could end up tagging a
lot of things you would rather have a chance to review first.  I often
use '*' to tag:deleted a large number of mails once I have a query I
like, so deleting things sight unseen is an unpleasant thought.

I know that there is an emacs-region patch, which could probably be
extended to implement the behavior I'm talking about, but I have a hard
enough time adding keys to the keymaps, and alas, the time thing...

Maybe someone else will decide it's important enough to do, or has
already done it.

Enjoy,
-Mark

> ___
> notmuch mailing list
> notmuch at notmuchmail.org
> http://notmuchmail.org/mailman/listinfo/notmuch
> 

Re: [notmuch] Bulk message tagging

[Mark Anderson]

On Mon, 5 Apr 2010 01:15:39 -0500, Xavier Maillard x...@gnu.org wrote:
 On Sun, 04 Apr 2010 07:38:03 -0400, Jesse Rosenthal jrosent...@jhu.edu 
 wrote:
  On Sun, 04 Apr 2010 06:37:53 +0200, Xavier Maillard x...@gnu.org wrote:
   Is there an easy way to mark a whole bunch of message (restricted
   in a region, result of a search, ...) ?
  
  In addition to the * command that was mentioned, there is a patch I
  wrote to tag messages in search view by region in emacs. You can find it
 
 Sounds interesting. * is good when your search criteria is
 perfect but you proposed patch is a good companion for all the
 rest.
 
 Sadly, git is not really something I know wll enough to play with
 all this stuff :(
 
 Xavier

Also, you will want to be careful with '*' in emacs.

It runs notmuch tag +tag search terms from buffer,
but the notmuch-search buffer can be woefully out of date.

I like to go through and add tags, archive, etc, lots of email.  Then I
'q' back to the search buffer, and I may realize that I want to tag
everything I can see.  '*' isn't really that command, it applies a tag
to the results of a search that you are looking at.  But it will use a
fresh version of the search, not the version in the buffer, which could
be minutes, hours, days or weeks old, depending on your Emacs buffer
management habits (or lack of them).

I think that '*' is definitely an awesome command, but I wonder if we
shouldn't have another command for the notmuch-search buffer which means
'tag all the threads that I can see in this buffer'.

If you have a crontab running 'notmuch new', you could end up tagging a
lot of things you would rather have a chance to review first.  I often
use '*' to tag:deleted a large number of mails once I have a query I
like, so deleting things sight unseen is an unpleasant thought.

I know that there is an emacs-region patch, which could probably be
extended to implement the behavior I'm talking about, but I have a hard
enough time adding keys to the keymaps, and alas, the time thing...

Maybe someone else will decide it's important enough to do, or has
already done it.

Enjoy,
-Mark

 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
 

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Any idea why my emacs doesn't want to accept my notmuch-folder keymap changes?

[Mark Anderson]

On Wed, 10 Mar 2010 16:01:28 -0600, James Vasile  
wrote:
> You have a typo: mot-much-folder-mode-map should be
> notmuch-folder-mode-map.  (s/motmuch/notmuch/)

Wow, how long can you stare at something and not see the obvious?

Thanks,
-Mark

Re: [notmuch] Any idea why my emacs doesn't want to accept my notmuch-folder keymap changes?

[Mark Anderson]

On Wed, 10 Mar 2010 16:01:28 -0600, James Vasile ja...@hackervisions.org 
wrote:
 You have a typo: mot-much-folder-mode-map should be
 notmuch-folder-mode-map.  (s/motmuch/notmuch/)

Wow, how long can you stare at something and not see the obvious?

Thanks,
-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Thoughts on not seeing messages I can't deal with (yet, or now, or here...)

[Mark Anderson]

Carl,

There's a post from a while ago about using GTD on Remember The Milk.

Remember the Milk as described here is mainly a todo manager, but the
saved search (as a list of todo tasks that match the criterion) is
what's being utilized here that makes me think so much of notmuch.

This seems to capture some of the things that you want to see, since you
are trying to manage action items which happen to live in mail bodies.

http://blog.rememberthemilk.com/2008/05/guest-post-advanced-gtd-with-remember-the-milk/

apparently another post has expired on the topic, with some focus on
contexts:

http://web.archive.org/web/20080331121051/http://www.geektronica.com/2007-01-15-gtd-with-rtm-getting-things-done-with-remember-the-milk

The main tool in use here is a viewport on the collection of things that
might need to be reviewed. RTM tabs (which are saved searches) which you
have designed to be the contexts you might be executing in, in this
case.

I know this is related pretty heavily to what you are looking for, but
how to bring this about specifically, that's going to be up to you.

Enjoy,
-Mark

On Fri, 26 Feb 2010 14:00:06 -0600, Carl Worth  wrote:
> [This mail started as some off-topic rambling in my reply to the
> notmuch-reply script. So that's why it starts on one topic and ends
> somewhere else entirely.]
> 
> I'm currently avoiding any locking failures with notmuch commands by
> running "notmuch new" manually, (rather than from a cron job). And it
> occurs to me that running "notmuch new" manually has a certain
> benefit---it allows me to bring in a chunk of mail, and then process all
> of that (either by replying, or setting aside to a particular project or
> "todo" tag, etc.) without getting distracted by other mail coming in.
> 
> It almost makes me want to just run "notmuch new" something like once
> per day.
> 
> But then, of course, there are times where there are important messages
> I need to get to quickly, or fast-moving threads where I need to reply
> several times throughout the day.
> 
> But I do have particular mailing lists that I don't want to read more
> than once a day, for instance. If I wait until I have about a days worth
> of mail in those lists, then I can deal with them very efficiently,
> (just scan all the subjects, read a thread or two and the "* -inbox" the
> rest). This gets a lot less efficient if I have to deal with those lists
> on a regular basis throughout the day, (particularly before we have
> support for "muted" threads).
> 
> So I'd like to be able to deal with important messages as they come in,
> but postpone bulk stuff.
> 
> But I also notice that there's a bad tendency I have if I try to do this
> postponing manually. Mail starts collecting in one of these bulk-list
> folders, and I start training myself to ignore the folder, then it gets
> huge, (which discourages me even more from looking at it, etc.). [*]
> 
> So I want better support from notmuch to tell me what things deserve my
> attention, so that I can avoid training myself to ignore things. I think
> what I want here is the ability to set a threshold on a particular
> folder based on number of messages or date. Something like: "Don't show
> me this folder at all until it has more than X messages or until the
> oldest message is at least Y hours old".
> 
> [*] A similar, (but more dangerous), problem occurs with manually
> postponing things into a "todo" folder. If I just added a bunch of
> things to my todo folder then I have a tendency to think, "I don't need
> to look at that---that's got a bunch of things I just decided to
> postpone". But then I forget that I put some things in there previously
> that really need my attention now.
> 
> I know that what I really want instead of "todo" is a way to express the
> reason I'm postponing a message. There's probably some resource I'm
> missing that I need before I can deal with it. Perhaps that's:
> 
>   * I can't decide on this until I'm with co-workers and can talk about
> this.
> 
>   * I can't resolve this until I'm at the office with the right hardware
> to test.
> 
>   * I need to remember to do something with this when I'm at home.
> 
>   * I need a nice block of "discretionary time" to be able to give this
> topic the attention I want to.
> 
>   * I need to look at this message again on this Saturday.
> 
> So what I really want to do is to tag things based on those criteria,
> ("office", "magic-hardware", "home", "discretionary", "2010-02-27"),
> which I can at least do now.
> 
> But what I'm currently missing is a way for the folders based on these
> tags to only appear at the right times, (when the resource is
> available).
> 
> When the messages appear at the wrong times, it just trains me to ignore
> things, and that's when I start forgetting things and let things fall
> through.
> 
> No concrete proposal here, but just some musings on the kinds of issues
> I'd really like to be exploring with notmuch, (once we can get past all
> 

Re: [notmuch] Thoughts on not seeing messages I can't deal with (yet, or now, or here...)

[Mark Anderson]

Carl,

There's a post from a while ago about using GTD on Remember The Milk.

Remember the Milk as described here is mainly a todo manager, but the
saved search (as a list of todo tasks that match the criterion) is
what's being utilized here that makes me think so much of notmuch.

This seems to capture some of the things that you want to see, since you
are trying to manage action items which happen to live in mail bodies.

http://blog.rememberthemilk.com/2008/05/guest-post-advanced-gtd-with-remember-the-milk/

apparently another post has expired on the topic, with some focus on
contexts:

http://web.archive.org/web/20080331121051/http://www.geektronica.com/2007-01-15-gtd-with-rtm-getting-things-done-with-remember-the-milk

The main tool in use here is a viewport on the collection of things that
might need to be reviewed. RTM tabs (which are saved searches) which you
have designed to be the contexts you might be executing in, in this
case.

I know this is related pretty heavily to what you are looking for, but
how to bring this about specifically, that's going to be up to you.

Enjoy,
-Mark

On Fri, 26 Feb 2010 14:00:06 -0600, Carl Worth cwo...@cworth.org wrote:
 [This mail started as some off-topic rambling in my reply to the
 notmuch-reply script. So that's why it starts on one topic and ends
 somewhere else entirely.]
 
 I'm currently avoiding any locking failures with notmuch commands by
 running notmuch new manually, (rather than from a cron job). And it
 occurs to me that running notmuch new manually has a certain
 benefit---it allows me to bring in a chunk of mail, and then process all
 of that (either by replying, or setting aside to a particular project or
 todo tag, etc.) without getting distracted by other mail coming in.
 
 It almost makes me want to just run notmuch new something like once
 per day.
 
 But then, of course, there are times where there are important messages
 I need to get to quickly, or fast-moving threads where I need to reply
 several times throughout the day.
 
 But I do have particular mailing lists that I don't want to read more
 than once a day, for instance. If I wait until I have about a days worth
 of mail in those lists, then I can deal with them very efficiently,
 (just scan all the subjects, read a thread or two and the * -inbox the
 rest). This gets a lot less efficient if I have to deal with those lists
 on a regular basis throughout the day, (particularly before we have
 support for muted threads).
 
 So I'd like to be able to deal with important messages as they come in,
 but postpone bulk stuff.
 
 But I also notice that there's a bad tendency I have if I try to do this
 postponing manually. Mail starts collecting in one of these bulk-list
 folders, and I start training myself to ignore the folder, then it gets
 huge, (which discourages me even more from looking at it, etc.). [*]
 
 So I want better support from notmuch to tell me what things deserve my
 attention, so that I can avoid training myself to ignore things. I think
 what I want here is the ability to set a threshold on a particular
 folder based on number of messages or date. Something like: Don't show
 me this folder at all until it has more than X messages or until the
 oldest message is at least Y hours old.
 
 [*] A similar, (but more dangerous), problem occurs with manually
 postponing things into a todo folder. If I just added a bunch of
 things to my todo folder then I have a tendency to think, I don't need
 to look at that---that's got a bunch of things I just decided to
 postpone. But then I forget that I put some things in there previously
 that really need my attention now.
 
 I know that what I really want instead of todo is a way to express the
 reason I'm postponing a message. There's probably some resource I'm
 missing that I need before I can deal with it. Perhaps that's:
 
   * I can't decide on this until I'm with co-workers and can talk about
 this.
 
   * I can't resolve this until I'm at the office with the right hardware
 to test.
 
   * I need to remember to do something with this when I'm at home.
 
   * I need a nice block of discretionary time to be able to give this
 topic the attention I want to.
 
   * I need to look at this message again on this Saturday.
 
 So what I really want to do is to tag things based on those criteria,
 (office, magic-hardware, home, discretionary, 2010-02-27),
 which I can at least do now.
 
 But what I'm currently missing is a way for the folders based on these
 tags to only appear at the right times, (when the resource is
 available).
 
 When the messages appear at the wrong times, it just trains me to ignore
 things, and that's when I start forgetting things and let things fall
 through.
 
 No concrete proposal here, but just some musings on the kinds of issues
 I'd really like to be exploring with notmuch, (once we can get past all
 these little things like maildir flags, keybindings, failed HTML
 rendering, missing FCC support, 

[notmuch] Mail in git

[Mark Anderson]

On Wed, 17 Feb 2010 10:03:36 -0500, Ben Gamari  wrote:
> > notmuch would then only search and provide the hash ID(s); tags
> > would be a function of storage.
> > 
> > Is it possible to find out all trees that reference a given object
> > with Git in constant or sub-linear time?
> > 
> I don't believe so. I think this is one of the reasons why git gc is so
> expensive.

But if we have notmuch as a cache of the tags, then don't we already
know the tree objects that need updating?  Yes, we would probably need
some consistency checks for when things don't work as planned, but in
the common case we ought to always know.

Perhaps I'm misunderstanding these tree objects, and you're suggesting
that we don't even tell notmuch about them.

-Mark

Just poking my nose where it don't belong, since 1984.

Re: [notmuch] Mail in git

[Mark Anderson]

On Wed, 17 Feb 2010 10:03:36 -0500, Ben Gamari bgam...@gmail.com wrote:
  notmuch would then only search and provide the hash ID(s); tags
  would be a function of storage.
  
  Is it possible to find out all trees that reference a given object
  with Git in constant or sub-linear time?
  
 I don't believe so. I think this is one of the reasons why git gc is so
 expensive.

But if we have notmuch as a cache of the tags, then don't we already
know the tree objects that need updating?  Yes, we would probably need
some consistency checks for when things don't work as planned, but in
the common case we ought to always know.

Perhaps I'm misunderstanding these tree objects, and you're suggesting
that we don't even tell notmuch about them.

-Mark

Just poking my nose where it don't belong, since 1984.

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Threading

[Mark Anderson]

On Tue, 15 Dec 2009 16:54:20 +0100, Marten Veldthuis  
wrote:
> On Thu, 10 Dec 2009 13:30:13 -0800, Carl Worth  wrote:
> > But I still have a hard time justifying user operations to manipulate
> > threading. The whole point of threading is to make it faster to process
> > and read messages. But manual operations like joining and splitting
> > threads seem like the user just doing more work, and that *after* having
> > read the messages. So that seems mostly backwards to me.
> 
> By the way, Outlook & Exchange suck (or at least some versions do), and
> don't seem to generate In-Reply-To and References: headers. Just got a
> mail which prompted me to write this mail. I'd really like to be able to
> join messages in a case like this.

It's actually worse than that.  I was looking into why some of my
threads weren't coalescing.  Some of it seems to be a very difficult bug
DB that doesn't use identical Message-ID's to refer to the parent bug
mail.  I don't know how that works at all.  Sometimes it uses the same
Message-ID, but sometimes it changes a number in the ID.

However, this isn't the worst news, because I work with a lot of
Exchange users, and I noticed that their mail was also refusing to
thread.

I was looking at the message bodies, and they led me to these links
about mail processing.

The problem identified:
http://blog.postmaster.gr/2007/12/11/trying-to-make-use-of-outlooks-thread-index-header/

How to read it, or how Exchange goes its own way:
http://blog.postmaster.gr/2007/12/23/more-fun-with-message-threading/

With a fairly loose understanding of how notmuch detects threads, and
how much information it places in the Xapian database (only the
msg-id?), I can't suggest much of the how.

But I would like to propose that we consider handling the Exchange
non-standard threading method as well as the RFC822 threading in the
headers.

Reactions?

-Mark

[notmuch] Rather simple optimization for notmuch tag

[Mark Anderson]

On Wed, 23 Dec 2009 03:45:14 +, Olly Betts  wrote:
> Carl Worth writes:
> > On Fri, 18 Dec 2009 00:49:00 -0700, Mark Anderson wrote:
> > > I was updating my poll script that tags messages, and a common idiom is
> > > to put
> > >  tag +mytag  and not tag:mytag
> > > 
> > > I don't know anything about efficiency, but for the simple single-tag
> > > case, couldn't we imply the "and not tag:mytag" from the +mytag action
> > > list for the tag command?
> > 
> > On one level, it really shouldn't be a performance issue to tag messages
> > that already have a particular tag. (And in fact, the recently proposed
> > patches to fix Xapian defect 250 even address this I think.)
> 
> Applying a filter up-front like this is likely to still help I think as it
> avoids Xapian having to reverse-engineer this information internally.

That's good to hear.

> Actually, you could do this with multiple tags - you just need to build
> a filter for documents which might be affected.
> 
> So if you're adding tags a1 and a2, you want:  AND_NOT (a1 AND a2)
> since documents which already have tags a1 and a2 can be ignored.
> 
> If you're removing d1 and d2, then the filter is:  AND (d1 OR d2)
> since documents have to be tagged d1 or d2 in order for the removal to
> do anything.
> 
> Handling a combination of removals and additions is trickier, but probably
> possible, although the more tags you are dealing with, the less profitable
> the filtering is likely to be (as the filter is likely to cull fewer
> documents yet be more expensive to evaluate).

But the transform is pretty simple, I think that any combination of
additions and removals could be transformed according to the following
formula.

notmuch tag +a1 +a2 +a3 -d1 -d2 -d3 

would transform to something like:

 and ( not(a1) or not(a2) or not(a3) or d1 or d2 or d3) 

There are certainly may be much more optimal ways to do it depending on
the specific corpus of the database, considering if the tags a1 and a2
and a3 are usually added as one tag, or if the addition is done
individually, because if I know that a3 implies a1 and a2, the first 3
terms could be combined to not(a1 and a2 and a3), or I could just
exclude a3 tagged messages for nearly the same effect, with expected
performance improvements.

Unfortunately this requires that I know more about how the tags are used
than I ever want notmuch to deal with.  Perhaps a follow-on or parallel
project with less emphasis on minimalism.


This looks pretty good to me.  Easy to implement and not likely to break
things.  I've been wondering about whether there should be a repository
of mail added to the notmuch git so that we can start testing these
kinds of features on a consistent body of mail.

I doubt that I'll be the one to write this, since I don't have any time
set aside for real coding, but if it takes long enough, I'll probably
pick this one up eventually.

-Mark

[notmuch] [Orgmode] Notmuch: An emacs interface for fast global search and tagging of email

[Mark Anderson]

Excerpts from Jed Brown's message of Thu Dec 10 05:26:25 -0700 2009:
> On Wed, 09 Dec 2009 12:00:21 -0800, Carl Worth  wrote:
> > 1. Rewriting the code to not use apply-partially
> 
> 1b. Use apply-partially
> 
> (defun apply-partially (fun  args)
>   "Return a function that is a partial application of FUN to ARGS.
> ARGS is a list of the first N arguments to pass to FUN.
> The result is a new function which does the same as FUN, except that
> the first N arguments are fixed at the values with which this function
> was called."
>   (lexical-let ((fun fun) (args1 args))
> (lambda ( args2) (apply fun (append args1 args2)

Thanks, that is nice to have. Maybe after about 13 years of Emacs use I'll try 
to learn more lisp than Google-copy-paste.

Unfortunately for my specific case, that still didn't let the '?' key work.

if: Symbol's function definition is void: mouse-event-p

I seem to have lots of issues in emacs 22.2, but my Elisp is weak enough that I 
don't know how this ought to be handled.

There is the possibility of requiring 23.1 for notmuch.el, that may be easier, 
then I'll just stop trying with 22.2

Thanks again,
-Mark

Re: [notmuch] [Orgmode] Notmuch: An emacs interface for fast global search and tagging of email

[Mark Anderson]

Excerpts from Jed Brown's message of Thu Dec 10 05:26:25 -0700 2009:
 On Wed, 09 Dec 2009 12:00:21 -0800, Carl Worth cwo...@cworth.org wrote:
  1. Rewriting the code to not use apply-partially
 
 1b. Use apply-partially
 
 (defun apply-partially (fun rest args)
   Return a function that is a partial application of FUN to ARGS.
 ARGS is a list of the first N arguments to pass to FUN.
 The result is a new function which does the same as FUN, except that
 the first N arguments are fixed at the values with which this function
 was called.
   (lexical-let ((fun fun) (args1 args))
 (lambda (rest args2) (apply fun (append args1 args2)

Thanks, that is nice to have. Maybe after about 13 years of Emacs use I'll try 
to learn more lisp than Google-copy-paste.

Unfortunately for my specific case, that still didn't let the '?' key work.

if: Symbol's function definition is void: mouse-event-p

I seem to have lots of issues in emacs 22.2, but my Elisp is weak enough that I 
don't know how this ought to be handled.

There is the possibility of requiring 23.1 for notmuch.el, that may be easier, 
then I'll just stop trying with 22.2

Thanks again,
-Mark

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Threading

[Mark Anderson]

I was wondering if there's a way in notmuch to group un-associated threads into 
a single thread.

I have a bug tracking system that doesn't give me emails that thread naturally 
in notmuch.

I wouldn't mind writing a filter that could help identify a thread id which 
should apply to a message, and suggest that to notmuch.

Is there a method for this existing in notmuch yet?


Mark Anderson

[notmuch] [PATCH] Added regress option to tags iterator

[Mark Anderson]

Excerpts from Carl Worth's message of Wed Dec 09 13:08:43 -0700 2009:
> On Wed, 9 Dec 2009 14:24:46 +0100, Ruben Pollan  
> wrote:
> > Do you like to call them regress? Should I change that?
> 
> I don't love the name, (since it's so close to the word "regression"
> which has a totally different meaning in software context). But I also
> don't have an immediate suggestion for an improved name yet either.
> 
> > What about the functions notmuch_*_is_first? Is kind of reversed logic than
> > notmuch_*_has_more, the last are true when is not reach the limit but the
> > first ones are true when the limit is reached. But I think it make sense 
> > like
> > that.
> 
> I'd like a more symmetric API here. Anyone have a favorite set of names
> for iterating a list in two directions?

I like vocabulary games:

fwd/bck
forward/reverse
next/prev
advance/retreat
inc/dec
iter_fwd/iter_back
earlier/later
younger/older

I think that changing has_more is going to be a requirement to come up with a 
consistent set of names.

> 
> -Carl

Re: [notmuch] [PATCH] Added regress option to tags iterator

[Mark Anderson]

Excerpts from Carl Worth's message of Wed Dec 09 13:08:43 -0700 2009:
 On Wed, 9 Dec 2009 14:24:46 +0100, Ruben Pollan mes...@sindominio.net wrote:
  Do you like to call them regress? Should I change that?
 
 I don't love the name, (since it's so close to the word regression
 which has a totally different meaning in software context). But I also
 don't have an immediate suggestion for an improved name yet either.
 
  What about the functions notmuch_*_is_first? Is kind of reversed logic than
  notmuch_*_has_more, the last are true when is not reach the limit but the
  first ones are true when the limit is reached. But I think it make sense 
  like
  that.
 
 I'd like a more symmetric API here. Anyone have a favorite set of names
 for iterating a list in two directions?

I like vocabulary games:

fwd/bck
forward/reverse
next/prev
advance/retreat
inc/dec
iter_fwd/iter_back
earlier/later
younger/older

I think that changing has_more is going to be a requirement to come up with a 
consistent set of names.

 
 -Carl

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Threading

[Mark Anderson]

I was wondering if there's a way in notmuch to group un-associated threads into 
a single thread.

I have a bug tracking system that doesn't give me emails that thread naturally 
in notmuch.

I wouldn't mind writing a filter that could help identify a thread id which 
should apply to a message, and suggest that to notmuch.

Is there a method for this existing in notmuch yet?


Mark Anderson

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[notmuch] Hi, I'm Mark, and I'm addicted to effective mail handling

[Mark Anderson]

I've been a good friend of Carl's since college, and was following his interest 
in 'sup'.

I have transitioned to 'sup', and I'm ready to move on.

I just want a text email system that looks a lot like GMail, but isn't owned 
elsewhere. I want to be able to control the data.

I just recenty found a card game version of the old RPG Paranoia, I suppose I 
may have caught something from Friend Computer. (... No, Friend Computer, 
that's not treason, I'm trying to understand the Traitors so that I can capture 
or eliminate them. ) :)

The main issue I've had with getting notmuch working is that I've been using 
difficult distros.

Cygwin is one, although there must be some way to get it working properly. I'll 
be playing glib hokey-pokey if I can figure it out.

I also have an Ubuntu machine at home, but it is sitting on an LTS release so 
that I don't have to mess with my MythTV setup regularly, this makes some 
packages more difficult to find.

Anyway, more power to the mailer!!

-Mark