notmuch.el question: reading all messages in thread

[Matt Armstrong]

Sometimes a notmuch query matches only a subset of messages in a thread.
When this happens only that subset of messages will be "open".  Many
notmuch commands operate on the "open" messages only.  For example: SPC,
'n', 'p'.

Often this is what I want.  It works well when I'm looking for a
specific piece of information, or when I am interested in reading only
the unread messages in a long thread that I have seen previously.

Often this is not what I want.  I'm interested in seeing all messages
when I am looking for entire conversations where a topic is being
discussed.  E.g. when I want to re-read a long forgotten thread, or one
that surfaced through specific search terms that appear only in a few of
the messages in the thread.

For example, I subscribe to some lists but don't read all messages in a
timely manner, if I ever read them at all.  A search might surface
threads from those lists before I read them with my usual "tag:inbox
tag:somelist" query.  I typically find these threads with a query
"tag:inbox some other search term".

In the second case I can achieve what I want with queries like:

  thread:"{tag:inbox some other search term}"

But this is a burden to type and I often realize I want this after the
usual query has produced the search results, and then pressed 'Z' to get
a tree view, and realized that most of the messages are "closed."

I looked for a way to easily re-query a tree view buffer such that all
messages in all threads shown are "open" but did not find it.  Does this
exist?

I suppose I'm looking for the opposite of `notmuch-tree-filter'.  Maybe
`notmuch-tree-widen-to-thread' that produces a new notmuch tree widened
to the entire thread of the current message.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

notmuch.el and replying to multiple emails at once

[Matt Armstrong]

A feature I miss from Gnus is being able to reply to multiple emails at
once.  Has anyone else found to be a missing feature?  I wonder if it
would be easy to add to notmuch's Emacs MUA.

Background: In Gnus' equivalent to notmuch.el's tree mode it is possible
to "mark" multiple messages at once in a manner similar to dired for
directory contents.  Gnus has a number of different operations that can
be performed on messaged so marked.  One is reply.  When replying to
multiple messages at once, the In-Reply-To: header is set appropriately,
and the quoted contents of each is right there in the message buffer.

I used to use this feature of Gnus frequently, especially in busy
threads.  I often find multiple messages that need replying to, but that
replying in one message tends to be more effective than scattered
replies to multiple messages.

Problem is, I can't think of an unobtrusive way to add this to notmuch
without adding a lot of machinery.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [PATCH v7 0/1] emacs: notmuch-tree-outline-mode

[Matt Armstrong]

Jonathan Wilner  writes:

> I love this feature! I hope it could get mainstreamed. :-)

Yes, seems like a nice idea to me.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: [PATCH] emacs: fix dangling overlays in notmuch-search

[Matt Armstrong]

Matt Armstrong  writes:

> See Emacs bug#58479.  An upcoming change in Emacs will make these
> dangling overlays visible to the user.

It seems that the ellipsis overlays were shown due to a bug in Emacs,
but this patch is still a reasonable idea.  Otherwise, every time a
notmuch search buffer is refreshed (by pressing 'g') the overlays from
the previous run will collapse to zero-width overlays in position 1.
They're harmless there, but removing them is what 'evaporate is for.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH] emacs: fix dangling overlays in notmuch-search

[Matt Armstrong]

notmuch-search-insert-authors now sets the evaporate property on the
ellipsis overlays.  Emacs will delete them when the buffer contents
are zeroed out, which happens with `notmuch-refresh-buffer`.  This
prevents them from being collapsed to zero-width overlays in position
1.  See Emacs bug#58479.  An upcoming change in Emacs will make these
dangling overlays visible to the user.
---
 emacs/notmuch.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 26181758..6eef4af1 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -841,6 +841,7 @@ non-authors is found, assume that all of the authors match."
  overlay)
  (insert invisible-string)
  (setq overlay (make-overlay start (point)))
+ (overlay-put overlay 'evaporate t)
  (overlay-put overlay 'invisible 'ellipsis)
  (overlay-put overlay 'isearch-open-invisible #'delete-overlay)))
   (insert padding
-- 
2.35.1

___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH] Fix author-scan.sh on BSD systems.

[Matt Armstrong]

BSD xargs does not have the -d option. Here we use tr to convert
newlines to NUL characters, then pass -0 to xargs (which BSD does
support).

I looked at passing -z to 'git ls-files', but I did not find a BSD
grep option to turn on NUL deliminted line processing.
---
 devel/author-scan.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devel/author-scan.sh b/devel/author-scan.sh
index 2d9c4af8..23854f39 100644
--- a/devel/author-scan.sh
+++ b/devel/author-scan.sh
@@ -5,7 +5,7 @@ AUTHOR_EXCLUDE='uncrustify'
 # based on the FSF guideline, for want of a better idea.
 THRESHOLD=15
 
-git ls-files | grep -v -e "$FILE_EXCLUDE" | xargs -n 1 -d \\n \
+git ls-files | grep -v -e "$FILE_EXCLUDE" | tr '\n' '\0' | xargs -0 -n 1 \
   git blame -w 
--line-porcelain -- | \
 sed -n "/$AUTHOR_EXCLUDE/d; s/^[aA][uU][tT][hH][Oo][rR] //p" | \
 sort -fd | uniq -ic | awk "\$1 >= $THRESHOLD" | sort -nr
-- 
2.30.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Notmuch and backups

[Matt Armstrong]

David Bremner  writes:

[...]

> +case NOTMUCH_CONFIG_BACKUP_DIR:
> + return "database.backup_dir";

[...]

David, your recent changes that allow configurable separation of the
mail store and .notmuch files look like good ideas to me.

Separately, I've recently set up backups of my system and noticed that
the Xapian files are an order of magnitude larger than the notmuch dumps
(3.6 GiB vs. 15 MiB).  Is there any turn key code in notmuch itself to
keep a rolling set of "recent" notmuch dumps?

Alternatively, have folks contributed any such thing?

Failing everything, I can roll my own with cron and maybe logrotate or
something similar.  I thought I'd ask before going that way.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Emacs sending email via gmailieer

[Matt Armstrong]

Mark Gardner  writes:

> I am so close to getting my ideal email setup working...
>
> Instead of offlineimap or mbsync, I use gmailieer to pull down emails. It
> is quite fast and works with notmuch to map Gmail labels to notmuch tags.
> Much cleaner than the awkward way of accessing Gmail through IMAP. Where I
> am stuck is in sending the email.
>
> I could set up smtpmail as described in
> https://notmuchmail.org/emacstips/#index12h2 but gmailieer recently (v1.1)
> gained the capability to send mail. I would like to use that if I can so.
>
> Gmailieer sends a draft through Gmail without a problem (once I create a
> draft using message mode in emacs). Both sending a file and sending from
> stdin work:
> $ gmi send -C path/to/mailbox 1586809748.A123456Z12345.test\:2\,D
> $ cat 1586809748.A123456Z12345.test\:2\,D | gmi send -C path/to/mailbox
>
> But I am stuck configuring emacs to use gmailieer instead of sendmail or
> smtpmail.el. According to the emacs manual, the variable send-mail-function
> can be set but none of the choices looks appropriate for gmailieer:
> https://www.gnu.org/software/emacs/manual/html_node/emacs/Mail-Sending.html
>
> Anyone have an idea how to configure emacs to use gmailieer to send emails?

Mark, you'll likely have to write a send-mail-function that works with
ilieer. Or, you'll have to write a wrapper script that makes ilieer's
command line behave similarly to the classic Unix sendmail. At my former
employer we took the second option. SMTP was locked down but the
security folks provided a command line tool that could send email. The
Emacs afficionados patched that program to be behavior-compatible with
sendmail so we could set Emacs' 'send-mail-function' to
the 'sendmail-send-it' function.

Hope that helps!
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: how to search for hyphenated words? (was: how to search for Morse code?)

[Matt Armstrong]

David Bremner  writes:

> Matt Armstrong  writes:
>
>> Carl Worth  writes:
>>
>>> Hi Gregor,
>>>
>>> The trick here is that when notmuch is indexing body text it feeds it
>>> into a Xapian function that parses the text by finding "terms" in the
>>> text. And this parser considers both punctuation and whitespace as
>>> separators between terms.
>>
>> I notice that Xapian supports something called "phrase searches",
>> documented as:
>>
>>   "A phrase surrounded with double quotes ("") matches documents
>>   containing that exact phrase. Hyphenated words are also treated as
>>   phrases, as are cases such as filenames and email addresses
>>   (e.g. /etc/passwd or presid...@whitehouse.gov)."
>>
>> I assume that this particular Xapian feature is unavailable in notmuch?
>> If so, I wonder if enabling has ever been considered?
>
> It is enabled, and documented in notmuch-search-terms(7). Unfortunately
> I don't think it's related to the original request. The mention of
> hyphenated words is about the input to the query parser, not the
> (necessarily) the retrieved text.

Ah, so it boils down to the Xapian definition of "exact phrase."
Notably, "exact phrase" is not "identical sequence of characters" as
some people might expect.

Quick tests with various search engines reveal their phrase search as
operating the same way.  E.g. searching for "org notmuch" finds all
sorts of results:

  org-notmuch.el
  notmuchmail.org/notmuch-emacs/
  to:devicet...@vger.kernel.org notmuch tag +inbox +unread -new
  (require 'org-notmuch nil t)
  https://notmuchmail.org/notmuch-emacs/. *
  imaps://mail.example.org/Notmuch/search

For what it is worth, one thing I've taken to doing is using period
separators in the notmuch phrase searches I use in scripts and even
interactively.  Using periods is generally immune to confusing issues
related to quoting double quoted things, and always remains a single
shell "word."  They are also, most often, clearly not the exact content
I'm searching for, so they make it clear than the match algorithm is
inexact.  E.g.

  subject:notmuch.is.wonderful

instead of:

  subject:"notmuch is wonderful"
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Fix notmuch-describe-key

[Matt Armstrong]

Sheng Yang  writes:

> For the second case, I mean the call of notmuch-help fails with the
> following error:
>
> notmuch-documentation-first-line: Symbol’s function definition is void: 
> aya-persist-snippet
>
> Sorry for the confusion. Other void functions include the following for me
>
> spacemacs/search-pt-grep-region-or-symbol
> spacemacs/search-pt-grep
> spacemacs/search-rg-grep-region-or-symbol
> spacemacs/search-rg-grep
> spacemacs/search-ack-grep-region-or-symbol
> spacemacs/search-ack-grep

Thank you, that makes sense to me now.


> I think either is fine: notmuch-documentation-first-line is only
> called from notmuch-describe-key, we can do it in either function. The
> advantage of fixing it in notmuch-describe-key is that we can avoid
> some lines that have key bindings without description.

I agree.  Given all this, I think your patch looks good.


> Here are some known problems that are not fixed by this patch:
> 1. For me, 600+ lines of help are produced by notmuch-hello, which is a
> lot. (Maybe not as much, as counsel-desbinds tells me there are 3000+
> key bindings)

It is interesting that your help buffer displays 600 lines.  The intent
of the notmuch help is to display help for the notmuch commands, not the
myriad of other key bindings available.

I use vanilla Emacs (not spacemacs, not evil mode, etc.), and the help
buffers are about a page long.  For example, in a search buffer the key
bindings display as this:

Complete list of currently available key bindings:

?   Display help for the current notmuch mode.
q   Undisplay the current buffer.
s   Search for messages.
z   Display threads matching QUERY in Tree View.
g   Refresh the current buffer.
=   Refresh the current buffer.
M-= Invoke ‘notmuch-refresh-this-buffer’ on all notmuch major-mode buffers.
G   Invoke ‘notmuch-poll’ to import mail, then refresh the current buffer.
j   Jump to a saved search by shortcut key.
x   Undisplay the current buffer.
DEL Move backward through the search results by one window’s worth.
b   Move backward through the search results by one window’s worth.
SPC Move forward through search results by one window’s worth.
<   Select the first thread in the search results.
>   Select the last thread in the search results.
p   Select the previous thread in the search results.
n   Select the next thread in the search results.
r   Begin composing a reply to the entire current thread in a new buffer.
R   Begin composing a reply-all to the entire current thread in a new 
buffer.
o   Toggle the current search order.
c i Copy thread ID of current thread to kill-ring.
c q Copy current query to kill-ring.
c ? Show help for a subkeymap.
t   Filter the current search results based on a single tag.
l   Filter or LIMIT the current search results based on an additional query 
string.
k   Create a jump menu for tagging operations.
*   Add/remove tags from all messages in current search buffer.
a   Archive the currently selected thread or region.
C-u a   Un-archive the currently selected thread.
-   Change tags for the current thread or region (defaulting to remove).
+   Change tags for the current thread or region (defaulting to add).
RET Display the currently selected thread.
Z   Call notmuch tree with the current query
m   Mute the currently selected thread or region.


> 2. Key bindings using leader key (in spacemacs, it is ESC) are not
> shown correctly: instead of showing "SPC q q", notmuch-help shows "q q"
> for a key binding . 
> 3. Some lines show key bindings with no explanation. Possibly because no
> doc string is available for these functions.
>
> If we do not need to stick to the current implementation of
> notmuch-help, a simple way is to port counsel-desbinds or
> counsel--desbinds-cands from counsel.el.

Yes, I can believe that spacemacs and/or evil mode do not display useful
help when using "vanilla" Emacs commands to display the help, since
vanilla Emacs doesn't understand the approach.

I think it would be great if someone worked with upstream Emacs to make
the `documentation' function do something useful for evil mode and
spacemacs.  If that happened, then all packages would benefit.  Porting
things like counsel-descbinds into packages like notmuch can work but
does not feel like the right long term solution to me.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] lib: add 'body:' field, stop indexing headers twice.

[Matt Armstrong]

David, interesting idea.  I'm not very familiar with this code or its
conventions so my feedback should be taken with that in mind.  More
below.


David Bremner  writes:

> diff --git a/lib/database.cc b/lib/database.cc
> index 9cf8062c..27c2d042 100644
> --- a/lib/database.cc
> +++ b/lib/database.cc
> @@ -259,6 +259,8 @@ prefix_t prefix_table[] = {
>  { "directory",   "XDIRECTORY",   NOTMUCH_FIELD_NO_FLAGS },
>  { "file-direntry",   "XFDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS 
> },
>  { "directory-direntry",  "XDDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
> +{ "body","", NOTMUCH_FIELD_EXTERNAL |
> + NOTMUCH_FIELD_PROBABILISTIC},
>  { "thread",  "G",NOTMUCH_FIELD_EXTERNAL |
>   NOTMUCH_FIELD_PROCESSOR },
>  { "tag", "K",NOTMUCH_FIELD_EXTERNAL |

Above this new code in database.cc there is a comment describing the
schema.  E.g. "Mail document" describes id:, thread:, etc.  Add a
description of body: there?

Also, near those comments there is a double-space in the phrase
'uniquely identified by its "id" field' that you might fix while you're
nearby.


> diff --git a/lib/message.cc b/lib/message.cc
> index 6f2f6345..64349f83 100644
> --- a/lib/message.cc
> +++ b/lib/message.cc
> @@ -1443,13 +1443,13 @@ _notmuch_message_gen_terms (notmuch_message_t 
> *message,
>   message->termpos = term_gen->get_termpos () + 100;
>  
>   _notmuch_message_invalidate_metadata (message, prefix_name);
> +} else {
> + term_gen->set_termpos (message->termpos);
> + term_gen->index_text (text);
> + /* Create a term gap, as above. */
> + message->termpos = term_gen->get_termpos () + 100;
>  }
>  
> -term_gen->set_termpos (message->termpos);
> -term_gen->index_text (text);
> -/* Create a term gap, as above. */
> -message->termpos = term_gen->get_termpos () + 100;
> -
>  return NOTMUCH_PRIVATE_STATUS_SUCCESS;
>  }

Instead of the above I think I find what follows more clear.  This makes
it obvious which logic depends on the presence of a prefix and which
logic does not, which was a question I immediately had reading the code.

term_gen->set_termpos(message->termpos);
if (prefix_name) {
  term_gen->index_text (text, 1, _find_prefix (prefix_name));
} else {
  term_gen->index_text (text);
}
/* Create a gap between this an the next terms so they don't appear to be a
 * phrase. */
message->termpos = term_gen->get_termpos () + 100;
if (prefix_name) {
  _notmuch_message_invalidate_metadata (message, prefix_name);
}
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

notmuch.el: "wash" date in message display

[Matt Armstrong]

I miss some of the Gnus "wash" functions available for message display.
There was one that either reformatted the Date: header into my time
zone, or displayed the Date in terms of elapsed time from now (e.g. 1
hour ago, 1 day ago, etc.).  I don't remember which, but I'd be happy
with either.

Has anybody integrated a feature like that into notmuch.el?  It wouldn't
be hard, but I'd feel dumb if I did the work and there was some variable
I could set to get a similar effect.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] crypto: gracefully handle gmime errors

[Matt Armstrong]

David Bremner  writes:

> Jan Malakhovski  writes:
>
>>
>> The test suite has such a message already. "signature verification
>> (notmuch CLI)" test fails with a SIGSEGV when built with gmime-2.6.23.
>>
>> T355-smime: Testing S/MIME signature verification and decryption
>> gpgsm: keybox 
>> '/tmp/nix-build-notmuch-0.25.drv-0/notmuch-test-8789.qe2sRf/gnupg/pubring.kbx'
>>  created
>>  PASS   emacs delivery of S/MIME signed message
>>  PASS   emacs delivery of S/MIME encrypted + signed message
>>  PASS   Signature verification (openssl)
>>  FAIL   signature verification (notmuch CLI)
>
> Actually now that I try, I can't duplicate this failure on Debian (also
> with gmime-2.6.23). Matt, can you confirm or deny that the failures you
> were having were with SMIME signed messages?

I've lost track of the original thread, but I believe they were.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: all intermediate drafts appear in threads (v0.24.2, Emacs 25.2)

[Matt Armstrong]

David Bremner  writes:

> Matt Armstrong  writes:
>
>> David Bremner  writes:
>
>> This is happening to me on a fairly regular basis too.  Is there
>> something I can bind C-x C-s to in notmuch-show that doesn't create a
>> new draft?
>>
>> I like the idea of saving the drafts in the DB, so perhaps just getting
>> the deleted stuff working in an unsurprising way in notmuch-show is the
>> best way forward.
>
> For me, when I save a second draft, the first is tagged as deleted, and
> then mostly not shown. It's not perfect, but it's good enough at the
> moment. One person I talked to who was confused by the number of drafts
> in their inbox had excludes turned off. Excludes won't prevent deleted
> drafts showing up in threads, but they will be collapsed.

Ah thanks, that was it for me too.  I've restored exclude_tags to the
default "deleted;spam;" setting.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: bug: notmuch show --decrypt leads to SIGSEGV

[Matt Armstrong]

David Bremner  writes:

> Matt Armstrong  writes:
>
>> David Bremner  writes:
>>
>>> Matt Armstrong  writes:
>>>
>>>> I've been able to diagnose a SIGSEGV, and I have a workaround that
>>>> satisfies me.  I'm unsure how to fix it, so I'll describe the problem
>>>> and leave it at that.
>>>>
>>>> Repro:
>>>>
>>>> % notmuch --version
>>>> notmuch 0.25+22~g0967e46 (a recent git @HEAD)
>>>> % notmuch show --format=sexp --decrypt thread:0002ad2c
>>>> -> SIGSEGV
>>>
>>> Do you have a way for people other than you to reproduce it? I assume
>>> not all threads are a problem for you with --decrypt?
>>
>> The times I have encountered this involve email I can't reveal, so a
>> repro would take some work.  I'm happy to try to dig out specific
>> details, but I'm at a slight disadvantage here due to unfamiliarity.
>>
>
> Understood. If you manage to bisect the commit that introduces the
> problem (I suspect the rearrangement to support gmime-3.0, but you never
> know), that might be helpful.

David, your suspicions may have been correct.  The bisect came up with
the following commit.

1fdc08d0ffab9b211861de5d148d0a79eae840bc is the first bad commit
commit 1fdc08d0ffab9b211861de5d148d0a79eae840bc
Author: David Bremner 
Date:   Sun Jul 16 01:01:43 2017 +0200

cli/crypto: treat failure to create a crypto context as fatal.
  
Silently ignoring signed/encrypted parts seems like the wrong idea,
and it also complicates future gmime-3.0 compatibility changes.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: all intermediate drafts appear in threads (v0.24.2, Emacs 25.2)

[Matt Armstrong]

David Bremner  writes:

> Sanjoy Mahajan  writes:
>
>> I save an outgoing email as I write it and often (as if it were a
>> regular document) by using the usual Emacs key sequence C-x C-s.  With
>> notmuch v0.24, notmuch saves a proper draft each time I use C-x C-s.
>> The result is a million drafts, which may be good.
>>
>> However, it means that the thread view (in notmuch-show mode) includes
>> the zillion drafts.  Attached is a screenshot of what I mean (sorry that
>> it is so small -- I downsampled it to get within the message-size
>> limit).  I made the example by sending myself (sanjoy@localhost) a short
>> msg.  Then I replied to it with 'r', and saved the reply a few times
>> (first with "Draft v1" as the body text, the next time with "Draft v2",
>> etc.).
>
> It could be a nice feature to toggle visibility of drafts (and other
> excluded messages) in notmuch-show mode. So far nobody wrote such a
> thing.
>
> Two things you might find useful to know:
>
> - you can resume editing a previous draft with 'e'
> - autosaving does not create drafts
>
> If it really bugs you, you could override the binding C-x C-s in
> notmuch-show mode.

This is happening to me on a fairly regular basis too.  Is there
something I can bind C-x C-s to in notmuch-show that doesn't create a
new draft?

I like the idea of saving the drafts in the DB, so perhaps just getting
the deleted stuff working in an unsurprising way in notmuch-show is the
best way forward.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: bug: notmuch show --decrypt leads to SIGSEGV

[Matt Armstrong]

David Bremner  writes:

> Matt Armstrong  writes:
>
>> I've been able to diagnose a SIGSEGV, and I have a workaround that
>> satisfies me.  I'm unsure how to fix it, so I'll describe the problem
>> and leave it at that.
>>
>> Repro:
>>
>> % notmuch --version
>> notmuch 0.25+22~g0967e46 (a recent git @HEAD)
>> % notmuch show --format=sexp --decrypt thread:0002ad2c
>> -> SIGSEGV
>
> Do you have a way for people other than you to reproduce it? I assume
> not all threads are a problem for you with --decrypt?

The times I have encountered this involve email I can't reveal, so a
repro would take some work.  I'm happy to try to dig out specific
details, but I'm at a slight disadvantage here due to unfamiliarity.

I didn't this notice this before, but I am getting this on stderr:

"Failed to construct pkcs7 context."

So g_mime_gpg_context_new() is returning NULL.

I am running an old Ubuntu variant.  My libgmime is libgmime-2.6-dev.
Looks like nothing in the API contract for g_mime_gpg_context_new()
states that it never returns NULL.  Still, it appears that libgmime is
built with ENABLE_CRYPTOGRAPHY (when I download gmime source and
debbuild it, config.h sets that macro), so I'm at a loss for why that
function would return an error.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

bug: notmuch show --decrypt leads to SIGSEGV

[Matt Armstrong]

I've been able to diagnose a SIGSEGV, and I have a workaround that
satisfies me.  I'm unsure how to fix it, so I'll describe the problem
and leave it at that.

Repro:

% notmuch --version
notmuch 0.25+22~g0967e46 (a recent git @HEAD)
% notmuch show --format=sexp --decrypt thread:0002ad2c
-> SIGSEGV

Workaround:

Don't pass --decrypt.  In Emacs, configure notmuch-crypto-process-mime
to shut off crypto processing, or C-u before in notmuch-show before
viewing a problematic thread.

Diagnosis:

mime-node.c's _mime_node_create() can return NULL in various scenarios
yet few to none of its callers appear to handle it properly.  In this
particular case, the NULL is returned here:

#if (GMIME_MAJOR_VERSION < 3)
if ((GMIME_IS_MULTIPART_ENCRYPTED (part) && node->ctx->crypto->decrypt)
|| (GMIME_IS_MULTIPART_SIGNED (part) && node->ctx->crypto->verify)) {
GMimeContentType *content_type = g_mime_object_get_content_type (part);
const char *protocol = g_mime_content_type_get_parameter (content_type, 
"protocol");
cryptoctx = notmuch_crypto_get_context (node->ctx->crypto, protocol);
if (!cryptoctx) {
return NULL;
}
}
#endif

Note above a missing call to talloc_free(node) before the return, which
suggests a kind of bit-rot has set in for the GMIME_MAJOR_VERSION<3
case?  Anyway...

mime_node_child() calls _mime_node_create() and will SIGSEGV:

node = _mime_node_create (parent, sub);

if (child == parent->next_child && parent->next_part_num != -1) {
/* We're traversing in depth-first order.  Record the child's
 * depth-first numbering. */
node->part_num = parent->next_part_num;
node->next_part_num = node->part_num + 1;


If I address that by returning NULL from mime_node_child() when
_mime_node_create() does, then the problem cascades to callers.  None of
the callers of mime_node_child() explicitly handle the NULL return case:

mime-node.c:mime_node_t *child = mime_node_child (node, i);
notmuch-show.c:571: format_part_text (ctx, sp, mime_node_child (node, i), 
indent, params);
notmuch-show.c:622: format_part_sprinter (ctx, sp, mime_node_child 
(node, 0), first, TRUE, include_html);
notmuch-show.c:724: format_part_sprinter (ctx, sp, mime_node_child (node, 
i), i == 0, TRUE, include_html);

..._mime_node_seek_dfs_walk will proceed to SIGSEGV, and so will
format_part_...().
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: finding incoming messages in threads in which i've participated [was: Re: find threads where I and Jian participated but not Dave]

[Matt Armstrong]

Daniel Kahn Gillmor  writes:

> Hey all--
>
> I really appreciate the thought and experimentation and research that's
> gone into this thread!
>
> On Thu 2017-06-22 17:00:58 -0700, Matt Armstrong wrote:
>> # All threads in which I participate get tag:participated
>> #  1) Find all threads with a message tagged new
>> # (finding all 'today' messages helps during testing,
>> # but isn't necessary)
>> #  2) Run through "xargs -s 2048 echo" to to group threads
>> # lines of about 2K in size.
>> #  3) For each line (2) produces, narrow the threads to
>> # those containing a message from me.
>> #  4) For each such thread, tag every message with +participated.
>> notmuch search --output=threads tag:new OR date:today | \
>>   xargs -s 2048 echo | \
>>   xargs -I '{}' notmuch search \
>>   --output=threads from:marmstrong AND \( '{}' \) | \
>>   sed -e 's,^,+participated -- ,' | \
>>   notmuch tag --batch
>
> This makes sense to me, modulo the split into 2048-octet lines (magic
> numbers make me nervous, though i think i understand why you've included
> it).

Yes, the two xargs commands and "2048 business" is just a hack to work
around the documented limitations of "xargs -I".  I'd love to come up
with a simpler way to do this.  I suspect there is one, but when it
comes to this kind of Unix shell hackery, I usually stop once I get to
something that works.  :)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: finding incoming messages in threads in which i've participated [was: Re: find threads where I and Jian participated but not Dave]

[Matt Armstrong]

David Bremner  writes:

> Daniel Kahn Gillmor  writes:
>
>>
>> For example, would it make sense to have "notmuch new" (and "notmuch
>> insert") do "thread-based propagation" of specific tags?  for example,
>> consider the following (i've just made up the config options):
>>
>> notmuch config set new.from_self_tags participated
>> notmuch config set new.propagate_thread_tags participated
>>
>> the idea is that "new.from_self_tags" would be applied by "notmuch new" or
>> "notmuch insert" if the message was explicitly from: user.primary_email
>> or user.other_email.
>
> At the moment I'm more inclined to work on "doing things right" by
> adding xapian documents (database items) for threads. Many of the
> ideas in this thread amount to working around their absence. OTOH,
> it's certainly true that this last idea (unlike some of the query
> ideas) would be relatively straightforward to impliment.

David, that makes a lot of sense.

Take two other examples from my post-hook:

# Unmute all threads with new messages sent to me.
notmuch search --output=threads tag:new AND tag:me | \
  xargs --no-run-if-empty notmuch tag -muted --

# Remove all muted threads from the inbox and mark
# every message in them muted.
notmuch search --output=threads tag:muted | \
  xargs --no-run-if-empty notmuch tag -inbox +muted --

Above I'm just operating on all the messages in a thread as a unit.  If
notmuch had a 'database item' for each thread, I could potentially tag
just that to achieve similar results with simpler commands.

It has some subtle implications.  E.g. when I tag through a particular
UI do I mean to tag a particular message or the thread?  Is it worth
making the user think about the difference?  Is there some way to
express this such that they never do?  Are some tags configured to
always apply to the thread?  Are these the only tags that do?  Or do
"thread tags" always contain the union of all associated message tags?

Fun stuff.  :)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: find threads where I and Jian participated but not Dave

[Matt Armstrong]

Gaute Hope  writes:

> Gaute Hope writes on juni 22, 2017 8:08:
>> Daniel Kahn Gillmor writes on juni 21, 2017 23:30:
>>> On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote:
>>>> For what it is worth, I've found this idea from Daniel intriguing and
>>>> pretty useful in practice:
>>>>
>>>>   "show me threads in which i've participated, where there are some
>>>>messages flagged with 'inbox'"
>>>>
>>>> I implement it like this in my post-new hook:
>>>>
>>>> # All messages in threads in which I participate get tag:participated
>>>> notmuch search --output=threads from:marmstrong | \
>>>>   sed -e 's,^,+participated -- ,' | \
>>>>   notmuch tag --batch
>>> 
>>> cool, thx for the suggestion.
>>> 
>>> the "notmuch search" part of the pipeline alone takes ~19s (wall time,
>>> and actual CPU time) for me though :/  It returns 30504 threads!  how
>>> many threads do you get?
>> 
>> Is there any reason why you do not filter on a tag 'new' as well?
>> 
>>  notmuch search --output=threads from:marmstrong and tag:new | \
>>sed -e 's,^,+participated -- ,' | \
>>notmuch tag --batch
>> 
>
> Nevermind, I get it - it might be possible to add a temporary tag 
> new-tag to the whole thread first and not just new messages. That might 
> be faster. As long as all sent messages get the new tag as well.

Gaute, I took this as a challenge and came up with what I think is an
equivalent but more efficient approach.  The disadvantage is that it is
much more complex.  The advantage is that it runs in under 0.2 seconds
to process a day's worth of my "new" mail.

I now have this in my notmuch post-hook.  I believe I could change the
"tag:new OR date:today" query to just "tag:new".  The "OR date:today"
helped during interactive development.

# All threads in which I participate get tag:participated
#  1) Find all threads with a message tagged new
# (finding all 'today' messages helps during testing,
# but isn't necessary)
#  2) Run through "xargs -s 2048 echo" to to group threads
# lines of about 2K in size.
#  3) For each line (2) produces, narrow the threads to
# those containing a message from me.
#  4) For each such thread, tag every message with +participated.
notmuch search --output=threads tag:new OR date:today | \
  xargs -s 2048 echo | \
  xargs -I '{}' notmuch search \
  --output=threads from:marmstrong AND \( '{}' \) | \
  sed -e 's,^,+participated -- ,' | \
  notmuch tag --batch


The basic idea is that each run of the notmuch post-hook will
incorporate relatively little mail, so the number of unique threads will
be relatively small.  So, we just list them all by thread ID.

Then for each thread with new messages, we figure out which threads have
a message from:marmstrong (it need not be the new message).

We then tag all messages in each of those threads with +participated.

You said "it might be possible to add a temporary tag new-tag to the
whole thread first and not just new messages." -- Yes, and that is
implicitly what I am doing, except that each such thread is instead
tracked in an ephemeral way through the xargs based shell pipeline.

I did try an approach of explicitly labeling all messages in "new"
threads, temporarily, but that was slower.

You said "As long as all sent messages get the new tag as well." --
true, and I'm not sure about that.  My primary use for this is to
discover new activity from others *after* I've participated in a thread,
so I don't much care if a thread that is "participated in" is not tagged
that way until some mail from somebody else arrives.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: find threads where I and Jian participated but not Dave

[Matt Armstrong]

Daniel Kahn Gillmor  writes:

> On Wed 2017-06-21 13:04:53 -0700, Matt Armstrong wrote:
>> For what it is worth, I've found this idea from Daniel intriguing and
>> pretty useful in practice:
>>
>>   "show me threads in which i've participated, where there are some
>>messages flagged with 'inbox'"
>>
>> I implement it like this in my post-new hook:
>>
>> # All messages in threads in which I participate get tag:participated
>> notmuch search --output=threads from:marmstrong | \
>>   sed -e 's,^,+participated -- ,' | \
>>   notmuch tag --batch
>
> cool, thx for the suggestion.
>
> the "notmuch search" part of the pipeline alone takes ~19s (wall time,
> and actual CPU time) for me though :/  It returns 30504 threads!  how
> many threads do you get?

The query returns 6600 threads.  I'm getting 2 seconds wall clock time.


> you're effectively re-tagging every single message in every participated
> thread every time you run "notmuch new", right?

Yeah, the "batch script" that the above search+sed creates and pipes
into "notmuch tag --batch" is 265K, but it only takes 0.5 seconds to
execute.  My understanding is that "notmuch tag" is smart enough to do
no work if the tag is already present on a message, so the only changes
happening in the database are actually for new stuff.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: find threads where I and Jian participated but not Dave

[Matt Armstrong]

Gaute Hope  writes:

> David Bremner writes on juni 15, 2017 22:20:
>> Daniel Kahn Gillmor  writes:
>>>
>>> One of my long-standing wishes is to be able to say "show me mails in my
>>> inbox from people who have replied to messages i've sent them".
>>>
>>> This could be re-framed as "show me threads in which i've participated,
>>> where there are some messages flagged with 'inbox'".  but generating a
>>> huge list of all threads in which i've participated, just to be able to
>>> do an intersection operation with a (much smaller) list of all threads
>>> that have a message with the inbox flag seems like a pretty gross
>>> inefficiency.
>> 
>> At the moment the best we could do is essentially the same algorithm,
>> but in C instead of shell / python. Threads are not documents in the
>> database, so they can't efficiently be searched for.  Of course we could
>> change that, but those kind of changes take a fair amount of effort, and
>> some careful design work.
>
> There are probably multiple earlier references to this, but here's one:
>
>   id:1471858269.x2m28lgosh.astroid@strange
>
> matching against the whole thread vs the individual messages would be
> very useful!

For what it is worth, I've found this idea from Daniel intriguing and
pretty useful in practice:

  "show me threads in which i've participated, where there are some
   messages flagged with 'inbox'"

I implement it like this in my post-new hook:

# All messages in threads in which I participate get tag:participated
notmuch search --output=threads from:marmstrong | \
  sed -e 's,^,+participated -- ,' | \
  notmuch tag --batch

On my database the query takes about two seconds to run, and lets me to
searches like "tag:inbox and tag:participated".  The set of threads
found is typically a subset of "tag:inbox and to:marmstrong", but not
always, and I now have two canned "inbox" searches:

  "participated" -> "tag:inbox and tag:participated"
  "me" -> "tag:inbox and to:marmstrong and not tag:participated"

The "me" search tends to be new stuff, bot-generated notifications, and
such.  The "participated" is typically active conversations and stuff
I've already engaged with, or initiated myself.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: find threads where I and Jian participated but not Dave

[Matt Armstrong]

David Bremner  writes:

> Daniel Kahn Gillmor  writes:
>
>>
>> One of my long-standing wishes is to be able to say "show me mails in my
>> inbox from people who have replied to messages i've sent them".
>>
>> This could be re-framed as "show me threads in which i've participated,
>> where there are some messages flagged with 'inbox'".  but generating a
>> huge list of all threads in which i've participated, just to be able to
>> do an intersection operation with a (much smaller) list of all threads
>> that have a message with the inbox flag seems like a pretty gross
>> inefficiency.
>
> At the moment the best we could do is essentially the same algorithm,
> but in C instead of shell / python. Threads are not documents in the
> database, so they can't efficiently be searched for.  Of course we could
> change that, but those kind of changes take a fair amount of effort, and
> some careful design work.

Even if the C level does the same algorithm, it may be able to do some
optimizations on behalf of the "scripting layer" queries.

I suspect that a separate "thread based" query language may be an
interesting area of investigation.

Taking Daniel's last example, "show me mails in my inbox from people who
have replied to messages I've sent them".  That isn't even an entirely
unambiguous query specification.  What is *actually* desired:

a) show me messages from X that are part of threads where at least one
message is in the inbox and for which at least one message is from me.

or,

b) same as (a) but the "message from X" must be in the inbox (not just
any other message in the thread)

or,

c) same as (a) or (b) but the "message from X" is a reply (e.g. dated
after, or in-reply-to) a message from me.

or,

d) same as (c) but "message from X" is "unread", etc.

Like David's 'comm -12 A B' solution, these pretty quickly start looking
like multi-pass, or structed/nested, queries.  They are a lot more like
relational database queries (SQL) than the single-pass, flat (NoSQL)
queries we typically use with notmuch.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] NEWS for notmuch-cycle-notmuch-buffers

[Matt Armstrong]

Mark Walters  writes:

> +Fix notmuch-interesting-buffer and notmuch-cycle-notmuch-buffers.
> +
> +  The functions `notmuch-interesting-buffer` which detects notmuch
> +  buffers, and `notmuch-cycle-notmuch-buffers` which cycles between
> +  notmuch buffers, now additionally detect notmuch-tree-mode and
> +  notmuch-message-mode buffers.
> +

If you're not happy with the wording I'd just say:

> notmuch-tree-mode and notmuch-message-mode-buffers are now considered
> interesting by `notmuch-interesting-buffer` and
> `notmuch-cycle-notmuch-buffers`.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Notmuch dev workflow: applying patches to local trees

[Matt Armstrong]

Jani Nikula  writes:

> On Tue, Nov 8, 2016 at 1:16 AM, Matt Armstrong  wrote:
>> I'm not experienced with managing patches via email on this list,
>> especially when it comes to applying patches sent out for review and
>> testing.  Is there a documented best practice?
>
> Not sure about best practice, but this works for me:
>
> M-x cd RET /path/to/repo RET
>
> Apply a single patch in show mode:
> | git am RET
>
> Apply all expanded patch messages in a thread in show mode:
> C-u | git am RET

Ah, notmuch-show-pipe-message is what I was missing.  Thanks Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Notmuch dev workflow: applying patches to local trees

[Matt Armstrong]

I'm not experienced with managing patches via email on this list,
especially when it comes to applying patches sent out for review and
testing.  Is there a documented best practice?

Ideally, I'd like a "dwim" command that does something reasonable, such
as take messages from the current thread and apply the patch series to a
new/pristine git branch in my local repo.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: mua: add a pre-send-check-hook

[Matt Armstrong]

Hey Mark,

For consistency with Emacs' own elisp, perhaps rename
notmuch-mua-pre-send-check-hooks to
notmuch-mua-pre-send-check-functions?

This patch reminded me of a recent discussion on emacs-devel about the
definition of "hook" in Emacs code and documentation.  There is the
broad meaning of "arbitrary extension point", and the narrow meaning of
"list of functions run by `run-hook'".  This is "hook" in the former
sense.

From the Elisp manual's Modes > Hooks section:

   If the hook variable’s name does not end with ‘-hook’, that indicates
it is probably an "abnormal hook".  That means the hook functions are
called with arguments, or their return values are used in some way.  The
hook’s documentation says how the functions are called.  You can use
‘add-hook’ to add a function to an abnormal hook, but you must write the
function to follow the hook’s calling convention.  By convention,
abnormal hook names end in ‘-functions’.

[...]

 -- Function: run-hook-with-args-until-failure hook &rest args
 This function runs an abnormal hook by calling each hook function
 in turn, stopping if one of them fails by returning ‘nil’.  Each
 hook function is passed the arguments ARGS.  If this function stops
 because one of the hook functions fails, it returns ‘nil’;
 otherwise it returns a non-‘nil’ value.


When calling hooks by run-hook-with-args-until-failure/-success, Emacs'
own elisp code follows the -functions convention for the most part.



Mark Walters  writes:

> This add a pre-send hook for any checks the user wants to run before
> sending the message. If any function in the hook returns nil then the
> send will abort.
>
> One use would be to check that the from address is appropriate for the
> recipients (i.e., test From: based on To: and Cc:), but many other
> uses are possible: checking spelling, checking that the message is set
> to be encrypted etc.
> ---
>
> bremner and I were discussing the address-completion-hook on irc, and
> before he implemented that I suggested an alternative of adding a
> pre-send-check-hook. The idea is that functions in this hook can force
> abort sending (or at least get confirmation from the user) based on
> the message.
>
> For example I would be quite likely to use something like the
> following in the hook.
>
> (lambda ()
>   (save-excursion
> (save-restriction
>   (let ((to (message-fetch-field "To"))
>   (from (message-fetch-field "From"))
>   (case-fold-search t))
>   (or (not (string-match "work-domain-address" to))
>   (string= from "my-work-address")
>   (yes-or-no-p "Message to work but not from work address. Really 
> send? "))
>
> I think this is reasonably orthogonal to the
> notmuch-address-completion-hook. Setting the from address based on the
> to addresses makes a lot of sense, but checking on send also does --
> if the from is correct then the check is silent, and if the user types
> in the To: without using completion then the check will catch it.
>
>
> Best wishes
>
> Mark
>
>
> emacs/notmuch-mua.el | 12 +++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
>
> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
> index f333655..78130e6 100644
> --- a/emacs/notmuch-mua.el
> +++ b/emacs/notmuch-mua.el
> @@ -36,6 +36,15 @@
>  
>  ;;
>  
> +(defcustom notmuch-mua-pre-send-check-hook nil
> +  "Hook of checks run before sending messages.
> +
> +If any of the functions in the hook return nil then the send will
> +be aborted."
> +  :type 'hook
> +  :group 'notmuch-send
> +  :group 'notmuch-hooks)
> +
>  (defcustom notmuch-mua-send-hook '(notmuch-mua-message-send-hook)
>"Hook run before sending messages."
>:type 'hook
> @@ -538,7 +547,8 @@ unencrypted.  Really send? "
>  
>  (defun notmuch-mua-send-common (arg &optional exit)
>(interactive "P")
> -  (when (and (notmuch-mua-check-no-misplaced-secure-tag)
> +  (when (and (run-hook-with-args-until-failure 
> 'notmuch-mua-pre-send-check-hook)
> +  (notmuch-mua-check-no-misplaced-secure-tag)
>(notmuch-mua-check-secure-tag-has-newline))
>  (letf (((symbol-function 'message-do-fcc) 
> #'notmuch-maildir-message-do-fcc))
> (if exit
> -- 
> 2.1.4
>
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Is there a separate git repo for the notmuch website?

[Matt Armstrong]

I noticed that https://notmuchmail.org/emacstips/#index21h2 is stale
(the patch it mentions is now part of notmuch).  Is there a git
repository for the website that I can suggest patches against?
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] emacs: add compatability functions for emacs 23

[Matt Armstrong]

Mark Walters  writes:

[...]

> Version 1 of this patch (with some discussion) is at
> id:1477191835-17828-1-git-send-email-markwalters1009 at gmail.com
>
> The general consensus is that we should not define functions outside
> our namespace, even when they are just backports of functions from
> later emacs.
>
> rlb on irc gave one additional reason not mentioned earlier in the
> thread -- it could be that some other package choses to test for the
> setq-local say and perhaps draws incorrect conclusions about the
> environment. In particular this would be a case where we were breaking
> otherwise working packages.

Ahh, good point.

Thanks for sticking with the patch.


> I think it makes sense to included the whole of read-char-choice, not
> some cutdown version, as we may use other functionality from it later
> (eg help-forms etc), and it would be confusing if the change only
> worked on emacs 24.
>
> Finally, I have two questions
>
> 1) please could someone with emacs 23 see if the testsuite passes? My
> system with emacs 23 is so outdated the test suite doesn't run (wrong
> python versions I think).

I tried.

FWIW, the patch didn't apply cleanly for me, though I'm not used to
slurping patches around in email.  I used: "git apply -3 mark.patch".
"git apply --reject mark.patch" left a few .rej files that were easily
applied by hand (it looked like some code had moved too far?).

I'm stuck on an Ubuntu trusty release, which is rather old and has a
packaged emacs 23.  I'm not sure how useful my results are, though,
because I show a lot of test errors regardless.  This may be a result of
me compiling my own xapian, which the notmuch build system does not
handle particularly gracefully.  Most failures are of the form "A Xapian
exception occurred performing query", and unrelated to your patch.

Before your patch I got the expected byte compiler warnings:

% XAPIAN_CONFIG=$HOME/opt/xapian-core-1.4.0/bin/xapian-config ./configure
% LD_RUN_PATH=$HOME/opt/xapian-core-1.4.0/lib make test

[...]

notmuch-company.el:95:1:Warning: the function `setq-local' is not known to be
defined.
notmuch-address.el:351:1:Warning: the function `setq-local' is not known to be
defined.
notmuch-maildir-fcc.el:376:1:Warning: the function `read-char-choice' is not
known to be defined.

[...then I see...]

Notmuch test suite complete.
635/783 tests passed.
2 broken tests failed as expected.
43 tests failed.
103 tests skipped.

[...with this patch the byte compiler shuts up, but I get the same test
suite errors...]

Notmuch test suite complete.
635/783 tests passed.
2 broken tests failed as expected.
43 tests failed.
103 tests skipped.

[...but not to worry, I get the same running Emacs 25 too...]

I'll attach full output of the notmuch test run.


> 2) Is the copyright notice I have included above the two functions
> sufficient, and suitably placed?

An alternative is to place these functions in a new file named something
like notmuch-backport.el, with clear FSF copyright.  Since what
constitutes "functions in this section" may drift over time, the rigor
of a separate file might be desirable.


> --- a/emacs/notmuch-lib.el
> +++ b/emacs/notmuch-lib.el
> @@ -966,6 +966,78 @@ status."
>  (defvar notmuch-show-process-crypto nil)
>  (make-variable-buffer-local 'notmuch-show-process-crypto)
>  
> +;; Compatibility functions for emacs 23.
> +
> +;; The functions in this section are copied from eamcs 24.4 and are

typo: eamcs


-- next part --
Use "make V=1" to print test headings and PASSing results.
INFO: using 2 minute timeout for tests

T050-new: Testing "notmuch new" in several variations
 FAIL   Xapian exception: read only files
--- T050-new.30.expected2016-11-02 23:49:06.740165419 +
+++ T050-new.30.output  2016-11-02 23:49:06.740165419 +
@@ -1 +1,17 @@
-A Xapian exception occurred opening database
+(D) add_files, pass 1
+(D) add_files, pass 1
+(D) add_files, pass 1
+(D) add_files, pass 1
+(D) add_files, pass 1
+(D) add_files, pass 1
+(D) add_files, pass 2
+(D) add_files, pass 2
+(D) add_files, pass 1
+(D) add_files, pass 2
+(D) add_files, pass 1
+(D) add_files, pass 2
+(D) add_files, pass 2
+(D) add_files, pass 2
+(D) add_files, pass 2
+(D) add_files, pass 2
+No new mail.
chmod: cannot access 
'/usr/local/google/home/marmstrong/git/notmuch/test/tmp.T050-new/mail/.notmuch/xapian/*.DB':
 No such file or directory
chmod: cannot access 
'/usr/local/google/home/marmstrong/git/notmuch/test/tmp.T050-new/mail/.notmuch/xapian/*.DB':
 No such file or directory

T060-count: Testing "notmuch count" for messages and threads
 FAIL   error message for database open
--- T060-count.13.expected  2016-11-02 23:49:06.984168633 +
+++ T060-count.13.output2016-11-02 23:49:06.984168633 +
@@ -1 +1 @@

[PATCH v3] emacs: make faces readable on dark backgrounds.

[Matt Armstrong]

The notmuch-tag-flagged, notmuch-search-flagged-face and
notmuch-crypto-part-header faces defaulted to "blue", which is nearly
unreadable when a dark background is in use.  This is addressed by using
"LightBlue1" for dark backgrounds.

As a side effect, these faces are now no-op definitions for grayscale or
mono displays.
---
 emacs/notmuch-crypto.el | 7 ++-
 emacs/notmuch-tag.el| 7 ++-
 emacs/notmuch.el| 6 +-
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el
index e376aa8..68a7e9f 100644
--- a/emacs/notmuch-crypto.el
+++ b/emacs/notmuch-crypto.el
@@ -42,7 +42,12 @@ mode."
   :group 'notmuch-crypto)
 
 (defface notmuch-crypto-part-header
-  '((t (:foreground "blue")))
+  'class color)
+  (background dark))
+ (:foreground "LightBlue1"))
+(((class color)
+  (background light))
+ (:foreground "blue")))
   "Face used for crypto parts headers."
   :group 'notmuch-crypto
   :group 'notmuch-faces)
diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el
index 1b2ce5c..09d182d 100644
--- a/emacs/notmuch-tag.el
+++ b/emacs/notmuch-tag.el
@@ -115,7 +115,12 @@ Used in the default value of `notmuch-tag-formats`."
   :group 'notmuch-faces)
 
 (defface notmuch-tag-flagged
-  '((t :foreground "blue"))
+  'class color)
+  (background dark))
+ (:foreground "LightBlue1"))
+(((class color)
+  (background light))
+ (:foreground "blue")))
   "Face used for the flagged tag.
 
 Used in the default value of `notmuch-tag-formats`."
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 2fc72b7..c0e9e1d 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -313,7 +313,11 @@ there will be called at other points of notmuch execution."
   :group 'notmuch-faces)
 
 (defface notmuch-search-flagged-face
-  '((t
+  'class color)
+  (background dark))
+ (:foreground "LightBlue1"))
+(((class color)
+  (background light))
  (:foreground "blue")))
   "Face used in search mode face for flagged threads.
 
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] emacs: make faces readable on dark backgrounds.

[Matt Armstrong]

This is a followup to
id:1476987754-13672-1-git-send-email-marmstr...@google.com with:

 - Addition of ((class color) (background light)) tests for the light
   faces, as requested by Mark.  This is consistent with how many
   other light/dark faces are already defined in notmuch.

 - Switch to a blue color, as opposed to "gold".  This makes sense,
   because light backgrond face uses "blue".  David suggested "deep
   sky blue" but I found that pretty fruit-salady and actually darker
   than the normal text.  This tended to deemphasize flagged email
   when mixed among non-flagged mail in notmuch-show mode.  The point
   is the opposite.  I settled on LightBlue1, which becomes cyan on 8
   color displays (xterm), while likely to be subtly different from
   whatever default text color the user has set up.

 - renamed patch from "emacs: make faces clear on dark backgrounds"
   (s/clear/readable/).
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matt Armstrong]

Matthew Lear  writes:

> Hi,
> I switched to trying emacs 25 (25.1) with notmuch the other day.

I'm on 25.1 as well, but haven't experienced the symptoms you describe.

>> !!! Bodypart handler `notmuch-show-insert-part-text/html' threw an error:
>> !!! Window is dedicated to ‘*unsent mail to u...@domain.com*’
>
> Errors like these are generated by replying from either notmuch-tree or
> notmuch-show.

Below I tried replying from notmuch-show.

> I'm of the opinion that this is probably some sort of set up /
> configuration issue specific to 25, but I'm quite surprised. I don't
> have many settings for handling html stuff:
>
> (setq mm-text-html-renderer 'shr)
> (setq shr-inhibit-images nil)
> (setq mime-view-text/html-previewer 'shr)
> (setq mm-inline-text-html-with-images t)
> (setq notmuch-multipart/alternative-discouraged '("text/plain"))
>
> Is this reproducible for anyone else with notmuch from git/master as of
> today?

Matt, I couldn't resist helping another, Matt, so I copy-pasted your
setq incantations, executed them (thus "poisoning" my precious Emacs!),
and was able to initiate replies to several emails.  Some were
multipart/mixed with HTML alternatives, some (like yours) were
text/plain.

I'm running recent-ish notmuch binaries from git/master, running even
fresher elisp from git/master (I think I pulled last night).

It may be necessary for you to play around with the various Emacs
options to eliminate site specific stuff: -Q, -q, etc.  Here at work a
bunch of people are moving to Emacs 25 and a common theme is some old
package in their setup somehow conflicts with Emacs 25 (e.g. it shadows
something that is now part of Emacs but has changed in subtle ways,
etc.).
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: rfc for notmuch remote access script

[Matt Armstrong]

Neat.  Basics of it look correct to me.  Personally, I'd abandon most of
these environment variables and edit the script directly, but that goes
against your stated goal.

Other comments below.


Tomi Ollila  writes:

> Hi
>
> j4ni on irc expressed interest of having an installation option for
> notmuch-emacs and a notmuch remote access script as 'notmuch'
>
> This got me thinking what kind of script would fulfill all the needs
> that I know of.
>
> and this is the result of that:
>
> --8<8<8<8<8<8<8<8<--
>
> #!/bin/bash
>
> set -euf
>
> if [ "${NOTMUCH_REMOTE_XTRACE_FILE-}" ]; then

In bash, I often see people favoring this style:

if [[ -n ${NOTMUCH_REMOTE_XTRACE_FILE-} ]]; then


>   exec 6>>"$NOTMUCH_REMOTE_XTRACE_FILE"
>   BASH_XTRACEFD=6
>   echo -- >&6
>   set -x
>   env >&6

This BASH_XTRACEFD stuff is nice, and the technique is new to me.  Is
there a way to close the trace file descriptor so it isn't available to
the ssh command?  Maybe something like this works?

exec 6>&- "$NOTMUCH_REMOTE_SSH_COMMAND" $NOTMUCH_REMOTE_SSH_ARGS \
  "$NOTMUCH_REMOTE_HOST" $NOTMUCH_REMOTE_COMMAND $ARGS

(though that only needs to happen optionally, which makes it tricky).

> : ${NOTMUCH_REMOTE_SSH_COMMAND:=ssh}
> : ${NOTMUCH_REMOTE_SSH_ARGS=}
> : ${NOTMUCH_REMOTE_HOST:=notmuch}
> : ${NOTMUCH_REMOTE_COMMAND:=notmuch}
>
> printf -v ARGS '%q ' "$@"

This is the real secret sauce.  When I was looking at removing the
extraneous quoting we have in notmuch-show.el, it boiled down to
figuring this out.  The web has a lot of people asking how to do this
properly, but very few good answers.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: add compatability functions for emacs 23

[Matt Armstrong]

David Bremner  writes:

> Mark Walters  writes:
>
>> This is a good point. I think I don't mind too much if they do -- they
>> should see it is provided by notmuch-lib if they do describe-function
>> etc. But maybe bremner would like to comment?
>>
>> However, maybe other packages are doing the same. Thus I think we should
>> not put in a cut down version of read-char-choice but just include the
>> whole command from the emacs24 source. That way if any other package is
>> doing the same load order doesn't matter -- we don't stomp on them and
>> they don't stomp on us.
>
> There is a sort of precedent with the package cl-lib.el.  Of course it's
> not exactly the same since it's mainly providing new names for
> functionality that already existed.

I'm obviously in no place to object strongly, but I like this approach
less the more I think about it.  I've had similar things come back to
bite me too many times in the past.

Here I find Steve Purcell raising the same concern:

https://github.com/melpa/melpa/pull/1748#issuecomment-45082451
https://github.com/swift-emacs/swift-mode/issues/10

I'm concerned entirely about surprises.  Folks don't expect packages to
provide future compatibility functions.  Imagine using Emacs 23 and,
maybe because you're copy/paste hacking your personal elisp, you come to
use setq-local.  Or perhaps you use some *other* add-on that uses
setq-local but makes no effort to support Emacs 23.  Then you decide to
move (require 'notmuch) a little lower in your .emacs and *boom*, your
config is broken.

So I'd rather see this kind of thing:

;; TODO: remove and use setq-local directly when Emacs 23 support is dropped.
(if (fboundp 'setq-local)
(defalias 'notmuch-setq-local 'setq-local)
  (defmacro notmuch-setq-local (var val)
"Set variable VAR to value VAL in current buffer."
`(set (make-local-variable ',var) ,val)))

Then just call notmuch-setq-local from notmuch code.  That follows the
expected convention that packages add symbols under their own namespace.

But, as far as prior art, we have examples of other packages patching in
a setq-local:

https://github.com/flycheck/flycheck/commit/b19bd4fdf118c55a96d72b2b3c034a0393cfcae2
https://github.com/fxbois/web-mode/issues/438

...and I have to agree with Steve when he says "This isn't the most
egregious example, though. :-)"
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: add compatability functions for emacs 23

[Matt Armstrong]

Mark Walters  writes:

> +;; Compatability functions for emacs 23.
> +
> +(unless (fboundp 'setq-local)
> +  (defmacro setq-local (var val)
> +`(set (make-local-variable ',var) ,val)))

A concern I have with this approach is that it modifies symbols outside
the notmuch namespace.  Could people on old Emacsen come to rely on the
newer Emacs interfaces inadvertently, and wonder how they came to be,
etc?

The only other package I have non-trivial experience working with is
Gnus, and the practice there was to place portability interfaces in the
gnus- namespace, and refrain from calling the "native" interfaces
directly.

Outside Gnus, I'm ignorant of what Emacs packages typically do for this
problem.  I have no strong feelings either way.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] emacs: make faces clear on dark backgrounds.

[Matt Armstrong]

Mark Walters  writes:

> Broadly this looks good to me -- though I think blue is OK on some dark
> backgrounds so it would be good to have confirmation from some people
> who do use a dark background normally as to whether they had to
> customise these faces.

I'm happy to wait for further feedback.  See below for what I'm seeing. 


> One small point: I think I would prefer a ((class color) (background
> light)) test for the "light" face so that it is consistent with the other 
> deffaces
> which already distinguish between light and dark background.

I'll send something out in a few days.  I'm thinking, though, that
perhaps options that default "blue" on light backgrounds should default
"light blue" on dark backgrounds.


Attached is a PNG capture of my M-x list-colors-display.  I'm on Ubuntu
trusty.  Attached is a screen capture of what "blue" looks like to me.

-- next part --
A non-text attachment was scrubbed...
Name: calc.png
Type: image/png
Size: 256282 bytes
Desc: blue on a dark background
URL: 

[PATCH v2] emacs: make faces clear on dark backgrounds.

[Matt Armstrong]

The notmuch-tag-flagged, notmuch-search-flagged-face and
notmuch-crypto-part-header faces defaulted to "blue", which is nearly
unreadable when a dark background is in use.  This is addressed by using
"gold" for dark backgrounds.
---
 emacs/notmuch-crypto.el | 5 -
 emacs/notmuch-tag.el| 5 -
 emacs/notmuch.el| 5 -
 3 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-crypto.el b/emacs/notmuch-crypto.el
index e376aa8..3f4b3f6 100644
--- a/emacs/notmuch-crypto.el
+++ b/emacs/notmuch-crypto.el
@@ -42,7 +42,10 @@ mode."
   :group 'notmuch-crypto)
 
 (defface notmuch-crypto-part-header
-  '((t (:foreground "blue")))
+  'class color)
+  (background dark))
+ (:foreground "gold"))
+(t (:foreground "blue")))
   "Face used for crypto parts headers."
   :group 'notmuch-crypto
   :group 'notmuch-faces)
diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el
index 1b2ce5c..199582b 100644
--- a/emacs/notmuch-tag.el
+++ b/emacs/notmuch-tag.el
@@ -115,7 +115,10 @@ Used in the default value of `notmuch-tag-formats`."
   :group 'notmuch-faces)
 
 (defface notmuch-tag-flagged
-  '((t :foreground "blue"))
+  'class color)
+  (background dark))
+ (:foreground "gold"))
+(t :foreground "blue"))
   "Face used for the flagged tag.
 
 Used in the default value of `notmuch-tag-formats`."
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index bd08aa0..9246344 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -313,7 +313,10 @@ there will be called at other points of notmuch execution."
   :group 'notmuch-faces)
 
 (defface notmuch-search-flagged-face
-  '((t
+  'class color)
+  (background dark))
+ (:foreground "gold"))
+(t
  (:foreground "blue")))
   "Face used in search mode face for flagged threads.
 
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] emacs: make faces clear on dark backgrounds.

[Matt Armstrong]

I found a multipart/signed message and verified that indeed the "blue"
button created by way of notmuch-crypto-part-header is unreadable on
dark backgrounds.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: fix notmuch-search-line-faces defcustom

[Matt Armstrong]

Mark Walters  writes:

> In commit 2a7b11b064233afc4feead876fa396e3c18a6b91 the default value
> for notmuch-search-line-faces was changed so that it didn't match the
> specification in the corresponding defcustom. This meant that it was
> difficult for the user to customize this variable as they got a type
> mismatch error.

Mark, thanks for fixing the defcustom that I broke.  :-) I have verified
that it now works for me.  This testing induced me to zero-out my
notmuch face customizations, which reminded me that the defaults are bad
in places for Emacs themes with dark backgrounds.  I'll send a patch for
that.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1] emacs: make faces clear on dark backgrounds.

[Matt Armstrong]

The notmuch-tag-flagged and notmucy-search-flagged-face faces defaulted
to "blue", which is nearly unreadable when a dark background is in use.
This is addressed by using "gold" for dark backgrounds.

There is one remaining unconditional use of "blue" at
notmuch-crypto-part-header, but I don't have any SMIME email, so I'm
unable to see the impact of changing that variable.  I've left it alone.
---
 emacs/notmuch-tag.el | 5 -
 emacs/notmuch.el | 5 -
 2 files changed, 8 insertions(+), 2 deletions(-)

diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el
index 1b2ce5c..199582b 100644
--- a/emacs/notmuch-tag.el
+++ b/emacs/notmuch-tag.el
@@ -115,7 +115,10 @@ Used in the default value of `notmuch-tag-formats`."
   :group 'notmuch-faces)
 
 (defface notmuch-tag-flagged
-  '((t :foreground "blue"))
+  'class color)
+  (background dark))
+ (:foreground "gold"))
+(t :foreground "blue"))
   "Face used for the flagged tag.
 
 Used in the default value of `notmuch-tag-formats`."
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index bd08aa0..9246344 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -313,7 +313,10 @@ there will be called at other points of notmuch execution."
   :group 'notmuch-faces)
 
 (defface notmuch-search-flagged-face
-  '((t
+  'class color)
+  (background dark))
+ (:foreground "gold"))
+(t
  (:foreground "blue")))
   "Face used in search mode face for flagged threads.
 
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4] Add notmuch-show--build-queries.

[Matt Armstrong]

notmuch-show--build-buffer now queries a list of queries built by the
former.  This simplifies the logic.  It also provides an easy place to
experiment with alternate sets of queries for given notmuch-show-*
variables (e.g. users can use advice-add to do so in a surgical way).
---
 emacs/notmuch-show.el | 35 +--
 1 file changed, 21 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 16ff45a..fcf7e6e 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1263,6 +1263,18 @@ matched."
(message "No messages matched the query!")
nil
 
+(defun notmuch-show--build-queries (thread context)
+  "Return a list of queries to try for this search.
+
+THREAD and CONTEXT are both strings, though CONTEXT may be nil.
+When CONTEXT is not nil, the first query is the conjunction of it
+and THREAD.  The next query is THREAD alone, and serves as a
+fallback if the prior matches no messages."
+  (let (queries)
+(push (list thread) queries)
+(if context (push (list thread "and (" context ")") queries))
+queries))
+
 (defun notmuch-show--build-buffer (&optional state)
   "Display messages matching the current buffer context.
 
@@ -1270,25 +1282,20 @@ Apply the previously saved STATE if supplied, otherwise 
show the
 first relevant message.
 
 If no messages match the query return NIL."
-  (let* ((basic-args (list notmuch-show-thread-id))
-(args (if notmuch-show-query-context
-  (append (list "\'") basic-args
-  (list "and (" notmuch-show-query-context ")\'"))
-(append (list "\'") basic-args (list "\'"
-(cli-args (cons "--exclude=false"
+  (let* ((cli-args (cons "--exclude=false"
 (when notmuch-show-elide-non-matching-messages
   (list "--entire-thread=false"
-
-(forest (or (notmuch-query-get-threads (append cli-args args))
-;; If a query context reduced the number of
-;; results to zero, try again without it.
-(and notmuch-show-query-context
- (notmuch-query-get-threads (append cli-args 
basic-args)
-
+(queries (notmuch-show--build-queries
+  notmuch-show-thread-id notmuch-show-query-context))
+(forest nil)
 ;; Must be reset every time we are going to start inserting
 ;; messages into the buffer.
 (notmuch-show-previous-subject ""))
-
+;; Use results from the first query that returns some.
+(while (and (not forest) queries)
+  (setq forest (notmuch-query-get-threads
+   (append cli-args (list "'") (car queries) (list "'"
+  (setq queries (cdr queries)))
 (when forest
   (notmuch-show-insert-forest forest)
 
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3] Add notmuch-show--build-queries.

[Matt Armstrong]

Mark Walters  writes:

>> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el

[...]

>> +(defun notmuch-show--build-queries ()

[...]

> We may want to call this from tree-mode later, so I wonder whether
> passing it notmuch-show-thread-id and notmuch-show-query-context as
> arguments would make sense. I quite like the idea of this function being
> self contained (ie not referencing global state variables). My instinct
> is that referencing truly global variables like customise variables is
> fine, but I am less keen on referencing buffer-local global variables.

Done.

Perhaps a useful rule of thumb is that functions referencing
buffer-local vars should be interactive?  I'm not sure that follows for
all cases, but I certainly agree that global and buffer-local vars
should probably not be referenced gratuitously, and pure functions are
generally good.


> However, some of this depends on what your later plans are.

Calling what I have in mind "plans" may be a stretch.  ;-) I'm not sure
this function is useful for notmuch-tree as written.  At minimum, I
would expect a shared function to have a different name.  But I think
you're right that there is probably opportunity for code sharing,
especially if notmuch grows more complex features with respect to
searching vs. expanding messages in threads.

Look for a v4 of this patch soon.  I appreciate your review so far.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: feature request: highlighting of partial jump sequences

[Matt Armstrong]

Mark Walters  writes:

> On Mon, 17 Oct 2016, Matt Armstrong  wrote:
>> David Bremner  writes:
>>
>>> I (very) recently started using longer key sequences with Mark's
>>> tag-jump feature. One thing I miss from a similar feature in org-mode
>>> (e.g. exporting) is some visual feedback on what I have typed so far,
>>> and thus what my next key is likely to do.
>>
>> Tangentially, has an alternative UI been considered such as something
>> based on ido?  I mention this merely because building on something
>> standard is probably going to be less work and more familiar to users.
>
> The nice feature of jump is that it is very fast to type: I can get to
> my common saved searches in two keystrokes. I think the multistep is
> more specialised and probably more useful for tag-jump -- I imagine
> uses along the lines of k (to enter tag jump) n to enter notmuch::
> prefixed tags, and then a letter for the appropriate notmuch:: tag.
>
>> The other day I was thinking that if no "jumps" are configured, and the
>> user has ido mode on, selecting among all saved searches with ido would
>> be a natural thing for 'j' to do.  Some users (probably me) may then
>> never bother to set up shortcut keys.
>
> I think this would make it very confusing when/if people did add a saved
> jump. If we want to offer something like ido completion then I would
> suggest putting it under a further character in the jump map. eg j j
> would lead to ido completion, and then if people add normal jump short
> cuts everything stays the same. (We would want to make sure that user
> jump binding override this j binding so that we don't break anybodies
> existing setup.)
>
> Best wishes

Mark, no disagreement from me.  I might, someday, hack up something
based on ido (or an ido-like UI) to see how it works out.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: feature request: highlighting of partial jump sequences

[Matt Armstrong]

David Bremner  writes:

> I (very) recently started using longer key sequences with Mark's
> tag-jump feature. One thing I miss from a similar feature in org-mode
> (e.g. exporting) is some visual feedback on what I have typed so far,
> and thus what my next key is likely to do.

Tangentially, has an alternative UI been considered such as something
based on ido?  I mention this merely because building on something
standard is probably going to be less work and more familiar to users.

The other day I was thinking that if no "jumps" are configured, and the
user has ido mode on, selecting among all saved searches with ido would
be a natural thing for 'j' to do.  Some users (probably me) may then
never bother to set up shortcut keys.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re:

[Matt Armstrong]

Matt Armstrong  writes:

> This supercedes
> id:1476207707-21827-1-git-send-email-marmstr...@google.com with
> changes steming from Mark's helpful feedback.

Apologies for the lack of a subject here.  I'm still learning the ins
and outs of 'git send-email'.  I can't say I'd call it a friendly
facility.  :)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] Add notmuch-show--build-queries.

[Matt Armstrong]

notmuch-show--build-buffer now queries a list of queries built by the
former.  This simplifies the logic.  It also provides an easy place to
experiment with alternate sets of queries for given notmuch-show-*
variables (e.g. users can use advice-add to do so in a surgical way).
---
 emacs/notmuch-show.el | 36 ++--
 1 file changed, 22 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index f2487ab..b393c11 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1261,6 +1261,20 @@ matched."
(message "No messages matched the query!")
nil
 
+(defun notmuch-show--build-queries ()
+  "Return a list of queries to try for this search.
+
+If `notmuch-show-query-context` is not nil, the first query is
+the conjunction of it and `notmuch-show-thread-id`.  The next
+query is `notmuch-show-thread-id` alone, and serves as a fallback
+if the prior matches no messages."
+  (let* ((thread notmuch-show-thread-id)
+(context notmuch-show-query-context)
+queries)
+(push (list thread) queries)
+(if context (push (list thread "and (" context ")") queries))
+queries))
+
 (defun notmuch-show--build-buffer (&optional state)
   "Display messages matching the current buffer context.
 
@@ -1268,25 +1282,19 @@ Apply the previously saved STATE if supplied, otherwise 
show the
 first relevant message.
 
 If no messages match the query return NIL."
-  (let* ((basic-args (list notmuch-show-thread-id))
-(args (if notmuch-show-query-context
-  (append (list "\'") basic-args
-  (list "and (" notmuch-show-query-context ")\'"))
-(append (list "\'") basic-args (list "\'"
-(cli-args (cons "--exclude=false"
+  (let* ((cli-args (cons "--exclude=false"
 (when notmuch-show-elide-non-matching-messages
   (list "--entire-thread=false"
-
-(forest (or (notmuch-query-get-threads (append cli-args args))
-;; If a query context reduced the number of
-;; results to zero, try again without it.
-(and notmuch-show-query-context
- (notmuch-query-get-threads (append cli-args 
basic-args)
-
+(queries (notmuch-show--build-queries))
+(forest nil)
 ;; Must be reset every time we are going to start inserting
 ;; messages into the buffer.
 (notmuch-show-previous-subject ""))
-
+;; Use results from the first query that returns some.
+(while (and (not forest) (consp queries))
+  (setq forest (notmuch-query-get-threads
+   (append cli-args (list "'") (car queries) (list "'"
+  (setq queries (cdr queries)))
 (when forest
   (notmuch-show-insert-forest forest)
 
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[no subject]

[Matt Armstrong]

This supercedes
id:1476207707-21827-1-git-send-email-marmstr...@google.com with
changes steming from Mark's helpful feedback.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v2] Add notmuch-show--build-queries.

[Matt Armstrong]

Mark, thanks for the review.  I agree with your comments.  I'll send
another patch.

As you might guess, I'm new to elisp.  Use of push makes some of this
fall out nicely.

For what it is worth, I did experiment with using loop to find the first
functional "forest".  There are several other calls to loop in this
file.  It was something like:

(let* ((cli-args ())
   (forest (loop for query in (notmuch-show--build-queries)
 for forest = (notmuch-query-get-threads
   (append cli-args
   (list "'") query (list "'")))
 until forest
 finally return forest))
  ...)
  ...)


I won't go that route in my next patch (I'm not convinced it is better).
But I also am not particularly invested in any one way to do this.



Mark Walters  writes:

> On Tue, 11 Oct 2016, Matt Armstrong  wrote:
>> notmuch-show--build-buffer now queries a list of queries built by the
>> former.  This simplifies the logic.  It also provides an easy place to
>> experiment with alternate sets of queries for given notmuch-show-*
>> variables (e.g. users can use advice-add to do so in a surgical way).
>
> Hi
>
> I think I like the overall logic, but I do have some comments below.
>
>> ---
>>  emacs/notmuch-show.el | 37 +++--
>>  1 file changed, 23 insertions(+), 14 deletions(-)
>>
>> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
>> index f2487ab..0727319 100644
>> --- a/emacs/notmuch-show.el
>> +++ b/emacs/notmuch-show.el
>> @@ -1261,6 +1261,20 @@ matched."
>>  (message "No messages matched the query!")
>>  nil
>>  
>> +(defun notmuch-show--build-queries ()
>> +  "Return a list of queries to try for this search.
>> +
>> +If `notmuch-show-query-context` is not nil, the first query is
>> +the conjunction of it and `notmuch-show-thread-id`.  The next
>> +query is `notmuch-show-thread-id` alone, and serves as a fallback
>> +if the prior matches no messages."
>> +  (let* ((thread notmuch-show-thread-id)
>> + (context notmuch-show-query-context)
>> + (queries `((,thread
>> +(if context
>> +(setq queries (cons `(,thread "and (" ,context ")") queries)))
>> +queries))
>
> I think it would be better to keep closer to the existing style -- using
> list and append. At least do that as a first patch and then do a change
> as above with ` and , so we can review them separately. Note I find the
> list append style much easier to read, and I am guessing someone else
> did (perhaps cworth?) as they wrote it that way.
>
> I think I would go for something like
>
> (let* ((...
> queries))
>   (push (list thread) queries)
>   (if context (push (list thread "and (" context ")") queries))
>   queries)



>>  (defun notmuch-show--build-buffer (&optional state)
>>"Display messages matching the current buffer context.
>>  
>> @@ -1268,25 +1282,20 @@ Apply the previously saved STATE if supplied, 
>> otherwise show the
>>  first relevant message.
>>  
>>  If no messages match the query return NIL."
>> -  (let* ((basic-args (list notmuch-show-thread-id))
>> - (args (if notmuch-show-query-context
>> -   (append (list "\'") basic-args
>> -   (list "and (" notmuch-show-query-context ")\'"))
>> - (append (list "\'") basic-args (list "\'"
>> - (cli-args (cons "--exclude=false"
>> +  (let* ((cli-args (cons "--exclude=false"
>>   (when notmuch-show-elide-non-matching-messages
>> (list "--entire-thread=false"
>> -
>> - (forest (or (notmuch-query-get-threads (append cli-args args))
>> - ;; If a query context reduced the number of
>> - ;; results to zero, try again without it.
>> - (and notmuch-show-query-context
>> -  (notmuch-query-get-threads (append cli-args 
>> basic-args)
>> -
>> + (queries (notmuch-show--build-queries))
>> + (forest nil)
>>   ;; Must be reset every time we are going to start inserting
>>   ;; messages into the buffer.
>>   (notmuch-show-previous-subject ""))
>> -
>> +;; Use results from the first query that returns some.
>> +(while (and (consp queries)
>> +(not (setq forest
>> +   (notmuch-query-get-threads
>> +`(,@cli-args "'" ,@(car queries) "'")
>> +  (setq queries (cdr queries)))
>
> Similarly here I would avoid the ` , @ . I think I would also suggest
> making pushing the setq outside the while condition. Something like
>
> (while (and (consp queries)
> (not forest))
>(setq forest (notmuch-query-get-threads
>   (append cli-args (list "'") (car queries) (list "'"

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] Add notmuch-show--build-queries.

[Matt Armstrong]

notmuch-show--build-buffer now queries a list of queries built by the
former.  This simplifies the logic.  It also provides an easy place to
experiment with alternate sets of queries for given notmuch-show-*
variables (e.g. users can use advice-add to do so in a surgical way).
---
 emacs/notmuch-show.el | 37 +++--
 1 file changed, 23 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index f2487ab..0727319 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1261,6 +1261,20 @@ matched."
(message "No messages matched the query!")
nil
 
+(defun notmuch-show--build-queries ()
+  "Return a list of queries to try for this search.
+
+If `notmuch-show-query-context` is not nil, the first query is
+the conjunction of it and `notmuch-show-thread-id`.  The next
+query is `notmuch-show-thread-id` alone, and serves as a fallback
+if the prior matches no messages."
+  (let* ((thread notmuch-show-thread-id)
+(context notmuch-show-query-context)
+(queries `((,thread
+(if context
+   (setq queries (cons `(,thread "and (" ,context ")") queries)))
+queries))
+
 (defun notmuch-show--build-buffer (&optional state)
   "Display messages matching the current buffer context.
 
@@ -1268,25 +1282,20 @@ Apply the previously saved STATE if supplied, otherwise 
show the
 first relevant message.
 
 If no messages match the query return NIL."
-  (let* ((basic-args (list notmuch-show-thread-id))
-(args (if notmuch-show-query-context
-  (append (list "\'") basic-args
-  (list "and (" notmuch-show-query-context ")\'"))
-(append (list "\'") basic-args (list "\'"
-(cli-args (cons "--exclude=false"
+  (let* ((cli-args (cons "--exclude=false"
 (when notmuch-show-elide-non-matching-messages
   (list "--entire-thread=false"
-
-(forest (or (notmuch-query-get-threads (append cli-args args))
-;; If a query context reduced the number of
-;; results to zero, try again without it.
-(and notmuch-show-query-context
- (notmuch-query-get-threads (append cli-args 
basic-args)
-
+(queries (notmuch-show--build-queries))
+(forest nil)
 ;; Must be reset every time we are going to start inserting
 ;; messages into the buffer.
 (notmuch-show-previous-subject ""))
-
+;; Use results from the first query that returns some.
+(while (and (consp queries)
+   (not (setq forest
+  (notmuch-query-get-threads
+   `(,@cli-args "'" ,@(car queries) "'")
+  (setq queries (cdr queries)))
 (when forest
   (notmuch-show-insert-forest forest)
 
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] emacs: Add notmuch-show--build-queries.

[Matt Armstrong]

This supercedes
id:1474003701-19831-1-git-send-email-marmstr...@google.com with a much
simpler patch.  My goal here is to make it easier to tweak
notmuch-show behavior without hacking on notmuch-show--build-buffer
itself, since it is responsible for a host of other tasks.

I'm still working (slowly) on coming up with a nice mental model for
the way notmuch's Emacs interface treats threads -vs- messages with
respect to "expansion" in show and tree results.  Although, to be
honest, wiht this patch and the following piece of advice,
notmuch-show buffers are doing what I want in all cases:

(defun my-notmuch-show--prepend-tag-unread (queries)
  "Prepend a tag:unread query to QUERIES.

Intended to be used as :filter-return advice on
`notmuch-show--build-queries'."
  (cons `(,notmuch-show-thread-id "and tag:unread") queries))

(advice-add 'notmuch-show--build-queries :filter-return
#'my-notmuch-show--prepend-tag-unread)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: add tag jump menu

[Matt Armstrong]

David Bremner  writes:

> Mark Walters  writes:
>
> If we think about the operation as reverse rather than undo

I'd be happy without undo/reverse in the first version of this feature.
I have a few key bindings I've set up and I have not bothered to set up
"reverse/undo" for them.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [WIP PATCH] emacs: add "--" to finish notmuch command line args before search terms

[Matt Armstrong]

Tomi Ollila  writes:

> I don't (yet) know how complete this is (Someone's "workflow" it may break)
> but tests pass...

[...]

> -(defun notmuch-query-get-threads (search-terms)
> +(defun notmuch-query-get-threads (cli-args search-terms)

I'm new here and still have a low reputation :-) but fwiw, I like this.
I especially the change to the notmuch-query-search-args function.
I think it cleans up the code a bit, and makes it more flexible.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: tag deleted face bugfix

[Matt Armstrong]

(notmuch-apply-face tag
   (if (display-supports-face-attributes-p'(:strike-through "red"))
   '(:strike-through "red")
 '(:inverse-video t)))

Mark Walters  writes:

> Commit d25d33ff cleaned up some of the tag face code. However, for the
> face notmuch-tag-deleted it used the test
>
> ((class color) (supports :strike-through))
>
> to decide whether to use red strikethrough or inverse-video (emacs in
> a terminal typically doesn't support red strikethrough, but in X it does).
>
> However, it seems that test often returns true even though red
> strikethrough is not supported. This breaks the tag update code -- the
> wrong thing is displayed to the user.
>
> Thus we make the test explicitly more specific, changing the test to
>
> ((class color) (supports :strike-through "red"))
> ---
>
> Tomi found this bug today, and narrowed it down to a recent notmuch
> change. This seems to fix it, and the code now seems to work as
> expected in terminals and in X. However I am not an expert on emacs
> faces so there may be a better way.
>
> Best wishes
>
> Mark

Mark, thanks, the patch looks correct to me.  I apologize for the
regression.

My commit d25d33ff intended to adapt following literal face spec in the
call to notmuch-apply-face into a new face declared with defface:

(notmuch-apply-face
 tag (if (display-supports-face-attributes-p
  '(:strike-through "red"))
 '(:strike-through "red")
   '(:inverse-video t)))

Which would be what you've done in this patch:

(((class color) (supports :strike-through "red")) :strike-through "red")

I omitted the "red" portion of the supports string, which looks to be an
error.

Info docs for face attributes has:

‘:strike-through’
 Whether or not characters should be strike-through, and in what
 color.  The value is used like that of ‘:overline’.

This suggests that supplying a color is appropriate.





> emacs/notmuch-tag.el | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el
> index 644ce40..e59f148 100644
> --- a/emacs/notmuch-tag.el
> +++ b/emacs/notmuch-tag.el
> @@ -137,7 +137,7 @@ with images."
>:type 'notmuch-tag-format-type)
>  
>  (defface notmuch-tag-deleted
> -  'class color) (supports :strike-through)) :strike-through "red")
> +  'class color) (supports :strike-through "red")) :strike-through "red")
>  (t :inverse-video t))
>"Face used to display deleted tags.
>  
> -- 
> 2.1.4
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: notmuch-show: remove extraneous shell quoting

[Matt Armstrong]

Mark Walters  writes:

> Hi
>
> On Fri, 16 Sep 2016, Matt Armstrong  wrote:
>> Tomi, thanks for your reply asking for some motivation behind this
>> patch.  I can't reply directly to your message because, for some reason,
>> it doesn't appear in my mailbox (I discovered your message while reading
>> the mail archive on notmuchmail.org).
>>
>> The code dealing with this quoting issue was last touched in commit
>> b57d9635f50d5e9b53092218e81f6d2c391c363e, where Carl recognizes the
>> quoting is a bit of a hack and asks for a better fix.  This is my
>> attempt.
>>
>> I am motivated by a concern for code health.  I saw the quoting, did not
>> understand it, recognized it as probably wrong, investigated how the
>> quotes were actually passed from Emacs to the shell, and still believed
>> it wrong.
>>
>> I think this kind of flaw can be placed in the category of security fix.
>> Quoting issues often are.  But, I'm not a security person.
>
> I think all the data being passed is generated by notmuch so I don't
> see a security issue.

The search phrase is entered by the user.  Imagine a user searching for
an exact substring they copy/paste from source code, etc.  It is
contrived, but it isn't hard to imagine users using single quotes in
their searches.


>> By my reasoning, the rationale for the change is simple:
>>
>> a) It is the job of notmuch elisp to pass call-process the args in an
>> appropriate manner for notmuch-command (which is always a local
>> command).  Because call-process takes a list of strings, and no shell is
>> involved, using shell quotes is wrong.  It just so happens that Xapian
>> ignores the quotes, but taking advantage of that is not a great thing.
>>
>> b) If notmuch-command is doing something fancy, as is the case with the
>> "remote" script on https://notmuchmail.org/remoteusage/, it is the job
>> of that script to quote its own args properly for ssh.  It looks like it
>> already does this.
>
> That one script does -- there are at least two others even on the wiki
> (see the links at the bottom of the above page) -- they also seem to be
> fine. But there could be other user scripts that do need the quoting.

I agree that there is risk to the change.  However, the scripts that
require the quoting from notmuch.el are already broken.  They will break
whenever a user happens to include a single quote in their query.


> So the question is do we mind breaking a few currently working setups
> for the purpose of a mild cleanup. Since the current code is confusing I
> think a comment would be in order if we don't apply this patch.

Yes, some debate about the importance of the cleanup, but I agree that
if we do keep the quoting it deserves some comment.



>> So, the quoting is unnecessary on both accounts.
>>
>>
>> Matt Armstrong  writes:
>>
>>> Remove shell quoting from notmuch-show--build-buffer.  The args list
>>> is ultimately passed to call-process, which passes them verbatim to
>>> the subprocess (typically, notmuch).  The quoting, intended for a
>>> shell, is unnecessary and confusing.
>> ___
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v1] WIP: emacs: show: expand unread tags

[Matt Armstrong]

Mark Walters  writes:

> Hi
>
> On Fri, 16 Sep 2016, Matt Armstrong  wrote:
>> Expand unread messages by default in show mode.
>>
>> Show mode now expands messages matching tags designated by
>> notmuch-show-unread-tags (in addition to those matching the search
>> query).  By default, this list is `("unread").  This way, one can still
>> tag and search for messages however one likes, but if the thread is
>> displayed those messages matching the search term OR unread are
>> expanded.
>
> Broadly I like the idea.
>
>> See id:qf54m70o7h5@marmstrong-linux.kir.corp.google.com for a
>> related discussion thread.
>>
>> What happens today when notmuch-show does the query to show a thread, it
>> takes the user's query Q and constructs:
>>
>>   thread:THREAD and Q
>>
>> This changes the above to:
>>
>>   thread:THREAD and ((Q) or (tag:unread))
>
> This seems a reasonable way of doing this. If you are doing it this way
> then presumably the extra query (tag:unread in the above) could be any
> query, not just a tag query? This might make sense, as it would be
> analogous to count-queries in notmuch-hello. 

I like the idea of a query.  Carl had said he preferred the tag based
approach, but this was when contrasted with an approach where the entire
search query was replaced with an entirely different "expansion query".
See id:qf54m70o7h5@marmstrong-linux.kir.corp.google.com.

> I will mention a couple of possible extensions as it might be worth
> making sure they are possible even if they aren't implemented yet. One
> is that the user might want to not open matching messages: that is to
> have the thread opened as e.g., thread:THREAD and (tag:unread)
>
> Another is that we might want the this extra search to be settable on a
> per saved search basis. Saved searches are a plist so that should be
> easy.

Yes, these second two suggestions are similar to the ideas Jani
presented on the original discussion thread.  I like them.

>> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
>> index 5a585f3..f20eec6 100644
>> --- a/emacs/notmuch-show.el
>> +++ b/emacs/notmuch-show.el
>> @@ -142,10 +142,21 @@ indentation."
>>  notmuch-show-interactively-view-part)))
>>  
>>  (defcustom notmuch-show-only-matching-messages nil
>> -  "Only matching messages are shown by default."
>> +  "When non-nil only matching messages are shown.
>> +
>> +See `notmuch-search-show-thread'."
>>:type 'boolean
>
> This seems to be unrelated so perhaps omit (or split into a separate patch).

Agreed, will do.


>>:group 'notmuch-show)
>>  
>> +(defcustom notmuch-show-unread-tags '("unread")
>> +  "A list of tags to expand by default in `notmuch-show-mode'.
>
> Even if you stick with just allowing tags I think the name here should
> not involve unread as it is more general than that. However, since it
> doesn't appear to be any more difficult, I would suggest that you allow
> more general queries.

Agreed, will do.


>> +In addition to the search query, any messages that match one of
>> +the tags in the list are expanded.  Has no effect if
>> +`notmuch-show-only-matching-messages', which see."
>> +  :type '(repeat string)
>> +  :group 'notmuch-show)
>> +
>>  ;; By default, block all external images to prevent privacy leaks and
>>  ;; potential attacks.
>>  (defcustom notmuch-show-text/html-blocked-images "."
>> @@ -1258,6 +1269,24 @@ matched."
>>  (message "No messages matched the query!")
>>  nil
>>  
>> +(defun notmuch-show--query-or (a b)
>> +  "Form a query string \"(A) or (B)\".
>> +
>> +The args must be strings. If either is nil, returns the other."
>> +  (cond
>> +   ((null a) b)
>> +   ((null b) a)
>> +   (t (concat "(" a ") or (" b ")"
>> +
>> +(defun notmuch-show--query-and (a b)
>> +  "Form a query string \"(A) and (B)\".
>> +
>> +The args must be strings. If either is nil, returns the other."
>> +  (cond
>> +   ((null a) b)
>> +   ((null b) a)
>> +   (t (concat "(" a ") and (" b ")"
>> +
>>  (defun notmuch-show--build-buffer (&optional state)
>>"Display messages matching the current buffer context.
>>  
>> @@ -1265,21 +1294,29 @@ Apply the previously saved STATE if supplied, 
>> otherwise show the
>>  first relevant me

Re: notmuch.el: controlling what does and doesn't get expanded in searches

[Matt Armstrong]

Jani Nikula  writes:

> On Thu, 04 Aug 2016, Carl Worth  wrote:
>>>  i) notmuch could have an "also expand tags" feature, where thread based
>>> results would auto expand matching tags.  I would set this to
>>> "unread".
>>
>> This approach makes a lot of sense to me based on how notmuch.el works.
>
> My idea on how to do this: I'd like to have a key binding in the show
> view to go through a customizable list of rules on how to
> collapse/expand the messages. The rules could be:
>
> * [ ] expand all matching messages
>   [ ] expand messages having any of the specified tags
>   [ ] expand messages having all of the specified tags
> * expand all messages
> * collapse all messages
>
> (* are mutually exclusive, [ ] are not)
>
> The first rule would define what is displayed by default. So you could
> have, for example, "expand all matching messages and any messages that
> have both inbox and unread tags", followed by "expand all matching
> messages", followed by "expand messages that have inbox tag", followed
> by "expand all messages", etc. any way you wish.
>
> It would be a nice bonus if you could specify at which rule to start per
> each saved search, instead of the first in the list.
>
> I think this could replace the current M-RET and C-u M-RET
> expand/collapse all bindings. Maybe M-RET could be reused for this.
>
> This would obviously not require any changes to the SPC, n, p or other
> navigation bindings, which I think are currently just fine.

I've begun a WIP patch series for this at
id:1474003701-19831-1-git-send-email-marmstr...@google.com under subject
"emacs: show: expand unread tags"

Jani, compared to your idea what I have is very simple.  notmuch-show
retains just the two modes when initially displaying a thread,
controlled by notmuch-show-elide-non-matching-messages.  This is usually
toggled with a prefix arg to RET (e.g. the command that shows a thread).

It employs a search you didn't include explicitly in your list:

* [ ] expand all matching messages and all messages having
  any of the specified flags.

I find it quite useful.

As for your proposal, I'll respond to each:

> * [ ] expand all matching messages

Agreed.  This is the default today.

>   [ ] expand messages having any of the specified tags

See above: I think it makes sense to expand the both the matching
messages and those matching the additional expansion tags
(e.g. tag:unread).

>   [ ] expand messages having all of the specified tags

In the context of a configuration for notmuch-search-messages, this
feels overly flexible.  You could just as well include the required tags
in the original query.  It may make sense as a global default?

I call this out because I think as soon as the feature allows expressing
both a tag conjunction (sequence of AND) and tag disjunction (OR), I
begin to wonder why we don't just ask the user to supply a query.

Perhaps you had a use case in mind?

> * expand all messages
> * collapse all messages

Agreed. These are available today.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: notmuch-show: remove extraneous shell quoting

[Matt Armstrong]

Tomi, thanks for your reply asking for some motivation behind this
patch.  I can't reply directly to your message because, for some reason,
it doesn't appear in my mailbox (I discovered your message while reading
the mail archive on notmuchmail.org).

The code dealing with this quoting issue was last touched in commit
b57d9635f50d5e9b53092218e81f6d2c391c363e, where Carl recognizes the
quoting is a bit of a hack and asks for a better fix.  This is my
attempt.

I am motivated by a concern for code health.  I saw the quoting, did not
understand it, recognized it as probably wrong, investigated how the
quotes were actually passed from Emacs to the shell, and still believed
it wrong.

I think this kind of flaw can be placed in the category of security fix.
Quoting issues often are.  But, I'm not a security person.

By my reasoning, the rationale for the change is simple:

a) It is the job of notmuch elisp to pass call-process the args in an
appropriate manner for notmuch-command (which is always a local
command).  Because call-process takes a list of strings, and no shell is
involved, using shell quotes is wrong.  It just so happens that Xapian
ignores the quotes, but taking advantage of that is not a great thing.

b) If notmuch-command is doing something fancy, as is the case with the
"remote" script on https://notmuchmail.org/remoteusage/, it is the job
of that script to quote its own args properly for ssh.  It looks like it
already does this.

So, the quoting is unnecessary on both accounts.


Matt Armstrong  writes:

> Remove shell quoting from notmuch-show--build-buffer.  The args list
> is ultimately passed to call-process, which passes them verbatim to
> the subprocess (typically, notmuch).  The quoting, intended for a
> shell, is unnecessary and confusing.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1] WIP: emacs: show: expand unread tags

[Matt Armstrong]

Expand unread messages by default in show mode.

Show mode now expands messages matching tags designated by
notmuch-show-unread-tags (in addition to those matching the search
query).  By default, this list is `("unread").  This way, one can still
tag and search for messages however one likes, but if the thread is
displayed those messages matching the search term OR unread are
expanded.

---

See id:qf54m70o7h5@marmstrong-linux.kir.corp.google.com for a
related discussion thread.

What happens today when notmuch-show does the query to show a thread, it
takes the user's query Q and constructs:

  thread:THREAD and Q

This changes the above to:

  thread:THREAD and ((Q) or (tag:unread))

I've found this to be a much better day to day experience for myself and
am interesting on iterating on this idea until it is can be
committed.  (settling on interface first, then testing)

As I explain in the thread noted above, I found the default notmuch
experience to be quite confusing, at least when processing "inbox" mail.

The problem is that my work experience is based on a large number of
mailing lists that cross post to each other, often initiated mid-thread.
I aggressively archive these threads as "unread", but sometimes a long
thread is cross-posted to my team mailing list, or to me directly.  If
these show up in my "team" or "me" tags, I want to have a clear
indication that I have yet to read the entire thread.  Expanding
"tag:unread" messages makes sense here.

It might make less sense if drilling down for those specific messages.
In that case I wonder if C-u RET (to toggle
notmuch-show-elide-non-matching-messages) is sufficient?
---
 emacs/notmuch-show.el | 59 +--
 emacs/notmuch.el  |  4 
 2 files changed, 52 insertions(+), 11 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 5a585f3..f20eec6 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -142,10 +142,21 @@ indentation."
notmuch-show-interactively-view-part)))
 
 (defcustom notmuch-show-only-matching-messages nil
-  "Only matching messages are shown by default."
+  "When non-nil only matching messages are shown.
+
+See `notmuch-search-show-thread'."
   :type 'boolean
   :group 'notmuch-show)
 
+(defcustom notmuch-show-unread-tags '("unread")
+  "A list of tags to expand by default in `notmuch-show-mode'.
+
+In addition to the search query, any messages that match one of
+the tags in the list are expanded.  Has no effect if
+`notmuch-show-only-matching-messages', which see."
+  :type '(repeat string)
+  :group 'notmuch-show)
+
 ;; By default, block all external images to prevent privacy leaks and
 ;; potential attacks.
 (defcustom notmuch-show-text/html-blocked-images "."
@@ -1258,6 +1269,24 @@ matched."
(message "No messages matched the query!")
nil
 
+(defun notmuch-show--query-or (a b)
+  "Form a query string \"(A) or (B)\".
+
+The args must be strings. If either is nil, returns the other."
+  (cond
+   ((null a) b)
+   ((null b) a)
+   (t (concat "(" a ") or (" b ")"
+
+(defun notmuch-show--query-and (a b)
+  "Form a query string \"(A) and (B)\".
+
+The args must be strings. If either is nil, returns the other."
+  (cond
+   ((null a) b)
+   ((null b) a)
+   (t (concat "(" a ") and (" b ")"
+
 (defun notmuch-show--build-buffer (&optional state)
   "Display messages matching the current buffer context.
 
@@ -1265,21 +1294,29 @@ Apply the previously saved STATE if supplied, otherwise 
show the
 first relevant message.
 
 If no messages match the query return NIL."
-  (let* ((basic-args (list notmuch-show-thread-id))
-(args (if notmuch-show-query-context
-  (append (list "\'") basic-args
-  (list "and (" notmuch-show-query-context ")\'"))
-(append (list "\'") basic-args (list "\'"
+  (let* ((unread (if notmuch-show-unread-tags
+(mapconcat
+ '(lambda (tag) (concat "tag:" tag))
+ notmuch-show-unread-tags
+ " or ")))
+(basic-query (notmuch-show--query-or
+  notmuch-show-thread-id unread))
+(context-query (if notmuch-show-query-context
+   (notmuch-show--query-and
+notmuch-show-thread-id
+(notmuch-show--query-or
+ notmuch-show-query-context
+ unread
 (cli-args (cons "--exclude=false"
 (when notmuch-show-elide-non-matching-messages
   (list "--entire-thread=false"
-
-(forest (or (notmuch-query-get-threads (append cli-args args))
+(forest (or (and context-query
+ (notmuch-query-get-threads
+  (append cli-args (list "\'" context-query "\'"
 ;; If a

Re: [PATCH] emacs: notmuch-show: remove extraneous shell quoting

[Matt Armstrong]

Sorry, ignore this one.
id:1473834053-17591-1-git-send-email-marmstr...@google.com has some
typo fixes to the commit message.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: notmuch-show: remove extraneous shell quoting

[Matt Armstrong]

Remove shell quoting from notmuch-show--build-buffer.  The args list
is ultimately passed to call-process, which passes them verbatim to
the subprocess (typically, notmuch).  The quoting, intended for a
shell, is unnecessary and confusing.

This code originally appeared in
9193455fa1476ea3957475e77379b75efa6dd90b, aiming to support remote
notmuch execution over ssh.  The commit message included this example
shell script (simplified):

  #!/bin/sh
  ssh remotehost notmuch $@

That script doesn't properly quote "$@".  Even if it did, it wouldn't
work because the shell ssh starts on the remote host does another
round of shell word splitting.  Fortunately, notmuch already provides
an example of a correct shell script on
https://notmuchmail.org/remoteusage/ (again simplified):

  #!/bin/bash
  printf -v ARGS "%q " "$@"
  ssh remotehost notmuch $ARGS

With the above script I'm able to use notmuch in "remote" mode.
Quoted search terms work as expected, etc.

---
 emacs/notmuch-show.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 5a585f3..da2d685 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1267,9 +1267,9 @@ first relevant message.
 If no messages match the query return NIL."
   (let* ((basic-args (list notmuch-show-thread-id))
 (args (if notmuch-show-query-context
-  (append (list "\'") basic-args
-  (list "and (" notmuch-show-query-context ")\'"))
-(append (list "\'") basic-args (list "\'"
+  (append basic-args
+  (list "and (" notmuch-show-query-context ")"))
+basic-args))
 (cli-args (cons "--exclude=false"
 (when notmuch-show-elide-non-matching-messages
   (list "--entire-thread=false"
-- 
tg: (89c8d27..) t/remote-quoting (depends on: master)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: notmuch-show: remove extraneous shell quoting

[Matt Armstrong]

Remove shell quoting from notmuch-show--build-buffer.  The args list
is ultimately passed to call-process, which passes them verbatim to
the subprocess (typically, notmuch).  The quoting, intended for a
shell, are unnecessary and confusing.

This code originally appeared in
9193455fa1476ea3957475e77379b75efa6dd90b, aiming to support remote
notmuch execution over ssh.  The commit message included this example
shell script (simplified):

  #!/bin/sh
  ssh remotehost notmuch $@

That script doesn't properly quote "$@".  Even if it did, it wouldn't
work because the shell ssh starts on the remote host does another
round of shell word splitting.  Fortunately, notmuch already provides
an example of a correct shell script on
https://notmuchmail.org/remoteusage/ (again simplified):

  #!/bin/bash
  printf -v ARGS "%q " "$@"
  ssh remotehost notmuch $ARGS

With the above script I'm able to use notmuch in "remote" mode.
Quoted search terms work as expected, etc.

(this is my t/remote-quoting)

---
 emacs/notmuch-show.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 5a585f3..da2d685 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1267,9 +1267,9 @@ first relevant message.
 If no messages match the query return NIL."
   (let* ((basic-args (list notmuch-show-thread-id))
 (args (if notmuch-show-query-context
-  (append (list "\'") basic-args
-  (list "and (" notmuch-show-query-context ")\'"))
-(append (list "\'") basic-args (list "\'"
+  (append basic-args
+  (list "and (" notmuch-show-query-context ")"))
+basic-args))
 (cli-args (cons "--exclude=false"
 (when notmuch-show-elide-non-matching-messages
   (list "--entire-thread=false"
-- 
tg: (89c8d27..) t/remote-quoting (depends on: master)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: make the remaining faces configurable.

[Matt Armstrong]

I believe this moves all "anonymous" face specifications in notmuch
code into a configurable defface.
---
 NEWS |  8 
 emacs/notmuch-tag.el | 45 ++---
 2 files changed, 38 insertions(+), 15 deletions(-)

diff --git a/NEWS b/NEWS
index 3a9c8d3..2ff78e9 100644
--- a/NEWS
+++ b/NEWS
@@ -6,10 +6,10 @@ Emacs
 
 Face customization is easier
 
-  New faces `notmuch-search-flagged-face` and
-  `notmuch-search-unread-face` are used by default by
-  `notmuch-search-line-faces`. Customize `notmuch-faces` to modify
-  them.
+  New faces `notmuch-tag-unread`, `notmuch-tag-flagged`,
+  `notmuch-tag-deleted`, `notmuch-tag-added`,
+  `notmuch-search-flagged-face` and `notmuch-search-unread-face` are
+  now used by default. Customize `notmuch-faces` to modify them.
 
 Ruby Bindings
 -
diff --git a/emacs/notmuch-tag.el b/emacs/notmuch-tag.el
index a3f0c52..ec3c964 100644
--- a/emacs/notmuch-tag.el
+++ b/emacs/notmuch-tag.el
@@ -56,9 +56,23 @@
  (string :tag "Custom")))
(sexp :tag "Custom")
 
+(defface notmuch-tag-unread
+  '((t :foreground "red"))
+  "Default face used for the unread tag.
+
+Used in the default value of `notmuch-tag-formats`."
+  :group 'notmuch-faces)
+
+(defface notmuch-tag-flagged
+  '((t :foreground "blue"))
+  "Face used for the flagged tag.
+
+Used in the default value of `notmuch-tag-formats`."
+  :group 'notmuch-faces)
+
 (defcustom notmuch-tag-formats
-  '(("unread" (propertize tag 'face '(:foreground "red")))
-("flagged" (propertize tag 'face '(:foreground "blue"))
+  '(("unread" (propertize tag 'face 'notmuch-tag-unread))
+("flagged" (propertize tag 'face 'notmuch-tag-flagged)
  (notmuch-tag-format-image-data tag (notmuch-tag-star-icon
   "Custom formats for individual tags.
 
@@ -90,15 +104,17 @@ with images."
   :group 'notmuch-faces
   :type 'notmuch-tag-format-type)
 
+(defface notmuch-tag-deleted
+  'class color) (supports :strike-through)) :strike-through "red")
+(t :inverse-video t))
+  "Face used to display deleted tags.
+
+Used in the default value of `notmuch-tag-deleted-formats`."
+  :group 'notmuch-faces)
+
 (defcustom notmuch-tag-deleted-formats
-  '(("unread" (notmuch-apply-face bare-tag
- (if (display-supports-face-attributes-p 
'(:strike-through "red"))
- '(:strike-through "red")
-   '(:inverse-video t
-(".*" (notmuch-apply-face tag
- (if (display-supports-face-attributes-p 
'(:strike-through "red"))
- '(:strike-through "red")
-   '(:inverse-video t)
+  '(("unread" (notmuch-apply-face bare-tag `notmuch-tag-deleted))
+(".*" (notmuch-apply-face tag `notmuch-tag-deleted)))
   "Custom formats for tags when deleted.
 
 For deleted tags the formats in `notmuch-tag-formats` are applied
@@ -118,8 +134,15 @@ See `notmuch-tag-formats' for full documentation."
   :group 'notmuch-faces
   :type 'notmuch-tag-format-type)
 
+(defface notmuch-tag-added
+  '((t :underline "green"))
+  "Default face used for added tags.
+
+Used in the default value for `notmuch-tag-added-formats`."
+  :group 'notmuch-faces)
+
 (defcustom notmuch-tag-added-formats
-  '((".*" (notmuch-apply-face tag '(:underline "green"
+  '((".*" (notmuch-apply-face tag 'notmuch-tag-added)))
   "Custom formats for tags when added.
 
 For added tags the formats in `notmuch-tag-formats` are applied
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] emacs: implement notmuch-search-color-line with dolist.

[Matt Armstrong]

While passing a lambda to mapc is idiomatic elisp, dolist is easier
to understand, and there are a few other calls to it in this file.
---
 emacs/notmuch.el | 13 ++---
 1 file changed, 6 insertions(+), 7 deletions(-)

diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 43d56f7..6307b37 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -700,13 +700,12 @@ background."
 
 (defun notmuch-search-color-line (start end line-tag-list)
   "Colorize lines in `notmuch-show' based on tags."
-  (mapc (lambda (elem)
- (let ((tag (car elem))
-   (attributes (cdr elem)))
-   (when (member tag line-tag-list)
- (notmuch-apply-face nil attributes nil start end
-   ;; Reverse the list so earlier entries take precedence
-   (reverse notmuch-search-line-faces)))
+  ;; Reverse the list so earlier entries take precedence
+  (dolist (elem (reverse notmuch-search-line-faces))
+(let ((tag (car elem))
+ (face (cdr elem)))
+  (when (member tag line-tag-list)
+   (notmuch-apply-face nil face nil start end)
 
 (defun notmuch-search-author-propertize (authors)
   "Split `authors' into matching and non-matching authors and
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Hi all

[Matt Armstrong]

Ken Stevens  writes:

> I am new to notmuch. I am using the emacs interface for notmuch. I would
> like to be able to use bbdb to manage my contacts. Is their a simple way
> to integrate bbdb and notmuch so I cann add addresses, etc.

https://notmuchmail.org/emacstips/ has some coverage for bbdb in the
"Address completion when composing" section.  I haven't tried it myself.
I'm using notmuch's native address completion (based on the indexed
content of my email) with some success.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

"snoozing" with notmuch?

[Matt Armstrong]

My mail yesterday about muting
(id:qf5vazjjciq@marmstrong-linux.kir.corp.google.com) was in part
motivated by this question:

Has anybody implemented something like "snooze" in notmuch?

I think of a "snooze" as a temporary mute.

The intent would be to hide messages from the inbox until some future
time, upon which they are automatically promoted back into the inbox.

A more precise spec:

 - if a new message arrives into a "snoozed" thread it too is snoozed
   and is not tagged 'inbox'.

 - (perhaps optinoally) unless it is "to me", which always
   auto-unsnoozes the threads it is part of.

 - when unsnoozed, or when a snooze expires, messages in the thread are
   tagged with inbox.

The difficulty I have is that notmuch doesn't support tags with
timestamps or numerical values.  It might be possible to snooze based on
fixed intervals.  E.g. one unique "snoozed until" tag per day of the
year.  Using this would require some UI facade.


Prior art that I'm aware of:

a) Google Inbox has snooze.  You choose a time and the thread appears in
   your inbox as "new mail", as if it arrived then.  It appears to be
   implemented as a tag on the thread that suppresses message display by
   the client (e.g. if you view your "Google Inbox" mail in Gmail the
   "snoozed" messages appear there as normal).

b) in a past life I filtered low priority mail into hourly and daily
   temporary mailboxes, and had cron entries set up to re-incorporate
   those as incoming mail on an hourly/daily basis.  This was a great
   way to encourage myself to batch-process less important mail.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Flat search and threaded views

[Matt Armstrong]

Yuri D'Elia  writes:

> For example, for a query like "tag:unread AND date:24h..now", I'm shown
> all threads containing unread messages within the last day, which is
> perfect. But when I select a thread (with RET), I'm shown the thread
> from the start.

Yuri, I see you're running into issues similar to what I raised today in
id:qf57fbw4fx4@marmstrong-linux.kir.corp.google.com

So far I think notmuch attemps to give you a useful
https://en.wikipedia.org/wiki/DWIM default behavior with respect to how
it handles threads, but both you and I find it to be surprising or even
undesirable in certain situations.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch.el: controlling what does and doesn't get expanded in searches

[Matt Armstrong]

Jani Nikula  writes:

> On Thu, 04 Aug 2016, Matt Armstrong  wrote:
>> This question pertains to notmuch built from recent git HEAD, using
>> notmuch.el in show mode (i.e. not tree mode).
>>
>> I sometimes read a thread with a bunch of messages and notmuch.el
>> collapses a bunch of them (even if unread and the search matches tags
>> in every message).  I can't figure out the heuristic notmuch is
>> applying here.
>
> The idea is that all the messages matching the query are expanded, the
> others collapsed. Each expanded message must fully match the query. The
> unread or inbox tags are not special in this regard.
>
> I am not saying this is ideal, but this is how it's supposed to
> work. (Indeed I'd personally like to define e.g. saved search specific
> tags or queries to use for deciding which messages to expand.)

Thank you.

Yes, I find the query semantics with respect to tags and threads a bit
confusing at times.  This is not a problem specific to notmuch, as I
find the same kinds of issues in GMail.  Usually the problem occurs at
the semantic border between per-message tags and thread-based
operations.

I can now explain what I am seeing.  In my notmuch-post-hook I tag
messages according to various categories (mailing lists, etc.).  Every
time I do this I also tag them with 'filtered'.  My
`notmuch-saved-searches` has about 15 sub-queries of the form "tag:inbox
AND tag:".

I also have a saved search to catch the "unfiltered" stuff:

  tag:inbox AND tag:unfiltered

So this occurred:

1) One message was sent to a foo-announce mailing list.  This was not
   caught by my filters, so it was simply tagged 'inbox'.

2) All replies were sent to the main "foo" mailing list, which *was*
   caught by my filters and tagged 'inbox' and 'foo' and 'filtered'.

This is bad for two reasons:

a) If I observed this thread by searching for 'tag:inbox AND tag:foo',
   the initial foo-announce message would not be expanded.  But, as the
   first message in the thread, it is the most important!

b) If I observed this thread by searching for 'tag:inbox AND not
   tag:filtered' (as I do to find all the "uncategorized" stuff in my
   inbox), then the foo-announce mail is expanded but none of the
   others.

This problem isn't specific to my 'filtered' tag approach.  I think it
generalizes to any approach that attempts to split incoming mail.

I've been seeing this problem quite frequently because I'm in an
environment where messages are cc:'d to different mailing lists all the
time.  It is common for threads to be cc'd to new mailing lists
mid-thread, or for people to pull lists off the cc: list mid-thread
(sometimes into private per-person distribution).

In this environment, different messages in those threads area expanded
depending on which query I use to find them.  This is undesirable
because, generally, I want to read every message I have not yet read in
these threads.


>> In particular, pressing SPC does not seem to navigate to the collapsed
>> messages (again, even if they are unread).
>
> SPC and n and p are supposed to navigate expanded messages only. N and P
> navigate all messages (but do not expand by default). Again, the tags
> the messages have do not matter. You can manually expand/collapse
> messages, and that'll affect the navigation.

Note that SPC also archives, and when it does, it archives the entire
thread, not just the expanded messages (i.e. those that match the
current search).  So, this gave rise to the above situation, where I
pressed SPC twice and archived a 40 message thread, with most messages
still unread.


>> Worst case: only the first messages is initially expanded and all
>> subsequent are collapsed.  I press SPC and the cursor goes to the end
>> of search results.  SPC again all the entire thread is archived.
>>
>> This behavior has caused me to accidentally skip messages.  First
>> step for me is understanding what is going on so I can fix it.
>
> Yes, let's first check that notmuch behaves as it is expected, and
> then figure out how to improve it.

I think the semantics make coherent sense for ad-hoc searches.  Things
break down for me when considering an inbox processing workflow heavily
based on archiving.

If I return to a thread after reading the first N messages I need not
see those messages expanded.  I have already read them, so I'd prefer
they be collapsed.  Not much usually does this for me because I archive
aggressively, but I don't always archive.  In these cases I think I'd
prefer expansion instead be based on whether I've read the message
(tag:unread).

Also, I do want threads consisting entirely of read messages to appear
in my inbox sear

notmuch.el: controlling what does and doesn't get expanded in searches

[Matt Armstrong]

This question pertains to notmuch built from recent git HEAD, using
notmuch.el in show mode (i.e. not tree mode).

I sometimes read a thread with a bunch of messages and notmuch.el
collapses a bunch of them (even if unread and the search matches tags in
every message).  I can't figure out the heuristic notmuch is applying
here.  In particular, pressing SPC does not seem to navigate to the
collapsed messages (again, even if they are unread).

Worst case: only the first messages is initially expanded and all
subsequent are collapsed.  I press SPC and the cursor goes to the end of
search results.  SPC again all the entire thread is archived.

This behavior has caused me to accidentally skip messages.  First step
for me is understanding what is going on so I can fix it.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Linking a privately built -lxapian

[Matt Armstrong]

I've got a privately built xapian 4.0 in my $HOME/opt/xapian-core-1.4.0
dir, and its bin dir in my path.

xapian-config is working like this:

% xapian-config --libs
-L/usr/local/google/home/marmstrong/opt/xapian-core-1.4.0/lib -lxapian

Then "./configure; make" doesn't produce a functioning build unless I
run notmuch's make one of these two ways:

1) make LDFLAGS=-Wl,-rpath-link,$HOME/opt/xapian-core-1.4.0/lib
2) LD_RUN_PATH=$HOME/opt/xapian-core-1.4.0/lib make

I can live with that, but it brings back decade old memories.  Is there
a better way?

(I don't want to get into packaging up a xapian 1.4 .deb and installing
it on my box just to build notmuch)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch and "mute" -- useful to anyone?

[Matt Armstrong]

David Bremner  writes:

> Matt Armstrong  writes:
>
>> Is anyone else interested in Gmail-like "mute" support in notmuch.el?
>> If so, I can think about polishing the below off and adding it to
>> notmuch.
>>
>> I've managed to implement Gmail's "mute" in notmuch as follows in my
>> notmuch-post-new:
>>
>> --
>> # Unmute all threads with new messages sent to me.
>> notmuch search --output=threads tag:new AND tag:me | \
>>   xargs --no-run-if-empty notmuch tag -muted --
>>
>> # Remove all muted threads from the inbox and mark every message in them
>> # muted.  Ideally this would be atomic with the above.
>> notmuch search --output=threads tag:muted | \
>>   xargs --no-run-if-empty notmuch tag -inbox +muted --
>> --
>
> See also the example of https://notmuchmail.org/excluding/
>
> This kind of thing is what message exclusion is invented for.
>
> I guess you have to adjust to get precisely the semantics you want where
> messages specifically to you are not muted.

"mute" is a bit odd.  It is essentially "new messages in this thread
skip the inbox".  It is a property of a thread, not a message.

It has little to do with excluding messages from arbitrary searches.  In
particular, muted threads should show up in normal free text searches.
I don't think message exclusion is the right mechanism.

The spec is: when a message arrives,
  if it is "to me" unmute any thread it is part of
  otherwise if it is in a muted thread, avoid tagging it "inbox"

This is a little bit hard to deal with in a the notmuch model, which
deals more naturally with mechanisms that tag individual messages and
handles threading as a post-search message gathering step (as far as I
can tell).

I'm not claiming that what I've got is optimal -- it feels like a hack
to me, but is the best I came up with.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: A systematic way of handling Xapian lock errors?

[Matt Armstrong]

David Bremner  writes:

> Matt Armstrong  writes:
>
>
>> To address both, has something like this ever been considered:
>>
>>   notmuch --lock_timeout  COMMAND ARG...
>>
>> Then frontends like Emacs could lean on retry logic written in C.  If
>> this seems workable, it is something I can try implementing.
>
> notmuch in git master uses xapians lock retry if present (xapian
> 1.3+). Currently there is no timeout, but Olly seemed receptive to the
> idea of adding one, so I'd suggest that's the right place to work on
> improving this. You may find that just having unbounded lock retries
> (i.e. blocking opens) is enough for your issues.

Ah great.  I am still on Xapian 1.2 by way of running a relatively old
Linux distro.  Building a personal version of Xapian as we speak...
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] Add Google Inc. to AUTHORS as a contributor.

[Matt Armstrong]

Nicolas Petton  writes:

> [ Unknown signature status ]
> Matt Armstrong  writes:
>
>>> The AUTHORS file is not unmaintained, why are you saying that?
>>
>> I was going by David's related comment where he said the file had not
>> changed in seven years:
>> id:1469191654-5922-1-git-send-email-da...@tethera.net.  So, it is not
>> maintained as an authoritative list of copyright holders
>
> The only AUTHORS file I know about is etc/AUTHORS, which was last
> updated on the emacs-25 branch by commit d08afa1 one week ago.

Are you thinking of emacs?  This thread is about notmuch, not emacs.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

notmuch and "mute" -- useful to anyone?

[Matt Armstrong]

Is anyone else interested in Gmail-like "mute" support in notmuch.el?
If so, I can think about polishing the below off and adding it to
notmuch.

I've managed to implement Gmail's "mute" in notmuch as follows in my
notmuch-post-new:

--
# Unmute all threads with new messages sent to me.
notmuch search --output=threads tag:new AND tag:me | \
  xargs --no-run-if-empty notmuch tag -muted --

# Remove all muted threads from the inbox and mark every message in them
# muted.  Ideally this would be atomic with the above.
notmuch search --output=threads tag:muted | \
  xargs --no-run-if-empty notmuch tag -inbox +muted --
--

Then in .emacs:

--
(defcustom my-notmuch-mute-tags '("+muted" "-inbox")
  "List of tag changes to apply to a message or a thread when it is muted.

Tags starting with \"+\" (or not starting with either \"+\" or
\"-\") in the list will be added, and tags starting with \"-\"
will be removed from the message or thread being archived.

For example, if you wanted to remove an \"inbox\" tag and add an
\"archived\" tag, you would set:
(\"-inbox\" \"+archived\")"
  :type '(repeat string)
  :group 'notmuch-search
  :group 'notmuch-show)

;; TODO: consider defadvice?
(defun my-notmuch-search-mute-thread (&optional unarchive beg end)
  "Mute the currently selected thread or region.

Mute  each message in the currently selected thread by applying the
tag changes in `my-notmuch-mute-tags' to each (remove the \"inbox\"
tag by default). If a prefix argument is given, the messages will
be \"unarchived\" (i.e. the tag changes in `my-notmuch-mute-tags'
will be reversed).

This function advances the next thread when finished."
  (interactive (cons current-prefix-arg (notmuch-search-interactive-region)))
  (let ((notmuch-archive-tags my-notmuch-mute-tags))
(notmuch-search-archive-thread unarchive beg end)))

(defun my-notmuch-show-mute-thread-then-next ()
  (interactive)
  "Mute all messages in the current buffer, then show next thread from search."
  (let ((notmuch-archive-tags my-notmuch-mute-tags))
(notmuch-show-archive-thread-then-next)))

(define-key notmuch-search-mode-map "m" 'my-notmuch-search-mute-thread)
(define-key notmuch-show-mode-map "m" 'my-notmuch-show-mute-thread-then-next)
--
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

A systematic way of handling Xapian lock errors?

[Matt Armstrong]

Simple notmuch commands like "notmuch tag" can fail to grab the Xapian
lock.  When this occurs they bail with:

A Xapian exception occurred opening database: Unable to get write lock on 
/example/.notmuch/xapian: already locked

I've noticed a few issues with this:

1) The notmuch command line doesn't define semantics for exit codes.  I
notice that notmuch exits with 1 for "xapian write lock error".  I
suspect this code is not reserved for write lock errors?  That would be
needed for any sensible downgrade of the write lock error into a soft
failure with a retry loop in things like shell scripts.

2) The notmuch Emacs client just bails whatever it was trying when this
occurs.  I run background mail fetching, which I expect is typical, and
this makes for an unpleasant experience as errors are thrown at me
randomly.  Since my post-injection filter commands run quickly, I'd
rather Emacs simply spin a short while against the lock trying to
perform the command.  Even a one second spin attempt is likely to
succeed most of the time.  But I would happily configure an infinite
retry loop -- Emacs can show me a spinning ball, and I'd rather not be
subject to odd effects due to failures.

To address both, has something like this ever been considered:

  notmuch --lock_timeout  COMMAND ARG...

Then frontends like Emacs could lean on retry logic written in C.  If
this seems workable, it is something I can try implementing.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] Add Google Inc. to AUTHORS as a contributor.

[Matt Armstrong]

Nicolas Petton  writes:

> [ Unknown signature status ]
> Matt Armstrong  writes:
>
>> Looking into this further, if AUTHORS file is essentially unmaintained,
>> I need not touch it.
>
> The AUTHORS file is not unmaintained, why are you saying that?

I was going by David's related comment where he said the file had not
changed in seven years:
id:1469191654-5922-1-git-send-email-da...@tethera.net.  So, it is not
maintained as an authoritative list of copyright holders, as is typical
in fastidiously maintained GNU software (i.e. those projects fully
subscribing to the FSF standards).  See
https://www.gnu.org/prep/maintain/html_node/Recording-Contributors.html

Personally, I think this is fine.  In the era of reliable DVCS that keep
revision history forever and have commands like git-annotate, AUTHORS
files in the GNU tradition are probably obsolete.  But, I'm not a
lawyer.  :-)
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] Add Google Inc. to AUTHORS as a contributor.

[Matt Armstrong]

David Bremner  writes:

> Never argue with a happy lawyer, that's motto :). It sounds like we're
> good to go; I merged patch 2/2 after moving the NEWS. Then I wanted to
> shorten the summary line of the commit, which got a bit more intrusive
> than I wanted, hopefully it conveys the intent of the existing message.

Thanks David, especially for your patience with how to best handle the
copyright questions.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH 1/2] Add Google Inc. to AUTHORS as a contributor.

[Matt Armstrong]

David Bremner  writes:

> Matt Armstrong  writes:
>
>> By virtue of being a Google employee I'm required to contribute
>> software in the name of Google and not myself.

[...]

> I've no objection to this in principle, but
>
> 1) if we start updating AUTHORS after 7 years of neglect, I think we
> need to do a more systematic job.

Looking into this further, if AUTHORS file is essentially unmaintained,
I need not touch it.


> I'm tempted to script the output of "git shortlog" as part of the
> release process.
>
> In id:1469191654-5922-1-git-send-email-da...@tethera.net, I've posted
> the beginnings of a mailmap file. What about adding something like
>
> Google Inc. (by way of Matt Armstrong)  
>
> to the mailmap file?
>
> If that sounds reasonable to you, I'll leave it to you to play with the
> mailmap syntax to get what you want.

I think the @google.com in my email address is enough.  I don't need to
tweak my display name to mention "Google Inc." via .mailcap.


> 2) If you think the corporate overlords are fine with a mention of
> contribution in AUTHORS, that's also fine for me, but note that this is
> distinct from a claim of copyright.  There are copyright headers in
> every source file, so if you want to claim copyright on behalf of your
> employer (which is fine, I think we're not fussed about people adding
> themselves to copyright headers for anything more than 2 lines of change
> or so) then you'll want to update those headers. At least, that would be
> that safer option, rather than thinking such a claim is implicit.

Thanks for thinking about this David.

The good news: in looking into this further, the issue of leaving (c)
claims in code is secondary.  The primary thing I'm supposed to do is
send patches with my work email.

If I become a significant author in some file, adding a (c) Google makes
sense if that the normal thing to do in notmuch.  I'll keep that in
mind.

Google's open source lawyers tell me that regardless of what (c) marks
appear in code, the author of code (e.g. significant patches) retains
copyright unless other arrangements are made, such as those you see in
FSF projects (https://www.gnu.org/licenses/why-assign.en.html).  When it
comes to a real legal investigation things like the git commit log come
into play.  A "(c) Google Inc." is nice to have, especially if a project
typically maintains them with some rigor, but it isn't required or
particularly significant.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] emacs: express notmuch-search-line-faces in terms of two new faces: notmuch-search-flagged-face and notmuch-search-unread-face.

[Matt Armstrong]

This makes it easier to find the relevant face by customizing
notmuch-faces.  I plan to do the same to the other alists of faces
found elsewhere.
---
 NEWS |  7 +++
 emacs/notmuch.el | 39 ---
 2 files changed, 39 insertions(+), 7 deletions(-)

diff --git a/NEWS b/NEWS
index ab661d4..49f8bb5 100644
--- a/NEWS
+++ b/NEWS
@@ -37,6 +37,13 @@ Respect charset of MIME parts when reading them
 
   Fix previous assumption that everyone uses UTF-8.
 
+Face customization is easier
+
+  New faces `notmuch-search-flagged-face` and
+  `notmuch-search-unread-face` are used by default by
+  `notmuch-search-line-faces`. Customize `notmuch-faces` to modify
+  them.
+
 Notmuch 0.22 (2016-04-26)
 =
 
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 8acdef3..43d56f7 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -311,6 +311,26 @@ there will be called at other points of notmuch execution."
   :group 'notmuch-search
   :group 'notmuch-faces)
 
+(defface notmuch-search-flagged-face
+  '((t
+ (:weight bold)))
+  "Face used in search mode face for flagged threads.
+
+This face is the default value for the \"flagged\" tag in
+`notmuch-search-line-faces`."
+  :group 'notmuch-search
+  :group 'notmuch-faces)
+
+(defface notmuch-search-unread-face
+  '((t
+ (:foreground "blue")))
+  "Face used in search mode for unread threads.
+
+This face is the default value for the \"unread\" tag in
+`notmuch-search-line-faces`."
+  :group 'notmuch-search
+  :group 'notmuch-faces)
+
 (defun notmuch-search-mode ()
   "Major mode displaying results of a notmuch search.
 
@@ -654,9 +674,12 @@ of the result."
  (goto-char (point-min))
  (forward-line (1- notmuch-search-target-line)
 
-(defcustom notmuch-search-line-faces '(("unread" :weight bold)
-  ("flagged" :foreground "blue"))
-  "Tag/face mapping for line highlighting in notmuch-search.
+(defcustom notmuch-search-line-faces
+  '(("unread" 'notmuch-search-unread-face)
+("flagged" 'notmuch-search-flagged-face))
+  "Alist of tags to faces for line highlighting in notmuch-search.
+Each element looks like (TAG . FACE).
+A thread with TAG will have FACE applied.
 
 Here is an example of how to color search results based on tags.
  (the following text would be placed in your ~/.emacs file):
@@ -665,10 +688,12 @@ Here is an example of how to color search results based 
on tags.
(\"deleted\" . (:foreground \"red\"
  :background \"blue\"
 
-The attributes defined for matching tags are merged, with earlier
-attributes overriding later. A message having both \"deleted\"
-and \"unread\" tags with the above settings would have a green
-foreground and blue background."
+The FACE must be a face name (a symbol or string), a property
+list of face attributes, or a list of these.  The faces for
+matching tags are merged, with earlier attributes overriding
+later. A message having both \"deleted\" and \"unread\" tags with
+the above settings would have a green foreground and blue
+background."
   :type '(alist :key-type (string) :value-type (custom-face-edit))
   :group 'notmuch-search
   :group 'notmuch-faces)
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/2] Add Google Inc. to AUTHORS as a contributor.

[Matt Armstrong]

By virtue of being a Google employee I'm required to contribute
software in the name of Google and not myself.  Google retains
whatever rights (and liabilities) that come from having done so, not
myself.  As long as this is clear, and my association with Google is
clear, my employer is quite supportive of my making these
contributions.
---
 AUTHORS | 1 +
 1 file changed, 1 insertion(+)

diff --git a/AUTHORS b/AUTHORS
index 6d0f2de..51d40b4 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -28,3 +28,4 @@ ideas, inspiration, testing or feedback):
 Martin Krafft
 Keith Packard
 Jamey Sharp
+Google Inc. (by way of Matt Armstrong )
-- 
2.8.0.rc3.226.g39d4020

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Improving face customization in notmuch.el

[Matt Armstrong]

Hi folks, this is my first contribution to the project.  I'm new to
the git workflow, so go easy on me.  I also know next to nothing about
elisp.  ;-)

I work for Google, which strongly supports employees contributing to
OSS projects.  Some minor legal red tape must occur because it must be
clear that Google is the legal entity that owns whatever rights (and
liabilities!) exists for the work I do.  So, I use
marmstr...@google.com and not my personal mail, and the contributor
line should say Google and not "Matt Armstrong".  Of course, given
this is a GPL project, those rights are limited and well defined.

I can, if needed, supply whatever "Contributor License Agreement" is
needed if notmuch maintains these (as the FSF does with what they
typically call "papers").

My goal here is to make the out-of-box experience better for
notmuch.el users with respect to the choice of faces.  I find that on
terminals with few colors the fonts used by notmuch are often bad
(e.g. black text on a black background).  I had a heck of a time
figuring out how some of these face choices were made.  I want to both
improve the default faces and make them easier to discover and
customize.

Trivia: I wrote https://github.com/matta/rubymail 15 years ago before
I joined Google.  RubyMail was used by "sup" after sup's author
interned at Google and learned the power of the tags+threading+search
email model.  And then, sup inspired notmuch.  If you squint hard
enough you could say I'm already a notmuch contributor, it is just
that all the code I wrote has been deleted.  ;-)

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch