Database locked when running hooks

[Matthew Lear]

Hello all,
I recently updated to notmuch 0.31.4+137~g6967dcb and made some changes to the 
way I sync my mail using lieer.
I wanted to utilise the pre-new and post-new notmuch hooks.
However, when running an operation to import mail in the context of the notmuch 
pre-new hook (by running gmi pull), I can see that the database is locked, 
which means that running notmuch new hangs.
From man (5) notmuch-hooks the wording is such that it implies that the 
database is not locked when the pre-new and post-new hooks are run.

If I strace notmuch new, I can see that the database is opened for writing 
before the pre-new hook is run, and it is not closed before the hook is run.
This means that any operation which needs write access to database (e.g. 
lieer’s gmi pull) cannot obtain the database lock, so it hangs.

The problem can be reproduced by creating $DATABASEDIR/.notmuch/hooks/pre-new 
and having it perform a notmuch CLI command to write to the DB, such as apply a 
tag.

I described the problem in https://github.com/gauteh/lieer/issues/195 


Assuming the database should be able to be opened for writing in the context of 
the pre-new (and post-new) hooks, this seems like a bug.
Here is some output...

Running notmuch new before the pre-new hook is executed:

..
openat(AT_FDCWD, "/home/myhome/path/to/maildir//.notmuch/xapian/flintlock", 
O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 3
fcntl(3, F_OFD_SETLKW, {l_type=F_WRLCK, l_whence=SEEK_SET, l_start=0, l_len=1}) 
= 0
openat(AT_FDCWD, "/home/ml943601/b/mail/brcm//.notmuch/xapian/iamglass", 
O_RDONLY) = 4
..

It’s never closed before the hook is executed.
What the hang looks like:

 [~]
$ notmuch new
path: /home/myhome/path/to/maildir/gmaildir/
pull: partial synchronization.. (hid: 66446860)
fetching changes fetching changes fetching changes fetching changes 
...resolving changes (376) receiving 
content (156) ...^C
Traceback (most recent call last):
  File "/home/myhome/git/lieer/gmi", line 24, in 
g.main ()
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 215, in main
args.func (args)
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 418, in pull
self.partial_pull ()
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 548, in partial_pull
updated = self.get_content (message_gids)
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 754, in get_content
self.remote.get_messages (need_content, _got_msgs, 'raw')
  File "/home/myhome/git/lieer/lieer/remote.py", line 133, in func_wrap
return func (self, *args, **kwargs)
  File "/home/myhome/git/lieer/lieer/remote.py", line 384, in get_messages
cb (msg_batch)
  File "/home/myhome/git/lieer/lieer/gmailieer.py", line 749, in _got_msgs
with notmuch.Database (mode = notmuch.Database.MODE.READ_WRITE) as db:
  File "/usr/local/lib/python3.8/dist-packages/notmuch/database.py", line 164, 
in __init__
self.open(path, mode)
  File "/usr/local/lib/python3.8/dist-packages/notmuch/database.py", line 223, 
in open
status = Database._open(_str(path), mode, byref(db))

There is no other program running at the time of the hang which accesses the 
database.
The problem seems to be caused by the database being opened for writing before 
the hook is executed.

Does notmuch require the database to be open when calling the hook?

Cheers,
—  Matt

___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

[PATCH] Clarify the syntax required when searching using timestamps.

[Matthew Lear]

Need to be clearer about specifying time ranges using timestamps.
Legacy syntax which predates the date prefix is still supported, but
timestamps used in conjunction with the date prefix require additional
syntax.
---
 doc/man7/notmuch-search-terms.rst | 14 +++---
 1 file changed, 11 insertions(+), 3 deletions(-)

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 6d2bf62a..b4d784a3 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -121,13 +121,14 @@ date:.. or date:
 expression, and supported syntax for  and  date and
 time expressions.
 
-The time range can also be specified using timestamps with a
-syntax of:
+The time range can also be specified using timestamps without
+including the date prefix using a syntax of:
 
 ..
 
 Each timestamp is a number representing the number of seconds
-since 1970-01-01 00:00:00 UTC.
+since 1970-01-01 00:00:00 UTC. Specifying a time range this way
+is considered legacy and predates the date prefix.
 
 lastmod:..
 The **lastmod:** prefix can be used to restrict the result by the
@@ -296,6 +297,13 @@ In this case,  is taken as the earliest time it 
could describe
 could describe (the end of yesterday). Similarly, date:january..february
 matches from the beginning of January to the end of February.
 
+If specifying a time range using timestamps in conjunction with the
+date prefix, each timestamp must be preceded by @ (ASCII hex 40). As
+above, each timestamp is a number representing the number of seconds
+since 1970-01-01 00:00:00 UTC. For example:
+
+date:@..@
+
 date:..! can be used as a shorthand for date:... The
 expansion takes place before interpretation, and thus, for example,
 date:monday..! matches from the beginning of Monday until the end of
-- 
2.14.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: trouble searching with unix timestamps

[Matthew Lear]

Sure thing. Just done it now.

On Wed, Jan 31, 2018 at 3:04 AM, Daniel Kahn Gillmor <d...@fifthhorseman.net>
wrote:

> On Mon 2018-01-29 16:13:38 +, Matthew Lear wrote:
> > [ Jani wrote: ]
> >> Alternatively, you can use @ in date: queries, although this
> >> seems to be completely undocumented:
> >
> > I guess this relates to similar syntax requirements of date -d in that a
> > unix timestamp is required to be preceded by an @
>
> agreed, it seems in line with that.
>
> Matthew, since you've been reading the docs there most recently, maybe
> you'd be up for sending a patch to make sure this feature is findable in
> the documentation?
>
> --dkg
>
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[no subject]

[Matthew Lear]

From: Matthew Lear <m...@bubblegen.co.uk>
To: notmuch@notmuchmail.org
Cc: Matthew Lear <m...@bubblegen.co.uk>
Subject: [PATCH] Update date search syntax.
Date: Thu,  1 Feb 2018 20:52:18 +
Message-Id: <20180201205218.4368-1-m...@bubblegen.co.uk>
X-Mailer: git-send-email 2.14.1

If searching using the date prefix and timestamps, each timestamp
is required to be prefixed with an @
Legacy syntax of .. without the
date prefix is still honoured, only without the @ specifiers.
---
 doc/man7/notmuch-search-terms.rst | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 6d2bf62a..b6e7079a 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -124,10 +124,13 @@ date:.. or date:
 The time range can also be specified using timestamps with a
 syntax of:
 
-..
+@..@
 
 Each timestamp is a number representing the number of seconds
-since 1970-01-01 00:00:00 UTC.
+since 1970-01-01 00:00:00 UTC. A date range search using
+timestamps is also permitted without using the date prefix and
+@ specifiers, although this is considered legacy and pre-dates
+the date prefix.
 
 lastmod:..
 The **lastmod:** prefix can be used to restrict the result by the
-- 
2.14.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: trouble searching with unix timestamps

[Matthew Lear]

Wonderful! Thanks Jani.

> Alternatively, you can use @ in date: queries, although this
> seems to be completely undocumented:

I guess this relates to similar syntax requirements of date -d in that a
unix timestamp is required to be preceded by an @

# date -d 1517152333
date: invalid date ‘1517152333’
# date -d @1517152333
Sun Jan 28 15:12:13 GMT 2018

On Mon, Jan 29, 2018 at 4:03 PM, Jani Nikula <j...@nikula.org> wrote:

> On Mon, 29 Jan 2018, Matthew Lear <m...@bubblegen.co.uk> wrote:
> > Hi. I've noticed that I'm unable to notmuch search using date with unix
> > timestamps (ie number of seconds since Jan 1st 1970 UTC). I get a xapian
> > error. The notmuch man pages state that searching using
> > 'date:..'
> > where "each timestamp is a number representing the number of seconds
> since
> > 1970-01-01 00:00:00 UTC" is supported, so I'm wondering what's wrong with
> > my installation.
>
> The man page could use some clarification.
>
> The .. syntax predates the date:
> prefix, and needs to be given as-is:
>
> $ notmuch search 1517152333..1517238733
>
> Alternatively, you can use @ in date: queries, although this
> seems to be completely undocumented:
>
> $ notmuch search date:@1517152333..@1517238733
>
> HTH,
> Jani.
>
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

trouble searching with unix timestamps

[Matthew Lear]

Hi. I've noticed that I'm unable to notmuch search using date with unix
timestamps (ie number of seconds since Jan 1st 1970 UTC). I get a xapian
error. The notmuch man pages state that searching using
'date:..'
where "each timestamp is a number representing the number of seconds since
1970-01-01 00:00:00 UTC" is supported, so I'm wondering what's wrong with
my installation.

* notmuch config get built_with.field_processor
true
* xapian-config - xapian-core 1.4.4
* notmuch 0.26+8~ga727e7d

Here's an example:

# date +%s
1517238733

# date -d @1517238733
Mon Jan 29 15:12:13 GMT 2018

# date -d @1517152333
Sun Jan 28 15:12:13 GMT 2018


# notmuch search date:1517152333..1517238733
notmuch search: A Xapian exception occurred
A Xapian exception occurred parsing query: Didn't understand date
specification '1517152333..1517238733'
Query string was: date:1517152333..1517238733

Yet I can do:

# notmuch search date:yesterday..today
or
# notmuch search date:1/28/2018..1/29/2018
or
# notmuch search date:20180128..20180129

..and all show 216 results.

Any thoughts as to why Xapian doesn't like this type of query? Is it just
me? :-)

Thanks,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

[Matthew Lear]

On Wed, 1 Nov 2017, 14:38 Brian Sniffen, <b...@evenmere.org> wrote:

> On Oct 31, 2017, at 5:32 PM, Matthew Lear <m...@bubblegen.co.uk> wrote
>
> I’ll push some fixes to encoding to github later today.
>

I've just tested with e3ba123 and there is a significant improvement. Also,
attachments can be accessed from html too. And it's even faster than
before! Sweet :-) I've still got a couple of mails which don't throw
exceptions but do have issues when rendered in the browser (encoding of the
body generates unreadable characters). I'll try to send you some examples
tomorrow.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

[Matthew Lear]

Comparing with the Haskell based notmuch-web, while slightly slower to
render a browser page with the same search terms as nmweb, I can view the
email which causes nmweb to throw the encoding exception just fine in it. I
guess something in that implementation is able to handle encoding
differently.
Regards,
  Matt

On Tue, 31 Oct 2017, 21:32 Matthew Lear, <m...@bubblegen.co.uk> wrote:

> On Tue, Oct 31, 2017 at 7:21 PM, Brian Sniffen <b...@evenmere.org> wrote:
>
>>
>> > I'm no Python expert, but from a quick google it would seem like the
>> cause
>> > of such an exception is related to not using utf-8.
>>
>> Neat.  So to get there, this has to be a text/html part.  It has to have
>> been decoded, either with the declared content type or with ascii.  If a
>> \u201c (left double quote) showed up, it didn't get decoded as
>> ascii---and indeed, it looks like the content-type specifies latin-1.
>> But now when we try to encode back, using the same latin-1, it fails?
>> That's really neat.
>>
>> > Brian - do you think something needs modifying in nmweb.py to cater for
>> > this type of thing, or is this somehow related my own mailstore (not
>> sure
>> > why that would be as my messages haven't been modified).
>>
>> Lots of mail has busted encoding.  I've done some defensive work against
>> that---look at decodeAnyway and shed a tear for purity---but clearly not
>> enough.  Can you send me a message that causes the problem?
>>
>
> I'll need to fix up the text in the message because it's confidential.
> That should be easy enough to do.
> I'll send it to you once I've done that.
>
> One other thing - it looks like accessing attachments should work, but
> I've seen messages in my local set up here which have attachments shown,
> but I've not been able to retrieve them.
> Not sure what would cause that. Also some messages which are tagged as
> having attachments, don't have them shown by nmweb.
>
> FWIW this link (
> https://nmweb.evenmere.org/show/CACMMjMLecmXopb8AATjE3UuCnNLOO%2B5Nmev5X8K-UostDEUdrQ%40mail.gmail.com)
> has the tag attachment applied to the message, but there is no attachment
> shown.  And another (
> https://nmweb.evenmere.org/show/87d31artti.fsf%40inf-8657.int-evry.fr).
>
> Maybe text/plain only emails are the ones which aren't problematic w.r.t.
> having their attachments shown?
> Cheers,
> --  Matt
>
>
>
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

[Matthew Lear]

On Tue, Oct 31, 2017 at 7:21 PM, Brian Sniffen  wrote:

>
> > I'm no Python expert, but from a quick google it would seem like the
> cause
> > of such an exception is related to not using utf-8.
>
> Neat.  So to get there, this has to be a text/html part.  It has to have
> been decoded, either with the declared content type or with ascii.  If a
> \u201c (left double quote) showed up, it didn't get decoded as
> ascii---and indeed, it looks like the content-type specifies latin-1.
> But now when we try to encode back, using the same latin-1, it fails?
> That's really neat.
>
> > Brian - do you think something needs modifying in nmweb.py to cater for
> > this type of thing, or is this somehow related my own mailstore (not sure
> > why that would be as my messages haven't been modified).
>
> Lots of mail has busted encoding.  I've done some defensive work against
> that---look at decodeAnyway and shed a tear for purity---but clearly not
> enough.  Can you send me a message that causes the problem?
>

I'll need to fix up the text in the message because it's confidential. That
should be easy enough to do.
I'll send it to you once I've done that.

One other thing - it looks like accessing attachments should work, but I've
seen messages in my local set up here which have attachments shown, but
I've not been able to retrieve them.
Not sure what would cause that. Also some messages which are tagged as
having attachments, don't have them shown by nmweb.

FWIW this link (
https://nmweb.evenmere.org/show/CACMMjMLecmXopb8AATjE3UuCnNLOO%2B5Nmev5X8K-UostDEUdrQ%40mail.gmail.com)
has the tag attachment applied to the message, but there is no attachment
shown.  And another (
https://nmweb.evenmere.org/show/87d31artti.fsf%40inf-8657.int-evry.fr).

Maybe text/plain only emails are the ones which aren't problematic w.r.t.
having their attachments shown?
Cheers,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

[Matthew Lear]

On Fri, 27 Oct 2017, 18:52 Brian Sniffen,  wrote:

> Daniel Kahn Gillmor  writes:
>
> > On Fri 2017-10-27 00:04:21 -0400, Brian Sniffen wrote:
> >> With bleach integrated (all of five lines), I think this is safe enough
> >> to let random notmuch users run it.
> >
> > hm, bleach might be a little too aggressive.
> >
> > jrollins just pointed toward:
> >
> > https://nmweb.evenmere.org/show/87innmvvam.fsf%40ligo.caltech.edu
>
> That's fixed in 53403ecd, and there's some examples of bleach on a rope
> at
> https://nmweb.evenmere.org/show/20141107190321.GL23609%40odin.tremily.us
>
> The mbox URL is linkified, the many other link-like texts aren't.
>
>
> Next/prev links are at the bottom, and a thread listing.  I haven't
> thought through how to get the body delivered immediately, but speed
> seems acceptable.  Next up, some style revisions---and I'd love
> proposals for something that looks less awful, or at least makes the
> interface more clear.  UI design is a strong anti-specialty for me.
>

I've been running this today - standalone on localhost port 80 with the
built in CherryPy web server on my mail store. First impressions are it's
terrific :-) As my intended 'target' mail store will be geared towards a
'internal work stuff knowledge collection', lots of emails contain html
links to intranet pages and sites. I can adapt the bleach usage to suit (or
just remove it), but along the way of searching and viewing mail, I've
encountered quite a few occurrences of failing to UnicodeEncode. An example
backtrace looks like this:

Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/web/application.py", line 239, in
process
return self.handle()
  File "/usr/lib/python2.7/dist-packages/web/application.py", line 230, in
handle
return self._delegate(fn, self.fvars, args)
  File "/usr/lib/python2.7/dist-packages/web/application.py", line 420, in
_delegate
return handle_class(cls)
  File "/usr/lib/python2.7/dist-packages/web/application.py", line 396, in
handle_class
return tocall(*args)
  File "/b/git/notmuch-brians.git/contrib/notmuch-web/nmweb.py", line 153,
in GET
sprefix=webprefix)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 989,
in render
return self.environment.handle_exception(exc_info, True)
  File "/usr/lib/python2.7/dist-packages/jinja2/environment.py", line 754,
in handle_exception
reraise(exc_type, exc_value, tb)
  File "templates/show.html", line 1, in top-level template code
{% extends "base.html" %}
  File "templates/base.html", line 32, in top-level template code
{% block content %}
  File "templates/show.html", line 12, in block "content"
{% for part in format_message(m.get_filename(),mid): %}{{ part|safe
}}{% endfor %}
  File "/b/git/notmuch-brians.git/contrib/notmuch-web/nmweb.py", line 245,
in format_message_walk
tags=safe_tags).encode(part.get_content_charset('ascii')))
UnicodeEncodeError: 'latin-1' codec can't encode character u'\u201c' in
position 1141: ordinal not in range(256)

127.0.0.1:60968 - - [31/Oct/2017 17:00:02] "HTTP/1.1 GET /show/
665d8c5c2b024898ae21951c4b8b4...@co2pr05mb747.namprd05.prod.outlook.com" -
500 Internal Server Error

I'm no Python expert, but from a quick google it would seem like the cause
of such an exception is related to not using utf-8.

Brian - do you think something needs modifying in nmweb.py to cater for
this type of thing, or is this somehow related my own mailstore (not sure
why that would be as my messages haven't been modified).
Cheers,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

[Matthew Lear]

I've had a play with this this morning. It's great! The speed and page
loading efficiency is fantastic. Would be really nice if we could go
next/previous in the thread (yes I know I'm complaining about one extra
mouse click). Also, if I select a date via the drop down I need to delete
the timestamp that appears prior to searching, otherwise there is a xapian
error.
This is definitely a candidate solution for me, though.
Thanks Brian!
Cheers,
  Matt

On Fri, 27 Oct 2017, 05:24 Daniel Kahn Gillmor, 
wrote:

> On Fri 2017-10-27 00:04:21 -0400, Brian Sniffen wrote:
> > Thanks!  The part I'm happiest about is the speed:
>
> amen, it feels very lightweight.
>
> > Very careful examination would have shown that the em-dashes between
> > author and subject were red for matches.  Now matches are in italics.
>
> cool.  perhaps assigning a class to those elements and stashing some CSS
> would make that easier for folks to experiment with (and probably reduce
> the bytecount transfered)?
>
> or would that hurt the rendering time for some reason i'm unaware of?  i
> haven't thought about these mechanics as much as you have.
>
> > Yup.  The thread object isn't accessible by then: it existed in the
> > scope of the search query, and is gone by the time we show the message.
> > get_replies isn't available.  So what's the alternative?
> > get_thread_id(), search for that thread id, identify this message *in*
> > that thread id, and then link to the next message with a "next" link?
> > While doing it, why not show the thread structure at the bottom of the
> > message, I guess.
>
> yep, i think that's right.
>
> > With bleach integrated (all of five lines), I think this is safe enough
> > to let random notmuch users run it.  The worst they'll do is expose
> > their mailstore on tcp/8080.  Any interest in taking this into the
> > upstream contrib directory?
>
> Yes, i think this should move into contrib/ upstream.  And we should
> think about what might be the appropriate way to package it for debian,
> too.
>
> --dkg
>
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: web interface to notmuch

[Matthew Lear]

Thanks for doing that, and thanks to all for the feedback and input so far.
For the interface I want to set up, I'd like the ability to enter notmuch
search syntax in an input box, and also show tags applied to messages.
The interface presented by the current version of notmuch-web ticks a lot
of boxes for me. Speed of being able to enter free-form search syntax (or
ideally, selecting from a list of favourite or predefined searches) and
returning the results quickly, I think are critical for how I'd like the
interface to be used. I see that the DFeed instance has an advanced search
facility, but it's a few clicks away.
I've not looked seriously at the other suggestions so far in this thread,
though.
Cheers,
--  Matt


On Tue, Oct 24, 2017 at 4:00 AM, Vladimir Panteleev <
thecybersha...@gmail.com> wrote:

> Hi,
>
> Sorry to barge in, I noticed this thread and thought I'd try to have a go
> at setting up a test DFeed instance.
>
> Here it is:
>
> http://dfeed-notmuch.k3.1azy.net/
>
> There is some more info on the help page:
>
> http://dfeed-notmuch.k3.1azy.net/help
>
> Posting is supported, but it is currently (intentionally) unconfigured for
> now.
>
> What do you think?
>
>
> On 2017-10-21 22:21, Daniel Kahn Gillmor wrote:
>
>> On Sat 2017-10-21 23:00:00 +0300, Jani Nikula wrote:
>>
>>> For the list archive, we could restrict to displaying text/plain only.
>>>
>>
>> and text/x-diff, surely :)
>>
>> But yeah, good point.
>>
>> Brian, what do you think about such a constraint?  would that make your
>> implementation safe enough to put on the public Internet for a read-only
>> archive?
>>
>>  --dkg
>>
>>
>>
>> ___
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch
>>
>>
> --
> Best regards,
>  Vladimir
>
>
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

web interface to notmuch

[Matthew Lear]

Hello all. A little side project at work involves me trying to put together
part of a knowledge share system where users can query and search email
stored and indexed centrally (by offlineimap & notmuch). My intention is to
provide a means to support multiple concurrent read-only accesses to the
notmuch database from users' web browsers so they can query and search mail.

Consider a few different email addresses being plugged into various
systems, all receiving email on different topics. I'd like to build an
application which presents a web frontend which I can run on the server
which fetches and indexes the mail, and thus present a web interface to
search all mail using notmuch.

notmuch-web has not seen much development for a few years.
noservice looks pretty nifty but I'm a little unsure of the status and if
it's missing anything fundamental.

I think my requirements are pretty basic:

* Read-only access
* Search and display mail only (no sending), including html mails
* Freeform entry of search terms in accordance with notmuch-search-terms(7).

Would anybody have any ideas about the best way to undertake such a project?

notmuch-web and noservice definitively look like they could be leveraged,
but I don't know if I'd be better trying to construct something from the
ground up which is better suited / tailored to my requirements (which are
much less than either of the above were intended to fulfil).

A standalone app would be preferred rather than having to rely on a web
server, although I'm not picky about infrastructure. Web based programming
is not my forte so I'd appreciate any feedback relating also to
implementation, currently available open source web frameworks which could
be used / considered / leveraged, etc.

Many thanks,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

Hi David. Finally got back round to checking this with recent sw. Apologies
for the long delay. I can confirm that the problem still exists as before,
ie replying to my test html emails when composing the message in a new
frame throws the bodypart errors. ...But I'm not too fussed about it
because I've got used to not composing messages in new frames ;-)

I've synced my emacs.git to:
* c3ff6712ad -  (HEAD, origin/emacs-25) * lisp/net/tls.el (tls-program):
Fix :version (do not merge to master). (2017-09-14 16:38:29 -0400) 

Emacs version is reported as 25.3.50.1.

And notmuch.git to:
* ab8b40f7 -  (HEAD -> master, origin/master, origin/HEAD) cli: make
notmuch count --exclude a boolean argument (2017-10-15 19:22:40 -0300)


Using the tiny .emacs below is enough to trigger the problem:

(require 'shr)
(setq mm-text-html-renderer 'shr)
(setq mime-view-text/html-previewer 'shr)
(setq mm-html-inhibit-images nil)
(setq mm-html-blocked-images nil)
(require 'notmuch)
(setq notmuch-mua-compose-in 'new-frame)
(setq notmuch-multipart/alternative-discouraged '("text/plain"))

Not sure what you'd like to do with this from here...
Cheers,
--  Matt

On Fri, Jun 30, 2017 at 1:47 PM, David Bremner <da...@tethera.net> wrote:

> Matthew Lear <m...@bubblegen.co.uk> writes:
>
>
> > It was indeed fixed...that is until I updated my emacs git tree to
> 25.2.1.
> > I've given up composing emails in new frames as it's now broken again -
> > this time when replying to emails which have quoted/embedded html in the
> > body. Feels like two steps forward, one step back :-(
> > --  Matt
>
> Unfortunately I don't have easy access to emacs25.2; I've been waiting
> for it show up in Debian. Can you see if the relevant fix referenced in
> that bug log included in 25.2? I did wonder if maybe you had a local
> modification that was lost when you updated to 25.2.
>
> d
>
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 4 Mar 2017 11:03, "David Bremner" <da...@tethera.net> wrote:

David Bremner <da...@tethera.net> writes:

> Matthew Lear <m...@bubblegen.co.uk> writes:
>
>
>> Thanks David. Yes it does. After recompiling the v25 lisp with these
>> changes, I'm unable to reproduce the problems with both the test emails I
>> sent you. Wonderful :-)
>> Are you going to raise this with upstream?
>> Cheers,
>>  Matt
>
> Yes, I've filed
>
>  https://debbugs.gnu.org/cgi/bugreport.cgi?bug=25828
>

That bug should be fixed in emacs master branch.

d


It was indeed fixed...that is until I updated my emacs git tree to 25.2.1.
I've given up composing emails in new frames as it's now broken again -
this time when replying to emails which have quoted/embedded html in the
body. Feels like two steps forward, one step back :-(
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

error building against xapian 1.3.4

[Matthew Lear]

Hi. On a new machine that I've been setting up (running Ubuntu 16.04 LTS),
I'm unable to build notmuch from git with HEAD @

* 77c9ec1 -  (HEAD -> master, origin/master, origin/HEAD) test: add known
broken test for indexing html (2017-04-20 06:59:40 -0300) 

The error looks like this:
...
CXX  -g -O2 lib/database.o
CXX  -g -O2 lib/parse-time-vrp.o
CXX  -g -O2 lib/directory.o
CXX  -g -O2 lib/index.o
CXX  -g -O2 lib/message.o
CXX  -g -O2 lib/message-property.o
CXX  -g -O2 lib/query.o
CXX  -g -O2 lib/query-fp.o
CXX  -g -O2 lib/config.o
CXX  -g -O2 lib/regexp-fields.o
lib/regexp-fields.cc: In member function ‘virtual Xapian::Query
RegexpFieldProcessor::operator()(const string&)’:
lib/regexp-fields.cc:159:38: error: ‘class RegexpPostingSource’ has no
member named ‘release’
  return Xapian::Query (postings->release ());
  ^
make: *** [lib/regexp-fields.o] Error 1

Even though xapian 1.4 is marked as stable, Ubuntu 16.04 has xapian 1.3.4
installed. I'd rather not start hacking around updating xapian and
dependant packages since the whole point of me having this machine is to
use it as a stable baseline with other colleagues who I work with.

Here's the output from the configure checks:

Sanity checking C compilation environment... OK.
Sanity checking C++ compilation environment... OK.
Reading libnotmuch version from source... OK.
Checking for Xapian development files... Yes (1.3.4).
Checking for Xapian compaction support... Yes.
Checking for Xapian FieldProcessor API... Yes.
Checking for Xapian lock retry support... Yes.
Testing default Xapian backend... chert
Checking for GMime development files... Yes.
Checking for Glib development files (>= 2.22)... Yes.
Checking for zlib (>= 1.2.5.2)... Yes.
Checking for talloc development files... Yes.
Checking for python... Yes (python).
Checking for valgrind development files... No (but that's fine).
Checking for bash-completion (>= 1.90)... Yes.
Checking if emacs is available... Yes.
Checking if doxygen is available... No (so will not install api docs)
Checking for ruby development files... No (skipping ruby bindings)
Checking if sphinx is available and supports nroff output... No (so will
not install man pages).
Checking if desktop-file-install is available... Yes.
Checking which platform we are on... Linux
Checking for /usr/local/lib in ldconfig... Yes
Checking for canonicalize_file_name... Yes.
Checking for getline... Yes.
Checking for strcasestr... Yes.
Checking for strsep... Yes.
Checking for timegm... Yes.
Checking for dirent.d_type... Yes.
Checking for standard version of getpwuid_r... Yes.
Checking for standard version of asctime_r... Yes.
Checking for rpath support... Yes.
Checking for -Wl,--as-needed... Yes.
Checking for -Wl,--no-undefined... Yes.
Checking for available C++ compiler warning flags...
-Wall -Wextra -Wwrite-strings
Checking for available C compiler warning flags...
-Wall -Wextra -Wwrite-strings -Wmissing-declarations

All required packages were found. You may now run the following
...

regexp-fields.cc was created in commit:

* 55524bb -  lib: regexp matching in 'subject' and 'from' (2017-03-03
17:46:48 -0400) 

If I revert to the previous commit:

* | 31b8ce4 -  lib: create field processors from prefix table (2017-03-03
07:15:13 -0400) 

I can build ok.

Is anybody still using xapian 1.3.x ? I was really hoping to keep up to
date with notmuch on my Ubuntu machine...

Cheers,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Moving mail store

[Matthew Lear]

On Tue, Mar 21, 2017 at 5:59 PM, David Bremner <da...@tethera.net> wrote:

> Matthew Lear <m...@bubblegen.co.uk> writes:
>
> > Hi all. I plan to migrate my offlineimap mail store and notmuch database
> to
> > a new machine in the next week or so. I wanted to check for any potential
> > gotchas before I did this. I don't plan on keeping the same absolute
> paths
> > to the maildir files on the new machine because the new top level
> mailstore
> > location will be different. Will notmuch care about this change if I use
> > the exported database (obtained from the old machine) on the new
> machine? I
> > expect notmuch to only care about the message IDs themselves and update
> its
> > database on the new machine when notmuch new is run. Is that correct?
>
> yes
>
> >
> > Is there a recommended way to move mailstore in such a way that all tags
> > are preserved even though the paths to the top level location of the
> > mailstore is different?
> >
>
> tags are not connected with paths, only with message-ids.
>

Excellent. Nothing else to be wary of then I guess. Thanks David.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Moving mail store

[Matthew Lear]

Hi all. I plan to migrate my offlineimap mail store and notmuch database to
a new machine in the next week or so. I wanted to check for any potential
gotchas before I did this. I don't plan on keeping the same absolute paths
to the maildir files on the new machine because the new top level mailstore
location will be different. Will notmuch care about this change if I use
the exported database (obtained from the old machine) on the new machine? I
expect notmuch to only care about the message IDs themselves and update its
database on the new machine when notmuch new is run. Is that correct?

Is there a recommended way to move mailstore in such a way that all tags
are preserved even though the paths to the top level location of the
mailstore is different?

Offlineimap shouldn't care as I think it just needs a top level directory
to sync into, so a sync run on the new machine post-migration of mailstore
shouldn't notice anything different provided that the offlineimap database
and config files are moved and tweaked accordingly.

Cheers,
 Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 18 Feb 2017 01:01, "David Bremner"  wrote:


So I _finally_ got around to looking at these, and I think it's roughly
the same shr bug as before but some different functions.

I could actually only duplicate the bug with emacs-reply-fail-ec
message, but that was fixed by the following patch against the emacs-25
branch. Does this patch fix both failures for you?

diff --git a/lisp/net/shr.el b/lisp/net/shr.el
index 6c35a33c9c..2bc37c64bd 100644
--- a/lisp/net/shr.el
+++ b/lisp/net/shr.el
@@ -1993,6 +1993,9 @@ shr-pixel-buffer-width
 (if (get-buffer-window)
(car (window-text-pixel-size nil (point-min) (point-max)))
   (save-window-excursion
+;; Avoid errors if the selected window is a dedicated one,
+;; and they just want to insert a document into it.
+(set-window-dedicated-p nil nil)
(set-window-buffer nil (current-buffer))
(car (window-text-pixel-size nil (point-min) (point-max)))

@@ -2036,6 +2039,9 @@ shr-render-td-1
(shr-indentation 0))
(shr-descend dom))
   (save-window-excursion
+;; Avoid errors if the selected window is a dedicated one,
+;; and they just want to insert a document into it.
+(set-window-dedicated-p nil nil)
(set-window-buffer nil (current-buffer))
(unless fill
  (setq natural-width


Thanks David. Yes it does. After recompiling the v25 lisp with these
changes, I'm unable to reproduce the problems with both the test emails I
sent you. Wonderful :-)
Are you going to raise this with upstream?
Cheers,
 Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 03/12/16 01:58, David Bremner wrote:
> Matthew Lear <m...@bubblegen.co.uk> writes:
> 
>>
>> I'd happily post some debug info to help analyse this if somebody could
>> give me a few pointers about what would be needed and the emacs-fu
>> needed to generate this.
>>
> 
> I think the best thing at this point would be to find (or create)
> another public message where you can duplicate the problem.

I've sent two test emails directly to David for analysis.
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 23/11/16 11:57, David Bremner wrote:
> In a fresh emacs, can you generate an error using the recipe in
> 
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24950 ?

No. I can evaluate the recipe ok and eval of the last line yields:
10 (#o12, #xa, ?\C-j)


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 22/11/16 14:24, Matthew Lear wrote:
> I can confirm that it is indeed fixed. I'm not able to reproduce it with
> the latest emacs 25 from git.

Unfortunately, it seems I was too hasty in confirming this.
For some reason unknown to me, replying to the html in some
multipart/alternative works fine, but others cause the same error to be
thrown. I've also seen the error occur when replying to a
multipart/mixed message, too.

Even more bizarrely, I'm currently looking at a thread with two emails
where both are multipart/alternative and contain text/plain and
text/html parts, and replying to the first message triggers the
error/bug but replying to the second message does not. I can't see any
formatting difference between the two.

Sorry, but it would appear as though there is still some work needed to
resolve this in it's entirety. I suppose it's likely that this could be
an emacs-25 issue, but what I mentioned above makes no sense to me so I
can't say for sure.

I'd happily post some debug info to help analyse this if somebody could
give me a few pointers about what would be needed and the emacs-fu
needed to generate this.

Cheers,
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 20/11/16 20:01, David Bremner wrote:
>> I have reported https://debbugs.gnu.org/cgi/bugreport.cgi?bug=24950
>> upstream. We'll see if they agree it's bug in shr.
> 
> This bug will be fixed in the next release of emacs 25, so I'm tagging
> it fixed.

I can confirm that it is indeed fixed. I'm not able to reproduce it with
the latest emacs 25 from git.
Thanks for raising it with upstream, David.
Cheers,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 12/11/16 00:20, David Bremner wrote:
> Matthew Lear <m...@bubblegen.co.uk> writes:
> 
>> I can trigger the bug if I discard text/plain though. Did you try that? 
> 
> I'm not sure what you mean by "discard", but closing it isn't enough. I
> think more of your settings are needed to replicate the problem with a
> multipart/alternative message.
> 

Apologies, David. I wasn't clear at all. I meant that if I discourage
text/plain in a multipart/alternative message, I can trigger the bug.
If I use the following tiny-init.el...

(require 'shr)
(setq mm-text-html-renderer 'shr)
(setq mime-view-text/html-previewer 'shr)
(setq mm-html-inhibit-images nil)
(setq mm-html-blocked-images nil)

(require 'notmuch)
(global-set-key [f1]  'notmuch)
(setq notmuch-mua-compose-in 'new-frame)
(setq notmuch-multipart/alternative-discouraged '("text/plain"))


...and load it using 'emacs-25 -q --load ~/.emacs.d/tiny-init.el',
searching for any mimetype:text/html and replying to it triggers the bug
for me.

Commenting out the setting of notmuch-mua-compose-in is enough to
prevent the problem from occurring.
Cheers,
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

I can trigger the bug if I discard text/plain though. Did you try that? 
 Original message From: David Bremner <da...@tethera.net> Date: 
11/11/2016  11:40  (GMT+00:00) To: Matthew Lear <m...@bubblegen.co.uk>, Matt 
Armstrong <marmstr...@google.com> Cc: notmuch@notmuchmail.org Subject: Re: some 
issues with emacs 25 
Matthew Lear <m...@bubblegen.co.uk> writes:

> On 09/11/16 23:55, David Bremner wrote:
>> I can't find an html only public-message so far to give a full
>> backtrace.
> If it helps, here is some text in *bold *with some changes in red that
> you can test with :-D

FWIW, this doesn't trigger the bug with default settings + (setq
notmuch-mua-compose-in 'new-window) because it has a text part.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 09/11/16 23:55, David Bremner wrote:
> Matthew Lear <m...@bubblegen.co.uk> writes:
>
>> On 28/10/16 22:34, Matthew Lear wrote:
>>> I will commence with trying to figure out what is causing the problem. I've 
>>> seen some weirdness with margins when viewing HTML messages too. Maybe 
>>> that's some kind of bizarre toolkit thing... Anyway, thanks a lot for 
>>> testing and for confirming that there's no problem.
>> From what I can tell, this is purely caused by using:
>>
>> (setq notmuch-mua-compose-in 'new-frame)
>> or
>> (setq notmuch-mua-compose-in 'new-window)
> Yes, I can replicate this bug in emacs 25.1 by
>
> 1) ./devel/try-emacs-mua -q
>
> 2) (setq notmuch-mua-compose-in 'new-frame)
>
> 3) reply to an html only message.
>
>
> If I M-x toggle-debug-on-error I get a backtrace ending in
>
> set-window-buffer(nil #)
> shr-pixel-column
> shr-string-pixe-width
> shr-insert-document
> notmuch-show--insert-part-text/html-shr
>
> I can't find an html only public-message so far to give a full
> backtrace.
If it helps, here is some text in *bold *with some changes in red that
you can test with :-D
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

On 28/10/16 22:34, Matthew Lear wrote:
> 
> I will commence with trying to figure out what is causing the problem. I've 
> seen some weirdness with margins when viewing HTML messages too. Maybe that's 
> some kind of bizarre toolkit thing... Anyway, thanks a lot for testing and 
> for confirming that there's no problem.

From what I can tell, this is purely caused by using:

(setq notmuch-mua-compose-in 'new-frame)
or
(setq notmuch-mua-compose-in 'new-window)

when using (setq mm-text-html-renderer 'shr)

If I keep shr and use (setq notmuch-mua-compose-in 'current-window) or
remove this setting completely (to get the default behaviour), there is
no problem in replying to an html message and the html text being quoted.

If I switch to using different values of mm-text-html-renderer such as
w3m, links, lynx or html2text there is no problem replying when using
a new-frame or new-window via notmuch-mua-compose-in.

Seems like the problem I've seen is purely limited to using
mm-text-html-renderer equal to 'shr along with notmuch settings to
compose in either a new frame or a new window.

I've tested this with a .emacs of only a few lines and am convinced.

Please can this be verified by somebody else?

Cheers,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: some issues with emacs 25

[Matthew Lear]

> On 28 Oct 2016, at 01:52, Matt Armstrong <marmstr...@google.com> wrote:
> 
> Matthew Lear <m...@bubblegen.co.uk> writes:
> 
>> Hi,
>> I switched to trying emacs 25 (25.1) with notmuch the other day.
> 
> I'm on 25.1 as well, but haven't experienced the symptoms you describe.

Thanks Matt. I feel reassured! :-) I will commence with trying to figure out 
what is causing the problem. I've seen some weirdness with margins when viewing 
HTML messages too. Maybe that's some kind of bizarre toolkit thing... Anyway, 
thanks a lot for testing and for confirming that there's no problem.
Cheers,
  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

some issues with emacs 25

[Matthew Lear]

Hi,
I switched to trying emacs 25 (25.1) with notmuch the other day. I
really like some of the improvements that have been made (in particular,
SHR seems to do a much nicer and quicker job of rendering html emails).
However, things came a little unstuck when I tried to reply to an email
with notmuch. I have notmuch set to discourage plain text simply because
I deal with a lot of email which is sent in html (people frequently
in-line tables, images etc) and when I replied, notmuch generated the
following in the new message buffer:

> !!! Bodypart handler `notmuch-show-insert-part-text/html' threw an error:
> !!! Window is dedicated to ‘*unsent mail to u...@domain.com*’
>

Errors like these are generated by replying from either notmuch-tree or
notmuch-show.

I'd made no changes to my .emacs before switching to 25 and just to
check, I switched back to 24 (24.5) and I could reply to the same email
without any trouble.

I'm of the opinion that this is probably some sort of set up /
configuration issue specific to 25, but I'm quite surprised. I don't
have many settings for handling html stuff:

(setq mm-text-html-renderer 'shr)
(setq shr-inhibit-images nil)
(setq mime-view-text/html-previewer 'shr)
(setq mm-inline-text-html-with-images t)
(setq notmuch-multipart/alternative-discouraged '("text/plain"))

Is this reproducible for anyone else with notmuch from git/master as of
today?

Cheers,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re:

[Matthew Lear]

Hi Mark. Excellent :-) I'll look out for it in the repository at some point 
soon. Cheers,   Matt 
 Original message From: Mark Walters 
<markwalters1...@gmail.com> Date: 15/10/2016  08:09  (GMT+00:00) To: 
m...@bubblegen.co.uk, notmuch@notmuchmail.org, m...@bubblegen.co.uk Subject: 
Re: 

On Wed, 12 Oct 2016, Mark Walters <markwalters1...@gmail.com> wrote:
> On Tue, 11 Oct 2016, m...@bubblegen.co.uk wrote:
>> From: Matthew Lear <m...@bubblegen.co.uk>
>> To: notmuch@notmuchmail.org
>> Cc: Matthew Lear <m...@bubblegen.co.uk>
>> Subject: [PATCH] Fix reply to encrypted mail when discouraging plain text.
>> Date: Tue, 11 Oct 2016 22:24:18 +0100
>> Message-Id: <1476221058-10431-1-git-send-email-m...@bubblegen.co.uk>
>> X-Mailer: git-send-email 2.4.10
>>
>> If an encrypted multipart message is received which contains html and
>> notmuch-multipart/alternative-discouraged is set to discourage "text/plain",
>> any encrypted parts are not decrypted during generation of the reply
>> text. This fixes that problem by making sure notmuch-mua-reply does
>> that.
>
> Hi
>
> I haven't tested this but it looks correct: more broadly I think this is
> needed whenever notmuch-show has to get a part directly rather than just
> from the sexp reply.

Hi

Just to confirm I have now tested this -- it compiles and test suite
passes. (Note I don't have suitable encrypted messages to test).

Anyway LGTM +1

Best wishes

Mark



>
> Best wishes
>
> Mark
>
>
>> ---
>>  emacs/notmuch-mua.el | 4 
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
>> index c567173..f333655 100644
>> --- a/emacs/notmuch-mua.el
>> +++ b/emacs/notmuch-mua.el
>> @@ -251,6 +251,10 @@ mutiple parts get a header."
>>     (notmuch-show-max-text-part-size 0)
>>     ;; Insert headers for parts as appropriate for replying.
>>     (notmuch-show-insert-header-p-function 
>>notmuch-mua-reply-insert-header-p-function)
>> +   ;; Ensure that any encrypted parts are
>> +   ;; decrypted during the generation of the reply
>> +   ;; text.
>> +   (notmuch-show-process-crypto process-crypto)
>>     ;; Don't indent multipart sub-parts.
>>     (notmuch-show-indent-multipart nil))
>>      ;; We don't want sigstatus buttons (an information leak and 
>>usually wrong anyway).
>> -- 
>> 2.4.10
>>
>> ___
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Missing headers when forwarding html message as RFC822

[Matthew Lear]

Hi Mark. I will do, but would you mind sending me one of your test messages? At 
least that way, I can test against something that is 'known working'.
Thanks,
  Matt

> On 23 Jul 2016, at 08:59, Mark Walters <markwalters1...@gmail.com> wrote:
> 
> 
> Hi
> 
> I have tried several cases, including generating a message with just the
> two parts [ multipart/related ] [ text/html ] and forwarding that, and
> they all work for me. Can you provide a test message where it fails?
> 
> Also what version of emacs are you running? I think all the rfc822
> forwarding is handled by emacs/gnus message-mode
> 
> Best wishes
> 
> Mark
> 
> 
> 
>> On Fri, 22 Jul 2016, Matthew Lear <m...@bubblegen.co.uk> wrote:
>> Hi,
>> 
>> With notmuch built from very recent git (sha 3a45d29) it would seem that
>> when forwarding an HTML only message as RFC822, once received, the message
>> does not have any headers and notmuch-show merely displays 'Date: Thu, 01
>> Jan 1970 00:00:00 +'. I've stripped my .emacs back to a bare minimum
>> configuration and the behaviour is the same.
>> 
>> With a message (incidentally sent by MS Outlook) that contains the
>> following parts:
>> 
>> [ multipart/related ]
>> [ text/html ]
>> 
>> Once forwarded as RFC822 by notmuch and received, it is displayed by
>> notmuch-show as:
>> 
>> [ message/rfc822 ]
>> Date: Thu, 01 Jan 1970 00:00:00 +
>> 
>> [ multipart/mixed ]
>> [ multipart/related ]
>> [ text/html ]
>> 
>> While with a message that contains the following parts:
>> 
>> [ multipart/mixed ]
>> [ text/plain ]
>> 
>> Once forwarded as RFC822 by notmuch and received, it is displayed by
>> notmuch-show as:
>> 
>> [ message/rfc822 ]
>> From: 
>> Subject: 
>> To: 
>> Date: Fri, 15 Jul 2016 17:48:17 +
>> 
>> [ multipart/mixed ]
>> [ text/plain ]
>> 
>> So the headers are missing in the first case. I'm sure this is a regression.
>> 
>> Can anybody verify this?
>> Cheers,
>> --  Matt
>> 
>> 
>> ___
>> notmuch mailing list
>> notmuch@notmuchmail.org
>> https://notmuchmail.org/mailman/listinfo/notmuch

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

Hi David,

> OK, with that setting I can duplicate the problem replying to encrypted
> HTML mail.
>
> It's also true that notmuch-reply doesn't include the content in the
> html version of the multipart/alternative. Unlike notmuch show, it
> doesn't have the ability to send parts later, so I guess that's a
> problem there.
>
> The following patch seems to fix it for me. What I haven't worked out is
> if it will break other things. Perhaps David E or Mark (in copy) can
> comment on whether this is a sensible change, given that we now give the
> user control of showing the part on the emacs side.
>
> diff --git a/notmuch-reply.c b/notmuch-reply.c
> index 3c6d685..cd88d79 100644
> --- a/notmuch-reply.c
> +++ b/notmuch-reply.c
> @@ -690,7 +690,7 @@ notmuch_reply_format_sprinter(void *ctx,
>
>  /* Start the original */
>  sp->map_key (sp, "original");
> -format_part_sprinter (ctx, sp, node, TRUE, TRUE, FALSE);
> +format_part_sprinter (ctx, sp, node, TRUE, TRUE, TRUE);
>
>  /* End */
>  sp->end (sp);

I could have sworn that I replied to this thread months ago but can't find
a record it in the archives... I think I also missed David E's reply to
yours mentioning the patch from id:1459708823-1852-1-git-send-email-dme at
dme.org, so I'm following up...

With David E's patch, the decryption problem I reported is resolved.
However, I notice that this change hasn't yet been committed / pushed. Is
there a particular reason for this?

I'm currently maintaining a version of this patch compatible with the
current master which makes this change to notmuch-mua.el
notmuch-mua-reply() by adding (notmuch-show-process-crypto process-crypto)
before doing the show. I've attached that patch here.

I just wondered if we were going to see this pushed to master any time soon?

Cheers,
--  Matt
diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 0445975..0dcdb06 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -250,7 +250,11 @@ mutiple parts get a header."
 		   ;; Don't omit long parts.
 		   (notmuch-show-max-text-part-size 0)
 		   ;; Insert headers for parts as appropriate for replying.
-		   (notmuch-show-insert-header-p-function notmuch-mua-reply-insert-header-p-function)
+   (notmuch-show-insert-header-p-function notmuch-mua-reply-insert-header-p-function)
+   ;; Ensure that any encrypted parts are
+   ;; decrypted during the generation of the reply
+   ;; text.
+   (notmuch-show-process-crypto process-crypto)
 		   ;; Don't indent multipart sub-parts.
 		   (notmuch-show-indent-multipart nil))
 		(notmuch-show-insert-body original (plist-get original :body) 0)___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Missing headers when forwarding html message as RFC822

[Matthew Lear]

Hi,

With notmuch built from very recent git (sha 3a45d29) it would seem that
when forwarding an HTML only message as RFC822, once received, the message
does not have any headers and notmuch-show merely displays 'Date: Thu, 01
Jan 1970 00:00:00 +'. I've stripped my .emacs back to a bare minimum
configuration and the behaviour is the same.

With a message (incidentally sent by MS Outlook) that contains the
following parts:

[ multipart/related ]
[ text/html ]

Once forwarded as RFC822 by notmuch and received, it is displayed by
notmuch-show as:

[ message/rfc822 ]
Date: Thu, 01 Jan 1970 00:00:00 +

[ multipart/mixed ]
[ multipart/related ]
[ text/html ]

While with a message that contains the following parts:

[ multipart/mixed ]
[ text/plain ]

Once forwarded as RFC822 by notmuch and received, it is displayed by
notmuch-show as:

[ message/rfc822 ]
From: 
Subject: 
To: 
Date: Fri, 15 Jul 2016 17:48:17 +

[ multipart/mixed ]
[ text/plain ]

So the headers are missing in the first case. I'm sure this is a regression.

Can anybody verify this?
Cheers,
--  Matt


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

>> Hmm. I can't duplicate the problem replying from show mode with the test
>> message you provided.  Is the appropriate key in your gpg keyring?
>
> Yes it is. I get the gpg gui prompt when opening the encrypted email and
> can decrypt it prior to show. As I suspected earlier in the thread, I
> wondered if this was related to some emacs config settings/options that I
> have. I can't immediately think what they would be though, and why they
> would trigger this particular issue for html only. Strange...

Do you have the following in your .emacs ?

(setq notmuch-multipart/alternative-discouraged '("text/plain"))

I think that's the work/not-work switch. I'm not sure if it makes sense
or not in terms of seeing the behaviour I do. I discard plain text
because I get a ton of html email at work with various embedded images
etc and I want to be able to read that (which works fine). But I can see
that to quote a reply from an encrypted message, you need to quote the
original text which will be plain text.
What do you think?
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

> Matthew Lear <m...@bubblegen.co.uk> writes:
>
>> Thanks David. I've just pulled and rebuilt and can see a difference.
>> When
>> replying from show view, I see no quoted text from the original message,
>> but when I reply from search view I see the inline PGP ciphertext quoted
>> in the reply. I can understand the latter but not the former - this
>> still
>> looks like a bug, especially since if I reply from show view to a plain
>> text encrypted message, I do see the original message quoted in the
>> reply.
>> --  Matt
>
> Hmm. I can't duplicate the problem replying from show mode with the test
> message you provided.  Is the appropriate key in your gpg keyring?

Yes it is. I get the gpg gui prompt when opening the encrypted email and
can decrypt it prior to show. As I suspected earlier in the thread, I
wondered if this was related to some emacs config settings/options that I
have. I can't immediately think what they would be though, and why they
would trigger this particular issue for html only. Strange...

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

> Matthew Lear <m...@bubblegen.co.uk> writes:
>
>> On Sat, Mar 12 2016 at  2:37:44 pm GMT, David Edmondson <d...@dme.org>
>> wrote:
>>> On Sat, Mar 12 2016, David Bremner wrote:
>>>> Tomi Ollila <tomi.oll...@iki.fi> writes:
>>>>
>>>>> 2016-03-08 (Tue) 14:16:01: reply --format=sexp --format-version=1
>>>>> --reply-to=sender thread:4a6f
>>>>> 2016-03-08 (Tue) 14:16:01: show --format=raw --part=2
>>>>> id:56dde706.6060...@bubblegen.co.uk
>>>>> 2016-03-08 (Tue) 14:16:01: show --format=sexp --include-html --part=3
>>>>> id:56dde706.6060...@bubblegen.co.uk
>>>>> 2016-03-08 (Tue) 14:16:01: show --format=raw --part=3
>>>>> id:56dde706.6060...@bubblegen.co.uk
>>>>>
>>>>
>>>> I think the problem is in the following let form, which uses
>>>> notmuch-show-process-crypto. That isn't set in search mode. I'm
>>>> unclear
>>>> on what the right solution is here; we could check
>>>> notmuch-crypto-process-mime, but I'm not sure how this would interact
>>>> with the handling of prefix arguments (for temporary toggling).
>>>
>>> Is "reply" from search mode a common pattern of use? Currently it seems
>>> generally un-useful (the thread has to contain just a single message -
>>> not just a single matching message).
>>>
>>>> (defun notmuch-mua-reply (query-string  sender reply-all)
>>>>   (let ((args '("reply" "--format=sexp" "--format-version=1"))
>>>>(process-crypto notmuch-show-process-crypto)
>>>>reply
>>>>original)
>>
>> Hi. I just wondered if anyone had managed to make any progress on this?
>
> Hi Matthew;
>
> I'm not sure if you noticed, but there are patches in master now (and
> will be in a release pretty soon) that handle the part of the problem
> related to encrypted html, but nothing so far for the problem(s) of
> replying from search view.

Thanks David. I've just pulled and rebuilt and can see a difference. When
replying from show view, I see no quoted text from the original message,
but when I reply from search view I see the inline PGP ciphertext quoted
in the reply. I can understand the latter but not the former - this still
looks like a bug, especially since if I reply from show view to a plain
text encrypted message, I do see the original message quoted in the reply.
--  Matt


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

On Sat, Mar 12 2016 at  2:37:44 pm GMT, David Edmondson  wrote:
> On Sat, Mar 12 2016, David Bremner wrote:
>> Tomi Ollila  writes:
>>
>>> 2016-03-08 (Tue) 14:16:01: reply --format=sexp --format-version=1 
>>> --reply-to=sender thread:4a6f
>>> 2016-03-08 (Tue) 14:16:01: show --format=raw --part=2 
>>> id:56dde706.6060...@bubblegen.co.uk
>>> 2016-03-08 (Tue) 14:16:01: show --format=sexp --include-html --part=3 
>>> id:56dde706.6060...@bubblegen.co.uk
>>> 2016-03-08 (Tue) 14:16:01: show --format=raw --part=3 
>>> id:56dde706.6060...@bubblegen.co.uk
>>>
>>
>> I think the problem is in the following let form, which uses
>> notmuch-show-process-crypto. That isn't set in search mode. I'm unclear
>> on what the right solution is here; we could check
>> notmuch-crypto-process-mime, but I'm not sure how this would interact
>> with the handling of prefix arguments (for temporary toggling).
>
> Is "reply" from search mode a common pattern of use? Currently it seems
> generally un-useful (the thread has to contain just a single message -
> not just a single matching message).
>
>> (defun notmuch-mua-reply (query-string  sender reply-all)
>>   (let ((args '("reply" "--format=sexp" "--format-version=1"))
>>  (process-crypto notmuch-show-process-crypto)
>>  reply
>>  original)

Hi. I just wondered if anyone had managed to make any progress on this?
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

> On Tue, Mar 08 2016, Tomi Ollila wrote:
>
>> [ text/plain ]
>> On Tue, Mar 08 2016, David Bremner wrote:
>>
>>> [ text/plain ]
>>> Matthew Lear <m...@bubblegen.co.uk> writes:
>>>
>>>
>>> Thanks for the test case. I can duplicate a (the?) bug as
>>> follows. Replying to that message from notmuch-show mode works, but
>>> replying from notmuch-search mode yields
>>>
>>>> !!! Bodypart insert error: Wrong type argument: number-or-marker-p,
>>>> nil !!!
>>>> [ application/pgp-encrypted ]
>>>> Version: 1
>>>> [ encrypted.asc: application/octet-stream (as text/plain) ]
>>>> !!! Bodypart insert error: Internal error: No :content from ("show"
>>>> "--format=sexp" "--include-html" "--part=3"
>>>> "id:56dde706.6060...@bubblegen.co.uk") !!!
>>>
>>> Followed by the ciphertext.
>>>> -BEGIN PGP MESSAGE-
>>>
>>> If anyone wants to play with this, you just need to import the notmuch
>>> test key into your keyring, something like
>>>
>>> gpg --import path/to/notmuch-source/test/gnupg-secret-key.asc
>>>
>>> What's odd is that the notmuch command in the error message works on
>>> the
>>> command line.
>>
>> I got these to my log:
>>
>> when replying from search:
>>
>> 2016-03-08 (Tue) 14:16:01: reply --format=sexp --format-version=1
>> --reply-to=sender thread:4a6f
>> 2016-03-08 (Tue) 14:16:01: show --format=raw --part=2
>> id:56dde706.6060...@bubblegen.co.uk
>> 2016-03-08 (Tue) 14:16:01: show --format=sexp --include-html --part=3
>> id:56dde706.6060...@bubblegen.co.uk
>
> The above line returns (same as in latest !!! bodypart insert error !!!)
>
> (:id 3 :content-type "application/octet-stream" :filename "encrypted.asc"
> :content-length 1875)
>
> -- there is no content to be included from that line :/ (and wat is this
>"encrypted.asc" anyway ?)

encrypted.asc is what Enigmail uses when sending using PGP/MIME. I think
it wraps up the message body in this file (armor encrypted) and marks it
with disposition inline. Enigmail also sends PGP/MIME version
identification information in another MIME part
(application/pgp-encrypted) - so you have two MIME parts per PGP/MIME
encrypted message.

You can view all headers in the test message if you're in notmuch-show by
using V. You'll see something like:

Content-Type: multipart/encrypted;
 protocol="application/pgp-encrypted";
 boundary="xxx"

This is an OpenPGP/MIME encrypted message (RFC 4880 and 3156)
--xxx
Content-Type: application/pgp-encrypted
Content-Description: PGP/MIME version identification

Version: 1

--xxx
Content-Type: application/octet-stream; name="encrypted.asc"
Content-Description: OpenPGP encrypted message
Content-Disposition: inline; filename="encrypted.asc"

-BEGIN PGP MESSAGE-
Version: GnuPG v2
...
-END PGP MESSAGE-

--xxx--


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

>> Hmm. In emacs, if I try to reply to the message from notmuch-search, I
>> get
>> no error but there is no quoted text from the original message in the
>> reply body and all I see is the 'on ,  wrote:' citation.
>> Replying from notmuch-show gives me the error I mentioned previously.
>
> I suppose the main difference is the setting of
> notmuch-multipart/alternative-discouraged. If I copy your setting I also
> get an error replying in notmuch-show mode.
>

I know this was mentioned earlier but just to avoid any doubt, my
discourage setting is currently:
(setq notmuch-multipart/alternative-discouraged '("text/plain"))

I removed the "multipart/related". (I agree it's probably not wise to
discourage it).


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

> Thanks for the test case. I can duplicate a (the?) bug as
> follows. Replying to that message from notmuch-show mode works, but
> replying from notmuch-search mode yields
>
>> !!! Bodypart insert error: Wrong type argument: number-or-marker-p, nil
>> !!!
>> [ application/pgp-encrypted ]
>> Version: 1
>> [ encrypted.asc: application/octet-stream (as text/plain) ]
>> !!! Bodypart insert error: Internal error: No :content from ("show"
>> "--format=sexp" "--include-html" "--part=3"
>> "id:56dde706.6060...@bubblegen.co.uk") !!!
>
> Followed by the ciphertext.
>> -BEGIN PGP MESSAGE-
>
> If anyone wants to play with this, you just need to import the notmuch
> test key into your keyring, something like
>
> gpg --import path/to/notmuch-source/test/gnupg-secret-key.asc
>
> What's odd is that the notmuch command in the error message works on the
> command line.
>

Hmm. In emacs, if I try to reply to the message from notmuch-search, I get
no error but there is no quoted text from the original message in the
reply body and all I see is the 'on ,  wrote:' citation.
Replying from notmuch-show gives me the error I mentioned previously.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

On Mon, Mar 07 2016 at 12:20:37 pm GMT, David Bremner <da...@tethera.net> wrote:
> Matthew Lear <m...@bubblegen.co.uk> writes:
>>
>> This is easy to reproduce on linux with thunderbird (I'm using v38.5.0).
>> Edit -> Account Settings -> Composition & Addressing and check 'Compose
>> messages in HTML format' and send yourself a pgp encrypted email using
>> PGP/MIME, open it in notmuch-emacs and reply to it. You'll see the
>> 'Bodypart insert error'.
>>
>> If you need any more info let me know...
>
> Please encrypt a message to the attached gpg key (from the notmuch test
> suite), and send it to the list as an attachement if you can replicate
> the bug.

Done. Attached here.



1457383253_0.15104.sunrise,U=4513,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,S
Description: Binary data
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

On 01/03/16 13:59, Matthew Lear wrote:
>> There is probably a bug here.
>>
>> On Tue, Mar 01 2016, Matthew Lear wrote:
>>> Notmuch shows this as:
>>>
>>> [ multipart/encrypted ]
>>> [ Decryption successful ]
>>> [ Good signature by:   ]
>>> [ application/pgp-encrypted ]
>>> Version: 1
>>> [ multipart/mixed ]
>>> [ multipart/alternative ]
>>> [ text/plain (hidden) ]
>>> [ multipart/related ]
>>> [ text/html (hidden) ]
>>> [ image.jpg: image/jpeg (hidden) ]
>>
>> You're saying here that you don't get shown the text/html part?
> 
> Apologies. No, I do get shown the text/html part. I just collapsed all the
> parts here for illustration purposes.
> 
>>> (setq notmuch-multipart/alternative-discouraged '("text/plain"
>>> "multipart/related"))
>>
>> I'm curious to understand why you discourage multipart/related. That's
>> almost always used to wrap a text/html part with an image/jpg that the
>> text/html part refers to by cid:.
> 
> Fair point. I think I had this discouraged as a hangover from having to
> deal with malformatted PGP encrypted emails. I don't receive emails from
> people which have been encrypted with that sw any more so I should
> probably remove it. Even with multipart/related discouraged though, I
> don't have any problem with embedded images / cid etc. All displayed ok.
> 
>> Are things different if you discourage only text/plain?
> 
> Nope. No change.

This is easy to reproduce on linux with thunderbird (I'm using v38.5.0).
Edit -> Account Settings -> Composition & Addressing and check 'Compose
messages in HTML format' and send yourself a pgp encrypted email using
PGP/MIME, open it in notmuch-emacs and reply to it. You'll see the
'Bodypart insert error'.

If you need any more info let me know...

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: (emacs) Parsing problems replying to encrypted html

[Matthew Lear]

> There is probably a bug here.
>
> On Tue, Mar 01 2016, Matthew Lear wrote:
>> Notmuch shows this as:
>>
>> [ multipart/encrypted ]
>> [ Decryption successful ]
>> [ Good signature by:   ]
>> [ application/pgp-encrypted ]
>> Version: 1
>> [ multipart/mixed ]
>> [ multipart/alternative ]
>> [ text/plain (hidden) ]
>> [ multipart/related ]
>> [ text/html (hidden) ]
>> [ image.jpg: image/jpeg (hidden) ]
>
> You're saying here that you don't get shown the text/html part?

Apologies. No, I do get shown the text/html part. I just collapsed all the
parts here for illustration purposes.

>> (setq notmuch-multipart/alternative-discouraged '("text/plain"
>> "multipart/related"))
>
> I'm curious to understand why you discourage multipart/related. That's
> almost always used to wrap a text/html part with an image/jpg that the
> text/html part refers to by cid:.

Fair point. I think I had this discouraged as a hangover from having to
deal with malformatted PGP encrypted emails. I don't receive emails from
people which have been encrypted with that sw any more so I should
probably remove it. Even with multipart/related discouraged though, I
don't have any problem with embedded images / cid etc. All displayed ok.

> Are things different if you discourage only text/plain?

Nope. No change.


___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

(emacs) Parsing problems replying to encrypted html

[Matthew Lear]

Hi,
I've noticed a problem when replying to an encrypted email when I have
html set as preferred over plain text. Looks like notmuch fails to parse
the body and the original body text in notmuch-reply just looks like this:

> [ text/html ]
> !!! Bodypart insert error: End of file during parsing !!!

A few things which might help understand what could be happening here...

notmuch from git with head @ 07b6220
gmime 2.6.20-r1 (gentoo)

(setq mm-text-html-renderer 'shr)
(setq shr-inhibit-images nil)
(setq mime-view-text/html-previewer 'shr)
(setq mm-inline-text-html-with-images t)

(set-locale-environment "en_GB.UTF-8")
(set-language-environment 'utf-8)
(set-selection-coding-system 'utf-8)
(prefer-coding-system 'utf-8)
(set-default-coding-systems 'utf-8)
(set-terminal-coding-system 'utf-8)
(set-keyboard-coding-system 'utf-8)
(setq default-buffer-file-coding-system 'utf-8)

(setq notmuch-multipart/alternative-discouraged '("text/plain"
"multipart/related"))

The message I'm replying to looks like:

User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101
 Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: 
Content-Type: multipart/encrypted;
 protocol="application/pgp-encrypted";
 boundary="et6k1RmUSFFnhWFnwewqvineB8rlw3boE"

This is an OpenPGP/MIME encrypted message (RFC 4880 and 3156)
--et6k1RmUSFFnhWFnwewqvineB8rlw3boE
Content-Type: application/pgp-encrypted
Content-Description: PGP/MIME version identification

Version: 1

--et6k1RmUSFFnhWFnwewqvineB8rlw3boE
Content-Type: application/octet-stream; name="encrypted.asc"
Content-Description: OpenPGP encrypted message
Content-Disposition: inline; filename="encrypted.asc"

-BEGIN PGP MESSAGE-
Version: GnuPG v2

hQEMA41hzuYEoCb4AQgApMvj7nt6Goj1kLDUheofRWwORI/CNFgExIIAAz8ivJxH
...
...
6nVgRBQIJp+59xsmfC2axrASSAh11/ZdVl3T60LvlrsoKLkBHNAC4W+H/cyxxDk0
BnJC8+QJbgbyRBANUV4d5PCUbKcuDRW/TVQhtZY=
=rKjq
-END PGP MESSAGE-

--et6k1RmUSFFnhWFnwewqvineB8rlw3boE--


Notmuch shows this as:

[ multipart/encrypted ]
[ Decryption successful ]
[ Good signature by:   ]
[ application/pgp-encrypted ]
Version: 1
[ multipart/mixed ]
[ multipart/alternative ]
[ text/plain (hidden) ]
[ multipart/related ]
[ text/html (hidden) ]
[ image.jpg: image/jpeg (hidden) ]

Things of interest:

1/ I have the same encrypted email in plain text only and replying to it
works fine with notmuch.

2/ I can view the html from the email in a browser, and it looks fine.

3/ If I switch emacs to use w3m instead of shr, I don't see the 'Bodypart
insert error' fro notmuch-reply but there is no original message included
in the reply composition either.

I'm not quite sure if this is a notmuch issue as such, or attributed to my
emacs config, or something else...

I don't know exactly what is supposed to happen when replying to html
emails, but if I reply to an html email which is not encrypted, there is
no '> [ text/html ]' in the reply composition window - it's just text from
the original message that I'm replying to and nothing more.

Since the encrypted email I'm replying has a text part, can notmuch use
this and quote it when replying (obviously ensuring that the entire reply
will be encrypted).

Any thoughts?

Thanks a lot.
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch-emacs: forward messages inline

[Matthew Lear]

>
> Hi
>
> Does adding something like the following to your .emacs file do what you
> want?
>
>
> (defun my-forward (prefix)
>   (interactive "P")
>   (let ((message-forward-as-mime nil)
>   (message-forward-ignored-headers ".*"))
> (notmuch-show-forward-message prefix)))
>
> (define-key notmuch-show-mode-map "F" 'my-forward)
>
> Then "F" should forward the message as a plain message rather than
> rfc822, and there should not be any spurious headers.

Hi Mark. Yes, thanks. For plain text emails this is perfect in that it
gives me what I want (and I can customise the start/end of forwarded
message separators). For messages with text and html versions of the body
it is less ideal because it generates an email with three attachments -
one with the plain text of the message body, one with the html of the
message body and one that just contains the 'end of forwarded message'
separator. I'd prefer that the plain text message body was included inline
and that the html portion was the only attachment in this case. Is this
possible?
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: notmuch-emacs: forward messages inline

[Matthew Lear]

Hi Daniel,

> On 5 Nov 2015, at 21:42, Daniel Kahn Gillmor <d...@fifthhorseman.net> wrote:
>> On Fri 2015-11-06 04:51:53 +0900, Matthew Lear wrote:
>> Are there any recommended ways to selectively forward an email as
>> inline with notmuch-emacs rather than as an RFC822 attachment, eg have
>> a new shortcut key that can be used for this purpose in addition to f?
>> I have toggled one of the mml configuration variables (I forget
>> exactly what it was) which switched the forwarding method to inline
>> (ACAICT this also removed the ability of being able to forward as
>> RFC822), but it also included various header information in the body
>> of the email. Not good.
> 
> I suppose you could reply to the message (r) and just change the
> addresses, which would include the previous message "inline".  If you
> don't like that it leaks the previous message-id in the In-Reply-To and
> References headers, you can just kill them from the buffer manually.

I guess that's one way, but it's a bit of a faff. Unless it was possible to wrap
it all up in lisp, I don't really think it's a good option.

> But can i ask why you'd want this?  forwarded messages as RFC 822
> attachments are significantly more sane for any MUA to deal with.

No arguments on the 'being sane' front, although I have seen
notmuch-emacs fail to correctly formulate an RFC822 attachment of the
original email message a few times. I suspect this was due to MS Outlook
formatting but can't be sure, though. My main use of notmuch is at work
where I have to handle large amounts of email such as bug notifications from
a couple of systems, messages to/from lists, auto generated stuff for tracking,
plus the usual reams of corporate email from teams and colleagues. Notmuch
allows me to handle this fantastically. A common use case of forwarding
messages inline is to take an email already received, and send it onto
colleagues. It's not uncommon for this to initiate a new thread of conversation
and other people could be added to the thread as appropriate. If I were to
forward a message I received as an RFC822 attachment, in order for the
conversation to be coherent and contained in the text when other people were
added to the thread, the email containing my attachment would need to be
forwarded to (additional) recipients because 'replying to all' and including new
recipients wouldn't contain the original message. As I see it, to be able to
forward and include people starting a new thread based on the forwarded
message, it needs to be inline. Make sense?
--  Matt 
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

notmuch-emacs: forward messages inline

[Matthew Lear]

Hi all. Are there any recommended ways to selectively forward an email as 
inline with notmuch-emacs rather than as an RFC822 attachment, eg have a new 
shortcut key that can be used for this purpose in addition to f? I have toggled 
one of the mml configuration variables (I forget exactly what it was) which 
switched the forwarding method to inline (ACAICT this also removed the ability 
of being able to forward as RFC822), but it also included various header 
information in the body of the email. Not good.
I was wondering if any of you have such lisp-fu in your .emacs, or have any 
thoughts on how one could go about going this through a custom key binding.
Cheers,
  Matt


 
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

(emacs) customise key bindings taking text input

[Matthew Lear]

>
> On Tue, 14 Jul 2015, Matthew Lear  wrote:
>> Hi all,
>>
>> I was wondering if there was a way to add a new search similar to what
>> pressing s does, but make it use a predefined list of search terms in
>> addition to the text being entered? For example, I have reams of bug
>> notification emails which currently have various tags applied. If I
>> wanted
>> to search a subset of these tags with a number (bug id) in the subject,
>> right now I have to press s and type tag:foo and tag:bar and
>> subject:1234.
>> If I could add a new search option (eg 'B') and after pressing that I
>> simply enter the bug id and hit return, it would save me a lot of
>> typing.
>> I have custom searches set up which do the 'tag:foo and tag:bar' and I
>> can
>> filter on the subject, but that still means pressing f and typing
>> subject:1234.
>>
>> I fear I'm rambling here but hopefully I've conveyed what I'm trying to
>> do.
>> Is this possible?
>> Just trying to cut down on my typing ;-)
>
> I don't think there is anything built in but adding something like this
> snippet
>
> (defun my-search ( bug)
>   (interactive)
>   (let* ((bug (or bug (notmuch-read-query "Bug number: ")))
>(query (concat "tag:foo and tag:bar and subject:" bug)))
> (notmuch-search query)))
>
> to your .emacs file and binding it to B in the notmuch-common-keymap would
> probably
> do what you want.
>
> If people think this would be more generally useful it may be possible
> to add something like this to saved searches (ie a partial saved search)
> and then they would be accessible from notmuch-hello or via notmuch
> jump.

That's a good idea. Hooking into saved searches and then being able to
refine/enhance them via a customisable text input field would be really
nice.
The above lisp-fu works a treat. Thank you very much, and thanks for
fixing the mark read logic in notmuch-tree :-)
Cheers,
--  Matt

Re: (emacs) customise key bindings taking text input

[Matthew Lear]

 On Tue, 14 Jul 2015, Matthew Lear m...@bubblegen.co.uk wrote:
 Hi all,

 I was wondering if there was a way to add a new search similar to what
 pressing s does, but make it use a predefined list of search terms in
 addition to the text being entered? For example, I have reams of bug
 notification emails which currently have various tags applied. If I
 wanted
 to search a subset of these tags with a number (bug id) in the subject,
 right now I have to press s and type tag:foo and tag:bar and
 subject:1234.
 If I could add a new search option (eg 'B') and after pressing that I
 simply enter the bug id and hit return, it would save me a lot of
 typing.
 I have custom searches set up which do the 'tag:foo and tag:bar' and I
 can
 filter on the subject, but that still means pressing f and typing
 subject:1234.

 I fear I'm rambling here but hopefully I've conveyed what I'm trying to
 do.
 Is this possible?
 Just trying to cut down on my typing ;-)

 I don't think there is anything built in but adding something like this
 snippet

 (defun my-search (optional bug)
   (interactive)
   (let* ((bug (or bug (notmuch-read-query Bug number: )))
(query (concat tag:foo and tag:bar and subject: bug)))
 (notmuch-search query)))

 to your .emacs file and binding it to B in the notmuch-common-keymap would
 probably
 do what you want.

 If people think this would be more generally useful it may be possible
 to add something like this to saved searches (ie a partial saved search)
 and then they would be accessible from notmuch-hello or via notmuch
 jump.

That's a good idea. Hooking into saved searches and then being able to
refine/enhance them via a customisable text input field would be really
nice.
The above lisp-fu works a treat. Thank you very much, and thanks for
fixing the mark read logic in notmuch-tree :-)
Cheers,
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

(emacs) customise key bindings taking text input

[Matthew Lear]

Hi all,

I was wondering if there was a way to add a new search similar to what
pressing s does, but make it use a predefined list of search terms in
addition to the text being entered? For example, I have reams of bug
notification emails which currently have various tags applied. If I wanted
to search a subset of these tags with a number (bug id) in the subject,
right now I have to press s and type tag:foo and tag:bar and subject:1234.
If I could add a new search option (eg 'B') and after pressing that I
simply enter the bug id and hit return, it would save me a lot of typing.
I have custom searches set up which do the 'tag:foo and tag:bar' and I can
filter on the subject, but that still means pressing f and typing
subject:1234.

I fear I'm rambling here but hopefully I've conveyed what I'm trying to do.
Is this possible?
Just trying to cut down on my typing ;-)

Thanks,
--  Matt

(emacs) customise key bindings taking text input

[Matthew Lear]

Hi all,

I was wondering if there was a way to add a new search similar to what
pressing s does, but make it use a predefined list of search terms in
addition to the text being entered? For example, I have reams of bug
notification emails which currently have various tags applied. If I wanted
to search a subset of these tags with a number (bug id) in the subject,
right now I have to press s and type tag:foo and tag:bar and subject:1234.
If I could add a new search option (eg 'B') and after pressing that I
simply enter the bug id and hit return, it would save me a lot of typing.
I have custom searches set up which do the 'tag:foo and tag:bar' and I can
filter on the subject, but that still means pressing f and typing
subject:1234.

I fear I'm rambling here but hopefully I've conveyed what I'm trying to do.
Is this possible?
Just trying to cut down on my typing ;-)

Thanks,
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

quirks with subject searching

[Matthew Lear]

Hello all,
This may well be due to my novice use of search syntax, but I'm having
issues find needle in haystack within the subject of an email thread.
This simple example highlights the problem I'm having:

When using notmuch search subject:foo for threads with the following
subjects...

item: foo
an item: foo
an item: XYfoo1234
an item: XYfoo bar

... only the first two are found.

Also:
* notmuch searching for subject:"item: f" gives zero results.

* notmuch searching for subject:"(*foo*)" results in the first two threads
being found and also a thread with the word footprint in the middle of the
subject being found (subject is "memory footprint without feature").

* notmuch searching for subject:"an item: XYfoo1234" results in the third
being found.

Why doesn't notmuch search subject:foo find all four threads, and why
doesn't notmuch search subject:"item: f" find anything at all?

This was spurred on by me having a thread with the following as the subject..

Release Update: BT52_ETW_MD4.5.4alpha1_take1

..and notmuch search subject:4.5.4 finding nothing.

Something doesn't seem right here. man notmuch-search-terms doesn't seem
to explain this as far as I can see.
FWIW I'm using notmuch from git, master @ 294bb6d.
Any thoughts / tips?
Thanks,
--  Matt

quirks with subject searching

[Matthew Lear]

Hello all,
This may well be due to my novice use of search syntax, but I'm having
issues find needle in haystack within the subject of an email thread.
This simple example highlights the problem I'm having:

When using notmuch search subject:foo for threads with the following
subjects...

item: foo
an item: foo
an item: XYfoo1234
an item: XYfoo bar

... only the first two are found.

Also:
* notmuch searching for subject:item: f gives zero results.

* notmuch searching for subject:(*foo*) results in the first two threads
being found and also a thread with the word footprint in the middle of the
subject being found (subject is memory footprint without feature).

* notmuch searching for subject:an item: XYfoo1234 results in the third
being found.

Why doesn't notmuch search subject:foo find all four threads, and why
doesn't notmuch search subject:item: f find anything at all?

This was spurred on by me having a thread with the following as the subject..

Release Update: BT52_ETW_MD4.5.4alpha1_take1

..and notmuch search subject:4.5.4 finding nothing.

Something doesn't seem right here. man notmuch-search-terms doesn't seem
to explain this as far as I can see.
FWIW I'm using notmuch from git, master @ 294bb6d.
Any thoughts / tips?
Thanks,
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

emacs / vim - time taken to open threads

[Matthew Lear]

On 01/04/15 05:08, David Bremner wrote:
> Matthew Lear  writes:
> 
>> Here's two attachments obtained using the in-built profiler in emacs 24 -
>> one for memory profiling and one for cpu. This particular thread took > 30
>> secs to load and emacs was locked up the entire time. I'm no lisp expert,
>> but perhaps somebody could interpret this and possible suggest what could
>> be going on..? W.r.t. both cpu and memory usage, quite a lot of time seems
>> to be spent in indent-rigidly and notmuch-show-insert-thread.
>>
> 
> That's consistent with what I've seen with very large text attachements.
> Can you try the patch
> 
> id:1427540939-10055-1-git-send-email-markwalters1009 at gmail.com [1]
> 
> [1] http://mid.gmane.org/1427540939-10055-1-git-send-email-markwalters1009 at 
> gmail.com
> 

Sorry for the late reply. That has definitely made an improvement.
Opening a large thread takes much less time with the patch applied.
--  Matt

Re: emacs / vim - time taken to open threads

[Matthew Lear]

On 01/04/15 05:08, David Bremner wrote:
 Matthew Lear m...@bubblegen.co.uk writes:
 
 Here's two attachments obtained using the in-built profiler in emacs 24 -
 one for memory profiling and one for cpu. This particular thread took  30
 secs to load and emacs was locked up the entire time. I'm no lisp expert,
 but perhaps somebody could interpret this and possible suggest what could
 be going on..? W.r.t. both cpu and memory usage, quite a lot of time seems
 to be spent in indent-rigidly and notmuch-show-insert-thread.

 
 That's consistent with what I've seen with very large text attachements.
 Can you try the patch
 
 id:1427540939-10055-1-git-send-email-markwalters1...@gmail.com [1]
 
 [1] 
 http://mid.gmane.org/1427540939-10055-1-git-send-email-markwalters1...@gmail.com
 

Sorry for the late reply. That has definitely made an improvement.
Opening a large thread takes much less time with the patch applied.
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

emacs / vim - time taken to open threads

[Matthew Lear]

>> My experience is that the emacs interface is faster than the vim one.
>> At least I don't have an examples handy where emacs is slower than the
>> vim interface at rendering a thread. At a wild guess, I suspect it has
>> to do with how many attachments there are, and the emacs UI being
>> overenthusiastic about processing attachements.
>
> I've seen over 15 seconds of load time when opening some 20+ message
> thread -- and jumping through messages has been slow in emacs ui.

Here's two attachments obtained using the in-built profiler in emacs 24 -
one for memory profiling and one for cpu. This particular thread took > 30
secs to load and emacs was locked up the entire time. I'm no lisp expert,
but perhaps somebody could interpret this and possible suggest what could
be going on..? W.r.t. both cpu and memory usage, quite a lot of time seems
to be spent in indent-rigidly and notmuch-show-insert-thread.

Cheers,
--  Matt
-- next part --
An embedded and charset-unspecified text was scrubbed...
Name: cpu.txt
URL: 

-- next part --
An embedded and charset-unspecified text was scrubbed...
Name: mem.txt
URL: 

Re: emacs / vim - time taken to open threads

[Matthew Lear]

 My experience is that the emacs interface is faster than the vim one.
 At least I don't have an examples handy where emacs is slower than the
 vim interface at rendering a thread. At a wild guess, I suspect it has
 to do with how many attachments there are, and the emacs UI being
 overenthusiastic about processing attachements.

 I've seen over 15 seconds of load time when opening some 20+ message
 thread -- and jumping through messages has been slow in emacs ui.

Here's two attachments obtained using the in-built profiler in emacs 24 -
one for memory profiling and one for cpu. This particular thread took  30
secs to load and emacs was locked up the entire time. I'm no lisp expert,
but perhaps somebody could interpret this and possible suggest what could
be going on..? W.r.t. both cpu and memory usage, quite a lot of time seems
to be spent in indent-rigidly and notmuch-show-insert-thread.

Cheers,
--  Matt- command-execute   1,274,239,745  99%
 - call-interactively   1,274,239,745  99%
  - notmuch-search-show-thread  1,244,222,992  97%
   - notmuch-show   1,244,222,992  97%
- notmuch-show-build-buffer 1,244,184,644  97%
 - notmuch-show-insert-forest   1,052,295,452  82%
  - mapc1,052,295,452  82%
   - #compiled 0xd784f3   1,052,295,452  82%
- notmuch-show-insert-thread1,052,295,452  82%
 - mapc 1,052,295,452  82%
  - #compiled 0xd784cd1,052,295,452  82%
   - notmuch-show-insert-tree   1,052,295,452  82%
- notmuch-show-insert-thread  840,006,831  65%
 - mapc   840,006,831  65%
  - #compiled 0xd784cd  840,006,831  65%
   - notmuch-show-insert-tree 840,006,831  65%
- notmuch-show-insert-thread  475,772,280  37%
 - mapc   475,772,280  37%
  - #compiled 0xd784cd  475,772,280  37%
   - notmuch-show-insert-tree 475,772,280  37%
- notmuch-show-insert-thread  393,675,698  30%
 - mapc   393,675,698  30%
  - #compiled 0xd784cd  393,675,698  30%
   - notmuch-show-insert-tree 393,675,698  30%
- notmuch-show-insert-msg 393,670,578  30%
   indent-rigidly 378,976,399  29%
 + notmuch-show-insert-headers  8,573,181   0%
 + notmuch-show-strip-re   38,912   0%
 + notmuch-show-insert-headerline21,748   0%
 + notmuch-show-message-visible 5,264   0%
   notmuch-show-set-message-properties 3,096   
0%
   make-overlay 1,418   0%
+ notmuch-show-insert-msg  82,096,582   6%
- notmuch-show-insert-msg 364,233,495  28%
 + notmuch-show-insert-body   237,892,232  18%
   indent-rigidly 123,850,002   9%
 + notmuch-show-insert-headers377,077   0%
 + notmuch-show-insert-headerline  85,340   0%
 + notmuch-show-strip-re   20,556   0%
 + notmuch-show-headers-visible 4,208   0%
 + notmuch-show-message-visible 1,056   0%
- notmuch-show-insert-msg 212,283,421  16%
   indent-rigidly 155,843,920  12%
 + notmuch-show-insert-body51,496,368   4%
 + notmuch-show-insert-headers379,990   0%
 + notmuch-show-insert-headerline  93,694   0%
 + notmuch-show-strip-re   17,802   0%
 + notmuch-show-message-visible 1,056   0%
 + notmuch-show-set-message-properties  1,056   0%
 - notmuch-query-get-threads  180,741,671  14%
  - apply 180,741,671  14%
   - notmuch-call-notmuch-sexp180,741,671  14%
- notmuch-call-notmuch--helper175,917,905  13%
   apply  175,917,905  13%
+ make-temp-file 

emacs / vim - time taken to open threads

[Matthew Lear]

Hi all,
Continuing my journey with notmuch... It has been very enjoyable so far.
One thing which has been puzzling me greatly is the time it can take to
notmuch-show a moderately sized thread (eg one with 32 messages with a
three to four sentence paragraph per message - approx 32 x 35kB with
some small attachments). Opening the thread using notmuch-show in emacs
takes about six seconds. If I build the ruby bindings and load up
notmuch in vim or gvim, opening the same thread takes less than a
second. It's *loads* quicker.

I've rolled back and rolled forwards the version of emacs in my gentoo
installation (various versions of 24) and built it without all available
options, run it bare bones in X and without X. The behaviour is the
same. I've also stripped back my .emacs to nothing more than pulling in
notmuch.el but there is no change in the behaviour.

I've read some posts about emacs hanging when opening emails due to
attached files, but that was quite a while ago (pre 24). Some of the
emails in my thread have photos (a few hundred kB), others have URLs but
this shouldn't slow emacs down.

Despite being a vi guy for years, I prefer the emacs interface to
notmuch and really like what it provides. I'm sticking with it but there
is clearly a problem and I'd like to solve it. It's annoying when you
know that something isn't working as well as it should be.

Is it known that emacs is slower with notmuch or do you think I have
some kind of build / environment problem? Or could it be due to some of
the email content (doubtful) or something else such as encoding etc?

Cheers,
--  Matt

emacs / vim - time taken to open threads

[Matthew Lear]

Hi all,
Continuing my journey with notmuch... It has been very enjoyable so far.
One thing which has been puzzling me greatly is the time it can take to
notmuch-show a moderately sized thread (eg one with 32 messages with a
three to four sentence paragraph per message - approx 32 x 35kB with
some small attachments). Opening the thread using notmuch-show in emacs
takes about six seconds. If I build the ruby bindings and load up
notmuch in vim or gvim, opening the same thread takes less than a
second. It's *loads* quicker.

I've rolled back and rolled forwards the version of emacs in my gentoo
installation (various versions of 24) and built it without all available
options, run it bare bones in X and without X. The behaviour is the
same. I've also stripped back my .emacs to nothing more than pulling in
notmuch.el but there is no change in the behaviour.

I've read some posts about emacs hanging when opening emails due to
attached files, but that was quite a while ago (pre 24). Some of the
emails in my thread have photos (a few hundred kB), others have URLs but
this shouldn't slow emacs down.

Despite being a vi guy for years, I prefer the emacs interface to
notmuch and really like what it provides. I'm sticking with it but there
is clearly a problem and I'd like to solve it. It's annoying when you
know that something isn't working as well as it should be.

Is it known that emacs is slower with notmuch or do you think I have
some kind of build / environment problem? Or could it be due to some of
the email content (doubtful) or something else such as encoding etc?

Cheers,
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

emacs: notmuch-tree, reading messages and tags

[Matthew Lear]

Hi Mark, 
That would be great. Yes b and space are useful but even navigating using them 
doesn't clear the unread flag in the message pane so refreshing the 
notmuch-tree view still shows the message(s) as unread. 
I'll keep an eye on git for the changes going in. 
Cheers, 
 Matt 

On 11 March 2015 13:52:57 GMT+00:00, Mark Walters  wrote:
>
>Hi
>
>I think that is a bug (my fault) which happened when the unread tag
>handling went in. I will try and fix it.
>
>Incidentally you can scroll the message pane with space and b without
>having to switch frames.
>
>Best wishes
>
>Mark
>
>On Wed, 11 Mar 2015, Matthew Lear  wrote:
>> Hi,
>> If I switch to viewing a thread with notmuch-tree, I can navigate
>through,
>> and read messages using up/down + RET just fine. Each message is
>shown in
>> a seperate window. However, if point never enters that message
>window, the
>> unread tag is removed (strikethrough) from the message in the
>notmuch-tree
>> view but not from the message in the message window itself. If I
>refresh
>> the notmuch tree view, all messages still have their unread tag
>applied.
>>
>> The only time that the unread tag is removed is if point enters the
>> message window. For messages that are only a few lines long (eg a
>reply to
>> the previous message in the thread is only a few lines) and which can
>be
>> read without putting point in the message window to scroll, having to
>> specifically select the message window in order for the unread tag to
>be
>> removed from the message seems totally unnecessary and is
>inconvenient.
>>
>> Is this by design or am I using it wrongly? Is this behaviour
>configurable?
>>
>> Cheers,
>> --  Matt
>>
>> ___
>> notmuch mailing list
>> notmuch at notmuchmail.org
>> http://notmuchmail.org/mailman/listinfo/notmuch
-- next part --
An HTML attachment was scrubbed...
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20150311/92fe2f48/attachment.html>

emacs: notmuch-tree, reading messages and tags

[Matthew Lear]

Hi,
If I switch to viewing a thread with notmuch-tree, I can navigate through,
and read messages using up/down + RET just fine. Each message is shown in
a seperate window. However, if point never enters that message window, the
unread tag is removed (strikethrough) from the message in the notmuch-tree
view but not from the message in the message window itself. If I refresh
the notmuch tree view, all messages still have their unread tag applied.

The only time that the unread tag is removed is if point enters the
message window. For messages that are only a few lines long (eg a reply to
the previous message in the thread is only a few lines) and which can be
read without putting point in the message window to scroll, having to
specifically select the message window in order for the unread tag to be
removed from the message seems totally unnecessary and is inconvenient.

Is this by design or am I using it wrongly? Is this behaviour configurable?

Cheers,
--  Matt

Re: emacs: notmuch-tree, reading messages and tags

[Matthew Lear]

Hi Mark, 
That would be great. Yes b and space are useful but even navigating using them 
doesn't clear the unread flag in the message pane so refreshing the 
notmuch-tree view still shows the message(s) as unread. 
I'll keep an eye on git for the changes going in. 
Cheers, 
 Matt 

On 11 March 2015 13:52:57 GMT+00:00, Mark Walters markwalters1...@gmail.com 
wrote:

Hi

I think that is a bug (my fault) which happened when the unread tag
handling went in. I will try and fix it.

Incidentally you can scroll the message pane with space and b without
having to switch frames.

Best wishes

Mark

On Wed, 11 Mar 2015, Matthew Lear m...@bubblegen.co.uk wrote:
 Hi,
 If I switch to viewing a thread with notmuch-tree, I can navigate
through,
 and read messages using up/down + RET just fine. Each message is
shown in
 a seperate window. However, if point never enters that message
window, the
 unread tag is removed (strikethrough) from the message in the
notmuch-tree
 view but not from the message in the message window itself. If I
refresh
 the notmuch tree view, all messages still have their unread tag
applied.

 The only time that the unread tag is removed is if point enters the
 message window. For messages that are only a few lines long (eg a
reply to
 the previous message in the thread is only a few lines) and which can
be
 read without putting point in the message window to scroll, having to
 specifically select the message window in order for the unread tag to
be
 removed from the message seems totally unnecessary and is
inconvenient.

 Is this by design or am I using it wrongly? Is this behaviour
configurable?

 Cheers,
 --  Matt

 ___
 notmuch mailing list
 notmuch@notmuchmail.org
 http://notmuchmail.org/mailman/listinfo/notmuch
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

emacs: notmuch-tree, reading messages and tags

[Matthew Lear]

Hi,
If I switch to viewing a thread with notmuch-tree, I can navigate through,
and read messages using up/down + RET just fine. Each message is shown in
a seperate window. However, if point never enters that message window, the
unread tag is removed (strikethrough) from the message in the notmuch-tree
view but not from the message in the message window itself. If I refresh
the notmuch tree view, all messages still have their unread tag applied.

The only time that the unread tag is removed is if point enters the
message window. For messages that are only a few lines long (eg a reply to
the previous message in the thread is only a few lines) and which can be
read without putting point in the message window to scroll, having to
specifically select the message window in order for the unread tag to be
removed from the message seems totally unnecessary and is inconvenient.

Is this by design or am I using it wrongly? Is this behaviour configurable?

Cheers,
--  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Inspired but need help searching folder names containing spaces

[Matthew Lear]

> On 20 Feb 2015, at 18:01, Jani Nikula  wrote:
> 
>> On Fri, 20 Feb 2015, Matthew Lear  wrote:
>> A couple of others things have surprised me, though. Using the date search
>> option, such as date:"today" or date:"yesterday", hasn't quite thrown up
>> what I expected. Searching with date:"today" shows up no results (which I
>> know is wrong) and searching with date:"yesterday" shows up a handful of
>> emails all from the same person, in different threads which all arrived on
>> a day in May 2014... Got to be a bug there somewhere :-D
> 
> This is a known limitation in the date: prefix queries. You have to
> specify it as a range, for example date:today..today (from the beginning
> of today until the end of today). If the underlying Xapian search engine
> is not given a range expression (with the "..") it searches for the
> words "date today" without punctuation, matching whatever it matches,
> and notmuch does not even get an error for it.

Thanks. Explains everything. Any thoughts on searching on 'to' only? I'm not 
sure if this is also related to what Xapian needs to be provided with. If not, 
and if only searching in the to, cc and bcc fields of the header is possible, 
maybe we could consider adopting and integrating case sensitive syntax in 
notmuch such as TO:me rather than to:me..?
--  Matt

Inspired but need help searching folder names containing spaces

[Matthew Lear]

Hi David,

>> notmuch search 'folder:"Deleted Items"'
>>
>> .. throws up zero results.
>>
>
> Hmm. I tried to duplicate your situation here, and it works for me. Two
> things to try:
>
> 1) notmuch search --output=files $QUERY
>
> where $QUERY is some query (using e.g. id:) that matches one of those
> messages. Check that you have have the whole anchored path from the top
> level.
>
> 2) NOTMUCH_DEBUG_QUERY=1 notmuch count -'folder:"Deleted Items"'
>
> have a look for XFOLDER:Deleted Items
>
> 3) try both of the above with "--exclude=false" in case you have tagged
> them with an excluded tag.  Note that by default "deleted" is such a tag.
> You can change the excluded tags with "notmuch config"

As part of still playing around with notmuch and strategies for tagging
etc, I decided to start afresh with the database so I deleted it. After
running notmuch new, I could quite happily search with notmuch search
'folder:"Deleted Items"'. This surprised me greatly because I know that I
didn't apply any tags to mails in Deleted Items (couldn't find any in
there to tag) and I also already tried searching with --exclude=false. In
any case, it seems that searching in folders with spaces now works. Hurrah
:-)

A couple of others things have surprised me, though. Using the date search
option, such as date:"today" or date:"yesterday", hasn't quite thrown up
what I expected. Searching with date:"today" shows up no results (which I
know is wrong) and searching with date:"yesterday" shows up a handful of
emails all from the same person, in different threads which all arrived on
a day in May 2014... Got to be a bug there somewhere :-D

One last thing which I'd like to explore if I may is searching for emails
sent to me. By default, I think notmuch classes 'to me' as 'to or cc or
bcc me'. Is there a way to search/filter on emails sent only to me (ie
where my name / address is only in the to: field in the message header)?
This seems a reasonable requirement / use case so I'm assuming that
there's a good reason that notmuch views it differently.

An example is an automated bug tracking system which sends emails to the
current assignee of the bug (to) and also watchers (cc). I'd like to be
able to filter these emails differently (ie bugs I'm watching versus bugs
I'm responsible for fixing). Apart from using other fields in the message
body (which may be subject to change), there doesn't seem an easy way to
do this which is surprising given notmuch's flexibility and power.

Thoughts?

I'm still exploring strategies for tagging and searching etc. The links on
the wiki are helpful. The possibilities are almost infinite and while
everybody's mail is different, having a great deal of flexibility can
sometimes makes it tricky to decide on exactly what's best. Still, it's an
enjoyable task, despite being time consuming, and is well worth the
effort.

  Matt

Re: Inspired but need help searching folder names containing spaces

[Matthew Lear]

Hi David,

 notmuch search 'folder:Deleted Items'

 .. throws up zero results.


 Hmm. I tried to duplicate your situation here, and it works for me. Two
 things to try:

 1) notmuch search --output=files $QUERY

 where $QUERY is some query (using e.g. id:) that matches one of those
 messages. Check that you have have the whole anchored path from the top
 level.

 2) NOTMUCH_DEBUG_QUERY=1 notmuch count -'folder:Deleted Items'

 have a look for XFOLDER:Deleted Items

 3) try both of the above with --exclude=false in case you have tagged
 them with an excluded tag.  Note that by default deleted is such a tag.
 You can change the excluded tags with notmuch config

As part of still playing around with notmuch and strategies for tagging
etc, I decided to start afresh with the database so I deleted it. After
running notmuch new, I could quite happily search with notmuch search
'folder:Deleted Items'. This surprised me greatly because I know that I
didn't apply any tags to mails in Deleted Items (couldn't find any in
there to tag) and I also already tried searching with --exclude=false. In
any case, it seems that searching in folders with spaces now works. Hurrah
:-)

A couple of others things have surprised me, though. Using the date search
option, such as date:today or date:yesterday, hasn't quite thrown up
what I expected. Searching with date:today shows up no results (which I
know is wrong) and searching with date:yesterday shows up a handful of
emails all from the same person, in different threads which all arrived on
a day in May 2014... Got to be a bug there somewhere :-D

One last thing which I'd like to explore if I may is searching for emails
sent to me. By default, I think notmuch classes 'to me' as 'to or cc or
bcc me'. Is there a way to search/filter on emails sent only to me (ie
where my name / address is only in the to: field in the message header)?
This seems a reasonable requirement / use case so I'm assuming that
there's a good reason that notmuch views it differently.

An example is an automated bug tracking system which sends emails to the
current assignee of the bug (to) and also watchers (cc). I'd like to be
able to filter these emails differently (ie bugs I'm watching versus bugs
I'm responsible for fixing). Apart from using other fields in the message
body (which may be subject to change), there doesn't seem an easy way to
do this which is surprising given notmuch's flexibility and power.

Thoughts?

I'm still exploring strategies for tagging and searching etc. The links on
the wiki are helpful. The possibilities are almost infinite and while
everybody's mail is different, having a great deal of flexibility can
sometimes makes it tricky to decide on exactly what's best. Still, it's an
enjoyable task, despite being time consuming, and is well worth the
effort.

  Matt

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Inspired but need help searching folder names containing spaces

[Matthew Lear]

Hello all,

As part of trying to cope with an extremely high amount of email at
work, after a couple years of using X1 Search on Windows in conjunction
with MS Outlook and many years getting incredibly frustrated at the lack
of, and then failures of, email search features built into linux desktop
environments such as KDE and Gnome, I craved a better tool for linux.
Recently, I stumbled on the combination of offlineimap and notmuch and
I'm convinced that this will make my world a better place. I'm still
finding my way in terms of optimum ways to tag and filter my email, but
this is a fun ongoing task that will continue over the coming weeks and
months and I'm seeing a light at the end of the tunnel. So firstly,
thank you.

So...One thing I'm struggling with is searching for emails in folders
which contain spaces in their name. The IMAP server at work is Exchange
based and I have Inbox, Deleted Items and Sent Items folders (among many
others) all at the same top level. I'm using the default offlineimap
configuration which uses '.' as the separator. After a sync I have...

mail/maildir/
??? INBOX
??? Deleted Items
...
??? Sent Items

on the file system. These directories have the cur, new and tmp sub
directories associated with maildir. All ok so far.

I have several emails in Deleted Items and Sent Items. I can ls and cat
them just fine. What I cannot do, however, is search them with notmuch.
Reading the very helpful replies in this thread
(http://notmuchmail.org/pipermail/notmuch/2014/019053.html), I thought
I'd stumbled on a solution. However, searching with..

notmuch search 'folder:"Deleted Items"'

.. throws up zero results.

Confusingly, if I sync a different IMAP account with offlineimap and
index it with notmuch, I can successfully search folders which contain
spaces in their name, but in that account, these folders exist *under*
INBOX, eg

mail/maildir2/
??? INBOX
??? INBOX.Deleted Items
...
??? INBOX.Sent Items

So searching with..

notmuch search 'folder:"INBOX.Deleted Items"'

..works.

So why the difference? AFAICT the syntax I'm using is correct and should
allow the string to be searched in the Xapian database. I just don't
understand why one directory level seems to make a difference between a
successful and an unsuccessful search.

Any thoughts?

If it helps I'm running notmuch 0.19 and xapian 1.2.18 on Gentoo linux.

I suppose I don't need to sync Deleted Items, and I can tag -inbox
emails sent by me, but I'd still like to be able to search both these
folders if I need to.

Can anybody help suggest what could be going wrong?

Many thanks from a very happy new notmuch user.
--  Matt

Inspired but need help searching folder names containing spaces

[Matthew Lear]

Hello all,

As part of trying to cope with an extremely high amount of email at
work, after a couple years of using X1 Search on Windows in conjunction
with MS Outlook and many years getting incredibly frustrated at the lack
of, and then failures of, email search features built into linux desktop
environments such as KDE and Gnome, I craved a better tool for linux.
Recently, I stumbled on the combination of offlineimap and notmuch and
I'm convinced that this will make my world a better place. I'm still
finding my way in terms of optimum ways to tag and filter my email, but
this is a fun ongoing task that will continue over the coming weeks and
months and I'm seeing a light at the end of the tunnel. So firstly,
thank you.

So...One thing I'm struggling with is searching for emails in folders
which contain spaces in their name. The IMAP server at work is Exchange
based and I have Inbox, Deleted Items and Sent Items folders (among many
others) all at the same top level. I'm using the default offlineimap
configuration which uses '.' as the separator. After a sync I have...

mail/maildir/
├── INBOX
├── Deleted Items
...
└── Sent Items

on the file system. These directories have the cur, new and tmp sub
directories associated with maildir. All ok so far.

I have several emails in Deleted Items and Sent Items. I can ls and cat
them just fine. What I cannot do, however, is search them with notmuch.
Reading the very helpful replies in this thread
(http://notmuchmail.org/pipermail/notmuch/2014/019053.html), I thought
I'd stumbled on a solution. However, searching with..

notmuch search 'folder:Deleted Items'

.. throws up zero results.

Confusingly, if I sync a different IMAP account with offlineimap and
index it with notmuch, I can successfully search folders which contain
spaces in their name, but in that account, these folders exist *under*
INBOX, eg

mail/maildir2/
├── INBOX
├── INBOX.Deleted Items
...
└── INBOX.Sent Items

So searching with..

notmuch search 'folder:INBOX.Deleted Items'

..works.

So why the difference? AFAICT the syntax I'm using is correct and should
allow the string to be searched in the Xapian database. I just don't
understand why one directory level seems to make a difference between a
successful and an unsuccessful search.

Any thoughts?

If it helps I'm running notmuch 0.19 and xapian 1.2.18 on Gentoo linux.

I suppose I don't need to sync Deleted Items, and I can tag -inbox
emails sent by me, but I'd still like to be able to search both these
folders if I need to.

Can anybody help suggest what could be going wrong?

Many thanks from a very happy new notmuch user.
--  Matt
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch