[PATCH] Use the message cite variable instead of a hard coded function

[Todd]

---
 emacs/notmuch-mua.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 2c58886..e90ca55 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -220,8 +220,8 @@ list."
(date (plist-get original-headers :Date))
(start (point)))
 
-   ;; message-cite-original constructs a citation line based on the From 
and Date
-   ;; headers of the original message, which are assumed to be in the 
buffer.
+   ;; message-cite-function should construct a citation line based on the 
From and
+   ;; Date headers of the original message, which are assumed to be in the 
buffer.
(insert "From: " from "\n")
(insert "Date: " date "\n\n")
 
@@ -233,7 +233,7 @@ list."
(set-mark (point))
(goto-char start)
;; Quote the original message according to the user's configured style.
-   (message-cite-original)))
+   (funcall message-cite-function)))
 
 ;; Crypto processing based crypto content of the original message
 (when process-crypto
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Use the message cite variable instead of a hard coded function

[Todd]

It's defaulted to message-cite-original in Emacs 24, and may have been
back to Emacs 22.3 ("This variable was introduced, or its default value
was changed, in version 22.3 of Emacs.").  

I can re-work the patch to fall back to message-cite-original if it's
been unset, if that's preferred.

- Todd


>>>>> "DB" == David Bremner  writes:
DB> Hi Todd;

DB> Thanks for the patch. What happens to the behaviour of notmuch-reply for
DB> people that have not customized this variable?

DB> d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] emacs: Use the message cite variable

[Todd]

I checked in the Emacs 22.1 source code, and there it was a defaulted
to message-cite-original as well.  This patch falls back to
message-cite-original in the unlikely case that someone has customized
message-cite-function to nil.

- Todd

---
 emacs/notmuch-mua.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 2c58886..2a01455 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -220,8 +220,8 @@ list."
(date (plist-get original-headers :Date))
(start (point)))

-   ;; message-cite-original constructs a citation line based on the From 
and Date
-   ;; headers of the original message, which are assumed to be in the 
buffer.
+   ;; message-cite-function should construct a citation line based on the 
From and
+   ;; Date headers of the original message, which are assumed to be in the 
buffer.
(insert "From: " from "\n")
(insert "Date: " date "\n\n")

@@ -233,7 +233,7 @@ list."
(set-mark (point))
(goto-char start)
;; Quote the original message according to the user's configured style.
-   (message-cite-original)))
+   (funcall (or message-cite-function #'message-cite-original

 ;; Crypto processing based crypto content of the original message
 (when process-crypto
--
1.9.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] Index Content-Type of attachments with a contenttype prefix

[Todd]

I wanted to tag messages with calendar invitations, but couldn't as
the information wasn't indexed.

This patch allows for queries for like:

Find calendar invites
- contenttype:text/calendar or contenttype:applicaton/ics

Find any image attachments
- contenttype:image

Find all patches
- contenttype:text/x-patch


- Todd

---
 NEWS   |  6 ++
 completion/notmuch-completion.bash |  2 +-
 doc/man7/notmuch-search-terms.rst  |  6 ++
 emacs/notmuch.el   |  2 +-
 lib/database.cc|  1 +
 lib/index.cc   |  5 +
 test/T190-multipart.sh | 32 
 7 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 44e8d05..5f4622c 100644
--- a/NEWS
+++ b/NEWS
@@ -15,6 +15,12 @@ keyboard shortcuts to saved searches.
 Command-Line Interface
 --

+There is a new `contenttype:` search prefix
+
+  The new `contenttype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by `notmuch
+  insert`. See the `notmuch-search-terms` manual page for details.
+
 Stopped `notmuch dump` failing if someone writes to the database

   The dump command now takes the write lock when running. This
diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..05b5969 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: contenttype: 
tag: id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..d126ce6 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):

 -  attachment:

+-  contenttype:
+
 -  tag: (or is:)

 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.

+The **contenttype:** prefix can be used to search for specific
+content-types of attachments to email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..702700c 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "contenttype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..a7a64c9 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "contenttype",   "XCONTENTTYPE"},
 { "subject",   "XSUBJECT"},
 };

diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..c3f7c6b 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -346,6 +346,11 @@ _index_mime_part (notmuch_message_t *message,
return;
 }

+GMimeContentType*  content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   _notmuch_message_gen_terms (message, "contenttype", 
g_mime_content_type_to_string(content_type));
+}
+
 if (GMIME_IS_MESSAGE_PART (part)) {
GMimeMessage *mime_message;

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..e3270a7 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: t...@electricoding.com
+Subject: o

Re: [PATCH] Index Content-Type of attachments with a contenttype prefix

[Todd]

>>>>> "DB" == David Bremner  writes:

DB> Todd  writes:
>> I wanted to tag messages with calendar invitations, but couldn't as
>> the information wasn't indexed.
>> 
>> This patch allows for queries for like:
>> 
>> Find calendar invites
>> - contenttype:text/calendar or contenttype:applicaton/ics
>> 
>> Find any image attachments
>> - contenttype:image
>> 
>> Find all patches
>> - contenttype:text/x-patch

DB> The main issue,  that I won't really address in this message (because 
I'm
DB> hoping Austin finds time to comment) is upgrading the database. The 
short
DB> version is that a new "database feature" needs to be created.

I just discovered and starting using notmuch this week, so I'm not
entirely familiar with the project yet.  I had suspected there might
be more work :)  I'll look into the "database feature".

My other thought was to just index the content-type with the
attachment prefix.  This would have made the first search that I
attempted (without reading the docs/source) work
(e.g. attachment:text/calendar or attachment:application/ics).

Is this preferred to adding a new search term?


>> --- a/NEWS
>> +++ b/NEWS
>> @@ -15,6 +15,12 @@ keyboard shortcuts to saved searches.
>> Command-Line Interface
>> --

DB> Minor point, 0.19 was released, you should start a new NEWS section for
DB> 0.20 with date UNRELEASED

I'll make that change.

>> --- a/completion/notmuch-completion.bash
>> +++ b/completion/notmuch-completion.bash

DB> If you can without too much suffering, it would be nice to update the 
zsh completion at the same
DB> time.

I don't use zsh, but I'll look into what's needed.

>> +The **contenttype:** prefix can be used to search for specific
>> +content-types of attachments to email messages (as specified by the
>> +sender).
>> +

DB> I'm not 100% sure, but I did wonder if the docs should mention MIME
DB> somewhere, for people searching.

Agreed.

>> { "attachment",  "XATTACHMENT" },
>> +{ "contenttype","XCONTENTTYPE"},
>> { "subject", "XSUBJECT"},

DB> I didn't work through all the details, but I did wonder if it was in
DB> some sense redundant to be indexing contenttype and also the existing
DB> attachement and encrypted pseudotags.  I guess this might be one of
DB> those cases where we are stuck with the extra indexing for now, until we
DB> sort out some query parsing issues.

DB> d



- Todd
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3] emacs: Use the message cite variable instead of a calling cite-message-original directly

[Todd]

This fixes the test failure.  I didn't have dtach installed, so the
tests were being skipped earlier.  As to why the test fails, it
appears that the default for message-cite-function may depend on the
version of Emacs that you use.

This will likely cause notmuch replying to behave differently for
users if they upgrade, even if they don't customize
message-cite-function (in the case that it is not
message-cite-original).

Regarding the quotation changes in the test, it was not being
interpreted correctly without the double quotes.  I didn't look into
it in much depth, just fixed it for my case.

- Todd


---
 emacs/notmuch-mua.el | 6 +++---
 test/T310-emacs.sh   | 7 ---
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 2c58886..66a6b65 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -220,8 +220,8 @@ list."
(date (plist-get original-headers :Date))
(start (point)))
 
-   ;; message-cite-original constructs a citation line based on the From 
and Date
-   ;; headers of the original message, which are assumed to be in the 
buffer.
+   ;; message-cite-function should construct a citation line based on the 
From and
+   ;; Date headers of the original message, which are assumed to be in the 
buffer.
(insert "From: " from "\n")
(insert "Date: " date "\n\n")
 
@@ -233,7 +233,7 @@ list."
(set-mark (point))
(goto-char start)
;; Quote the original message according to the user's configured style.
-   (message-cite-original)))
+   (funcall (or message-cite-function 'message-cite-original
 
 ;; Crypto processing based crypto content of the original message
 (when process-crypto
diff --git a/test/T310-emacs.sh b/test/T310-emacs.sh
index d72799b..74d9f58 100755
--- a/test/T310-emacs.sh
+++ b/test/T310-emacs.sh
@@ -378,10 +378,11 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED
 
 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_emacs '(let ((message-hidden-headers ''()))
-   (notmuch-show "id:20091118002059.067214ed@hikari")
+test_emacs "(let ((message-hidden-headers '())
+  (message-cite-function 'message-cite-original))
+   (notmuch-show \"id:20091118002059.067214ed@hikari\")
(notmuch-show-reply)
-   (test-output))'
+   (test-output))"
 sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat <EXPECTED
 From: Notmuch Test Suite 
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] Index Content-Type of attachments with a contenttype prefix

[Todd]

>>>>> "Jani" == Jani Nikula  writes:

Jani> On Sat, 10 Jan 2015, Todd  wrote:
>> I wanted to tag messages with calendar invitations, but couldn't as
>> the information wasn't indexed.
>> 
>> This patch allows for queries for like:
>> 
>> Find calendar invites
>> - contenttype:text/calendar or contenttype:applicaton/ics
>> 
>> Find any image attachments
>> - contenttype:image
>> 
>> Find all patches
>> - contenttype:text/x-patch
>> 
>> 
>> - Todd
>> 
>> ---
>> NEWS   |  6 ++
>> completion/notmuch-completion.bash |  2 +-
>> doc/man7/notmuch-search-terms.rst  |  6 ++
>> emacs/notmuch.el   |  2 +-
>> lib/database.cc|  1 +
>> lib/index.cc   |  5 +
>> test/T190-multipart.sh | 32 

Jani> IMO these could be split into several patches.

No problem, I'll split them up the next time I post.

>> 7 files changed, 52 insertions(+), 2 deletions(-)
>> 
>> diff --git a/NEWS b/NEWS
>> index 44e8d05..5f4622c 100644
>> --- a/NEWS
>> +++ b/NEWS
>> @@ -15,6 +15,12 @@ keyboard shortcuts to saved searches.
>> Command-Line Interface
>> --
>> 
>> +There is a new `contenttype:` search prefix
>> +
>> +  The new `contenttype:` search prefix allows searching for the
>> +  content-type of attachments, which is now indexed by `notmuch
>> +  insert`. See the `notmuch-search-terms` manual page for details.
>> +

Jani> Admittedly I did not have the time to dig into details, but I think
Jani> "attachment" is misleading, as it's really all mime parts, right?

Jani> Will this also index the Content-Type: header of the message itself,
Jani> regardless of whether it has mime structure or not? Maybe it
Jani> should?

Yes, all mime-parts. It does not index the Content-Type of the
message itself.  That probably wouldn't be difficult to add if it's
a desired feature, but if there are plans for indexing other message
headers it may fit better there.

I also wasn't too happy with a "contenttype" keyword and debated
just indexing the information under "attachment" along with the
filename.

>> Stopped `notmuch dump` failing if someone writes to the database
>> 
>> The dump command now takes the write lock when running. This
>> diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
>> index d58dc8b..05b5969 100644
>> --- a/completion/notmuch-completion.bash
>> +++ b/completion/notmuch-completion.bash
>> @@ -61,7 +61,7 @@ _notmuch_search_terms()
>> sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
>> ;;
>> *)
>> -local search_terms="from: to: subject: attachment: tag: id: 
thread: folder: path: date:"
>> +local search_terms="from: to: subject: attachment: 
contenttype: tag: id: thread: folder: path: date:"
>> compopt -o nospace
>> COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
>> ;;
>> diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
>> index 1acdaa0..d126ce6 100644
>> --- a/doc/man7/notmuch-search-terms.rst
>> +++ b/doc/man7/notmuch-search-terms.rst
>> @@ -40,6 +40,8 @@ indicate user-supplied values):
>> 
>> -  attachment:
>> 
>> +-  contenttype:
>> +
>> -  tag: (or is:)
>> 
>> -  id:
>> @@ -66,6 +68,10 @@ by including quotation marks around the phrase, 
immediately following
>> The **attachment:** prefix can be used to search for specific filenames
>> (or extensions) of attachments to email messages.
>> 
>> +The **contenttype:** prefix can be used to search for specific
>> +content-types of attachments to email messages (as specified by the
>> +sender).
>> +
>> For **tag:** and **is:** valid tag values include **inbox** and
>> **unread** by default for new messages added by **notmuch new** as well
>> as any other tag values added manually with **notmuch tag**.
>> diff --git a/emacs/notmuch.el b/emacs/notmuch.el
>> index 218486a..702700c 100644
&g

[PATCH v2 1/5] Add a failing unit test for indexed mime types

[Todd]

---
 test/T190-multipart.sh | 32 
 1 file changed, 32 insertions(+)

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..26f7c7a 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: t...@example.com
+Subject: odd content types
+Date: Mon, 12 Jan 2014 18:12:32 +
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 
(i486-pc-linux-gnu)
+Message-ID: 
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: application/unique_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==
+Content-Type: text/some_other_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+cat content_types >> ${MAIL_DIR}/odd_content_type
 notmuch new > /dev/null
 
 test_begin_subtest "--format=text --part=0, full message"
@@ -727,4 +751,12 @@ test_begin_subtest "html parts included"
 notmuch show --format=json --include-html id:htmlmessage > OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"
 
+test_begin_subtest "indexes mime-type"
+output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+
 test_done
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 0/5] Index the content-type of MIME parts

[Todd]

This set of patches allows for queries for like:

Find calendar invites
- mimetype:text/calendar or mimetype:application/ics

Find any image attachments
- mimetype:image

Find all patches
- mimetype:text/x-patch

It is a rework of a previous set of patches that now adds a new
database feature, only available in newly created databases, which
indicates that all of the mime parts are indexed.  Indexing of new
messages on an existing database should work fine as well, but the
feature won't be enabled via upgrade.

Code reviews welcome, I'm not familiar with this code base.

Todd (5):
  Add a failing unit test for indexed mime types
  Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature
  Add indexing for the mimetype term
  Update completions for Emacs and bash
  Update documentation

 NEWS   | 16 
 completion/notmuch-completion.bash |  2 +-
 doc/man7/notmuch-search-terms.rst  |  6 ++
 emacs/notmuch.el   |  2 +-
 lib/database-private.h | 15 ---
 lib/database.cc| 11 +--
 lib/index.cc   |  6 ++
 test/T190-multipart.sh | 32 
 8 files changed, 83 insertions(+), 7 deletions(-)

--
1.9.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 4/5] Update completions for Emacs and bash

[Todd]

This adds completions for Emacs and bash, ZSH does not appear to have
completions for search terms.
---
 completion/notmuch-completion.bash | 2 +-
 emacs/notmuch.el   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..05b5969 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: contenttype: 
tag: id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..ab00454 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "mimetype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 3/5] Add indexing for the mimetype term

[Todd]

---
 lib/database.cc | 1 +
 lib/index.cc| 6 ++
 2 files changed, 7 insertions(+)

diff --git a/lib/database.cc b/lib/database.cc
index 2de60f8..7c7a267 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "mimetype",  "XMIMETYPE"},
 { "subject",   "XSUBJECT"},
 };
 
diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..2e27558 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -318,6 +318,12 @@ _index_mime_part (notmuch_message_t *message,
return;
 }
 
+
+GMimeContentType*  content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   _notmuch_message_gen_terms (message, "mimetype", 
g_mime_content_type_to_string(content_type));
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 5/5] Update documentation

[Todd]

---
 NEWS  | 16 
 doc/man7/notmuch-search-terms.rst |  6 ++
 2 files changed, 22 insertions(+)

diff --git a/NEWS b/NEWS
index 44e8d05..0df7977 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,19 @@
+Notmuch 0.20 (TBD)
+=
+
+Overview
+
+
+Command-Line Interface
+--
+
+There is a new `mimetype:` search prefix
+
+  The new `mimetype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by `notmuch
+  insert`. See the `notmuch-search-terms` manual page for details.
+
+
 Notmuch 0.19 (2014-11-14)
 =
 
diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..8d1acfc 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):
 
 -  attachment:
 
+-  mimetype:
+
 -  tag: (or is:)
 
 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.
 
+The **mimetype:** prefix can be used to search for the specific
+content-types of mime parts within email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

[Todd]

---
 lib/database-private.h | 15 ---
 lib/database.cc| 10 --
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 15e03cc..6d6fa2c 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -92,6 +92,14 @@ enum _notmuch_features {
  *
  * Introduced: version 3. */
 NOTMUCH_FEATURE_GHOSTS = 1 << 4,
+
+
+/* If set, then the database was created after the introduction of
+ * indexed mime types. If unset, then the database may contain a
+ * mixture of messages with indexed and non-indexed mime types.
+ *
+ * Introduced: version 3. */
+NOTMUCH_FEATURE_INDEXED_MIMETYPES = 1 << 5,
 };
 
 /* In C++, a named enum is its own type, so define bitwise operators
@@ -161,9 +169,10 @@ struct _notmuch_database {
 
 /* Current database features.  If any of these are missing from a
  * database, request an upgrade.
- * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES is not included because
- * upgrade doesn't currently introduce the feature (though brand new
- * databases will have it). */
+ * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES and
+ * NOTMUCH_FEATURE_INDEXED_MIMETYPES are not included because upgrade
+ * doesn't currently introduce the features (though brand new databases
+ * will have it). */
 #define NOTMUCH_FEATURES_CURRENT \
 (NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_DIRECTORY_DOCS | \
  NOTMUCH_FEATURE_BOOL_FOLDER | NOTMUCH_FEATURE_GHOSTS)
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..2de60f8 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -304,6 +304,11 @@ static const struct {
   "exact folder:/path: search", "rw" },
 { NOTMUCH_FEATURE_GHOSTS,
   "mail documents for missing messages", "w"},
+/* Knowledge of the index mime-types are not required for reading
+ * a database because a reader will just be unable to query
+ * them. */
+{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
+  "mime-types in database", "w"},
 };
 
 const char *
@@ -646,9 +651,10 @@ notmuch_database_create (const char *path, 
notmuch_database_t **database)
 if (status)
goto DONE;
 
-/* Upgrade doesn't add this feature to existing databases, but new
- * databases have it. */
+/* Upgrade doesn't add these feature to existing databases, but
+ * new databases have them. */
 notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
+notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;
 
 status = notmuch_database_upgrade (notmuch, NULL, NULL);
 if (status) {
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: make citation function customizable.

[Todd]

Looks good to me. I applied the patch and it works fine as well.

- Todd

>>>>> "DB" == David Bremner  writes:

DB> Make a new customizable variable instead of relying on
DB> message-cite-function because the default for the latter changed
DB> between emacs releases.

DB> The defcustom is borrowed from the message.el source, with minor
DB> modifications.
DB> ---

DB> Thanks to Todd for tracking down the wibbly wobbly timey wimey 
behaviour of this variable. I propose to add our own variable to avoid 
surprising notmuch-emacs users.

DB>  emacs/notmuch-mua.el | 23 ---
DB>  1 file changed, 20 insertions(+), 3 deletions(-)

DB> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
DB> index 2c58886..33f1399 100644
DB> --- a/emacs/notmuch-mua.el
DB> +++ b/emacs/notmuch-mua.el
DB> @@ -75,6 +75,22 @@ list."
DB>:type '(repeat string)
DB>:group 'notmuch-send)
 
DB> +(defgroup notmuch-reply nil
DB> +  "Replying to messages in notmuch"
DB> +  :group 'notmuch)
DB> +
DB> +(defcustom notmuch-mua-cite-function 'message-cite-original
DB> +  "*Function for citing an original message.
DB> +Predefined functions include `message-cite-original' and
DB> +`message-cite-original-without-signature'.
DB> +Note that these functions use `mail-citation-hook' if that is non-nil."
DB> +  :type '(radio (function-item message-cite-original)
DB> +   (function-item message-cite-original-without-signature)
DB> +   (function-item sc-cite-original)
DB> +   (function :tag "Other"))
DB> +  :link '(custom-manual "(message)Insertion Variables")
DB> +  :group 'notmuch-reply)
DB> +
DB>  ;;
 
DB>  (defun notmuch-mua-get-switch-function ()
DB> @@ -220,8 +236,9 @@ list."
DB> (date (plist-get original-headers :Date))
DB> (start (point)))
 
DB> -   ;; message-cite-original constructs a citation line based on 
the >From and Date
DB> -   ;; headers of the original message, which are assumed to be in 
the buffer.
DB> +   ;; notmuch-mua-cite-function constructs a citation line based
DB> +   ;; on the From and Date headers of the original message, which
DB> +   ;; are assumed to be in the buffer.
DB> (insert "From: " from "\n")
DB> (insert "Date: " date "\n\n")
 
DB> @@ -233,7 +250,7 @@ list."
DB> (set-mark (point))
DB> (goto-char start)
DB> ;; Quote the original message according to the user's 
configured style.
DB> -   (message-cite-original)))
DB> +   (funcall notmuch-mua-cite-function)))
 
DB>  ;; Crypto processing based crypto content of the original message
DB>  (when process-crypto
DB> -- 
DB> 2.1.4

DB> ___
DB> notmuch mailing list
DB> notmuch@notmuchmail.org
DB> http://notmuchmail.org/mailman/listinfo/notmuch


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 3/5] Add indexing for the mimetype term

[Todd]

Adds the indexing and removes the broken test flag
---
 lib/database.cc|  1 +
 lib/index.cc   | 10 ++
 test/T190-multipart.sh |  4 
 3 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 0d2c417..3974e2e 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "mimetype",  "XMIMETYPE"},
 { "subject",   "XSUBJECT"},
 };
 
diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..c88ed8d 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -318,6 +318,16 @@ _index_mime_part (notmuch_message_t *message,
return;
 }
 
+GMimeContentType *content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   char *mime_string = g_mime_content_type_to_string(content_type);
+   if (mime_string)
+   {
+   _notmuch_message_gen_terms (message, "mimetype", mime_string);
+   g_free(mime_string);
+   }
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 57f1b61..de7d361 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -752,22 +752,18 @@ notmuch show --format=json --include-html id:htmlmessage 
> OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"
 
 test_begin_subtest "indexes mime-type #1"
-test_subtest_known_broken
 output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
 
 test_begin_subtest "indexes mime-type #2"
-test_subtest_known_broken
 output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
 
 test_begin_subtest "indexes mime-type #3"
-test_subtest_known_broken
 output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
 
 test_begin_subtest "search for non-existent mime-type"
-test_subtest_known_broken
 output=$(notmuch search mimetype:non-existent/mime-type | 
notmuch_search_sanitize)
 test_expect_equal "$output" ""
 
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 4/5] Update completions for Emacs and bash

[Todd]

This adds completions for Emacs and bash, ZSH does not appear to have
completions for search terms.
---
 completion/notmuch-completion.bash | 2 +-
 emacs/notmuch.el   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..39320f8 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: mimetype: tag: 
id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..ab00454 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "mimetype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 5/5] Update documentation

[Todd]

---
 NEWS  | 11 +++
 doc/man7/notmuch-search-terms.rst |  6 ++
 2 files changed, 17 insertions(+)

diff --git a/NEWS b/NEWS
index abb5ad7..baa4519 100644
--- a/NEWS
+++ b/NEWS
@@ -1,4 +1,5 @@
 Notmuch 0.20 (UNRELEASED)
+=
 
 Contrib
 ---
@@ -7,6 +8,16 @@ Contrib
 previously provided by `notmuch-deliver` should now be provided by
 `notmuch insert`, provided by the main notmuch binary.
 
+Command-Line Interface
+--
+
+There is a new `mimetype:` search prefix
+
+  The new `mimetype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by `notmuch
+  insert`. See the `notmuch-search-terms` manual page for details.
+
+
 Notmuch 0.19 (2014-11-14)
 =
 
diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..8d1acfc 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):
 
 -  attachment:
 
+-  mimetype:
+
 -  tag: (or is:)
 
 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.
 
+The **mimetype:** prefix can be used to search for the specific
+content-types of mime parts within email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 1/5] Add failing unit tests for indexed mime types

[Todd]

---
 test/T190-multipart.sh | 44 
 1 file changed, 44 insertions(+)

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..57f1b61 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: t...@example.com
+Subject: odd content types
+Date: Mon, 12 Jan 2014 18:12:32 +
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 
(i486-pc-linux-gnu)
+Message-ID: 
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: application/unique_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==
+Content-Type: text/some_other_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+cat content_types >> ${MAIL_DIR}/odd_content_type
 notmuch new > /dev/null
 
 test_begin_subtest "--format=text --part=0, full message"
@@ -727,4 +751,24 @@ test_begin_subtest "html parts included"
 notmuch show --format=json --include-html id:htmlmessage > OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"
 
+test_begin_subtest "indexes mime-type #1"
+test_subtest_known_broken
+output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #2"
+test_subtest_known_broken
+output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #3"
+test_subtest_known_broken
+output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "search for non-existent mime-type"
+test_subtest_known_broken
+output=$(notmuch search mimetype:non-existent/mime-type | 
notmuch_search_sanitize)
+test_expect_equal "$output" ""
+
 test_done
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v3 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

[Todd]

---
 lib/database-private.h | 15 ---
 lib/database.cc| 10 --
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 15e03cc..6d6fa2c 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -92,6 +92,14 @@ enum _notmuch_features {
  *
  * Introduced: version 3. */
 NOTMUCH_FEATURE_GHOSTS = 1 << 4,
+
+
+/* If set, then the database was created after the introduction of
+ * indexed mime types. If unset, then the database may contain a
+ * mixture of messages with indexed and non-indexed mime types.
+ *
+ * Introduced: version 3. */
+NOTMUCH_FEATURE_INDEXED_MIMETYPES = 1 << 5,
 };
 
 /* In C++, a named enum is its own type, so define bitwise operators
@@ -161,9 +169,10 @@ struct _notmuch_database {
 
 /* Current database features.  If any of these are missing from a
  * database, request an upgrade.
- * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES is not included because
- * upgrade doesn't currently introduce the feature (though brand new
- * databases will have it). */
+ * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES and
+ * NOTMUCH_FEATURE_INDEXED_MIMETYPES are not included because upgrade
+ * doesn't currently introduce the features (though brand new databases
+ * will have it). */
 #define NOTMUCH_FEATURES_CURRENT \
 (NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_DIRECTORY_DOCS | \
  NOTMUCH_FEATURE_BOOL_FOLDER | NOTMUCH_FEATURE_GHOSTS)
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..0d2c417 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -304,6 +304,11 @@ static const struct {
   "exact folder:/path: search", "rw" },
 { NOTMUCH_FEATURE_GHOSTS,
   "mail documents for missing messages", "w"},
+/* Knowledge of the index mime-types are not required for reading
+ * a database because a reader will just be unable to query
+ * them. */
+{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
+  "indexed MIME types", "w"},
 };
 
 const char *
@@ -646,9 +651,10 @@ notmuch_database_create (const char *path, 
notmuch_database_t **database)
 if (status)
goto DONE;
 
-/* Upgrade doesn't add this feature to existing databases, but new
- * databases have it. */
+/* Upgrade doesn't add these feature to existing databases, but
+ * new databases have them. */
 notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
+notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;
 
 status = notmuch_database_upgrade (notmuch, NULL, NULL);
 if (status) {
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 0/2] Fixes for issues discovered with -fsanitize=address

[Todd]

I built a copy of notmuch with gcc 4.8's new -fsanitize=address
feature and ran the test bench. It pointed out these two issues.


Todd (2):
  lib: Fix potential invalid read past an empty string
  lib: Fix use after free

 lib/thread.cc | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)

--
1.9.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 1/2] lib: Fix potential invalid read past an empty string

[Todd]

==22884== ERROR: AddressSanitizer: heap-buffer-overflow on address 
0x60168291 at pc 0x7ff6295680e5 bp 0x7fff4ab9aa40 sp 0x7fff4ab9aa08
READ of size 1 at 0x60168291 thread T0
#0 0x7ff6295680e4 in __interceptor_strcmp ??:?
#1 0x44763b in _thread_add_message 
/home/todd/.apps/notmuch/lib/thread.cc:255
#2 0x4459e8 in notmuch_threads_get /home/todd/.apps/notmuch/lib/query.cc:496
#3 0x41e2a7 in do_search_threads 
/home/todd/.apps/notmuch/notmuch-search.c:131
#4 0x40a408 in main /home/todd/.apps/notmuch/notmuch.c:345
#5 0x7ff627cb9ec4 in __libc_start_main 
/build/buildd/eglibc-2.19/csu/libc-start.c:287
#6 0x40abf3 in _start ??:?
0x60168291 is located 0 bytes to the right of 97-byte region 
[0x60168230,0x60168291)
allocated by thread T0 here:
#0 0x7ff62956e41a in malloc ??:?
#1 0x7ff628b8ab5d in talloc_strdup ??:?
---
 lib/thread.cc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index 4542505..4c49f98 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -251,8 +251,8 @@ _thread_add_message (notmuch_thread_t *thread,
 term != NULL;
 term = term->next)
{
-   /* We ignore initial 'K'. */
-   if (strcmp(tag, (term->string + 1)) == 0) {
+   /* Check for an empty string, and then ignore initial 'K'. */
+   if (*(term->string) && strcmp(tag, (term->string + 1)) == 0) {
message_excluded = TRUE;
break;
}
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v1 2/2] lib: Fix use after free

[Todd]

_thread_set_subject_from_message sometimes replaces the subject, making the
cur_subject point to free'd memory

==6550== ERROR: AddressSanitizer: heap-use-after-free on address 0x601abec0 
at pc 0x4464a4 bp 0x7fffa40be910 sp 0x7fffa40be908
READ of size 1 at 0x601abec0 thread T0
#0 0x4464a3 in _thread_add_matched_message 
/home/todd/.apps/notmuch/lib/thread.cc:369
#1 0x443c2c in notmuch_threads_get /home/todd/.apps/notmuch/lib/query.cc:496
#2 0x41d947 in do_search_threads 
/home/todd/.apps/notmuch/notmuch-search.c:131
#3 0x40a3fe in main /home/todd/.apps/notmuch/notmuch.c:345
#4 0x7f4e535b4ec4 in __libc_start_main 
/build/buildd/eglibc-2.19/csu/libc-start.c:287
#5 0x40abe6 in _start ??:?
0x601abec0 is located 96 bytes inside of 134-byte region 
[0x601abe60,0x601abee6)
freed by thread T0 here:
#0 0x7f4e54e6933a in __interceptor_free ??:?
#1 0x7f4e54482fab in _talloc_free ??:?
previously allocated by thread T0 here:
#0 0x7f4e54e6941a in malloc ??:?
#1 0x7f4e54485b5d in talloc_strdup ??:?
---
 lib/thread.cc | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index 4c49f98..9847cf8 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -353,10 +353,8 @@ _thread_add_matched_message (notmuch_thread_t *thread,
 {
 time_t date;
 notmuch_message_t *hashed_message;
-const char *cur_subject;
 
 date = notmuch_message_get_date (message);
-cur_subject = notmuch_thread_get_subject(thread);
 
 if (date < thread->oldest || ! thread->matched_messages) {
thread->oldest = date;
@@ -366,6 +364,7 @@ _thread_add_matched_message (notmuch_thread_t *thread,
 
 if (date > thread->newest || ! thread->matched_messages) {
thread->newest = date;
+   const char *cur_subject = notmuch_thread_get_subject(thread);
if (sort != NOTMUCH_SORT_OLDEST_FIRST || EMPTY_STRING(cur_subject))
_thread_set_subject_from_message (thread, message);
 }
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v3 3/5] Add indexing for the mimetype term

[Todd]

>>>>> "DB" == David Bremner  writes:

DB> Todd  writes:
>> Adds the indexing and removes the broken test flag
>> ---
>> lib/database.cc|  1 +
>> lib/index.cc   | 10 ++
>> test/T190-multipart.sh |  4 
>> 3 files changed, 11 insertions(+), 4 deletions(-)
>>
>> diff --git a/lib/database.cc b/lib/database.cc
>> index 0d2c417..3974e2e 100644
>> --- a/lib/database.cc
>> +++ b/lib/database.cc
>> @@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
>> { "from","XFROM" },
>> { "to",  "XTO" },
>> { "attachment",  "XATTACHMENT" },
>> +{ "mimetype",   "XMIMETYPE"},
>> { "subject", "XSUBJECT"},
>> };

DB> I think the commit message should articulate why we are indexing this as
DB> a probabilistic prefix, rather than as a boolean prefix. In particular,
DB> this gives people a last chance to complain.

DB> The reference I know is http://xapian.org/docs/queryparser.html

DB> If I understand correctly (it would be great if you could test this
DB> Todd) , with a probabilistic prefix,

DB>mimetime:pdf

DB> will match

DB> application/pdf
DB> image/pdf
DB> application/x-pdf
DB> application/x-ext-pdf

DB> but not

DB> application/x-bzpdf
DB> application/x-gzpdf
DB> application/x-xzpdf

I just tested, and it does work this way with your examples.  I
*believe* from reading the docs, that xapian is treating the full
MIME-type queries as phrase searches anyway due to the embedded
slashes.

From http://xapian.org/docs/queryparser.html:

 A phrase surrounded with double quotes ("") matches documents
 containing that exact phrase. Hyphenated words are also treated
 as phrases, as are cases such as filenames and email addresses
 (e.g. /etc/passwd or presid...@whitehouse.gov).

I think that we'll get good behavior from the types of queries that
will typically be performed due to this automatic phrasing.



DB> On the whole, this is probably more beneficial than bad.  The downside
DB> of probabilistic prefixes/fields is that they are not "anchored", so
DB> there is no easy way to distinguish

DB>   application/pdf

DB> from

DB>   pdf
DB>   application/x-pdf

DB> I guess in a perfect world this would also be explained in
DB> notmuch-search-terms(7), but that's pretty much orthogonal to this
DB> series.

If separate messages with application/pdf and application/x-pdf are
indexed, then:

mimetype:application/x-pdf finds only the application/x-pdf
mimetype:application/pdf finds only the application/pdf
mimetype:pdf finds both of the messages

I am fairly sure that this behaviour is a result of the automatic
phrasing mentioned above.

- Todd

DB> d


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4 3/5] Add indexing for the mimetype term

[Todd]

This adds the indexing support for the "mimetype:" term and removes
the broken test flag.  The indexing is probablistic in Xapian terms,
which gives a better experience to end users.  Standard content-types
of the form "foo/bar" are automatically interpreted as phrases in
Xapian due to the embedded slash.

Assume, separate messages with application/pdf and application/x-pdf
are indexed, then:

- mimetype:application/x-pdf will find only the application/x-pdf
- mimetype:application/pdf will find only the application/pdf
- mimetype:pdf will find both of the messages
---
 lib/database.cc|  1 +
 lib/index.cc   | 10 ++
 test/T190-multipart.sh |  3 ---
 3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 0d2c417..3974e2e 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "mimetype",  "XMIMETYPE"},
 { "subject",   "XSUBJECT"},
 };
 
diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..c88ed8d 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -318,6 +318,16 @@ _index_mime_part (notmuch_message_t *message,
return;
 }
 
+GMimeContentType *content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   char *mime_string = g_mime_content_type_to_string(content_type);
+   if (mime_string)
+   {
+   _notmuch_message_gen_terms (message, "mimetype", mime_string);
+   g_free(mime_string);
+   }
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index a97dc28..ad8d29e 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -752,17 +752,14 @@ notmuch show --format=json --include-html id:htmlmessage 
> OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"
 
 test_begin_subtest "indexes mime-type #1"
-test_subtest_known_broken
 output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
 
 test_begin_subtest "indexes mime-type #2"
-test_subtest_known_broken
 output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
 
 test_begin_subtest "indexes mime-type #3"
-test_subtest_known_broken
 output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
 
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4 0/5] Index the content-type of MIME parts

[Todd]

I think I've finished incorporating the feedback.  The final
notmuch-search-terms.rst could use more details, but it should
probably occur after the recent patch that was posted documenting the
probablistic indexing/searching has been committed.

Todd (5):
  test: Add failing unit tests for indexed mime types
  Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature
  Add indexing for the mimetype term
  Update completions for Emacs and bash
  Update documentation

 NEWS   | 11 +++
 completion/notmuch-completion.bash |  2 +-
 doc/man7/notmuch-search-terms.rst  |  6 ++
 emacs/notmuch.el   |  2 +-
 lib/database-private.h | 15 ---
 lib/database.cc| 11 +--
 lib/index.cc   | 10 ++
 test/T190-multipart.sh | 36 
 8 files changed, 86 insertions(+), 7 deletions(-)

--
1.9.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4 1/5] test: Add failing unit tests for indexed mime types

[Todd]

Adds three failing unit tests for searching of mime-types.

An attempt was made at adding a negative test (i.e. searching for a
non-existent mime-type and ensuring it didn't return a message), but
that test would always pass making it pointless.
---
 test/T190-multipart.sh | 39 +++
 1 file changed, 39 insertions(+)

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..a97dc28 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: t...@example.com
+Subject: odd content types
+Date: Mon, 12 Jan 2014 18:12:32 +
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 
(i486-pc-linux-gnu)
+Message-ID: 
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: application/unique_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==
+Content-Type: text/some_other_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+cat content_types >> ${MAIL_DIR}/odd_content_type
 notmuch new > /dev/null
 
 test_begin_subtest "--format=text --part=0, full message"
@@ -727,4 +751,19 @@ test_begin_subtest "html parts included"
 notmuch show --format=json --include-html id:htmlmessage > OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"
 
+test_begin_subtest "indexes mime-type #1"
+test_subtest_known_broken
+output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #2"
+test_subtest_known_broken
+output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #3"
+test_subtest_known_broken
+output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
 test_done
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4 4/5] Update completions for Emacs and bash

[Todd]

This adds completions for both Emacs and bash. ZSH does not appear to
have completions for search terms.
---
 completion/notmuch-completion.bash | 2 +-
 emacs/notmuch.el   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..39320f8 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: mimetype: tag: 
id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..ab00454 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "mimetype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

[Todd]

This feature will exist in all newly created databases, but there is
no upgrade provided for it.  If this flag exists, it indicates that
the database was created after the indexed MIME-types feature was
added.
---
 lib/database-private.h | 15 ---
 lib/database.cc| 10 --
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 15e03cc..6d6fa2c 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -92,6 +92,14 @@ enum _notmuch_features {
  *
  * Introduced: version 3. */
 NOTMUCH_FEATURE_GHOSTS = 1 << 4,
+
+
+/* If set, then the database was created after the introduction of
+ * indexed mime types. If unset, then the database may contain a
+ * mixture of messages with indexed and non-indexed mime types.
+ *
+ * Introduced: version 3. */
+NOTMUCH_FEATURE_INDEXED_MIMETYPES = 1 << 5,
 };
 
 /* In C++, a named enum is its own type, so define bitwise operators
@@ -161,9 +169,10 @@ struct _notmuch_database {
 
 /* Current database features.  If any of these are missing from a
  * database, request an upgrade.
- * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES is not included because
- * upgrade doesn't currently introduce the feature (though brand new
- * databases will have it). */
+ * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES and
+ * NOTMUCH_FEATURE_INDEXED_MIMETYPES are not included because upgrade
+ * doesn't currently introduce the features (though brand new databases
+ * will have it). */
 #define NOTMUCH_FEATURES_CURRENT \
 (NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_DIRECTORY_DOCS | \
  NOTMUCH_FEATURE_BOOL_FOLDER | NOTMUCH_FEATURE_GHOSTS)
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..0d2c417 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -304,6 +304,11 @@ static const struct {
   "exact folder:/path: search", "rw" },
 { NOTMUCH_FEATURE_GHOSTS,
   "mail documents for missing messages", "w"},
+/* Knowledge of the index mime-types are not required for reading
+ * a database because a reader will just be unable to query
+ * them. */
+{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
+  "indexed MIME types", "w"},
 };
 
 const char *
@@ -646,9 +651,10 @@ notmuch_database_create (const char *path, 
notmuch_database_t **database)
 if (status)
goto DONE;
 
-/* Upgrade doesn't add this feature to existing databases, but new
- * databases have it. */
+/* Upgrade doesn't add these feature to existing databases, but
+ * new databases have them. */
 notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
+notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;
 
 status = notmuch_database_upgrade (notmuch, NULL, NULL);
 if (status) {
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v4 5/5] Update documentation

[Todd]

Adds new entry to the NEWS file, and updates the search terms section
of the man page.  The search terms section needs to be updated again
once the new section in the documentation covering probablistic terms
has been committed.
---
 NEWS  | 11 +++
 doc/man7/notmuch-search-terms.rst |  6 ++
 2 files changed, 17 insertions(+)

diff --git a/NEWS b/NEWS
index 53e06aa..a4f2a3f 100644
--- a/NEWS
+++ b/NEWS
@@ -1,4 +1,5 @@
 Notmuch 0.20 (UNRELEASED)
+=
 
 Documentation
 -
@@ -13,6 +14,16 @@ Contrib
 previously provided by `notmuch-deliver` should now be provided by
 `notmuch insert`, provided by the main notmuch binary.
 
+Command-Line Interface
+--
+
+There is a new `mimetype:` search prefix
+
+  The new `mimetype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by. See the
+  `notmuch-search-terms` manual page for details.
+
+
 Notmuch 0.19 (2014-11-14)
 =
 
diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..315e849 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):
 
 -  attachment:
 
+-  mimetype:
+
 -  tag: (or is:)
 
 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.
 
+The **mimetype:** prefix will be used to match text from the
+content-types of MIME parts within email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
-- 
1.9.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Unable to search for emails with content with a given MIME type

[Todd]

Petter,

The feature you wanted was committed a few days ago (searching with a
"mimetype:" prefix).  Searching for "mimetype:calendar" would get any
"text/calendar" or "ics/calendar" attachments.

(Replying to the list so it will appear in the list archive as well.)

Thanks,
Todd

Petter Reinholdtsen  writes:

> Not quite sure where to send a feature request, so I try to follow the
> instructions for bug reporst.
>
> I would like to be able to search for emails with content with a given
> MIME type.  The latest use case I ran into was my need to locate Notes
> and Exchange emails with text/calendar attachments, but I've also needed
> to find application/pdf and application/vnd.oasis.opendocument.text
> emails.
>
> Please consider extending notmuch to be able to search for specific mime
> types, for example using attachment:text/calendar or mime:text/calendar
> or something like that.  Or perhaps just index up the mime content type
> value, to allow me to search for "tag:attachment and text/calendar".
>
> PS: Please CC me on replies, as I am not subscribed to the list.
> -- 
> Happy hacking
> Petter Reinholdtsen


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] Use the message cite variable instead of a hard coded function

[Todd]

---
 emacs/notmuch-mua.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 2c58886..e90ca55 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -220,8 +220,8 @@ list."
(date (plist-get original-headers :Date))
(start (point)))

-   ;; message-cite-original constructs a citation line based on the From 
and Date
-   ;; headers of the original message, which are assumed to be in the 
buffer.
+   ;; message-cite-function should construct a citation line based on the 
From and
+   ;; Date headers of the original message, which are assumed to be in the 
buffer.
(insert "From: " from "\n")
(insert "Date: " date "\n\n")

@@ -233,7 +233,7 @@ list."
(set-mark (point))
(goto-char start)
;; Quote the original message according to the user's configured style.
-   (message-cite-original)))
+   (funcall message-cite-function)))

 ;; Crypto processing based crypto content of the original message
 (when process-crypto
-- 
1.9.1

[PATCH] Use the message cite variable instead of a hard coded function

[Todd]

It's defaulted to message-cite-original in Emacs 24, and may have been
back to Emacs 22.3 ("This variable was introduced, or its default value
was changed, in version 22.3 of Emacs.").  

I can re-work the patch to fall back to message-cite-original if it's
been unset, if that's preferred.

- Todd


>>>>> "DB" == David Bremner  writes:
DB> Hi Todd;

DB> Thanks for the patch. What happens to the behaviour of notmuch-reply for
DB> people that have not customized this variable?

DB> d

[PATCH v2] emacs: Use the message cite variable

[Todd]

I checked in the Emacs 22.1 source code, and there it was a defaulted
to message-cite-original as well.  This patch falls back to
message-cite-original in the unlikely case that someone has customized
message-cite-function to nil.

- Todd

---
 emacs/notmuch-mua.el | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 2c58886..2a01455 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -220,8 +220,8 @@ list."
(date (plist-get original-headers :Date))
(start (point)))

-   ;; message-cite-original constructs a citation line based on the From 
and Date
-   ;; headers of the original message, which are assumed to be in the 
buffer.
+   ;; message-cite-function should construct a citation line based on the 
From and
+   ;; Date headers of the original message, which are assumed to be in the 
buffer.
(insert "From: " from "\n")
(insert "Date: " date "\n\n")

@@ -233,7 +233,7 @@ list."
(set-mark (point))
(goto-char start)
;; Quote the original message according to the user's configured style.
-   (message-cite-original)))
+   (funcall (or message-cite-function #'message-cite-original

 ;; Crypto processing based crypto content of the original message
 (when process-crypto
--
1.9.1

[PATCH] Index Content-Type of attachments with a contenttype prefix

[Todd]

I wanted to tag messages with calendar invitations, but couldn't as
the information wasn't indexed.

This patch allows for queries for like:

Find calendar invites
- contenttype:text/calendar or contenttype:applicaton/ics

Find any image attachments
- contenttype:image

Find all patches
- contenttype:text/x-patch


- Todd

---
 NEWS   |  6 ++
 completion/notmuch-completion.bash |  2 +-
 doc/man7/notmuch-search-terms.rst  |  6 ++
 emacs/notmuch.el   |  2 +-
 lib/database.cc|  1 +
 lib/index.cc   |  5 +
 test/T190-multipart.sh | 32 
 7 files changed, 52 insertions(+), 2 deletions(-)

diff --git a/NEWS b/NEWS
index 44e8d05..5f4622c 100644
--- a/NEWS
+++ b/NEWS
@@ -15,6 +15,12 @@ keyboard shortcuts to saved searches.
 Command-Line Interface
 --

+There is a new `contenttype:` search prefix
+
+  The new `contenttype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by `notmuch
+  insert`. See the `notmuch-search-terms` manual page for details.
+
 Stopped `notmuch dump` failing if someone writes to the database

   The dump command now takes the write lock when running. This
diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..05b5969 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: contenttype: 
tag: id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..d126ce6 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):

 -  attachment:

+-  contenttype:
+
 -  tag: (or is:)

 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.

+The **contenttype:** prefix can be used to search for specific
+content-types of attachments to email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..702700c 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "contenttype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..a7a64c9 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "contenttype",   "XCONTENTTYPE"},
 { "subject",   "XSUBJECT"},
 };

diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..c3f7c6b 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -346,6 +346,11 @@ _index_mime_part (notmuch_message_t *message,
return;
 }

+GMimeContentType*  content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   _notmuch_message_gen_terms (message, "contenttype", 
g_mime_content_type_to_string(content_type));
+}
+
 if (GMIME_IS_MESSAGE_PART (part)) {
GMimeMessage *mime_message;

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..e3270a7 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: todd at electricoding.com
+Subject

[PATCH] Index Content-Type of attachments with a contenttype prefix

[Todd]

>>>>> "DB" == David Bremner  writes:

DB> Todd  writes:
>> I wanted to tag messages with calendar invitations, but couldn't as
>> the information wasn't indexed.
>> 
>> This patch allows for queries for like:
>> 
>> Find calendar invites
>> - contenttype:text/calendar or contenttype:applicaton/ics
>> 
>> Find any image attachments
>> - contenttype:image
>> 
>> Find all patches
>> - contenttype:text/x-patch

DB> The main issue,  that I won't really address in this message (because 
I'm
DB> hoping Austin finds time to comment) is upgrading the database. The 
short
DB> version is that a new "database feature" needs to be created.

I just discovered and starting using notmuch this week, so I'm not
entirely familiar with the project yet.  I had suspected there might
be more work :)  I'll look into the "database feature".

My other thought was to just index the content-type with the
attachment prefix.  This would have made the first search that I
attempted (without reading the docs/source) work
(e.g. attachment:text/calendar or attachment:application/ics).

Is this preferred to adding a new search term?


>> --- a/NEWS
>> +++ b/NEWS
>> @@ -15,6 +15,12 @@ keyboard shortcuts to saved searches.
>> Command-Line Interface
>> --

DB> Minor point, 0.19 was released, you should start a new NEWS section for
DB> 0.20 with date UNRELEASED

I'll make that change.

>> --- a/completion/notmuch-completion.bash
>> +++ b/completion/notmuch-completion.bash

DB> If you can without too much suffering, it would be nice to update the 
zsh completion at the same
DB> time.

I don't use zsh, but I'll look into what's needed.

>> +The **contenttype:** prefix can be used to search for specific
>> +content-types of attachments to email messages (as specified by the
>> +sender).
>> +

DB> I'm not 100% sure, but I did wonder if the docs should mention MIME
DB> somewhere, for people searching.

Agreed.

>> { "attachment",  "XATTACHMENT" },
>> +{ "contenttype","XCONTENTTYPE"},
>> { "subject", "XSUBJECT"},

DB> I didn't work through all the details, but I did wonder if it was in
DB> some sense redundant to be indexing contenttype and also the existing
DB> attachement and encrypted pseudotags.  I guess this might be one of
DB> those cases where we are stuck with the extra indexing for now, until we
DB> sort out some query parsing issues.

DB> d



- Todd

[PATCH v3] emacs: Use the message cite variable instead of a calling cite-message-original directly

[Todd]

This fixes the test failure.  I didn't have dtach installed, so the
tests were being skipped earlier.  As to why the test fails, it
appears that the default for message-cite-function may depend on the
version of Emacs that you use.

This will likely cause notmuch replying to behave differently for
users if they upgrade, even if they don't customize
message-cite-function (in the case that it is not
message-cite-original).

Regarding the quotation changes in the test, it was not being
interpreted correctly without the double quotes.  I didn't look into
it in much depth, just fixed it for my case.

- Todd


---
 emacs/notmuch-mua.el | 6 +++---
 test/T310-emacs.sh   | 7 ---
 2 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
index 2c58886..66a6b65 100644
--- a/emacs/notmuch-mua.el
+++ b/emacs/notmuch-mua.el
@@ -220,8 +220,8 @@ list."
(date (plist-get original-headers :Date))
(start (point)))

-   ;; message-cite-original constructs a citation line based on the From 
and Date
-   ;; headers of the original message, which are assumed to be in the 
buffer.
+   ;; message-cite-function should construct a citation line based on the 
From and
+   ;; Date headers of the original message, which are assumed to be in the 
buffer.
(insert "From: " from "\n")
(insert "Date: " date "\n\n")

@@ -233,7 +233,7 @@ list."
(set-mark (point))
(goto-char start)
;; Quote the original message according to the user's configured style.
-   (message-cite-original)))
+   (funcall (or message-cite-function 'message-cite-original

 ;; Crypto processing based crypto content of the original message
 (when process-crypto
diff --git a/test/T310-emacs.sh b/test/T310-emacs.sh
index d72799b..74d9f58 100755
--- a/test/T310-emacs.sh
+++ b/test/T310-emacs.sh
@@ -378,10 +378,11 @@ EOF
 test_expect_equal_file OUTPUT EXPECTED

 test_begin_subtest "Reply within emacs to a multipart/mixed message"
-test_emacs '(let ((message-hidden-headers ''()))
-   (notmuch-show "id:20091118002059.067214ed at hikari")
+test_emacs "(let ((message-hidden-headers '())
+  (message-cite-function 'message-cite-original))
+   (notmuch-show \"id:20091118002059.067214ed at hikari\")
(notmuch-show-reply)
-   (test-output))'
+   (test-output))"
 sed -i -e 's,^User-Agent: Notmuch/.* Emacs/.*,User-Agent: Notmuch/XXX 
Emacs/XXX,' OUTPUT
 cat <EXPECTED
 From: Notmuch Test Suite 
-- 
1.9.1

[PATCH] Index Content-Type of attachments with a contenttype prefix

[Todd]

>>>>> "Jani" == Jani Nikula  writes:

Jani> On Sat, 10 Jan 2015, Todd  wrote:
>> I wanted to tag messages with calendar invitations, but couldn't as
>> the information wasn't indexed.
>> 
>> This patch allows for queries for like:
>> 
>> Find calendar invites
>> - contenttype:text/calendar or contenttype:applicaton/ics
>> 
>> Find any image attachments
>> - contenttype:image
>> 
>> Find all patches
>> - contenttype:text/x-patch
>> 
>> 
>> - Todd
>> 
>> ---
>> NEWS   |  6 ++
>> completion/notmuch-completion.bash |  2 +-
>> doc/man7/notmuch-search-terms.rst  |  6 ++
>> emacs/notmuch.el   |  2 +-
>> lib/database.cc|  1 +
>> lib/index.cc   |  5 +
>> test/T190-multipart.sh | 32 

Jani> IMO these could be split into several patches.

No problem, I'll split them up the next time I post.

>> 7 files changed, 52 insertions(+), 2 deletions(-)
>> 
>> diff --git a/NEWS b/NEWS
>> index 44e8d05..5f4622c 100644
>> --- a/NEWS
>> +++ b/NEWS
>> @@ -15,6 +15,12 @@ keyboard shortcuts to saved searches.
>> Command-Line Interface
>> --
>> 
>> +There is a new `contenttype:` search prefix
>> +
>> +  The new `contenttype:` search prefix allows searching for the
>> +  content-type of attachments, which is now indexed by `notmuch
>> +  insert`. See the `notmuch-search-terms` manual page for details.
>> +

Jani> Admittedly I did not have the time to dig into details, but I think
Jani> "attachment" is misleading, as it's really all mime parts, right?

Jani> Will this also index the Content-Type: header of the message itself,
Jani> regardless of whether it has mime structure or not? Maybe it
Jani> should?

Yes, all mime-parts. It does not index the Content-Type of the
message itself.  That probably wouldn't be difficult to add if it's
a desired feature, but if there are plans for indexing other message
headers it may fit better there.

I also wasn't too happy with a "contenttype" keyword and debated
just indexing the information under "attachment" along with the
filename.

>> Stopped `notmuch dump` failing if someone writes to the database
>> 
>> The dump command now takes the write lock when running. This
>> diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
>> index d58dc8b..05b5969 100644
>> --- a/completion/notmuch-completion.bash
>> +++ b/completion/notmuch-completion.bash
>> @@ -61,7 +61,7 @@ _notmuch_search_terms()
>> sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
>> ;;
>> *)
>> -local search_terms="from: to: subject: attachment: tag: id: 
thread: folder: path: date:"
>> +local search_terms="from: to: subject: attachment: 
contenttype: tag: id: thread: folder: path: date:"
>> compopt -o nospace
>> COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
>> ;;
>> diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
>> index 1acdaa0..d126ce6 100644
>> --- a/doc/man7/notmuch-search-terms.rst
>> +++ b/doc/man7/notmuch-search-terms.rst
>> @@ -40,6 +40,8 @@ indicate user-supplied values):
>> 
>> -  attachment:
>> 
>> +-  contenttype:
>> +
>> -  tag: (or is:)
>> 
>> -  id:
>> @@ -66,6 +68,10 @@ by including quotation marks around the phrase, 
immediately following
>> The **attachment:** prefix can be used to search for specific filenames
>> (or extensions) of attachments to email messages.
>> 
>> +The **contenttype:** prefix can be used to search for specific
>> +content-types of attachments to email messages (as specified by the
>> +sender).
>> +
>> For **tag:** and **is:** valid tag values include **inbox** and
>> **unread** by default for new messages added by **notmuch new** as well
>> as any other tag values added manually with **notmuch tag**.
>> diff --git a/emacs/notmuch.el b/emacs/notmuch.el
>> index 218486a..702700c 100644
&g

[PATCH v2 0/5] Index the content-type of MIME parts

[Todd]

This set of patches allows for queries for like:

Find calendar invites
- mimetype:text/calendar or mimetype:application/ics

Find any image attachments
- mimetype:image

Find all patches
- mimetype:text/x-patch

It is a rework of a previous set of patches that now adds a new
database feature, only available in newly created databases, which
indicates that all of the mime parts are indexed.  Indexing of new
messages on an existing database should work fine as well, but the
feature won't be enabled via upgrade.

Code reviews welcome, I'm not familiar with this code base.

Todd (5):
  Add a failing unit test for indexed mime types
  Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature
  Add indexing for the mimetype term
  Update completions for Emacs and bash
  Update documentation

 NEWS   | 16 
 completion/notmuch-completion.bash |  2 +-
 doc/man7/notmuch-search-terms.rst  |  6 ++
 emacs/notmuch.el   |  2 +-
 lib/database-private.h | 15 ---
 lib/database.cc| 11 +--
 lib/index.cc   |  6 ++
 test/T190-multipart.sh | 32 
 8 files changed, 83 insertions(+), 7 deletions(-)

--
1.9.1

[PATCH v2 1/5] Add a failing unit test for indexed mime types

[Todd]

---
 test/T190-multipart.sh | 32 
 1 file changed, 32 insertions(+)

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..26f7c7a 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: todd at example.com
+Subject: odd content types
+Date: Mon, 12 Jan 2014 18:12:32 +
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 
(i486-pc-linux-gnu)
+Message-ID: 
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: application/unique_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==
+Content-Type: text/some_other_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+cat content_types >> ${MAIL_DIR}/odd_content_type
 notmuch new > /dev/null

 test_begin_subtest "--format=text --part=0, full message"
@@ -727,4 +751,12 @@ test_begin_subtest "html parts included"
 notmuch show --format=json --include-html id:htmlmessage > OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"

+test_begin_subtest "indexes mime-type"
+output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+
 test_done
-- 
1.9.1

[PATCH v2 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

[Todd]

---
 lib/database-private.h | 15 ---
 lib/database.cc| 10 --
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 15e03cc..6d6fa2c 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -92,6 +92,14 @@ enum _notmuch_features {
  *
  * Introduced: version 3. */
 NOTMUCH_FEATURE_GHOSTS = 1 << 4,
+
+
+/* If set, then the database was created after the introduction of
+ * indexed mime types. If unset, then the database may contain a
+ * mixture of messages with indexed and non-indexed mime types.
+ *
+ * Introduced: version 3. */
+NOTMUCH_FEATURE_INDEXED_MIMETYPES = 1 << 5,
 };

 /* In C++, a named enum is its own type, so define bitwise operators
@@ -161,9 +169,10 @@ struct _notmuch_database {

 /* Current database features.  If any of these are missing from a
  * database, request an upgrade.
- * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES is not included because
- * upgrade doesn't currently introduce the feature (though brand new
- * databases will have it). */
+ * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES and
+ * NOTMUCH_FEATURE_INDEXED_MIMETYPES are not included because upgrade
+ * doesn't currently introduce the features (though brand new databases
+ * will have it). */
 #define NOTMUCH_FEATURES_CURRENT \
 (NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_DIRECTORY_DOCS | \
  NOTMUCH_FEATURE_BOOL_FOLDER | NOTMUCH_FEATURE_GHOSTS)
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..2de60f8 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -304,6 +304,11 @@ static const struct {
   "exact folder:/path: search", "rw" },
 { NOTMUCH_FEATURE_GHOSTS,
   "mail documents for missing messages", "w"},
+/* Knowledge of the index mime-types are not required for reading
+ * a database because a reader will just be unable to query
+ * them. */
+{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
+  "mime-types in database", "w"},
 };

 const char *
@@ -646,9 +651,10 @@ notmuch_database_create (const char *path, 
notmuch_database_t **database)
 if (status)
goto DONE;

-/* Upgrade doesn't add this feature to existing databases, but new
- * databases have it. */
+/* Upgrade doesn't add these feature to existing databases, but
+ * new databases have them. */
 notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
+notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;

 status = notmuch_database_upgrade (notmuch, NULL, NULL);
 if (status) {
-- 
1.9.1

[PATCH v2 4/5] Update completions for Emacs and bash

[Todd]

This adds completions for Emacs and bash, ZSH does not appear to have
completions for search terms.
---
 completion/notmuch-completion.bash | 2 +-
 emacs/notmuch.el   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..05b5969 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: contenttype: 
tag: id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..ab00454 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "mimetype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
-- 
1.9.1

[PATCH v2 3/5] Add indexing for the mimetype term

[Todd]

---
 lib/database.cc | 1 +
 lib/index.cc| 6 ++
 2 files changed, 7 insertions(+)

diff --git a/lib/database.cc b/lib/database.cc
index 2de60f8..7c7a267 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "mimetype",  "XMIMETYPE"},
 { "subject",   "XSUBJECT"},
 };

diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..2e27558 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -318,6 +318,12 @@ _index_mime_part (notmuch_message_t *message,
return;
 }

+
+GMimeContentType*  content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   _notmuch_message_gen_terms (message, "mimetype", 
g_mime_content_type_to_string(content_type));
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
-- 
1.9.1

[PATCH v2 5/5] Update documentation

[Todd]

---
 NEWS  | 16 
 doc/man7/notmuch-search-terms.rst |  6 ++
 2 files changed, 22 insertions(+)

diff --git a/NEWS b/NEWS
index 44e8d05..0df7977 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,19 @@
+Notmuch 0.20 (TBD)
+=
+
+Overview
+
+
+Command-Line Interface
+--
+
+There is a new `mimetype:` search prefix
+
+  The new `mimetype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by `notmuch
+  insert`. See the `notmuch-search-terms` manual page for details.
+
+
 Notmuch 0.19 (2014-11-14)
 =

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..8d1acfc 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):

 -  attachment:

+-  mimetype:
+
 -  tag: (or is:)

 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.

+The **mimetype:** prefix can be used to search for the specific
+content-types of mime parts within email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
-- 
1.9.1

[PATCH] emacs: make citation function customizable.

[Todd]

Looks good to me. I applied the patch and it works fine as well.

- Todd

>>>>> "DB" == David Bremner  writes:

DB> Make a new customizable variable instead of relying on
DB> message-cite-function because the default for the latter changed
DB> between emacs releases.

DB> The defcustom is borrowed from the message.el source, with minor
DB> modifications.
DB> ---

DB> Thanks to Todd for tracking down the wibbly wobbly timey wimey 
behaviour of this variable. I propose to add our own variable to avoid 
surprising notmuch-emacs users.

DB>  emacs/notmuch-mua.el | 23 ---
DB>  1 file changed, 20 insertions(+), 3 deletions(-)

DB> diff --git a/emacs/notmuch-mua.el b/emacs/notmuch-mua.el
DB> index 2c58886..33f1399 100644
DB> --- a/emacs/notmuch-mua.el
DB> +++ b/emacs/notmuch-mua.el
DB> @@ -75,6 +75,22 @@ list."
DB>:type '(repeat string)
DB>:group 'notmuch-send)

DB> +(defgroup notmuch-reply nil
DB> +  "Replying to messages in notmuch"
DB> +  :group 'notmuch)
DB> +
DB> +(defcustom notmuch-mua-cite-function 'message-cite-original
DB> +  "*Function for citing an original message.
DB> +Predefined functions include `message-cite-original' and
DB> +`message-cite-original-without-signature'.
DB> +Note that these functions use `mail-citation-hook' if that is non-nil."
DB> +  :type '(radio (function-item message-cite-original)
DB> +   (function-item message-cite-original-without-signature)
DB> +   (function-item sc-cite-original)
DB> +   (function :tag "Other"))
DB> +  :link '(custom-manual "(message)Insertion Variables")
DB> +  :group 'notmuch-reply)
DB> +
DB>  ;;

DB>  (defun notmuch-mua-get-switch-function ()
DB> @@ -220,8 +236,9 @@ list."
DB> (date (plist-get original-headers :Date))
DB> (start (point)))

DB> -   ;; message-cite-original constructs a citation line based on 
the >From and Date
DB> -   ;; headers of the original message, which are assumed to be in 
the buffer.
DB> +   ;; notmuch-mua-cite-function constructs a citation line based
DB> +   ;; on the From and Date headers of the original message, which
DB> +   ;; are assumed to be in the buffer.
DB> (insert "From: " from "\n")
DB> (insert "Date: " date "\n\n")

DB> @@ -233,7 +250,7 @@ list."
DB> (set-mark (point))
DB> (goto-char start)
DB> ;; Quote the original message according to the user's 
configured style.
DB> -   (message-cite-original)))
DB> +   (funcall notmuch-mua-cite-function)))

DB>  ;; Crypto processing based crypto content of the original message
DB>  (when process-crypto
DB> -- 
DB> 2.1.4

DB> ___
DB> notmuch mailing list
DB> notmuch at notmuchmail.org
DB> http://notmuchmail.org/mailman/listinfo/notmuch
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20150114/4587ed23/attachment.pgp>

[PATCH v3 1/5] Add failing unit tests for indexed mime types

[Todd]

---
 test/T190-multipart.sh | 44 
 1 file changed, 44 insertions(+)

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..57f1b61 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: todd at example.com
+Subject: odd content types
+Date: Mon, 12 Jan 2014 18:12:32 +
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 
(i486-pc-linux-gnu)
+Message-ID: 
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: application/unique_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==
+Content-Type: text/some_other_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+cat content_types >> ${MAIL_DIR}/odd_content_type
 notmuch new > /dev/null

 test_begin_subtest "--format=text --part=0, full message"
@@ -727,4 +751,24 @@ test_begin_subtest "html parts included"
 notmuch show --format=json --include-html id:htmlmessage > OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"

+test_begin_subtest "indexes mime-type #1"
+test_subtest_known_broken
+output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #2"
+test_subtest_known_broken
+output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #3"
+test_subtest_known_broken
+output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "search for non-existent mime-type"
+test_subtest_known_broken
+output=$(notmuch search mimetype:non-existent/mime-type | 
notmuch_search_sanitize)
+test_expect_equal "$output" ""
+
 test_done
-- 
1.9.1

[PATCH v3 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

[Todd]

---
 lib/database-private.h | 15 ---
 lib/database.cc| 10 --
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 15e03cc..6d6fa2c 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -92,6 +92,14 @@ enum _notmuch_features {
  *
  * Introduced: version 3. */
 NOTMUCH_FEATURE_GHOSTS = 1 << 4,
+
+
+/* If set, then the database was created after the introduction of
+ * indexed mime types. If unset, then the database may contain a
+ * mixture of messages with indexed and non-indexed mime types.
+ *
+ * Introduced: version 3. */
+NOTMUCH_FEATURE_INDEXED_MIMETYPES = 1 << 5,
 };

 /* In C++, a named enum is its own type, so define bitwise operators
@@ -161,9 +169,10 @@ struct _notmuch_database {

 /* Current database features.  If any of these are missing from a
  * database, request an upgrade.
- * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES is not included because
- * upgrade doesn't currently introduce the feature (though brand new
- * databases will have it). */
+ * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES and
+ * NOTMUCH_FEATURE_INDEXED_MIMETYPES are not included because upgrade
+ * doesn't currently introduce the features (though brand new databases
+ * will have it). */
 #define NOTMUCH_FEATURES_CURRENT \
 (NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_DIRECTORY_DOCS | \
  NOTMUCH_FEATURE_BOOL_FOLDER | NOTMUCH_FEATURE_GHOSTS)
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..0d2c417 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -304,6 +304,11 @@ static const struct {
   "exact folder:/path: search", "rw" },
 { NOTMUCH_FEATURE_GHOSTS,
   "mail documents for missing messages", "w"},
+/* Knowledge of the index mime-types are not required for reading
+ * a database because a reader will just be unable to query
+ * them. */
+{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
+  "indexed MIME types", "w"},
 };

 const char *
@@ -646,9 +651,10 @@ notmuch_database_create (const char *path, 
notmuch_database_t **database)
 if (status)
goto DONE;

-/* Upgrade doesn't add this feature to existing databases, but new
- * databases have it. */
+/* Upgrade doesn't add these feature to existing databases, but
+ * new databases have them. */
 notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
+notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;

 status = notmuch_database_upgrade (notmuch, NULL, NULL);
 if (status) {
-- 
1.9.1

[PATCH v3 4/5] Update completions for Emacs and bash

[Todd]

This adds completions for Emacs and bash, ZSH does not appear to have
completions for search terms.
---
 completion/notmuch-completion.bash | 2 +-
 emacs/notmuch.el   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..39320f8 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: mimetype: tag: 
id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..ab00454 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "mimetype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
-- 
1.9.1

[PATCH v3 3/5] Add indexing for the mimetype term

[Todd]

Adds the indexing and removes the broken test flag
---
 lib/database.cc|  1 +
 lib/index.cc   | 10 ++
 test/T190-multipart.sh |  4 
 3 files changed, 11 insertions(+), 4 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 0d2c417..3974e2e 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "mimetype",  "XMIMETYPE"},
 { "subject",   "XSUBJECT"},
 };

diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..c88ed8d 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -318,6 +318,16 @@ _index_mime_part (notmuch_message_t *message,
return;
 }

+GMimeContentType *content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   char *mime_string = g_mime_content_type_to_string(content_type);
+   if (mime_string)
+   {
+   _notmuch_message_gen_terms (message, "mimetype", mime_string);
+   g_free(mime_string);
+   }
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 57f1b61..de7d361 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -752,22 +752,18 @@ notmuch show --format=json --include-html id:htmlmessage 
> OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"

 test_begin_subtest "indexes mime-type #1"
-test_subtest_known_broken
 output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"

 test_begin_subtest "indexes mime-type #2"
-test_subtest_known_broken
 output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"

 test_begin_subtest "indexes mime-type #3"
-test_subtest_known_broken
 output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"

 test_begin_subtest "search for non-existent mime-type"
-test_subtest_known_broken
 output=$(notmuch search mimetype:non-existent/mime-type | 
notmuch_search_sanitize)
 test_expect_equal "$output" ""

-- 
1.9.1

[PATCH v3 5/5] Update documentation

[Todd]

---
 NEWS  | 11 +++
 doc/man7/notmuch-search-terms.rst |  6 ++
 2 files changed, 17 insertions(+)

diff --git a/NEWS b/NEWS
index abb5ad7..baa4519 100644
--- a/NEWS
+++ b/NEWS
@@ -1,4 +1,5 @@
 Notmuch 0.20 (UNRELEASED)
+=

 Contrib
 ---
@@ -7,6 +8,16 @@ Contrib
 previously provided by `notmuch-deliver` should now be provided by
 `notmuch insert`, provided by the main notmuch binary.

+Command-Line Interface
+--
+
+There is a new `mimetype:` search prefix
+
+  The new `mimetype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by `notmuch
+  insert`. See the `notmuch-search-terms` manual page for details.
+
+
 Notmuch 0.19 (2014-11-14)
 =

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..8d1acfc 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):

 -  attachment:

+-  mimetype:
+
 -  tag: (or is:)

 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.

+The **mimetype:** prefix can be used to search for the specific
+content-types of mime parts within email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
-- 
1.9.1

[PATCH v1 0/2] Fixes for issues discovered with -fsanitize=address

[Todd]

I built a copy of notmuch with gcc 4.8's new -fsanitize=address
feature and ran the test bench. It pointed out these two issues.


Todd (2):
  lib: Fix potential invalid read past an empty string
  lib: Fix use after free

 lib/thread.cc | 7 +++
 1 file changed, 3 insertions(+), 4 deletions(-)

--
1.9.1

[PATCH v1 1/2] lib: Fix potential invalid read past an empty string

[Todd]

==22884== ERROR: AddressSanitizer: heap-buffer-overflow on address 
0x60168291 at pc 0x7ff6295680e5 bp 0x7fff4ab9aa40 sp 0x7fff4ab9aa08
READ of size 1 at 0x60168291 thread T0
#0 0x7ff6295680e4 in __interceptor_strcmp ??:?
#1 0x44763b in _thread_add_message 
/home/todd/.apps/notmuch/lib/thread.cc:255
#2 0x4459e8 in notmuch_threads_get /home/todd/.apps/notmuch/lib/query.cc:496
#3 0x41e2a7 in do_search_threads 
/home/todd/.apps/notmuch/notmuch-search.c:131
#4 0x40a408 in main /home/todd/.apps/notmuch/notmuch.c:345
#5 0x7ff627cb9ec4 in __libc_start_main 
/build/buildd/eglibc-2.19/csu/libc-start.c:287
#6 0x40abf3 in _start ??:?
0x60168291 is located 0 bytes to the right of 97-byte region 
[0x60168230,0x60168291)
allocated by thread T0 here:
#0 0x7ff62956e41a in malloc ??:?
#1 0x7ff628b8ab5d in talloc_strdup ??:?
---
 lib/thread.cc | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index 4542505..4c49f98 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -251,8 +251,8 @@ _thread_add_message (notmuch_thread_t *thread,
 term != NULL;
 term = term->next)
{
-   /* We ignore initial 'K'. */
-   if (strcmp(tag, (term->string + 1)) == 0) {
+   /* Check for an empty string, and then ignore initial 'K'. */
+   if (*(term->string) && strcmp(tag, (term->string + 1)) == 0) {
message_excluded = TRUE;
break;
}
-- 
1.9.1

[PATCH v1 2/2] lib: Fix use after free

[Todd]

_thread_set_subject_from_message sometimes replaces the subject, making the
cur_subject point to free'd memory

==6550== ERROR: AddressSanitizer: heap-use-after-free on address 0x601abec0 
at pc 0x4464a4 bp 0x7fffa40be910 sp 0x7fffa40be908
READ of size 1 at 0x601abec0 thread T0
#0 0x4464a3 in _thread_add_matched_message 
/home/todd/.apps/notmuch/lib/thread.cc:369
#1 0x443c2c in notmuch_threads_get /home/todd/.apps/notmuch/lib/query.cc:496
#2 0x41d947 in do_search_threads 
/home/todd/.apps/notmuch/notmuch-search.c:131
#3 0x40a3fe in main /home/todd/.apps/notmuch/notmuch.c:345
#4 0x7f4e535b4ec4 in __libc_start_main 
/build/buildd/eglibc-2.19/csu/libc-start.c:287
#5 0x40abe6 in _start ??:?
0x601abec0 is located 96 bytes inside of 134-byte region 
[0x601abe60,0x601abee6)
freed by thread T0 here:
#0 0x7f4e54e6933a in __interceptor_free ??:?
#1 0x7f4e54482fab in _talloc_free ??:?
previously allocated by thread T0 here:
#0 0x7f4e54e6941a in malloc ??:?
#1 0x7f4e54485b5d in talloc_strdup ??:?
---
 lib/thread.cc | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/lib/thread.cc b/lib/thread.cc
index 4c49f98..9847cf8 100644
--- a/lib/thread.cc
+++ b/lib/thread.cc
@@ -353,10 +353,8 @@ _thread_add_matched_message (notmuch_thread_t *thread,
 {
 time_t date;
 notmuch_message_t *hashed_message;
-const char *cur_subject;

 date = notmuch_message_get_date (message);
-cur_subject = notmuch_thread_get_subject(thread);

 if (date < thread->oldest || ! thread->matched_messages) {
thread->oldest = date;
@@ -366,6 +364,7 @@ _thread_add_matched_message (notmuch_thread_t *thread,

 if (date > thread->newest || ! thread->matched_messages) {
thread->newest = date;
+   const char *cur_subject = notmuch_thread_get_subject(thread);
if (sort != NOTMUCH_SORT_OLDEST_FIRST || EMPTY_STRING(cur_subject))
_thread_set_subject_from_message (thread, message);
 }
-- 
1.9.1

[PATCH v3 3/5] Add indexing for the mimetype term

[Todd]

>>>>> "DB" == David Bremner  writes:

DB> Todd  writes:
>> Adds the indexing and removes the broken test flag
>> ---
>> lib/database.cc|  1 +
>> lib/index.cc   | 10 ++
>> test/T190-multipart.sh |  4 
>> 3 files changed, 11 insertions(+), 4 deletions(-)
>>
>> diff --git a/lib/database.cc b/lib/database.cc
>> index 0d2c417..3974e2e 100644
>> --- a/lib/database.cc
>> +++ b/lib/database.cc
>> @@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
>> { "from","XFROM" },
>> { "to",  "XTO" },
>> { "attachment",  "XATTACHMENT" },
>> +{ "mimetype",   "XMIMETYPE"},
>> { "subject", "XSUBJECT"},
>> };

DB> I think the commit message should articulate why we are indexing this as
DB> a probabilistic prefix, rather than as a boolean prefix. In particular,
DB> this gives people a last chance to complain.

DB> The reference I know is http://xapian.org/docs/queryparser.html

DB> If I understand correctly (it would be great if you could test this
DB> Todd) , with a probabilistic prefix,

DB>mimetime:pdf

DB> will match

DB> application/pdf
DB> image/pdf
DB> application/x-pdf
DB> application/x-ext-pdf

DB> but not

DB> application/x-bzpdf
DB> application/x-gzpdf
DB> application/x-xzpdf

I just tested, and it does work this way with your examples.  I
*believe* from reading the docs, that xapian is treating the full
MIME-type queries as phrase searches anyway due to the embedded
slashes.

From http://xapian.org/docs/queryparser.html:

 A phrase surrounded with double quotes ("") matches documents
 containing that exact phrase. Hyphenated words are also treated
 as phrases, as are cases such as filenames and email addresses
 (e.g. /etc/passwd or president at whitehouse.gov).

I think that we'll get good behavior from the types of queries that
will typically be performed due to this automatic phrasing.



DB> On the whole, this is probably more beneficial than bad.  The downside
DB> of probabilistic prefixes/fields is that they are not "anchored", so
DB> there is no easy way to distinguish

DB>   application/pdf

DB> from

DB>   pdf
DB>   application/x-pdf

DB> I guess in a perfect world this would also be explained in
DB> notmuch-search-terms(7), but that's pretty much orthogonal to this
DB> series.

If separate messages with application/pdf and application/x-pdf are
indexed, then:

mimetype:application/x-pdf finds only the application/x-pdf
mimetype:application/pdf finds only the application/pdf
mimetype:pdf finds both of the messages

I am fairly sure that this behaviour is a result of the automatic
phrasing mentioned above.

- Todd

DB> d
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20150117/d46cb7a9/attachment.pgp>

[PATCH v4 0/5] Index the content-type of MIME parts

[Todd]

I think I've finished incorporating the feedback.  The final
notmuch-search-terms.rst could use more details, but it should
probably occur after the recent patch that was posted documenting the
probablistic indexing/searching has been committed.

Todd (5):
  test: Add failing unit tests for indexed mime types
  Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature
  Add indexing for the mimetype term
  Update completions for Emacs and bash
  Update documentation

 NEWS   | 11 +++
 completion/notmuch-completion.bash |  2 +-
 doc/man7/notmuch-search-terms.rst  |  6 ++
 emacs/notmuch.el   |  2 +-
 lib/database-private.h | 15 ---
 lib/database.cc| 11 +--
 lib/index.cc   | 10 ++
 test/T190-multipart.sh | 36 
 8 files changed, 86 insertions(+), 7 deletions(-)

--
1.9.1

[PATCH v4 3/5] Add indexing for the mimetype term

[Todd]

This adds the indexing support for the "mimetype:" term and removes
the broken test flag.  The indexing is probablistic in Xapian terms,
which gives a better experience to end users.  Standard content-types
of the form "foo/bar" are automatically interpreted as phrases in
Xapian due to the embedded slash.

Assume, separate messages with application/pdf and application/x-pdf
are indexed, then:

- mimetype:application/x-pdf will find only the application/x-pdf
- mimetype:application/pdf will find only the application/pdf
- mimetype:pdf will find both of the messages
---
 lib/database.cc|  1 +
 lib/index.cc   | 10 ++
 test/T190-multipart.sh |  3 ---
 3 files changed, 11 insertions(+), 3 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 0d2c417..3974e2e 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -254,6 +254,7 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { "from",  "XFROM" },
 { "to","XTO" },
 { "attachment","XATTACHMENT" },
+{ "mimetype",  "XMIMETYPE"},
 { "subject",   "XSUBJECT"},
 };

diff --git a/lib/index.cc b/lib/index.cc
index 1a2e63d..c88ed8d 100644
--- a/lib/index.cc
+++ b/lib/index.cc
@@ -318,6 +318,16 @@ _index_mime_part (notmuch_message_t *message,
return;
 }

+GMimeContentType *content_type = g_mime_object_get_content_type(part);
+if (content_type) {
+   char *mime_string = g_mime_content_type_to_string(content_type);
+   if (mime_string)
+   {
+   _notmuch_message_gen_terms (message, "mimetype", mime_string);
+   g_free(mime_string);
+   }
+}
+
 if (GMIME_IS_MULTIPART (part)) {
GMimeMultipart *multipart = GMIME_MULTIPART (part);
int i;
diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index a97dc28..ad8d29e 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -752,17 +752,14 @@ notmuch show --format=json --include-html id:htmlmessage 
> OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"

 test_begin_subtest "indexes mime-type #1"
-test_subtest_known_broken
 output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"

 test_begin_subtest "indexes mime-type #2"
-test_subtest_known_broken
 output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"

 test_begin_subtest "indexes mime-type #3"
-test_subtest_known_broken
 output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"

-- 
1.9.1

[PATCH v4 1/5] test: Add failing unit tests for indexed mime types

[Todd]

Adds three failing unit tests for searching of mime-types.

An attempt was made at adding a negative test (i.e. searching for a
non-existent mime-type and ensuring it didn't return a message), but
that test would always pass making it pointless.
---
 test/T190-multipart.sh | 39 +++
 1 file changed, 39 insertions(+)

diff --git a/test/T190-multipart.sh b/test/T190-multipart.sh
index 85cbf67..a97dc28 100755
--- a/test/T190-multipart.sh
+++ b/test/T190-multipart.sh
@@ -104,6 +104,30 @@ Content-Transfer-Encoding: base64
 7w0K
 --==-=-=--
 EOF
+
+cat < content_types
+From: Todd 
+To: todd at example.com
+Subject: odd content types
+Date: Mon, 12 Jan 2014 18:12:32 +
+User-Agent: Notmuch/0.5 (http://notmuchmail.org) Emacs/23.3.1 
(i486-pc-linux-gnu)
+Message-ID: 
+MIME-Version: 1.0
+Content-Type: multipart/alternative; boundary="==-=-=="
+
+--==-=-==
+Content-Type: application/unique_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==
+Content-Type: text/some_other_identifier
+
+This is an embedded message, with a multipart/alternative part.
+
+--==-=-==--
+EOF
+cat content_types >> ${MAIL_DIR}/odd_content_type
 notmuch new > /dev/null

 test_begin_subtest "--format=text --part=0, full message"
@@ -727,4 +751,19 @@ test_begin_subtest "html parts included"
 notmuch show --format=json --include-html id:htmlmessage > OUTPUT
 test_expect_equal_json "$(cat OUTPUT)" "$(cat EXPECTED.withhtml)"

+test_begin_subtest "indexes mime-type #1"
+test_subtest_known_broken
+output=$(notmuch search mimetype:application/unique_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #2"
+test_subtest_known_broken
+output=$(notmuch search mimetype:text/some_other_identifier | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
+test_begin_subtest "indexes mime-type #3"
+test_subtest_known_broken
+output=$(notmuch search from:todd and mimetype:multipart/alternative | 
notmuch_search_sanitize)
+test_expect_equal "$output" "thread:XXX   2014-01-12 [1/1] Todd; odd content 
types (inbox unread)"
+
 test_done
-- 
1.9.1

[PATCH v4 4/5] Update completions for Emacs and bash

[Todd]

This adds completions for both Emacs and bash. ZSH does not appear to
have completions for search terms.
---
 completion/notmuch-completion.bash | 2 +-
 emacs/notmuch.el   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/completion/notmuch-completion.bash 
b/completion/notmuch-completion.bash
index d58dc8b..39320f8 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -61,7 +61,7 @@ _notmuch_search_terms()
sed "s|^$path/||" | grep -v "\(^\|/\)\(cur\|new\|tmp\)$" ) )
;;
*)
-   local search_terms="from: to: subject: attachment: tag: id: thread: 
folder: path: date:"
+   local search_terms="from: to: subject: attachment: mimetype: tag: 
id: thread: folder: path: date:"
compopt -o nospace
COMPREPLY=( $(compgen -W "${search_terms}" -- ${cur}) )
;;
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index 218486a..ab00454 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -858,7 +858,7 @@ PROMPT is the string to prompt with."
   (lexical-let
   ((completions
(append (list "folder:" "path:" "thread:" "id:" "date:" "from:" "to:"
- "subject:" "attachment:")
+ "subject:" "attachment:" "mimetype:")
(mapcar (lambda (tag)
  (concat "tag:" (notmuch-escape-boolean-term tag)))
(process-lines notmuch-command "search" "--output=tags" 
"*")
-- 
1.9.1

[PATCH v4 5/5] Update documentation

[Todd]

Adds new entry to the NEWS file, and updates the search terms section
of the man page.  The search terms section needs to be updated again
once the new section in the documentation covering probablistic terms
has been committed.
---
 NEWS  | 11 +++
 doc/man7/notmuch-search-terms.rst |  6 ++
 2 files changed, 17 insertions(+)

diff --git a/NEWS b/NEWS
index 53e06aa..a4f2a3f 100644
--- a/NEWS
+++ b/NEWS
@@ -1,4 +1,5 @@
 Notmuch 0.20 (UNRELEASED)
+=

 Documentation
 -
@@ -13,6 +14,16 @@ Contrib
 previously provided by `notmuch-deliver` should now be provided by
 `notmuch insert`, provided by the main notmuch binary.

+Command-Line Interface
+--
+
+There is a new `mimetype:` search prefix
+
+  The new `mimetype:` search prefix allows searching for the
+  content-type of attachments, which is now indexed by. See the
+  `notmuch-search-terms` manual page for details.
+
+
 Notmuch 0.19 (2014-11-14)
 =

diff --git a/doc/man7/notmuch-search-terms.rst 
b/doc/man7/notmuch-search-terms.rst
index 1acdaa0..315e849 100644
--- a/doc/man7/notmuch-search-terms.rst
+++ b/doc/man7/notmuch-search-terms.rst
@@ -40,6 +40,8 @@ indicate user-supplied values):

 -  attachment:

+-  mimetype:
+
 -  tag: (or is:)

 -  id:
@@ -66,6 +68,10 @@ by including quotation marks around the phrase, immediately 
following
 The **attachment:** prefix can be used to search for specific filenames
 (or extensions) of attachments to email messages.

+The **mimetype:** prefix will be used to match text from the
+content-types of MIME parts within email messages (as specified by the
+sender).
+
 For **tag:** and **is:** valid tag values include **inbox** and
 **unread** by default for new messages added by **notmuch new** as well
 as any other tag values added manually with **notmuch tag**.
-- 
1.9.1

[PATCH v4 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

[Todd]

This feature will exist in all newly created databases, but there is
no upgrade provided for it.  If this flag exists, it indicates that
the database was created after the indexed MIME-types feature was
added.
---
 lib/database-private.h | 15 ---
 lib/database.cc| 10 --
 2 files changed, 20 insertions(+), 5 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 15e03cc..6d6fa2c 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -92,6 +92,14 @@ enum _notmuch_features {
  *
  * Introduced: version 3. */
 NOTMUCH_FEATURE_GHOSTS = 1 << 4,
+
+
+/* If set, then the database was created after the introduction of
+ * indexed mime types. If unset, then the database may contain a
+ * mixture of messages with indexed and non-indexed mime types.
+ *
+ * Introduced: version 3. */
+NOTMUCH_FEATURE_INDEXED_MIMETYPES = 1 << 5,
 };

 /* In C++, a named enum is its own type, so define bitwise operators
@@ -161,9 +169,10 @@ struct _notmuch_database {

 /* Current database features.  If any of these are missing from a
  * database, request an upgrade.
- * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES is not included because
- * upgrade doesn't currently introduce the feature (though brand new
- * databases will have it). */
+ * NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES and
+ * NOTMUCH_FEATURE_INDEXED_MIMETYPES are not included because upgrade
+ * doesn't currently introduce the features (though brand new databases
+ * will have it). */
 #define NOTMUCH_FEATURES_CURRENT \
 (NOTMUCH_FEATURE_FILE_TERMS | NOTMUCH_FEATURE_DIRECTORY_DOCS | \
  NOTMUCH_FEATURE_BOOL_FOLDER | NOTMUCH_FEATURE_GHOSTS)
diff --git a/lib/database.cc b/lib/database.cc
index 3601f9d..0d2c417 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -304,6 +304,11 @@ static const struct {
   "exact folder:/path: search", "rw" },
 { NOTMUCH_FEATURE_GHOSTS,
   "mail documents for missing messages", "w"},
+/* Knowledge of the index mime-types are not required for reading
+ * a database because a reader will just be unable to query
+ * them. */
+{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
+  "indexed MIME types", "w"},
 };

 const char *
@@ -646,9 +651,10 @@ notmuch_database_create (const char *path, 
notmuch_database_t **database)
 if (status)
goto DONE;

-/* Upgrade doesn't add this feature to existing databases, but new
- * databases have it. */
+/* Upgrade doesn't add these feature to existing databases, but
+ * new databases have them. */
 notmuch->features |= NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES;
+notmuch->features |= NOTMUCH_FEATURE_INDEXED_MIMETYPES;

 status = notmuch_database_upgrade (notmuch, NULL, NULL);
 if (status) {
-- 
1.9.1

Unable to search for emails with content with a given MIME type

[Todd]

Petter,

The feature you wanted was committed a few days ago (searching with a
"mimetype:" prefix).  Searching for "mimetype:calendar" would get any
"text/calendar" or "ics/calendar" attachments.

(Replying to the list so it will appear in the list archive as well.)

Thanks,
Todd

-- next part --
Petter Reinholdtsen  writes:

> Not quite sure where to send a feature request, so I try to follow the
> instructions for bug reporst.
>
> I would like to be able to search for emails with content with a given
> MIME type.  The latest use case I ran into was my need to locate Notes
> and Exchange emails with text/calendar attachments, but I've also needed
> to find application/pdf and application/vnd.oasis.opendocument.text
> emails.
>
> Please consider extending notmuch to be able to search for specific mime
> types, for example using attachment:text/calendar or mime:text/calendar
> or something like that.  Or perhaps just index up the mime content type
> value, to allow me to search for "tag:attachment and text/calendar".
>
> PS: Please CC me on replies, as I am not subscribed to the list.
> -- 
> Happy hacking
> Petter Reinholdtsen
-- next part --
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20150127/d9a572d8/attachment.pgp>

Difficulty understanding maildir.synchronize_flags behavior

[Andrew Todd]

Hello,

I'm trying to set up notmuch, and I think that the behavior I want 
should be possible (and is desirable), but I can't seem to make it work. 
I also can't find any reference online suggesting that it *shouldn't* 
work, and I would have thought that it's a very common use case for 
people using multiple mail clients with one IMAP server.


Running Arch Linux, so on latest (currently 0.38.1). Working with the 
CLI and Emacs frontend.


I'm synchronizing mail from a remote IMAP server to local maildir using 
isync/mbsync.


What I want is for messages that do not have maildir files ending in 
':2,S' to be tagged as 'unread' in notmuch.


In other words, I want notmuch to respect the read/unread state of 
messages from the IMAP server and not make any other changes.


My understanding is that this is the purpose of the 
maildir.synchronize_flags option. I have explicitly set this to true in 
my config:


[maildir]
synchronize_flags=true

Moreover, I have also turned off all default tags when running notmuch new:

[new]
tags=

What I see is:

1) When I pull in messages with `mbsync`, they end up in the `new/` 
directory as expected.
2) When I run `notmuch new`, the messages remain in the `new/` 
directory. I guess this is correct. I don't really mind where they are.
3) When I open the Emacs interface, I see that the message is not tagged 
as `unread`. Moreover, when I read the message, it is not moved from 
`new/` to `cur/`, and the filename does not change from ':2,' to ':2,S'.


This is the first part of the problem.

The second part is that when I try to implement tagging in a post-new 
hook, all messages are immediately marked as read, e.g. they are moved 
from `new/` to `cur/` and their filename ending changes from ':2,' to 
':2,S'.


This is the only tag operation I'm performing:

+inbox -- folder:current/Inbox

My understanding of what the behavior should be is that:

1) `notmuch new` should automatically tag messages in `new/` and `cur/` 
that do not have 'S' in their filename as 'unread'.
2) `notmuch tag` should not mark an unread message as read; maybe this 
is happening because `notmuch new` didn't add the `unread` tag beforehand?


On the other hand, I have confirmed that if I go into the emacs 
interface and add the 'unread' tag to messages, their maildir files do 
get the 'S' removed from them. So the tag synchronization is working in 
the opposite direction.


I feel like I must be missing something fundamental in my understanding, 
or there is a fundamental bug in maildir.synchronize_flags. I would 
appreciate any advice.


Thank you,
Andrew Todd
a...@auspicacious.org
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org

Re: Difficulty understanding maildir.synchronize_flags behavior

[Andrew Todd]

On 12/12/23 20:25, Michael J Gruber wrote:
The manual is not overly clear about the interaction between this config 
and flag sync, but AFAIU, `notmuch new` sets `unread` and then possibly 
unsets it during the flag sync (maybe in one atomic db write, i.e. you 
don't see this). By removing `unread` from `new.tags` you keep `notmuch 
new` from doing that first step.


Thanks for your help; it pushed me to explore a bit more. I have 
realized that something much more subtle is going on. In fact, you do 
not have to set `tags=unread` in the configuration for this to work.


First, I noticed that other maildir flags, such as D/draft and 
R/replied, were synchronizing correctly.


Then, I realized that I was doing my testing by sending emails to 
myself. This created duplicate copies of the same message, with the same 
message ID, in both `Inbox` and `Sent`.


However, the copy of the message in `Sent` was, of course, already 
marked as read. notmuch's synchronization was working as intended, but 
since notmuch considers both copies to be the same message, it 
presumably synchronized the copy in `Sent` after the copy in `Inbox` and 
therefore marked both copies as already read.


With `tags=` (empty), I sent a message from another email account to my 
test account. Synchronization of S not present/`unread` worked correctly.


I'm not sure if this behavior is considered a bug, given notmuch's model 
for dealing with duplicate messages. I can live with this corner case, 
anyway. Also, for the record, I understand that I will probably have to 
switch to `tags=new` or something in the future to avoid reprocessing 
old mail. (Although emacs seems to treat `new` as a special tag? Maybe 
I'll use something else).



And just in case this feels like a "duh moment" to you - we all have them ;)


All the same, from reading the documentation, I felt like the `tags` and 
`synchronize_flags` options operate independently of each other, which 
seems to be mostly correct. However, some more details of how things are 
processed and how it interacts with duplicate messages would help build 
a stronger mental model on which to make decisions.


I also found this while searching around, which was helpful, but I can't 
figure out how to access it from the notmuch homepage. Maybe it's out of 
date?:


https://github.com/notmuch/notmuch-wiki/blob/master/special-tags.mdwn

Thanks,
Andrew Todd
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org