v3 of postpone patches

[David Bremner]

This obsoletes, and hopefully doesn't break

 id:1478196548-19596-1-git-send-email-markwalters1...@gmail.com

Compared to Mark's original patches I added the check for encryption
tags discussed in that thread, and split the draft stuff to it's own
file.

I have to admit I (or someone else) still need to line by line review
this, but I have tested it a bit more while working on the encryption
tag check. On the other hand, splitting the draft functions out into
notmuch-draft.el did involve a certain amount of blind search and
replace.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[Patch v3 2/2] emacs: postpone/resume support

[David Bremner]

From: Mark Walters 

This provides preliminary support for postponing and resuming in the
emacs frontend. On postponing it uses notmuch insert to put the
message in the notmuch database; resume gets the raw file from notmuch
and using the emacs function mime-to-mml reconstructs the message
(including attachments).

Current bindings are C-x C-s to save a draft, C-c C-p to postpone a
draft (save and exit compose buffer), and e to resume a draft from
show or tree mode.

Previous drafts get tagged deleted on subsequent saves, or on the
message being sent.

Each draft gets its own message-id, and we use the namespace
draft- for draft message ids (so, at least for most people, drafts
are easily distinguisable).
---
 emacs/notmuch-draft.el | 261 +
 emacs/notmuch-mua.el   |   4 +
 emacs/notmuch-show.el  |  10 ++
 emacs/notmuch-tree.el  |   1 +
 4 files changed, 276 insertions(+)
 create mode 100644 emacs/notmuch-draft.el

diff --git a/emacs/notmuch-draft.el b/emacs/notmuch-draft.el
new file mode 100644
index 000..806c1a7
--- /dev/null
+++ b/emacs/notmuch-draft.el
@@ -0,0 +1,261 @@
+;;; notmuch-draft.el --- functions for postponing and editing drafts
+;;
+;; Copyright © Mark Walters
+;; Copyright © David Bremner
+;;
+;; This file is part of Notmuch.
+;;
+;; Notmuch is free software: you can redistribute it and/or modify it
+;; under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+;;
+;; Notmuch is distributed in the hope that it will be useful, but
+;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+;; General Public License for more details.
+;;
+;; You should have received a copy of the GNU General Public License
+;; along with Notmuch.  If not, see .
+;;
+;; Authors: Mark Walters 
+;; David Bremner 
+
+;;; Code:
+
+(require 'notmuch-maildir-fcc)
+
+(declare-function notmuch-show-get-message-id "notmuch-show" ( bare))
+
+(defgroup notmuch-draft nil
+  "Saving and editing drafts in Notmuch."
+  :group 'notmuch)
+
+(defcustom notmuch-draft-tags '("+draft")
+  "List of tags changes to apply to a draft message when it is saved in the 
database.
+
+Tags starting with \"+\" (or not starting with either \"+\" or
+\"-\") in the list will be added, and tags starting with \"-\"
+will be removed from the message being stored.
+
+For example, if you wanted to give the message a \"draft\" tag
+but not the (normally added by default) \"inbox\" tag, you would
+set:
+(\"+draft\" \"-inbox\")"
+  :type '(repeat string)
+  :group 'notmuch-draft)
+
+(defcustom notmuch-draft-folder "drafts"
+  "Folder to save draft messages in.
+
+This should be specified relative to the root of the notmuch
+database. It will be created if necessary."
+  :type 'string
+  :group 'notmuch-draft)
+
+(defcustom notmuch-draft-quoted-tags '()
+  "Mml tags to quote.
+
+This should be a list of mml tags to quote before saving. You do
+not need to include \"secure\" as that is handled separately.
+
+If you include \"part\" then attachments will not be saved with
+the draft -- if not then they will be saved with the draft. The
+former means the attachments may not still exist when you resume
+the message, the latter means that the attachments as they were
+when you postponed will be sent with the resumed message.
+
+Note you may get strange results if you change this between
+postponing and resuming a message."
+  :type '(repeat string)
+  :group 'notmuch-send)
+
+(defcustom notmuch-draft-save-plaintext 'ask
+  "Should notmuch save/postpone in plaintext messages that seem
+  like they are intended to be sent encrypted
+(i.e with an mml encryption tag in it)."
+  :type '(radio
+ (const :tag "Never" nil)
+ (const :tag "Ask every time" ask)
+ (const :tag "Always" t))
+  :group 'notmuch-draft
+  :group 'notmuch-crypto)
+
+(defvar notmuch-draft-encryption-tag-regex
+  "<#\\(part encrypt\\|secure.*mode=.*encrypt>\\)"
+  "Regular expression matching mml tags indicating encryption of part or 
message")
+
+(defvar notmuch-draft-id nil
+  "Message-id of the most recent saved draft of this message")
+(make-variable-buffer-local 'notmuch-draft-id)
+
+(defun notmuch-draft--mark-deleted ()
+  "Tag the last saved draft deleted.
+
+Used when a new version is saved, or the message is sent."
+  (when notmuch-draft-id
+(notmuch-tag notmuch-draft-id '("+deleted"
+
+(defun notmuch-draft-quote-some-mml ()
+  "Quote the mml tags in `notmuch-draft-quoted-tags`."
+  (save-excursion
+;; First we deal with any secure tag separately.
+(message-goto-body)
+(when (looking-at "<#secure[^\n]*>\n")
+  (let ((secure-tag (match-string 0)))
+   (delete-region 

[PATCH] notmuch-config: replace config reading function

[Ioan-Adrian Ratiu]

Config files are currently read using glib's g_key_file_load_from_file
function which is very inconvenient because it's limited by design to read
only from "regular data files" in a filesystem. Because of this limitation
notmuch can't read configs from pipes, fifos, sockets, stdin, etc. Not even
"notmuch --config=/dev/stdin" works:

Error reading configuration file /dev/stdin: Not a regular file

So replace g_key_file_load_from_file with g_key_file_load_from_data which
gives us much more freedom to read configs from multiple sources.

This also helps the more security sensitive users: If someone has private
information in the config file, it can be encrypted on disk, then decrypted
in RAM and passed through a pipe directly to notmuch without the use of
intermediate plain text files.

Signed-off-by: Ioan-Adrian Ratiu 
---
 notmuch-config.c | 63 
 1 file changed, 59 insertions(+), 4 deletions(-)

diff --git a/notmuch-config.c b/notmuch-config.c
index e5d42a0..8435815 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -202,6 +202,64 @@ get_username_from_passwd_file (void *ctx)
 return name;
 }
 
+static gboolean
+get_config_from_file (notmuch_config_t *config, GError **error)
+{
+#define BUF_SIZE 4096
+char buffer[BUF_SIZE];
+size_t content_size = 1; // includes NULL
+
+FILE *fp = fopen(config->filename, "r");
+if (fp == NULL) {
+   *error = g_error_new(G_FILE_ERROR,
+G_FILE_ERROR_NOENT,
+"Couldn't open config file '%s': %s.\n",
+config->filename,
+strerror(errno));
+   return FALSE;
+}
+
+char *config_str = talloc_zero_array (config, char, BUF_SIZE);
+if (config_str == NULL)
+{
+   *error = g_error_new(G_FILE_ERROR,
+G_FILE_ERROR_NOMEM,
+"Out of memory while reading config.\n");
+   return FALSE;
+}
+
+while (fgets (buffer, BUF_SIZE, fp))
+{
+   content_size += strlen(buffer);
+   config_str = talloc_realloc(config, config_str, char, content_size);
+   if (config_str == NULL)
+   {
+   *error = g_error_new(G_FILE_ERROR,
+G_FILE_ERROR_NOMEM,
+"Failed to reallocate config memory.\n");
+   return FALSE;
+   }
+   strcat (config_str, buffer);
+}
+
+if (ferror (fp))
+{
+   *error = g_error_new(G_FILE_ERROR,
+G_FILE_ERROR_IO,
+"I/O error reading configuration from '%s'.\n",
+config->filename);
+   return FALSE;
+}
+
+fclose(fp);
+
+return g_key_file_load_from_data (config->key_file,
+ config_str,
+ strlen(config_str),
+ G_KEY_FILE_KEEP_COMMENTS,
+ error);
+}
+
 /* Open the named notmuch configuration file. If the filename is NULL,
  * the value of the environment variable $NOTMUCH_CONFIG will be used.
  * If $NOTMUCH_CONFIG is unset, the default configuration file
@@ -289,10 +347,7 @@ notmuch_config_open (void *ctx,
 config->search_exclude_tags_length = 0;
 config->crypto_gpg_path = NULL;
 
-if (! g_key_file_load_from_file (config->key_file,
-config->filename,
-G_KEY_FILE_KEEP_COMMENTS,
-))
+if (! get_config_from_file (config, ))
 {
if (error->domain == G_FILE_ERROR && error->code == G_FILE_ERROR_NOENT) 
{
/* If create_new is true, then the caller is prepared for a
-- 
2.10.2

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 9/9] build: only install known man pages

[Jani Nikula]

Install man pages based on $(MAN_GZIP_FILES), which directly
corresponds to the man page source rst files. This way we can filter
the man pages to be installed as needed.
---
 doc/Makefile.local | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index 8a2f656bcad4..c6f05ca879c0 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -68,7 +68,7 @@ MAN_GZIP_FILES += ${APIMAN}.gz
 apidocs: $(APIMAN)
 install-apidocs: ${APIMAN}.gz
mkdir -p "$(DESTDIR)$(mandir)/man3"
-   install -m0644  $(DOCBUILDDIR)/man/man3/*.3.gz  
$(DESTDIR)/$(mandir)/man3
+   install -m0644 $(filter %.3.gz,$(MAN_GZIP_FILES)) 
$(DESTDIR)/$(mandir)/man3
 
 $(APIMAN): $(dir)/config.dox $(srcdir)/$(dir)/doxygen.cfg 
$(srcdir)/lib/notmuch.h
mkdir -p $(DOCBUILDDIR)/man/man3
@@ -92,9 +92,9 @@ install-man: ${MAN_GZIP_FILES}
mkdir -p "$(DESTDIR)$(mandir)/man1"
mkdir -p "$(DESTDIR)$(mandir)/man5"
mkdir -p "$(DESTDIR)$(mandir)/man7"
-   install -m0644 $(DOCBUILDDIR)/man/man1/*.1.gz $(DESTDIR)/$(mandir)/man1
-   install -m0644 $(DOCBUILDDIR)/man/man5/*.5.gz $(DESTDIR)/$(mandir)/man5
-   install -m0644 $(DOCBUILDDIR)/man/man7/*.7.gz $(DESTDIR)/$(mandir)/man7
+   install -m0644 $(filter %.1.gz,$(MAN_GZIP_FILES)) 
$(DESTDIR)/$(mandir)/man1
+   install -m0644 $(filter %.5.gz,$(MAN_GZIP_FILES)) 
$(DESTDIR)/$(mandir)/man5
+   install -m0644 $(filter %.7.gz,$(MAN_GZIP_FILES)) 
$(DESTDIR)/$(mandir)/man7
cd $(DESTDIR)/$(mandir)/man1 && ln -sf notmuch.1.gz notmuch-setup.1.gz
 endif
 
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 8/9] build: generate man page list from source files, not conf.py

[Jani Nikula]

Use $(wildcard ...) to generate the list of man pages based on the rst
source files present in the man page directories, instead of reading
conf.py. This has three main benefits:

1) This makes the man page build slightly less complicated and easier
   to understand. At least there are fewer moving parts.

2) This makes the build fail if we add a man page rst file, but fail
   to add it to conf.py.

3) We can use Sphinx constructs in conf.py that are not available when
   importing the file into a normal python program such as
   mkdocdeps.py.
---
 doc/.gitignore |  1 -
 doc/Makefile.local | 23 +--
 doc/mkdocdeps.py   | 18 --
 3 files changed, 13 insertions(+), 29 deletions(-)
 delete mode 100644 doc/mkdocdeps.py

diff --git a/doc/.gitignore b/doc/.gitignore
index d0da78e510f1..9fa35d08a95e 100644
--- a/doc/.gitignore
+++ b/doc/.gitignore
@@ -1,4 +1,3 @@
 *.pyc
-docdeps.mk
 _build
 config.dox
diff --git a/doc/Makefile.local b/doc/Makefile.local
index 5dc1cad489cc..8a2f656bcad4 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -7,13 +7,23 @@ SPHINXOPTS:= -q
 SPHINXBUILD   = sphinx-build
 DOCBUILDDIR  := $(dir)/_build
 
-mkdocdeps := $(PYTHON) $(srcdir)/$(dir)/mkdocdeps.py
-
 # Internal variables.
 ALLSPHINXOPTS   := -d $(DOCBUILDDIR)/doctrees $(SPHINXOPTS) $(srcdir)/$(dir)
 APIMAN := $(DOCBUILDDIR)/man/man3/notmuch.3
 DOXYFILE   := $(srcdir)/$(dir)/doxygen.cfg
 
+MAN1_RST := $(wildcard $(srcdir)/doc/man1/*.rst)
+MAN5_RST := $(wildcard $(srcdir)/doc/man5/*.rst)
+MAN7_RST := $(wildcard $(srcdir)/doc/man7/*.rst)
+MAN_RST_FILES := $(MAN1_RST) $(MAN5_RST) $(MAN7_RST)
+
+MAN1_ROFF := $(patsubst 
$(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN1_RST:.rst=.1))
+MAN5_ROFF := $(patsubst 
$(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN5_RST:.rst=.5))
+MAN7_ROFF := $(patsubst 
$(srcdir)/doc/%,$(DOCBUILDDIR)/man/%,$(MAN7_RST:.rst=.7))
+MAN_ROFF_FILES := $(MAN1_ROFF) $(MAN5_ROFF) $(MAN7_ROFF)
+
+MAN_GZIP_FILES := $(addsuffix .gz,${MAN_ROFF_FILES})
+
 .PHONY: sphinx-html sphinx-texinfo sphinx-info
 
 .PHONY: install-man build-man apidocs install-apidocs
@@ -30,10 +40,6 @@ sphinx-texinfo:
 sphinx-info: sphinx-texinfo
make -C $(DOCBUILDDIR)/texinfo info
 
--include $(dir)/docdeps.mk
-
-MAN_GZIP_FILES := $(addsuffix .gz,${MAN_ROFF_FILES})
-
 # Use the man page converter that is available. We should never depend
 # on MAN_ROFF_FILES if a converter is not available.
 ${MAN_ROFF_FILES}: $(DOCBUILDDIR)/.roff.stamp
@@ -96,8 +102,5 @@ $(dir)/config.dox: version.stamp
echo "PROJECT_NAME = \"Notmuch $(VERSION)\"" > $@
echo "INPUT=${srcdir}/lib/notmuch.h" >> $@
 
-$(dir)/docdeps.mk: $(dir)/conf.py $(dir)/mkdocdeps.py
-   $(mkdocdeps) $(srcdir)/doc $(DOCBUILDDIR) $@
-
-CLEAN := $(CLEAN) $(DOCBUILDDIR) $(dir)/docdeps.mk $(DOCBUILDDIR)/.roff.stamp
+CLEAN := $(CLEAN) $(DOCBUILDDIR) $(DOCBUILDDIR)/.roff.stamp
 CLEAN := $(CLEAN) $(MAN_GZIP_FILES) $(MAN_ROFF_FILES) $(dir)/conf.pyc 
$(dir)/config.dox
diff --git a/doc/mkdocdeps.py b/doc/mkdocdeps.py
deleted file mode 100644
index b87fe3e80c0f..
--- a/doc/mkdocdeps.py
+++ /dev/null
@@ -1,18 +0,0 @@
-import sys
-
-srcdir = sys.argv[1]
-builddir = sys.argv[2]
-outfile = sys.argv[3]
-
-sys.path.insert(0, srcdir)
-import conf
-
-roff_files = []
-rst_files = []
-for page in conf.man_pages:
-rst_files = rst_files + ["{0:s}/{1:s}.rst".format(srcdir,page[0])]
-roff_files = roff_files + 
["{0:s}/man/{1:s}.{2:d}".format(builddir,page[0],page[4])]
-
-with open(outfile, 'w') as out:
-out.write('MAN_ROFF_FILES := ' + ' \\\n\t'.join(roff_files) + '\n')
-out.write('MAN_RST_FILES := ' + ' \\\n\t'.join(rst_files) + '\n')
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 7/9] build: do not touch roff files after sphinx-build

[Jani Nikula]

If Sphinx fails to create any of the roff files, having touch create
them hides the errors until someone realizes, possibly much later,
that the resulting files are empty. (Note that gzip doesn't fail on
empty input files.) Sphinx will change the timestamps of any files it
has written anyway.
---
 doc/Makefile.local | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/doc/Makefile.local b/doc/Makefile.local
index 8633cfcd3b51..5dc1cad489cc 100644
--- a/doc/Makefile.local
+++ b/doc/Makefile.local
@@ -53,7 +53,7 @@ else
@echo "Fatal: build dependency fail."
@false
 endif
-   touch ${MAN_ROFF_FILES} $@
+   touch $@
 
 install-man: install-apidocs
 
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 6/9] doc: fix references in notmuch-emacs.rst

[Jani Nikula]

Fix make sphinx-texinfo warnings:

WARNING: undefined label: notmuch-jump (if the link has no caption the
label must precede a section header)

WARNING: undefined label: notmuch-saved-searches (if the link has no
caption the label must precede a section header)
---
 doc/notmuch-emacs.rst | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/doc/notmuch-emacs.rst b/doc/notmuch-emacs.rst
index d68542d349a0..5e25996f69b4 100644
--- a/doc/notmuch-emacs.rst
+++ b/doc/notmuch-emacs.rst
@@ -189,10 +189,12 @@ following key bindings:
 ``j``
 Jump to saved searches using :ref:`notmuch-jump`.
 
+.. _notmuch-jump:
+
 notmuch-jump
 
 
-Saved searches configured through :ref:`notmuch-saved-searches` can
+Saved searches configured through :ref:`saved-searches` can
 include a "shortcut key" that's accessible through notmuch-jump.
 Pressing ``j`` anywhere in notmuch followed by the configured shortcut
 key of a saved search will immediately jump to that saved search.  For
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 5/9] doc/conf.py: include command name in texinfo document title

[Jani Nikula]

Seems to make more sense than to have the same title for everything.
---
 doc/conf.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/doc/conf.py b/doc/conf.py
index f8da1a422f9a..b6cfa1aa3499 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -132,7 +132,7 @@ man_pages = [
 texinfo_no_detailmenu = True
 
 texinfo_documents = [
-('notmuch-emacs', 'notmuch-emacs', u'notmuch Documentation',
+('notmuch-emacs', 'notmuch-emacs', u'notmuch-emacs documentation',
  notmuch_authors, 'notmuch-emacs',
  'emacs based front-end for notmuch', 'Miscellaneous'),
 ]
@@ -142,7 +142,7 @@ texinfo_documents += [
 (
 x[0],  # source start file
 x[1],  # target name
-u'notmuch Documentation',  # title
+x[1] + u' documentation',  # title
 x[3][0],   # author
 x[1],  # dir menu entry
 x[2],  # description
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/9] doc/conf.py: add notmuch-emacs-mua to texinfo documents

[Jani Nikula]

This was overlooked when adding the man page.
---
 doc/conf.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/doc/conf.py b/doc/conf.py
index 8b9329662bf0..fce5a3afdfc2 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -153,6 +153,9 @@ texinfo_documents = [
 ('man1/notmuch-dump','notmuch-dump',u'notmuch Documentation',
   u'Carl Worth and many others', 'notmuch-dump',
   'creates a plain-text dump of the tags of each message','Miscellaneous'),
+('man1/notmuch-emacs-mua','notmuch-emacs-mua',u'notmuch Documentation',
+  u'Carl Worth and many others', 'notmuch-emacs-mua',
+  u'send mail with notmuch and emacs','Miscellaneous'),
 ('man5/notmuch-hooks','notmuch-hooks',u'notmuch Documentation',
   u'Carl Worth and many others', 'notmuch-hooks',
   'hooks for notmuch','Miscellaneous'),
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 3/9] doc/conf.py: abstract notmuch authors

[Jani Nikula]

Don't repeat the same stuff so much. We probably don't want to track
individual authors for man/texinfo pages anyway.

No user visible changes.
---
 doc/conf.py | 36 +++-
 1 file changed, 19 insertions(+), 17 deletions(-)

diff --git a/doc/conf.py b/doc/conf.py
index aa02b9a8bb7b..3cec01451f35 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -52,71 +52,73 @@ htmlhelp_basename = 'notmuchdoc'
 # One entry per manual page. List of tuples
 # (source start file, name, description, authors, manual section).
 
+notmuch_authors = u'Carl Worth and many others'
+
 man_pages = [
 
 ('man1/notmuch','notmuch',
 u'thread-based email index, search, and tagging',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-address','notmuch-address',
 u'output addresses from matching messages',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-compact','notmuch-compact',
 u'compact the notmuch database',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-config','notmuch-config',
 u'access notmuch configuration file',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-count','notmuch-count',
 u'count messages matching the given search terms',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-dump','notmuch-dump',
 u'creates a plain-text dump of the tags of each message',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-emacs-mua','notmuch-emacs-mua',
 u'send mail with notmuch and emacs',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man5/notmuch-hooks','notmuch-hooks',
 u'hooks for notmuch',
-[u'Carl Worth and many others'], 5),
+[notmuch_authors], 5),
 
 ('man1/notmuch-insert','notmuch-insert',
 u'add a message to the maildir and notmuch database',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-new','notmuch-new',
 u'incorporate new mail into the notmuch database',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-reply','notmuch-reply',
 u'constructs a reply template for a set of messages',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-restore','notmuch-restore',
 u'restores the tags from the given file (see notmuch dump)',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-search','notmuch-search',
 u'search for messages matching the given search terms',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man7/notmuch-search-terms','notmuch-search-terms',
 u'syntax for notmuch queries',
-[u'Carl Worth and many others'], 7),
+[notmuch_authors], 7),
 
 ('man1/notmuch-show','notmuch-show',
 u'show messages matching the given search terms',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 ('man1/notmuch-tag','notmuch-tag',
 u'add/remove tags for all messages matching the search terms',
-[u'Carl Worth and many others'], 1),
+[notmuch_authors], 1),
 
 
 ]
@@ -133,7 +135,7 @@ texinfo_no_detailmenu = True
 
 texinfo_documents = [
  ('notmuch-emacs', 'notmuch-emacs', u'notmuch Documentation',
-   u'Carl Worth and many others', 'notmuch-emacs',
+   notmuch_authors, 'notmuch-emacs',
'emacs based front-end for notmuch', 'Miscellaneous'),
 ]
 
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/9] doc/conf.py: generate texinfo_documents list from man_pages list

[Jani Nikula]

No need to repeat mostly the same information twice in conf.py. We
probably want to have a corresponding texinfo document for all the man
pages. Python list comprehension to the rescue. (The reverse is not
true; we have a texinfo document for notmuch-emacs we don't want as a
man page.)

There should be no user visible changes.
---
 doc/conf.py | 60 
 1 file changed, 12 insertions(+), 48 deletions(-)

diff --git a/doc/conf.py b/doc/conf.py
index fce5a3afdfc2..aa02b9a8bb7b 100644
--- a/doc/conf.py
+++ b/doc/conf.py
@@ -135,52 +135,16 @@ texinfo_documents = [
  ('notmuch-emacs', 'notmuch-emacs', u'notmuch Documentation',
u'Carl Worth and many others', 'notmuch-emacs',
'emacs based front-end for notmuch', 'Miscellaneous'),
-('man1/notmuch','notmuch',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch',
-  'thread-based email index, search, and tagging','Miscellaneous'),
-('man1/notmuch-address','notmuch-address',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-address',
-  'output addresses from matching messages','Miscellaneous'),
-('man1/notmuch-compact','notmuch-compact',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-compact',
-  'compact the notmuch database','Miscellaneous'),
-('man1/notmuch-config','notmuch-config',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-config',
-  'access notmuch configuration file','Miscellaneous'),
-('man1/notmuch-count','notmuch-count',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-count',
-  'count messages matching the given search terms','Miscellaneous'),
-('man1/notmuch-dump','notmuch-dump',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-dump',
-  'creates a plain-text dump of the tags of each message','Miscellaneous'),
-('man1/notmuch-emacs-mua','notmuch-emacs-mua',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-emacs-mua',
-  u'send mail with notmuch and emacs','Miscellaneous'),
-('man5/notmuch-hooks','notmuch-hooks',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-hooks',
-  'hooks for notmuch','Miscellaneous'),
-('man1/notmuch-insert','notmuch-insert',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-insert',
-  'add a message to the maildir and notmuch database','Miscellaneous'),
-('man1/notmuch-new','notmuch-new',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-new',
-  'incorporate new mail into the notmuch database','Miscellaneous'),
-('man1/notmuch-reply','notmuch-reply',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-reply',
-  'constructs a reply template for a set of messages','Miscellaneous'),
-('man1/notmuch-restore','notmuch-restore',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-restore',
-  'restores the tags from the given file (see notmuch 
dump)','Miscellaneous'),
-('man1/notmuch-search','notmuch-search',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-search',
-  'search for messages matching the given search terms','Miscellaneous'),
-('man7/notmuch-search-terms','notmuch-search-terms',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-search-terms',
-  'syntax for notmuch queries','Miscellaneous'),
-('man1/notmuch-show','notmuch-show',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-show',
-  'show messages matching the given search terms','Miscellaneous'),
-('man1/notmuch-tag','notmuch-tag',u'notmuch Documentation',
-  u'Carl Worth and many others', 'notmuch-tag',
-  'add/remove tags for all messages matching the search 
terms','Miscellaneous'),
 ]
+
+# generate texinfo list from man page list
+texinfo_documents += [
+(
+x[0],  # source start file
+x[1],  # target name
+u'notmuch Documentation',  # title
+x[3][0],   # author
+x[1],  # dir menu entry
+x[2],  # description
+'Miscellaneous'# category
+) for x in man_pages]
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 0/9] doc: config and build improvements

[Jani Nikula]

Here's a bunch of fixes to the documentation config and build.

BR,
Jani.


Jani Nikula (9):
  doc/conf.py: add notmuch-emacs-mua to texinfo documents
  doc/conf.py: generate texinfo_documents list from man_pages list
  doc/conf.py: abstract notmuch authors
  doc/conf.py: reindent, whitespace clean man page, texinfo lists
  doc/conf.py: include command name in texinfo document title
  doc: fix references in notmuch-emacs.rst
  build: do not touch roff files after sphinx-build
  build: generate man page list from source files, not conf.py
  build: only install known man pages

 doc/.gitignore|   1 -
 doc/Makefile.local|  33 +-
 doc/conf.py   | 165 --
 doc/mkdocdeps.py  |  18 --
 doc/notmuch-emacs.rst |   4 +-
 5 files changed, 87 insertions(+), 134 deletions(-)
 delete mode 100644 doc/mkdocdeps.py

-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] cli: consider files vanishing during notmuch new non-fatal

[Paul Wise]

On Sat, 2016-11-05 at 14:57 +0200, Jani Nikula wrote:

> Add a new exit code for when files vanished, so the caller has a
> chance to detect the race and re-run notmuch new to recover.

I don't think this is the right approach for two reasons:

The exit code you have chosen is still a failure so I will still get
notified for a minor issue. I use chronic to detect fail scenarios.

This is a pretty normal scenario when you have a mail program open and
are auto-running `notmuch new` on a scheduled basis or when new mail
arrives. notmuch should just ignore the error and continue as normal.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [Paul Wise] Bug#843127: notmuch: race condition in `notmuch new`?

[Paul Wise]

On Fri, 2016-11-04 at 20:47 +0200, Jani Nikula wrote:

> Do you have some other software modifying your mail store while
> you're running notmuch new?

The folder in question has my laptop's exim4 service writing to it when
my cron jobs generate email.

On Fri, 2016-11-04 at 13:26 -0300, David Bremner wrote:

> inotify sounds a bit overcomplicated and perhaps non-portable?

There are similar APIs on non-Linux OSes but there is indeed no common
API or library to abstract away this sort of feature AFAIK.

> It should probably just tolerate disappearing files better, consider
> that a warning.

That sounds like the correct solution indeed. Probably if the code
notices that a file disappeared, it should also rescan the nearby
folders to see if the file was moved instead of deleted.

> As a workaround, if you can replace background use of notmuch-new
> with notmuch-insert (and I understand this doesn't work for
> everyone), you will eliminate this kind of race condition.

Hmm, I don't think that will work for me.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise


signature.asc
Description: This is a digitally signed message part
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] cli: consider files vanishing during notmuch new non-fatal

[Jani Nikula]

If some software other than notmuch new renames or removes files
during the notmuch new scan (specifically after scandir but before
indexing the file), keep going instead of bailing out. Failing to
index the file is just a race condition between notmuch and the other
software; the rename could happen after the notmuch new scan
anyway. It's not fatal, and we'll catch the renamed files on the next
scan.

Add a new exit code for when files vanished, so the caller has a
chance to detect the race and re-run notmuch new to recover.

Reported by Paul Wise  at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=843127

---

Having notmuch new re-run (parts of) the scan automatically seems a
rather more involved change. So does inotify support. This simple
change both finishes the scan and lets the user recover, IMO a
reasonable first step.

Please suggest a better alternative to "vanish" in code...
---
 notmuch-client.h |  8 
 notmuch-new.c| 15 ---
 2 files changed, 20 insertions(+), 3 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index 9ce2aef17431..d2057e26c5cd 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -114,6 +114,14 @@ chomp_newline (char *str)
str[strlen(str)-1] = '\0';
 }
 
+/* Exit status code indicating that file(s) in the mail store were
+ * removed or renamed after notmuch new scanned the directories but
+ * before indexing the file(s). If the file was renamed, the indexing
+ * might not be complete, and the user is advised to re-run notmuch
+ * new.
+ */
+#define NOTMUCH_EXIT_VANISHED_FILES 10
+
 /* Exit status code indicating the requested format version is too old
  * (support for that version has been dropped).  CLI code should use
  * notmuch_exit_if_unsupported_format rather than directly exiting
diff --git a/notmuch-new.c b/notmuch-new.c
index c55dea7bc1b7..e694a6adcee1 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -53,6 +53,7 @@ typedef struct {
 int total_files;
 int processed_files;
 int added_messages, removed_messages, renamed_messages;
+int vanished_files;
 struct timeval tv_start;
 
 _filename_list_t *removed_files;
@@ -280,11 +281,13 @@ add_file (notmuch_database_t *notmuch, const char 
*filename,
 case NOTMUCH_STATUS_FILE_NOT_EMAIL:
fprintf (stderr, "Note: Ignoring non-mail file: %s\n", filename);
break;
-/* Fatal issues. Don't process anymore. */
 case NOTMUCH_STATUS_FILE_ERROR:
+   /* Someone renamed/removed the file between scandir and now. */
+   state->vanished_files++;
fprintf (stderr, "Unexpected error with file %s\n", filename);
(void) print_status_database ("add_file", notmuch, status);
-   goto DONE;
+   break;
+/* Fatal issues. Don't process anymore. */
 case NOTMUCH_STATUS_READ_ONLY_DATABASE:
 case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
 case NOTMUCH_STATUS_OUT_OF_MEMORY:
@@ -1151,5 +1154,11 @@ notmuch_new_command (notmuch_config_t *config, int argc, 
char *argv[])
 if (!no_hooks && !ret && !interrupted)
ret = notmuch_run_hook (db_path, "post-new");
 
-return ret || interrupted ? EXIT_FAILURE : EXIT_SUCCESS;
+if (ret || interrupted)
+   return EXIT_FAILURE;
+
+if (add_files_state.vanished_files)
+   return NOTMUCH_EXIT_VANISHED_FILES;
+
+return EXIT_SUCCESS;
 }
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: add check for encryption before saving.

[Mark Walters]

On Sat, 05 Nov 2016, David Bremner  wrote:
> Mark Walters  writes:
>
>>
>> I think this is an excellent thing to add. I agree that false positives
>> aren't much of a worry. If someone bumps into them a lot then they can
>> complain or come up with a better regex.
>>
>
> Should the regex also be a defcustom?

I think not a defcustom, but perhaps a defvar (the difference being that
defvar's are hidden from most users -- anyone who can write a better
regex can use setq in their init file). I think even if we don't expect
users to change it making it a defvar is quite clean so that might be
best.

Best wishes

Mark





___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: add check for encryption before saving.

[David Bremner]

Mark Walters  writes:

>
> I think this is an excellent thing to add. I agree that false positives
> aren't much of a worry. If someone bumps into them a lot then they can
> complain or come up with a better regex.
>

Should the regex also be a defcustom?

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] emacs: add check for encryption before saving.

[Mark Walters]

On Sat, 05 Nov 2016, David Bremner  wrote:
> This is intended to decrease the chance of people ending up with a bunch
> of plaintext drafts of encrypted messages without knowing it.
>
> The check is intentionally overcautious; I think the false positive of
> misplaced #secure tag is probably OK here.
> ---
>
> This is somewhat RFC. The regex needs to be double checked, and the
> variable name is not ideal. However it does solve reduce a worry I
> have about this code saving drafts of sensitive messages in plaintext
> that are effectively invisible because they are tagged deleted.

Hi

I think this is an excellent thing to add. I agree that false positives
aren't much of a worry. If someone bumps into them a lot then they can
complain or come up with a better regex.

>  emacs/notmuch-message.el | 25 +
>  1 file changed, 25 insertions(+)
>
> diff --git a/emacs/notmuch-message.el b/emacs/notmuch-message.el
> index a503296..a2b079d 100644
> --- a/emacs/notmuch-message.el
> +++ b/emacs/notmuch-message.el
> @@ -80,6 +80,12 @@ postponing and resuming a message."
>:type '(repeat string)
>:group 'notmuch-send)
>  
> +(defcustom notmuch-message-warn-encryption t
> +  "Warn if the user postpones or saves a message with an mml encryption tag 
> in it"
> +  :type 'boolean
> +  :group 'notmuch-send
> +  :group 'notmuch-crypto)

I think it would be good if the variable name contained postpone or save
in it as it is not part of the normal send message route. Perhaps
notmuch-message-warn-unencrypted-save ? (not perfect I know)

Maybe change the docstring to something like "Warn if the user postpones
or saves a message that would be encrypted if sent (i.e., has an mml
encryption tag)."

> +(defun notmuch-message-check-encryption ()
> +  "Query user if there an mml tag that looks like it might indicate 
> encryption.

Maybe a name like notmuch-message-check-has-encrypt-tag (or omit "check")?

But this is probably excessive bikeshedding on my part. In any case the
only one I care about above is the name of the defcustom variable

(and there is one trivial typo below)

Best wishes

Mark

> +Returns t if there is no such tag, or the user confirms they mean
> +it."
> +  (save-excursion
> +(message-goto-body)
> +  (or
> +   ;; We fine if there is no secure tag, and no #part encryption
 ^^^
 are
  

> +   (not (re-search-forward "<#\\(part 
> encrypt\\|secure.*mode=.*encrypt>\\)" nil 't))
> +   ;; The user confirms they means it.
> +   (yes-or-no-p "\
> +This message contains mml tags that suggest it is intended to be encrypted.
> +Really save and index an unencrypted copy?
> +(Customize `notmuch-message-warn-encrypted' to avoid this warning)"
> +
>  (defun notmuch-message-save-draft ()
>"Save the current draft message in the notmuch database.
>  
> @@ -147,6 +169,9 @@ This saves the current message in the database with tags
>  `notmuch-message-draft-tags` (in addition to any default tags
>  applied to newly inserted messages)."
>(interactive)
> +  (when (and notmuch-message-warn-encryption
> +  (not (notmuch-message-check-encryption))
> +  (error "Save aborted")))
>(let (;; We need the message id as we need it for tagging. Note
>   ;; message-make-message-id gives the id inside a "<" ">" pair,
>   ;; but notmuch doesn't want that form, so remove them.
> -- 
> 2.10.1
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch