Re: [PATCH v4 3/3] cli/reply: make --decrypt take a keyword

2017-12-30 Thread Daniel Kahn Gillmor
On Sat 2017-12-30 09:05:40 -0400, David Bremner wrote:
> I need more time to think about this, so I'd rather defer till after the
> release in any case. 

are you saying that you want to defer this whole series until after the
release?  that would be a real shame, since it would mean we'd have
both:

notmuch show --decrypt

and

notmuch new --decrypt=true

which seems particularly troubling.  please let's at least make it a
keyword in all cases.

> But at some point we collectively (I think? Maybe Jamie and I browbeat
> you into it) decided that it was OK for --decrypt=true to have context
> dependent behaviour.  It seems to me that "different things in a
> different context" issue already exists.

Oh, i'm not saying that "notmuch show --decrypt=true" must mean exactly
the same thing as "notmuch new --decrypt=true" -- i understand that it
does not mean the same thing, because the contexts are different.  My
complaint was that documenting "notmuch show --decrypt=nostash" seems
like it introduces confusion around the fact that --decrypt=nostash *is*
identical to --decrypt=true in one place, and is functionally
(significantly) different from --decrypt=true in another place.

iow: i'm fine with --decrypt=true being a coherent policy about message
decryption that does different things in different contexts.  I think
explaining about --decrypt=nostash being the same as that policy in some
contexts and different in others is pretty awkward, but if people really
want it, i won't block on it, and i look forward to seeing the patches
to the documentation.

   --dkg


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] NEWS: library changes for 0.26

2017-12-30 Thread David Bremner
Floris Bruynooghe  writes:


>> +Library Changes
>> +---
>> +
>> +Indexing files with duplicate message-id
>> +
>> +  Files with message-id's are now indexed, and searchable via terms
>
> "Files with duplicate message-IDs are now ..."?  

Thanks, fixed in git.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] NEWS: library changes for 0.26

2017-12-30 Thread Floris Bruynooghe
David Bremner  writes:

> These are just "my" changes, as arbited by "git shortlog", which
> sometimes lies.
> ---
>  NEWS | 21 +
>  1 file changed, 21 insertions(+)
>
> diff --git a/NEWS b/NEWS
> index 3b6404e7..1c5edf4c 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -41,6 +41,27 @@ Indexing cleartext of encrypted e-mails
>that the notmuch index itself is adequately protected.  DO NOT USE
>this feature without considering the security of your index.
>  
> +Library Changes
> +---
> +
> +Indexing files with duplicate message-id
> +
> +  Files with message-id's are now indexed, and searchable via terms

"Files with duplicate message-IDs are now ..."?  
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] WIP: support XDG database directory

2017-12-30 Thread David Bremner
---

Just a rebase against current master, based on discussion in IRC
today.  AFAIK, the general approach could be extended to support a
"NOTMUCH_DATABASE_DIRECTORY" environment variable, which if think is
what Tomi was suggesting previously.

 lib/database.cc| 65 --
 test/T560-lib-error.sh |  2 +-
 test/T590-libconfig.sh | 35 +++
 3 files changed, 89 insertions(+), 13 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 02444e09..521949cd 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -800,6 +800,23 @@ notmuch_database_open (const char *path,
 return status;
 }
 
+static char *
+_xdg_database_path (void *ctx) {
+
+const char *data_dir = NULL;
+
+data_dir = getenv ("XDG_DATA_HOME");
+
+if (! data_dir) {
+   const char *home = getenv ("HOME");
+   if (! home)
+   return NULL;
+
+   data_dir = talloc_asprintf (ctx, "%s/.local/share", home);
+}
+return talloc_asprintf (ctx, "%s/notmuch", data_dir);
+}
+
 notmuch_status_t
 notmuch_database_open_verbose (const char *path,
   notmuch_database_mode_t mode,
@@ -810,6 +827,7 @@ notmuch_database_open_verbose (const char *path,
 void *local = talloc_new (NULL);
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path, *xapian_path, *incompat_features;
+char *xdg_path = NULL;
 char *message = NULL;
 struct stat st;
 int err;
@@ -817,21 +835,29 @@ notmuch_database_open_verbose (const char *path,
 static int initialized = 0;
 
 if (path == NULL) {
-   message = strdup ("Error: Cannot open a database for a NULL path.\n");
-   status = NOTMUCH_STATUS_NULL_POINTER;
-   goto DONE;
+   xdg_path = _xdg_database_path (local);
+   if (! xdg_path) {
+   message = strdup ("Error: NULL path, and cannot compute 
XDG_DATA_HOME.\n");
+   status = NOTMUCH_STATUS_NULL_POINTER;
+   goto DONE;
+   }
 }
 
-if (path[0] != '/') {
+if (path && path[0] != '/') {
message = strdup ("Error: Database path must be absolute.\n");
status = NOTMUCH_STATUS_PATH_ERROR;
goto DONE;
 }
 
-if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch"))) 
{
-   message = strdup ("Out of memory\n");
-   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
-   goto DONE;
+if (xdg_path) {
+   notmuch_path = xdg_path;
+} else {
+   notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch");
+   if (! (notmuch_path)) {
+   message = strdup ("Out of memory\n");
+   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
+   goto DONE;
+   }
 }
 
 err = stat (notmuch_path, );
@@ -862,9 +888,12 @@ notmuch_database_open_verbose (const char *path,
 notmuch = talloc_zero (NULL, notmuch_database_t);
 notmuch->exception_reported = false;
 notmuch->status_string = NULL;
-notmuch->path = talloc_strdup (notmuch, path);
-
-strip_trailing(notmuch->path, '/');
+if (path) {
+   notmuch->path = talloc_strdup (notmuch, path);
+strip_trailing(notmuch->path, '/');
+} else {
+   notmuch->path = NULL;
+}
 
 notmuch->mode = mode;
 notmuch->atomic_nesting = 0;
@@ -1254,7 +1283,19 @@ notmuch_database_destroy (notmuch_database_t *notmuch)
 const char *
 notmuch_database_get_path (notmuch_database_t *notmuch)
 {
-return notmuch->path;
+char *path = NULL;
+notmuch_status_t status;
+
+if (notmuch->path)
+   return notmuch->path;
+
+status = notmuch_database_get_config (notmuch, "maildir_root", );
+if (status) {
+   _notmuch_database_log (notmuch, "unable to find maildir_root\n");
+   return NULL;
+}
+
+return path;
 }
 
 unsigned int
diff --git a/test/T560-lib-error.sh b/test/T560-lib-error.sh
index 06a6b860..115a0a35 100755
--- a/test/T560-lib-error.sh
+++ b/test/T560-lib-error.sh
@@ -22,7 +22,7 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 == stderr ==
-Error: Cannot open a database for a NULL path.
+Error opening database at CWD/home/.local/share/notmuch: No such file or 
directory
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 46f3a76d..eaaec43f 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -132,4 +132,39 @@ notmuch restore --include=config OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+XDG_DIR=$HOME/.local/share/notmuch
+test_begin_subtest "Split database and maildir"
+xapian-metadata set ${MAIL_DIR}/.notmuch/xapian Cmaildir_root ${MAIL_DIR}
+mkdir -p $XDG_DIR
+mv ${MAIL_DIR}/.notmuch/xapian $XDG_DIR
+test_C 

Re: [PATCH v3] doc: unify definition list usage across man pages

2017-12-30 Thread Jani Nikula
On Sat, 30 Dec 2017, David Bremner  wrote:
> Jani Nikula  writes:
>
>> Make all parameter descriptions etc. use reStructuredText definition
>> lists with uniform style and indentation. Remove redundant indentation
>> from around the lists. Remove blank lines between term lines and
>> definition blocks. Use four spaces for indentation.
>>
>> This is almost completely whitespace and paragraph reflow changes.
>
> My inclination is to apply this to the master branch, but not to include
> it in the 0.26 release. Any comments?

Make it so.

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH v3] doc: unify definition list usage across man pages

2017-12-30 Thread David Bremner
Jani Nikula  writes:

> Make all parameter descriptions etc. use reStructuredText definition
> lists with uniform style and indentation. Remove redundant indentation
> from around the lists. Remove blank lines between term lines and
> definition blocks. Use four spaces for indentation.
>
> This is almost completely whitespace and paragraph reflow changes.

My inclination is to apply this to the master branch, but not to include
it in the 0.26 release. Any comments?

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH v3] doc: unify definition list usage across man pages

2017-12-30 Thread Jani Nikula
Make all parameter descriptions etc. use reStructuredText definition
lists with uniform style and indentation. Remove redundant indentation
from around the lists. Remove blank lines between term lines and
definition blocks. Use four spaces for indentation.

This is almost completely whitespace and paragraph reflow changes.
---
 doc/man1/notmuch-address.rst| 170 ++-
 doc/man1/notmuch-compact.rst|  16 +-
 doc/man1/notmuch-config.rst | 354 +++-
 doc/man1/notmuch-count.rst  |  65 
 doc/man1/notmuch-dump.rst   | 116 ++---
 doc/man1/notmuch-emacs-mua.rst  |  64 
 doc/man1/notmuch-insert.rst |  82 +-
 doc/man1/notmuch-new.rst|  44 +++--
 doc/man1/notmuch-reindex.rst|  45 +++--
 doc/man1/notmuch-reply.rst  | 102 ++--
 doc/man1/notmuch-restore.rst| 118 +++---
 doc/man1/notmuch-search.rst | 220 -
 doc/man1/notmuch-show.rst   | 264 +++---
 doc/man1/notmuch-tag.rst|  33 ++--
 doc/man1/notmuch.rst|  33 ++--
 doc/man5/notmuch-hooks.rst  |  55 +++
 doc/man7/notmuch-properties.rst |   1 -
 17 files changed, 865 insertions(+), 917 deletions(-)

diff --git a/doc/man1/notmuch-address.rst b/doc/man1/notmuch-address.rst
index 68415d13c5b6..c00d7d743e3e 100644
--- a/doc/man1/notmuch-address.rst
+++ b/doc/man1/notmuch-address.rst
@@ -18,93 +18,89 @@ See **notmuch-search-terms(7)** for details of the 
supported syntax for
 
 Supported options for **address** include
 
-``--format=``\ (**json**\ \|\ **sexp**\ \|\ **text**\ \|\ **text0**)
-Presents the results in either JSON, S-Expressions, newline
-character separated plain-text (default), or null character
-separated plain-text (compatible with **xargs(1)** -0 option
-where available).
-
-``--format-version=N``
-Use the specified structured output format version. This is
-intended for programs that invoke **notmuch(1)** internally. If
-omitted, the latest supported version will be used.
-
-``--output=(sender|recipients|count|address)``
-
-Controls which information appears in the output. This option
-can be given multiple times to combine different outputs.
-When neither --output=sender nor --output=recipients is
-given, --output=sender is implied.
-
-**sender**
-Output all addresses from the *From* header.
-
-Note: Searching for **sender** should be much faster than
-searching for **recipients**, because sender addresses are
-cached directly in the database whereas other addresses
-need to be fetched from message files.
-
-**recipients**
-Output all addresses from the *To*, *Cc* and *Bcc*
-headers.
-
-**count**
-Print the count of how many times was the address
-encountered during search.
-
-Note: With this option, addresses are printed only after
-the whole search is finished. This may take long time.
-
-**address**
-Output only the email addresses instead of the full
-mailboxes with names and email addresses. This option has
-no effect on the JSON or S-Expression output formats.
-
-``--deduplicate=(no|mailbox|address)``
-
-Control the deduplication of results.
-
-**no**
-Output all occurrences of addresses in the matching
-messages. This is not applicable with --output=count.
-
-**mailbox**
-Deduplicate addresses based on the full, case sensitive
-name and email address, or mailbox. This is effectively
-the same as piping the --deduplicate=no output to **sort |
-uniq**, except for the order of results. This is the
-default.
-
-**address**
-Deduplicate addresses based on the case insensitive
-address part of the mailbox. Of all the variants (with
-different name or case), print the one occurring most
-frequently among the matching messages. If --output=count
-is specified, include all variants in the count.
-
-``--sort=``\ (**newest-first**\ \|\ **oldest-first**)
-This option can be used to present results in either
-chronological order (**oldest-first**) or reverse chronological
-order (**newest-first**).
-
-By default, results will be displayed in reverse chronological
-order, (that is, the newest results will be displayed first).
-
-However, if either --output=count or --deduplicate=address is
-specified, this option is ignored and the order of the results
-is unspecified.
-
-``--exclude=(true|false)``
-A message is called "excluded" if it matches at least one tag in
-search.tag\_exclude that 

[PATCH] NEWS: Python bindings changes for 0.26

2017-12-30 Thread David Bremner
These are my own changes, plus those of Florian Klink and Lucas
Hoffmann. Gaute's change fixed a bug that was never in a released
version, so I left it out.
---
 NEWS | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/NEWS b/NEWS
index 3049fc78..fe426bad 100644
--- a/NEWS
+++ b/NEWS
@@ -73,6 +73,20 @@ Change of return value of `notmuch_thread_get_authors`
   returned NULL.  This has been replaced by an empty string, since the
   possibility of NULL was not documented.
 
+Python Bindings
+---
+
+Python bindings specific Debian packaging is removed
+
+  The bindings have been build by the top level Debian packaging for a
+  long time, and `bindings/python/debian` has bit-rotted.
+
+Open mail files in binary mode when using Python 3
+
+  This avoids certain encoding related crashes under Python 3.
+
+Add python bindings for notmuch_database_{get,set}_config*
+
 nmbug
 -
 
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] NEWS: CLI changes for 0.26

2017-12-30 Thread David Bremner
As with the lib changes, these are based on the commits with me as author.
---
 NEWS | 11 +++
 1 file changed, 11 insertions(+)

diff --git a/NEWS b/NEWS
index 1c5edf4c..3049fc78 100644
--- a/NEWS
+++ b/NEWS
@@ -24,6 +24,17 @@ Support for re-indexing existing messages
   archive, the recorded Subject: of may change upon reindexing,
   depending on the order in which the variants are indexed.
 
+Improved error reporting in notmuch new
+
+  Give more details when reporting certain Xapian exceptions.
+
+Support maildir synced tags in `new.tags`
+
+  Tags `draft`, `flagged`, `passed`, and `replied` are now supported
+  in `new.tags`. The tag `unread` is still special in the presence of
+  maildir syncing, and will be added for files in `new/` regardless of
+  the setting of `new.tags`.
+
 Encrypted Mail
 --
 
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] NEWS: library changes for 0.26

2017-12-30 Thread David Bremner
These are just "my" changes, as arbited by "git shortlog", which
sometimes lies.
---
 NEWS | 21 +
 1 file changed, 21 insertions(+)

diff --git a/NEWS b/NEWS
index 3b6404e7..1c5edf4c 100644
--- a/NEWS
+++ b/NEWS
@@ -41,6 +41,27 @@ Indexing cleartext of encrypted e-mails
   that the notmuch index itself is adequately protected.  DO NOT USE
   this feature without considering the security of your index.
 
+Library Changes
+---
+
+Indexing files with duplicate message-id
+
+  Files with message-id's are now indexed, and searchable via terms
+  and phrases. There are known issues related to presentation of
+  results and regular-expression search, but in principle no mail file
+  should be completely unsearchable now.
+
+New functions to count files
+
+  Two new functions in the libnotmuch API:
+  `notmuch_message_count_files`, and `notmuch_thread_get_total_files`.
+
+Change of return value of `notmuch_thread_get_authors`
+
+  In certain corner cases, `notmuch_thread_get_authors` previously
+  returned NULL.  This has been replaced by an empty string, since the
+  possibility of NULL was not documented.
+
 nmbug
 -
 
-- 
2.15.1

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH v4 3/3] cli/reply: make --decrypt take a keyword

2017-12-30 Thread David Bremner
Daniel Kahn Gillmor  writes:

> It sounds like you were suggesting "--decrypt=nostash" as a synonym for
> "--decrypt=true" on show/reply, which i confess i didn't fully
> understand when i wrote my response.  If it really makes you feel better
> to add the alias/synonym, i wouldn't block such a change.

Yes, that's what I was suggesting.

> But I think the documentation is tricky to write (and trickier to read
> and trickier still to understand!)  if "--decrypt=nostash" means the
> same thing as "--decrypt=true" in one context, but they mean different
> things in a different context.

I need more time to think about this, so I'd rather defer till after the
release in any case. But at some point we collectively (I think? Maybe
Jamie and I browbeat you into it) decided that it was OK for
--decrypt=true to have context dependent behaviour. It seems to me that
"different things in a different context" issue already exists.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch