Re: [PATCH] configure: replace $(realpath emacs) with $(cd emacs && pwd -P)

2020-10-21 Thread David Bremner
Tomi Ollila  writes:

> For portability; the realpath command (e.g. from GNU coreutils)
> is not so common outside Linux systems.
>
> The "$(cd emacs && pwd -P)" replaces that realpath(1) execution
> suitably in this context (using just bash(1) builtins).

Applied to master. It seems not to make things worse, although my
testing of the OOT build was hampered by the failure of the new python
bindings to build OOT [1].


[1]: id:87h7qnubsw@tethera.net
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


OOT bindings and python-cffi

2020-10-21 Thread David Bremner


I noticed, while testing something unrelated, that the version copying
we set up for bindings/python-cffi does not work (does not find the file
'version') when doing an out of tree build.

I'm sending this to remind us to fix that at some point.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs docs: rstdoc.el: consistent single quote conversions

2020-10-21 Thread David Bremner
David Bremner  writes:

> David Bremner  writes:
>
>> Tomi Ollila  writes:
>>
>>> With text-quoting-style 'grave keeps "'" and "`" quotes unaltered
>>> for further processing done by this code (regardless of locale...).
>>> The tools that read the reStructuredText markup generated can do
>>> their styling instead.
>>
>> Not a direct comment on your patch, but should we be setting some fixed
>> locale when running emacs at build time?
>
> Now that I try, this doesn't apply to master.

As you surmised on IRC, it was a line ending problem, that I solved with
my previously developed script. I guess I should include said script in
our devel/ collection. Or someone(TM) could fix mailman.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs docs: rstdoc.el: consistent single quote conversions

2020-10-17 Thread David Bremner
David Bremner  writes:

> Tomi Ollila  writes:
>
>> With text-quoting-style 'grave keeps "'" and "`" quotes unaltered
>> for further processing done by this code (regardless of locale...).
>> The tools that read the reStructuredText markup generated can do
>> their styling instead.
>
> Not a direct comment on your patch, but should we be setting some fixed
> locale when running emacs at build time?

Now that I try, this doesn't apply to master.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: Dependencies should include "realpath"

2020-10-17 Thread David Bremner
Ralph Seichter  writes:

> * David Bremner:
>
>> Do you have a suggested replacement? I guess some inline perl with "use
>> Cwd 'realpath'" would probably work, although I haven't tested it.
>
> At a quick glance, that particular section of "configure" is run by
> doc/conf.py to generate three lines of Python code and store the result
> as sphinx.config, correct? If so, my preferred choice would be to use
> Python to figure out the absolute path, e.g. like so:
>
>   rsti_dir = os.path.abspath('emacs')
>
> This shows the generated result, and I assume that emacs is a directory
> in the source tree? I also wonder if an absolute directory path is really
> required for the doc-build to work.

Someone (TM) would have to check that this version did not break
out-of-tree builds. Other than that it seems plausible.

> The segment of conf.py which uses the generated config file does not
> look convincing to me anyway. Apparently the original author did not
> like it either, which is why the segment is labelled as "hacky". It
> should probably be overhauled, and not only because it uses the
> statement open(rsti_dir+'/'+file) which will potentially fail, depending
> on the build platform.

It's somewhat orthogonal to this discussion, but I don't object to a
patch replacing the hardcoded '/' with the appropriate python path
manipulation code. I think windows compatibility for notmuch has much
bigger issues, but it doesn't hurt to clean up the code.

Other than that, I'm not sure what an "overhaul" would involve.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: Dependencies should include "realpath"

2020-10-16 Thread David Bremner
Ralph Seichter  writes:

> Since this issue just came up related to my MacPorts port: The list of
> dependencies for notmuch should include "realpath". configure [1] relies
> on realpath to generate the value for rsti_dir.
>

Do you have a suggested replacement?  I guess some inline perl with "use
Cwd 'realpath'" would probably work, although I haven't tested it.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs docs: rstdoc.el: consistent single quote conversions

2020-10-09 Thread David Bremner
Tomi Ollila  writes:

> With text-quoting-style 'grave keeps "'" and "`" quotes unaltered
> for further processing done by this code (regardless of locale...).
> The tools that read the reStructuredText markup generated can do
> their styling instead.

Not a direct comment on your patch, but should we be setting some fixed
locale when running emacs at build time?

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: muchsync segfault

2020-09-28 Thread David Bremner
Brian May  writes:
>
> Anything else I should try?
>

Generic advice would be to run in gdb and get a backtrace, assuming you
have debugging symbols.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: emacs: How to tab-complete destination email addresses?

2020-09-24 Thread David Bremner
George Kadianakis  writes:

>
> So I guess this ordering should happen internally in notmuch-address,
> right? Perhaps as a new type of "--sort" option like "--most-frequent"
> or "--best-fit".
>
> If this is the right way to do it, perhaps I'll take a stab at it over
> the next days. If it's not the right way to do it, please let me know so
> that I don't do useless things! :)
>

there was some discussion about upstreaming some of the features of
notmuch-addrlookup-c into notmuch-address [1]. I'm not sure what
happened there, but maybe this helps.

>
> Hmm, I've never used this interface but if you are talking about the
> "--output" switch I see that they can be combined. So like you can do:
>
>$ notmuch address --output sender --output recipients jameson 
>
> to combine both To: and From:.

Maybe the elisp 'internal' front-end needs to be updated to (optionally) do what
Jamie asks.

[1]: https://github.com/aperezdc/notmuch-addrlookup-c/issues/23
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs: Remove notmuch-mua-message-send-hook

2020-09-19 Thread David Bremner
Tomi Ollila  writes:

> On Sat, Sep 12 2020, Tim Quelch wrote:
>
>> Currently `message-send-hook` functions are being called twice: In
>> notmuch send common when `notmuch-mua-send-hook` functions are
>> run (which by default includes `notmuch-mua-message-send-hook`) and in
>> `message-send` itself.
>>
>> Because `message-send-hook` functions are run in `message-send` itself,
>> we don't need also need to run them before we delegate to `message-send`
>>
>> Calling `notmuch-mua-message-send-hook` resulted in functions in
>> `message-send-hook` to be called twice. This causes bugs in
>> non-idempotent hook functions.
>
> Looks good to me. everyone who rewrote notmuch-mua-message-send-hook
> (i hope none) have to find better solution (or just...)
>

applied to master

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: emacs: error decrypting s/mime

2020-09-12 Thread David Bremner
Alexander Adolf  writes:

> Dear Notmuch Developers,
>
> I have trouble getting going with S/MIME.
>
> Emacs version: GNU Emacs 27.1 (build 1, x86_64-apple-darwin18.7.0, NS
> appkit-1671.60 Version 10.14.6 (Build 18G95)) of 2020-08-12
>
> notmuch version: notmuch 0.31+2~gd127b16
> (brew install notmuch --head)

I'm a bit at a loss how to investigate this, not using SMIME or macOS
myself. Could you try duplicating the problem with the notmuch test key
/ certificate? I've attached the test key and the ca from from the
notmuch source.

If you have the same behaviour with this key, then please share a
message encrypted to this key that manifests the problem for you
(probably as an attachement)



test.crt
Description: application/x509-ca-cert


ca.crt
Description: application/x509-ca-cert


d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs/tree: declare notmuch-search functions

2020-09-10 Thread David Bremner
"Danilo G. Baio"  writes:

>
> That helps, thanks!
>
> In fact, the notmuch-emacs port on FreeBSD is not executing the
> './configure'. [1]

I guess this issue is exactly the sort of change in the behaviour of
configure that might bite you in that case. It might be worth scanning
the upstream git log for configure if you're going do this.

> I'll add sphinx.config for now and take a look on these dependencies
> for building it in the proper way.

There is also sh.config generated by configure. It is currently only
used by the test suite, which I'm guessing you are not running.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] emacs/tree: declare notmuch-search functions

2020-09-09 Thread David Bremner
"Danilo G. Baio"  writes:
> Hi William.
>
> I can confirm this patch fixes that issue on
> `emacs/notmuch-tree.el:1143` I'm seeing here. Thank you!
>
>
> Now I have other issue, on the docs:
>
> ```
> gmake -C doc/_build/texinfo info
> [...]
> mkdir -m0755 -p 
> "/wrkdirs/usr/ports/mail/notmuch-emacs/work-full/stage/usr/local/share/info"
> install -m0644 doc/_build/texinfo/notmuch-address.info 
> doc/_build/texinfo/notmuch-compact.info 
> doc/_build/texinfo/notmuch-config.info doc/_build/texinfo/notmuch-count.info 
> doc/_build/texinfo/notmuch-dump.info 
> doc/_build/texinfo/notmuch-emacs-mua.info 
> doc/_build/texinfo/notmuch-insert.info doc/_build/texinfo/notmuch-new.info 
> doc/_build/texinfo/notmuch-reindex.info doc/_build/texinfo/notmuch-reply.info 
> doc/_build/texinfo/notmuch-restore.info 
> doc/_build/texinfo/notmuch-search.info doc/_build/texinfo/notmuch-show.info 
> doc/_build/texinfo/notmuch-tag.info doc/_build/texinfo/notmuch.info 
> doc/_build/texinfo/notmuch-hooks.info 
> doc/_build/texinfo/notmuch-properties.info 
> doc/_build/texinfo/notmuch-search-terms.info 
> doc/_build/texinfo/notmuch-emacs.info 
> /wrkdirs/usr/ports/mail/notmuch-emacs/work-full/stage/usr/local/share/info
> install: doc/_build/texinfo/notmuch-emacs.info: No such file or directory
> gmake[1]: *** [doc/Makefile.local:135: install-info] Error 71
> gmake[1]: Leaving directory 
> '/wrkdirs/usr/ports/mail/notmuch-emacs/work-full/notmuch-0.31'
> *** Error code 2
> ```

I guess I'm curious what is in the [...]. There should be a call to
sphinx-build that produces the info files. Whether emacs.info is
produced is controlled by the (generated) file sphinx.config, which
e.g. on my Debian machine looks like

,
| # Generated by configure, run from doc/conf.py
| tags.add('WITH_EMACS')
| tags.add('WITH_PYTHON')
| rsti_dir = '/home/bremner/software/upstream/notmuch/emacs'
`
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


test failures on ppc64

2020-09-08 Thread David Bremner
Dan Čermák  writes:
>
> I was wrong, "get total messages with closed database" from
> T568-lib-thread still fails sometimes on ppc64, but unfortunately not
> consistently. It appears to mostly fail on a worker with relatively
> little memory (3584 MB vs 8192 MB or 11264MB). I'll try to keep an eye
> out on that hoping to find some indicator what could be wrong.
>

If you have an interactive ppc64 machine (I'm guessing not, but someone
might), it would be interesting to run something like

cd test && count=0 && while ./T568-lib-thread.sh; do
   count=$((count + 1));
   echo $count
done

And see if the test is just flaky, and fails after some number of
repetitions.

FWIW, it passes 150 times on amd64.

I guess another thing to try would be serialize the test suite on the
low memory machines (set NOTMUCH_TEST_SERIALIZE to some non-empty value,
or remove 'parallel' from the path

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: Last call for NEWS entries for 0.31

2020-09-07 Thread David Bremner
Dan Čermák  writes:

> David Bremner  writes:
>
>> Dan Čermák  writes:
>>
>>> - on ppc64 "comparing existing to exported symbols" from
>>>   T360-symbol-hiding fails as well
>>
>> This is more-or-less known [1]. The problem is that ppc64 reports
>> defined functions as 'D' in nm output. At least on ELF architectures,
>> readelf -Ws is more reliable. How many people would using readelf break
>> things for? Currently the test does not run on macOS, afaik.
>
> So it is ok if I just skip this test on ppc64? Or can anything be done
> about this (I must admit that I have relatively little insight into
> this)?

I'd say it's OK to skip it for now. Hopefully we will fix it for the
next release; I didn't want to make the switch to readelf in the late
stages of a release.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] News: add entry for unthreaded mode

2020-09-06 Thread David Bremner
Mark Walters  writes:

> A belated NEWS entry for the new un-threaded mode introduced in Notmuch
> 0.30.
> ---

included with the 0.31 release. thanks.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Notmuch 0.31 is now available

2020-09-05 Thread David Bremner

Where to obtain notmuch 0.31
===
  https://notmuchmail.org/releases/notmuch-0.31.tar.xz

Which can be verified with:

  https://notmuchmail.org/releases/notmuch-0.31.tar.xz.sha256.asc
  -BEGIN PGP SIGNED MESSAGE-
Hash: SHA256

571fa0e1539c86612b1f2b2c80a398e08ecfef52e27ef7e48cf8e3b84fa18394  
notmuch-0.31.tar.xz
-BEGIN PGP SIGNATURE-

iQIzBAEBCAAdFiEEkiyHYXwaY0SiY6fqA0U5G1WqFSEFAl9UMs4ACgkQA0U5G1Wq
FSGwTw//Z1uzrNFgOnYYv68M5/ghrhcfXENfKhbtXIv/UyphZ7YWxnUYbNCDV56Y
PS7X8Q8i5kcvmXD4QQ+ln0jvKHfoqaiaUExKkQXYkpQFqaSS635s9q8OYgB/mwIF
Chbe3BJXCimYXSCjbmQye+h/P6yAh5WALs5yPeVsmpfIcXsbvvTD8Nj0q08flwe8
qxjpNHNyWcvZ5tauA63ZC9Ts8SWTt3SoIerinD5jyZedu5yPbLv0jLXJnjcKZCGO
LwOpwjJISOTBO5qYHijrgk5UZ2UqCmzsrAJTfRG2wxaW+3bV7IlEnW7YQB2s3eb9
zgq0q85RxhK0CLDr07JYrWH1e54Ilr4UaBRdcm5VvxDWH3lvZZ5RnqnfEXxx1M/t
3WH81WOh4P72YHZg7dk1GELldlzgRTSHhuYH3BZT4s+fXA1T7N+hLFgsnL/VExXz
51r/AnPly4fBoxo3m/n00IwHnI3D6hOr4ulR+lEj4ATw5bDgh3QSdF7dYhaUTbHH
kdc7papJ1Z4FWgLMWv2z4Mx/SVhj02IdZHpnEc802bOvaMlLlYkftTHuz/8EDFc/
3rT6DiI3VqCioGuDRVeb1eHOReEGgvHIBOyoYlWYiypVwvOfuh9hJfoIOPJitO7G
C0Kgrb4hmYuO+7Bd3M0tNhX9jGDcaHfFNhizjFGmpEopCkx0QUk=
=Fb0V
-END PGP SIGNATURE-

  https://notmuchmail.org/releases/notmuch-0.31.tar.xz.asc
  (signed by David Bremner)

What's new in notmuch 0.31
=

Emacs
-

Notmuch now supports Emacs 27.1. You may need to set
`mml-secure-openpgp-sign-with-sender` and/or
`mml-secure-smime-sign-with-sender` to continue signing messages.

The minimum supported major version of GNU Emacs is now 25.1.

Add support for moving between threads after notmuch-tree-from-search-thread.

New `notmuch-unthreaded` mode (added in Notmuch 0.30)

  Unthreaded view is a mode where each matching message is shown on a
  separate line.

  The main key entries to unthreaded view are

  'u' enter a query to view in unthreaded mode (works in hello,
  search, show and tree mode)

  'U' view the current query in unthreaded mode (works from search,
  show and tree)

  Saved searches can also specify that they should open in unthreaded
  view.

  Currently it is not possible to specify the sort order: it will
  always be newest first.

Notmuch-Mutt


The shell pipeline executed by notmuch-mutt, which symlinked matched
files to a maildir for mutt to access is replaced with internal perl
processing. This search operation is now more portable, and somewhat
faster.

Library
---

Improve exception handling in the library. This should
largely eliminate terminations inside the library due to uncaught
exceptions or internal errors.  No doubt there are a few uncovered
code paths still; please report them as bugs.

Add `notmuch_message_get_flag_st` and
`notmuch_message_has_maildir_flag_st`, and deprecate the existing
non-status providing versions.

Move memory de-allocation from `notmuch_database_close` to
`notmuch_database_destroy`.

Handle relative filenames in `notmuch_database_index_file`, as
promised in the documentation.

Python Bindings
---

Documentation for the python bindings is merged into the main
sphinx-doc documentation tree. The merged documentation can be built
with e.g. `make sphinx-html`

Dependencies


We now support building notmuch against Xapian 1.5 (the current
development version).

Test Suite
--

Test suite fixes for compatibility with Emacs 27.1.

Build System


Man pages are now compressed reproducibly.

What is notmuch
===
Notmuch is a system for indexing, searching, reading, and tagging
large collections of email messages in maildir or mh format. It uses
the Xapian library to provide fast, full-text search with a convenient
search syntax.

For more about notmuch, see https://notmuchmail.org



signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] test: fix syntax errors in erroring calls to notmuch insert

2020-09-04 Thread David Bremner
David Bremner  writes:

> notmuch insert does not currently support passing a filename for the
> input, so all of these tests have an extra error in addition to the
> one being tested for.
>

applied to master

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: call for NEWS entries for 0.31

2020-09-04 Thread David Bremner
Mark Walters  writes:

> Hi
>
>> Here's a lightly edited output from git shortlog to jog your collective
>> memories. Please send patches for NEWS for any non-trivial user visible
>> changes.
>
> I think I should have sent a news patch for the "unthreaded mode"
> addition which went into the 0.30 release (sorry about that). What is
> the best way to add it now?
>

Maybe add it to the 0.31 NEWS with a note that the change actually
happened in 0.30. I suspect people are more likely to find it if it's in
the most recent release NEWS.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] test: fix syntax errors in erroring calls to notmuch insert

2020-09-03 Thread David Bremner
notmuch insert does not currently support passing a filename for the
input, so all of these tests have an extra error in addition to the
one being tested for.

Currently this does not make a difference because the error being
tested for is caught before the error of an extra command line
argument. In the future it might make a difference, and in any case it
is confusing.
---
 test/T070-insert.sh | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/test/T070-insert.sh b/test/T070-insert.sh
index c8161e1e..1c7ca846 100755
--- a/test/T070-insert.sh
+++ b/test/T070-insert.sh
@@ -222,24 +222,24 @@ test_expect_equal "$output" "2"
 
 test_begin_subtest "Insert message, create invalid subfolder"
 gen_insert_msg
-test_expect_code 1 "notmuch insert --folder=../G --create-folder 
$gen_msg_filename"
+test_expect_code 1 "notmuch insert --folder=../G --create-folder < 
$gen_msg_filename"
 
 OLDCONFIG=$(notmuch config get new.tags)
 
 test_begin_subtest "Empty tags in new.tags are forbidden"
 notmuch config set new.tags "foo;;bar"
 gen_insert_msg
-output=$(notmuch insert $gen_msg_filename 2>&1)
+output=$(notmuch insert < $gen_msg_filename 2>&1)
 test_expect_equal "$output" "Error: tag '' in new.tags: empty tag forbidden"
 
 test_begin_subtest "Tags starting with '-' in new.tags are forbidden"
 notmuch config set new.tags "-foo;bar"
 gen_insert_msg
-output=$(notmuch insert $gen_msg_filename 2>&1)
+output=$(notmuch insert < $gen_msg_filename 2>&1)
 test_expect_equal "$output" "Error: tag '-foo' in new.tags: tag starting with 
'-' forbidden"
 
 test_begin_subtest "Invalid tags set exit code"
-test_expect_code 1 "notmuch insert $gen_msg_filename 2>&1"
+test_expect_code 1 "notmuch insert < $gen_msg_filename 2>&1"
 
 notmuch config set new.tags $OLDCONFIG
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: Last call for NEWS entries for 0.31

2020-09-01 Thread David Bremner
Dan Čermák  writes:

> Hi David,
>
> David Bremner  writes:
>
>> David Bremner  writes:
>>
>>> David Bremner  writes:
>>>
>>> I've sent a couple of patches [1] for things that stood out to
>>> me. Please send me anything you think I missed ASAP; I'd like to release
>>> the (hopefully) final release candidate tommorow.
>>
>> I've tagged 0.31~rc2, and uploaded to notmuchmail.org and debian
>> experimental.
>
> I have started seeing consistent test failures with rc2 on ppc64 and
> ppc64le:
> - on both platforms "get total messages with closed database" from
>   T568-lib-thread fails

I don't see this failure on ppc64el on Debian [0]. Perhaps there is some
toolchain difference?

> - on ppc64 "comparing existing to exported symbols" from
>   T360-symbol-hiding fails as well

This is more-or-less known [1]. The problem is that ppc64 reports
defined functions as 'D' in nm output. At least on ELF architectures,
readelf -Ws is more reliable. How many people would using readelf break
things for? Currently the test does not run on macOS, afaik.

[0]: 
https://buildd.debian.org/status/architecture.php?a=ppc64el=experimental=buildd_ppc64el-ppc64el-osuosl-01
[1]: 
https://buildd.debian.org/status/fetch.php?pkg=notmuch=ppc64=0.31%7Erc2-1=1598797686=0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: Last call for NEWS entries for 0.31

2020-08-30 Thread David Bremner
David Bremner  writes:

> David Bremner  writes:
>
> I've sent a couple of patches [1] for things that stood out to
> me. Please send me anything you think I missed ASAP; I'd like to release
> the (hopefully) final release candidate tommorow.

I've tagged 0.31~rc2, and uploaded to notmuchmail.org and debian
experimental.

d


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH 1/3] debian: suggest elpa-mailscripts

2020-08-29 Thread David Bremner
David Bremner  writes:

> These contain some useful functions for notmuch users, mainly wrappers
> for scripts from mailscripts.

Series applied to release and master

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Last call for NEWS entries for 0.31

2020-08-29 Thread David Bremner
David Bremner  writes:

> Here's a lightly edited output from git shortlog to jog your collective
> memories. Please send patches for NEWS for any non-trivial user visible
> changes.

I've sent a couple of patches [1] for things that stood out to
me. Please send me anything you think I missed ASAP; I'd like to release
the (hopefully) final release candidate tommorow.

[1]: id:20200829122310.801682-1-da...@tethera.net, 
id:20200829122310.801682-2-da...@tethera.net
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 2/2] NEWS: mention reproducibility fix

2020-08-29 Thread David Bremner
---
 NEWS | 5 +
 1 file changed, 5 insertions(+)

diff --git a/NEWS b/NEWS
index f226dffb..aa62c391 100644
--- a/NEWS
+++ b/NEWS
@@ -56,6 +56,11 @@ Test Suite
 
 Test suite fixes for compatibility with Emacs 27.1.
 
+Build System
+
+
+Man pages are now compressed reproducibly
+
 Notmuch 0.30 (2020-07-10)
 =
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 1/2] NEWS: mention Emacs 27.1 compatibility fixes

2020-08-29 Thread David Bremner
---
 NEWS | 9 +
 1 file changed, 9 insertions(+)

diff --git a/NEWS b/NEWS
index 16a5b116..f226dffb 100644
--- a/NEWS
+++ b/NEWS
@@ -4,6 +4,10 @@ Notmuch 0.31 (UNRELEASED)
 Emacs
 -
 
+Notmuch now supports Emacs 27.1. You may need to set
+`mml-secure-openpgp-sign-with-sender` and/or
+`mml-secure-smime-sign-with-sender` to continue signing messages.
+
 The minimum supported major version of GNU Emacs is now 25.1.
 
 Add support for moving between threads after notmuch-tree-from-search-thread.
@@ -47,6 +51,11 @@ Dependencies
 We now support building notmuch against Xapian 1.5 (the current
 development version).
 
+Test Suite
+--
+
+Test suite fixes for compatibility with Emacs 27.1.
+
 Notmuch 0.30 (2020-07-10)
 =
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 3/3] debian: drop notmuch-emacs dependency package

2020-08-25 Thread David Bremner
elpa-notmuch is now present in oldstable, and we don't need to support
direct upgrades from older releases than that.
---
 debian/changelog | 4 +++-
 debian/control   | 9 -
 2 files changed, 3 insertions(+), 10 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index b2da56c5..b800e361 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -5,8 +5,10 @@ notmuch (0.31~rc2-1) experimental; urgency=medium
 #944269).
   * Bug fix: "suggest mailscripts", thanks to Sean Whitton (Closes:
 #944270).
+  * Bug fix: "please drop transitional package notmuch-emacs from
+src:notmuch", thanks to Holger Levsen (Closes: #940738).
 
- -- David Bremner   Tue, 25 Aug 2020 07:37:30 -0300
+ -- David Bremner   Tue, 25 Aug 2020 07:51:33 -0300
 
 notmuch (0.31~rc1-1) experimental; urgency=medium
 
diff --git a/debian/control b/debian/control
index 7cbe9508..75359bc4 100644
--- a/debian/control
+++ b/debian/control
@@ -148,15 +148,6 @@ Description: Ruby interface to the notmuch mail search and 
index library
  This package provides a Ruby interface to the notmuch
  functionality, directly interfacing with a shared notmuch library.
 
-Package: notmuch-emacs
-Section: oldlibs
-Architecture: all
-Depends:
- elpa-notmuch,
- ${misc:Depends},
-Description: thread-based email index, search and tagging (transitional 
package)
- This dummy package help ease transition to the new package elpa-notmuch
-
 Package: elpa-notmuch
 Architecture: all
 Depends:
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 1/3] debian: suggest elpa-mailscripts

2020-08-25 Thread David Bremner
These contain some useful functions for notmuch users, mainly wrappers
for scripts from mailscripts.
---
 debian/changelog | 8 
 debian/control   | 1 +
 2 files changed, 9 insertions(+)

diff --git a/debian/changelog b/debian/changelog
index c645dd12..41e036e7 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,11 @@
+notmuch (0.31~rc2-1) experimental; urgency=medium
+
+  * New upstream release candidate
+  * Bug fix: "suggest elpa-mailscripts", thanks to Sean Whitton (Closes:
+#944269).
+
+ -- David Bremner   Tue, 25 Aug 2020 07:36:00 -0300
+
 notmuch (0.31~rc1-1) experimental; urgency=medium
 
   * Fix buggy test in T562-lib-database
diff --git a/debian/control b/debian/control
index e2bb166f..6e5a2750 100644
--- a/debian/control
+++ b/debian/control
@@ -160,6 +160,7 @@ Architecture: all
 Depends:
  ${elpa:Depends},
  ${misc:Depends},
+Suggests: elpa-mailscripts
 Description: thread-based email index, search and tagging (emacs interface)
  Notmuch is a system for indexing, searching, reading, and tagging
  large collections of email messages in maildir or mh format. It uses
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 2/3] debian: suggest mailscripts

2020-08-25 Thread David Bremner
These contain several useful tools for the notmuch user, particularly
on Debian.
---
 debian/changelog | 4 +++-
 debian/control   | 2 ++
 2 files changed, 5 insertions(+), 1 deletion(-)

diff --git a/debian/changelog b/debian/changelog
index 41e036e7..b2da56c5 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -3,8 +3,10 @@ notmuch (0.31~rc2-1) experimental; urgency=medium
   * New upstream release candidate
   * Bug fix: "suggest elpa-mailscripts", thanks to Sean Whitton (Closes:
 #944269).
+  * Bug fix: "suggest mailscripts", thanks to Sean Whitton (Closes:
+    #944270).
 
- -- David Bremner   Tue, 25 Aug 2020 07:36:00 -0300
+ -- David Bremner   Tue, 25 Aug 2020 07:37:30 -0300
 
 notmuch (0.31~rc1-1) experimental; urgency=medium
 
diff --git a/debian/control b/debian/control
index 6e5a2750..7cbe9508 100644
--- a/debian/control
+++ b/debian/control
@@ -53,6 +53,8 @@ Recommends:
  elpa-notmuch | notmuch-vim | notmuch-mutt | alot,
  gnupg-agent,
  gpgsm,
+Suggests:
+ mailscripts
 Description: thread-based email index, search and tagging
  Notmuch is a system for indexing, searching, reading, and tagging
  large collections of email messages in maildir or mh format. It uses
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH 2/5] NEWS: mention exception handling changes

2020-08-23 Thread David Bremner
William Casarin  writes:

> subjective nit: 
>
>   Many small changes to exception handling in the library. This should
>   largely eliminate terminations due to uncaught exceptions or internal
>   errors. No doubt there are a few uncovered code paths still; please
>   report them as bugs.
>
> Otherwise series LGTM

Sure, I used your version. Applied series to release and master.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] NEWS: add news entry for tree navigation changes

2020-08-23 Thread David Bremner
William Casarin  writes:


> The elisp code has plenty of comments that document each function,
> perhaps there could be a way to generate documentation from that? Not
> sure how magit does it.

We already have the docstrings available in the notmuch-emacs manual,
but they need to be referenced in the rst source.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 5/5] NEWS: remaining user visible library changes

2020-08-23 Thread David Bremner
These could both cause / fix crashes for user code.
---
 NEWS | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/NEWS b/NEWS
index a1106130..9f649ff6 100644
--- a/NEWS
+++ b/NEWS
@@ -28,6 +28,12 @@ Add `notmuch_message_get_flag_st` and
 `notmuch_message_has_maildir_flag_st`, and deprecate the existing
 non-status providing versions.
 
+Move memory de-allocation from `notmuch_database_close` to
+`notmuch_database_destroy`.
+
+Handle relative filenames in `notmuch_database_index_file`, as
+promised in the documentation.
+
 Python Bindings
 ---
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 2/5] NEWS: mention exception handling changes

2020-08-23 Thread David Bremner
---
 NEWS | 8 
 1 file changed, 8 insertions(+)

diff --git a/NEWS b/NEWS
index c1f09b18..e6392e81 100644
--- a/NEWS
+++ b/NEWS
@@ -16,6 +16,14 @@ files to a maildir for mutt to access is replaced with 
internal perl
 processing. This search operation is now more portable, and somewhat
 faster.
 
+Library
+---
+
+Many small changes to exception handling in the library; this should
+largely eliminate terminations inside the library due to uncaught
+exceptions or internal errors.  No doubt there are a few uncovered
+code paths still; please report them as bugs.
+
 Python Bindings
 ---
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 3/5] NEWS: mention port to Xapian 1.5

2020-08-23 Thread David Bremner
---
 NEWS | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/NEWS b/NEWS
index e6392e81..ece208a3 100644
--- a/NEWS
+++ b/NEWS
@@ -31,6 +31,12 @@ Documentation for the python bindings is merged into the main
 sphinx-doc documentation tree. The merged documentation can be built
 with e.g. `make sphinx-html`
 
+Dependencies
+
+
+We now support building notmuch against Xapian 1.5 (the current
+development version).
+
 Notmuch 0.30 (2020-07-10)
 =
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 4/5] NEWS: mention new API entries

2020-08-23 Thread David Bremner
---
 NEWS | 4 
 1 file changed, 4 insertions(+)

diff --git a/NEWS b/NEWS
index ece208a3..a1106130 100644
--- a/NEWS
+++ b/NEWS
@@ -24,6 +24,10 @@ largely eliminate terminations inside the library due to 
uncaught
 exceptions or internal errors.  No doubt there are a few uncovered
 code paths still; please report them as bugs.
 
+Add `notmuch_message_get_flag_st` and
+`notmuch_message_has_maildir_flag_st`, and deprecate the existing
+non-status providing versions.
+
 Python Bindings
 ---
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 1/5] NEWS: mention merging of documentation for python bindings

2020-08-23 Thread David Bremner
---
 NEWS | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/NEWS b/NEWS
index 040df4fd..c1f09b18 100644
--- a/NEWS
+++ b/NEWS
@@ -16,6 +16,13 @@ files to a maildir for mutt to access is replaced with 
internal perl
 processing. This search operation is now more portable, and somewhat
 faster.
 
+Python Bindings
+---
+
+Documentation for the python bindings is merged into the main
+sphinx-doc documentation tree. The merged documentation can be built
+with e.g. `make sphinx-html`
+
 Notmuch 0.30 (2020-07-10)
 =
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] NEWS: add news entry for tree navigation changes

2020-08-23 Thread David Bremner
William Casarin  writes:

> Signed-off-by: William Casarin 
> ---
>  NEWS | 2 ++
>  1 file changed, 2 insertions(+)
>

Applied to master and release, with a tiny wording change.

I realized when looking at this that I had completely forgotten
notmuch-tree-from-search-thread.  I suspect some of emacs interface is
not as discoverable as we think it is, and it would be worth updating
(not for the upcoming release, but afterwards) doc/notmuch-emacs.rst
with some of the nifty features that have been added since I
optimistically wrote (eb6fb36d)

This is pretty minimal, but will hopefully inspire others to
contribute more complete documentation

___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


call for NEWS entries for 0.31

2020-08-22 Thread David Bremner

Here's a lightly edited output from git shortlog to jog your collective
memories. Please send patches for NEWS for any non-trivial user visible
changes.

David Bremner (125):
  test: add known broken test for error handling on closed database
  lib: catch error from closed db in n_m_get_message_id
  lib: migrate to post Xapian 1.3.4 compact support
  lib: migrate from Xapian ValueRangeProcessor to RangeProcessor
  test: drop upgrade from v1 tests
  lib: add regression test for n_m_get_date; clarify API
  doc: replace use of environment variables with a generated config
  doc: add new python bindings to main documentation tree.
  lib: catch exceptions in n_m_get_flag, provide n_m_get_flag_st
  lib: add notmuch_message_has_maildir_flag_st
  cli/new: replace newly deprecated n_m_has_maildir_flag
  test: add regression test for n_m_maildir_flags_to_tags
  test: add broken test for n_m_remove_all_tags
  lib: handle xapian exception in n_m_remove_all_tags
  lib: fix return value for n_m_reindex
  lib: move deallocation of memory from n_d_close to n_d_destroy
  lib/n_d_get_version: catch exceptions and clarify the API
  lib: convert relative filenames to absolute in n_d_index_file
  lib: replace use of static_cast for writable databases
  build: upload html docs as part of release process
  lib: return NULL from n_d_get_default_indexopts on error
  lib: catch exceptions in n_directory_get_child_directories
  lib: catch exceptions in n_directory_get_child_files
  lib: fix return value for n_directory_delete

Jonas Bernoulli (38):
  test: Deal with Emacs 27 switching to lexical scope by default
  test: Remove misguided emacs testing utilities
  test: Explicitly state that we want to sign with sender
  emacs: Extend face to window edge again
  emacs: Fix some function declarations
  emacs: notmuch-poll: Let the user know we are polling
  emacs: Improve doc-strings
  emacs: Autoload notmuch-jump-search only once
  emacs: Autoload notmuch-jump using an autoload cookie
  emacs: Use cl-incf where appropriate
  emacs: Remove notmuch-setq-local
  emacs: Remove notmuch-read-char-choice
  emacs: Drop old advices that were only need for Emacs 23
  emacs: Do not abuse advice to monkey patch while testing
  emacs: Use new advice mechanism do advice mm-shr


Jonas Witschel (1):
  doc: make gzipped man pages reproducible

Sean Whitton (2):
  emacs: notmuch-search: avoid wiping out buffer-local variables
  emacs: Use pop-to-buffer-same-window rather than switch-to-buffer

Teemu Likonen (3):
  Emacs: Ensure left-to-right display for message headers
  Emacs: Indent first header line only when indentation is turned on

William Casarin (3):
  emacs/tree: introduce notmuch-tree-parent-buffer variable
  emacs/tree: enable moving to next thread in search results
  emacs/tree: add notmuch-tree-archive-thread-then-next

Đoàn Trần Công Danh (3):
  configure: check for pytest with python -m pytest
  configure: drop check for default xapian backend
  T355: specify hash algorithm explicitly
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2] emacs: Use pop-to-buffer-same-window rather than switch-to-buffer

2020-08-22 Thread David Bremner
Sean Whitton  writes:

> This means that notmuch commands obey display-buffer-alist so the user
> can customize how buffers show up.
>

applied to release and master
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] AUTHORS: update for 0.31

2020-08-22 Thread David Bremner
David Bremner  writes:

> Just shuffles existing authors around, mainly due to Jonas's
> enthusiastic cleanup work.

applied to release and master

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2] Emacs: Fix notmuch-message-summary-face definition

2020-08-22 Thread David Bremner
Teemu Likonen  writes:

>
> This change fixes the face definition form to produce:
>
> ((DISPLAY
>   :extend t :background "#f0f0f0")
>  (DISPLAY
>   :extend t :background "#303030"))
>
> which follows the (DISPLAY . PLIST) format (see above).

applied to release and master

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: starting release process for 0.31

2020-08-18 Thread David Bremner
David Bremner  writes:

> David Bremner  writes:
>
>> I've tagged the first release candidate for 0.31 (0.31_rc0), and
>> uploaded signed tarballs to the usual place [1].
>>
>> We need to sort out NEWS, and possibly add one or two small patches
>> before final release of 0.31
>
> Did the same for rc1. Mainly this fixes a silly copy-paste bug in
> T590-libconfig

That should be T562-lib-database. Sigh. And I made the same mistake in
debian/changelog.

d


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] AUTHORS: update for 0.31

2020-08-18 Thread David Bremner
Just shuffles existing authors around, mainly due to Jonas's
enthusiastic cleanup work.
---
 AUTHORS | 24 
 1 file changed, 12 insertions(+), 12 deletions(-)

diff --git a/AUTHORS b/AUTHORS
index 83113f4c..6e872084 100644
--- a/AUTHORS
+++ b/AUTHORS
@@ -23,7 +23,7 @@ system libraries, compilers, and the kernel that make it all 
work
 (thanks GNU, thanks Linux). Thanks to everyone who has played a part!
 
 The following list of people have at least 15 lines of code in the
-Notmuch 0.30 release (calculated by devel/author-scan.sh).
+Notmuch 0.31 release (calculated by devel/author-scan.sh).
 
  David Bremner
  Carl Worth
@@ -42,12 +42,13 @@ Notmuch 0.30 release (calculated by devel/author-scan.sh).
  W. Trevor King
  Jameson Graef Rollins
  Felipe Contreras
+ Jonas Bernoulli
  Pieter Praet
  Peter Feigl
  Dmitry Kurochkin
  Peter Wang
- Daniel Schoepe
  Gregor Zattler
+ Daniel Schoepe
  Keith Packard
  Adam Wolfe Gordon
  Stefano Zacchiroli
@@ -55,27 +56,26 @@ Notmuch 0.30 release (calculated by devel/author-scan.sh).
  laochailan
  Ben Gamari
  Aaron Ecay
- Jesse Rosenthal
  l-...@web.de
  Thomas Jost
+ Jesse Rosenthal
  Dirk Hohndel
  Blake Jones
- Jonas Bernoulli
  Damien Cassou
- Vladimir Panteleev
  Anton Khirnov
  Matt Armstrong
+ Vladimir Panteleev
+ William Casarin
  Örjan Ekeberg
  Jan Janak
  Patrick Totzke
- Chunyang Xu
- rhn
  Ruben Pollan
+ rhn
  Ioan-Adrian Ratiu
  Ethan Glasser-Camp
+ Chunyang Xu
  Todd
  Chris Wilson
- William Casarin
  Yuri Volchkov
  Cédric Cabessa
  Mark Anderson
@@ -84,17 +84,17 @@ Notmuch 0.30 release (calculated by devel/author-scan.sh).
  Ludovic LANGE
  Sebastian Poeplau
  Mikhail
- Gaute Hope
  Keith Amidon
+ Gaute Hope
  martin f. krafft
  Jeffrey C. Ollie
- Bart Trojanowski
  Jameson Rollins
  Scott Henson
+ Bart Trojanowski
  Vladimir Marek
  Servilio Afre Puentes
- Kevin McCarthy
  Tomas Carnecky
+ Kevin McCarthy
  Kevin J. McCarthy
  Scott Robinson
  Wael M. Nasreddine
@@ -111,9 +111,9 @@ Notmuch 0.30 release (calculated by devel/author-scan.sh).
  Luis Ressel
  Sergei Shilovsky
  Trevor Jim
- Jinwoo Lee
  Uli Scholler
  Matthew Lear
+ Jinwoo Lee
  Amadeusz Żołnowski
 
 Here is an incomplete list of other people that have made
-- 
2.28.0

___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: starting release process for 0.31

2020-08-18 Thread David Bremner
David Bremner  writes:

> I've tagged the first release candidate for 0.31 (0.31_rc0), and
> uploaded signed tarballs to the usual place [1].
>
> We need to sort out NEWS, and possibly add one or two small patches
> before final release of 0.31

Did the same for rc1. Mainly this fixes a silly copy-paste bug in
T590-libconfig

d


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] devel/release-checks.sh: use grep to find copyright year.

2020-08-17 Thread David Bremner
Tomi Ollila  writes:

> On Sun, Aug 16 2020, David Bremner wrote:
>
>
> For consistency and to save one fork(2) $(exec grep ...), but I could
> live with this, too ;D
>
> So LGTM.

I guess I was a bit impatient. We can add the exec in the next cleanup
pass of configure I guess.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] test: fix uninitialized variable use in T562-lib-database

2020-08-16 Thread David Bremner
Fix a copy paste error of using the boolean ret as a notmuch_status_t,
and uninitialized.
---
 test/T562-lib-database.sh | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/test/T562-lib-database.sh b/test/T562-lib-database.sh
index d6097418..dd4f2566 100755
--- a/test/T562-lib-database.sh
+++ b/test/T562-lib-database.sh
@@ -154,11 +154,9 @@ test_expect_equal_file EXPECTED OUTPUT
 test_begin_subtest "upgrade a closed db"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 {
-notmuch_bool_t ret;
-
 EXPECT0(notmuch_database_close (db));
 stat = notmuch_database_upgrade (db, NULL, NULL);
-printf ("%d\n", ret == NOTMUCH_STATUS_SUCCESS);
+printf ("%d\n", stat == NOTMUCH_STATUS_SUCCESS);
 }
 EOF
 cat < EXPECTED
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] build: clean up sphinx.config

2020-08-16 Thread David Bremner
Follow the existing practice and remove it under "distclean", same as
sh.config and Makefile.config
---
 Makefile.local | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile.local b/Makefile.local
index 156c8ce1..c65cee7c 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -294,7 +294,7 @@ CLEAN := $(CLEAN) notmuch notmuch-shared 
$(notmuch_client_modules)
 CLEAN := $(CLEAN) version.stamp notmuch-*.tar.gz.tmp
 CLEAN := $(CLEAN) .deps
 
-DISTCLEAN := $(DISTCLEAN) .first-build-message Makefile.config sh.config
+DISTCLEAN := $(DISTCLEAN) .first-build-message Makefile.config sh.config 
sphinx.config
 
 CPPCHECK_STAMPS := $(SRCS:%=.stamps/cppcheck/%)
 .PHONY: cppcheck
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


starting release process for 0.31

2020-08-16 Thread David Bremner

I've tagged the first release candidate for 0.31 (0.31_rc0), and
uploaded signed tarballs to the usual place [1].

We need to sort out NEWS, and possibly add one or two small patches
before final release of 0.31

[1]: https://notmuchmail.org/releases/


signature.asc
Description: PGP signature
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] devel/release-checks.sh: use grep to find copyright year.

2020-08-16 Thread David Bremner
This is quite fragile, but it works for now, unlike the python
version.

In general it seems conf.py is not intended to be evaluated outside of
sphinx, as it assumes certain global names (in particular "tags") are
defined.
---

 I am going to apply this to unblock the release process, but I
 welcome better solutions.
 
 devel/release-checks.sh | 5 +
 1 file changed, 1 insertion(+), 4 deletions(-)

diff --git a/devel/release-checks.sh b/devel/release-checks.sh
index 7ba94822..cfa208d5 100755
--- a/devel/release-checks.sh
+++ b/devel/release-checks.sh
@@ -178,10 +178,7 @@ esac
 year=`exec date +%Y`
 echo -n "Checking that copyright in documentation contains 2009-$year... "
 # Read the value of variable `copyright' defined in 'doc/conf.py'.
-# As __file__ is not defined when python command is given from command line,
-# it is defined before contents of 'doc/conf.py' (which dereferences __file__)
-# is executed.
-copyrightline=`exec python -c "with open('doc/conf.py') as cf: __file__ = ''; 
exec(cf.read()); print(copyright)"`
+copyrightline=$(grep ^copyright doc/conf.py)
 case $copyrightline in
*2009-$year*)
echo Yes. ;;
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2 1/3] emacs/tree: introduce notmuch-tree-parent-buffer variable

2020-08-16 Thread David Bremner
William Casarin  writes:

> This variable will be used in a similar fashion to
> notmuch-show-parent-buffer. It will be used to navigate between
> threads from the parent search buffer.
>
> Signed-off-by: William Casarin 

v2 applied to master
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] test: update README to reflect dropping upgrade tests

2020-08-16 Thread David Bremner
David Bremner  writes:

> These test databases have been unneeded since ee897cab8.

applied to master.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH 3/3] test: Explicitly state that we want to sign with sender

2020-08-15 Thread David Bremner
Jonas Bernoulli  writes:

> Since Emacs 27 'mml-secure-epg-sign' errors out if we don't opt-in to
> signing as the sender using 'mml-secure-openpgp-sign-with-sender'.
> ---
>  test/test-lib.sh | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/test/test-lib.sh b/test/test-lib.sh
> index 8c331b88..90e26639 100644
> --- a/test/test-lib.sh
> +++ b/test/test-lib.sh
> @@ -414,7 +414,8 @@ emacs_fcc_message ()
>  (message-goto-body)
>  (insert \"${body}\")
>  $*
> -(notmuch-mua-send-and-exit))" || return 1
> +(let ((mml-secure-openpgp-sign-with-sender t))
> +  (notmuch-mua-send-and-exit)))" || return 1
>  notmuch new $nmn_args >/dev/null

It seems that notmuch-emacs users will have to set this variable, or
perform some other configuration to keep signing working when upgrading
to 27.1. Should notmuch be helping this transition somehow?

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2 2/3] emacs/tree: enable moving to next thread in search results

2020-08-15 Thread David Bremner
William Casarin  writes:

> This introduces a new function called
> notmuch-tree-next-thread-from-search which is analogous to
> notmuch-show-next-thread. It will switch to the next or previous
> thread from the parent search results.
>

This seems to work, and is pretty slick. If there are no objections
today, I'll apply to master,

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2 1/3] emacs/tree: introduce notmuch-tree-parent-buffer variable

2020-08-15 Thread David Bremner
David Bremner  writes:

> William Casarin  writes:
>
>> This variable will be used in a similar fashion to
>> notmuch-show-parent-buffer. It will be used to navigate between
>> threads from the parent search buffer.
>>
>> Signed-off-by: William Casarin 
>
> Can someone explain the difference in behaviour before and after this
> patch?

Ah, reading ahead I see the main point is to use it in the next
patch. That's fine. I guess in general a hint in the commit message is
nice, but not worth respinning the series just for that.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2 1/3] emacs/tree: introduce notmuch-tree-parent-buffer variable

2020-08-15 Thread David Bremner
William Casarin  writes:

> This variable will be used in a similar fashion to
> notmuch-show-parent-buffer. It will be used to navigate between
> threads from the parent search buffer.
>
> Signed-off-by: William Casarin 

Can someone explain the difference in behaviour before and after this
patch?

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v3] Emacs: Indent first header line only when indentation is turned on

2020-08-15 Thread David Bremner
Teemu Likonen  writes:

> Previously in message-show mode message's first header line (From
> header) was always indented, even if user had turned thread
> indentation off with "<" (notmuch-show-toggle-thread-indentation)
> command.
>
> This change modifies notmuch-show-insert-headerline function so that
> it doesn't indent the first header line if notmuch-show-indent-content
> variable is nil.
>
> This change also modifies tests so that they expect this new output
> format:
> test/emacs-show.expected-output/notmuch-show-indent-thread-content-off

Applied to master

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] test: update README to reflect dropping upgrade tests

2020-08-15 Thread David Bremner
These test databases have been unneeded since ee897cab8.
---

 You're correct that the README needs updating, but download-corpus is
 for the performance tests.

 test/README | 8 
 1 file changed, 8 deletions(-)

diff --git a/test/README b/test/README
index 3f54af58..11eaf18f 100644
--- a/test/README
+++ b/test/README
@@ -79,14 +79,6 @@ The following command-line options are available when 
running tests:
As the names depend on the tests' file names, it is safe to
run the tests with this option in parallel.
 
-Certain tests require precomputed databases to complete. You can fetch these
-databases with
-
-   make download-test-databases
-
-If you do not download the test databases, the relevant tests will be
-skipped.
-
 When invoking the test suite via "make test" any of the above options
 can be specified as follows:
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2] emacs: Use pop-to-buffer-same-window rather than switch-to-buffer

2020-08-15 Thread David Bremner
Sean Whitton  writes:

> This means that notmuch commands obey display-buffer-alist so the user
> can customize how buffers show up.
>
> It also permits the use of C-x 4 4, C-x 5 5 and C-x t t, available in
> Emacs 28.  For example, one can use C-x 4 4 M-x notmuch-jump-search RET
> to open a saved search in another window rather than the current window.
> Or in notmuch-search mode, C-x 5 5 RET to view the message at point in
> a new frame.
>
> notmuch-tree has custom buffer display logic, so bind
> display-buffer-overriding-action to make pop-to-buffer-same-window
> behave exactly as switch-to-buffer while that function is running.

I only tested this in emacs 26.3, but I think I hit all the modified
code paths and it didn't crash.  It would be nice to have one or two
more people try it out before I apply to master; I tend to be somewhat
oblivious to subtle changes in emacs window behaviour.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: Signing messages with emacs 27.1

2020-08-13 Thread David Bremner
Alan Schmitt  writes:

> Hello,
>
> I just updated to emacs 27.1, and I'm wondering if this is why I no
> longer can sign messages. I get the error "Couldn’t find any signer
> names."
>
> I tried debugging it, and it seems that the problem is that
> `mml-secure-epg-encrypt' cannot find the name of the sender
> ((message-options-get 'message-sender) returns nil).
>
> Is it a problem with notmuch, or should I looks elsewhere?

I _think_ it's a change in message-mode. Does it work for you if you set
mml-secure-openpgp-sign-with-sender?

(or smime, if using smime)

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] configure: Check if emacs >= 25 (instead of >= 24) is available

2020-08-12 Thread David Bremner
Tomi Ollila  writes:

> "The minimum supported major version of GNU Emacs is now 25.1."
>
> 25.1 is the first "released" version of Emacs 25.

applied to master.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] notmuch-mutt: replace shell pipeline with internal pipe processing

2020-08-12 Thread David Bremner
Tomi Ollila  writes:

> The shell pipeline used to symlink files based in search results
> to "cache" directory for mutt(1) to use was prone to portability
> problems (due to /bin/sh differences).
>
> The replacement executes `notmuch search` without intermediate shell
> (so shell_quote was removed in this case), reads the filenames from
> piped output and symlinks files internally.

applied to master.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2] Emacs: Indent first header line only when indentation is turned on

2020-08-12 Thread David Bremner
Teemu Likonen  writes:


> diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
> index 0eb27e33..444b2a45 100644
> --- a/emacs/notmuch-show.el
> +++ b/emacs/notmuch-show.el
> @@ -474,7 +474,10 @@ message at DEPTH in the current thread."
>;; invisible U+200E LEFT-TO-RIGHT MARK character which forces
>;; the header paragraph as left-to-right text.
>(insert (propertize (string ?\x200e) 'invisible t)))
> -(insert (notmuch-show-spaces-n (* notmuch-show-indent-messages-width 
> depth))
> +(insert (if notmuch-show-indent-content
> + (notmuch-show-spaces-n (* notmuch-show-indent-messages-width
> +   depth))
> +   "")
>   from
>   " ("
>   date

the test "notmuch-show: disable indentation of thread content (w/
notmuch-show-toggle-thread-indentation)" in T450-emacs-show needs to be
adjusted for this change (i.e. it fails as is).
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 2/6] lib: factor out feature name related code.

2020-08-12 Thread David Bremner
database.cc is uncomfortably large, and some of the static data
structures do not need to be shared as much as they are.

This is a somewhat small piece to factor out, but it will turn out to
be helpful to further refactoring.
---
 lib/Makefile.local |   3 +-
 lib/database-private.h |  14 +
 lib/database.cc| 118 +
 lib/features.cc| 114 +++
 4 files changed, 132 insertions(+), 117 deletions(-)
 create mode 100644 lib/features.cc

diff --git a/lib/Makefile.local b/lib/Makefile.local
index a6400126..04418fa8 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -59,7 +59,8 @@ libnotmuch_cxx_srcs = \
$(dir)/config.cc\
$(dir)/regexp-fields.cc \
$(dir)/thread.cc \
-   $(dir)/thread-fp.cc
+   $(dir)/thread-fp.cc \
+   $(dir)/features.cc
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 
diff --git a/lib/database-private.h b/lib/database-private.h
index 041602cd..2d220811 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -32,6 +32,8 @@
 
 #include "notmuch-private.h"
 
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
+
 #ifdef SILENCE_XAPIAN_DEPRECATION_WARNINGS
 #define XAPIAN_DEPRECATED(D) D
 #endif
@@ -263,4 +265,16 @@ _notmuch_database_find_doc_ids (notmuch_database_t 
*notmuch,
const char *value,
Xapian::PostingIterator *begin,
Xapian::PostingIterator *end);
+
+#define NOTMUCH_DATABASE_VERSION 3
+
+/* features.cc */
+
+_notmuch_features
+_notmuch_database_parse_features (const void *ctx, const char *features, 
unsigned int version,
+ char mode, char **incompat_out);
+
+char *
+_notmuch_database_print_features (const void *ctx, unsigned int features);
+
 #endif
diff --git a/lib/database.cc b/lib/database.cc
index 75189685..e08d43ca 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -39,8 +39,6 @@
 
 using namespace std;
 
-#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
-
 typedef struct {
 const char *name;
 const char *prefix;
@@ -458,41 +456,6 @@ _notmuch_database_prefix (notmuch_database_t *notmuch, 
const char *name)
 return NULL;
 }
 
-static const struct {
-/* NOTMUCH_FEATURE_* value. */
-_notmuch_features value;
-/* Feature name as it appears in the database.  This name should
- * be appropriate for displaying to the user if an older version
- * of notmuch doesn't support this feature. */
-const char *name;
-/* Compatibility flags when this feature is declared. */
-const char *flags;
-} feature_names[] = {
-{ NOTMUCH_FEATURE_FILE_TERMS,
-  "multiple paths per message", "rw" },
-{ NOTMUCH_FEATURE_DIRECTORY_DOCS,
-  "relative directory paths", "rw" },
-/* Header values are not required for reading a database because a
- * reader can just refer to the message file. */
-{ NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES,
-  "from/subject/message-ID in database", "w" },
-{ NOTMUCH_FEATURE_BOOL_FOLDER,
-  "exact folder:/path: search", "rw" },
-{ NOTMUCH_FEATURE_GHOSTS,
-  "mail documents for missing messages", "w" },
-/* Knowledge of the index mime-types are not required for reading
- * a database because a reader will just be unable to query
- * them. */
-{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
-  "indexed MIME types", "w" },
-{ NOTMUCH_FEATURE_LAST_MOD,
-  "modification tracking", "w" },
-/* Existing databases will work fine for all queries not involving
- * 'body:' */
-{ NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY,
-  "index body and headers separately", "w" },
-};
-
 const char *
 notmuch_status_to_string (notmuch_status_t status)
 {
@@ -817,83 +780,6 @@ _notmuch_database_new_revision (notmuch_database_t 
*notmuch)
 return new_revision;
 }
 
-/* Parse a database features string from the given database version.
- * Returns the feature bit set.
- *
- * For version < 3, this ignores the features string and returns a
- * hard-coded set of features.
- *
- * If there are unrecognized features that are required to open the
- * database in mode (which should be 'r' or 'w'), return a
- * comma-separated list of unrecognized but required features in
- * *incompat_out suitable for presenting to the user.  *incompat_out
- * will be allocated from ctx.
- */
-static _notmuch_features
-_parse_features (const void *ctx, const char *features, unsigned int version,
-char mode, char **incompat_out)
-{
-_notmuch_features res = static_cast<_notmuch_features>(0);
-unsigned int namelen, i;
-size_t llen = 0;
-const char *flags;
-
-/* Prior to database version 3, features were implied by the
- * version number. */
-if (version == 0)
-   return NOTMUCH_FEATURES_V0;
-else if (version == 1)
-   return 

[PATCH 6/6] lib: factor out notmuch_database_open* related code to own file

2020-08-12 Thread David Bremner
Reduce the size of database.cc, and prepare for extending the database
opening API
---
 lib/Makefile.local |   3 +-
 lib/database.cc| 219 -
 lib/open.cc| 218 
 3 files changed, 220 insertions(+), 220 deletions(-)
 create mode 100644 lib/open.cc

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 3aa9e80f..ddd169dc 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -61,7 +61,8 @@ libnotmuch_cxx_srcs = \
$(dir)/thread.cc \
$(dir)/thread-fp.cc \
$(dir)/features.cc  \
-   $(dir)/prefix.cc
+   $(dir)/prefix.cc\
+   $(dir)/open.cc
 
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/database.cc b/lib/database.cc
index defa3062..0f4e2ff9 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,10 +19,6 @@
  */
 
 #include "database-private.h"
-#include "parse-time-vrp.h"
-#include "query-fp.h"
-#include "thread-fp.h"
-#include "regexp-fields.h"
 #include "string-util.h"
 
 #include 
@@ -50,12 +46,6 @@ typedef struct {
 #define STRINGIFY(s) _SUB_STRINGIFY (s)
 #define _SUB_STRINGIFY(s) #s
 
-#if HAVE_XAPIAN_DB_RETRY_LOCK
-#define DB_ACTION (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK)
-#else
-#define DB_ACTION Xapian::DB_CREATE_OR_OPEN
-#endif
-
 #define LOG_XAPIAN_EXCEPTION(message, error) _log_xapian_exception 
(__location__, message, error)
 
 static void
@@ -594,215 +584,6 @@ _notmuch_database_new_revision (notmuch_database_t 
*notmuch)
 return new_revision;
 }
 
-notmuch_status_t
-notmuch_database_open (const char *path,
-  notmuch_database_mode_t mode,
-  notmuch_database_t **database)
-{
-char *status_string = NULL;
-notmuch_status_t status;
-
-status = notmuch_database_open_verbose (path, mode, database,
-   _string);
-
-if (status_string) {
-   fputs (status_string, stderr);
-   free (status_string);
-}
-
-return status;
-}
-
-notmuch_status_t
-notmuch_database_open_verbose (const char *path,
-  notmuch_database_mode_t mode,
-  notmuch_database_t **database,
-  char **status_string)
-{
-notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
-void *local = talloc_new (NULL);
-notmuch_database_t *notmuch = NULL;
-char *notmuch_path, *xapian_path, *incompat_features;
-char *message = NULL;
-struct stat st;
-int err;
-unsigned int i, version;
-static int initialized = 0;
-
-if (path == NULL) {
-   message = strdup ("Error: Cannot open a database for a NULL path.\n");
-   status = NOTMUCH_STATUS_NULL_POINTER;
-   goto DONE;
-}
-
-if (path[0] != '/') {
-   message = strdup ("Error: Database path must be absolute.\n");
-   status = NOTMUCH_STATUS_PATH_ERROR;
-   goto DONE;
-}
-
-if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch"))) 
{
-   message = strdup ("Out of memory\n");
-   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
-   goto DONE;
-}
-
-err = stat (notmuch_path, );
-if (err) {
-   IGNORE_RESULT (asprintf (, "Error opening database at %s: %s\n",
-notmuch_path, strerror (errno)));
-   status = NOTMUCH_STATUS_FILE_ERROR;
-   goto DONE;
-}
-
-if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, 
"xapian"))) {
-   message = strdup ("Out of memory\n");
-   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
-   goto DONE;
-}
-
-/* Initialize the GLib type system and threads */
-#if ! GLIB_CHECK_VERSION (2, 35, 1)
-g_type_init ();
-#endif
-
-/* Initialize gmime */
-if (! initialized) {
-   g_mime_init ();
-   initialized = 1;
-}
-
-notmuch = talloc_zero (NULL, notmuch_database_t);
-notmuch->exception_reported = false;
-notmuch->status_string = NULL;
-notmuch->path = talloc_strdup (notmuch, path);
-
-strip_trailing (notmuch->path, '/');
-
-notmuch->writable_xapian_db = NULL;
-notmuch->atomic_nesting = 0;
-notmuch->view = 1;
-try {
-   string last_thread_id;
-   string last_mod;
-
-   if (mode == NOTMUCH_DATABASE_MODE_READ_WRITE) {
-   notmuch->writable_xapian_db = new Xapian::WritableDatabase 
(xapian_path,
-   
DB_ACTION);
-   notmuch->xapian_db = notmuch->writable_xapian_db;
-   } else {
-   notmuch->xapian_db = new Xapian::Database (xapian_path);
-   }
-
-   /* Check version.  As of database version 3, we represent
-* changes in terms of features, so assume a version bump
-* means a dramatically incompatible change. */
-   version = notmuch_database_get_version (notmuch);
-   if (version > 

preliminaries for merged config

2020-08-12 Thread David Bremner
This collects all of the patches from 
id:20200808141653.1124111-1-da...@tethera.net that
are reasonable changes on their own, without wading into the new API design.

Two updates to tests

[PATCH 1/6] test: use keys with group 'test' in T590-libconfig
[PATCH 5/6] test: add regression test for searching with alternate

Three patches breaking up database.cc

[PATCH 2/6] lib: factor out feature name related code.
[PATCH 3/6] lib: factor out prefix related code to its own file
[PATCH 6/6] lib: factor out notmuch_database_open* related code to

One genuine (if a bit obscure) bug fix:

[PATCH 4/6] lib/config: delay setting talloc destructor

___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 3/6] lib: factor out prefix related code to its own file

2020-08-12 Thread David Bremner
Reduce the size of database.cc, and limit the scope of prefix_table,
make sure it's accessed via a well-defined internal API.
---
 lib/Makefile.local |   4 +-
 lib/database-private.h |   7 ++
 lib/database.cc| 201 ++-
 lib/prefix.cc  | 210 +
 4 files changed, 228 insertions(+), 194 deletions(-)
 create mode 100644 lib/prefix.cc

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 04418fa8..3aa9e80f 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -60,7 +60,9 @@ libnotmuch_cxx_srcs = \
$(dir)/regexp-fields.cc \
$(dir)/thread.cc \
$(dir)/thread-fp.cc \
-   $(dir)/features.cc
+   $(dir)/features.cc  \
+   $(dir)/prefix.cc
+
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 
diff --git a/lib/database-private.h b/lib/database-private.h
index 2d220811..c9bc712b 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -277,4 +277,11 @@ _notmuch_database_parse_features (const void *ctx, const 
char *features, unsigne
 char *
 _notmuch_database_print_features (const void *ctx, unsigned int features);
 
+/* prefix.cc */
+notmuch_status_t
+_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch);
+
+notmuch_status_t
+_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch);
+
 #endif
diff --git a/lib/database.cc b/lib/database.cc
index e08d43ca..defa3062 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -263,80 +263,6 @@ _notmuch_database_mode (notmuch_database_t *notmuch)
  * same thread.
  */
 
-/* With these prefix values we follow the conventions published here:
- *
- * https://xapian.org/docs/omega/termprefixes.html
- *
- * as much as makes sense. Note that I took some liberty in matching
- * the reserved prefix values to notmuch concepts, (for example, 'G'
- * is documented as "newsGroup (or similar entity - e.g. a web forum
- * name)", for which I think the thread is the closest analogue in
- * notmuch. This in spite of the fact that we will eventually be
- * storing mailing-list messages where 'G' for "mailing list name"
- * might be even a closer analogue. I'm treating the single-character
- * prefixes preferentially for core notmuch concepts (which will be
- * nearly universal to all mail messages).
- */
-
-static const
-prefix_t prefix_table[] = {
-/* nameterm prefix flags */
-{ "type",   "T",NOTMUCH_FIELD_NO_FLAGS },
-{ "reference",  "XREFERENCE",   NOTMUCH_FIELD_NO_FLAGS },
-{ "replyto","XREPLYTO", NOTMUCH_FIELD_NO_FLAGS },
-{ "directory",  "XDIRECTORY",   NOTMUCH_FIELD_NO_FLAGS },
-{ "file-direntry",  "XFDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
-{ "directory-direntry", "XDDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
-{ "body",   "", NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "thread", "G",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "tag","K",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "is", "K",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "id", "Q",NOTMUCH_FIELD_EXTERNAL },
-{ "mid","Q",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "path",   "P",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "property",   "XPROPERTY",NOTMUCH_FIELD_EXTERNAL },
-/*
- * Unconditionally add ':' to reduce potential ambiguity with
- * overlapping prefixes and/or terms that start with capital
- * letters. See Xapian document termprefixes.html for related
- * discussion.
- */
-{ "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "date",   NULL,   NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "query",  NULL,   NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "from",   "XFROM",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "to", "XTO",  NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "attachment", "XATTACHMENT",  NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "mimetype",   "XMIMETYPE",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "subject","XSUBJECT", NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC |
-  NOTMUCH_FIELD_PROCESSOR },
-};
-
-static void
-_setup_query_field_default 

[PATCH 1/6] test: use keys with group 'test' in T590-libconfig

2020-08-12 Thread David Bremner
In a future commit we want to interoperate better with glib KeyFiles,
which need groups for all keys.
---
 test/T590-libconfig.sh | 36 ++--
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 360e45b0..8c34acf9 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -28,18 +28,18 @@ EOF
 test_begin_subtest "notmuch_database_{set,get}_config"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 {
-   EXPECT0(notmuch_database_set_config (db, "testkey1", "testvalue1"));
-   EXPECT0(notmuch_database_set_config (db, "testkey2", "testvalue2"));
-   EXPECT0(notmuch_database_get_config (db, "testkey1", ));
-   printf("testkey1 = %s\n", val);
-   EXPECT0(notmuch_database_get_config (db, "testkey2", ));
-   printf("testkey2 = %s\n", val);
+   EXPECT0(notmuch_database_set_config (db, "test.key1", "testvalue1"));
+   EXPECT0(notmuch_database_set_config (db, "test.key2", "testvalue2"));
+   EXPECT0(notmuch_database_get_config (db, "test.key1", ));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", ));
+   printf("test.key2 = %s\n", val);
 }
 EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
-testkey1 = testvalue1
-testkey2 = testvalue2
+test.key1 = testvalue1
+test.key2 = testvalue2
 == stderr ==
 EOF
 test_expect_equal_file EXPECTED OUTPUT
@@ -93,8 +93,8 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 aaabefore beforeval
-testkey1 testvalue1
-testkey2 testvalue2
+test.key1 testvalue1
+test.key2 testvalue2
 zzzafter afterval
 == stderr ==
 EOF
@@ -115,8 +115,8 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 aaabefore 1
-testkey1 1
-testkey2 1
+test.key1 1
+test.key2 1
 zzzafter 1
 == stderr ==
 EOF
@@ -126,7 +126,7 @@ test_begin_subtest "notmuch_database_get_config_list: one 
prefix"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 {
notmuch_config_list_t *list;
-   EXPECT0(notmuch_database_get_config_list (db, "testkey", ));
+   EXPECT0(notmuch_database_get_config_list (db, "test.key", ));
for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next 
(list)) {
   printf("%s %s\n", notmuch_config_list_key (list), 
notmuch_config_list_value(list));
}
@@ -135,8 +135,8 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
-testkey1 testvalue1
-testkey2 testvalue2
+test.key1 testvalue1
+test.key2 testvalue2
 == stderr ==
 EOF
 test_expect_equal_file EXPECTED OUTPUT
@@ -152,8 +152,8 @@ cat <<'EOF' >EXPECTED
 #notmuch-dump batch-tag:3 config
 #@ aaabefore beforeval
 #@ key%20with%20spaces value,%20with,%20spaces%21
-#@ testkey1 testvalue1
-#@ testkey2 testvalue2
+#@ test.key1 testvalue1
+#@ test.key2 testvalue2
 #@ zzzafter afterval
 EOF
 test_expect_equal_file EXPECTED OUTPUT
@@ -162,7 +162,7 @@ test_begin_subtest "restore config"
 notmuch dump --include=config >EXPECTED
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 {
-EXPECT0(notmuch_database_set_config (db, "testkey1", "mutatedvalue"));
+EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue"));
 }
 EOF
 notmuch restore --include=config 

[PATCH 5/6] test: add regression test for searching with alternate config

2020-08-12 Thread David Bremner
Make sure upcoming changes to config handling do not break command
line specification.
---
 test/T140-excludes.sh | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh
index 0cf69975..cef07095 100755
--- a/test/T140-excludes.sh
+++ b/test/T140-excludes.sh
@@ -39,6 +39,16 @@ deleted_id=$gen_msg_id
 output=$(notmuch search subject:deleted | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; 
Not deleted (inbox unread)"
 
+test_begin_subtest "Search, exclude \"deleted\" messages; alternate config 
file"
+cp ${NOTMUCH_CONFIG} alt-config
+notmuch config set search.exclude_tags
+notmuch --config=alt-config search subject:deleted | notmuch_search_sanitize > 
OUTPUT
+cp alt-config ${NOTMUCH_CONFIG}
+cat < EXPECTED
+thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Search, exclude \"deleted\" messages from message search"
 output=$(notmuch search --output=messages subject:deleted | 
notmuch_search_sanitize)
 test_expect_equal "$output" "id:$not_deleted_id"
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 4/6] lib/config: delay setting talloc destructor

2020-08-12 Thread David Bremner
If Xapian has thrown an exception, it is not safe to invoke the
destuctor when freeing the list struct.
---
 lib/config.cc | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/lib/config.cc b/lib/config.cc
index dae0ff0e..d0497052 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -113,7 +113,6 @@ notmuch_database_get_config_list (notmuch_database_t 
*notmuch,
goto DONE;
 }
 
-talloc_set_destructor (list, _notmuch_config_list_destroy);
 list->notmuch = notmuch;
 list->current_key = NULL;
 list->current_val = NULL;
@@ -133,8 +132,15 @@ notmuch_database_get_config_list (notmuch_database_t 
*notmuch,
 *out = list;
 
   DONE:
-if (status && list)
-   talloc_free (list);
+if (status) {
+   if (list) {
+   talloc_free (list);
+   if (status != NOTMUCH_STATUS_XAPIAN_EXCEPTION)
+   _notmuch_config_list_destroy (list);
+   }
+}  else {
+   talloc_set_destructor (list, _notmuch_config_list_destroy);
+}
 
 return status;
 }
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] notmuch-mutt: replace shell pipeline with internal pipe processing

2020-08-11 Thread David Bremner
Tomi Ollila  writes:

> The shell pipeline used to symlink files based in search results
> to "cache" directory for mutt(1) to use was prone to portability
> problems (due to /bin/sh differences).
>
> The replacement executes `notmuch search` without intermediate shell
> (so shell_quote was removed in this case), reads the filenames from
> piped output and symlinks files internally.
> ---
>
> Now also tested a bit by me (remoteusage provides stale symlinks ;)

tested by the original bug reporter

   https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=966100#22

seems to work.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v3 00/34] A great number of cosmetic changes

2020-08-09 Thread David Bremner
Tomi Ollila  writes:

>
> --ignore-whitespace could help...
>
> for the --no-keep-cr (the default) not to work, in thread containing this
>
>https://lkml.org/lkml/2019/12/1/69
>
> may give some hint why the situation is as unclear as it is now...

--ignore-whitespace adds CRs to the files, which is pretty ick.

I ended up writing a python script to massage the patches, and added
that to my patch application alias.

The series is now applied to master.




email-to-8bit
Description: Binary data
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v3 00/34] A great number of cosmetic changes

2020-08-09 Thread David Bremner
Jonas Bernoulli  writes:
>
> Jonas Bernoulli (34):
>   emacs: Shorten long lines
>   emacs: Remove excess empty lines


These patches (and Đoàn Trần Công Danh's that I just applied by hand)
both seem to have problem with base64 encoded patches documented at.

 https://lore.kernel.org/git/20191218194144.gl17...@pobox.com/

It seems that the patches have CRLF line endings inside the base64
encoding, and git fails to strip the CRs.

I'm not sure why this happens for some patches and not for others;
I hope there is not re-encoding happening on the mailing list. If there
is we should fix that.

The practical upshot is that I can't really apply such a long series by
hand. Ideas welcome. A short term fix would be to fetch a ref from
somewhere. OTOH, I hope this isn't going to be a recurring problem.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH 02/19] lib: add stub for notmuch_database_open_with_config

2020-08-09 Thread David Bremner
Reto  writes:

>> + * Mode to open database. Use one of #NOTMUCH_DATABASE_MODE_READ_WRITE
>> + * or #NOTMUCH_DATABASE_MODE_READ_WRITE
>
> I think you want to have NOTMUCH_DATABASE_MODE_READ_ONLY here?
>
> Greetings,
> Reto

yes, you are correct.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v5] Emacs: Ensure left-to-right display for message headers

2020-08-08 Thread David Bremner
Teemu Likonen  writes:

> In notmuch-show buffer insert invisible U+200E LEFT-TO-RIGHT MARK
> character at the beginning of message header paragraph if the From
> header contains a right-to-left character. This ensures that the
> header paragraph is always rendered in left-to-right mode.
>
> See Emacs Lisp reference manual section "(elisp) Bidirectional
> Display" for more info.

v5 applied to master.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2] configure: check for pytest with python -m pytest

2020-08-08 Thread David Bremner
Tomi Ollila  writes:

> On Mon, Aug 03 2020, Đoàn Trần Công Danh wrote:
>
>> On different distro, pytest is suffixed with different patterns.
>>
>> On the other hand, `python3-pytest' could be invoked correctly,
>> via `python3 -m pytest', the latter is used by our tests, now.
>>
>> Switch to `$python -m pytest` to fix address all incompatible naming.
>>
>> Signed-off-by: Đoàn Trần Công Danh 
>
> THis series looks good to me on my terminal, but does not apply 
> to my tree (HEAD commit: 0e4695ab (origin/master) test: regression tests for 
> n_indexopts_{get,set}_decrypt_policy)
>
> Tomi

Yes, I had the the same problem, but I managed to get the patches to
apply with some trickery involving "patch". I'm still not sure what the
issue was, I guess the patches were based on an old tree because the
changes to configure were offset by two lines.

Series is applied to master now.
d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 01/19] test: use keys with group 'test' in T590-libconfig

2020-08-08 Thread David Bremner
In a future commit we want to interoperate better with glib KeyFiles,
which need groups for all keys.
---
 test/T590-libconfig.sh | 36 ++--
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 360e45b0..8c34acf9 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -28,18 +28,18 @@ EOF
 test_begin_subtest "notmuch_database_{set,get}_config"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 {
-   EXPECT0(notmuch_database_set_config (db, "testkey1", "testvalue1"));
-   EXPECT0(notmuch_database_set_config (db, "testkey2", "testvalue2"));
-   EXPECT0(notmuch_database_get_config (db, "testkey1", ));
-   printf("testkey1 = %s\n", val);
-   EXPECT0(notmuch_database_get_config (db, "testkey2", ));
-   printf("testkey2 = %s\n", val);
+   EXPECT0(notmuch_database_set_config (db, "test.key1", "testvalue1"));
+   EXPECT0(notmuch_database_set_config (db, "test.key2", "testvalue2"));
+   EXPECT0(notmuch_database_get_config (db, "test.key1", ));
+   printf("test.key1 = %s\n", val);
+   EXPECT0(notmuch_database_get_config (db, "test.key2", ));
+   printf("test.key2 = %s\n", val);
 }
 EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
-testkey1 = testvalue1
-testkey2 = testvalue2
+test.key1 = testvalue1
+test.key2 = testvalue2
 == stderr ==
 EOF
 test_expect_equal_file EXPECTED OUTPUT
@@ -93,8 +93,8 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 aaabefore beforeval
-testkey1 testvalue1
-testkey2 testvalue2
+test.key1 testvalue1
+test.key2 testvalue2
 zzzafter afterval
 == stderr ==
 EOF
@@ -115,8 +115,8 @@ EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
 aaabefore 1
-testkey1 1
-testkey2 1
+test.key1 1
+test.key2 1
 zzzafter 1
 == stderr ==
 EOF
@@ -126,7 +126,7 @@ test_begin_subtest "notmuch_database_get_config_list: one 
prefix"
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 {
notmuch_config_list_t *list;
-   EXPECT0(notmuch_database_get_config_list (db, "testkey", ));
+   EXPECT0(notmuch_database_get_config_list (db, "test.key", ));
for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next 
(list)) {
   printf("%s %s\n", notmuch_config_list_key (list), 
notmuch_config_list_value(list));
}
@@ -135,8 +135,8 @@ cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 EOF
 cat <<'EOF' >EXPECTED
 == stdout ==
-testkey1 testvalue1
-testkey2 testvalue2
+test.key1 testvalue1
+test.key2 testvalue2
 == stderr ==
 EOF
 test_expect_equal_file EXPECTED OUTPUT
@@ -152,8 +152,8 @@ cat <<'EOF' >EXPECTED
 #notmuch-dump batch-tag:3 config
 #@ aaabefore beforeval
 #@ key%20with%20spaces value,%20with,%20spaces%21
-#@ testkey1 testvalue1
-#@ testkey2 testvalue2
+#@ test.key1 testvalue1
+#@ test.key2 testvalue2
 #@ zzzafter afterval
 EOF
 test_expect_equal_file EXPECTED OUTPUT
@@ -162,7 +162,7 @@ test_begin_subtest "restore config"
 notmuch dump --include=config >EXPECTED
 cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
 {
-EXPECT0(notmuch_database_set_config (db, "testkey1", "mutatedvalue"));
+EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue"));
 }
 EOF
 notmuch restore --include=config 

[PATCH 04/19] WIP: add notmuch_config_get

2020-08-08 Thread David Bremner
this just accesses the cached config data

API is to be decided; maybe we should return a status value.
---
 lib/config.cc  |  6 ++
 lib/notmuch.h  |  2 ++
 test/T590-libconfig.sh | 15 +++
 3 files changed, 23 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index 2d4c3801..0a91ec03 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -219,3 +219,9 @@ _notmuch_config_load_from_database (notmuch_database_t 
*notmuch)
 
 return status;
 }
+
+const char *
+notmuch_config_get (notmuch_database_t *notmuch, const char *key) {
+
+return _notmuch_string_map_get (notmuch->config, key);
+}
diff --git a/lib/notmuch.h b/lib/notmuch.h
index e61634a1..777116bb 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2397,6 +2397,8 @@ notmuch_config_list_move_to_next (notmuch_config_list_t 
*config_list);
 void
 notmuch_config_list_destroy (notmuch_config_list_t *config_list);
 
+const char *
+notmuch_config_get (notmuch_database_t *notmuch, const char *key);
 
 /**
  * get the current default indexing options for a given database.
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 8c34acf9..8a4519e9 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -169,4 +169,19 @@ notmuch restore --include=config OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
+test_begin_subtest "notmuch_config_get"
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+{
+   printf("test.key1 = %s\n", notmuch_config_get (db, "test.key1"));
+   printf("test.key2 = %s\n", notmuch_config_get (db, "test.key2"));
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = testvalue1
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_done
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 05/19] WIP: add notmuch_config_set

2020-08-08 Thread David Bremner
To be decided: is the write_through paramater a good idea? Should we
simplify the API?
---
 lib/config.cc  | 16 
 lib/notmuch-private.h  |  5 +
 lib/notmuch.h  |  5 +
 lib/string-map.c   | 16 
 test/T590-libconfig.sh | 30 ++
 5 files changed, 72 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index 0a91ec03..2cfd2882 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -225,3 +225,19 @@ notmuch_config_get (notmuch_database_t *notmuch, const 
char *key) {
 
 return _notmuch_string_map_get (notmuch->config, key);
 }
+
+notmuch_status_t
+notmuch_config_set (notmuch_database_t *notmuch,
+   const char *key,
+   const char *val,
+   notmuch_bool_t write_through)
+{
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+
+_notmuch_string_map_set (notmuch->config, key, val);
+
+if (write_through)
+   status = notmuch_database_set_config (notmuch, key, val);
+
+return status;
+}
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 0f26b371..b545fc46 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -638,6 +638,11 @@ _notmuch_string_map_append (notmuch_string_map_t *map,
const char *key,
const char *value);
 
+void
+_notmuch_string_map_set (notmuch_string_map_t *map,
+   const char *key,
+   const char *value);
+
 const char *
 _notmuch_string_map_get (notmuch_string_map_t *map, const char *key);
 
diff --git a/lib/notmuch.h b/lib/notmuch.h
index 777116bb..e147d5e6 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -2400,6 +2400,11 @@ notmuch_config_list_destroy (notmuch_config_list_t 
*config_list);
 const char *
 notmuch_config_get (notmuch_database_t *notmuch, const char *key);
 
+notmuch_status_t
+notmuch_config_set (notmuch_database_t *notmuch, const char *key,
+   const char *val,
+   notmuch_bool_t write_through);
+
 /**
  * get the current default indexing options for a given database.
  *
diff --git a/lib/string-map.c b/lib/string-map.c
index a88404c7..9774dbe8 100644
--- a/lib/string-map.c
+++ b/lib/string-map.c
@@ -143,6 +143,22 @@ bsearch_first (notmuch_string_pair_t *array, size_t len, 
const char *key, bool e
 
 }
 
+void
+_notmuch_string_map_set (notmuch_string_map_t *map, const char *key, const 
char *val)
+{
+notmuch_string_pair_t *pair;
+
+/* this means that calling string_map_set invalidates iterators */
+_notmuch_string_map_sort (map);
+pair = bsearch_first (map->pairs, map->length, key, true);
+if (! pair)
+   _notmuch_string_map_append (map, key, val);
+else {
+   talloc_free (pair->value);
+   pair->value = talloc_strdup (map->pairs, val);
+}
+}
+
 const char *
 _notmuch_string_map_get (notmuch_string_map_t *map, const char *key)
 {
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index 8a4519e9..c2bce4a2 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -184,4 +184,34 @@ test.key2 = testvalue2
 EOF
 test_expect_equal_file EXPECTED OUTPUT
 
+backup_database
+test_begin_subtest "notmuch_config_set"
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+{
+   char *val;
+   printf("test.key1 = %s\n", notmuch_config_get (db, "test.key1"));
+   EXPECT0(notmuch_config_set (db, "test.key1", "overridden", FALSE));
+   printf("test.key1 = %s\n", notmuch_config_get (db, "test.key1"));
+   printf("test.key2 = %s\n", notmuch_config_get (db, "test.key2"));
+   EXPECT0(notmuch_database_get_config (db, "test.key1", ));
+   printf("test.key1 (db) = %s\n", val);
+   EXPECT0(notmuch_config_set (db, "test.key2", "overridden2", TRUE));
+   printf("test.key2 = %s\n", notmuch_config_get (db, "test.key2"));
+   EXPECT0(notmuch_database_get_config (db, "test.key2", ));
+   printf("test.key2 (db) = %s\n", val);
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = testvalue1
+test.key1 = overridden
+test.key2 = testvalue2
+test.key1 (db) = testvalue1
+test.key2 = overridden2
+test.key2 (db) = overridden2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
 test_done
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 09/19] lib: factor out feature name related code.

2020-08-08 Thread David Bremner
database.cc is uncomfortably large, and some of the static data
structures do not need to be shared as much as they are.

This is a somewhat small piece to factor out, but it will turn out to
be helpful to further refactoring.
---
 lib/Makefile.local |   3 +-
 lib/database-private.h |  14 +
 lib/database.cc| 116 +
 lib/features.cc| 114 
 4 files changed, 131 insertions(+), 116 deletions(-)
 create mode 100644 lib/features.cc

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 5dc057c0..b95ae216 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -59,7 +59,8 @@ libnotmuch_cxx_srcs = \
$(dir)/config.cc\
$(dir)/regexp-fields.cc \
$(dir)/thread.cc \
-   $(dir)/thread-fp.cc
+   $(dir)/thread-fp.cc \
+   $(dir)/features.cc
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 
diff --git a/lib/database-private.h b/lib/database-private.h
index bfeef869..3b365c41 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -32,6 +32,8 @@
 
 #include "notmuch-private.h"
 
+#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
+
 #ifdef SILENCE_XAPIAN_DEPRECATION_WARNINGS
 #define XAPIAN_DEPRECATED(D) D
 #endif
@@ -266,4 +268,16 @@ _notmuch_database_find_doc_ids (notmuch_database_t 
*notmuch,
const char *value,
Xapian::PostingIterator *begin,
Xapian::PostingIterator *end);
+
+#define NOTMUCH_DATABASE_VERSION 3
+
+/* features.cc */
+
+_notmuch_features
+_notmuch_database_parse_features (const void *ctx, const char *features, 
unsigned int version,
+ char mode, char **incompat_out);
+
+char *
+_notmuch_database_print_features (const void *ctx, unsigned int features);
+
 #endif
diff --git a/lib/database.cc b/lib/database.cc
index 93fef107..ceba6dd5 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -39,8 +39,6 @@
 
 using namespace std;
 
-#define ARRAY_SIZE(arr) (sizeof (arr) / sizeof (arr[0]))
-
 typedef struct {
 const char *name;
 const char *prefix;
@@ -458,41 +456,6 @@ _notmuch_database_prefix (notmuch_database_t *notmuch, 
const char *name)
 return NULL;
 }
 
-static const struct {
-/* NOTMUCH_FEATURE_* value. */
-_notmuch_features value;
-/* Feature name as it appears in the database.  This name should
- * be appropriate for displaying to the user if an older version
- * of notmuch doesn't support this feature. */
-const char *name;
-/* Compatibility flags when this feature is declared. */
-const char *flags;
-} feature_names[] = {
-{ NOTMUCH_FEATURE_FILE_TERMS,
-  "multiple paths per message", "rw" },
-{ NOTMUCH_FEATURE_DIRECTORY_DOCS,
-  "relative directory paths", "rw" },
-/* Header values are not required for reading a database because a
- * reader can just refer to the message file. */
-{ NOTMUCH_FEATURE_FROM_SUBJECT_ID_VALUES,
-  "from/subject/message-ID in database", "w" },
-{ NOTMUCH_FEATURE_BOOL_FOLDER,
-  "exact folder:/path: search", "rw" },
-{ NOTMUCH_FEATURE_GHOSTS,
-  "mail documents for missing messages", "w" },
-/* Knowledge of the index mime-types are not required for reading
- * a database because a reader will just be unable to query
- * them. */
-{ NOTMUCH_FEATURE_INDEXED_MIMETYPES,
-  "indexed MIME types", "w" },
-{ NOTMUCH_FEATURE_LAST_MOD,
-  "modification tracking", "w" },
-/* Existing databases will work fine for all queries not involving
- * 'body:' */
-{ NOTMUCH_FEATURE_UNPREFIX_BODY_ONLY,
-  "index body and headers separately", "w" },
-};
-
 const char *
 notmuch_status_to_string (notmuch_status_t status)
 {
@@ -817,83 +780,6 @@ _notmuch_database_new_revision (notmuch_database_t 
*notmuch)
 return new_revision;
 }
 
-/* Parse a database features string from the given database version.
- * Returns the feature bit set.
- *
- * For version < 3, this ignores the features string and returns a
- * hard-coded set of features.
- *
- * If there are unrecognized features that are required to open the
- * database in mode (which should be 'r' or 'w'), return a
- * comma-separated list of unrecognized but required features in
- * *incompat_out suitable for presenting to the user.  *incompat_out
- * will be allocated from ctx.
- */
-static _notmuch_features
-_parse_features (const void *ctx, const char *features, unsigned int version,
-char mode, char **incompat_out)
-{
-_notmuch_features res = static_cast<_notmuch_features>(0);
-unsigned int namelen, i;
-size_t llen = 0;
-const char *flags;
-
-/* Prior to database version 3, features were implied by the
- * version number. */
-if (version == 0)
-   return NOTMUCH_FEATURES_V0;
-else if (version == 1)
-   return 

[PATCH 06/19] WIP: initial retrieval of database path from config file

2020-08-08 Thread David Bremner
---
 lib/database.cc | 18 ++
 1 file changed, 18 insertions(+)

diff --git a/lib/database.cc b/lib/database.cc
index 350abb11..2ab9170d 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -935,12 +935,27 @@ notmuch_database_open_with_config (const char 
*database_path,
 void *local = talloc_new (NULL);
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path, *xapian_path, *incompat_features;
+char *configured_database_path = NULL;
 char *message = NULL;
 struct stat st;
 int err;
 unsigned int i, version;
+GKeyFile *key_file = NULL;
 static int initialized = 0;
 
+/* XXX TODO: default locations for NULL case, handle profiles */
+if (config_path != NULL && ! EMPTY_STRING (config_path)) {
+   key_file = g_key_file_new ();
+   if (! g_key_file_load_from_file (key_file, config_path, 
G_KEY_FILE_NONE, NULL)) {
+   status = NOTMUCH_STATUS_FILE_ERROR;
+   goto DONE;
+   }
+   configured_database_path = g_key_file_get_value (key_file, "database", 
"path", NULL);
+}
+
+if (database_path == NULL)
+   database_path = configured_database_path;
+
 if (database_path == NULL) {
message = strdup ("Error: Cannot open a database for a NULL path.\n");
status = NOTMUCH_STATUS_NULL_POINTER;
@@ -1100,6 +1115,9 @@ notmuch_database_open_with_config (const char 
*database_path,
   DONE:
 talloc_free (local);
 
+if (key_file)
+   g_key_file_free (key_file);
+
 if (message) {
if (status_string)
*status_string = message;
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 08/19] WIP: add _notmuch_config_load_from_file

2020-08-08 Thread David Bremner
---
 lib/config.cc  | 34 ++
 lib/database.cc|  4 
 lib/notmuch-private.h  |  3 +++
 test/T590-libconfig.sh | 23 ++-
 4 files changed, 63 insertions(+), 1 deletion(-)

diff --git a/lib/config.cc b/lib/config.cc
index 2cfd2882..f5def3aa 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -241,3 +241,37 @@ notmuch_config_set (notmuch_database_t *notmuch,
 
 return status;
 }
+
+notmuch_status_t
+_notmuch_config_load_from_file (notmuch_database_t *notmuch,
+   GKeyFile *file)
+{
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+gchar **groups,**keys, *val;
+
+if (notmuch->config == NULL)
+   notmuch->config = _notmuch_string_map_create (notmuch);
+
+if (unlikely(notmuch->config == NULL)) {
+   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
+   goto DONE;
+}
+
+for (groups = g_key_file_get_groups (file, NULL); *groups; groups++) {
+   for (keys = g_key_file_get_keys (file, *groups, NULL, NULL); *keys; 
keys++) {
+   char *absolute_key = talloc_asprintf(notmuch, "%s.%s", *groups,  
*keys);
+   val = g_key_file_get_value (file, *groups, *keys, NULL);
+   if (! val) {
+   status = NOTMUCH_STATUS_FILE_ERROR;
+   goto DONE;
+   }
+   status = notmuch_config_set (notmuch, absolute_key, val, FALSE);
+   talloc_free (absolute_key);
+   if (status)
+   goto DONE;
+   }
+}
+
+ DONE:
+return status;
+}
diff --git a/lib/database.cc b/lib/database.cc
index 2ab9170d..93fef107 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -1104,6 +1104,10 @@ notmuch_database_open_with_config (const char 
*database_path,
 
if (status == NOTMUCH_STATUS_SUCCESS)
status = _notmuch_config_load_from_database (notmuch);
+
+   if (status == NOTMUCH_STATUS_SUCCESS && key_file)
+   status = _notmuch_config_load_from_file (notmuch, key_file);
+
 } catch (const Xapian::Error ) {
IGNORE_RESULT (asprintf (, "A Xapian exception occurred opening 
database: %s\n",
 error.get_msg ().c_str ()));
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index b545fc46..192a0771 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -707,6 +707,9 @@ struct _notmuch_indexopts {
 /* config.cc */
 notmuch_status_t
 _notmuch_config_load_from_database (notmuch_database_t * db);
+
+notmuch_status_t
+_notmuch_config_load_from_file (notmuch_database_t * db, GKeyFile *file);
 NOTMUCH_END_DECLS
 
 #ifdef __cplusplus
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index a34eae0b..a303319d 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -128,7 +128,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_database_get_config_list: one prefix"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
notmuch_config_list_t *list;
EXPECT0(notmuch_database_get_config_list (db, "test.key", ));
@@ -219,4 +219,25 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 restore_database
 
+backup_database
+test_begin_subtest "override config from file"
+notmuch config set test.key1 overridden
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+{
+   printf("test.key1 = %s\n", notmuch_config_get (db, "test.key1"));
+   EXPECT0(notmuch_database_get_config (db, "test.key1", ));
+   printf("test.key1 (db) = %s\n", val);
+   printf("test.key2 = %s\n", notmuch_config_get (db, "test.key2"));
+}
+EOF
+cat <<'EOF' >EXPECTED
+== stdout ==
+test.key1 = overridden
+test.key1 (db) = testvalue1
+test.key2 = testvalue2
+== stderr ==
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+restore_database
+
 test_done
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 14/19] WIP: add strsplit_len

2020-08-08 Thread David Bremner
---
 util/string-util.c | 23 +++
 util/string-util.h | 14 ++
 2 files changed, 37 insertions(+)

diff --git a/util/string-util.c b/util/string-util.c
index de8430b2..27f8a26b 100644
--- a/util/string-util.c
+++ b/util/string-util.c
@@ -24,6 +24,7 @@
 
 #include 
 #include 
+#include 
 
 char *
 strtok_len (char *s, const char *delim, size_t *len)
@@ -37,6 +38,28 @@ strtok_len (char *s, const char *delim, size_t *len)
 return *len ? s : NULL;
 }
 
+const char *
+strsplit_len (const char *s, char delim, size_t *len)
+{
+bool escaping = false;
+size_t count = 0;
+
+/* Skip initial unescaped delimiters */
+while (*s && *s == delim)
+   s++;
+
+while (s[count] && (escaping || s[count] != delim)) {
+   escaping = (s[count] == '\\');
+   count++;
+}
+
+if (count==0)
+   return NULL;
+
+*len = count;
+return s;
+}
+
 const char *
 strtok_len_c (const char *s, const char *delim, size_t *len)
 {
diff --git a/util/string-util.h b/util/string-util.h
index fb95a740..80647c5f 100644
--- a/util/string-util.h
+++ b/util/string-util.h
@@ -26,6 +26,20 @@ char *strtok_len (char *s, const char *delim, size_t *len);
 /* Const version of strtok_len. */
 const char *strtok_len_c (const char *s, const char *delim, size_t *len);
 
+/* Simplified version of strtok_len, with a single delimiter.
+ * Handles escaping delimiters with \
+ * Usage pattern:
+ *
+ * const char *tok = input;
+ * const char *delim = ';';
+ * size_t tok_len = 0;
+ *
+ * while ((tok = strsplit_len (tok + tok_len, delim, _len)) != NULL) {
+ * // do stuff with string tok of length tok_len
+ * }
+ */
+const char *strsplit_len (const char *s, char delim, size_t *len);
+
 /* Return a talloced string with str sanitized.
  *
  * Whitespace characters (tabs and newlines) are replaced with spaces,
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 03/19] lib: cache configuration information from database

2020-08-08 Thread David Bremner
The main goal is to allow configuration information to be temporarily
overridden by a separate config file.
---
 lib/config.cc  | 25 +
 lib/database-private.h |  3 +++
 lib/database.cc|  4 
 lib/notmuch-private.h  |  3 +++
 4 files changed, 35 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index dae0ff0e..2d4c3801 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -194,3 +194,28 @@ notmuch_config_list_destroy (notmuch_config_list_t *list)
 {
 talloc_free (list);
 }
+
+notmuch_status_t
+_notmuch_config_load_from_database (notmuch_database_t *notmuch)
+{
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+notmuch_config_list_t *list;
+
+if (notmuch->config == NULL)
+   notmuch->config = _notmuch_string_map_create (notmuch);
+
+if (unlikely(notmuch->config == NULL))
+   return NOTMUCH_STATUS_OUT_OF_MEMORY;
+
+status = notmuch_database_get_config_list (notmuch, "", );
+if (status)
+   return status;
+
+for (; notmuch_config_list_valid (list); notmuch_config_list_move_to_next 
(list)) {
+   _notmuch_string_map_append (notmuch->config,
+   notmuch_config_list_key (list),
+   notmuch_config_list_value (list));
+}
+
+return status;
+}
diff --git a/lib/database-private.h b/lib/database-private.h
index 041602cd..bfeef869 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -226,6 +226,9 @@ struct _notmuch_database {
  * here, but at least they are small */
 notmuch_string_map_t *user_prefix;
 notmuch_string_map_t *user_header;
+
+/* Cached and possibly overridden configuration */
+notmuch_string_map_t *config;
 };
 
 /* Prior to database version 3, features were implied by the database
diff --git a/lib/database.cc b/lib/database.cc
index acc686c0..350abb11 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -987,6 +987,7 @@ notmuch_database_open_with_config (const char 
*database_path,
 notmuch = talloc_zero (NULL, notmuch_database_t);
 notmuch->exception_reported = false;
 notmuch->status_string = NULL;
+notmuch->config = NULL;
 notmuch->path = talloc_strdup (notmuch, database_path);
 
 strip_trailing (notmuch->path, '/');
@@ -1085,6 +1086,9 @@ notmuch_database_open_with_config (const char 
*database_path,
}
}
status = _setup_user_query_fields (notmuch);
+
+   if (status == NOTMUCH_STATUS_SUCCESS)
+   status = _notmuch_config_load_from_database (notmuch);
 } catch (const Xapian::Error ) {
IGNORE_RESULT (asprintf (, "A Xapian exception occurred opening 
database: %s\n",
 error.get_msg ().c_str ()));
diff --git a/lib/notmuch-private.h b/lib/notmuch-private.h
index 57ec7f72..0f26b371 100644
--- a/lib/notmuch-private.h
+++ b/lib/notmuch-private.h
@@ -699,6 +699,9 @@ struct _notmuch_indexopts {
 
 #define EMPTY_STRING(s) ((s)[0] == '\0')
 
+/* config.cc */
+notmuch_status_t
+_notmuch_config_load_from_database (notmuch_database_t * db);
 NOTMUCH_END_DECLS
 
 #ifdef __cplusplus
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 12/19] WIP: adding fallbacks for NULL config_path

2020-08-08 Thread David Bremner
still need to test combinations of NULL and non-NULL arguments
---
 lib/open.cc|  95 
 test/T590-libconfig.sh | 138 +
 2 files changed, 209 insertions(+), 24 deletions(-)

diff --git a/lib/open.cc b/lib/open.cc
index 9b7da161..213dc21e 100644
--- a/lib/open.cc
+++ b/lib/open.cc
@@ -37,6 +37,82 @@ notmuch_database_open_verbose (const char *path,
  database, status_string);
 }
 
+static const char *
+_xdg_dir (void *ctx,
+ const char *xdg_root_variable,
+ const char *xdg_prefix,
+ const char *profile_name)
+{
+const char *xdg_root = getenv (xdg_root_variable);
+const char *home = getenv ("HOME");
+
+if (! xdg_root) {
+   if (! home) return NULL;
+
+   xdg_root = talloc_asprintf (ctx,
+   "%s/%s",
+   home,
+   xdg_prefix);
+}
+
+if (! profile_name)
+   profile_name = getenv ("NOTMUCH_PROFILE");
+
+if (! profile_name)
+   profile_name = "default";
+
+return talloc_asprintf (ctx,
+   "%s/notmuch/%s",
+   xdg_root,
+   profile_name);
+}
+
+static notmuch_status_t
+_load_key_file (const char *path,
+   const char *profile,
+   GKeyFile **key_file)
+{
+notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
+void *local = talloc_new (NULL);
+
+if (path && EMPTY_STRING (path))
+   goto DONE;
+
+if (! path)
+   path = getenv ("NOTMUCH_CONFIG");
+
+if (! path) {
+   const char *dir = _xdg_dir (local, "XDG_CONFIG_HOME", ".config", 
profile);
+
+   if (dir) {
+   path = talloc_asprintf (local, "%s/config", dir);
+   if (access (path, R_OK) !=0)
+   path = NULL;
+   }
+}
+
+if (! path) {
+   const char *home = getenv ("HOME");
+
+   path = talloc_asprintf (local, "%s/.notmuch-config", home);
+
+   if (! profile)
+   profile = getenv ("NOTMUCH_PROFILE");
+
+   if (profile)
+   path = talloc_asprintf (local, "%s.%s", path, profile);
+}
+
+*key_file = g_key_file_new ();
+if (! g_key_file_load_from_file (*key_file, path, G_KEY_FILE_NONE, NULL)) {
+   status = NOTMUCH_STATUS_FILE_ERROR;
+}
+
+DONE:
+talloc_free (local);
+return status;
+}
+
 notmuch_status_t
 notmuch_database_open_with_config (const char *database_path,
   notmuch_database_mode_t mode,
@@ -49,7 +125,6 @@ notmuch_database_open_with_config (const char *database_path,
 void *local = talloc_new (NULL);
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path, *xapian_path, *incompat_features;
-char *configured_database_path = NULL;
 char *message = NULL;
 struct stat st;
 int err;
@@ -57,18 +132,14 @@ notmuch_database_open_with_config (const char 
*database_path,
 GKeyFile *key_file = NULL;
 static int initialized = 0;
 
-/* XXX TODO: default locations for NULL case, handle profiles */
-if (config_path != NULL && ! EMPTY_STRING (config_path)) {
-   key_file = g_key_file_new ();
-   if (! g_key_file_load_from_file (key_file, config_path, 
G_KEY_FILE_NONE, NULL)) {
-   status = NOTMUCH_STATUS_FILE_ERROR;
-   goto DONE;
-   }
-   configured_database_path = g_key_file_get_value (key_file, "database", 
"path", NULL);
+status = _load_key_file (config_path, profile, _file);
+if (status) {
+   message = strdup ("Error: cannot load config file");
+   goto DONE;
 }
-
-if (database_path == NULL)
-   database_path = configured_database_path;
+   
+if (! database_path && key_file)
+   database_path = g_key_file_get_value (key_file, "database", "path", 
NULL);
 
 if (database_path == NULL) {
message = strdup ("Error: Cannot open a database for a NULL path.\n");
diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index a303319d..cc814efa 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -15,14 +15,21 @@ int main (int argc, char** argv)
notmuch_database_t *db;
char *val;
notmuch_status_t stat;
+   char *msg = NULL;
 
-   EXPECT0(notmuch_database_open_with_config (argv[1],
+   for (int i=1; i c_tail
@@ -51,7 +58,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 
 test_begin_subtest "notmuch_database_get_config_list: empty list"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG} %NULL%
 {
notmuch_config_list_t *list;
EXPECT0(notmuch_database_get_config_list (db, "nonexistent", ));
@@ -83,7 +90,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_database_get_config_list: all pairs"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
+cat c_head - 

[PATCH 10/19] lib: factor out prefix related code to its own file

2020-08-08 Thread David Bremner
Reduce the size of database.cc, and limit the scope of prefix_table,
make sure it's accessed via a well-defined internal API.
---
 lib/Makefile.local |   4 +-
 lib/database-private.h |   7 ++
 lib/database.cc| 204 ++-
 lib/prefix.cc  | 210 +
 4 files changed, 229 insertions(+), 196 deletions(-)
 create mode 100644 lib/prefix.cc

diff --git a/lib/Makefile.local b/lib/Makefile.local
index b95ae216..15e71ef8 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -60,7 +60,9 @@ libnotmuch_cxx_srcs = \
$(dir)/regexp-fields.cc \
$(dir)/thread.cc \
$(dir)/thread-fp.cc \
-   $(dir)/features.cc
+   $(dir)/features.cc  \
+   $(dir)/prefix.cc
+
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 
diff --git a/lib/database-private.h b/lib/database-private.h
index 3b365c41..d83cf0d0 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -280,4 +280,11 @@ _notmuch_database_parse_features (const void *ctx, const 
char *features, unsigne
 char *
 _notmuch_database_print_features (const void *ctx, unsigned int features);
 
+/* prefix.cc */
+notmuch_status_t
+_notmuch_database_setup_standard_query_fields (notmuch_database_t *notmuch);
+
+notmuch_status_t
+_notmuch_database_setup_user_query_fields (notmuch_database_t *notmuch);
+
 #endif
diff --git a/lib/database.cc b/lib/database.cc
index ceba6dd5..0620de5a 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -263,80 +263,6 @@ _notmuch_database_mode (notmuch_database_t *notmuch)
  * same thread.
  */
 
-/* With these prefix values we follow the conventions published here:
- *
- * https://xapian.org/docs/omega/termprefixes.html
- *
- * as much as makes sense. Note that I took some liberty in matching
- * the reserved prefix values to notmuch concepts, (for example, 'G'
- * is documented as "newsGroup (or similar entity - e.g. a web forum
- * name)", for which I think the thread is the closest analogue in
- * notmuch. This in spite of the fact that we will eventually be
- * storing mailing-list messages where 'G' for "mailing list name"
- * might be even a closer analogue. I'm treating the single-character
- * prefixes preferentially for core notmuch concepts (which will be
- * nearly universal to all mail messages).
- */
-
-static const
-prefix_t prefix_table[] = {
-/* nameterm prefix flags */
-{ "type",   "T",NOTMUCH_FIELD_NO_FLAGS },
-{ "reference",  "XREFERENCE",   NOTMUCH_FIELD_NO_FLAGS },
-{ "replyto","XREPLYTO", NOTMUCH_FIELD_NO_FLAGS },
-{ "directory",  "XDIRECTORY",   NOTMUCH_FIELD_NO_FLAGS },
-{ "file-direntry",  "XFDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
-{ "directory-direntry", "XDDIRENTRY",   NOTMUCH_FIELD_NO_FLAGS },
-{ "body",   "", NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "thread", "G",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "tag","K",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "is", "K",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "id", "Q",NOTMUCH_FIELD_EXTERNAL },
-{ "mid","Q",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "path",   "P",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "property",   "XPROPERTY",NOTMUCH_FIELD_EXTERNAL },
-/*
- * Unconditionally add ':' to reduce potential ambiguity with
- * overlapping prefixes and/or terms that start with capital
- * letters. See Xapian document termprefixes.html for related
- * discussion.
- */
-{ "folder", "XFOLDER:", NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "date",   NULL,   NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "query",  NULL,   NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "from",   "XFROM",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC |
-  NOTMUCH_FIELD_PROCESSOR },
-{ "to", "XTO",  NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "attachment", "XATTACHMENT",  NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "mimetype",   "XMIMETYPE",NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC },
-{ "subject","XSUBJECT", NOTMUCH_FIELD_EXTERNAL |
-  NOTMUCH_FIELD_PROBABILISTIC |
-  NOTMUCH_FIELD_PROCESSOR },
-};
-
-static void
-_setup_query_field_default 

[PATCH 16/19] test: add regression test for searching with alternate config

2020-08-08 Thread David Bremner
Make sure upcoming changes to config handling do not break command
line specification.
---
 test/T140-excludes.sh | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/test/T140-excludes.sh b/test/T140-excludes.sh
index 0cf69975..cef07095 100755
--- a/test/T140-excludes.sh
+++ b/test/T140-excludes.sh
@@ -39,6 +39,16 @@ deleted_id=$gen_msg_id
 output=$(notmuch search subject:deleted | notmuch_search_sanitize)
 test_expect_equal "$output" "thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; 
Not deleted (inbox unread)"
 
+test_begin_subtest "Search, exclude \"deleted\" messages; alternate config 
file"
+cp ${NOTMUCH_CONFIG} alt-config
+notmuch config set search.exclude_tags
+notmuch --config=alt-config search subject:deleted | notmuch_search_sanitize > 
OUTPUT
+cp alt-config ${NOTMUCH_CONFIG}
+cat < EXPECTED
+thread:XXX   2001-01-05 [1/1] Notmuch Test Suite; Not deleted (inbox unread)
+EOF
+test_expect_equal_file EXPECTED OUTPUT
+
 test_begin_subtest "Search, exclude \"deleted\" messages from message search"
 output=$(notmuch search --output=messages subject:deleted | 
notmuch_search_sanitize)
 test_expect_equal "$output" "id:$not_deleted_id"
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 19/19] WIP: switch notmuch-show to new config framework

2020-08-08 Thread David Bremner
---
 notmuch-show.c | 25 +++--
 1 file changed, 15 insertions(+), 10 deletions(-)

diff --git a/notmuch-show.c b/notmuch-show.c
index dd836add..094cd689 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1234,6 +1234,7 @@ notmuch_show_command (notmuch_config_t *config, int argc, 
char *argv[])
 bool entire_thread_set = false;
 bool single_message;
 bool unthreaded = FALSE;
+char *status_string = NULL;
 
 notmuch_opt_desc_t options[] = {
{ .opt_keyword = , .name = "format", .keywords =
@@ -1323,7 +1324,20 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
fprintf (stderr, "Warning: --include-html only implemented for 
format=text, format=json and format=sexp\n");
 }
 
-query_string = query_string_from_args (config, argc - opt_index, argv + 
opt_index);
+notmuch_database_mode_t mode = NOTMUCH_DATABASE_MODE_READ_ONLY;
+if (params.crypto.decrypt == NOTMUCH_DECRYPT_TRUE)
+   mode = NOTMUCH_DATABASE_MODE_READ_WRITE;
+if (notmuch_database_open_with_config (NULL,
+  mode,
+  _notmuch_config_get_path (config),
+  NULL,
+  ,
+  _string))
+   return EXIT_FAILURE;
+
+notmuch_exit_if_unmatched_db_uuid (notmuch);
+
+query_string = query_string_from_args (notmuch, argc - opt_index, argv + 
opt_index);
 if (query_string == NULL) {
fprintf (stderr, "Out of memory\n");
return EXIT_FAILURE;
@@ -1334,15 +1348,6 @@ notmuch_show_command (notmuch_config_t *config, int 
argc, char *argv[])
return EXIT_FAILURE;
 }
 
-notmuch_database_mode_t mode = NOTMUCH_DATABASE_MODE_READ_ONLY;
-if (params.crypto.decrypt == NOTMUCH_DECRYPT_TRUE)
-   mode = NOTMUCH_DATABASE_MODE_READ_WRITE;
-if (notmuch_database_open (notmuch_config_get_database_path (config),
-  mode, ))
-   return EXIT_FAILURE;
-
-notmuch_exit_if_unmatched_db_uuid (notmuch);
-
 query = notmuch_query_create (notmuch, query_string);
 if (query == NULL) {
fprintf (stderr, "Out of memory\n");
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 02/19] lib: add stub for notmuch_database_open_with_config

2020-08-08 Thread David Bremner
Initially document the intended API and copy the code from
notmuch_database_open_verbose. Most of the documented functionality is
not there yet.
---
 lib/database.cc |  21 ++--
 lib/notmuch.h   | 138 ++--
 2 files changed, 127 insertions(+), 32 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 75189685..acc686c0 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -918,6 +918,18 @@ notmuch_database_open_verbose (const char *path,
   notmuch_database_mode_t mode,
   notmuch_database_t **database,
   char **status_string)
+{
+return notmuch_database_open_with_config (path, mode, "", NULL,
+ database, status_string);
+}
+
+notmuch_status_t
+notmuch_database_open_with_config (const char *database_path,
+  notmuch_database_mode_t mode,
+  const char *config_path,
+  const char *profile,
+  notmuch_database_t **database,
+  char **status_string)
 {
 notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
 void *local = talloc_new (NULL);
@@ -929,19 +941,19 @@ notmuch_database_open_verbose (const char *path,
 unsigned int i, version;
 static int initialized = 0;
 
-if (path == NULL) {
+if (database_path == NULL) {
message = strdup ("Error: Cannot open a database for a NULL path.\n");
status = NOTMUCH_STATUS_NULL_POINTER;
goto DONE;
 }
 
-if (path[0] != '/') {
+if (database_path[0] != '/') {
message = strdup ("Error: Database path must be absolute.\n");
status = NOTMUCH_STATUS_PATH_ERROR;
goto DONE;
 }
 
-if (! (notmuch_path = talloc_asprintf (local, "%s/%s", path, ".notmuch"))) 
{
+if (! (notmuch_path = talloc_asprintf (local, "%s/%s", database_path, 
".notmuch"))) {
message = strdup ("Out of memory\n");
status = NOTMUCH_STATUS_OUT_OF_MEMORY;
goto DONE;
@@ -975,7 +987,7 @@ notmuch_database_open_verbose (const char *path,
 notmuch = talloc_zero (NULL, notmuch_database_t);
 notmuch->exception_reported = false;
 notmuch->status_string = NULL;
-notmuch->path = talloc_strdup (notmuch, path);
+notmuch->path = talloc_strdup (notmuch, database_path);
 
 strip_trailing (notmuch->path, '/');
 
@@ -1101,7 +1113,6 @@ notmuch_database_open_verbose (const char *path,
 
 return status;
 }
-
 notmuch_status_t
 notmuch_database_close (notmuch_database_t *notmuch)
 {
diff --git a/lib/notmuch.h b/lib/notmuch.h
index f3cb0fe2..e61634a1 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -301,52 +301,136 @@ typedef enum {
 } notmuch_database_mode_t;
 
 /**
- * Open an existing notmuch database located at 'path'.
+ * Deprecated alias for notmuch_database_open_with_config with
+ * config_path=error_message=NULL
+ * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
+ */
+/* NOTMUCH_DEPRECATED(5, 4) */
+notmuch_status_t
+notmuch_database_open (const char *path,
+  notmuch_database_mode_t mode,
+  notmuch_database_t **database);
+/**
+ * Deprecated alias for notmuch_database_open_with_config with
+ * config_path=NULL
+ *
+ * @deprecated Deprecated as of libnotmuch 5.4 (notmuch 0.32)
+ *
+ */
+/* NOTMUCH_DEPRECATED(5, 4) */
+notmuch_status_t
+notmuch_database_open_verbose (const char *path,
+  notmuch_database_mode_t mode,
+  notmuch_database_t **database,
+  char **error_message);
+
+/**
+ * Open an existing notmuch database located at 'database_path', using
+ * configuration in 'config_path'.
+ *
+ * @param[in]  database_path
+ * @parblock
+ * Path to existing database.
+ *
+ * A notmuch database is a Xapian database containing appropriate
+ * metadata.
  *
  * The database should have been created at some time in the past,
  * (not necessarily by this process), by calling
- * notmuch_database_create with 'path'. By default the database should be
- * opened for reading only. In order to write to the database you need to
- * pass the NOTMUCH_DATABASE_MODE_READ_WRITE mode.
+ * notmuch_database_create.
+ *
+ * If 'database_path' is NULL, use the location specified
+ *
+ * - in the environment variable NOTMUCH_DATABASE, if non-empty
+ *
+ * - by $XDG_DATA_HOME/notmuch/$PROFILE where XDG_DATA_HOME defaults
+ *   to "$HOME/.local/share" and PROFILE as as discussed in
+ *   'profile'
+ *
+ * If 'database_path' is non-NULL, but does not appear to be a Xapian
+ * database, check for a directory '.notmuch/xapian' below
+ * 'database_path' (this is the behavior of
+ * notmuch_database_open_verbose pre-0.32).
+ *
+ * @endparblock
+ * @param[in]  mode
+ * @parblock
+ * Mode to open database. Use one of 

[PATCH 13/19] lib/config: delay setting talloc destructor

2020-08-08 Thread David Bremner
If Xapian has thrown an exception, it is not safe to invoke the
destuctor when freeing the list struct.
---
 lib/config.cc | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/lib/config.cc b/lib/config.cc
index f5def3aa..ca7ac2a8 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -113,7 +113,6 @@ notmuch_database_get_config_list (notmuch_database_t 
*notmuch,
goto DONE;
 }
 
-talloc_set_destructor (list, _notmuch_config_list_destroy);
 list->notmuch = notmuch;
 list->current_key = NULL;
 list->current_val = NULL;
@@ -133,8 +132,15 @@ notmuch_database_get_config_list (notmuch_database_t 
*notmuch,
 *out = list;
 
   DONE:
-if (status && list)
-   talloc_free (list);
+if (status) {
+   if (list) {
+   talloc_free (list);
+   if (status != NOTMUCH_STATUS_XAPIAN_EXCEPTION)
+   _notmuch_config_list_destroy (list);
+   }
+}  else {
+   talloc_set_destructor (list, _notmuch_config_list_destroy);
+}
 
 return status;
 }
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 11/19] lib: factor out notmuch_database_open* related code to own file

2020-08-08 Thread David Bremner
Reduce the size of database.cc, and prepare for adding missing features to
notmuch_database_open_with_config (to match API documentation already
in notmuch.h).
---
 lib/Makefile.local |   3 +-
 lib/database.cc| 255 -
 lib/open.cc| 253 
 3 files changed, 255 insertions(+), 256 deletions(-)
 create mode 100644 lib/open.cc

diff --git a/lib/Makefile.local b/lib/Makefile.local
index 15e71ef8..01d3b34b 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -61,7 +61,8 @@ libnotmuch_cxx_srcs = \
$(dir)/thread.cc \
$(dir)/thread-fp.cc \
$(dir)/features.cc  \
-   $(dir)/prefix.cc
+   $(dir)/prefix.cc\
+   $(dir)/open.cc
 
 
 libnotmuch_modules := $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
diff --git a/lib/database.cc b/lib/database.cc
index 0620de5a..0f4e2ff9 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -19,10 +19,6 @@
  */
 
 #include "database-private.h"
-#include "parse-time-vrp.h"
-#include "query-fp.h"
-#include "thread-fp.h"
-#include "regexp-fields.h"
 #include "string-util.h"
 
 #include 
@@ -50,12 +46,6 @@ typedef struct {
 #define STRINGIFY(s) _SUB_STRINGIFY (s)
 #define _SUB_STRINGIFY(s) #s
 
-#if HAVE_XAPIAN_DB_RETRY_LOCK
-#define DB_ACTION (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK)
-#else
-#define DB_ACTION Xapian::DB_CREATE_OR_OPEN
-#endif
-
 #define LOG_XAPIAN_EXCEPTION(message, error) _log_xapian_exception 
(__location__, message, error)
 
 static void
@@ -594,251 +584,6 @@ _notmuch_database_new_revision (notmuch_database_t 
*notmuch)
 return new_revision;
 }
 
-notmuch_status_t
-notmuch_database_open (const char *path,
-  notmuch_database_mode_t mode,
-  notmuch_database_t **database)
-{
-char *status_string = NULL;
-notmuch_status_t status;
-
-status = notmuch_database_open_verbose (path, mode, database,
-   _string);
-
-if (status_string) {
-   fputs (status_string, stderr);
-   free (status_string);
-}
-
-return status;
-}
-
-notmuch_status_t
-notmuch_database_open_verbose (const char *path,
-  notmuch_database_mode_t mode,
-  notmuch_database_t **database,
-  char **status_string)
-{
-return notmuch_database_open_with_config (path, mode, "", NULL,
- database, status_string);
-}
-
-notmuch_status_t
-notmuch_database_open_with_config (const char *database_path,
-  notmuch_database_mode_t mode,
-  const char *config_path,
-  const char *profile,
-  notmuch_database_t **database,
-  char **status_string)
-{
-notmuch_status_t status = NOTMUCH_STATUS_SUCCESS;
-void *local = talloc_new (NULL);
-notmuch_database_t *notmuch = NULL;
-char *notmuch_path, *xapian_path, *incompat_features;
-char *configured_database_path = NULL;
-char *message = NULL;
-struct stat st;
-int err;
-unsigned version;
-GKeyFile *key_file = NULL;
-static int initialized = 0;
-
-/* XXX TODO: default locations for NULL case, handle profiles */
-if (config_path != NULL && ! EMPTY_STRING (config_path)) {
-   key_file = g_key_file_new ();
-   if (! g_key_file_load_from_file (key_file, config_path, 
G_KEY_FILE_NONE, NULL)) {
-   status = NOTMUCH_STATUS_FILE_ERROR;
-   goto DONE;
-   }
-   configured_database_path = g_key_file_get_value (key_file, "database", 
"path", NULL);
-}
-
-if (database_path == NULL)
-   database_path = configured_database_path;
-
-if (database_path == NULL) {
-   message = strdup ("Error: Cannot open a database for a NULL path.\n");
-   status = NOTMUCH_STATUS_NULL_POINTER;
-   goto DONE;
-}
-
-if (database_path[0] != '/') {
-   message = strdup ("Error: Database path must be absolute.\n");
-   status = NOTMUCH_STATUS_PATH_ERROR;
-   goto DONE;
-}
-
-if (! (notmuch_path = talloc_asprintf (local, "%s/%s", database_path, 
".notmuch"))) {
-   message = strdup ("Out of memory\n");
-   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
-   goto DONE;
-}
-
-err = stat (notmuch_path, );
-if (err) {
-   IGNORE_RESULT (asprintf (, "Error opening database at %s: %s\n",
-notmuch_path, strerror (errno)));
-   status = NOTMUCH_STATUS_FILE_ERROR;
-   goto DONE;
-}
-
-if (! (xapian_path = talloc_asprintf (local, "%s/%s", notmuch_path, 
"xapian"))) {
-   message = strdup ("Out of memory\n");
-   status = NOTMUCH_STATUS_OUT_OF_MEMORY;
-   goto DONE;
-}
-
-/* Initialize the GLib type system and threads */
-#if ! 

[PATCH 17/19] cli/config: add accessor for config file name

2020-08-08 Thread David Bremner
This is intended for use in temporary code transitioning to the new
configuration system. The name is chosen to avoid cluttering the
notmuch_config_* namespace further with non-library functions.
---
 notmuch-client.h | 2 ++
 notmuch-config.c | 3 +++
 2 files changed, 5 insertions(+)

diff --git a/notmuch-client.h b/notmuch-client.h
index ebd43e8d..2ec84678 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -332,6 +332,8 @@ void
 notmuch_config_set_search_exclude_tags (notmuch_config_t *config,
const char *list[],
size_t length);
+const char *
+_notmuch_config_get_path (notmuch_config_t *config);
 
 int
 notmuch_run_hook (const char *db_path, const char *hook);
diff --git a/notmuch-config.c b/notmuch-config.c
index 19c2ddb3..c7b05d8c 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -510,6 +510,9 @@ notmuch_config_close (notmuch_config_t *config)
 talloc_free (config);
 }
 
+const char *_notmuch_config_get_path (notmuch_config_t *config) {
+return config->filename;
+}
 /* Save any changes made to the notmuch configuration.
  *
  * Any comments originally in the file will be preserved.
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 15/19] WIP: add config values iterator

2020-08-08 Thread David Bremner
---
 lib/config.cc | 49 +
 lib/notmuch.h | 17 +
 2 files changed, 66 insertions(+)

diff --git a/lib/config.cc b/lib/config.cc
index ca7ac2a8..dd042ab2 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -31,6 +31,11 @@ struct _notmuch_config_list {
 char *current_val;
 };
 
+struct _notmuch_config_values {
+const char *iterator;
+size_t tok_len;
+};
+
 static int
 _notmuch_config_list_destroy (notmuch_config_list_t *list)
 {
@@ -232,6 +237,50 @@ notmuch_config_get (notmuch_database_t *notmuch, const 
char *key) {
 return _notmuch_string_map_get (notmuch->config, key);
 }
 
+notmuch_config_values_t *
+notmuch_config_get_values (notmuch_database_t *notmuch, const char *key)
+{
+notmuch_config_values_t *values;
+
+const char *str;
+str  = _notmuch_string_map_get (notmuch->config, key);
+if (! str)
+   return NULL;
+
+values = talloc (notmuch, notmuch_config_values_t);
+if (unlikely(! values))
+   return NULL;
+
+values->iterator = str;
+values->tok_len = 0;
+return values;
+}
+
+notmuch_bool_t
+notmuch_config_values_valid (notmuch_config_values_t *values) {
+if (! values)
+   return false;
+
+return (values->iterator != NULL);
+}
+
+const char *
+notmuch_config_values_get (notmuch_config_values_t *values) {
+return talloc_strndup (values, values->iterator, values->tok_len);
+}
+
+void
+notmuch_config_values_move_to_next (notmuch_config_values_t *values) {
+values->iterator += values->tok_len;
+values->iterator = strsplit_len (values->iterator, ';', 
&(values->tok_len));
+}
+
+void
+notmuch_config_values_destroy (notmuch_config_values_t *values) {
+talloc_free (values);
+}
+
+
 notmuch_status_t
 notmuch_config_set (notmuch_database_t *notmuch,
const char *key,
diff --git a/lib/notmuch.h b/lib/notmuch.h
index e147d5e6..7ae69265 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -236,6 +236,7 @@ typedef struct _notmuch_tags notmuch_tags_t;
 typedef struct _notmuch_directory notmuch_directory_t;
 typedef struct _notmuch_filenames notmuch_filenames_t;
 typedef struct _notmuch_config_list notmuch_config_list_t;
+typedef struct _notmuch_config_values notmuch_config_values_t;
 typedef struct _notmuch_indexopts notmuch_indexopts_t;
 #endif /* __DOXYGEN__ */
 
@@ -2405,6 +2406,22 @@ notmuch_config_set (notmuch_database_t *notmuch, const 
char *key,
const char *val,
notmuch_bool_t write_through);
 
+
+notmuch_config_values_t *
+notmuch_config_get_values (notmuch_database_t *notmuch, const char *key);
+
+notmuch_bool_t
+notmuch_config_values_valid (notmuch_config_values_t *values);
+
+const char *
+notmuch_config_values_get (notmuch_config_values_t *values);
+
+void
+notmuch_config_values_move_to_next (notmuch_config_values_t *values);
+
+void
+notmuch_config_values_destroy (notmuch_config_values_t *values);
+
 /**
  * get the current default indexing options for a given database.
  *
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 18/19] WIP converting notmuch search to new style config

2020-08-08 Thread David Bremner
this breaks passing the config file as a command line argument, but
that is not currently tested for notmuch-search
---
 notmuch-search.c | 45 +++--
 1 file changed, 27 insertions(+), 18 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 2805d960..4a5a57c9 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -52,6 +52,7 @@ typedef enum {
 
 typedef struct {
 notmuch_database_t *notmuch;
+void *talloc_ctx;
 int format_sel;
 sprinter_t *format;
 int exclude;
@@ -677,28 +678,32 @@ do_search_tags (const search_context_t *ctx)
 }
 
 static int
-_notmuch_search_prepare (search_context_t *ctx, notmuch_config_t *config, int 
argc, char *argv[])
+_notmuch_search_prepare (search_context_t *ctx,
+const char  *config_path,
+int argc, char *argv[])
 {
 char *query_str;
-unsigned int i;
 char *status_string = NULL;
 
+if (! ctx->talloc_ctx)
+   ctx->talloc_ctx = talloc_new (NULL);
+
 switch (ctx->format_sel) {
 case NOTMUCH_FORMAT_TEXT:
-   ctx->format = sprinter_text_create (config, stdout);
+   ctx->format = sprinter_text_create (ctx->talloc_ctx, stdout);
break;
 case NOTMUCH_FORMAT_TEXT0:
if (ctx->output == OUTPUT_SUMMARY) {
fprintf (stderr, "Error: --format=text0 is not compatible with 
--output=summary.\n");
return EXIT_FAILURE;
}
-   ctx->format = sprinter_text0_create (config, stdout);
+   ctx->format = sprinter_text0_create (ctx->talloc_ctx, stdout);
break;
 case NOTMUCH_FORMAT_JSON:
-   ctx->format = sprinter_json_create (config, stdout);
+   ctx->format = sprinter_json_create (ctx->talloc_ctx, stdout);
break;
 case NOTMUCH_FORMAT_SEXP:
-   ctx->format = sprinter_sexp_create (config, stdout);
+   ctx->format = sprinter_sexp_create (ctx->talloc_ctx, stdout);
break;
 default:
/* this should never happen */
@@ -707,9 +712,12 @@ _notmuch_search_prepare (search_context_t *ctx, 
notmuch_config_t *config, int ar
 
 notmuch_exit_if_unsupported_format ();
 
-if (notmuch_database_open_verbose (
-   notmuch_config_get_database_path (config),
-   NOTMUCH_DATABASE_MODE_READ_ONLY, >notmuch, _string)) {
+if (notmuch_database_open_with_config (
+   NULL,
+   NOTMUCH_DATABASE_MODE_READ_ONLY,
+   config_path,
+   NULL,
+   >notmuch, _string)) {
 
if (status_string) {
fputs (status_string, stderr);
@@ -748,21 +756,20 @@ _notmuch_search_prepare (search_context_t *ctx, 
notmuch_config_t *config, int ar
 }
 
 if (ctx->exclude != NOTMUCH_EXCLUDE_FALSE) {
-   const char **search_exclude_tags;
-   size_t search_exclude_tags_length;
+   notmuch_config_values_t *exclude_tags;
notmuch_status_t status;
 
-   search_exclude_tags = notmuch_config_get_search_exclude_tags (
-   config, _exclude_tags_length);
+   for (exclude_tags = notmuch_config_get_values (ctx->notmuch, 
"search.exclude_tags");
+notmuch_config_values_valid (exclude_tags);
+notmuch_config_values_move_to_next (exclude_tags)) {
 
-   for (i = 0; i < search_exclude_tags_length; i++) {
-   status = notmuch_query_add_tag_exclude (ctx->query, 
search_exclude_tags[i]);
+   status = notmuch_query_add_tag_exclude (ctx->query,
+   notmuch_config_values_get 
(exclude_tags));
if (status && status != NOTMUCH_STATUS_IGNORED) {
print_status_query ("notmuch search", ctx->query, status);
return EXIT_FAILURE;
}
}
-
notmuch_query_set_omit_excluded (ctx->query, ctx->exclude);
 }
 
@@ -845,7 +852,8 @@ notmuch_search_command (notmuch_config_t *config, int argc, 
char *argv[])
return EXIT_FAILURE;
 }
 
-if (_notmuch_search_prepare (ctx, config,
+if (_notmuch_search_prepare (ctx,
+_notmuch_config_get_path (config),
 argc - opt_index, argv + opt_index))
return EXIT_FAILURE;
 
@@ -911,7 +919,8 @@ notmuch_address_command (notmuch_config_t *config, int 
argc, char *argv[])
return EXIT_FAILURE;
 }
 
-if (_notmuch_search_prepare (ctx, config,
+if (_notmuch_search_prepare (ctx,
+_notmuch_config_get_path (config),
 argc - opt_index, argv + opt_index))
return EXIT_FAILURE;
 
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH 07/19] test/libconfig; use n_database_open_with_config

2020-08-08 Thread David Bremner
This allows testing the "override database config with file
functionality". It also requires passing a config file explicitly to
each test program.
---
 test/T590-libconfig.sh | 21 +
 1 file changed, 13 insertions(+), 8 deletions(-)

diff --git a/test/T590-libconfig.sh b/test/T590-libconfig.sh
index c2bce4a2..a34eae0b 100755
--- a/test/T590-libconfig.sh
+++ b/test/T590-libconfig.sh
@@ -16,7 +16,12 @@ int main (int argc, char** argv)
char *val;
notmuch_status_t stat;
 
-   EXPECT0(notmuch_database_open (argv[1], NOTMUCH_DATABASE_MODE_READ_WRITE, 
));
+   EXPECT0(notmuch_database_open_with_config (argv[1],
+  NOTMUCH_DATABASE_MODE_READ_WRITE,
+  argv[2],
+  NULL,
+  ,
+  NULL));
 
 EOF
 
@@ -26,7 +31,7 @@ cat < c_tail
 EOF
 
 test_begin_subtest "notmuch_database_{set,get}_config"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
EXPECT0(notmuch_database_set_config (db, "test.key1", "testvalue1"));
EXPECT0(notmuch_database_set_config (db, "test.key2", "testvalue2"));
@@ -46,7 +51,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 
 test_begin_subtest "notmuch_database_get_config_list: empty list"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
notmuch_config_list_t *list;
EXPECT0(notmuch_database_get_config_list (db, "nonexistent", ));
@@ -78,7 +83,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_database_get_config_list: all pairs"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
notmuch_config_list_t *list;
EXPECT0(notmuch_database_set_config (db, "zzzafter", "afterval"));
@@ -142,7 +147,7 @@ EOF
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "dump config"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
 EXPECT0(notmuch_database_set_config (db, "key with spaces", "value, with, 
spaces!"));
 }
@@ -160,7 +165,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "restore config"
 notmuch dump --include=config >EXPECTED
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
 EXPECT0(notmuch_database_set_config (db, "test.key1", "mutatedvalue"));
 }
@@ -170,7 +175,7 @@ notmuch dump --include=config >OUTPUT
 test_expect_equal_file EXPECTED OUTPUT
 
 test_begin_subtest "notmuch_config_get"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
printf("test.key1 = %s\n", notmuch_config_get (db, "test.key1"));
printf("test.key2 = %s\n", notmuch_config_get (db, "test.key2"));
@@ -186,7 +191,7 @@ test_expect_equal_file EXPECTED OUTPUT
 
 backup_database
 test_begin_subtest "notmuch_config_set"
-cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR}
+cat c_head - c_tail <<'EOF' | test_C ${MAIL_DIR} ${NOTMUCH_CONFIG}
 {
char *val;
printf("test.key1 = %s\n", notmuch_config_get (db, "test.key1"));
-- 
2.28.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Initial implementation of merged config

2020-08-08 Thread David Bremner
This follow-up to

 id:20200703134338.3311659-1-da...@tethera.net

is a first draft of a rewrite of the notmuch configuration system. It
aims to remove several annoyances including:

- not being able to have certain configuration items in a plain text config file
- bindings support for configuration being somewhat half-baked
- not supporting XDG_foo

Not all of that promise is realized (or at least tested) in this
initial draft. On the other hand, hopefully a bunch of new tests in
T590-libconfig.sh make the intended API more clear. 

This patch implements the logic for finding config files:

 [PATCH 12/19] WIP: adding fallbacks for NULL config_path

Something similar needs to be done for finding database files.

In the last two commands you can see roughly how much
work is needed to convert to the style of config handling

 [PATCH 18/19] WIP converting notmuch search to new style config
 [PATCH 19/19] WIP: switch notmuch-show to new config framework

The is some awkardness due to converting one subcommand at a time: the
subcommand table uses a fixed function signature and we can't change
that until all the subcommands are updated. So we end up re-opening the config 
file in the modified subcommands.

Some design questions people might like to give feedback on

- the switch in notmuch-search.c to using the string
  "search.exclude_tags" directly to fetch tags.  I'm hoping that a
  reduction in boilerplate compensates for a decrease in "type safety"
  here, compared to the existing
  notmuch_config_get_search_exclude_tags. I'm hoping substantial
  portions of notmuch-config.c can disappear during this rewrite.

- the use of simple keys for the new merged notmuch_config_{get,set}_*
  in lib/config.cc, rather than the section, pair file suggested by the glib 
key_file format. 


My rough plan is to do a release 0.31 with all the cleanup we have
accumulated since 0.30, and target these changes for 0.32
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2 06/23] emacs: Use cl-incf where appropriate

2020-08-07 Thread David Bremner
Tomi Ollila  writes:

> NEWS for notmuch 0.29 says that:
>
> "
> The minimum supported major version of Emacs is now 24.
>
> Support for GNU Emacs older than 25.1 is deprecated with this release,
> and may be removed in a future release.
> "
>
> So we have 3 options:
>
> 1) apply this patch and now drop support for emacs 24
>
> 2) apply this patch and somehow infor emacs 24 users to install cl-lib from
> ELPA

I'm fine with either of these options. I'd hope we can specify what
versions we need in emacs/notmuch-pkg.el.templ

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v4] Emacs: Force left-to-right display for message headers

2020-08-06 Thread David Bremner
Teemu Likonen  writes:

> Insert invisible U+200E LEFT-TO-RIGHT MARK character at the beginning
> of message header paragraph in notmuch-show buffer. The U+200E
> character forces the header paragraph as left-to-right text even if
> the header content started with right-to-left characters.
>
> See Emacs Lisp reference manual section "(elisp) Bidirectional
> Display" for more info.

This causes 10 tests to fail for me. At a guess, the added LEFT-TO-RIGHT MARK
should probably be stripped out in the test framework. Either that or
added to test output files. The latter sounds easy to miss when editing.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH v2 06/23] emacs: Use cl-incf where appropriate

2020-08-06 Thread David Bremner
Jonas Bernoulli  writes:

> It's shorter.  That's it pretty much.
> ---
>  emacs/notmuch-hello.el | 2 +-
>  emacs/notmuch-lib.el   | 2 +-

"cl-incf is a Lisp macro in ‘aquamacs-tools.el’."

What the actual heck?

Anyway, that aside, what version of emacs is this available from?

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


[PATCH] lib/config: delay setting talloc destructor

2020-08-04 Thread David Bremner
If Xapian has thrown an exception, it is not safe to invoke the
destuctor when freeing the list struct.
---

this is a latent exception handling bug I missed before. It showed up
when updating some config related code, but I guess we were just lucky(?)
before. The bug manifests as one of the tests in T590-libconfig failing.

 lib/config.cc | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/lib/config.cc b/lib/config.cc
index f5def3aa..ca7ac2a8 100644
--- a/lib/config.cc
+++ b/lib/config.cc
@@ -113,7 +113,6 @@ notmuch_database_get_config_list (notmuch_database_t 
*notmuch,
goto DONE;
 }
 
-talloc_set_destructor (list, _notmuch_config_list_destroy);
 list->notmuch = notmuch;
 list->current_key = NULL;
 list->current_val = NULL;
@@ -133,8 +132,15 @@ notmuch_database_get_config_list (notmuch_database_t 
*notmuch,
 *out = list;
 
   DONE:
-if (status && list)
-   talloc_free (list);
+if (status) {
+   if (list) {
+   talloc_free (list);
+   if (status != NOTMUCH_STATUS_XAPIAN_EXCEPTION)
+   _notmuch_config_list_destroy (list);
+   }
+}  else {
+   talloc_set_destructor (list, _notmuch_config_list_destroy);
+}
 
 return status;
 }
-- 
2.27.0
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


Re: [PATCH] build: upload html docs as part of release process

2020-08-03 Thread David Bremner
David Bremner  writes:

>  Oops. Of course as soon as I send it I see a flaw. It's hard to test
>  this stuff without doing another release; in some sense this patch is
>  a reminder to me to debug it next time I do a release.

Applied to master.

d
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org


  1   2   3   4   5   6   7   8   9   10   >