v2 sexpr parser

This is a substantially revised version of the series at [1]. As far as I know, it now understands (a translation of) most of the queries handled by the existing query parser. Some remaining limitations/issues 1) The new query parser is only hooked into the notmuch search subcommand. It should be

[PATCH 19/25] lib/parse-sexp: add support for regexp fields

This commit connects the previously added keyword / flag handling with the previously refactored regexp to query refactoring. --- lib/parse-sexp.cc | 35 +++ lib/regexp-fields.h | 8 +++--- test/T081-sexpr-search.sh | 59 +++

[PATCH 18/25] lib: factor out query construction from regexp

This will allow re-use of this code outside of the Xapian query parser. --- lib/regexp-fields.cc | 81 +++- lib/regexp-fields.h | 6 2 files changed, 63 insertions(+), 24 deletions(-) diff --git a/lib/regexp-fields.cc b/lib/regexp-fields.cc index

[PATCH 11/25] lib: factor out date to query conversion

This is a bit messy, but throwing and catching Xapian::QueryParserError exceptions outside of the Xapian query parser seems worse. --- lib/parse-time-vrp.cc | 97 +++ lib/parse-time-vrp.h | 8 2 files changed, 79 insertions(+), 26 deletions(-) diff

[PATCH 17/25] lib/query: generalize exclude handling to s-expression queries

In fact most of the code path is in common, only the caching of terms in the query needs to be added for s-expression queries. --- lib/query.cc | 34 +++--- test/T081-sexpr-search.sh | 37 + 2 files changed, 60

[PATCH 04/25] CLI/search+address: support sexpr queries

Initially support selection of query syntax in two subcommands to enable testing. --- notmuch-search.c | 16 +++- test/T080-search.sh | 5 + test/T095-address.sh | 5 + 3 files changed, 25 insertions(+), 1 deletion(-) diff --git a/notmuch-search.c b/notmuch-search.c

[PATCH 01/25] configure: optional library sfsexp

This is essentially the same as the other checks using pkg-config. --- configure | 23 ++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/configure b/configure index cfa9c09b..15c1924f 100755 --- a/configure +++ b/configure @@ -820,6 +820,19 @@ else

[PATCH 24/25] lib/parse-sexp: support infix subqueries

This is necessary so that programs can take infix syntax queries from a user and use the sexp query syntax to construct e.g. a refinement of that query. --- lib/parse-sexp.cc | 32 test/T081-sexpr-search.sh | 35 +++ 2 files

[PATCH 07/25] lib/parse-sexp: parse 'subject'

Initially this only works for lists of individual terms, which is a bit inconvenient. This will be improved in a following commit. --- lib/parse-sexp.cc | 37 + test/T081-sexpr-search.sh | 15 +++ 2 files changed, 52 insertions(+) diff

[PATCH 12/25] lib/parse-sexp: parse date fields

This is mainly just marshalling the arguments and calling the existing query generator. It's not obvious why date is in the field table now, but we will later use this to track what flags (options) are permitted per field. --- lib/parse-sexp.cc | 42 ++-

[PATCH 23/25] lib/parse-sexp: support thread subqueries.

This essentially mimics the ThreadFieldProcessor code, but generating the notmuch_query_t via the new notmuch_query_from_sexp using a subtree of the already parsed s-expression. --- lib/parse-sexp.cc | 27 +++ test/T081-sexpr-search.sh | 35

[PATCH 08/25] lib/parse-sexp: split terms in phrase mode

The goal is to have (subject foo-bar) match the same messages as subject:foo-bar. --- lib/parse-sexp.cc | 38 +- test/T081-sexpr-search.sh | 8 2 files changed, 41 insertions(+), 5 deletions(-) diff --git a/lib/parse-sexp.cc

[PATCH 21/25] lib: define _notmuch_query_from_sexp

This will be used in a following commit to provide the equivalant of thread:{} syntax for s-expression queries. --- lib/parse-sexp.cc | 13 + lib/parse-sexp.h | 14 -- lib/query.cc | 23 ++- 3 files changed, 39 insertions(+), 11 deletions(-) diff

[PATCH 16/25] lib/parse-sexp: initial support for wildcard queries

Because of the implicit way body: queries are implemented, it is not clear how to support (body :any) queries with current Xapian. --- lib/parse-sexp.cc | 18 +++--- test/T081-sexpr-search.sh | 50 +++ 2 files changed, 65 insertions(+), 3

[PATCH 15/25] lib/parse-sexp: add keyword arguments for fields

Nothing is done with the flags here, only parsing and validation. --- lib/parse-sexp.cc | 116 +++--- test/T081-sexpr-search.sh | 31 ++ 2 files changed, 127 insertions(+), 20 deletions(-) diff --git a/lib/parse-sexp.cc b/lib/parse-sexp.cc index

[PATCH 05/25] lib: add new status code for query syntax errors.

This will help provide more meaningful error messages without special casing on the client side. --- bindings/python-cffi/notmuch2/_build.py | 1 + bindings/python-cffi/notmuch2/_errors.py | 3 +++ lib/database.cc | 2 ++ lib/notmuch.h| 4

[PATCH 02/25] lib: split notmuch_query_create

Most of the function will be re-usable when creating a query from an s-expression. --- lib/query.cc | 19 --- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/lib/query.cc b/lib/query.cc index 792aba21..39b85e91 100644 --- a/lib/query.cc +++ b/lib/query.cc @@ -84,9

[PATCH 25/25] lib/parse-sexp: parse user headers

Rather than adding one prefix per user header, we create a single 'header' prefix that takes the configured name of the prefix as a parameter. --- lib/parse-sexp.cc | 50 +++ test/T081-sexpr-search.sh | 84 +++ 2 files changed, 134

[PATCH 10/25] lib/parse-sexp: handle unprefixed terms.

This is equivalent to adding the same field name "" for multiple prefixes in the Xapian query parser, but we have to explicitely construct the resulting query. --- lib/parse-sexp.cc | 15 --- test/T081-sexpr-search.sh | 27 +++ 2 files changed, 39

[PATCH 09/25] lib/parse-sexp: handle most fields

date and query will need to be handled separately because they do not fit the pattern of combining a bunch of terms with an operator. --- lib/parse-sexp.cc | 13 test/T081-sexpr-search.sh | 143 +- 2 files changed, 153 insertions(+), 3

[PATCH 14/25] lib/parse-sexp: handle saved queries

This provides functionality analogous to query: in the Xapian QueryParser based parser. Perhaps counterintuitively, the saved queries currently have to be in the original query syntax (i.e. not s-expressions). --- lib/parse-sexp.cc | 10 ++ test/T081-sexpr-search.sh | 28

[PATCH 06/25] lib/parse-sexp: parse 'and', 'not', 'or'

This is not too useful yet, but provides gluing together queries in various ways, which is actually one of the main motivations for the s-expression query format. --- lib/Makefile.local| 3 +- lib/parse-sexp.cc | 92 +++ lib/parse-sexp.h

[PATCH 22/25] lib: generate actual Xapian query for "*" and ""

The previous code had the somewhat bizarre effect that the (notmuch specific) query string was "*" (interpreted as MatchAll) and the allegedly parsed xapian_query was "MatchNothing". This commit also reduces code duplication. --- lib/query.cc | 34 ++ 1 file

[PATCH 20/25] lib/thread-fp: factor out query expansion

The non-obvious choice of creating the query outside the factored-out code is so the the new function can be used with different query syntaxes. --- lib/database-private.h | 5 + lib/query.cc | 26 ++ lib/thread-fp.cc | 25 + 3

[PATCH 03/25] lib: define notmuch_query_create_sexpr

Set the parsing syntax when the (notmuch) query object is created. Initially the library always returns a trivial query that matches all messages. --- lib/notmuch.h | 4 lib/query.cc | 52 +++ 2 files changed, 52 insertions(+), 4 deletions(-)

[PATCH 13/25] lib: factor out expansion of saved queries.

This is intended to allow use outside of the Xapian query parser. --- lib/query-fp.cc | 22 +++--- lib/query-fp.h | 4 2 files changed, 23 insertions(+), 3 deletions(-) diff --git a/lib/query-fp.cc b/lib/query-fp.cc index b980b7f0..75b1d875 100644 --- a/lib/query-fp.cc +++

[PATCH] emacs: Refine scope of notmuch-jump-key face

The intent of the 'notmuch-jump-key' face is to allow users/themes to differentiate the text of the minibuffer prompt from the keys that are associated with jump actions. Commit 5cc106b0 correctly introduced the 'notmuch-jump-key' face for keys, but mistakenly applied it to the prompt as well.