Re: [Paul Wise] Bug#843127: notmuch: race condition in `notmuch new`?

2016-11-12 Thread Austin Clements
Quoth David Bremner on Nov 04 at 1:26 pm: > > Paul Wise wrote: > > > Last night I got this errorĀ from my `notmuch new --quiet` cron job. The > > file that the error message complains about is now in the cur directory > > of the maildir at the following path. > > > >

Re: searching: '*analysis' vs 'reanalysis'

2016-06-06 Thread Austin Clements
Quoth Gaute Hope on Jun 06 at 8:08 pm: > Austin Clements writes on juni 6, 2016 21:20: > > > >The experiment was specifically for regexp matching subject, but it should > >work for any header we store a literal copy of in the database. > > Does it work for terms

Re: searching: '*analysis' vs 'reanalysis'

2016-06-06 Thread Austin Clements
On Mon, Jun 6, 2016 at 1:29 PM, David Bremner wrote: > Sebastian Fischmeister writes: > > > > > I ran into this problem before as well. Storage is cheap. Notmuch could > > index all emails with reversed text to get around some of this > > problem. It

Re: [PATCH v4 7/7] complete ghost-on-removal-when-shared-thread-exists

2016-04-19 Thread Austin Clements
On Mon, 11 Apr 2016, Daniel Kahn Gillmor wrote: > On Sun 2016-04-10 20:33:02 -0400, David Bremner wrote: >> Daniel Kahn Gillmor writes: >> >>> To fully complete the ghost-on-removal-when-shared-thread-exists >>> proposal, we

[PATCH 2/6] lib: Add per-message last modification tracking

2015-08-08 Thread Austin Clements
Hi David. I haven't had a chance to look back at the original code, but your follow-up expanded comment agrees with how I remember this code working. On Aug 7, 2015 4:41 PM, "David Bremner" wrote: > Daniel Schoepe writes: > > > > On Fri, 05 Jun 2015 19:28 +0200, David Bremner wrote: > >> +

Re: [PATCH 2/6] lib: Add per-message last modification tracking

2015-08-08 Thread Austin Clements
Hi David. I haven't had a chance to look back at the original code, but your follow-up expanded comment agrees with how I remember this code working. On Aug 7, 2015 4:41 PM, David Bremner da...@tethera.net wrote: Daniel Schoepe dan...@schoepe.org writes: On Fri, 05 Jun 2015 19:28 +0200,

Understanding the "replied" tag

2015-06-13 Thread Austin Clements
Hi Xu. I may be misunderstanding your email, but it sounds like you want to know if a message has *any* reply message. That's not what the replied tag indicates. The replied tag indicates that *you* have sent a reply to a message. Mechanically, when you hit, say, r to start a reply and then

Re: Understanding the replied tag

2015-06-13 Thread Austin Clements
Hi Xu. I may be misunderstanding your email, but it sounds like you want to know if a message has *any* reply message. That's not what the replied tag indicates. The replied tag indicates that *you* have sent a reply to a message. Mechanically, when you hit, say, r to start a reply and then

notmuch_thread_get_authors

2015-04-21 Thread Austin Clements
Quoth Ronny Chevalier on Apr 22 at 4:01 am: > On Wed, Apr 22, 2015 at 3:28 AM, Austin Clements > wrote: > > On Tue, 21 Apr 2015, Ronny Chevalier wrote: > >> On Tue, Apr 21, 2015 at 1:35 AM, David Bremner > >> wrote: > >>> Ronny Chevalier writes: >

notmuch_thread_get_authors

2015-04-21 Thread Austin Clements
On Tue, 21 Apr 2015, Ronny Chevalier wrote: > On Tue, Apr 21, 2015 at 1:35 AM, David Bremner wrote: >> Ronny Chevalier writes: > Austin Clements wrote: >> And I think there's a fairly easy way to do it in C code that will >> also prevent library interface bloat: in

Re: notmuch_thread_get_authors

2015-04-21 Thread Austin Clements
Quoth Ronny Chevalier on Apr 22 at 4:01 am: On Wed, Apr 22, 2015 at 3:28 AM, Austin Clements acleme...@csail.mit.edu wrote: On Tue, 21 Apr 2015, Ronny Chevalier chevalier.ro...@gmail.com wrote: On Tue, Apr 21, 2015 at 1:35 AM, David Bremner da...@tethera.net wrote: Ronny Chevalier

Re: notmuch_thread_get_authors

2015-04-21 Thread Austin Clements
On Tue, 21 Apr 2015, Ronny Chevalier chevalier.ro...@gmail.com wrote: On Tue, Apr 21, 2015 at 1:35 AM, David Bremner da...@tethera.net wrote: Ronny Chevalier chevalier.ro...@gmail.com writes: Austin Clements wrote: And I think there's a fairly easy way to do it in C code that will also

[PATCH v2 8/8] emacs: Support cid: references with shr renderer

2015-01-24 Thread Austin Clements
shr has really nice support for inline image rendering, but previously we only had the hooks for w3m cid: references. --- emacs/notmuch-show.el | 45 + 1 file changed, 37 insertions(+), 8 deletions(-) diff --git a/emacs/notmuch-show.el

[PATCH v2 7/8] emacs: Rewrite content ID handling

2015-01-24 Thread Austin Clements
Besides generally cleaning up the code and separating the general content ID handling from the w3m-specific code, this fixes several problems. Foremost is that, previously, the code roughly assumed that referenced parts would be in the same multipart/related as the reference. According to RFC

[PATCH v2 6/8] emacs: Use generalized content caching in w3m CID code

2015-01-24 Thread Austin Clements
Previously this did its own caching, but this is now supported by more generally by `notmuch-get-bodypart-binary'. --- emacs/notmuch-show.el | 23 --- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index

[PATCH v2 5/8] emacs: Support caching in notmuch-get-bodypart-{binary, text}

2015-01-24 Thread Austin Clements
(The actual code change here is small, but requires re-indenting existing code.) --- emacs/notmuch-lib.el | 64 1 file changed, 40 insertions(+), 24 deletions(-) diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el index 3154725..f8e5165

[PATCH v2 4/8] emacs: Return unibyte strings for binary part data

2015-01-24 Thread Austin Clements
Unibyte strings are meant for representing binary data. In practice, using unibyte versus multibyte strings affects *almost* nothing. It does happen to matter if we use the binary data in an image descriptor (which is, helpfully, not documented anywhere and getting it wrong results in opaque

[PATCH v2 2/8] emacs: Create an API for fetching parts as undecoded binary

2015-01-24 Thread Austin Clements
The new function, `notmuch-get-bodypart-binary', replaces `notmuch-get-bodypart-internal'. Whereas the old function was really meant for internal use in `notmuch-get-bodypart-content', it was used in a few other places. Since the difference between `notmuch-get-bodypart-content' and

[PATCH v2 1/8] emacs: Track full message and part descriptor in w3m CID store

2015-01-24 Thread Austin Clements
This will simplify later changes. --- emacs/notmuch-show.el | 33 ++--- 1 file changed, 10 insertions(+), 23 deletions(-) diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index 87b4881..df2389e 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el

[PATCH v2 0/8] Improve charset and cid: handling

2015-01-24 Thread Austin Clements
This is v2 of id:1398105468-14317-1-git-send-email-amdragon at mit.edu. This improves some comments/documentation, fixes a bug that caused cryptographic processing to not happen on HTML parts, and addresses some byte compiler warnings on Emacs 23. This version has also been rebased against the

[PATCH 08/11] emacs: Support caching in notmuch-get-bodypart-{binary, text}

2015-01-24 Thread Austin Clements
On Fri, 25 Apr 2014, Mark Walters wrote: > On Thu, 24 Apr 2014, Austin Clements wrote: >> Quoth Mark Walters on Apr 24 at 11:46 am: >>> >>> On Mon, 21 Apr 2014, Austin Clements wrote: >>> > (The actual code change here is small, but re

[PATCH 00/11] Improve charset and cid: handling

2015-01-24 Thread Austin Clements
> defined: libxml-parse-html-region, shr-insert-document > > Finally, I have not really tested it as I mainly use emacs23 > > Best wishes > > Mark > > > > > On Mon, 21 Apr 2014, Austin Clements wrote: >> I set out to quickly add support for cid: links

[PATCH 11/11] emacs: Support cid: references with shr renderer

2015-01-24 Thread Austin Clements
On Thu, 01 May 2014, David Edmondson wrote: > On Mon, Apr 21 2014, Austin Clements wrote: >> +(defun notmuch-show--insert-part-text/html-shr (msg part) >> + ;; Make sure shr is loaded before we start let-binding its globals >> + (require 'shr) >> + (let ((dom (let (

[PATCH 06/11] emacs: Remove broken `notmuch-get-bodypart-content' API

2015-01-24 Thread Austin Clements
On Fri, 11 Jul 2014, David Bremner wrote: > Austin Clements writes: > >> +This returns the content of the given part as a multibyte Lisp > > What does "multibyte" mean here? utf8? current encoding? Elisp has two kinds of stings: "unibyte strings" and "

[PATCH 04/11] emacs: Track full message and part descriptor in w3m CID store

2015-01-24 Thread Austin Clements
On Thu, 10 Jul 2014, David Bremner wrote: > Austin Clements writes: > >> This will simplify later changes. > > I'd have preferred the whitespace changes as a seperate patch, but OK. Not sure which whitespace changes you're referring to. Everything in this diff is actual (minor) code changes.

Re: [PATCH 08/11] emacs: Support caching in notmuch-get-bodypart-{binary, text}

2015-01-24 Thread Austin Clements
On Fri, 25 Apr 2014, Mark Walters markwalters1...@gmail.com wrote: On Thu, 24 Apr 2014, Austin Clements amdra...@mit.edu wrote: Quoth Mark Walters on Apr 24 at 11:46 am: On Mon, 21 Apr 2014, Austin Clements amdra...@mit.edu wrote: (The actual code change here is small, but requires re

Re: [PATCH 04/11] emacs: Track full message and part descriptor in w3m CID store

2015-01-24 Thread Austin Clements
On Thu, 10 Jul 2014, David Bremner da...@tethera.net wrote: Austin Clements amdra...@mit.edu writes: This will simplify later changes. I'd have preferred the whitespace changes as a seperate patch, but OK. Not sure which whitespace changes you're referring to. Everything in this diff

Re: [PATCH 11/11] emacs: Support cid: references with shr renderer

2015-01-24 Thread Austin Clements
On Thu, 01 May 2014, David Edmondson d...@dme.org wrote: On Mon, Apr 21 2014, Austin Clements wrote: +(defun notmuch-show--insert-part-text/html-shr (msg part) + ;; Make sure shr is loaded before we start let-binding its globals + (require 'shr) + (let ((dom (let (process-crypto notmuch

[PATCH v2 8/8] emacs: Support cid: references with shr renderer

2015-01-24 Thread Austin Clements
shr has really nice support for inline image rendering, but previously we only had the hooks for w3m cid: references. --- emacs/notmuch-show.el | 45 + 1 file changed, 37 insertions(+), 8 deletions(-) diff --git a/emacs/notmuch-show.el

[PATCH v2 2/8] emacs: Create an API for fetching parts as undecoded binary

2015-01-24 Thread Austin Clements
The new function, `notmuch-get-bodypart-binary', replaces `notmuch-get-bodypart-internal'. Whereas the old function was really meant for internal use in `notmuch-get-bodypart-content', it was used in a few other places. Since the difference between `notmuch-get-bodypart-content' and

[PATCH v2 3/8] emacs: Remove broken `notmuch-get-bodypart-content' API

2015-01-24 Thread Austin Clements
`notmuch-get-bodypart-content' could do two very different things, depending on conditions: for text/* parts other than text/html, it would return the part content as a multibyte Lisp string *after* charset conversion, while for other parts (including text/html), it would return binary part

[PATCH v2 4/8] emacs: Return unibyte strings for binary part data

2015-01-24 Thread Austin Clements
Unibyte strings are meant for representing binary data. In practice, using unibyte versus multibyte strings affects *almost* nothing. It does happen to matter if we use the binary data in an image descriptor (which is, helpfully, not documented anywhere and getting it wrong results in opaque

[PATCH v2 0/8] Improve charset and cid: handling

2015-01-24 Thread Austin Clements
This is v2 of id:1398105468-14317-1-git-send-email-amdra...@mit.edu. This improves some comments/documentation, fixes a bug that caused cryptographic processing to not happen on HTML parts, and addresses some byte compiler warnings on Emacs 23. This version has also been rebased against the

[PATCH v2 6/8] emacs: Use generalized content caching in w3m CID code

2015-01-24 Thread Austin Clements
Previously this did its own caching, but this is now supported by more generally by `notmuch-get-bodypart-binary'. --- emacs/notmuch-show.el | 23 --- 1 file changed, 8 insertions(+), 15 deletions(-) diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index

[PATCH v2 1/8] emacs: Track full message and part descriptor in w3m CID store

2015-01-24 Thread Austin Clements
This will simplify later changes. --- emacs/notmuch-show.el | 33 ++--- 1 file changed, 10 insertions(+), 23 deletions(-) diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el index 87b4881..df2389e 100644 --- a/emacs/notmuch-show.el +++ b/emacs/notmuch-show.el

[PATCH v2 5/8] emacs: Support caching in notmuch-get-bodypart-{binary, text}

2015-01-24 Thread Austin Clements
(The actual code change here is small, but requires re-indenting existing code.) --- emacs/notmuch-lib.el | 64 1 file changed, 40 insertions(+), 24 deletions(-) diff --git a/emacs/notmuch-lib.el b/emacs/notmuch-lib.el index 3154725..f8e5165

Re: [PATCH 06/11] emacs: Remove broken `notmuch-get-bodypart-content' API

2015-01-24 Thread Austin Clements
On Fri, 11 Jul 2014, David Bremner da...@tethera.net wrote: Austin Clements amdra...@mit.edu writes: +This returns the content of the given part as a multibyte Lisp What does multibyte mean here? utf8? current encoding? Elisp has two kinds of stings: unibyte strings and multibyte strings

Re: [PATCH 00/11] Improve charset and cid: handling

2015-01-24 Thread Austin Clements
to be defined: libxml-parse-html-region, shr-insert-document Finally, I have not really tested it as I mainly use emacs23 Best wishes Mark On Mon, 21 Apr 2014, Austin Clements amdra...@mit.edu wrote: I set out to quickly add support for cid: links in the shr renderer and wound up making our

privacy problem: text/html parts pull in network resources

2015-01-21 Thread Austin Clements
Quoth Daniel Kahn Gillmor on Jan 21 at 4:36 pm: > On Wed 2015-01-21 16:14:07 -0500, Austin Clements wrote: > > I have a fix for this on shr buried deep in an old patch series that I > > never got back to: id:1398105468-14317-12-git-send-email-amdragon at mit.edu > > > >

privacy problem: text/html parts pull in network resources

2015-01-21 Thread Austin Clements
I have a fix for this on shr buried deep in an old patch series that I never got back to: id:1398105468-14317-12-git-send-email-amdragon at mit.edu For shr, the key is to set shr-blocked-images to ".". However, IIRC, in the current notmuch message rendering pipeline, mm overrides this variable

Re: privacy problem: text/html parts pull in network resources

2015-01-21 Thread Austin Clements
I have a fix for this on shr buried deep in an old patch series that I never got back to: id:1398105468-14317-12-git-send-email-amdra...@mit.edu For shr, the key is to set shr-blocked-images to .. However, IIRC, in the current notmuch message rendering pipeline, mm overrides this variable with

Re: privacy problem: text/html parts pull in network resources

2015-01-21 Thread Austin Clements
Quoth Daniel Kahn Gillmor on Jan 21 at 4:36 pm: On Wed 2015-01-21 16:14:07 -0500, Austin Clements wrote: I have a fix for this on shr buried deep in an old patch series that I never got back to: id:1398105468-14317-12-git-send-email-amdra...@mit.edu For shr, the key is to set shr-blocked

[WIP PATCH 4/4] lib: Add "lastmod:" queries for filtering by last modification

2015-01-15 Thread Austin Clements
Quoth David Bremner on Jan 15 at 10:08 pm: > Austin Clements writes: > > > From: Austin Clements > > > > XXX Includes reference to notmuch search --db-revision, which doesn't > > exist. > > What would --db-revision do if it was implimented? Did you wan

[PATCH v2 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

2015-01-15 Thread Austin Clements
Just one nit. Otherwise this patch LGTM. On January 15, 2015 12:20:08 PM EST, Jani Nikula wrote: > >Austin, would you mind having a look at this one please? > >Thanks, >Jani. > >On Wed, 14 Jan 2015, Todd wrote: >> --- >> lib/database-private.h | 15 --- >> lib/database.cc|

Re: [PATCH v2 2/5] Add the NOTMUCH_FEATURE_INDEXED_MIMETYPES database feature

2015-01-15 Thread Austin Clements
Just one nit. Otherwise this patch LGTM. On January 15, 2015 12:20:08 PM EST, Jani Nikula j...@nikula.org wrote: Austin, would you mind having a look at this one please? Thanks, Jani. On Wed, 14 Jan 2015, Todd t...@electricoding.com wrote: --- lib/database-private.h | 15 ---

Re: [WIP PATCH 4/4] lib: Add lastmod: queries for filtering by last modification

2015-01-15 Thread Austin Clements
Quoth David Bremner on Jan 15 at 10:08 pm: Austin Clements acleme...@csail.mit.edu writes: From: Austin Clements amdra...@mit.edu XXX Includes reference to notmuch search --db-revision, which doesn't exist. What would --db-revision do if it was implimented? Did you want to pass

Negating searches in notmuch running via emacs?

2015-01-06 Thread Austin Clements
Hi David. In general, "-" will negate a query term. However, Xapian's query syntax (to which notmuch is beholden) ignores "-" at the beginning of a query (and immediately after an open paren). Hence, if you want to negate the first term in a query, you'll have to use "not" instead. Hope that

Re: Negating searches in notmuch running via emacs?

2015-01-06 Thread Austin Clements
Hi David. In general, - will negate a query term. However, Xapian's query syntax (to which notmuch is beholden) ignores - at the beginning of a query (and immediately after an open paren). Hence, if you want to negate the first term in a query, you'll have to use not instead. Hope that helps.

[PATCH] NEWS: Database version 3, API improvements, and ghost messages

2014-11-10 Thread Austin Clements
--- NEWS | 30 ++ 1 file changed, 30 insertions(+) diff --git a/NEWS b/NEWS index b30ed1b..7a121e4 100644 --- a/NEWS +++ b/NEWS @@ -36,9 +36,39 @@ Improved `q` binding in notmuch buffers Library changes --- +Introduced database version 3 with support

[PATCH] NEWS: Database version 3, API improvements, and ghost messages

2014-11-10 Thread Austin Clements
--- NEWS | 30 ++ 1 file changed, 30 insertions(+) diff --git a/NEWS b/NEWS index b30ed1b..7a121e4 100644 --- a/NEWS +++ b/NEWS @@ -36,9 +36,39 @@ Improved `q` binding in notmuch buffers Library changes --- +Introduced database version 3 with support

[PATCH v3.1 3/9] lib: Introduce macros for bit operations

2014-10-24 Thread Austin Clements
These macros help clarify basic bit-twiddling code and are written to be robust against C undefined behavior of shift operators. --- lib/message.cc| 6 +++--- lib/notmuch-private.h | 11 +++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/lib/message.cc

[PATCH v1 1/3] search: Seperately report matching and non-matching authors.

2014-10-24 Thread Austin Clements
Quoth Mark Walters on Oct 24 at 10:23 am: > > Hi > > I definitely like the idea: some comments below. Agreed. > On Fri, 24 Oct 2014, David Edmondson wrote: > > In addition to the :authors attribute of each search result, include > > :authors_matched and :authors_non_matched attributes. Both

Re: [PATCH v1 1/3] search: Seperately report matching and non-matching authors.

2014-10-24 Thread Austin Clements
Quoth Mark Walters on Oct 24 at 10:23 am: Hi I definitely like the idea: some comments below. Agreed. On Fri, 24 Oct 2014, David Edmondson d...@dme.org wrote: In addition to the :authors attribute of each search result, include :authors_matched and :authors_non_matched attributes.

[PATCH v3.1 3/9] lib: Introduce macros for bit operations

2014-10-24 Thread Austin Clements
These macros help clarify basic bit-twiddling code and are written to be robust against C undefined behavior of shift operators. --- lib/message.cc| 6 +++--- lib/notmuch-private.h | 11 +++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/lib/message.cc

[PATCH v3 9/9] lib: Remove unnecessary thread linking steps when using ghost messages

2014-10-23 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> Previously, it was necessary to link new messages to children to work around some (though not all) problems with the old metadata-based approach to stored thread IDs. With ghost messages, this is no longer necessary, so don't bother with child l

[PATCH v3 8/9] test: Test upgrade to ghost messages feature

2014-10-23 Thread Austin Clements
--- test/T530-upgrade.sh | 21 + 1 file changed, 21 insertions(+) diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh index c4c4ac8..6b42a69 100755 --- a/test/T530-upgrade.sh +++ b/test/T530-upgrade.sh @@ -116,4 +116,25 @@ MAIL_DIR/bar/new/21:2, MAIL_DIR/bar/new/22:2,

[PATCH v3 7/9] lib: Enable ghost messages feature

2014-10-23 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This fixes the broken thread order test. --- lib/database-private.h| 2 +- test/T260-thread-order.sh | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/database-private.h b/lib/database-private.h index e2e4bc8..15e03cc

[PATCH v3 6/9] lib: Implement upgrade to ghost messages feature

2014-10-23 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> Somehow this is the first upgrade pass that actually does *any* error checking, so this also adds the bit of necessary infrastructure to handle that. --- lib/database.cc | 66 +++-- 1 file chang

[PATCH v3 5/9] lib: Implement ghost-based thread linking

2014-10-23 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This updates the thread linking code to use ghost messages instead of user metadata to link messages into threads. In contrast with the old approach, this is actually correct. Previously, thread merging updated only the thread IDs of message doc

[PATCH v3 4/9] lib: Internal support for querying and creating ghost messages

2014-10-23 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This updates the message abstraction to support ghost messages: it adds a message flag that distinguishes regular messages from ghost messages, and an internal function for initializing a newly created (blank) message as a ghost message. --

[PATCH v3 3/9] lib: Introduce macros for bit operations

2014-10-23 Thread Austin Clements
These macros help clarify basic bit-twiddling code and are written to be robust against C undefined behavior of shift operators. --- lib/message.cc| 6 +++--- lib/notmuch-private.h | 11 +++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/lib/message.cc

[PATCH v3 2/9] lib: Update database schema doc for ghost messages

2014-10-23 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This describes the structure of ghost mail documents. Ghost messages are not yet implemented. --- lib/database.cc | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/lib/database.cc b/lib/database.cc index 8

[PATCH v3 1/9] lib: Add a ghost messages database feature

2014-10-23 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This will be implemented over the next several patches. The feature is not yet "enabled" (this does not add it to NOTMUCH_FEATURES_CURRENT). --- lib/database-private.h | 7 +++ lib/database.cc| 2 ++ 2 files changed, 9 ins

[PATCH v3 0/9] Add ghost messages and fix thread linking

2014-10-23 Thread Austin Clements
This is v3 of id:1412637438-4821-1-git-send-email-aclements at csail.mit.edu. This fixes some comments, as suggested by Mark. There are no code changes relative to v2. The diff from v2 is below. diff --git a/lib/database.cc b/lib/database.cc index 6e51a72..3601f9d 100644 --- a/lib/database.cc

[PATCH v3 0/9] Add ghost messages and fix thread linking

2014-10-23 Thread Austin Clements
This is v3 of id:1412637438-4821-1-git-send-email-acleme...@csail.mit.edu. This fixes some comments, as suggested by Mark. There are no code changes relative to v2. The diff from v2 is below. diff --git a/lib/database.cc b/lib/database.cc index 6e51a72..3601f9d 100644 --- a/lib/database.cc +++

[PATCH v3 3/9] lib: Introduce macros for bit operations

2014-10-23 Thread Austin Clements
These macros help clarify basic bit-twiddling code and are written to be robust against C undefined behavior of shift operators. --- lib/message.cc| 6 +++--- lib/notmuch-private.h | 11 +++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/lib/message.cc

[PATCH v3 2/9] lib: Update database schema doc for ghost messages

2014-10-23 Thread Austin Clements
From: Austin Clements amdra...@mit.edu This describes the structure of ghost mail documents. Ghost messages are not yet implemented. --- lib/database.cc | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/lib/database.cc b/lib/database.cc index 8fd7fad

[PATCH v3 1/9] lib: Add a ghost messages database feature

2014-10-23 Thread Austin Clements
From: Austin Clements amdra...@mit.edu This will be implemented over the next several patches. The feature is not yet enabled (this does not add it to NOTMUCH_FEATURES_CURRENT). --- lib/database-private.h | 7 +++ lib/database.cc| 2 ++ 2 files changed, 9 insertions(+) diff --git

[PATCH v3 6/9] lib: Implement upgrade to ghost messages feature

2014-10-23 Thread Austin Clements
From: Austin Clements amdra...@mit.edu Somehow this is the first upgrade pass that actually does *any* error checking, so this also adds the bit of necessary infrastructure to handle that. --- lib/database.cc | 66 +++-- 1 file changed, 64

[PATCH v3 8/9] test: Test upgrade to ghost messages feature

2014-10-23 Thread Austin Clements
--- test/T530-upgrade.sh | 21 + 1 file changed, 21 insertions(+) diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh index c4c4ac8..6b42a69 100755 --- a/test/T530-upgrade.sh +++ b/test/T530-upgrade.sh @@ -116,4 +116,25 @@ MAIL_DIR/bar/new/21:2, MAIL_DIR/bar/new/22:2,

[PATCH v3 4/9] lib: Internal support for querying and creating ghost messages

2014-10-23 Thread Austin Clements
From: Austin Clements amdra...@mit.edu This updates the message abstraction to support ghost messages: it adds a message flag that distinguishes regular messages from ghost messages, and an internal function for initializing a newly created (blank) message as a ghost message. --- lib/message.cc

[PATCH v3 5/9] lib: Implement ghost-based thread linking

2014-10-23 Thread Austin Clements
From: Austin Clements amdra...@mit.edu This updates the thread linking code to use ghost messages instead of user metadata to link messages into threads. In contrast with the old approach, this is actually correct. Previously, thread merging updated only the thread IDs of message documents

[PATCH v3 7/9] lib: Enable ghost messages feature

2014-10-23 Thread Austin Clements
From: Austin Clements amdra...@mit.edu This fixes the broken thread order test. --- lib/database-private.h| 2 +- test/T260-thread-order.sh | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/database-private.h b/lib/database-private.h index e2e4bc8..15e03cc 100644

[PATCH v3 9/9] lib: Remove unnecessary thread linking steps when using ghost messages

2014-10-23 Thread Austin Clements
From: Austin Clements amdra...@mit.edu Previously, it was necessary to link new messages to children to work around some (though not all) problems with the old metadata-based approach to stored thread IDs. With ghost messages, this is no longer necessary, so don't bother with child linking when

[PATCH v2 12/12] lib: Remove unnecessary thread linking steps when using ghost messages

2014-10-21 Thread Austin Clements
Quoth Mark Walters on Oct 22 at 12:17 am: > On Tue, 07 Oct 2014, Austin Clements wrote: > > From: Austin Clements > > > > Previously, it was necessary to link new messages to children to work > > around some (though not all) problems with the old metadata-based > &

[PATCH v2 08/12] lib: Implement ghost-based thread linking

2014-10-21 Thread Austin Clements
Quoth Mark Walters on Oct 22 at 12:10 am: > On Tue, 07 Oct 2014, Austin Clements wrote: > > From: Austin Clements > > > > This updates the thread linking code to use ghost messages instead of > > user metadata to link messages into threads. > > >

[PATCH v2 07/12] lib: Internal support for querying and creating ghost messages

2014-10-21 Thread Austin Clements
Quoth Mark Walters on Oct 22 at 12:05 am: > > Hi > > I am slowly working my way through this series: only two trivial queries > so far. > > On Tue, 07 Oct 2014, Austin Clements wrote: > > From: Austin Clements > > > > This updates the messag

Re: [PATCH v2 08/12] lib: Implement ghost-based thread linking

2014-10-21 Thread Austin Clements
Quoth Mark Walters on Oct 22 at 12:10 am: On Tue, 07 Oct 2014, Austin Clements acleme...@csail.mit.edu wrote: From: Austin Clements amdra...@mit.edu This updates the thread linking code to use ghost messages instead of user metadata to link messages into threads. In contrast

Re: [PATCH v2 12/12] lib: Remove unnecessary thread linking steps when using ghost messages

2014-10-21 Thread Austin Clements
Quoth Mark Walters on Oct 22 at 12:17 am: On Tue, 07 Oct 2014, Austin Clements acleme...@csail.mit.edu wrote: From: Austin Clements amdra...@mit.edu Previously, it was necessary to link new messages to children to work around some (though not all) problems with the old metadata-based

[WIP PATCH 4/4] lib: Add "lastmod:" queries for filtering by last modification

2014-10-13 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> XXX Includes reference to notmuch search --db-revision, which doesn't exist. --- doc/man7/notmuch-search-terms.rst | 8 lib/database-private.h| 1 + lib/database.cc | 4 3 files changed, 13 insertions(+)

[WIP PATCH 3/4] lib: API to retrieve database revision and UUID

2014-10-13 Thread Austin Clements
This exposes the committed database revision to library users along with a UUID that can be used to detect when revision numbers are no longer comparable (e.g., because the database has been replaced). --- lib/database-private.h | 1 + lib/database.cc| 11 +++ lib/notmuch.h

[WIP PATCH 2/4] lib: Add per-message last modification tracking

2014-10-13 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This adds a new document value that stores the revision of the last modification to message metadata, where the revision number increases monotonically with each database commit. An alternative would be to store the wall-clock time of th

[WIP PATCH 1/4] lib: Only sync modified message documents

2014-10-13 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> Previously, we updated the database copy of a message on every call to _notmuch_message_sync, even if nothing had changed. In particular, this always happens on a thaw, so a freeze/thaw pair with no modifications between still caused a database

[WIP PATCH 0/4] Add message revision tracking

2014-10-13 Thread Austin Clements
This implements message revision tracking. This is definitely a work-in-progress, but I wanted to post it since I don't know when I'll be able to work on it next (and maybe someone else can run with it in the mean time). I think this makes all of the necessary library-side changes, but doesn't

[WIP PATCH 0/4] Add message revision tracking

2014-10-13 Thread Austin Clements
This implements message revision tracking. This is definitely a work-in-progress, but I wanted to post it since I don't know when I'll be able to work on it next (and maybe someone else can run with it in the mean time). I think this makes all of the necessary library-side changes, but doesn't

[WIP PATCH 3/4] lib: API to retrieve database revision and UUID

2014-10-13 Thread Austin Clements
This exposes the committed database revision to library users along with a UUID that can be used to detect when revision numbers are no longer comparable (e.g., because the database has been replaced). --- lib/database-private.h | 1 + lib/database.cc| 11 +++ lib/notmuch.h

[WIP PATCH 4/4] lib: Add lastmod: queries for filtering by last modification

2014-10-13 Thread Austin Clements
From: Austin Clements amdra...@mit.edu XXX Includes reference to notmuch search --db-revision, which doesn't exist. --- doc/man7/notmuch-search-terms.rst | 8 lib/database-private.h| 1 + lib/database.cc | 4 3 files changed, 13 insertions(+) diff

[WIP PATCH 2/4] lib: Add per-message last modification tracking

2014-10-13 Thread Austin Clements
From: Austin Clements amdra...@mit.edu This adds a new document value that stores the revision of the last modification to message metadata, where the revision number increases monotonically with each database commit. An alternative would be to store the wall-clock time of the last modification

[WIP PATCH 1/4] lib: Only sync modified message documents

2014-10-13 Thread Austin Clements
From: Austin Clements amdra...@mit.edu Previously, we updated the database copy of a message on every call to _notmuch_message_sync, even if nothing had changed. In particular, this always happens on a thaw, so a freeze/thaw pair with no modifications between still caused a database update. We

[PATCH v2 12/12] lib: Remove unnecessary thread linking steps when using ghost messages

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> Previously, it was necessary to link new messages to children to work around some (though not all) problems with the old metadata-based approach to stored thread IDs. With ghost messages, this is no longer necessary, so don't bother with child l

[PATCH v2 11/12] test: Test upgrade to ghost messages feature

2014-10-06 Thread Austin Clements
--- test/T530-upgrade.sh | 21 + 1 file changed, 21 insertions(+) diff --git a/test/T530-upgrade.sh b/test/T530-upgrade.sh index c4c4ac8..6b42a69 100755 --- a/test/T530-upgrade.sh +++ b/test/T530-upgrade.sh @@ -116,4 +116,25 @@ MAIL_DIR/bar/new/21:2, MAIL_DIR/bar/new/22:2,

[PATCH v2 10/12] lib: Enable ghost messages feature

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This fixes the broken thread order test. --- lib/database-private.h| 2 +- test/T260-thread-order.sh | 1 - 2 files changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/database-private.h b/lib/database-private.h index e2e4bc8..15e03cc

[PATCH v2 09/12] lib: Implement upgrade to ghost messages feature

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> Somehow this is the first upgrade pass that actually does *any* error checking, so this also adds the bit of necessary infrastructure to handle that. --- lib/database.cc | 66 +++-- 1 file chang

[PATCH v2 08/12] lib: Implement ghost-based thread linking

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This updates the thread linking code to use ghost messages instead of user metadata to link messages into threads. In contrast with the old approach, this is actually correct. Previously, thread merging updated only the thread IDs of message doc

[PATCH v2 07/12] lib: Internal support for querying and creating ghost messages

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This updates the message abstraction to support ghost messages: it adds a message flag that distinguishes regular messages from ghost messages, and an internal function for initializing a newly created (blank) message as a ghost message. --

[PATCH v2 06/12] lib: Introduce macros for bit operations

2014-10-06 Thread Austin Clements
These macros help clarify basic bit-twiddling code and are written to be robust against C undefined behavior of shift operators. --- lib/message.cc| 6 +++--- lib/notmuch-private.h | 11 +++ 2 files changed, 14 insertions(+), 3 deletions(-) diff --git a/lib/message.cc

[PATCH v2 05/12] lib: Update database schema doc for ghost messages

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This describes the structure of ghost mail documents. Ghost messages are not yet implemented. --- lib/database.cc | 20 ++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/lib/database.cc b/lib/database.cc index 8

[PATCH v2 04/12] lib: Add a ghost messages database feature

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This will be implemented over the next several patches. The feature is not yet "enabled" (this does not add it to NOTMUCH_FEATURES_CURRENT). --- lib/database-private.h | 7 +++ lib/database.cc| 2 ++ 2 files changed, 9 ins

[PATCH v2 03/12] lib: Handle empty date value

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> In the interest of robustness, avoid undefined behavior of sortable_unserialise if the date value is missing. This shouldn't happen now, but ghost messages will have blank date values. --- lib/message.cc | 3 +++ 1 file changed, 3 insertions(+)

[PATCH v2 02/12] lib: Refactor _notmuch_database_link_message

2014-10-06 Thread Austin Clements
From: Austin Clements <amdra...@mit.edu> This moves the code to retrieve and clear the metadata thread ID out of _notmuch_database_link_message into its own function. This will simplify future changes. --- lib/database.cc | 69 +++--

  1   2   3   4   5   6   7   8   9   10   >