Re: [PATCH v4 1/3] cli: some keyword options can be supplied with no argument

[Daniel Kahn Gillmor]

On Sat 2017-12-23 10:29:30 -0400, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>
>>  
>> +bool incremented = false;
>>  if (next == '\0' && next_arg != NULL && ! try->opt_bool) {
>>  next = ' ';
>>  value = next_arg;
>> +incremented = true;
>>  opt_index ++;
>>  }
>
> Is incremented == true exactly when next == ' ' ? It might be nice to
> make that more explicit by setting one based on the other. You could
> also use (next == ' ') as your test condition, but I understand that
> might not be that obvious to read.

yes, i was aiming for readability.  I'm assuming that the compiler can
optimize this as needed.

> The thing I'm most nervous about here is the interaction between this
> new code and the relatively recent code that permits ' ' as a
> seperator. Would you mind adding one or more tests for that case? For
> example, that I checked that
>
>  ./notmuch show --format=json --decrypt true $id
>
> continues to work, and that's great, but it seems like something to
> check on the argument parsing level, i.e
>
> --keyword␣non-default-value
>
> (pardon my unicode)

I'm fine with that, and with your proposed revision of this patch that
includes the amended test.  thanks for pushing it forward.

 --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v4 2/3] cli/show: make --decrypt take a keyword.

[Daniel Kahn Gillmor]

On Sat 2017-12-23 10:39:47 -0400, David Bremner wrote:
> One thing that gave me pause is the fact that --decrypt=auto does not
> verify by default. What are the security implications of this?

The only issue is that some messages that are correctly signed are
marked as unsigned if the user hasn't chosen to explicitly --decrypt.
The reverse failure would be the real security problem, but that's not
happening here :)

> Do we verify when indexing? Does this require more documentation?

We do not verify when indexing at all, because we have nothing to do
with any statement about signature verification during indexing (we
aren't caching it in the database, for example)

If notmuch was to do something more clever with signature verification,
i'd be interested in working on it, but i'm not currently clear what
that would be, specifically (e.g. Patrick Brunschwig and i discussed
some interesting thoughts about signature verification for enigmail that
resulted in this mailing list post [0]).

But i don't think these decisions are ones that need to be made in the
context of this particular series.

all that said, if anyone has been thinking about signature verification
for e-mail, i'd be happy to hear your thoughts on how notmuch could do
better there.

 --dkg

[0] 
https://admin.hostpoint.ch/pipermail/enigmail-users_enigmail.net/2017-November/004683.html
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH v4 3/3] cli/reply: make --decrypt take a keyword

[Daniel Kahn Gillmor]

On Sat 2017-12-23 10:47:52 -0400, David Bremner wrote:
> Daniel Kahn Gillmor  writes:
>> +{ .opt_keyword = (int*)(), .name = "decrypt",
>> +  .keyword_no_arg_value = "true", .keywords =
>> +  (notmuch_keyword_t []){ { "false", NOTMUCH_DECRYPT_FALSE },
>> +  { "auto", NOTMUCH_DECRYPT_AUTO },
>> +  { "true", NOTMUCH_DECRYPT_NOSTASH },
>> +  { 0, 0 } } },
>
> same question as for show, should we support --decrypt=nostash ?

No, we should not support --decrypt=nostash for show or reply.  The
semantics of the display commands (show, reply) are such that they
*never* modify the index or stash anything in there.  The equivalent for
the indexing (new, insert, reindex) commands' "--decrypt=nostash" in the
display commands is simply "--decrypt=true".

If you take a look at the subsequent series that follows this one,
you'll see that i offer a "--decrypt=stash" command "notmuch show", but
it makes some unusual assumptions (in particular, about opening the
database read/write, which "notmuch show" has never needed to do
before).

So, in case it wasn't clear: i've thought pretty hard about these
commands and the options that users are likely to want or need.

But the current series (under discussion in this thread) should be
applied *without* those additional changes -- i don't want to have the
whole controversy about "notmuch show --decrypt=stash" now. :)

  --dkg
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Xapian exception leading to database corruption

[David Bremner]

David Edmondson  writes:

> Using current git notmuch on Debian testing a rebuild from scratch of my
> database fails:
>
>> agrajag-testing ~/s/notmuch % ./notmuch new
>> Found 605510 total files (that's not much mail).
>> add_file: A Xapian exception occurred 28m 37s remaining).
>> A Xapian exception occurred adding message: Unexpected end of posting list 
>> for 'G00014364'.
>> Processed 137296 total files in 8h 4m 45s (4 files/sec.).
>> Added 135950 new messages to the database.
>> Note: A fatal error was encountered: A Xapian exception occurred
>> agrajag-testing ~/s/notmuch %  
>

I can't duplicate this (probably not surprising) on debian testing. I
also have about 600k files, and I'm running debian testing. My total
index time is about 1h on a not-very-recent machine with an SSD. I'm
guessing you have a hard disk (or something is deeply wrong to take
that long). Even for a hard disk, 8h to index 137k files seems slow.
Did you happen to check for disk errors?

> I see that bremner reported something like this in #xapian, but not any
> resolution.

I guess your best bet is to write to

  xapian-disc...@lists.xapian.org

>
> Any suggestions? Is it possible to force a new chert database to be
> created rather than glass? (Mostly I'd like to get back to work!)

According to Olly Betts,

   With 1.4 you can pass Xapian::DB_BACKEND_CHERT in the flags when
   constructing the WritableDatabase object.

So you'd have to hack the notmuch source, I'm guessing around line 55 of
database.cc

#define DB_ACTION (Xapian::DB_CREATE_OR_OPEN | Xapian::DB_RETRY_LOCK | 
Xapian::DB_BACKEND_CHERT)

Do move the existing database out of the way, or apparently xapian can
get confused.

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Xapian exception leading to database corruption

[David Edmondson]

Using current git notmuch on Debian testing a rebuild from scratch of my
database fails:

> agrajag-testing ~/s/notmuch % ./notmuch new
> Found 605510 total files (that's not much mail).
> add_file: A Xapian exception occurred 28m 37s remaining).
> A Xapian exception occurred adding message: Unexpected end of posting list 
> for 'G00014364'.
> Processed 137296 total files in 8h 4m 45s (4 files/sec.).
> Added 135950 new messages to the database.
> Note: A fatal error was encountered: A Xapian exception occurred
> agrajag-testing ~/s/notmuch %  

Checking the database at this point seems fine:

> agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian 
> docdata:
> blocksize=8K items=0 firstunused=1 revision=1 levels=0 root=(faked)
> void B-tree checked okay
> docdata table structure checked OK
> 
> termlist:
> blocksize=8K items=0 firstunused=2 revision=1 levels=0 root=(faked)
> void B-tree checked okay
> termlist table structure checked OK
> 
> postlist:
> blocksize=8K items=2 firstunused=1 revision=1 levels=0 root=0
> B-tree checked okay
> postlist table structure checked OK
> 
> position:
> blocksize=8K items=0 firstunused=1 revision=1 levels=0 root=(faked)
> void B-tree checked okay
> position table structure checked OK
> 
> spelling:
> Lazily created, and not yet used.
> 
> synonym:
> Lazily created, and not yet used.
> 
> No errors found
> agrajag-testing ~/s/notmuch % 

If I run “notmuch new” again then the exception repeats, except the
database is listed as corrupt (this is actually from an earlier
cycle of testing, but it is reproducible):

> agrajag-testing ~/s/notmuch % ./notmuch new
> add_file: A Xapian exception occurred
> A Xapian exception occurred adding message: Unexpected end of posting list 
> for 'G00013f27'.
> Processed 16 total files in 59s (0 files/sec.).
> Added 15 new messages to the database.
> Note: A fatal error was encountered: A Xapian exception occurred
> agrajag-testing ~/s/notmuch % xapian-check ~/Maildir/.notmuch/xapian  
> docdata:
> blocksize=8K items=15 firstunused=3 revision=9 levels=0 root=2
> B-tree checked okay
> docdata table structure checked OK
> 
> termlist:
> blocksize=8K items=301848 firstunused=68658 revision=9 levels=2 root=14019
> xapian-check: DatabaseError: 1 unused block(s) missing from the free list, 
> first is 0
> agrajag-testing ~/s/notmuch % 

I see that bremner reported something like this in #xapian, but not any
resolution.

Any suggestions? Is it possible to force a new chert database to be
created rather than glass? (Mostly I'd like to get back to work!)

dme.
-- 
But uh oh, I love her because, she moves in her own way.
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 1/2] nmbug: Bump to version 0.3

[W. Trevor King]

Changes since 0.2:

* Accept failures to unset core.worktree in clone (0a155847,
  2017-10-10, unreleased).
* Use --no-renames in log (f9189a06, 2016-09-26, v0.24).
* Auto-checkout in clone if it wouldn't clobber (7ef3b653, 2017-10-10,
  unreleased).
* Add a 'help' command for folks who don't like --help
  (9d25c97d, 2014-10-03, v0.20).
* Setup a 'config' branch on clone to track origin/config (244f8739,
  2015-03-22, v0.20).  This branch may be consumed by
  notmuch-report(1).
* Only error for invalid diff lines in tags/ (57225988, 2017-10-16,
  unreleased).
* Ignore # comments in 'notmuch dump ...' output (9bbc54bd,
  2016-03-27, v0.22).
* Respect 'expect' in _spawn(..., wait=True) (e263c5b1, 2017-10-10,
  unreleased).
* Update URLs in documentation (554b90b5 and 6a833a6e8, 2016-06-02,
  v0.23).
---
 devel/nmbug/nmbug | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/devel/nmbug/nmbug b/devel/nmbug/nmbug
index 8c7e925c..c35dd75d 100755
--- a/devel/nmbug/nmbug
+++ b/devel/nmbug/nmbug
@@ -51,7 +51,7 @@ except ImportError:  # Python 2
 from urllib import unquote as _unquote
 
 
-__version__ = '0.2'
+__version__ = '0.3'
 
 _LOG = _logging.getLogger('nmbug')
 _LOG.setLevel(_logging.WARNING)
-- 
2.13.6

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 2/2] NEWS: Add nmbug 0.3 release notes to the notmuch 0.26 section

[W. Trevor King]

---
 NEWS | 13 +
 1 file changed, 13 insertions(+)

diff --git a/NEWS b/NEWS
index 6d2bf138..3b6404e7 100644
--- a/NEWS
+++ b/NEWS
@@ -41,6 +41,19 @@ Indexing cleartext of encrypted e-mails
   that the notmuch index itself is adequately protected.  DO NOT USE
   this feature without considering the security of your index.
 
+nmbug
+-
+
+nmbug's internal version increases to 0.3 in this notmuch release.
+User-facing changes with this notmuch release:
+
+* Accept failures to unset `core.worktree` in `clone`, which allows
+  nmbug to be used with Git 2.11.0 and later.
+* Auto-checkout in `clone` if it wouldn't clobber existing content,
+  which makes the initial clone more convenient.
+* Only error for invalid diff lines in `tags/`, which allows for
+  `README`s and similar in nmbug repositories.
+
 Notmuch 0.25.3 (2017-12-08)
 ===
 
-- 
2.13.6

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

[PATCH 0/2] nmbug: Bump to 0.3 and document in NEWS

[W. Trevor King]

As discussed previously in [1,2,3].  I've split this into two patches,
and only covered the changes since the last notmuch release.  There
have been additional changes since nmbug 0.2 (which went out with
notmuch 0.19), and I mention them in the commit message for patch 1/1,
but none of them seemed important enough to call out in NEWS.

I'm also fine leaving these changes out of NEWS and just bumping the
nmbug version (which is why I split this series into two patches).

Cheers,
Trevor

[1]: id:cover.1507675236.git.wk...@tremily.us
 Subject: [PATCH 0/3] nmbug:
 Date: Tue, 10 Oct 2017 15:49:48 -0700
[2]: id: id:20171211174707.gx19...@valgrind.us
 Subject: Re: [PATCH 3/4] nmbug: Auto-checkout in clone if it
   wouldn't clobber
 Date: Mon, 11 Dec 2017 09:47:07 -0800
[3]: id:87a7ypdm3p@tesseract.cs.unb.ca
 Subject: Re: [PATCH 3/4] nmbug: Auto-checkout in clone if it
   wouldn't clobber
 Date: Mon, 11 Dec 2017 16:51:38 -0400

W. Trevor King (2):
  nmbug: Bump to version 0.3
  NEWS: Add nmbug 0.3 release notes to the notmuch 0.26 section

 NEWS  | 13 +
 devel/nmbug/nmbug |  2 +-
 2 files changed, 14 insertions(+), 1 deletion(-)

-- 
2.13.6

___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: Notmuch 0.26 release schedule

[David Bremner]

David Bremner  writes:

> I plan a feature freeze for December 25, and a release (all going well)
> on or around January 1.

I've tagged 0.26_rc0 (and uploaded to Debian experimental). I expect a
few more changes before final release, but I want to start to shake out
any build problems or API change issues.

d


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] make release archive: common (or no) timestamps

[David Bremner]

Tomi Ollila  writes:

> The appended file 'version' has the same timestamp as the files added
> by `git archive`.

pushed.

d
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch