Re: [PATCH v2] emacs: wrap current search in parens when filtering

2015-09-05 Thread David Bremner

Some pretty fussy comments follow. Probably I could have fixed these in
the time it took to write this message ;).

Uli Scholler  writes:
> +  (let ((grouped-query (notmuch-maybe-group-query-string query))
> + (grouped-search-query (notmuch-maybe-group-query-string 
> notmuch-search-query-string)))

- I didn't find it very obvious which of these introduced variables was
  which. I thought maybe "grouped-original-query" for the second
  one. It's pretty subjective though, so your call.

- The lines get pretty long here. We try to keep code to 80 columns.

- Your revised patch isn't in quite the right format for git am;
  the actual commit message get's lost. The unintuitive trick is to add
  commentary in the patch after the ---
  
> +(notmuch-search (if (string= grouped-search-query "*")
>   grouped-query
> -   (concat notmuch-search-query-string " and " 
> grouped-query)) notmuch-search-oldest-first)))
> +   (concat grouped-search-query " and " grouped-query)) 
> notmuch-search-oldest-first)))
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 1/1] Store and search for canonical Unicode text [WIP]

2015-09-05 Thread Rob Browning
Rob Browning  writes:

> David Bremner  writes:

>> It seems plausible to specify UTF-8 input for the library, but what
>> about the CLI? It seems like the canonicalization operation increases
>> the chance of mangling user input in non-UTF-8 locales.
>
> Yes, the key question: what does notmuch intend?  i.e. given a sequence
> of bytes, how will notmuch interpret them?  I think we should decide
> that, and document it clearly somewhere.
>
> The commit message describes my understanding of how things currently
> work, and if/when I get time, I'd like to propose some related
> documentation updates (perhaps to notmuch-search-terms or
> notmuch-insert/new?).
>
> Oh, and if I do understand things correctly, notmuch may already stand a
> chance of mangling any bytes that aren't an invalid UTF-8 byte sequence,
> but also aren't actually in UTF-8 (excepting encodings that are a strict
> subset of UTF-8, like ASCII).
>
> For example (if I did this right), [0xd1 0xa1] is valid UTF-8, producing
> omega "ѡ", and also valid Latin-1, producing "Ñ¡".

So on this particular point, I'm perhaps too used to thinking about the
general encoding problem, and wasn't thinking about our specific
constraints.

If (1) "normal" message bodies are required to be US-ASCII (which I'd
neglected to remember might be the case), and (2) MIME handles the rest,
then perhaps notmuch will only receive raw bytes via user input
(i.e. query strings, etc.).

In which case, we could just document that notmuch interprets user input
as UTF-8 (and we might or might not mention the Latin-1 fallback).

Later locale support could be added if desired, and none of this would
involve the quite nasty problem of encoding detection.

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 2/2] cli: reset db directory mtime upon directory removal

2015-09-05 Thread Jani Nikula
The library does not have a function to remove a directory document
for a path. Usually this doesn't matter except for a slight waste of
space. However, if the same directory gets added to the filesystem
again, the old directory document is found with the old mtime. Reset
the directory mtime on removal to avoid problems.

The corner case that can hit this problem is renaming directories back
and forth. Renaming does not change the mtime of the directory in the
filesystem, and thus the old db directory document mtime may match the
fs mtime of the directory.

The long term fix might be to add a library function to remove a
directory document, however this is a much simpler and faster fix for
the time being.
---
 notmuch-new.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/notmuch-new.c b/notmuch-new.c
index 514e06a4d1f3..33645349cd5f 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -878,6 +878,15 @@ _remove_directory (void *ctx,
goto DONE;
 }
 
+/*
+ * XXX: The library does not have a function to remove a directory
+ * document for a path. Usually this doesn't matter except for a
+ * slight waste of space. However, if the directory gets added to
+ * the filesystem again, the old directory document is found with
+ * the old mtime. Reset the directory mtime to avoid problems.
+ */
+notmuch_directory_set_mtime (directory, 0);
+
   DONE:
 notmuch_directory_destroy (directory);
 return status;
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH 1/2] HACK: test: test folder renames

2015-09-05 Thread Jani Nikula
On Sun, 23 Feb 2014, Mark Walters  wrote [1]:
> I was experimenting with letting notmuch new take an argument to tell it
> to scan only a particular directory (and sub-directories) for new
> messages. I came across the following strange behaviour which is also
> present in master (with a fresh database)
>
> I have a bunch of maildirs in /home/mail: so folders .mail.foo/
> .mail.bar/ each of which has cur/new/tmp and all the messages are in
> cur.
>
> If I do mv .mail.foo .mail.bar/ and run notmuch new I get the expected
> lots of renames (900 or so in the case I was trying). But if I then do
> mv .mail.bar/.mail.foo . and run notmuch new almost all the messages get
> removed (but 30 renames do get detected). If I then do touch .mail.foo/*
> the messages get found again
>
> I am guessing the 30 renames might be because those 30 have duplicates
> somewhere else.
>
> But the other behaviour has me puzzled.

This test reproduces the problem for me, but it's not deterministic,
and I've been unable to make it so. Thus there's a loop of 100
attempts, and usually I hit the problem several times like this:

 FAIL   Rename folder back
 --- T051-new-renames.27.expected   2014-02-23 21:37:10.121774241 +
 +++ T051-new-renames.27.output 2014-02-23 21:37:10.121774241 +
 @@ -1 +1 @@
 -No new mail. Detected 10 file renames.
 +No new mail. Removed 10 messages.
 FAIL   Files remain the same
 --- T051-new-renames.28.expected   2014-02-23 21:37:10.133774652 +
 +++ T051-new-renames.28.output 2014-02-23 21:37:10.133774652 +
 @@ -1,13 +1,3 @@
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-121
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-122
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-123
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-124
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-125
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-126
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-127
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-128
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-129
 -/path/to/test/tmp.T051-new-renames/mail/foo/msg-130
  /path/to/test/tmp.T051-new-renames/mail/bar/msg-131
  /path/to/test/tmp.T051-new-renames/mail/bar/msg-132
  /path/to/test/tmp.T051-new-renames/mail/bar/msg-133

This test is not suitable for merging since it's not
deterministic. But maybe it's good enough to assess the patch that
follows.

[1] id:87siray6th@qmul.ac.uk
---
 test/T051-new-renames.sh | 40 
 1 file changed, 40 insertions(+)
 create mode 100755 test/T051-new-renames.sh

diff --git a/test/T051-new-renames.sh b/test/T051-new-renames.sh
new file mode 100755
index ..3c1515da1085
--- /dev/null
+++ b/test/T051-new-renames.sh
@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+test_description='"notmuch new" with directory renames'
+. ./test-lib.sh
+
+for loop in `seq 100`; do
+
+rm -rf ${MAIL_DIR}
+
+for i in `seq 10`; do
+generate_message '[dir]=foo' '[subject]="Message foo $i"'
+done
+
+for i in `seq 10`; do
+generate_message '[dir]=bar' '[subject]="Message bar $i"'
+done
+
+test_begin_subtest "Index the messages, round $loop"
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "Added 20 new messages to the database."
+
+all_files=$(notmuch search --output=files \*)
+count_foo=$(notmuch count folder:foo)
+
+test_begin_subtest "Rename folder foo -> baz"
+mv ${MAIL_DIR}/foo ${MAIL_DIR}/baz
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
+
+test_begin_subtest "Rename folder back baz -> foo"
+mv ${MAIL_DIR}/baz ${MAIL_DIR}/foo
+output=$(NOTMUCH_NEW)
+test_expect_equal "$output" "No new mail. Detected $count_foo file renames."
+
+test_begin_subtest "Files remain the same"
+output=$(notmuch search --output=files \*)
+test_expect_equal "$output" "$all_files"
+
+done
+
+test_done
-- 
2.1.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch