Improved search speed by 2.5X

2010-11-16 Thread Austin Clements
* to be in? -- Austin Clements MIT/'06/PhD/CSAIL amdra...@mit.edu http://web.mit.edu/amdragon Somewhere in the dream we call reality you will find me, searching for the reality we call dreams

Re: Improved search speed by 2.5X

2010-11-16 Thread Austin Clements
Quoth David Edmondson on Nov 16 at 4:58 pm: On Tue, 16 Nov 2010 11:18:43 -0500, Austin Clements amdra...@mit.edu wrote: [1] Except for 2 emacs tests that depend on author order. What order are matched authors *supposed* to be in? In IRC cworth confirmed that he considered the current

[PATCH 4/4] Mark broken author order tests as no longer broken.

2010-11-17 Thread Austin Clements
Update author order in Emacs tests to reflect correct order. --- test/author-order | 10 test/emacs |4 +- .../emacs.expected-output/notmuch-hello-view-inbox | 26 ++--

[PATCH 1/4] Fix handling of broken tests.

2010-11-17 Thread Austin Clements
Make sure to close the subtest for test_expect_equal_failure, just like in test_expect_equal. --- test/test-lib.sh |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) mode change 100644 = 100755 test/test-lib.sh diff --git a/test/test-lib.sh b/test/test-lib.sh old mode 100644 new mode

Re: Bug in remote use of notmuch

2010-11-18 Thread Austin Clements
Unfortunately, expansion *is* performed by the remote shell, which is why your shell quoting approach works (and is necessary). There's really no way around this, since the ssh client simply joins all of its trailing arguments with spaces and sends this single string to the ssh server, which

Re: [PATCH] Implement a simple read-eval-print loop.

2010-11-21 Thread Austin Clements
Out of curiosity, why not simply use SSH control mastering? You could even make that part of the standard remote notmuch script, without requiring the user to change anything in their ssh configuration. This should be just as fast as a remote notmuch shell, but retains the ability to run

Re: [PATCH] Implement a simple read-eval-print loop.

2010-11-21 Thread Austin Clements
What about notmuch count query1; notmuch count query2; ... all as a single command issued by notmuch-hello? That should give exactly the same output, but eliminates the network round-trips without special support from count. It would be interesting to see how this compares with your modified

Re: [PATCH 3/4] Optimize thread search using matched docid sets.

2010-12-08 Thread Austin Clements
Quoth Carl Worth on Dec 07 at 5:19 pm: On Wed, 17 Nov 2010 14:28:26 -0500, Austin Clements amdra...@mit.edu wrote: This reduces thread search's 1+2t Xapian queries (where t is the number of matched threads) to 1+t queries and constructs exactly one notmuch_message_t for each message

[PATCH] Various small clean-ups to doc ID set code.

2010-12-08 Thread Austin Clements
Remove the repeated sizeof (doc_ids-bitmap[0]) that bothered cworth by instead defining macros to compute the word and bit offset of a given bit in the bitmap. Don't require the caller of _notmuch_doc_id_set_init to pass in a correct bound; instead compute it from the array. This simplifies the

[PATCH 2/5] Implement an internal generic string list and use it.

2010-12-09 Thread Austin Clements
This replaces the guts of the filename list and tag list, making those interfaces simple iterators over the generic string list. The directory, message filename, and tags-related code now build generic string lists and then wraps them in specific iterators. The real wins come in later patches,

[PATCH 3/5] Add a generic function to get a list of terms with some prefix.

2010-12-09 Thread Austin Clements
Replace _notmuch_convert_tags with this and simplify _create_filenames_for_terms_with_prefix. This will also come in handy shortly to get the message file name list. --- lib/database-private.h | 16 +++- lib/database.cc| 36 ++--

[PATCH 4/5] Add the file name list to the unified message metadata pass.

2010-12-09 Thread Austin Clements
Even if the caller never uses the file names, there is little cost to simply fetching the file name terms. However, retrieving the full paths requires additional database work, so the expansion from terms to full paths is performed lazily. This also simplifies clearing the filename cache, since

[PATCH 5/5] Add the tag list to the unified message metadata pass.

2010-12-09 Thread Austin Clements
Now each caller of notmuch_message_get_tags only gets a new iterator, instead of a whole new list. In principle this could cause problems with iterating while modifying tags, but through the magic of talloc references, we keep the old tag list alive even after the cache in the message object is

Re: [PATCH] add headers cc: bcc: and to: (as exactto:) to search index

2010-12-11 Thread Austin Clements
Short of full header indexing, wouldn't a better way to achieve this be to store only the to header as XTO, the cc header XCC, and the bcc header as XBCC and use Xapian's multi-prefix support to map the to: query prefix to XTO, XCC, and XBCC? That way you're not storing twice as many copies of

Re: [PATCH] add headers cc: bcc: and to: (as exactto:) to search index

2010-12-12 Thread Austin Clements
might also want notmuch-query_parser-add_prefix (to, XTO); to maintain some form of backwards compatibility. On Sun, Dec 12, 2010 at 5:43 AM, Joel Borggrén-Franck joel.borggren.fra...@gmail.com wrote: On Sun, Dec 12, 2010 at 7:41 AM, Austin Clements amdra...@gmail.com wrote: Short of full

Re: [PATCH 2/5] Implement an internal generic string list and use it.

2010-12-22 Thread Austin Clements
It seems I somehow repeated the function prototypes for tags.c and filenames.c twice at the bottom of notmuch-private.h (probably through some rebase mishap). Obviously those should be deduplicated. On Thu, Dec 9, 2010 at 3:59 PM, Austin Clements amdra...@mit.edu wrote: diff --git a/lib

[PATCH 2/2] Replace Xapian query parser with custom query parser.

2010-12-22 Thread Austin Clements
Note that the type:mail filter is implemented as a transform pass, so it no longer has to be done everywhere queries are parsed. Furthermore, this filter now depends on the prefixing logic in the query parser instead of implementing this itself. --- lib/database-private.h |3 +--

[PATCH 1/2] Implement a custom query parser with a mostly Xapian-compatible grammar.

2010-12-22 Thread Austin Clements
b/lib/qparser.cc new file mode 100644 index 000..4804d06 --- /dev/null +++ b/lib/qparser.cc @@ -0,0 +1,941 @@ +/* qparser.cc - Notmuch query parser + * + * Copyright © 2010 Austin Clements + * + * This program is free software: you can redistribute it and/or modify + * it under the terms

Re: [ANN] notmuch-deliver

2011-01-12 Thread Austin Clements
Out of curiosity, has anyone considered using inotify to monitor maildirs for new mail to hand to notmuch? For systems supporting inotify (or equivalents), this would have the advantage of being compatible with any delivery mechanism, be it a mail server, procmail, or emacs fcc'ing a maildir. On

[RFC PATCH v2 0/8] Custom query parser, date search, folder search, and more

2011-01-16 Thread Austin Clements
This is version 2 of the custom query parser. It now supports date searches with sane syntax, folder searches (without any additions or changes to the database, unlike cworth's recent commit), and tag:* and -tag:* queries for finding tagged and untagged messages. I used these features to guide

[PATCH 1/8] Implement a custom query parser with a mostly Xapian-compatible grammar.

2011-01-16 Thread Austin Clements
100644 index 000..b86a445 --- /dev/null +++ b/lib/qparser.cc @@ -0,0 +1,920 @@ +/* qparser.cc - Notmuch query parser + * + * Copyright © 2010 Austin Clements + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License

[PATCH 3/8] Parse wildcard queries.

2011-01-16 Thread Austin Clements
This implements support in the lexer and generator for wildcard terms, expanding them into synonym queries the way Xapian does. Since this expansion is performed by the generator, it's easy to take advantage of in query transforms. With this, * works anywhere in the query, so we'll no longer

[PATCH 4/8] Replace Xapian query parser with custom query parser.

2011-01-16 Thread Austin Clements
Note that the type:mail filter is implemented as a transform pass, so it no longer has to be done everywhere queries are parsed. Furthermore, this filter now depends on the prefixing logic in the query parser instead of implementing this itself. Likewise, we don't need to special-case the queries

[PATCH 5/8] Support tag:* as well as NOT tag:* queries.

2011-01-16 Thread Austin Clements
This extends the syntactic-to-database prefix query transform to optionally expand wildcards for boolean prefixes. Support of NOT tag:* queries to find all untagged messages falls out as a convenient side-effect. --- TODO |2 -- lib/database.cc |4 ++--

Re: Folder-based searching

2011-01-18 Thread Austin Clements
The folder search implementation in the custom query parser is always rooted (both because that happened to be much easier to do in that design, and because I agree with you that rooted searches seem preferable most of the time). What arguments do people have for or against rooted folder

[PATCH 1.5/8] Query parser testing framework and basic tests.

2011-01-20 Thread Austin Clements
+++ b/test/qparser-test.cc @@ -0,0 +1,153 @@ +/* qparser-test - Display the lex, parse, and query tree for a query + * + * Copyright © 2011 Austin Clements + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License

[PATCH 2.5/8] Query parser tests for NEAR and ADJ operators.

2011-01-20 Thread Austin Clements
--- This is intended to be applied after patch 2/8 in this series, id:1295165458-9573-3-git-send-email-amdra...@mit.edu test/qparser |5 ++ test/qparser.expected-output/near-and-adj | 83 + test/qparser.expected-output/operators|

[PATCH 5.5/8] test: Wildcard tag search and untagged search.

2011-01-24 Thread Austin Clements
--- test/qparser-test.cc |6 +++--- test/qparser.expected-output/wildcards | 13 + test/search| 12 3 files changed, 28 insertions(+), 3 deletions(-) diff --git a/test/qparser-test.cc b/test/qparser-test.cc index

[PATCH 8.5/8] test: Add tests for search by date.

2011-01-24 Thread Austin Clements
--- test/search | 67 +++ 1 files changed, 67 insertions(+), 0 deletions(-) diff --git a/test/search b/test/search index 7d1dedb..6425359 100755 --- a/test/search +++ b/test/search @@ -73,6 +73,73 @@ add_message '[subject]=this phrase

Re: [PATCH 1/3] new: Do not defer maildir flag synchronization during the first run

2011-01-26 Thread Austin Clements
Quoth Carl Worth on Jan 26 at 9:59 pm: On Wed, 26 Jan 2011 19:15:21 +1000, Carl Worth cwo...@cworth.org wrote: But then I'm also wondering if perhaps we could do the processing undeferred in all cases? I haven't thought that through, but I'd be glad to hear your ideas. This is still

Re: [PATCH 1/3] new: Do not defer maildir flag synchronization during the first run

2011-01-26 Thread Austin Clements
Worth cwo...@cworth.org wrote: On Wed, 26 Jan 2011 10:07:28 -0500, Austin Clements amdra...@mit.edu wrote: Quoth Carl Worth on Jan 26 at 9:59 pm: I believe you're right that synchronization could always be done eagerly. In fact, I believe this is true even with the addition of conjunctive

Re: [PATCH 1/3] new: Do not defer maildir flag synchronization during the first run

2011-01-27 Thread Austin Clements
, Jan 27, 2011 at 12:43 AM, Austin Clements amdra...@mit.edu wrote: Sure. I've been wanting to take a crack at notmuch new's atomicity for a while. Though you'll have to get through some of my outstanding patches. I can only keep so many branches in my head. ]:--8

Re: Xapian locking errors with custom query parser

2011-01-28 Thread Austin Clements
This should most definitely not happen. I'll look in to it. On Fri, Jan 28, 2011 at 5:18 AM, Michal Sojka sojk...@fel.cvut.cz wrote: Hi Austin, when I switched to using your custom query parser I started experiencing Unable to get write lock errors when I run my initial tagging script. I

Re: notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Austin Clements
On Fri, Jan 28, 2011 at 10:36 AM, Mike Kelly pi...@pioto.org wrote: On Fri, 28 Jan 2011 10:45:19 +0100, Thomas Schwinge tho...@schwinge.name wrote: It would definitely be nice to avoid the complexity inherent in having a daemon, but how do you imagine queue on a lock to work? We don't

Re: notmuch's idea of concurrency / failing an invocation

2011-01-28 Thread Austin Clements
) flock -s 200;; *) flock -x 200;; esac $NOTMUCH_BIN $@ 200- ) 200$MAIL_DIR/.notmuch/lock On Fri, Jan 28, 2011 at 11:57 AM, Austin Clements amdra...@mit.edu wrote: On Fri, Jan 28, 2011 at 10:36 AM, Mike Kelly pi...@pioto.org wrote: On Fri, 28 Jan 2011 10

Re: Xapian locking errors with custom query parser

2011-01-28 Thread Austin Clements
) } delete notmuch-term_gen; +delete notmuch-xapian_db; talloc_free (notmuch); } On Fri, Jan 28, 2011 at 11:35 AM, Austin Clements amdra...@mit.edu wrote: This should most definitely not happen.  I'll look in to it. On Fri, Jan 28, 2011 at 5:18 AM, Michal Sojka sojk...@fel.cvut.cz

Re: [PATCH 0/4] Versatile date/time parser

2011-01-29 Thread Austin Clements
Jan 2011, Tom Prince wrote: On 2011-01-23, Michal Sojka wrote: Hi all, the following patch series brings into notmuch date/time parser stolen from GNU coreutils. It can be applied on top of custom query parser patches from Austin Clements. This is RFC and it not meant for merging

[PATCH 1/2] Remove code repetition in the doc ID bitmap code.

2011-01-30 Thread Austin Clements
Remove the repeated sizeof (doc_ids-bitmap[0]) that bothered cworth by instead defining macros to compute the word and bit offset of a given bit in the doc ID set bitmap. --- lib/query.cc | 14 +++--- 1 files changed, 7 insertions(+), 7 deletions(-) diff --git a/lib/query.cc

[PATCH 2/2] Simplify _notmuch_doc_id_set_init interface.

2011-01-30 Thread Austin Clements
Don't require the caller of _notmuch_doc_id_set_init to pass in a correct bound; instead compute it from the array. This simplifies the caller and makes this interface easier to use correctly. --- lib/query.cc | 17 + 1 files changed, 9 insertions(+), 8 deletions(-) diff --git

[PATCH 9/8] qparser: Delete (and thus close) the Xapian database.

2011-01-30 Thread Austin Clements
I removed this line in a bout of overzealous deletions. The visible consequence was that the database was being unlocked lazily, resulting in a brief window after notmuch had exited but the database was still locked. --- I admit this patch series numbering is getting a little out of hand. As

Re: [RFC PATCH v2 0/8] Custom query parser, date search, folder search, and more

2011-02-01 Thread Austin Clements
for the lexer, parser, and generator, but the infrastructure for testing them needs cleanup before I send that out. -- Austin Clements MIT/'06/PhD/CSAIL amdra...@mit.edu http://web.mit.edu/amdragon Somewhere in the dream we call

Folder search semantics (was Re: [RFC PATCH v2 0/8] Custom query parser, date search, folder search, and more)

2011-02-03 Thread Austin Clements
Quoth Carl Worth on Feb 02 at 2:48 pm: Restricting my reply to one tiny bit of your mail: You wrote: non-recursive is the only thing that makes sense for Maildir++ folders Either I'm not understanding Maildir++ folders, or I don't agree with you. I might have an email archive that

Re: [PATCH 2/2] notmuch.el:notmuch-search-process-filter: Rewritten. Cope with incomplete lines.

2011-02-03 Thread Austin Clements
Nice catch. Is there a reason you keep the remaining data in a string instead of taking the more idiomatic elisp approach of leaving it in the process buffer? In fact, the code would probably be simpler if you immediately appended the string to the process buffer like a normal process-filter and

Re: Spam and mailing list filtering?

2011-02-10 Thread Austin Clements
On Feb 10, 2011 8:33 AM, Ben Gamari bgamari.f...@gmail.com wrote: On Thu, 10 Feb 2011 12:20:55 +, Daniel Barlow d...@telent.net wrote: My questions 1) (How) can I filter on the X-Spam-Bar header to chop out spam and suspected spam? I simply run new mail through bogofilter in my

Re: [PATCH 0/5] Fetch all message metadata in a single pass

2011-02-14 Thread Austin Clements
I've rebased this against current master and fixed a few merge conflicts. The rebased patches are on the eager-metadata-v3 branch of http://awakening.csail.mit.edu/git/notmuch.git On Thu, Dec 9, 2010 at 3:59 PM, Austin Clements amdra...@mit.edu wrote: This is the second of the two

[PATCH 00/10] Fix 'notmuch new' atomicity issues

2011-02-17 Thread Austin Clements
This patch series modifies notmuch new to perform all operations atomically and to perform maildir flag synchronization eagerly. As a result, notmuch new can be interrupted without risking database consistency or losing track of messages, but still without losing progress in the middle of a big

[PATCH 02/10] new: Don't loose messages on SIGINT.

2011-02-17 Thread Austin Clements
Previously, message removals were always performed, even after a SIGINT. As a result, when a message was moved from one folder to another, a SIGINT between processing the directory the message was removed from and processing the directory it was added to would result in notmuch removing that

[PATCH 03/10] new: Defer updating directory mtimes until the end.

2011-02-17 Thread Austin Clements
Previously, if notmuch new were interrupted between updating the directory mtime and handling removals from that directory, a subsequent notmuch new would not handle those removals until something else changed in that directory. This defers recording the updated mtime until after removals are

[PATCH 04/10] lib: Make _notmuch_message_sync capable of deleting a message.

2011-02-17 Thread Austin Clements
--- lib/message.cc |7 ++- 1 files changed, 6 insertions(+), 1 deletions(-) diff --git a/lib/message.cc b/lib/message.cc index 0590f76..06747fe 100644 --- a/lib/message.cc +++ b/lib/message.cc @@ -29,6 +29,7 @@ struct _notmuch_message { notmuch_database_t *notmuch;

[PATCH 05/10] lib: Indicate if there are more filenames after removal.

2011-02-18 Thread Austin Clements
Make _notmuch_message_remove_filename return NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID if the message has more filenames and fix callers to handle this. --- lib/message.cc | 10 +- 1 files changed, 9 insertions(+), 1 deletions(-) diff --git a/lib/message.cc b/lib/message.cc index

[PATCH 06/10] lib: Add API's to find by filename and remove a filename from a message.

2011-02-18 Thread Austin Clements
The two new API functions, notmuch_database_find_message_by_filename and notmuch_message_remove_filename give library users more control over the filename removal process. notmuch_database_remove_message has been reimplemented in terms of these new functions. notmuch_message_remove_filename acts

[PATCH 09/10] lib: Add notmuch_database_{begin,end}_atomic.

2011-02-18 Thread Austin Clements
These operations translate into non-flushed Xapian transactions, allowing arbitrary groups of database operations to be performed atomically. --- lib/database.cc | 44 lib/notmuch.h | 30 ++ 2 files changed, 74

[PATCH 08/10] new: Synchronize maildir flags eagerly.

2011-02-18 Thread Austin Clements
Because flag synchronization is stateless, it can be performed at any time as long as it's guaranteed to be performed after any change to a message's filename list. Take advantage of this to synchronize tags immediately after a filename is added or removed while the message is still frozen. With

Re: [PATCH] new: read db_files and db_subdirs if mtime changed

2011-02-27 Thread Austin Clements
Looks good (faster than, but provably equivalent to the original code! notmuch_directory_get_child_* are side-effect free, db_files/db_subdirs aren't used between where they were set in the old code and where they are set in the new code, and db_files/db_subdirs are initialized to NULL when

Re: [PATCH v2 0/2] Small clean-ups to the doc ID set code

2011-03-09 Thread Austin Clements
Thanks! Hope you feel better tomorrow. Quoth Carl Worth on Mar 09 at 3:21 pm: On Sun, 30 Jan 2011 23:22:31 -0500, Austin Clements amdra...@mit.edu wrote: As usual, this is also available in my git repo http://awakening.csail.mit.edu/git/notmuch.git as the search-perf-3 branch

Re: Xapian locking errors with custom query parser

2011-03-10 Thread Austin Clements
Quoth Carl Worth on Mar 10 at 6:21 pm: On Fri, 28 Jan 2011 21:26:03 -0500, Austin Clements amdra...@mit.edu wrote: unlocked. Here's the fix. cworth, what's the most convenient way for me to slip this in to the patch series? I'd most prefer a rebased branch including the fix, along

Re: RFC: notmuch powered (personal) (end-to-end) e-mail system

2011-03-20 Thread Austin Clements
Much of the beauty of notmuch is how few assumptions it makes about your mail system. It plays well with others. For example, one deep insight of notmuch is that it *doesn't* require a custom mail store, even though a more obvious design might; in fact, it doesn't even require Maildir. That

Re: [PATCH 0/5] Fetch all message metadata in a single pass

2011-03-21 Thread Austin Clements
-0500, Austin Clements amdra...@mit.edu wrote: I've rebased this against current master and fixed a few merge conflicts.  The rebased patches are on the eager-metadata-v3 branch of   http://awakening.csail.mit.edu/git/notmuch.git Hi Austin, This looks like a great set of optimizations

Re: Xapian locking errors with custom query parser

2011-03-21 Thread Austin Clements
grind. Whoever said being a grad student was hitting the snooze button on life was a liar.) Responses inline. On Fri, Mar 11, 2011 at 12:26 AM, Carl Worth cwo...@cworth.org wrote: On Thu, 10 Mar 2011 21:47:30 -0500, Austin Clements amdra...@mit.edu wrote: Yes, qparser-3 is ready for you, and has

Re: Xapian locking errors with custom query parser

2011-03-21 Thread Austin Clements
On Fri, Mar 11, 2011 at 3:58 AM, Michal Sojka sojk...@fel.cvut.cz wrote: On Fri, 11 Mar 2011, Carl Worth wrote: I've finally had a chance to start looking at this. [...] 1. For new search features (ADJ,NEAR,etc.) I do not have a strong    interest in compatibility with Xapian.    I was

Re: [PATCH] use custom-face-edit value-type in notmuch-search-line-faces

2011-03-28 Thread Austin Clements
On Sun, Mar 27, 2011 at 1:26 AM, Jameson Rollins jroll...@finestructure.net wrote: On Sat, 26 Mar 2011 22:16:32 -0700, Jameson Graef Rollins jroll...@finestructure.net wrote: This enables the proper face customization UI for notmuch-search-line-faces. Hey, folks. amdragon was the one who

Re: (auto-)tagging sent messages

2011-04-18 Thread Austin Clements
On Sun, Apr 17, 2011 at 2:23 PM, Jameson Graef Rollins jroll...@finestructure.net wrote: By giving notmuch new a path to a message in the store:  notmuch new /path/to/message By feeding notmuch new a message on stdin, and then having it write the message to a specified location:  notmuch

[PATCH 01/10 v2] test: Test atomicity of notmuch new.

2011-05-04 Thread Austin Clements
This tests notmuch new's ability to recover from arbitrary stopping failures. It interrupts notmuch new after every database commit and, on every resulting database snapshot, re-runs notmuch new to completion and checks that the final database state is invariant. --- This addresses a timing bug

Re: [PATCH v2] test/search: add check for slightly trick search

2011-05-04 Thread Austin Clements
On Wed, May 4, 2011 at 4:26 PM, Felipe Contreras felipe.contre...@gmail.com wrote: +output=$(notmuch search from:'search-by-from@' | notmuch_search_sanitize) I don't think this does what you think it does. Xapian only understands double quotes around phrases, not single quotes. Furthermore, a

Re: [PATCH v2] test/search: add check for slightly trick search

2011-05-04 Thread Austin Clements
Quoth Felipe Contreras on May 04 at 11:54 pm: On Wed, May 4, 2011 at 11:46 PM, Austin Clements amdra...@mit.edu wrote: On Wed, May 4, 2011 at 4:26 PM, Felipe Contreras felipe.contre...@gmail.com wrote: +output=$(notmuch search from:'search-by-from@' | notmuch_search_sanitize) I don't

Re: storing From and Subject in xapian

2011-05-04 Thread Austin Clements
This is awesome. What was your machine configuration? As another data point, with a probably very different configuration (8 year old P4, new SSD), my test query was 1.9X faster uncached and 1.6X faster cached. It also produced 60% fewer disk reads. I saw the same 1% increase in database size.

Re: storing From and Subject in xapian

2011-05-04 Thread Austin Clements
On Wed, May 4, 2011 at 9:48 PM, Austin Clements amdra...@mit.edu wrote: As another data point, with a probably very different configuration (8 year old P4, new SSD), my test query was 1.9X faster uncached and 1.6X faster cached.  It also produced 60% fewer disk reads.  I saw the same 1

Re: [PATCH] Sanitize Subject: and Author: fields to not contain control characters in notmuch-search

2011-05-08 Thread Austin Clements
Cool. This seems very reasonable. Just some style nits: The three places where you have sanitize_string(, there should be a space between the function name and the paren. Relatedly, for(;*loop;loop++){ should be spaced out like for (; *loop; loop++) {. (Curiously, there seems to be

Re: [PATCH] python/thread: always return a string in get_subject/authors

2011-05-09 Thread Austin Clements
Your commit message is inconsistent with your change; is your intent to return None or the empty string? Also, could you modify your commit message to say what those are? On May 9, 2011 3:06 AM, Anton Khirnov an...@khirnov.net wrote: ___ notmuch mailing

Re: [PATCH] test: add %s format string to all `message' calls in emacs tests

2011-05-09 Thread Austin Clements
This looks good to me (and is certainly more correct), but seems rather roundabout. Is there a reason this code doesn't simply (princ (buffer-string))? On Tue, May 10, 2011 at 1:40 AM, Dmitry Kurochkin dmitry.kuroch...@gmail.com wrote: The patch replaces all (message (buffer-string)) calls in

[PATCH] Fix search output sanitization test.

2011-05-11 Thread Austin Clements
Previously, the test assumed the generated message would be assigned a specific thread ID; now it doesn't. Also, spelling fix. --- This applies to the current 0.6 RC head (9f8ef78e). test/search-output |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git

[PATCH] test: Fix message when skipping test_expect_equal* tests

2011-05-12 Thread Austin Clements
For these types of tests, the test name is previously recorded in a variable, not passed to the test function, so pass this variable to test_skip. --- test/test-lib.sh |6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) diff --git a/test/test-lib.sh b/test/test-lib.sh index

[PATCH] test: Report test failures from test_expect_*

2011-05-12 Thread Austin Clements
This makes test_expect_* return non-zero if the test fails, so the caller can make decisions based on this, such as setting test prerequisites. --- test/test-lib.sh |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/test/test-lib.sh b/test/test-lib.sh index 9b56406..955350a

Re: release-candidate/0.6

2011-05-12 Thread Austin Clements
-0400, Austin Clements amdra...@mit.edu wrote: On Thu, May 12, 2011 at 8:22 AM, Pieter Praet pie...@praet.org wrote: The atomicity tests were failing here because I didn't have GDB installed, so I've added it as a prereq. Sorry, I've had a patch to address that sitting around, but hadn't sent

Re: storing From and Subject in xapian

2011-05-14 Thread Austin Clements
I wonder if a better approach would be to use notmuch_message_get_header everywhere, rather than introducing _notmuch_message_get_header_value, and have it simply recognize headers that can be retrieved directly from the database. Then library callers could take advantage of this optimization and

Re: [PATCH] Save and restore point explicitly in `notmuch-wash-toggle-invisible-action'.

2011-05-24 Thread Austin Clements
On Tue, May 24, 2011 at 6:16 PM, Dmitry Kurochkin dmitry.kuroch...@gmail.com wrote: When a user clicks the button, the cursor is somewhere inside the old label.  If we save the point as a marker, after step 3 it would end up at the position where the old label was.  If the new label is inserted

Re: [PATCH] emacs: Make the queries used in the all-tags section

2011-05-24 Thread Austin Clements
Out of curiosity, what use cases do you envision for this? So far I've only heard two, both of which seem like great ideas, but neither of which require such a heavy-handed solution: displaying unread counts for tags rather than total counts, and hiding unused tags. I would argue that we

Re: [PATCH] emacs: Make the queries used in the all-tags section

2011-05-25 Thread Austin Clements
On Wed, May 25, 2011 at 6:04 AM, Daniel Schoepe daniel.scho...@googlemail.com wrote: On Wed, 25 May 2011 00:10:43 -0400, Austin Clements amdra...@mit.edu wrote: Out of curiosity, what use cases do you envision for this?  So far I've only heard two, both of which seem like great ideas

Re: [PATCH] emacs: Make the queries used in the all-tags section

2011-05-25 Thread Austin Clements
On Wed, May 25, 2011 at 1:56 PM, Daniel Schoepe daniel.scho...@googlemail.com wrote: I accidentally used `filter' in the previous patch which isn't defined by default during runtime, updated version in the attachment. Cool. My inner parser is happy. A few comments on the code: +(defcustom

Re: [PATCH] emacs: Make the queries used in the all-tags section

2011-05-25 Thread Austin Clements
On Wed, May 25, 2011 at 5:21 PM, Daniel Schoepe daniel.scho...@googlemail.com wrote: On Wed, 25 May 2011 15:11:16 -0400, Austin Clements amdra...@mit.edu wrote: So, perhaps something like (defcustom notmuch-hello-tag-list-counts nil   Method for generating counts displayed in the all tags

Re: [PATCH] emacs: Make the queries used in the all-tags section

2011-05-25 Thread Austin Clements
On May 25, 2011 7:21 PM, Daniel Schoepe daniel.scho...@googlemail.com wrote: On Wed, 25 May 2011 18:42:30 -0400, Austin Clements amdra...@mit.edu wrote: 'Doh. That's what I get for not reading the surrounding code. I misunderstood what your patch was going for and assumed it was what *I

Re: one-time-iterators

2011-05-26 Thread Austin Clements
On May 26, 2011 1:20 PM, Carl Worth cwo...@cworth.org wrote: The question: How do you solve this in the emacs code? do you store all tids of a query? The emacs code does not use the notmuch library interface like your python bindings do. Instead, it uses the notmuch command-line tool, (and

Re: one-time-iterators

2011-05-26 Thread Austin Clements
On Thu, May 26, 2011 at 6:22 PM, Patrick Totzke patricktot...@googlemail.com wrote: Excerpts from Austin Clements's message of Thu May 26 22:43:02 +0100 2011: Though, Patrick, that solution doesn't address your problem.  On the other hand, it's not clear to me what concurrent access

Re: one-time-iterators

2011-05-27 Thread Austin Clements
On Fri, May 27, 2011 at 2:04 PM, Patrick Totzke patricktot...@googlemail.com wrote: Excerpts from Austin Clements's message of Fri May 27 03:41:44 +0100 2011: Have you tried simply calling list() on your thread iterator to see how expensive it is?  My bet is that it's quite cheap,

Re: release-candidate/0.6 redux

2011-05-28 Thread Austin Clements
On Sat, May 28, 2011 at 5:58 PM, Jameson Graef Rollins jroll...@finestructure.net wrote: Austin: speaking of which, would you mind rebasing that patch series against notmuch/master at cb8418784c21155ffea79cce8409a7ea3c546937 and sending that to the list again?  That might help push Carl to

Re: [PATCH 00/10] Fix 'notmuch new' atomicity issues

2011-05-28 Thread Austin Clements
Rebased to current master (cb8418) as atomic-new-v4 (aka for-review/atomic-new-v4). On Wed, May 4, 2011 at 4:30 PM, Austin Clements amdra...@mit.edu wrote: jrollins found a timing bug in the atomicity test.  A fix, plus beefed up test comments are on a new atomic-new-v3 (and for-review/atomic

Re: one-time-iterators

2011-05-30 Thread Austin Clements
Quoth Patrick Totzke on May 28 at 9:58 am: Excerpts from Austin Clements's message of Fri May 27 20:29:24 +0100 2011: On Fri, May 27, 2011 at 2:04 PM, Patrick Totzke patricktot...@googlemail.com wrote: Excerpts from Austin Clements's message of Fri May 27 03:41:44 +0100 2011: Have

Re: [PATCH 0/4] set test prereqs (Emacs, GDB, GPG) v3

2011-06-02 Thread Austin Clements
This looks good to me, and should be merged sooner rather than later because it touches a lot of lines. If atomic-new-v4 doesn't happen to get merged before this, then id:1305206080-17461-1-git-send-email-amdra...@mit.edu and id:1305206110-17511-1-git-send-email-amdra...@mit.edu should get

Re: [python] get all messages of a thread

2011-06-02 Thread Austin Clements
On Thu, Jun 2, 2011 at 10:20 AM, Sebastian Spaeth sebast...@sspaeth.de wrote: On Thu, 2 Jun 2011 19:43:29 +1000, Brian May wrote: On 2 June 2011 17:05, Sebastian Spaeth sebast...@sspaeth.de wrote: What would be the best way to solve this (besides fixing the C api to allow to reset the

Re: [PATCH] emacs: Tab completion for notmuch-search and notmuch-search-filter

2011-06-04 Thread Austin Clements
Nifty! On Sat, Jun 4, 2011 at 8:19 AM, Daniel Schoepe daniel.scho...@googlemail.com wrote: +        (minibuffer-completion-table (completion-table-dynamic +                                      `(lambda (s) (notmuch-query-completions +                                               (quote

Re: [PATCH] emacs: Tab completion for notmuch-search and notmuch-search-filter

2011-06-04 Thread Austin Clements
Quoth Daniel Schoepe on Jun 04 at 9:55 pm: On Sat, 4 Jun 2011 11:32:15 -0400, Austin Clements amdra...@mit.edu wrote: Dynamic scoping is obnoxious, but I think programmed completion is steeped in the assumption that you'll use it. This code would be much simpler if notmuch-query

[PATCH] Add dir-locals style variables for C++ and Elisp code.

2011-06-06 Thread Austin Clements
Also, slightly reformat dir-locals.el so that the settings align and to make it friendlier for future additions. --- .dir-locals.el | 18 ++ 1 files changed, 14 insertions(+), 4 deletions(-) diff --git a/.dir-locals.el b/.dir-locals.el index cbdb1f9..eff29fc 100644 ---

Re: [PATCH] Add dir-locals style variables for C++ and Elisp code.

2011-06-07 Thread Austin Clements
Quoth Dmitry Kurochkin on Jun 07 at 9:27 am: Hi Austin. On Tue, 7 Jun 2011 01:20:25 -0400, Austin Clements amdra...@mit.edu wrote: Also, slightly reformat dir-locals.el so that the settings align and to make it friendlier for future additions. --- .dir-locals.el | 18

Re: [PATCH] Add dir-locals style variables for C++ and Elisp code.

2011-06-07 Thread Austin Clements
On Tue, Jun 7, 2011 at 2:38 AM, Dima Kogan notm...@dima.secretsauce.net wrote: On Tue,  7 Jun 2011 01:20:25 -0400 Austin Clements amdra...@mit.edu wrote: Also, slightly reformat dir-locals.el so that the settings align and to make it friendlier for future additions. Should we also add

[PATCH] Add dir-locals style variables for C++, Elisp, and shell code.

2011-06-09 Thread Austin Clements
Also, slightly reformat dir-locals.el so that the settings align and to make it friendlier for future additions. --- .dir-locals.el | 24 1 files changed, 20 insertions(+), 4 deletions(-) diff --git a/.dir-locals.el b/.dir-locals.el index cbdb1f9..aea630b 100644 ---

Re: [PATCH] emacs: User-defined sections in notmuch-hello

2011-06-09 Thread Austin Clements
This looks really interesting. I haven't examined the code very closely, but I have some high level comments. It seems that the code is simultaneously trying to do something very general, but also hard-coding a lot of behaviors, and I think the code's complexity suffers for it. What would this

Re: [PATCH 00/10] Fix 'notmuch new' atomicity issues

2011-06-10 Thread Austin Clements
Quoth Carl Worth on Jun 08 at 3:05 pm: On Sat, 28 May 2011 22:51:10 -0400, Austin Clements amdra...@mit.edu wrote: Rebased to current master (cb8418) as atomic-new-v4 (aka for-review/atomic-new-v4). Hi Austin, Thanks so much for sending this series (and 4 times, even!). I *really

[PATCH 06/17] new: Defer updating directory mtimes until the end.

2011-06-11 Thread Austin Clements
Previously, if notmuch new were interrupted between updating the directory mtime and handling removals from that directory, a subsequent notmuch new would not handle those removals until something else changed in that directory. This defers recording the updated mtime until after removals are

[PATCH 08/17] lib: Add support for nested atomic sections.

2011-06-11 Thread Austin Clements
notmuch_database_t now keeps a nesting count and we only start a transaction or commit for the outermost atomic section. Introduces a new error, NOTMUCH_STATUS_UNBALANCED_ATOMIC. --- lib/database-private.h |1 + lib/database.cc| 22 ++ lib/notmuch.h |

[PATCH 09/17] lib: Indicate if there are more filenames after removal.

2011-06-11 Thread Austin Clements
Make _notmuch_message_remove_filename return NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID if the message has more filenames and fix callers to handle this. --- lib/message.cc |9 - 1 files changed, 8 insertions(+), 1 deletions(-) diff --git a/lib/message.cc b/lib/message.cc index

  1   2   3   4   5   6   7   8   9   10   >