[PATCH] dump: Don't sort.

2011-10-29 Thread Thomas Schwinge 
From: Thomas Schwinge tho...@schwinge.name

This improves usage experience considerably in the given scenario.

---


Hi!

I decided that it'd be useful to put the reasoning and data right next to
the source code (as opposed to putting it into the commit message), for
the next guy to read this code has it all in one place.


Grüße,
 Thomas


---

 notmuch-dump.c |7 ++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/notmuch-dump.c b/notmuch-dump.c
index 7e7bc17..a431e23 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -45,7 +45,12 @@ notmuch_dump_command (unused (void *ctx), int argc, char 
*argv[])
fprintf (stderr, Out of memory\n);
return 1;
 }
-notmuch_query_set_sort (query, NOTMUCH_SORT_MESSAGE_ID);
+/* This used to use NOTMUCH_SORT_MESSAGE_ID.  On 2011-10-29, a measurement
+ * on a 372981 messages instance showed that wall time can be reduced from
+ * 28 minutes (sorted by Message-ID) to 15 minutes (unsorted), the latter
+ * being much more ``database-disk-layout-friendly''.  Subsequently sorting
+ * the 25 MiB of data is a no-brainer, if required.  */
+notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED);
 
 if (argc) {
output = fopen (argv[0], w);
-- 
tg: (3bafdfc..) t/dump_unsorted (depends on: baseline)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] restore: Be more liberal in which data to accept.

2011-10-29 Thread Thomas Schwinge 
From: Thomas Schwinge tho...@schwinge.name

There are ``Message-ID''s out in the wild that contain spaces.

---


Hi!

Carl, the main question for you is: does this break sup-import
operability?


Spammers are quite inventive for creating ``interesting Messages-ID''s.
Apparently, notmuch handles these fine internally, but it breaks a
dump/restore cycle:

$ notmuch restore  ~/tmp/Mail-notmuch_dump/dump
No filename given. Reading dump from stdin.
Warning: Ignoring invalid input line: 3791856948.991306994...@m0.net 
Received:fromdialup-62.215.274.4.dial1.stamford([62.215.274.4]  ([...])
Warning: Ignoring invalid input line: PM200010:29:54 AM ([...])
Warning: Ignoring invalid input line: PM200010:51:48 AM ([...])
Warning: Ignoring invalid input line: PM200011:47:35 AM ([...])
Warning: Ignoring invalid input line: PM200011:48:46 AM ([...])
Warning: Ignoring invalid input line: PM200011:50:10 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:05 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:17 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:18 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:32 AM ([...])
Warning: Ignoring invalid input line: PM20001:48:38 PM ([...])
Warning: Ignoring invalid input line: PM20001:53:07 PM ([...])
Warning: Ignoring invalid input line: PM20004:01:48 AM ([...])
Warning: Ignoring invalid input line: PM20004:01:59 AM ([...])
Warning: Ignoring invalid input line: PM20004:10:44 AM ([...])
Warning: Ignoring invalid input line: PM20004:20:00 AM ([...])
Warning: Ignoring invalid input line: PM20005:06:50 PM ([...])
Warning: Ignoring invalid input line: PM20005:14:17 AM ([...])
Warning: Ignoring invalid input line: PM20005:32:15 PM ([...])
Warning: Ignoring invalid input line: PM20005:32:22 PM ([...])
Warning: Ignoring invalid input line: PM20005:33:05 PM ([...])
Warning: Ignoring invalid input line: PM20005:33:57 AM ([...])
Warning: Ignoring invalid input line: PM20006:24:12 AM ([...])
Warning: Ignoring invalid input line: PM20006:25:04 AM ([...])
Warning: Ignoring invalid input line: PM20006:25:49 AM ([...])
Warning: Ignoring invalid input line: PM20006:26:11 AM ([...])
Warning: Ignoring invalid input line: PM20007:05:34 PM ([...])
Warning: Ignoring invalid input line: PM2000PM 04:09:15 ([...])
Warning: Ignoring invalid input line: PM2000¿ÀÀü 11:07:41 ([...])
Warning: Ignoring invalid input line: PM2000¿ÀÈÄ 12:42:47 ([...])
Warning: Ignoring invalid input line: PM2000¿ÀÈÄ 12:42:48 ([...])
Warning: Ignoring invalid input line: PM2000¿ÀÈÄ 5:58:28 ([...])
Warning: Ignoring invalid input line: PM2000¿ÀÈÄ 6:30:51 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:04 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:09 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:11 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:12 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:45 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:38:10 ([...])

Thus, dump; remove all tags; restore is not nullipotent, which it should
be.

Especially noteworthy is probably the first one: it happens to have
gotten a Received line mangled into the Message-ID, and it ends with a
space character.

Some more from the freak show:

$MESSAGE_ID ([...])

%CUSTOM_CHAR[8-10]$%CUSTOM_CHAR[8-10]$%CUSTOM_CHAR[8-10]@%CUSTOM_DOMAIN.msn.com 
([...])
%RNDDIGIT1025.%RNDDIGIT15%RNDLCCHAR15%RNDDIGIT110%RNDLCCHAR13@ ([...])
%rnddigit1025.%rnddigit15%rndlcchar15%rnddigit110...@yahoo.com ([...])
%rnddigit1025.%rnddigit15%rndlcchar15%rnddigit11...@yahoo.com ([...])
%RNDDIGIT27eq52md1$9rg57p%RNDDIGIT14$277ts40lsh@%RNDWORD13ivo4068 ([...])
%RNDDIGIT27g10u874$3cqh62f%RNDDIGIT14$7fgo121wnwt@%RNDWORD13quw32712 ([...])
%RNDDIGIT27mog75vx711$541xqm480xc%RNDDIGIT14$031nq1pk@%RNDWORD13av2979 
([...])
%RNDDIGIT27nqf761drk7$7l4mza%RNDDIGIT14$96ijq17zq@%RNDWORD13b1779 ([...])
%RNDDIGIT27q0tcg10$94pcn1mw%RNDDIGIT14$7x77pztx@%RNDWORD13ny7619 ([...])
%RNDDIGIT27uiw866tv49$5c3rg%RNDDIGIT14$6jl43vv@%RNDWORD13uwh17820 ([...])
%RNDDIGIT27x966lug3$0pr016r%RNDDIGIT14$8ye15k@%RNDWORD13qps90907 ([...])

%RNDDIGIT310%RNDLCCHAR15%RNDDIGIT15%RNDLCCHAR15$%RNDDIGIT17%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13$%RNDDIGIT15%RNDLCCHAR13%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13@
 ([...])

%RNDDIGIT310%RNDLCCHAR15%RNDDIGIT15%RNDLCCHAR15$%RNDDIGIT17%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13$%RNDDIGIT15%RNDLCCHAR13%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13@bambi
 ([...])

%RNDDIGIT310%RNDLCCHAR15%RNDDIGIT15%RNDLCCHAR15$%RNDDIGIT17%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13$%RNDDIGIT15%RNDLCCHAR13%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13@wheelchair
 ([...])
%RNDDIGIT520.%RNDDIGIT110.%RNDDIGIT110@-%RNDLCCHAR13%RNDDIGIT13. ([...])

[Patch 2/3] Respect window margins in notmuch-hello.el

2011-10-29 Thread Ivy Foster 
Hello again, folks,

The following patch makes notmuch take window margins into
account when calculating the width of the notmuch-hello
window in Emacs. As a side benefit (so that I only had to
write the code once), I've added a named function to return
said width.

Folks using linum-mode should still probably get a copy of
linum-off.el and disable linum for notmuch-hello, though,
since it doesn't set the margin until after the buffer is
done being created, causing text to overflow across the
edge. Alternately, take a look at my next message, with yet
another patch.

Enjoy!

iff

Signed-off-by: Ivy Foster joyfulg...@archlinux.us
From 0c1a0fa0858e3c22b416ce6bd9ad623198217d34 Mon Sep 17 00:00:00 2001
From: Ivy Foster joyfulg...@archlinux.us
Date: Fri, 28 Oct 2011 23:18:14 -0400
Subject: [PATCH 2/3] Respect margins in new named window-width function

This patch removes a function which is duplicated
several times in notmuch-hello.el, improving
readability. Also, factors in  window margins
when calculating width.
---
 emacs/notmuch-hello.el |   28 ++--
 1 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 65fde75..1a213f0 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -213,25 +213,27 @@ should be. Returns a cons cell `(tags-per-line width)'.
 (cond
  ((integerp notmuch-column-control)
   (max 1
-   (/ (- (window-width) notmuch-hello-indent)
+   (/ (notmuch-hello-available-width)
   ;; Count is 9 wide (8 digits plus space), 1 for the space
   ;; after the name.
   (+ 9 1 (max notmuch-column-control widest)
 
  ((floatp notmuch-column-control)
-  (let* ((available-width (- (window-width) notmuch-hello-indent))
- (proposed-width (max (* available-width 
notmuch-column-control) widest)))
-(floor available-width proposed-width)))
+  (let ((proposed-width (max
+(* (notmuch-hello-available-width)
+   notmuch-column-control)
+widest)))
+(floor (notmuch-hello-available-width) proposed-width)))
 
  (t
   (max 1
-   (/ (- (window-width) notmuch-hello-indent)
+   (/ (notmuch-hello-available-width)
   ;; Count is 9 wide (8 digits plus space), 1 for the space
   ;; after the name.
   (+ 9 1 widest)))
 
 (cons tags-per-line (/ (max 1
-   (- (window-width) notmuch-hello-indent
+   (- (notmuch-hello-available-width)
   ;; Count is 9 wide (8 digits plus
   ;; space), 1 for the space after the
   ;; name.
@@ -458,7 +460,7 @@ Complete list of currently available key bindings:
(widget-create 'editable-field
   ;; Leave some space at the start and end of the
   ;; search boxes.
-  :size (max 8 (- (window-width) notmuch-hello-indent
+  :size (max 8 (- (notmuch-hello-available-width)
   (length Search: )))
   :action (lambda (widget rest ignore)
 (notmuch-hello-search (widget-value widget
@@ -481,12 +483,13 @@ Complete list of currently available key bindings:
   ;; Don't let the search boxes be
   ;; less than 8 characters wide.
   :size (max 8
- (- (window-width)
+ (-
+  
(notmuch-hello-available-width)
 ;; Leave some space
 ;; at the start and
 ;; end of the
 ;; boxes.
-(* 2 notmuch-hello-indent)
+  notmuch-hello-indent
 ;; 1 for the space
 ;; before the
 ;; `[save]' button. 6
@@ -540,7 +543,7 @@ Complete list of currently available key bindings:
  (widget-insert Edit saved searches with the `edit' button.\n))
(widget-insert Hit RET or click on a saved search or tag name to view 
matching threads.\n)
(widget-insert `=' refreshes this screen. `s' jumps to the search box. 
`q' to quit.\n)
-   (let

[PATCH 3/3] Add notmuch-hello-mode-hook

2011-10-29 Thread Ivy Foster 
The following patch adds a mode-hook to notmuch-hello-mode.
Not much else to say, really.

Would people find hooks for before and/or after adding the
tags useful? Or hooks elsewhere in this mode (or other
modes)?

iff

Signed-off-by: Ivy Foster joyfulg...@archlinux.us
From e8cdead0222fc85dd01d8a0185b54aed5e0a6e1e Mon Sep 17 00:00:00 2001
From: Ivy Foster joyfulg...@archlinux.us
Date: Fri, 28 Oct 2011 23:47:16 -0400
Subject: [PATCH 3/3] Added notmuch-hello-mode-hook

---
 emacs/notmuch-hello.el |6 ++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 1a213f0..13356c5 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -118,6 +118,11 @@ Typically \,\ in the US and UK and \.\ in Europe.
   :group 'notmuch
   :type 'string)
 
+(defcustom notmuch-hello-mode-hook nil
+  Functions called after entering `notmuch-hello-mode'.
+  :group 'notmuch
+  :type 'hook)
+
 (defvar notmuch-hello-url http://notmuchmail.org;
   The `notmuch' web site.)
 
@@ -337,6 +342,7 @@ Complete list of currently available key bindings:
  (use-local-map notmuch-hello-mode-map)
  (setq major-mode 'notmuch-hello-mode
mode-name notmuch-hello)
+ (run-mode-hooks 'notmuch-hello-mode-hook)
  ;;(setq buffer-read-only t)
 )
 
-- 
1.7.7.1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Experimental Tag Sharing

2011-10-29 Thread David Bremner 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

OK, here is my rough and ready attempt at tag sharing.  I figure we
can smooth out the rough edges if/when we agree on a set of tags and
preferably on an on-disk format.

How to play?

- - Apply all the patches in this thread (starting
  id:1319383133-11006-1-git-send-email-da...@tethera.net)

- - to git master git clone git://pivot.cs.unb.ca/nmbug.git $HOME/.nmbug

- - put the script nmbug somewhere

# suck in the tags
$ nmbug restore

[ do some tagging ]

$ nmbug dump
$ cd $HOME/.nmbug  git add tags  git commit

now you have to get the changes into the master repo.  I think
hosting this on git.notmuchmail.org in a seperate repo will make
sense, but for now, if you want to participate in the experiment, send
me your public key (ideally in a gpg signed mail, but if not, oh well), 
and you can push to my repo at

g...@pivot.cs.unb.ca:pub/nmbug

Probably the most crucial thing is that we agree on some set of tags.
Here is the set of tags I am working with (output from 
 notmuch search --output=tags * | grep ^notmuch:: 
)

notmuch::bug is a bug report
notmuch::feature provides a new feature
notmuch::fix fixes a bug
notmuch::obsoletereplaced by some other patch
notmuch::patch   
notmuch::portability improves portability
notmuch::pushed  is pushed to master
notmuch::review  is a review
notmuch::reviewedhas been (well reviewed)
notmuch::testprovides a new test/or improves testing

The prefix notmuch:: is hardcoded into the script, but everything
else is easy to change with the usual notmuch operations.  Of course
we can change the prefix too. I decided on :: because it needed less
escaping.

On think I'm not sure about is how to handle ideas like voting, and
signing off with tags. I suspect we can come up with some
conventions like notmuch::review1, or
notmuch::vote1..notmuch::vote5. But maybe people have less kludgy
ideas.

Remember, perfect is the enemy of good ;)

David
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iJwEAQECAAYFAk6sLWQACgkQTiiN/0Um85k5JQP+PZ0ycrNTTZZ72PyXbDQzrv4+
dDKopxh+WpeORjWu8gvj+LNMM3A8n0t4/A12mxrG9cUJxg2HTPUtRo9Zi7dFQzct
uGvSOvQ6GocUretW0BaK/yp53v0R7b00qxvxAvUeoZPgIWGtqp52fXW8d0I2yAUn
079okQwPYdbowaPAX1A=
=+E8F
-END PGP SIGNATURE-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] contrib/nmbug: new script for sharing tags with prefix notmuch::

2011-10-29 Thread David Bremner 
From: David Bremner brem...@debian.org

In this initial version, we take care of only the base import and
export of the appropriate tags in line oriented format amenable to
easy merging.

The current plan to use git to share tags and resolve conflicts.
---
 contrib/nmbug |   69 +
 1 files changed, 69 insertions(+), 0 deletions(-)
 create mode 100755 contrib/nmbug

diff --git a/contrib/nmbug b/contrib/nmbug
new file mode 100755
index 000..e9d1b14
--- /dev/null
+++ b/contrib/nmbug
@@ -0,0 +1,69 @@
+#!/bin/bash
+# Copyright (c) 2011 David Bremner
+# License: same as notmuch
+
+NMHOME=${HOME}/.nmbug
+
+NMTAGS=($(notmuch search --output=tags *|grep ^notmuch::))
+
+function dump() {
+notmuch dump -- $(printf   tag:%s ${NMTAGS[*]}) |\
+while read  -r msgid rest 
+do
+   outfile=$NMHOME/tags/$(echo $msgid | sha1sum - | cut -f1 -d' ')
+   echo $outfile
+   printf msg-id: %s\n $msgid  $outfile
+   tmp=${rest#\(}
+   read -r -a tags ${tmp%\)};
+   for tag in ${tags[@]}; do
+   case $tag in
+   notmuch::*)
+   echo tag: $tag  $outfile
+   ;;
+   *)
+   # nothing
+   esac
+   done
+done
+
+}
+
+function cat_file() {
+tags=
+id=
+cat $1 |\
+while read -r what data 
+do
+   case $what in
+   msg-id:)
+   printf %s ( $data 
+   ;;
+   tag:)
+   printf %s  $data
+   ;;
+   *)
+   echo Syntax error $what
+   exit 1
+   esac
+done
+echo )
+}
+
+function restore() {
+find $NMHOME/tags -type f |\
+while read -r filename
+do
+   cat_file $filename
+done | notmuch restore --match=notmuch::
+}
+case $1 in
+dump)
+   dump
+   ;;
+restore)
+   restore
+   ;;
+*)
+   echo unknown command $1;
+esac
+   
-- 
1.7.6.3

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC PATCH 3/3] emacs: support limiting the number of messages shown in search results

2011-10-29 Thread Daniel Schoepe 
On Fri, 28 Oct 2011 23:59:31 +0300, Jani Nikula j...@nikula.org wrote:
 Add support for limiting the maximum number of messages initially displayed
 in search results. When enabled, the search results will contain push
 buttons to double the number of messages displayed or to show unlimited
 messages.

Nice patch, as it not only makes searches with a lot of results easier
to use on slower machines/hard drives, but I also find that seeing only
a few dozen threads in the buffer looks more orderly to me, compared
to a buffer with hundreds of lines.

A few comments about the code:

 @@ -373,6 +381,7 @@ Complete list of currently available key bindings:
(make-local-variable 'notmuch-search-oldest-first)
(make-local-variable 'notmuch-search-target-thread)
(make-local-variable 'notmuch-search-target-line)
 +  (make-local-variable 'notmuch-search-maxitems)
(set (make-local-variable 'notmuch-search-continuation) nil)
(set (make-local-variable 'scroll-preserve-screen-position) t)
(add-to-invisibility-spec 'notmuch-search)
 @@ -633,6 +642,8 @@ This function advances the next thread when finished.
   (insert End of search results.)
   (if (not (= exit-status 0))
   (insert (format  (process returned %d) 
 exit-status)))
 + (if notmuch-search-maxitems
 + (notmuch-search-setup-buttons))

As discussed on IRC, this causes `notmuch-search' to fail if the
maxitems argument is nil.

 +(defun notmuch-search-setup-buttons ()
 +  (widget-insert )
 +  (widget-create 'push-button
 +  :notify (lambda (rest ignore)
 +(set 'notmuch-search-maxitems
 + (* 2 notmuch-search-maxitems))
 +(notmuch-search-refresh-view))
 +  :help-echo Double the number of messages shown
 +  Show 2X messages)
 +  (widget-insert )
 +  (widget-create 'push-button
 +  :notify (lambda (rest ignore)
 +(set 'notmuch-search-maxitems 0)
 +(notmuch-search-refresh-view))
 +  :help-echo Show all search results
 +  Show unlimited messages)
 +  (widget-setup))

I think these notify-actions should be separate functions to make it
easier to bind them to keys.

 +
  (defcustom notmuch-poll-script 
An external script to incorporate new mail into the notmuch database.
  
 @@ -997,7 +1030,7 @@ current search results AND the additional query string 
 provided.
query)))
  (notmuch-search (if (string= notmuch-search-query-string *)
   grouped-query
 -   (concat notmuch-search-query-string  and  
 grouped-query)) notmuch-search-oldest-first)))
 +   (concat notmuch-search-query-string  and 
grouped-query)) notmuch-search-oldest-first
notmuch-search-maxitems)))

This causes notmuch-search-filter to fail (repeatedly), since `notmuch-search'
expects a TARGET-THREAD (or nil) as its third parameter, but is given
`notmuch-search-maxitems' instead.

  
  (defun notmuch-search-filter-by-tag (tag)
Filter the current search results based on a single tag.
 @@ -1006,7 +1039,7 @@ Runs a new search matching only messages that match 
 both the
  current search results AND that are tagged with the given tag.
(interactive
 (list (notmuch-select-tag-with-completion Filter by tag: )))
 -  (notmuch-search (concat notmuch-search-query-string  and tag: tag) 
 notmuch-search-oldest-first))
 +  (notmuch-search (concat notmuch-search-query-string  and tag:
tag) notmuch-search-oldest-first notmuch-search-maxitems))

Same here.

Cheers,
Daniel


pgp8LLZSow5q5.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH v2] contrib/nmbug: new script for sharing tags with prefix notmuch::

2011-10-29 Thread David Bremner 
From: David Bremner brem...@debian.org

In this initial version, we take care of import and export of the
appropriate tags in line oriented format amenable to easy merging.

We also provide (not very robust/clever) commands commit/push/pull
to deal with a git repo that as already been set up.
---

fixed silly debugging output.
add some convenience commands.

 contrib/nmbug |   77 +
 1 files changed, 77 insertions(+), 0 deletions(-)
 create mode 100755 contrib/nmbug

diff --git a/contrib/nmbug b/contrib/nmbug
new file mode 100755
index 000..a449d60
--- /dev/null
+++ b/contrib/nmbug
@@ -0,0 +1,77 @@
+#!/bin/bash
+# Copyright (c) 2011 David Bremner
+# License: same as notmuch
+
+NMHOME=${HOME}/.nmbug
+
+NMTAGS=($(notmuch search --output=tags *|grep ^notmuch::))
+
+function dump() {
+notmuch dump -- $(printf   tag:%s ${NMTAGS[*]}) |\
+while read  -r msgid rest 
+do
+   outfile=$NMHOME/tags/$(echo $msgid | sha1sum - | cut -f1 -d' ')
+   printf msg-id: %s\n $msgid  $outfile
+   tmp=${rest#\(}
+   read -r -a tags ${tmp%\)};
+   for tag in ${tags[@]}; do
+   case $tag in
+   notmuch::*)
+   echo tag: $tag  $outfile
+   ;;
+   *)
+   # nothing
+   esac
+   done
+done
+
+}
+
+function cat_file() {
+tags=
+id=
+cat $1 |\
+while read -r what data 
+do
+   case $what in
+   msg-id:)
+   printf %s ( $data 
+   ;;
+   tag:)
+   printf %s  $data
+   ;;
+   *)
+   echo Syntax error $what
+   exit 1
+   esac
+done
+echo )
+}
+
+function restore() {
+find $NMHOME/tags -type f |\
+while read -r filename
+do
+   cat_file $filename
+done | notmuch restore --match=notmuch::
+}
+case $1 in
+dump)
+   dump
+   ;;
+restore)
+   restore
+   ;;
+commit)
+   (cd $NMHOME  git add tags  git commit)
+   ;;
+push)
+   (cd $NMHOME  git push)
+   ;;
+pull)
+   (cd $NMHOME  git pull)
+   ;;
+*)
+   echo unknown command $1;
+esac
+   
-- 
1.7.6.3

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC PATCH 2/3] cli: add support for limiting the number of search results

2011-10-29 Thread Daniel Schoepe 
On Fri, 28 Oct 2011 23:59:30 +0300, Jani Nikula j...@nikula.org wrote:
 @@ -412,6 +413,14 @@ notmuch_search_command (void *ctx, int argc, char 
 *argv[])
   fprintf (stderr, Invalid value for --sort: %s\n, opt);
   return 1;
   }
 + } else if (STRNCMP_LITERAL (argv[i], --maxitems=) == 0) {
 + const char *p;
 + opt = argv[i] + sizeof (--maxitems=) - 1;
 + maxitems = strtoul(opt, p, 10);

p should be of type `char *', not `const char *', as it will be
modified by strtoul. (Otherwise, gcc will produce a warning about this).

Cheers,
Daniel


pgpjX7J5cST8t.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] xutil.c: remove duplicate copies, create new library libutil.a to contain xutil.

2011-10-29 Thread David Bremner 
On Sun, 23 Oct 2011 12:18:53 -0300, David Bremner da...@tethera.net wrote:
 From: David Bremner brem...@debian.org
 
 We keep the lib/xutil.c version. As a consequence, also factor out
 _internal_error and associated macros.  It might be overkill to make a
 new file error_util.c for this, but _internal_error does not really
 belong in database.cc.

It seems this patch was sent out base64 encoded, which confuses git-am.
I attach the patch here. You won't be able to git am this message, but
you can at least save the attachment and apply it. Sorry for the
inconvenience.



pgpOYzTX22cXT.pgp
Description: PGP signature
From 5c678c9bc96890ce9a5965d6654851ebbe2b4767 Mon Sep 17 00:00:00 2001
From: David Bremner brem...@debian.org
Date: Sun, 23 Oct 2011 12:05:13 -0300
Subject: [PATCH] xutil.c: remove duplicate copies, create new library
 libutil.a to contain xutil.

We keep the lib/xutil.c version. As a consequence, also factor out
_internal_error and associated macros.  It might be overkill to make a
new file error_util.c for this, but _internal_error does not really
belong in database.cc.
---
 Makefile  |2 +-
 Makefile.local|7 +--
 lib/Makefile.local|5 +-
 lib/database.cc   |   15 -
 lib/notmuch-private.h |   20 +---
 lib/xutil.c   |  134 ---
 lib/xutil.h   |   55 ---
 util/Makefile |5 ++
 util/Makefile.local   |   11 
 util/error_util.c |   41 +++
 util/error_util.h |   45 
 util/xutil.c  |  136 
 util/xutil.h  |   55 +++
 xutil.c   |  138 -
 14 files changed, 300 insertions(+), 369 deletions(-)
 delete mode 100644 lib/xutil.c
 delete mode 100644 lib/xutil.h
 create mode 100644 util/Makefile
 create mode 100644 util/Makefile.local
 create mode 100644 util/error_util.c
 create mode 100644 util/error_util.h
 create mode 100644 util/xutil.c
 create mode 100644 util/xutil.h
 delete mode 100644 xutil.c

diff --git a/Makefile b/Makefile
index 11e3a3d..2fb2a61 100644
--- a/Makefile
+++ b/Makefile
@@ -3,7 +3,7 @@
 all:
 
 # List all subdirectories here. Each contains its own Makefile.local
-subdirs = compat completion emacs lib test
+subdirs = compat completion emacs lib util test
 
 # We make all targets depend on the Makefiles themselves.
 global_deps = Makefile Makefile.config Makefile.local \
diff --git a/Makefile.local b/Makefile.local
index ec09f95..6ddef5c 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -39,7 +39,7 @@ PV_FILE=bindings/python/notmuch/version.py
 # Smash together user's values with our extra values
 FINAL_CFLAGS = -DNOTMUCH_VERSION=$(VERSION) $(CFLAGS) $(WARN_CFLAGS) $(CONFIGURE_CFLAGS) $(extra_cflags)
 FINAL_CXXFLAGS = $(CXXFLAGS) $(WARN_CXXFLAGS) $(CONFIGURE_CXXFLAGS) $(extra_cflags) $(extra_cxxflags)
-FINAL_NOTMUCH_LDFLAGS = $(LDFLAGS) -Llib -lnotmuch $(AS_NEEDED_LDFLAGS) $(GMIME_LDFLAGS) $(TALLOC_LDFLAGS)
+FINAL_NOTMUCH_LDFLAGS = $(LDFLAGS) -Lutil -lutil -Llib -lnotmuch $(AS_NEEDED_LDFLAGS) $(GMIME_LDFLAGS) $(TALLOC_LDFLAGS)
 FINAL_NOTMUCH_LINKER = CC
 ifneq ($(LINKER_RESOLVES_LIBRARY_DEPENDENCIES),1)
 FINAL_NOTMUCH_LDFLAGS += $(CONFIGURE_LDFLAGS)
@@ -299,12 +299,11 @@ notmuch_client_srcs =		\
 	notmuch-time.c		\
 	query-string.c		\
 	show-message.c		\
-	json.c			\
-	xutil.c
+	json.c
 
 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
 
-notmuch: $(notmuch_client_modules) lib/libnotmuch.a
+notmuch: $(notmuch_client_modules) lib/libnotmuch.a util/libutil.a
 	$(call quiet,CXX $(CFLAGS)) $^ $(FINAL_LIBNOTMUCH_LDFLAGS) -o $@
 
 notmuch-shared: $(notmuch_client_modules) lib/$(LINKER_NAME)
diff --git a/lib/Makefile.local b/lib/Makefile.local
index be51eaa..d58552c 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -54,8 +54,7 @@ libnotmuch_c_srcs =		\
 	$(dir)/message-file.c	\
 	$(dir)/messages.c	\
 	$(dir)/sha1.c		\
-	$(dir)/tags.c		\
-	$(dir)/xutil.c
+	$(dir)/tags.c
 
 libnotmuch_cxx_srcs =		\
 	$(dir)/database.cc	\
@@ -71,7 +70,7 @@ $(dir)/libnotmuch.a: $(libnotmuch_modules)
 	$(call quiet,AR) rcs $@ $^
 
 $(dir)/$(LIBNAME): $(libnotmuch_modules) notmuch.sym
-	$(call quiet,CXX $(CXXFLAGS)) $(libnotmuch_modules) $(FINAL_LIBNOTMUCH_LDFLAGS) $(LIBRARY_LINK_FLAG) -o $@
+	$(call quiet,CXX $(CXXFLAGS)) $(libnotmuch_modules) $(FINAL_LIBNOTMUCH_LDFLAGS) $(LIBRARY_LINK_FLAG) -o $@ -L$(srcdir)/util -lutil
 
 notmuch.sym: $(srcdir)/$(dir)/notmuch.h $(libnotmuch_modules)
 	sh $(srcdir)/$(lib)/gen-version-script.sh $ $(libnotmuch_modules)  $@
diff --git a/lib/database.cc b/lib/database.cc
index e77fd53..88be939 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -209,21 +209,6 @@ static prefix_t PROBABILISTIC_PREFIX[]= {
 { folder,			XFOLDER}
 };
 
-int
-_internal_error (const char *format, ...)
-{
-va_list va_args;
-
-va_start (va_args, format);
-
-fprintf

Re: [RFC PATCH 2/3] cli: add support for limiting the number of search results

2011-10-29 Thread Jani Nikula 
On Sat, 29 Oct 2011 19:30:50 +0200, Daniel Schoepe dan...@schoepe.org wrote:
 On Fri, 28 Oct 2011 23:59:30 +0300, Jani Nikula j...@nikula.org wrote:
  @@ -412,6 +413,14 @@ notmuch_search_command (void *ctx, int argc, char 
  *argv[])
  fprintf (stderr, Invalid value for --sort: %s\n, opt);
  return 1;
  }
  +   } else if (STRNCMP_LITERAL (argv[i], --maxitems=) == 0) {
  +   const char *p;
  +   opt = argv[i] + sizeof (--maxitems=) - 1;
  +   maxitems = strtoul(opt, p, 10);
 
 p should be of type `char *', not `const char *', as it will be
 modified by strtoul. (Otherwise, gcc will produce a warning about this).

strtoul() won't touch the data pointed to by p (it only modifies p), so
in that sense it could be const, but you're right in that it really
should be 'char *', just for a more complicated reason. Thanks for
making me look it up: http://c-faq.com/ansi/constmismatch.html (not the
best of explanations, perhaps, but gives an idea why the 2nd parameter
of strtoul() can't be 'const char **').

BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC PATCH 2/3] cli: add support for limiting the number of search results

2011-10-29 Thread Daniel Schoepe 
On Sat, 29 Oct 2011 23:08:04 +0300, Jani Nikula j...@nikula.org wrote:
 strtoul() won't touch the data pointed to by p (it only modifies p), so
 in that sense it could be const, but you're right in that it really
 should be 'char *', just for a more complicated reason. Thanks for
 making me look it up: http://c-faq.com/ansi/constmismatch.html (not the
 best of explanations, perhaps, but gives an idea why the 2nd parameter
 of strtoul() can't be 'const char **').

Oh, right, thanks for the clarification. As I often do in situations
such as this, I confused const char * and char const *.


pgpKRtOed6SyQ.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [RFC PATCH 3/3] emacs: support limiting the number of messages shown in search results

2011-10-29 Thread Jani Nikula 
On Sat, 29 Oct 2011 19:25:22 +0200, Daniel Schoepe dan...@schoepe.org wrote:
 On Fri, 28 Oct 2011 23:59:31 +0300, Jani Nikula j...@nikula.org wrote:
  Add support for limiting the maximum number of messages initially displayed
  in search results. When enabled, the search results will contain push
  buttons to double the number of messages displayed or to show unlimited
  messages.
 
 Nice patch, as it not only makes searches with a lot of results easier
 to use on slower machines/hard drives, but I also find that seeing only
 a few dozen threads in the buffer looks more orderly to me, compared
 to a buffer with hundreds of lines.

Thanks, I agree. Though having read your review below, it's not such a
nice patch after all. :)

Also, having chatted with amdragon on IRC, patches 1 and 2 should be
thought out better. Perhaps changing the lib is not the way to go after
all, even if that would help other lib users in paging the results. But
the whole thing was a quick proof of concept (hence RFC) to get to see
how this patch would work out, and I'm glad you liked it.

 A few comments about the code:
 
  @@ -373,6 +381,7 @@ Complete list of currently available key bindings:
 (make-local-variable 'notmuch-search-oldest-first)
 (make-local-variable 'notmuch-search-target-thread)
 (make-local-variable 'notmuch-search-target-line)
  +  (make-local-variable 'notmuch-search-maxitems)
 (set (make-local-variable 'notmuch-search-continuation) nil)
 (set (make-local-variable 'scroll-preserve-screen-position) t)
 (add-to-invisibility-spec 'notmuch-search)
  @@ -633,6 +642,8 @@ This function advances the next thread when finished.
  (insert End of search results.)
  (if (not (= exit-status 0))
  (insert (format  (process returned %d) 
  exit-status)))
  +   (if notmuch-search-maxitems
  +   (notmuch-search-setup-buttons))
 
 As discussed on IRC, this causes `notmuch-search' to fail if the
 maxitems argument is nil.

And as you pointed out, the parameter is optional so it must accept nil.

  +(defun notmuch-search-setup-buttons ()
  +  (widget-insert )
  +  (widget-create 'push-button
  +:notify (lambda (rest ignore)
  +  (set 'notmuch-search-maxitems
  +   (* 2 notmuch-search-maxitems))
  +  (notmuch-search-refresh-view))
  +:help-echo Double the number of messages shown
  +Show 2X messages)
  +  (widget-insert )
  +  (widget-create 'push-button
  +:notify (lambda (rest ignore)
  +  (set 'notmuch-search-maxitems 0)
  +  (notmuch-search-refresh-view))
  +:help-echo Show all search results
  +Show unlimited messages)
  +  (widget-setup))
 
 I think these notify-actions should be separate functions to make it
 easier to bind them to keys.

It's obvious now that you say it! :)

  +
   (defcustom notmuch-poll-script 
 An external script to incorporate new mail into the notmuch database.
   
  @@ -997,7 +1030,7 @@ current search results AND the additional query string 
  provided.
   query)))
   (notmuch-search (if (string= notmuch-search-query-string *)
  grouped-query
  - (concat notmuch-search-query-string  and  
  grouped-query)) notmuch-search-oldest-first)))
  + (concat notmuch-search-query-string  and 
   grouped-query)) notmuch-search-oldest-first
   notmuch-search-maxitems)))
 
 This causes notmuch-search-filter to fail (repeatedly), since `notmuch-search'
 expects a TARGET-THREAD (or nil) as its third parameter, but is given
 `notmuch-search-maxitems' instead.

Oh, yes, totally broken.

   
   (defun notmuch-search-filter-by-tag (tag)
 Filter the current search results based on a single tag.
  @@ -1006,7 +1039,7 @@ Runs a new search matching only messages that match 
  both the
   current search results AND that are tagged with the given tag.
 (interactive
  (list (notmuch-select-tag-with-completion Filter by tag: )))
  -  (notmuch-search (concat notmuch-search-query-string  and tag: tag) 
  notmuch-search-oldest-first))
  +  (notmuch-search (concat notmuch-search-query-string  and tag:
 tag) notmuch-search-oldest-first notmuch-search-maxitems))
 
 Same here.

Yup.

Many thanks for the review. I'll fix these for myself so I might send a
v2 just as well.


BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] docs: man, online help, and NEWS updated for notmuch restore --match

2011-10-29 Thread David Bremner 
From: David Bremner brem...@debian.org

I also reformatted the notmuch restore --accumulate docs to make it
work better with multiple options.
---
Better late docs than no docs...
also pushed to branch nmbug on git://pivot.cs.unb.ca/notmuch.git

 NEWS  |   12 
 notmuch.1 |   19 +--
 notmuch.c |   14 +++---
 3 files changed, 36 insertions(+), 9 deletions(-)

diff --git a/NEWS b/NEWS
index e00452a..9034d5c 100644
--- a/NEWS
+++ b/NEWS
@@ -4,11 +4,15 @@ Notmuch 0.10 (2011-xx-xx)
 New command-line features
 -
 
-Add notmuch restore --accumulate option
+Add --accumulate and --match options to notmuch restore.
 
-  The --accumulate switch causes the union of the existing and new tags to be
-  applied, instead of replacing each message's tags as they are read in from
-  the dump file.
+  The --accumulate switch causes the union of the existing and new
+  tags to be applied, instead of replacing each message's tags as they
+  are read in from the dump file.
+
+  The --match=regex option restricts the restore operation to tags
+  matching the regular expression regex. One potential application is
+  the implementation of namespaces for tags.
 
 Add search terms to  notmuch dump
 
diff --git a/notmuch.1 b/notmuch.1
index bba479e..57b2062 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -480,7 +480,7 @@ section below for details of the supported syntax for 
search-terms.
 .RE
 
 .TP
-.BR restore  [--accumulate] [filename]
+.BR restore  [--accumulate] [--match=regex] [filename]
 
 Restores the tags from the given file (see
 .BR notmuch dump ).
@@ -494,10 +494,25 @@ So if you've previously been using sup for mail, then the
 command provides you a way to import all of your tags (or labels as
 sup calls them).
 
-The --accumulate switch causes the union of the existing and new tags to be
+Supported options for
+.B restore
+include
+.RS
+.TP 4
+.BR \-\-accumulate
+The union of the existing and new tags is
 applied, instead of replacing each message's tags as they are read in from the
 dump file.
 .RE
+.RS
+.TP 4
+.BR \-\-match=regex
+add or delete only tags matching 
+.B regex.
+Other tags in the input (or the database) are ignored.
+.RE
+
+.RE
 
 The
 .B part
diff --git a/notmuch.c b/notmuch.c
index e004c6c..3a51fee 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -401,9 +401,17 @@ static command_t commands[] = {
   \tSo if you've previously been using sup for mail, then the\n
   \t\notmuch restore\ command provides you a way to import\n
   \tall of your tags (or labels as sup calls them).\n
-  \tThe --accumulate switch causes the union of the existing and new\n
-  \ttags to be applied, instead of replacing each message's tags as\n
-  \tthey are read in from the dump file.},
+  \tSupported options for restore include\n
+  \n
+  \t--accumulate\n
+  \t  The union of the existing and new tags is applied, instead of\n
+  \t  replacing each message's tags as they are read  in  from  the\n
+  \t  dump file.\n
+  \n
+  \t--match=regex\n
+  \t  Add or delete only tags matching regex.  Other tags in the\n
+  \t  input (or the database) are ignored.\n
+},
 { config, notmuch_config_command,
   [get|set] section.item [value ...],
   Get or set settings in the notmuch configuration file.,
-- 
1.7.6.3

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] contrib/nmbug: new script for sharing tags with prefix notmuch::

2011-10-29 Thread Ali Polatel 

On Sat, Oct 29, 2011 at 01:45:07PM -0300, David Bremner wrote:

From: David Bremner brem...@debian.org

In this initial version, we take care of only the base import and
export of the appropriate tags in line oriented format amenable to
easy merging.

The current plan to use git to share tags and resolve conflicts.
---
contrib/nmbug |   69 +
1 files changed, 69 insertions(+), 0 deletions(-)
create mode 100755 contrib/nmbug

diff --git a/contrib/nmbug b/contrib/nmbug
new file mode 100755
index 000..e9d1b14
--- /dev/null
+++ b/contrib/nmbug
@@ -0,0 +1,69 @@
+#!/bin/bash


^^ I'd make that:
#!/usr/bin/env bash
for systems where bash is not the default shell.


+# Copyright (c) 2011 David Bremner
+# License: same as notmuch
+
+NMHOME=${HOME}/.nmbug
+
+NMTAGS=($(notmuch search --output=tags *|grep ^notmuch::))
+
+function dump() {
+notmuch dump -- $(printf   tag:%s ${NMTAGS[*]}) |\
+while read  -r msgid rest
+do
+   outfile=$NMHOME/tags/$(echo $msgid | sha1sum - | cut -f1 -d' ')
+   echo $outfile
+   printf msg-id: %s\n $msgid  $outfile
+   tmp=${rest#\(}
+   read -r -a tags ${tmp%\)};
+   for tag in ${tags[@]}; do
+   case $tag in
+   notmuch::*)
+   echo tag: $tag  $outfile
+   ;;
+   *)
+   # nothing
+   esac
+   done
+done
+
+}
+
+function cat_file() {
+tags=
+id=
+cat $1 |\
+while read -r what data
+do
+   case $what in
+   msg-id:)
+   printf %s ( $data
+   ;;
+   tag:)
+   printf %s  $data
+   ;;
+   *)
+   echo Syntax error $what
+   exit 1
+   esac
+done
+echo )
+}
+
+function restore() {
+find $NMHOME/tags -type f |\
+while read -r filename
+do
+   cat_file $filename
+done | notmuch restore --match=notmuch::
+}
+case $1 in
+dump)
+   dump
+   ;;
+restore)
+   restore
+   ;;
+*)
+   echo unknown command $1;
+esac
+
--
1.7.6.3

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


pgpRivR7vRz02.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] contrib/nmbug: new script for sharing tags with prefix notmuch::

2011-10-29 Thread David Bremner 
On Sun, 30 Oct 2011 00:11:35 +0300, Ali Polatel pola...@gmail.com wrote:
 ^^ I'd make that:
 #!/usr/bin/env bash
 for systems where bash is not the default shell.
 

OK, I'll do that in the next version. I always forget about those people
who install bash in funny places.

d
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: [PATCH] contrib/nmbug: new script for sharing tags with prefix notmuch::

2011-10-29 Thread Ali Polatel 

On Sat, Oct 29, 2011 at 07:12:52PM -0300, David Bremner wrote:

On Sun, 30 Oct 2011 00:11:35 +0300, Ali Polatel pola...@gmail.com wrote:

^^ I'd make that:
#!/usr/bin/env bash
for systems where bash is not the default shell.



OK, I'll do that in the next version. I always forget about those people
who install bash in funny places.


Cool!
Just to note, it's the standards, not the people, which are funny...


d


-alip


pgpvandoqzlIl.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Experimental Tag Sharing

2011-10-29 Thread Jameson Graef Rollins 
On Sat, 29 Oct 2011 13:45:06 -0300, David Bremner da...@tethera.net wrote:
 OK, here is my rough and ready attempt at tag sharing.  I figure we
 can smooth out the rough edges if/when we agree on a set of tags and
 preferably on an on-disk format.

Great!  Thank you so much for working on this, David.  This is very
exciting work.

Comments below.

 How to play?
 
 - - Apply all the patches in this thread (starting
   id:1319383133-11006-1-git-send-email-da...@tethera.net)

These patches did not apply cleanly for me from the list.  I was able to
get them from bremner's nmbug branch [0], though.

[0] git://pivot.cs.unb.ca/git/notmuch

 $ nmbug dump
 $ cd $HOME/.nmbug  git add tags  git commit

This last command seems to be included in nmbug as commit.  Fwiw, I
don't like having to do two separate operations for dump and commit.
Could they be merged into a single operation?

 now you have to get the changes into the master repo.  I think
 hosting this on git.notmuchmail.org in a seperate repo will make
 sense, but for now, if you want to participate in the experiment, send
 me your public key (ideally in a gpg signed mail, but if not, oh well), 
 and you can push to my repo at
 
 g...@pivot.cs.unb.ca:pub/nmbug

While I think having a central shared tag repo is ok to get started, I
would really like to see this work in a distributed way.  I don't think
it's impossible to extend this model you have here to work in a
distributed way, though, so that will be something good to work on down
the line.

 Probably the most crucial thing is that we agree on some set of tags.
 Here is the set of tags I am working with (output from 
  notmuch search --output=tags * | grep ^notmuch:: 
 )
 
 notmuch::bug   is a bug report
 notmuch::feature   provides a new feature
 notmuch::fix   fixes a bug
 notmuch::obsolete  replaced by some other patch
 notmuch::patch 
 notmuch::portability   improves portability
 notmuch::pushedis pushed to master
 notmuch::reviewis a review
 notmuch::reviewed  has been (well reviewed)
 notmuch::test  provides a new test/or improves testing
 
 The prefix notmuch:: is hardcoded into the script, but everything
 else is easy to change with the usual notmuch operations.  Of course
 we can change the prefix too. I decided on :: because it needed less
 escaping.

I think these are a fine set to start with.

It might be nice to let the user configure what prefix they use
(ie. what goes before the :: (currently notmuch)).

Does this system support the removing of tags?  I guess I need to see
the documentation for the restore --match option...

It's too bad there's not a way to do scoped tag searches in notmuch
(ie. notmuch search tag:notmuch::*).  Any idea how hard it would be to
support something like that?

 On think I'm not sure about is how to handle ideas like voting, and
 signing off with tags. I suspect we can come up with some
 conventions like notmuch::review1, or
 notmuch::vote1..notmuch::vote5. But maybe people have less kludgy
 ideas.

What issue are you trying to solve here?  Whatever it is, I don't think
I like enumerating tags as the solution.

Again, thanks so much for working on this, David.  This is going to be
very cool.  I really look forward to fleshing it out, and using it!

jamie.


pgpTuchttBmoH.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

Re: Experimental Tag Sharing

2011-10-29 Thread David Bremner 
On Sat, 29 Oct 2011 17:44:56 -0700, Jameson Graef Rollins 
jroll...@finestructure.net wrote:
 On Sat, 29 Oct 2011 13:45:06 -0300, David Bremner da...@tethera.net wrote:
 
 These patches did not apply cleanly for me from the list.  I was able to
 get them from bremner's nmbug branch [0], though.
 [0] git://pivot.cs.unb.ca/git/notmuch

I did resend the one patch that we figured out was corrupted, but for
the moment I will just rebase the nmbug branch that jrollins mentioned.

  $ nmbug dump
  $ cd $HOME/.nmbug  git add tags  git commit
[snip]

 Could they be merged into a single operation?

Yes, commit now (in my git repo) does both the dump and the git
commit.

 While I think having a central shared tag repo is ok to get started, I
 would really like to see this work in a distributed way.

So far I don't see any technical issues with working in a distributed
way, just social ones. Whatever organization we set up, I think it is
important for there to be a way to submit and manage bug reports for
outsiders. Of course, requiring a notmuch install is already a burden.

 It might be nice to let the user configure what prefix they use
 (ie. what goes before the :: (currently notmuch)).

There are two simple-matters-of-programming here. One is how to do the
configuration; probably nmbug config can be an alias for git config
nmbug.$1 or something. The other is translating between prefixes.
Neither is hard, I guess.

 Does this system support the removing of tags?  I guess I need to see
 the documentation for the restore --match option...

Yes, barring bugs, removing a tag in notmuch and running nmbug commit
should remove from the files in ~/.nmbug/tags

 It's too bad there's not a way to do scoped tag searches in notmuch
 (ie. notmuch search tag:notmuch::*).  Any idea how hard it would be to
 support something like that?

I don't know. Carl thought it should work, but it doesn't seem to. 
My uneducated guess is it has something to do with query parsing.

  On think I'm not sure about is how to handle ideas like voting, and
  signing off with tags. I suspect we can come up with some

 What issue are you trying to solve here?  Whatever it is, I don't think
 I like enumerating tags as the solution.

I'm trying to handle the issue of sharing non-boolean data. 
For example, 
 - how can we record priorities bugs/patches
 - how can we conveniently find all of the emacs patches (b.t.w I
   started using notmuch::emacs for those) that jrollins has endorsed
   but are not pushed.

Of course, nmbug could share more metadata than tags. OTOH I'd like to
avoid completely reinventing a distributed [0] bug tracker.

David


[0] well, at least potentially distributed, depending on definitions.







pgpzv0CvVuq14.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[RFC PATCH 0/3] lib/cli/emacs: limit number of messages in search results

2011-10-29 Thread Jani Nikula 
Hi, here are a few RFC patches that add support for limiting the number of
messages in search results. The main goal was emacs; the lib/cli patches could
probably be better thought out to be more general.

The emacs interface was inspired by vc-print-log in Emacs vc.el. You'll get
buttons [Show 2X messages] and [Show unlimited messages] at the end of search
results to expand the results. You can use the 2X button repeatedly to increase
the limit for the buffer.

When the bulk of the work was done, I was told in IRC that patches exist
towards a similar goal:
 id:"87vctrq4or.fsf at hackervisions.org"
 id:"8739gyw0zh.fsf at opensourcematters.org"

My approach is slightly different. I think it's easier and better to do this in
lib rather than just in cli.

BR,
Jani.


Jani Nikula (3):
  lib: add support for limiting the number of search results
  cli: add support for limiting the number of search results
  emacs: support limiting the number of messages shown in search
results

 emacs/notmuch-hello.el |   19 ---
 emacs/notmuch.el   |   43 ++-
 lib/notmuch.h  |3 +++
 lib/query.cc   |   26 --
 notmuch-search.c   |   10 ++
 5 files changed, 91 insertions(+), 10 deletions(-)

-- 
1.7.5.4

[RFC PATCH 1/3] lib: add support for limiting the number of search results

2011-10-29 Thread Jani Nikula 
Add a function to support limiting the number of messages in search
results. This is a fairly straightforward implementation just to support
the following patches. The proper design should probably support paging of
results (i.e. first give me results 0...49, then 50...99, etc.) That should
not be too difficult, as long as the library interface is properly thought
out.

Signed-off-by: Jani Nikula 
---
 lib/notmuch.h |3 +++
 lib/query.cc  |   26 --
 2 files changed, 27 insertions(+), 2 deletions(-)

diff --git a/lib/notmuch.h b/lib/notmuch.h
index c4330e4..b5ef030 100644
--- a/lib/notmuch.h
+++ b/lib/notmuch.h
@@ -457,6 +457,9 @@ notmuch_query_set_sort (notmuch_query_t *query, 
notmuch_sort_t sort);
 notmuch_sort_t
 notmuch_query_get_sort (notmuch_query_t *query);

+void
+notmuch_query_set_maxitems (notmuch_query_t *query, unsigned int maxitems);
+
 /* Execute a query for threads, returning a notmuch_threads_t object
  * which can be used to iterate over the results. The returned threads
  * object is owned by the query and as such, will only be valid until
diff --git a/lib/query.cc b/lib/query.cc
index 6f02b04..04dfbc5 100644
--- a/lib/query.cc
+++ b/lib/query.cc
@@ -27,6 +27,7 @@ struct _notmuch_query {
 notmuch_database_t *notmuch;
 const char *query_string;
 notmuch_sort_t sort;
+Xapian::doccount maxitems;
 };

 typedef struct _notmuch_mset_messages {
@@ -76,6 +77,8 @@ notmuch_query_create (notmuch_database_t *notmuch,

 query->sort = NOTMUCH_SORT_NEWEST_FIRST;

+query->maxitems = 0;
+
 return query;
 }

@@ -97,6 +100,12 @@ notmuch_query_get_sort (notmuch_query_t *query)
 return query->sort;
 }

+void
+notmuch_query_set_maxitems(notmuch_query_t *query, unsigned int maxitems)
+{
+query->maxitems = maxitems;
+}
+
 /* We end up having to call the destructors explicitly because we had
  * to use "placement new" in order to initialize C++ objects within a
  * block that we allocated with talloc. So C++ is making talloc
@@ -181,8 +190,21 @@ notmuch_query_search_messages (notmuch_query_t *query)

mset = enquire.get_mset (0, notmuch->xapian_db->get_doccount ());

-   messages->iterator = mset.begin ();
-   messages->iterator_end = mset.end ();
+   if (query->maxitems && query->maxitems < mset.size()) {
+   if (query->sort == NOTMUCH_SORT_OLDEST_FIRST) {
+   /* Sort oldest first, but return the newest messages. */
+   messages->iterator = mset[mset.size() - query->maxitems];
+   messages->iterator_end = mset.end ();
+   } else {
+   /* This path could be optimized by using maxitems in
+* enquire.get_mset(). */
+   messages->iterator = mset.begin ();
+   messages->iterator_end = mset[query->maxitems];
+   }
+   } else {
+   messages->iterator = mset.begin ();
+   messages->iterator_end = mset.end ();
+   }

return >base;

-- 
1.7.5.4

[RFC PATCH 2/3] cli: add support for limiting the number of search results

2011-10-29 Thread Jani Nikula 
Add command line parameter --maxitems=N to notmuch search to limit the
number of displayed messages to N.

These two are equal:

$ notmuch search --output=messages --sort=newest-first --maxitems=10 SEARCH
$ notmuch search --output=messages --sort=newest-first SEARCH | head

As are these:

$ notmuch search --output=messages --sort=oldest-first --maxitems=10 SEARCH
$ notmuch search --output=messages --sort=oldest-first SEARCH | tail

Note that N refers to the maximum amount of messages, even for
--output=threads.

Signed-off-by: Jani Nikula 
---
 notmuch-search.c |   10 ++
 1 files changed, 10 insertions(+), 0 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 6f04d9a..a3a6475 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -394,6 +394,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 const search_format_t *format = _text;
 int i, ret;
 output_t output = OUTPUT_SUMMARY;
+unsigned int maxitems = 0;

 argc--; argv++; /* skip subcommand argument */

@@ -412,6 +413,14 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
fprintf (stderr, "Invalid value for --sort: %s\n", opt);
return 1;
}
+   } else if (STRNCMP_LITERAL (argv[i], "--maxitems=") == 0) {
+   const char *p;
+   opt = argv[i] + sizeof ("--maxitems=") - 1;
+   maxitems = strtoul(opt, , 10);
+   if (*opt == '\0' || p == opt || *p != '\0') {
+   fprintf (stderr, "Invalid value for --maxitems: %s\n", opt);
+   return 1;
+   }
} else if (STRNCMP_LITERAL (argv[i], "--format=") == 0) {
opt = argv[i] + sizeof ("--format=") - 1;
if (strcmp (opt, "text") == 0) {
@@ -473,6 +482,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 }

 notmuch_query_set_sort (query, sort);
+notmuch_query_set_maxitems (query, maxitems);

 switch (output) {
 default:
-- 
1.7.5.4

[RFC PATCH 3/3] emacs: support limiting the number of messages shown in search results

2011-10-29 Thread Jani Nikula 
Add support for limiting the maximum number of messages initially displayed
in search results. When enabled, the search results will contain push
buttons to double the number of messages displayed or to show unlimited
messages.

The approach is inspired by vc-print-log in Emacs vc.el.

Signed-off-by: Jani Nikula 
---
 emacs/notmuch-hello.el |   19 ---
 emacs/notmuch.el   |   43 ++-
 2 files changed, 54 insertions(+), 8 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 65fde75..16eecb6 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -26,7 +26,7 @@
 (require 'notmuch-lib)
 (require 'notmuch-mua)

-(declare-function notmuch-search "notmuch" (query  oldest-first 
target-thread target-line continuation))
+(declare-function notmuch-search "notmuch" (query  oldest-first 
target-thread target-line continuation maxitems))
 (declare-function notmuch-poll "notmuch" ())

 (defvar notmuch-hello-search-bar-marker nil
@@ -37,6 +37,18 @@
   :type 'integer
   :group 'notmuch)

+(defcustom notmuch-search-maxitems 100
+  "The maximum number of messages to show in search results.
+
+This variables controls the maximum number of messages to
+initially show in search results. Set to 0 to unlimited. If
+non-zero, the search results will contain push buttons to double
+the number (can be repeated) or show unlimited number of
+messages. Note that this controls the number of messages; in a
+threaded view this is not the number of lines to show."
+  :type 'integer
+  :group 'notmuch)
+
 (defcustom notmuch-show-empty-saved-searches nil
   "Should saved searches with no messages be listed?"
   :type 'boolean
@@ -151,7 +163,7 @@ Typically \",\" in the US and UK and \".\" in Europe."
 (defun notmuch-hello-search (search)
   (let ((search (notmuch-hello-trim search)))
 (notmuch-hello-remember-search search)
-(notmuch-search search notmuch-search-oldest-first nil nil 
#'notmuch-hello-search-continuation)))
+(notmuch-search search notmuch-search-oldest-first nil nil 
#'notmuch-hello-search-continuation notmuch-search-maxitems)))

 (defun notmuch-hello-add-saved-search (widget)
   (interactive)
@@ -201,7 +213,8 @@ diagonal."
   (notmuch-search (widget-get widget
  :notmuch-search-terms)
  notmuch-search-oldest-first
- nil nil #'notmuch-hello-search-continuation))
+ nil nil #'notmuch-hello-search-continuation
+ notmuch-search-maxitems))

 (defun notmuch-saved-search-count (search)
   (car (process-lines notmuch-command "count" search)))
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index f11ec24..bb805da 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -193,6 +193,7 @@ For a mouse binding, return nil."

 (defvar notmuch-search-mode-map
   (let ((map (make-sparse-keymap)))
+(set-keymap-parent map widget-keymap)
 (define-key map "?" 'notmuch-help)
 (define-key map "q" 'notmuch-search-quit)
 (define-key map "x" 'notmuch-search-quit)
@@ -217,7 +218,13 @@ For a mouse binding, return nil."
 (define-key map "a" 'notmuch-search-archive-thread)
 (define-key map "-" 'notmuch-search-remove-tag)
 (define-key map "+" 'notmuch-search-add-tag)
-(define-key map (kbd "RET") 'notmuch-search-show-thread)
+; Some hackery to allow RET both on buttons and messages. There's probably 
a
+; better way to do this...
+(define-key map (kbd "RET") '(lambda (pos)
+  (interactive "@d")
+  (if (get-char-property pos 'button)
+  (widget-button-press pos)
+(notmuch-search-show-thread
 (define-key map (kbd "M-RET") 'notmuch-search-show-thread-crypto-switch)
 map)
   "Keymap for \"notmuch search\" buffers.")
@@ -239,6 +246,7 @@ For a mouse binding, return nil."
 (defvar notmuch-search-target-thread)
 (defvar notmuch-search-target-line)
 (defvar notmuch-search-continuation)
+(defvar notmuch-search-maxitems)

 (defvar notmuch-search-disjunctive-regexp  "\\<[oO][rR]\\>")

@@ -373,6 +381,7 @@ Complete list of currently available key bindings:
   (make-local-variable 'notmuch-search-oldest-first)
   (make-local-variable 'notmuch-search-target-thread)
   (make-local-variable 'notmuch-search-target-line)
+  (make-local-variable 'notmuch-search-maxitems)
   (set (make-local-variable 'notmuch-search-continuation) nil)
   (set (make-local-variable 'scroll-preserve-screen-position) t)
   (add-to-invisibility-spec 'notmuch-search)
@@ -633,6 +642,8 @@ This function advances the next thread when finished."
(insert "End of search results.")
(if (not (= exit-status 0))
(insert (format " (process returned %d)" 
exit-status)))
+   (if notmuch-search-maxitems
+

No subject

2011-10-29 Thread David Bremner 
These still need doc updates, but you can probably figure how to use them by 
reviewing the patches ;).

[PATCH 1/4] notmuch-restore: add tests for new --match command line argument.

2011-10-29 Thread David Bremner 
From: David Bremner 

The syntax is notmuch restore --match=regex

We check here that
- missing regex fails
- bad regex fails
- good regex succeeds (at least the command line argument parsing).
---
 test/dump-restore |   17 -
 1 files changed, 16 insertions(+), 1 deletions(-)

diff --git a/test/dump-restore b/test/dump-restore
index 439e998..c176b52 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -50,9 +50,24 @@ test_expect_success 'Restore with nothing to do, III' \
   test_cmp dump.expected dump.actual'

 # notmuch restore currently only considers the first argument.
-test_expect_success 'Invalid restore invocation' \
+test_expect_success 'restore extra argument' \
   'test_must_fail notmuch restore dump.expected another_one'

+
+test_begin_subtest 'restore --match #missing arg'
+test_subtest_known_broken
+test_expect_equal "restore: option '--match' requires an argument"\
+  "$(notmuch restore --match 2>&1)"
+
+test_begin_subtest 'restore --match='
+test_subtest_known_broken
+test_expect_equal 'compiling regex notmuch.*[: Invalid regular expression'\
+  "$(notmuch restore --match='notmuch.*[' 2>&1)"
+
+test_subtest_known_broken
+test_expect_success 'restore --match=' \
+'notmuch restore --match="notmuch.*" < /dev/null > /dev/null 2>&1'
+
 test_begin_subtest "dump outfile"
 notmuch dump dump-outfile.actual
 test_expect_equal_file dump.expected dump-outfile.actual
-- 
1.7.6.3

[PATCH 3/4] test/dump-restore: add tests for notmuch restore --match

2011-10-29 Thread David Bremner 
From: David Bremner 

The first test passes now because it only needs command line parsing.
The other three are marked broken, with functionality to follow.
---
 test/dump-restore |   43 +++
 1 files changed, 43 insertions(+), 0 deletions(-)

diff --git a/test/dump-restore b/test/dump-restore
index c6089f9..18925a4 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -65,6 +65,49 @@ test_expect_equal 'compiling regex notmuch.*[: Invalid 
regular expression'\
 test_expect_success 'restore --match=' \
 'notmuch restore --match="notmuch.*" < /dev/null > /dev/null 2>&1'

+test_expect_success 'Restoring with trivial match' \
+  'notmuch restore --match="." dump.expected &&
+  notmuch dump > dump.actual &&
+  test_cmp dump.expected dump.actual'
+
+sed 's/inbox\|unread\|signed//g' < dump-ABC_DEF.expected > dump-ABC_DEF-only
+
+test_subtest_known_broken
+test_expect_success 'Simulate accumulate with match' \
+  'notmuch restore < dump.expected &&
+   notmuch restore --match="(ABC|DEF)" < dump-ABC_DEF-only &&
+   notmuch dump > dump-simulate.actual &&
+   notmuch restore < dump.expected &&
+   test_cmp dump-ABC_DEF.expected dump-simulate.actual'
+
+test_subtest_known_broken
+test_expect_success 'clear only matched tags' \
+  'notmuch restore < dump-ABC_DEF.expected &&
+   notmuch restore --match="(ABC|DEF)" < clear.expected &&
+   notmuch dump > dump-clear-match.actual &&
+   notmuch restore < dump.expected &&
+   test_cmp dump.expected dump-clear-match.actual'
+
+test_subtest_known_broken
+test_expect_success 'import only matched tags' \
+  'notmuch restore < dump.expected &&
+   notmuch restore --match="(inbox|unread)" < dump-ABC_DEF.expected &&
+   notmuch dump > dump-import-match.actual &&
+   notmuch restore < dump.expected &&
+   test_cmp dump.expected dump-import-match.actual'
+
+sed 's/inbox\|unread\|signed\|ABC//g' < dump-ABC_DEF.expected > dump-DEF-only
+sed 's/inbox\|unread\|signed\|DEF//g' < dump-ABC_DEF.expected > dump-ABC-only
+
+test_subtest_known_broken
+test_expect_success 'combine --match and --accumulate' \
+  'notmuch restore < dump.expected &&
+   notmuch restore --match="(ABC|DEF)" < dump-ABC-only &&
+   notmuch restore --accumulate --match="(ABC|DEF)" < dump-DEF-only &&
+   notmuch dump > dump-combine-match.actual &&
+   notmuch restore < dump.expected &&
+   test_cmp dump-ABC_DEF.expected dump-combine-match.actual'
+
 test_begin_subtest "dump outfile"
 notmuch dump dump-outfile.actual
 test_expect_equal_file dump.expected dump-outfile.actual
-- 
1.7.6.3

[PATCH 4/4] notmuch-restore: implement --match functionality

2011-10-29 Thread David Bremner 
From: David Bremner 

notmuch restore --match= will for each message in the dump
file, delete any tags from the database matching  and add any
from the dump file matching .

I tried to keep this simple, so in particular turning on regex
matching disables one of the optimizations.
---
 notmuch-restore.c |   16 +---
 test/dump-restore |4 
 2 files changed, 13 insertions(+), 7 deletions(-)

diff --git a/notmuch-restore.c b/notmuch-restore.c
index e5ac162..a1b6b81 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -64,7 +64,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char 
*argv[])
break;
case 'm':
match_enabled = TRUE;
-   if ( xregcomp (_regex, optarg, REG_EXTENDED) )
+   if ( xregcomp (_regex, optarg, REG_EXTENDED|REG_NOSUB) )
return 1;
break;
case '?':
@@ -147,6 +147,10 @@ notmuch_restore_command (unused (void *ctx), int argc, 
char *argv[])
{
const char *tag = notmuch_tags_get (db_tags);

+   if (match_enabled && !accumulate &&
+   !regexec (_regex, tag, 0, NULL, 0) )
+   notmuch_message_remove_tag (message, tag);
+
if (db_tags_str)
db_tags_str = talloc_asprintf_append (db_tags_str, " %s", tag);
else
@@ -155,14 +159,15 @@ notmuch_restore_command (unused (void *ctx), int argc, 
char *argv[])

if (((file_tags == NULL || *file_tags == '\0') &&
 (db_tags_str == NULL || *db_tags_str == '\0')) ||
-   (file_tags && db_tags_str && strcmp (file_tags, db_tags_str) == 0))
+   (!match_enabled && file_tags && db_tags_str && 
+strcmp (file_tags, db_tags_str) == 0))
{
goto NEXT_LINE;
}

notmuch_message_freeze (message);

-   if (!accumulate)
+   if (!accumulate && !match_enabled)
notmuch_message_remove_all_tags (message);

next = file_tags;
@@ -170,6 +175,11 @@ notmuch_restore_command (unused (void *ctx), int argc, 
char *argv[])
tag = strsep (, " ");
if (*tag == '\0')
continue;
+
+   if (match_enabled &&
+   regexec (_regex, tag, 0, NULL, 0))
+   continue;
+
status = notmuch_message_add_tag (message, tag);
if (status) {
fprintf (stderr,
diff --git a/test/dump-restore b/test/dump-restore
index 18925a4..a4517f6 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -72,7 +72,6 @@ test_expect_success 'Restoring with trivial match' \

 sed 's/inbox\|unread\|signed//g' < dump-ABC_DEF.expected > dump-ABC_DEF-only

-test_subtest_known_broken
 test_expect_success 'Simulate accumulate with match' \
   'notmuch restore < dump.expected &&
notmuch restore --match="(ABC|DEF)" < dump-ABC_DEF-only &&
@@ -80,7 +79,6 @@ test_expect_success 'Simulate accumulate with match' \
notmuch restore < dump.expected &&
test_cmp dump-ABC_DEF.expected dump-simulate.actual'

-test_subtest_known_broken
 test_expect_success 'clear only matched tags' \
   'notmuch restore < dump-ABC_DEF.expected &&
notmuch restore --match="(ABC|DEF)" < clear.expected &&
@@ -88,7 +86,6 @@ test_expect_success 'clear only matched tags' \
notmuch restore < dump.expected &&
test_cmp dump.expected dump-clear-match.actual'

-test_subtest_known_broken
 test_expect_success 'import only matched tags' \
   'notmuch restore < dump.expected &&
notmuch restore --match="(inbox|unread)" < dump-ABC_DEF.expected &&
@@ -99,7 +96,6 @@ test_expect_success 'import only matched tags' \
 sed 's/inbox\|unread\|signed\|ABC//g' < dump-ABC_DEF.expected > dump-DEF-only
 sed 's/inbox\|unread\|signed\|DEF//g' < dump-ABC_DEF.expected > dump-ABC-only

-test_subtest_known_broken
 test_expect_success 'combine --match and --accumulate' \
   'notmuch restore < dump.expected &&
notmuch restore --match="(ABC|DEF)" < dump-ABC-only &&
-- 
1.7.6.3

[PATCH 2/4] notmuch-restore: implement argument parsing for --match

2011-10-29 Thread David Bremner 
From: David Bremner 

- recognize the --match option
- require an argument
- check the argument is a correct regex.

Currently the arguments are ignored after parsing. Note that we have
to be a bit careful to avoid creating a resource leak here by error
returning before calling regfree. On the other hand, notmuch is
probably shutting down at that point, so it may not matter much.
---
 notmuch-restore.c |   18 ++
 test/dump-restore |3 ---
 2 files changed, 14 insertions(+), 7 deletions(-)

diff --git a/notmuch-restore.c b/notmuch-restore.c
index 13b4325..e5ac162 100644
--- a/notmuch-restore.c
+++ b/notmuch-restore.c
@@ -29,11 +29,12 @@ notmuch_restore_command (unused (void *ctx), int argc, char 
*argv[])
 notmuch_database_t *notmuch;
 notmuch_bool_t synchronize_flags;
 notmuch_bool_t accumulate = FALSE;
+notmuch_bool_t match_enabled = FALSE;
 FILE *input = stdin;
 char *line = NULL;
 size_t line_size;
 ssize_t line_len;
-regex_t regex;
+regex_t input_regex, match_regex;
 int rerr;

 config = notmuch_config_open (ctx, NULL, NULL);
@@ -49,6 +50,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char 
*argv[])

 struct option options[] = {
{ "accumulate",   no_argument,   0, 'a' },
+   { "match",required_argument, 0, 'm' },
{ 0, 0, 0, 0}
 };

@@ -60,6 +62,11 @@ notmuch_restore_command (unused (void *ctx), int argc, char 
*argv[])
case 'a':
accumulate = 1;
break;
+   case 'm':
+   match_enabled = TRUE;
+   if ( xregcomp (_regex, optarg, REG_EXTENDED) )
+   return 1;
+   break;
case '?':
return 1;
break;
@@ -88,7 +95,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char 
*argv[])
  * non-space characters for the message-id, then one or more
  * spaces, then a list of space-separated tags as a sequence of
  * characters within literal '(' and ')'. */
-if ( xregcomp (,
+if ( xregcomp (_regex,
   "^([^ ]+) \\(([^)]*)\\)$",
   REG_EXTENDED) )
INTERNAL_ERROR("compile time constant regex failed.");
@@ -103,7 +110,7 @@ notmuch_restore_command (unused (void *ctx), int argc, char 
*argv[])

chomp_newline (line);

-   rerr = xregexec (, line, 3, match, 0);
+   rerr = xregexec (_regex, line, 3, match, 0);
if (rerr == REG_NOMATCH)
{
fprintf (stderr, "Warning: Ignoring invalid input line: %s\n",
@@ -186,7 +193,10 @@ notmuch_restore_command (unused (void *ctx), int argc, 
char *argv[])
free (file_tags);
 }

-regfree ();
+regfree (_regex);
+
+if (match_enabled)
+   regfree (_regex);

 if (line)
free (line);
diff --git a/test/dump-restore b/test/dump-restore
index c176b52..c6089f9 100755
--- a/test/dump-restore
+++ b/test/dump-restore
@@ -55,16 +55,13 @@ test_expect_success 'restore extra argument' \


 test_begin_subtest 'restore --match #missing arg'
-test_subtest_known_broken
 test_expect_equal "restore: option '--match' requires an argument"\
   "$(notmuch restore --match 2>&1)"

 test_begin_subtest 'restore --match='
-test_subtest_known_broken
 test_expect_equal 'compiling regex notmuch.*[: Invalid regular expression'\
   "$(notmuch restore --match='notmuch.*[' 2>&1)"

-test_subtest_known_broken
 test_expect_success 'restore --match=' \
 'notmuch restore --match="notmuch.*" < /dev/null > /dev/null 2>&1'

-- 
1.7.6.3

[PATCH] dump: Don't sort.

2011-10-29 Thread Thomas Schwinge 
From: Thomas Schwinge 

This improves usage experience considerably in the given scenario.

---


Hi!

I decided that it'd be useful to put the reasoning and data right next to
the source code (as opposed to putting it into the commit message), for
the next guy to read this code has it all in one place.


Gr??e,
 Thomas


---

 notmuch-dump.c |7 ++-
 1 files changed, 6 insertions(+), 1 deletions(-)

diff --git a/notmuch-dump.c b/notmuch-dump.c
index 7e7bc17..a431e23 100644
--- a/notmuch-dump.c
+++ b/notmuch-dump.c
@@ -45,7 +45,12 @@ notmuch_dump_command (unused (void *ctx), int argc, char 
*argv[])
fprintf (stderr, "Out of memory\n");
return 1;
 }
-notmuch_query_set_sort (query, NOTMUCH_SORT_MESSAGE_ID);
+/* This used to use NOTMUCH_SORT_MESSAGE_ID.  On 2011-10-29, a measurement
+ * on a 372981 messages instance showed that wall time can be reduced from
+ * 28 minutes (sorted by Message-ID) to 15 minutes (unsorted), the latter
+ * being much more ``database-disk-layout-friendly''.  Subsequently sorting
+ * the 25 MiB of data is a no-brainer, if required.  */
+notmuch_query_set_sort (query, NOTMUCH_SORT_UNSORTED);

 if (argc) {
output = fopen (argv[0], "w");
-- 
tg: (3bafdfc..) t/dump_unsorted (depends on: baseline)

[PATCH] restore: Be more liberal in which data to accept.

2011-10-29 Thread Thomas Schwinge 
From: Thomas Schwinge 

There are ``Message-ID''s out in the wild that contain spaces.

---


Hi!

Carl, the main question for you is: does this break sup-import
operability?


Spammers are quite inventive for creating ``interesting Messages-ID''s.
Apparently, notmuch handles these fine internally, but it breaks a
dump/restore cycle:

$ notmuch restore < ~/tmp/Mail-notmuch_dump/dump
No filename given. Reading dump from stdin.
Warning: Ignoring invalid input line: 3791856948.991306994491 at m0.net 
Received:fromdialup-62.215.274.4.dial1.stamford([62.215.274.4]  ([...])
Warning: Ignoring invalid input line: PM200010:29:54 AM ([...])
Warning: Ignoring invalid input line: PM200010:51:48 AM ([...])
Warning: Ignoring invalid input line: PM200011:47:35 AM ([...])
Warning: Ignoring invalid input line: PM200011:48:46 AM ([...])
Warning: Ignoring invalid input line: PM200011:50:10 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:05 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:17 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:18 AM ([...])
Warning: Ignoring invalid input line: PM200012:21:32 AM ([...])
Warning: Ignoring invalid input line: PM20001:48:38 PM ([...])
Warning: Ignoring invalid input line: PM20001:53:07 PM ([...])
Warning: Ignoring invalid input line: PM20004:01:48 AM ([...])
Warning: Ignoring invalid input line: PM20004:01:59 AM ([...])
Warning: Ignoring invalid input line: PM20004:10:44 AM ([...])
Warning: Ignoring invalid input line: PM20004:20:00 AM ([...])
Warning: Ignoring invalid input line: PM20005:06:50 PM ([...])
Warning: Ignoring invalid input line: PM20005:14:17 AM ([...])
Warning: Ignoring invalid input line: PM20005:32:15 PM ([...])
Warning: Ignoring invalid input line: PM20005:32:22 PM ([...])
Warning: Ignoring invalid input line: PM20005:33:05 PM ([...])
Warning: Ignoring invalid input line: PM20005:33:57 AM ([...])
Warning: Ignoring invalid input line: PM20006:24:12 AM ([...])
Warning: Ignoring invalid input line: PM20006:25:04 AM ([...])
Warning: Ignoring invalid input line: PM20006:25:49 AM ([...])
Warning: Ignoring invalid input line: PM20006:26:11 AM ([...])
Warning: Ignoring invalid input line: PM20007:05:34 PM ([...])
Warning: Ignoring invalid input line: PM2000PM 04:09:15 ([...])
Warning: Ignoring invalid input line: PM2000 11:07:41 ([...])
Warning: Ignoring invalid input line: PM2000 12:42:47 ([...])
Warning: Ignoring invalid input line: PM2000 12:42:48 ([...])
Warning: Ignoring invalid input line: PM2000 5:58:28 ([...])
Warning: Ignoring invalid input line: PM2000 6:30:51 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:04 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:09 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:11 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:12 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:37:45 ([...])
Warning: Ignoring invalid input line: Prospect Mailer 2:38:10 ([...])

Thus, dump; remove all tags; restore is not nullipotent, which it should
be.

Especially noteworthy is probably the first one: it happens to have
gotten a Received line mangled into the Message-ID, and it ends with a
space character.

Some more from the freak show:

$MESSAGE_ID ([...])

%CUSTOM_CHAR[8-10]$%CUSTOM_CHAR[8-10]$%CUSTOM_CHAR[8-10]@%CUSTOM_DOMAIN.msn.com 
([...])
%RNDDIGIT1025.%RNDDIGIT15%RNDLCCHAR15%RNDDIGIT110%RNDLCCHAR13@ ([...])
%RNDDIGIT1025.%RNDDIGIT15%RNDLCCHAR15%RNDDIGIT110ucp at yahoo.com ([...])
%RNDDIGIT1025.%RNDDIGIT15%RNDLCCHAR15%RNDDIGIT110vs at yahoo.com ([...])
%RNDDIGIT27eq52md1$9rg57p%RNDDIGIT14$277ts40lsh@%RNDWORD13ivo4068 ([...])
%RNDDIGIT27g10u874$3cqh62f%RNDDIGIT14$7fgo121wnwt@%RNDWORD13quw32712 ([...])
%RNDDIGIT27mog75vx711$541xqm480xc%RNDDIGIT14$031nq1pk@%RNDWORD13av2979 
([...])
%RNDDIGIT27nqf761drk7$7l4mza%RNDDIGIT14$96ijq17zq@%RNDWORD13b1779 ([...])
%RNDDIGIT27q0tcg10$94pcn1mw%RNDDIGIT14$7x77pztx@%RNDWORD13ny7619 ([...])
%RNDDIGIT27uiw866tv49$5c3rg%RNDDIGIT14$6jl43vv@%RNDWORD13uwh17820 ([...])
%RNDDIGIT27x966lug3$0pr016r%RNDDIGIT14$8ye15k@%RNDWORD13qps90907 ([...])

%RNDDIGIT310%RNDLCCHAR15%RNDDIGIT15%RNDLCCHAR15$%RNDDIGIT17%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13$%RNDDIGIT15%RNDLCCHAR13%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13@
 ([...])

%RNDDIGIT310%RNDLCCHAR15%RNDDIGIT15%RNDLCCHAR15$%RNDDIGIT17%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13$%RNDDIGIT15%RNDLCCHAR13%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13
 at bambi ([...])

%RNDDIGIT310%RNDLCCHAR15%RNDDIGIT15%RNDLCCHAR15$%RNDDIGIT17%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13$%RNDDIGIT15%RNDLCCHAR13%RNDDIGIT13%RNDLCCHAR13%RNDDIGIT13
 at wheelchair ([...])
%RNDDIGIT520.%RNDDIGIT110.%RNDDIGIT110 at

[Patch 2/3] Respect window margins in notmuch-hello.el

2011-10-29 Thread Ivy Foster 
Hello again, folks,

The following patch makes notmuch take window margins into
account when calculating the width of the notmuch-hello
window in Emacs. As a side benefit (so that I only had to
write the code once), I've added a named function to return
said width.

Folks using linum-mode should still probably get a copy of
linum-off.el and disable linum for notmuch-hello, though,
since it doesn't set the margin until after the buffer is
done being created, causing text to overflow across the
edge. Alternately, take a look at my next message, with yet
another patch.

Enjoy!

iff

Signed-off-by: Ivy Foster 
>From 0c1a0fa0858e3c22b416ce6bd9ad623198217d34 Mon Sep 17 00:00:00 2001
From: Ivy Foster 
Date: Fri, 28 Oct 2011 23:18:14 -0400
Subject: [PATCH 2/3] Respect margins in new named window-width function

This patch removes a function which is duplicated
several times in notmuch-hello.el, improving
readability. Also, factors in  window margins
when calculating width.
---
 emacs/notmuch-hello.el |   28 ++--
 1 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 65fde75..1a213f0 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -213,25 +213,27 @@ should be. Returns a cons cell `(tags-per-line width)'."
 (cond
  ((integerp notmuch-column-control)
   (max 1
-   (/ (- (window-width) notmuch-hello-indent)
+   (/ (notmuch-hello-available-width)
   ;; Count is 9 wide (8 digits plus space), 1 for the space
   ;; after the name.
   (+ 9 1 (max notmuch-column-control widest)

  ((floatp notmuch-column-control)
-  (let* ((available-width (- (window-width) notmuch-hello-indent))
- (proposed-width (max (* available-width 
notmuch-column-control) widest)))
-(floor available-width proposed-width)))
+  (let ((proposed-width (max
+(* (notmuch-hello-available-width)
+   notmuch-column-control)
+widest)))
+(floor (notmuch-hello-available-width) proposed-width)))

  (t
   (max 1
-   (/ (- (window-width) notmuch-hello-indent)
+   (/ (notmuch-hello-available-width)
   ;; Count is 9 wide (8 digits plus space), 1 for the space
   ;; after the name.
   (+ 9 1 widest)))

 (cons tags-per-line (/ (max 1
-   (- (window-width) notmuch-hello-indent
+   (- (notmuch-hello-available-width)
   ;; Count is 9 wide (8 digits plus
   ;; space), 1 for the space after the
   ;; name.
@@ -458,7 +460,7 @@ Complete list of currently available key bindings:
(widget-create 'editable-field
   ;; Leave some space at the start and end of the
   ;; search boxes.
-  :size (max 8 (- (window-width) notmuch-hello-indent
+  :size (max 8 (- (notmuch-hello-available-width)
   (length "Search: ")))
   :action (lambda (widget  ignore)
 (notmuch-hello-search (widget-value widget
@@ -481,12 +483,13 @@ Complete list of currently available key bindings:
   ;; Don't let the search boxes be
   ;; less than 8 characters wide.
   :size (max 8
- (- (window-width)
+ (-
+  
(notmuch-hello-available-width)
 ;; Leave some space
 ;; at the start and
 ;; end of the
 ;; boxes.
-(* 2 notmuch-hello-indent)
+  notmuch-hello-indent
 ;; 1 for the space
 ;; before the
 ;; `[save]' button. 6
@@ -540,7 +543,7 @@ Complete list of currently available key bindings:
  (widget-insert "Edit saved searches with the `edit' button.\n"))
(widget-insert "Hit RET or click on a saved search or tag name to view 
matching threads.\n")
(widget-insert "`=' refreshes this screen. `s' jumps to the search box. 
`q' to quit.\n")
-   (let ((fill-column (-

[PATCH 3/3] Add notmuch-hello-mode-hook

2011-10-29 Thread Ivy Foster 
The following patch adds a mode-hook to notmuch-hello-mode.
Not much else to say, really.

Would people find hooks for before and/or after adding the
tags useful? Or hooks elsewhere in this mode (or other
modes)?

iff

Signed-off-by: Ivy Foster 
>From e8cdead0222fc85dd01d8a0185b54aed5e0a6e1e Mon Sep 17 00:00:00 2001
From: Ivy Foster 
Date: Fri, 28 Oct 2011 23:47:16 -0400
Subject: [PATCH 3/3] Added notmuch-hello-mode-hook

---
 emacs/notmuch-hello.el |6 ++
 1 files changed, 6 insertions(+), 0 deletions(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 1a213f0..13356c5 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -118,6 +118,11 @@ Typically \",\" in the US and UK and \".\" in Europe."
   :group 'notmuch
   :type 'string)

+(defcustom notmuch-hello-mode-hook nil
+  "Functions called after entering `notmuch-hello-mode'."
+  :group 'notmuch
+  :type 'hook)
+
 (defvar notmuch-hello-url "http://notmuchmail.org;
   "The `notmuch' web site.")

@@ -337,6 +342,7 @@ Complete list of currently available key bindings:
  (use-local-map notmuch-hello-mode-map)
  (setq major-mode 'notmuch-hello-mode
mode-name "notmuch-hello")
+ (run-mode-hooks 'notmuch-hello-mode-hook)
  ;;(setq buffer-read-only t)
 )

-- 
1.7.7.1

Experimental Tag Sharing

2011-10-29 Thread David Bremner 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

OK, here is my rough and ready attempt at tag sharing.  I figure we
can smooth out the rough edges if/when we agree on a set of tags and
preferably on an on-disk format.

How to play?

- - Apply all the patches in this thread (starting
  id:"1319383133-11006-1-git-send-email-david at tethera.net")

- - to git master git clone git://pivot.cs.unb.ca/nmbug.git $HOME/.nmbug

- - put the script nmbug somewhere

# suck in the tags
$ nmbug restore

[ do some tagging ]

$ nmbug dump
$ cd $HOME/.nmbug && git add tags && git commit

now you have to get the changes into the "master" repo.  I think
hosting this on git.notmuchmail.org in a seperate repo will make
sense, but for now, if you want to participate in the experiment, send
me your public key (ideally in a gpg signed mail, but if not, oh well), 
and you can push to my repo at

git at pivot.cs.unb.ca:pub/nmbug

Probably the most crucial thing is that we agree on some set of tags.
Here is the set of tags I am working with (output from 
 notmuch search --output=tags "*" | grep ^notmuch:: 
)

notmuch::bug is a bug report
notmuch::feature provides a new feature
notmuch::fix fixes a bug
notmuch::obsoletereplaced by some other patch
notmuch::patch   
notmuch::portability improves portability
notmuch::pushed  is pushed to master
notmuch::review  is a review
notmuch::reviewedhas been (well reviewed)
notmuch::testprovides a new test/or improves testing

The prefix "notmuch::" is hardcoded into the script, but everything
else is easy to change with the usual notmuch operations.  Of course
we can change the prefix too. I decided on :: because it needed less
escaping.

On think I'm not sure about is how to handle ideas like "voting", and
"signing off" with tags. I suspect we can come up with some
conventions like "notmuch::review1", or
"notmuch::vote1".."notmuch::vote5". But maybe people have less kludgy
ideas.

Remember, perfect is the enemy of good ;)

David
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.11 (GNU/Linux)

iJwEAQECAAYFAk6sLWQACgkQTiiN/0Um85k5JQP+PZ0ycrNTTZZ72PyXbDQzrv4+
dDKopxh+WpeORjWu8gvj+LNMM3A8n0t4/A12mxrG9cUJxg2HTPUtRo9Zi7dFQzct
uGvSOvQ6GocUretW0BaK/yp53v0R7b00qxvxAvUeoZPgIWGtqp52fXW8d0I2yAUn
079okQwPYdbowaPAX1A=
=+E8F
-END PGP SIGNATURE-

[PATCH] contrib/nmbug: new script for sharing tags with prefix notmuch::

2011-10-29 Thread David Bremner 
From: David Bremner 

In this initial version, we take care of only the base import and
export of the appropriate tags in line oriented format amenable to
easy merging.

The current plan to use git to share tags and resolve conflicts.
---
 contrib/nmbug |   69 +
 1 files changed, 69 insertions(+), 0 deletions(-)
 create mode 100755 contrib/nmbug

diff --git a/contrib/nmbug b/contrib/nmbug
new file mode 100755
index 000..e9d1b14
--- /dev/null
+++ b/contrib/nmbug
@@ -0,0 +1,69 @@
+#!/bin/bash
+# Copyright (c) 2011 David Bremner
+# License: same as notmuch
+
+NMHOME=${HOME}/.nmbug
+
+NMTAGS=($(notmuch search --output=tags "*"|grep "^notmuch::"))
+
+function dump() {
+notmuch dump -- $(printf  " tag:%s" ${NMTAGS[*]}) |\
+while read  -r msgid rest 
+do
+   outfile=$NMHOME/tags/$(echo $msgid | sha1sum - | cut -f1 -d' ')
+   echo $outfile
+   printf "msg-id: %s\n" $msgid > $outfile
+   tmp=${rest#\(}
+   read -r -a tags <<<${tmp%\)};
+   for tag in "${tags[@]}"; do
+   case $tag in
+   notmuch::*)
+   echo "tag: $tag" >> $outfile
+   ;;
+   *)
+   # nothing
+   esac
+   done
+done
+
+}
+
+function cat_file() {
+tags=""
+id=""
+cat $1 |\
+while read -r what data 
+do
+   case $what in
+   msg-id:)
+   printf "%s (" $data 
+   ;;
+   tag:)
+   printf "%s " $data
+   ;;
+   *)
+   echo "Syntax error $what"
+   exit 1
+   esac
+done
+echo ")"
+}
+
+function restore() {
+find $NMHOME/tags -type f |\
+while read -r filename
+do
+   cat_file $filename
+done | notmuch restore --match=notmuch::
+}
+case $1 in
+dump)
+   dump
+   ;;
+restore)
+   restore
+   ;;
+*)
+   echo unknown command $1;
+esac
+   
-- 
1.7.6.3

[RFC PATCH 3/3] emacs: support limiting the number of messages shown in search results

2011-10-29 Thread Daniel Schoepe 
On Fri, 28 Oct 2011 23:59:31 +0300, Jani Nikula  wrote:
> Add support for limiting the maximum number of messages initially displayed
> in search results. When enabled, the search results will contain push
> buttons to double the number of messages displayed or to show unlimited
> messages.

Nice patch, as it not only makes searches with a lot of results easier
to use on slower machines/hard drives, but I also find that seeing only
a few dozen threads in the buffer looks more "orderly" to me, compared
to a buffer with hundreds of lines.

A few comments about the code:

> @@ -373,6 +381,7 @@ Complete list of currently available key bindings:
>(make-local-variable 'notmuch-search-oldest-first)
>(make-local-variable 'notmuch-search-target-thread)
>(make-local-variable 'notmuch-search-target-line)
> +  (make-local-variable 'notmuch-search-maxitems)
>(set (make-local-variable 'notmuch-search-continuation) nil)
>(set (make-local-variable 'scroll-preserve-screen-position) t)
>(add-to-invisibility-spec 'notmuch-search)
> @@ -633,6 +642,8 @@ This function advances the next thread when finished."
>   (insert "End of search results.")
>   (if (not (= exit-status 0))
>   (insert (format " (process returned %d)" 
> exit-status)))
> + (if notmuch-search-maxitems
> + (notmuch-search-setup-buttons))

As discussed on IRC, this causes `notmuch-search' to fail if the
maxitems argument is nil.

> +(defun notmuch-search-setup-buttons ()
> +  (widget-insert "")
> +  (widget-create 'push-button
> +  :notify (lambda ( ignore)
> +(set 'notmuch-search-maxitems
> + (* 2 notmuch-search-maxitems))
> +(notmuch-search-refresh-view))
> +  :help-echo "Double the number of messages shown"
> +  "Show 2X messages")
> +  (widget-insert "")
> +  (widget-create 'push-button
> +  :notify (lambda ( ignore)
> +(set 'notmuch-search-maxitems 0)
> +(notmuch-search-refresh-view))
> +  :help-echo "Show all search results"
> +  "Show unlimited messages")
> +  (widget-setup))

I think these notify-actions should be separate functions to make it
easier to bind them to keys.

> +
>  (defcustom notmuch-poll-script ""
>"An external script to incorporate new mail into the notmuch database.
>  
> @@ -997,7 +1030,7 @@ current search results AND the additional query string 
> provided."
>query)))
>  (notmuch-search (if (string= notmuch-search-query-string "*")
>   grouped-query
> -   (concat notmuch-search-query-string " and " 
> grouped-query)) notmuch-search-oldest-first)))
> +   (concat notmuch-search-query-string " and "
>grouped-query)) notmuch-search-oldest-first
>notmuch-search-maxitems)))

This causes notmuch-search-filter to fail (repeatedly), since `notmuch-search'
expects a TARGET-THREAD (or nil) as its third parameter, but is given
`notmuch-search-maxitems' instead.

>  
>  (defun notmuch-search-filter-by-tag (tag)
>"Filter the current search results based on a single tag.
> @@ -1006,7 +1039,7 @@ Runs a new search matching only messages that match 
> both the
>  current search results AND that are tagged with the given tag."
>(interactive
> (list (notmuch-select-tag-with-completion "Filter by tag: ")))
> -  (notmuch-search (concat notmuch-search-query-string " and tag:" tag) 
> notmuch-search-oldest-first))
> +  (notmuch-search (concat notmuch-search-query-string " and tag:"
>tag) notmuch-search-oldest-first notmuch-search-maxitems))

Same here.

Cheers,
Daniel
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20111029/fb564033/attachment.pgp>

[PATCH v2] contrib/nmbug: new script for sharing tags with prefix notmuch::

2011-10-29 Thread David Bremner 
From: David Bremner 

In this initial version, we take care of import and export of the
appropriate tags in line oriented format amenable to easy merging.

We also provide (not very robust/clever) commands commit/push/pull
to deal with a git repo that as already been set up.
---

fixed silly debugging output.
add some convenience commands.

 contrib/nmbug |   77 +
 1 files changed, 77 insertions(+), 0 deletions(-)
 create mode 100755 contrib/nmbug

diff --git a/contrib/nmbug b/contrib/nmbug
new file mode 100755
index 000..a449d60
--- /dev/null
+++ b/contrib/nmbug
@@ -0,0 +1,77 @@
+#!/bin/bash
+# Copyright (c) 2011 David Bremner
+# License: same as notmuch
+
+NMHOME=${HOME}/.nmbug
+
+NMTAGS=($(notmuch search --output=tags "*"|grep "^notmuch::"))
+
+function dump() {
+notmuch dump -- $(printf  " tag:%s" ${NMTAGS[*]}) |\
+while read  -r msgid rest 
+do
+   outfile=$NMHOME/tags/$(echo $msgid | sha1sum - | cut -f1 -d' ')
+   printf "msg-id: %s\n" $msgid > $outfile
+   tmp=${rest#\(}
+   read -r -a tags <<<${tmp%\)};
+   for tag in "${tags[@]}"; do
+   case $tag in
+   notmuch::*)
+   echo "tag: $tag" >> $outfile
+   ;;
+   *)
+   # nothing
+   esac
+   done
+done
+
+}
+
+function cat_file() {
+tags=""
+id=""
+cat $1 |\
+while read -r what data 
+do
+   case $what in
+   msg-id:)
+   printf "%s (" $data 
+   ;;
+   tag:)
+   printf "%s " $data
+   ;;
+   *)
+   echo "Syntax error $what"
+   exit 1
+   esac
+done
+echo ")"
+}
+
+function restore() {
+find $NMHOME/tags -type f |\
+while read -r filename
+do
+   cat_file $filename
+done | notmuch restore --match=notmuch::
+}
+case $1 in
+dump)
+   dump
+   ;;
+restore)
+   restore
+   ;;
+commit)
+   (cd $NMHOME && git add tags && git commit)
+   ;;
+push)
+   (cd $NMHOME && git push)
+   ;;
+pull)
+   (cd $NMHOME && git pull)
+   ;;
+*)
+   echo unknown command $1;
+esac
+   
-- 
1.7.6.3

[RFC PATCH 2/3] cli: add support for limiting the number of search results

2011-10-29 Thread Daniel Schoepe 
On Fri, 28 Oct 2011 23:59:30 +0300, Jani Nikula  wrote:
> @@ -412,6 +413,14 @@ notmuch_search_command (void *ctx, int argc, char 
> *argv[])
>   fprintf (stderr, "Invalid value for --sort: %s\n", opt);
>   return 1;
>   }
> + } else if (STRNCMP_LITERAL (argv[i], "--maxitems=") == 0) {
> + const char *p;
> + opt = argv[i] + sizeof ("--maxitems=") - 1;
> + maxitems = strtoul(opt, , 10);

p should be of type `char *', not `const char *', as it will be
modified by strtoul. (Otherwise, gcc will produce a warning about this).

Cheers,
Daniel
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20111029/0d510152/attachment.pgp>

[PATCH] xutil.c: remove duplicate copies, create new library libutil.a to contain xutil.

2011-10-29 Thread David Bremner 
On Sun, 23 Oct 2011 12:18:53 -0300, David Bremner  wrote:
> From: David Bremner 
> 
> We keep the lib/xutil.c version. As a consequence, also factor out
> _internal_error and associated macros.  It might be overkill to make a
> new file error_util.c for this, but _internal_error does not really
> belong in database.cc.

It seems this patch was sent out base64 encoded, which confuses git-am.
I attach the patch here. You won't be able to git am this message, but
you can at least save the attachment and apply it. Sorry for the
inconvenience.

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 315 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20111029/60aabf1c/attachment-0001.pgp>
-- next part --
A non-text attachment was scrubbed...
Name: 0001-xutil.c-remove-duplicate-copies-create-new-library-l.patch
Type: text/x-diff
Size: 20990 bytes
Desc: xutil patch
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20111029/60aabf1c/attachment-0001.patch>

[RFC PATCH 2/3] cli: add support for limiting the number of search results

2011-10-29 Thread Daniel Schoepe 
On Sat, 29 Oct 2011 23:08:04 +0300, Jani Nikula  wrote:
> strtoul() won't touch the data pointed to by p (it only modifies p), so
> in that sense it could be const, but you're right in that it really
> should be 'char *', just for a more complicated reason. Thanks for
> making me look it up: http://c-faq.com/ansi/constmismatch.html (not the
> best of explanations, perhaps, but gives an idea why the 2nd parameter
> of strtoul() can't be 'const char **').

Oh, right, thanks for the clarification. As I often do in situations
such as this, I confused "const char *" and "char const *".
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20111029/89ea91d2/attachment.pgp>

[PATCH] docs: man, online help, and NEWS updated for notmuch restore --match

2011-10-29 Thread David Bremner 
From: David Bremner 

I also reformatted the notmuch restore --accumulate docs to make it
work better with multiple options.
---
Better late docs than no docs...
also pushed to branch nmbug on git://pivot.cs.unb.ca/notmuch.git

 NEWS  |   12 
 notmuch.1 |   19 +--
 notmuch.c |   14 +++---
 3 files changed, 36 insertions(+), 9 deletions(-)

diff --git a/NEWS b/NEWS
index e00452a..9034d5c 100644
--- a/NEWS
+++ b/NEWS
@@ -4,11 +4,15 @@ Notmuch 0.10 (2011-xx-xx)
 New command-line features
 -

-Add "notmuch restore --accumulate" option
+Add "--accumulate" and "--match" options to "notmuch restore".

-  The --accumulate switch causes the union of the existing and new tags to be
-  applied, instead of replacing each message's tags as they are read in from
-  the dump file.
+  The "--accumulate" switch causes the union of the existing and new
+  tags to be applied, instead of replacing each message's tags as they
+  are read in from the dump file.
+
+  The "--match=" option restricts the restore operation to tags
+  matching the regular expression regex. One potential application is
+  the implementation of namespaces for tags.

 Add search terms to  "notmuch dump"

diff --git a/notmuch.1 b/notmuch.1
index bba479e..57b2062 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -480,7 +480,7 @@ section below for details of the supported syntax for 
.
 .RE

 .TP
-.BR restore " [--accumulate] []"
+.BR restore " [--accumulate] [--match=] []"

 Restores the tags from the given file (see
 .BR "notmuch dump" ")."
@@ -494,10 +494,25 @@ So if you've previously been using sup for mail, then the
 command provides you a way to import all of your tags (or labels as
 sup calls them).

-The --accumulate switch causes the union of the existing and new tags to be
+Supported options for
+.B restore
+include
+.RS
+.TP 4
+.BR \-\-accumulate
+The union of the existing and new tags is
 applied, instead of replacing each message's tags as they are read in from the
 dump file.
 .RE
+.RS
+.TP 4
+.BR \-\-match=
+add or delete only tags matching 
+.B .
+Other tags in the input (or the database) are ignored.
+.RE
+
+.RE

 The
 .B part
diff --git a/notmuch.c b/notmuch.c
index e004c6c..3a51fee 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -401,9 +401,17 @@ static command_t commands[] = {
   "\tSo if you've previously been using sup for mail, then the\n"
   "\t\"notmuch restore\" command provides you a way to import\n"
   "\tall of your tags (or labels as sup calls them).\n"
-  "\tThe --accumulate switch causes the union of the existing and new\n"
-  "\ttags to be applied, instead of replacing each message's tags as\n"
-  "\tthey are read in from the dump file."},
+  "\tSupported options for restore include\n"
+  "\n"
+  "\t--accumulate\n"
+  "\t  The union of the existing and new tags is applied, instead of\n"
+  "\t  replacing each message's tags as they are read  in  from  the\n"
+  "\t  dump file.\n"
+  "\n"
+  "\t--match=\n"
+  "\t  Add or delete only tags matching .  Other tags in the\n"
+  "\t  input (or the database) are ignored.\n"
+},
 { "config", notmuch_config_command,
   "[get|set] . [value ...]",
   "Get or set settings in the notmuch configuration file.",
-- 
1.7.6.3

notmuch in Emacs very slow

2011-10-29 Thread notm...@bontempi.net 
Hello,

I would like to report my experience with notmuch (newbie here).

I absolutely love it. I would like very much to continue using it, but I am 
having a speed issue from inside Emacs.

I have installed notmuch 0.9, on OS X 10.6, using of course the chert database 
format. I am also using Emacs 24.0.90.1 on a quad-core Intel Core i7 (no SSD 
though).

I started doing 'notmuch new' on a Maildir folder with about 3 messages 
(about one fourth of my archive).

At beginning notmuch was indexing about 200 msgs/sec. The more it indexed, the 
slower it became. After about 2 msgs it slowed down to 70 msgs/sec. I am 
not sure if that is a behaviour due to Xapian.

Anyway, after it finished that first part, starting notmuch from inside emacs, 
it says:

"Welcome to notmuch. You have 34,366 messages."

Once I start browsing the inbox (34,208), Emacs displays the initial screenful, 
but the whole list takes quite a while to load, about 50 seconds. After that 
time, in the status bar I see "Top of 1.1M".

That happens every time I am browsing the inbox, in order to assign proper tags.

The real issue for me, though, is that scrolling down the list has a noticeable 
delay, enough to make it basically unusable.

Is there something I do not know that I should do to improve that speed (either 
from within Emacs, or in notmuch config file?

I would be very grateful for any advice or pointer on how to speed up the 
notmuch experience.

Thank you,

Sam