[notmuch] Asynchronous tagging

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 22:04:50 +0100, Jed Brown  wrote:
> >  2) A client changes the flags on a message, for example, when you
> > read a message or mark it as deleted. Maildir stores flags in
> > filenames.
> 
> This seems like a problem.  I'm not familiar with xapian, is it
> necessarily an expensive operation to correct these inconsistencies?

There's not really anything Xapian-specific here. It should be a
relatively easy change to make notmuch do the right thing here. It just
happens that the original author/user of notmuch isn't using anything
that changes his filenames---so I hadn't noticed. :-)

> Matching by thread id ought to be cheap.

Naturally. And that's of course exactly what notmuch does. So in my
usage, the only time "notmuch new" sees a Message-ID that it has seen
before, is when it encounters a duplicate copy of a message. So the code
currently just ignores it.

Mikhail wrote a patch:

1258491078-29658-1-git-send-email-dottedmag at dottedmag.net

that does the simple thing in this case of just noticing whether the old
filename has since been removed, and in this case updating the document
to the new filename.

The problem he ran into is that renames aren't updating mtimes and the
current "notmuch new" has an optimization to not even look at files
unless their mtime is newer than the mtime last seen for the directory
they are in.

So some investigation is needed to see how important that optimization
is, and if it's important to see whether there's another way to keep the
performance while being able to support renames. (Or alternately,
allowing the user to configure an option saying, "I need to support
renames even if that means that notmuch new is a bit slower.").

-Carl


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Jed Brown
On Sat, 21 Nov 2009 17:36:18 -0500, Brett Viren  
wrote:
> How can I purge the index?  I can't locate it.

I believe you can just remove /path/to/maildir/.notmuch

Jed


[notmuch] RFC: Multiple filenames for email messages

2009-11-21 Thread Jan Janak
The comment of _notmuch_message_set_filename says:

   XXX: We should still figure out if we think it's important to store
   multiple filenames for email messages with identical message IDs.

I have lots of such messages in my email collection, both in my local
copy of my Gmail account and also in the local copy of my company's
IMAP account.

My dream mail indexing tool should be able to apply tags automatically
based on, among other things, the name of the directory the message is
stored in. If there are multiple copies of the same message scattered
across multiple directories, I would like to apply more tags.

I assume that most tags will be applied (either manually or
automatically) after 'notmuch-new', I currently do some of it with a
simple shell script. The script does not apply tags based on directory
names yet, but it would make notmuch really flexible if we could do
that *and* if we could get access to all filenames of a particular
message.

I'd like to propose that we store all filenames for email messages in
the database, not just one per message. I'd be happy to work on it and
submit a patch if others think that this would be good to have.

  -- Jan


[notmuch] notmuch vim support

2009-11-21 Thread Bart Trojanowski
* Carl Worth  [091121 15:20]:
> On Sat, 21 Nov 2009 14:14:02 -0500, Bart Trojanowski  
> wrote:
> > Perfect.  How would you like to proceed with the initial integration?
> > Pulling my tree, or me squashing all the changes into a single patch?
> > Also, are you OK with where the files live (ie under a vim directory).
> 
> I'm glad to pull (it will be nice for you to have the accurate
> history). And a vim directory sounds perfect.

Indeed.

> Or should we do interfaces/emacs and interfaces/vim ? I don't know.

I don't really have a preference.  As long as 'make install' at the top
doesn't install both/either blindly.

> Email or pull requests are fine. You're clearly the "owner" of the vim
> stuff, so don't expect any refusals to pull from me. (And feel free to
> ping me out of band if you don't want to clutter the list with pull
> requests---but it's more likely that you'll want to let people on the
> list know about changes anyway.)

Great.  Thanks.  I'll keep the list informed.

-Bart

-- 
WebSig: http://www.jukie.net/~bart/sig/


[notmuch] notmuch vim support

2009-11-21 Thread Bart Trojanowski
* Carl Worth  [091121 15:14]:
> So, well done!

Thank you.

> I think the only thing we noticed is that "filter" doesn't add the "and"
> between the old and new search terms. But Chris may have more to comment
> soon.

I think I fixed that along the way.  I'll verify.

> I don't know how realistic it will be to keep both interfaces totally
> synchronized in terms of look-and-feel, colors, keybindings, etc. But it
> might be nice to do that as much as possible. I think it would be
> helpful if we could have interface discussions as an entire community
> and not split based on choice of editor.

Perfect.

-Bart

-- 
WebSig: http://www.jukie.net/~bart/sig/


[notmuch] Segfault with weird Message-ID

2009-11-21 Thread Mike Hommey
On Fri, Nov 20, 2009 at 10:05:56PM +0100, Mike Hommey wrote:
> On Fri, Nov 20, 2009 at 09:53:37PM +0100, Carl Worth wrote:
> > On Fri, 20 Nov 2009 14:26:25 +0100, Mike Hommey  > glandium.org> wrote:
> > > - for some reason, xapian doesn't want to add the document corresponding
> > >   to this old spam message: notmuch->xapian_db->add_document throws an
> > >   exception.
> > 
> > I think things had just gone wrong long before then.
> 
> I *did* see it throwing an exception from there. The sad thing is that I
> can't reproduce the problem anymore :-/
> 
> > > I can provide the spam if necessary, or can continue debugging the issue
> > > with some guidance.
> > 
> > Thanks for providing it. It turns out that the giant Message-Id value
> > wasn't causing the problem. Instead the message was corrupt by having a
> > stray new line at the third line. (So GMime is seeing only the first two
> > lines of headers). We *used* to have working code to detect this kind of
> > file as "not an email" but again, this broke when we changed
> > notmuch_message_get_header to return "" instead of NULL for missing
> > headers.
> 
> Interestingly, when I first traced on what message the crash was
> happening, I did see notmuch having the message-id in the message_id
> variable.

I just was able to reproduce after starting over.

header isn't "", and message_id is correctly filled. I can also confirm
the exception is thrown from notmuch->xapian_db->add_document.

> FWIW, that was using c05c3f1.

With 3ae12b1, I get the following output:
Error: A Xapian exception occurred. Halting processing.

But I confirm there is no crash, now.

Cheers,

Mike


[notmuch] [PATCH] Support multiple configuration files via $NOTMUCH_CONFIG

2009-11-21 Thread Alec Berryman
If present, $NOTMUCH_CONFIG will be used as the configuration file
location.
---
 notmuch-config.c |   16 +++-
 notmuch-setup.c  |3 ++-
 notmuch.1|7 ---
 notmuch.c|3 ++-
 4 files changed, 19 insertions(+), 10 deletions(-)

I'd like to have multiple configurations for personal and work email;
this solution worked for me.

I've been having trouble with git send-email and this list; sorry if
this comes a few times.

diff --git a/notmuch-config.c b/notmuch-config.c
index aaa0372..fc65d6b 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -132,9 +132,10 @@ get_username_from_passwd_file (void *ctx)
 return name;
 }

-/* Open the named notmuch configuration file. A filename of NULL will
- * be interpreted as the default configuration file
- * ($HOME/.notmuch-config).
+/* Open the named notmuch configuration file. If the filename is NULL,
+ * the value of the environment variable $NOTMUCH_CONFIG will be used.
+ * If $NOTMUCH_CONFIG is unset, the default configuration file
+ * ($HOME/.notmuch-config) will be used.
  *
  * If any error occurs, (out of memory, or a permission-denied error,
  * etc.), this function will print a message to stderr and return
@@ -168,6 +169,7 @@ notmuch_config_open (void *ctx,
 {
 GError *error = NULL;
 int is_new = 0;
+char *notmuch_config_env = NULL;

 if (is_new_ret)
*is_new_ret = 0;
@@ -180,11 +182,15 @@ notmuch_config_open (void *ctx,

 talloc_set_destructor (config, notmuch_config_destructor);

-if (filename)
+if (filename) {
config->filename = talloc_strdup (config, filename);
-else
+} else if ((notmuch_config_env = getenv ("NOTMUCH_CONFIG"))) {
+   config->filename = talloc_strdup (config, notmuch_config_env);
+   notmuch_config_env = NULL;
+} else {
config->filename = talloc_asprintf (config, "%s/.notmuch-config",
getenv ("HOME"));
+}

 config->key_file = g_key_file_new ();

diff --git a/notmuch-setup.c b/notmuch-setup.c
index 482efd2..d06fbf8 100644
--- a/notmuch-setup.c
+++ b/notmuch-setup.c
@@ -76,7 +76,8 @@ welcome_message_post_setup (void)
 "Notmuch is now configured, and the configuration settings are saved in\n"
 "a file in your home directory named .notmuch-config . If you'd like to\n"
 "change the configuration in the future, you can either edit that file\n"
-"directly or run \"notmuch setup\".\n\n"
+"directly or run \"notmuch setup\".  To choose an alternate configuration\n"
+"location, set ${NOTMUCH_CONFIG}.\n\n"

 "The next step is to run \"notmuch new\" which will create a database\n"
 "that indexes all of your mail. Depending on the amount of mail you have\n"
diff --git a/notmuch.1 b/notmuch.1
index 8a3fb40..eeb1a94 100644
--- a/notmuch.1
+++ b/notmuch.1
@@ -53,9 +53,10 @@ Interactively sets up notmuch for first use.
 The setup command will prompt for your full name, your primary email
 address, any alternate email addresses you use, and the directory
 containing your email archives. Your answers will be written to a
-configuration file in ${HOME}/.notmuch-config . This configuration
-file will be created with descriptive comments, making it easy to edit
-by hand later to change the configuration. Or you can run
+configuration file in ${NOTMUCH_CONFIG} (if set) or
+${HOME}/.notmuch-config . This configuration file will be created with
+descriptive comments, making it easy to edit by hand later to change the
+configuration. Or you can run
 .B "notmuch setup"
 again to change the configuration.

diff --git a/notmuch.c b/notmuch.c
index 5cc8e4c..d7ea5aa 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -98,7 +98,8 @@ command_t commands[] = {
   "\t\tThe setup command will prompt for your full name, your primary\n"
   "\t\temail address, any alternate email addresses you use, and the\n"
   "\t\tdirectory containing your email archives. Your answers will be\n"
-  "\t\twritten to a configuration file in ${HOME}/.notmuch-config .\n"
+  "\t\twritten to a configuration file in ${NOTMUCH_CONFIG} (if set)\n"
+  "\t\tor ${HOME}/.notmuch-config.\n"
   "\n"
   "\t\tThis configuration file will be created with descriptive\n"
   "\t\tcomments, making it easy to edit by hand later to change the\n"
-- 
1.6.5




[notmuch] [PATCH] notmuch-config: Fix memleaks.

2009-11-21 Thread Stefan Schmidt
>From 4bb598e6e0fa32d5184be54cb5c66fe364eba374 Mon Sep 17 00:00:00 2001
From: Holger Freyther 
Date: Sat, 21 Nov 2009 22:45:23 +0100
Subject: [PATCH] notmuch-config: Fix memleaks.

While talloc is great we need to free the g_error by hand.

Tested-by: Stefan Schmidt 
Signed-off-by: Holger Freyther 
---
 notmuch-config.c |3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

diff --git a/notmuch-config.c b/notmuch-config.c
index 7252a19..aaa0372 100644
--- a/notmuch-config.c
+++ b/notmuch-config.c
@@ -207,9 +207,11 @@ notmuch_config_open (void *ctx,
fprintf (stderr, "Error reading configuration file %s: %s\n",
 config->filename, error->message);
talloc_free (config);
+   g_error_free (error);
return NULL;
}

+   g_error_free (error);
is_new = 1;
 }

@@ -308,6 +310,7 @@ notmuch_config_save (notmuch_config_t *config)
 if (! g_file_set_contents (config->filename, data, length, &error)) {
fprintf (stderr, "Error saving configuration to %s: %s\n",
 config->filename, error->message);
+   g_error_free (error);
return 1;
 }

-- 
1.6.5.3



[notmuch] [PATCH] lib/database.cc: coding style

2009-11-21 Thread Chris Wilson
Carl claims he must have been distracted when he wrote this...

Signed-off-by: Chris Wilson 
---
 lib/database.cc |   18 +++---
 1 files changed, 11 insertions(+), 7 deletions(-)

diff --git a/lib/database.cc b/lib/database.cc
index 207246c..0f95aa1 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -147,17 +147,20 @@ _find_prefix (const char *name)
 {
 unsigned int i;

-for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_INTERNAL); i++)
+for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_INTERNAL); i++) {
if (strcmp (name, BOOLEAN_PREFIX_INTERNAL[i].name) == 0)
return BOOLEAN_PREFIX_INTERNAL[i].prefix;
+}

-for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++)
+for (i = 0; i < ARRAY_SIZE (BOOLEAN_PREFIX_EXTERNAL); i++) {
if (strcmp (name, BOOLEAN_PREFIX_EXTERNAL[i].name) == 0)
return BOOLEAN_PREFIX_EXTERNAL[i].prefix;
+}

-for (i = 0; i < ARRAY_SIZE (PROBABILISTIC_PREFIX); i++)
+for (i = 0; i < ARRAY_SIZE (PROBABILISTIC_PREFIX); i++) {
if (strcmp (name, PROBABILISTIC_PREFIX[i].name) == 0)
return PROBABILISTIC_PREFIX[i].prefix;
+}

 INTERNAL_ERROR ("No prefix exists for '%s'\n", name);

@@ -293,13 +296,14 @@ skip_space_and_comments (const char **str)
int nesting = 1;
s++;
while (*s && nesting) {
-   if (*s == '(')
+   if (*s == '(') {
nesting++;
-   else if (*s == ')')
+   } else if (*s == ')') {
nesting--;
-   else if (*s == '\\')
+   } else if (*s == '\\') {
if (*(s+1))
s++;
+   }
s++;
}
}
@@ -508,7 +512,7 @@ notmuch_database_open (const char *path)
 error.get_msg().c_str());
notmuch = NULL;
 }
-
+
   DONE:
 if (notmuch_path)
free (notmuch_path);
-- 
1.6.5.3



[notmuch] [PATCH 2/2] notmuch list: A new command to produce various lists.

2009-11-21 Thread Bart Trojanowski
* Carl Worth  [091121 18:08]:
> Another option would be to just call this "notmuch search" and have an
> option to control what is output:
> 
>notmuch search # for threads, as currently
>notmuch search --output=tags
>notmuch search --output=messages
> 
> Actually, I kind of like that. For one thing, it makes it easy to
> eliminate the redundancy I made with the option-parsing in both
> notmuch-search.c and notmuch-search-message.c.

I too prefer this option.  It keeps the interface and code simpler.

Moreover, it might also eliminate the need for 'notmuch count'. Simply
pass --count-only to the search command.

-Bart

-- 
WebSig: http://www.jukie.net/~bart/sig/


[notmuch] [PATCH] _READ_Write -> _READ_WRITE

2009-11-21 Thread Jed Brown
---
 lib/notmuch.h |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

-- next part --
A non-text attachment was scrubbed...
Name: 0001-_READ_Write-_READ_WRITE.patch
Type: text/x-patch
Size: 403 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/9f8bd9be/attachment.bin>


[notmuch] [PATCH] Permit opening the notmuch database in read-only mode.

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 12:06:43 -0800, Keith Packard  wrote:
> On Sat, 21 Nov 2009 19:54:25 +, Chris Wilson  chris-wilson.co.uk> wrote:
> 
> > +typedef enum {
> > +NOTMUCH_DATABASE_MODE_READONLY = 0,
> > +NOTMUCH_DATABASE_MODE_WRITABLE
> > +} notmuch_database_mode_t;
> 
> Wondering if MODE_READWRITE would be more descriptive?

I went with READ_ONLY and READ_WRITE and pushed this.

Thanks, Chris! It's a nice feature.

-Carl


[notmuch] Asynchronous tagging

2009-11-21 Thread Jed Brown
On Sat, 21 Nov 2009 22:00:13 +0100, Carl Worth  wrote:
> Ah, OK. So you made a change on the Gmail side and that caused a file to
> be renamed locally.

yes

> Or did you mean you removed the tag from within emacs? In that case, the
> search term used to find the message is the message id itself. (Try
> running "M-x visible-mode" from a notmuch-search view in emacs to see
> what those look like.)

Exactly, that's what I meant by manually.  Those messages don't match a
nice generic pattern.

> Meanwhile, just archiving the message won't make things perfect for
> you. The document in the database point to the broken file is still
> there. And it should still have all of its terms, so will likely show up
> if you do more searches. (The "(null)" stuff you're seeing isn't because
> the message is NULL---for example, notmuch was able to find the date,
> etc. It's just that notmuch couldn't find the subject and authors when
> it went to look for the file.)

Yeah.

> So if GMail+offlineimap continues to shuffle your files around, you're
> going to keep seeing more and more confusion like this buildup.
> 
> So we really just need to teach notmuch how to handle an unstable file
> store in order to be able to use it in this kind of setup.

This seems unavoidable with maildir in the presence of any
synchronization, or use of a different client.

An ugly, but possible solution would be to mirror the entire maildir via
hard links with whatever naming scheme you like.  You then have a stable
link to the file and can resolve changing names in the real maildir.
This eats up a lot of inodes.

Jed


[notmuch] Asynchronous tagging

2009-11-21 Thread Jan Janak
On Sat, Nov 21, 2009 at 10:04 PM, Jed Brown  wrote:
>> ?3) Message flags are updated on the IMAP server (for example when you
>> mark a message as read in gmail). Offlineimap keeps message flags
>> synchronized. ?If you mark a local message as read then the change is
>> propagated to the IMAP server and vice versa.
>
> Do you know if Offlineimap (or some similar tool) can be told not to
> bother keeping flags synchronized?

Try using the cmdline option -q, from offlineimap's help:

-q  Run  only quick synchronizations.   Ignore any flag updates on IMAP servers.

This kinda works, but even with this option I am still seeing missing
files if I work with my inbox in gmail. AFAIK there is currently no
easy way to prevent that.

  -- Jan


[notmuch] [PATCH 2/2] notmuch list: A new command to produce various lists.

2009-11-21 Thread Jan Janak
On Thu, Nov 19, 2009 at 3:41 PM, Carl Worth  wrote:
> On Thu, 19 Nov 2009 12:52:49 +0100, Jan Janak  wrote:
>> I implemented a new notmuch command that can be used to list all tags
>> present in the database:
>
> Ah, very very interesting! I'd been planning on doing something like
> this soon. And I was just thinking of "notmuch tags" as the command
> line. What other ideas do you have in mind for "notmuch list" ?
>
> One way we could possibly do "notmuch tags" is to have it accept search
> terms and then return the list of all tags from the matched messages.
> Then of course we'd need a syntax for a search term to match all
> messages, (which we want in any case).
>
>> The main reason why I implemented this is because I am also working on
>> adding the tag completion feature to the Emacs mode. This is very
>> useful if you have a large collection of tags--it can save you some
>> typing and, perhaps more importantly, it minimizes the risk of having
>> typos in tag names. I'll send a patch for that later too.
>
> That will be a very nice feature to have, yes.
>
> The other reason I've wanted this is have something like a "folder view"
> that would show a list of tags and a number of messages with each tag,
> (or a number of messages with that tag and the inbox tag).
>
> I know that Keith said he'd prefer to use a view like that as his
> primary way of reading mail.
>
> Actual review of the patch later.

Carl and others,

My patch no longer works and I have been thinking about updating it to
current HEAD. But before I do that, I wanted to check with you to see
if you would prefer to use a different name for the command, here are
some options:

  1) notmuch tags
  2) notmuch list tags
  3) notmuch list-tag

Any opinions?

I also plan to add support for search-terms later so that we can
produce tag lists for a set of messages, as you mentioned in one of
your previous emails.

A quick description for those who joined later: This command produces
a list of all tags defined in the database. The emacs interface uses
it to implement tag name completion.

  -- Jan


[notmuch] Asynchronous tagging

2009-11-21 Thread Jed Brown
On Sat, 21 Nov 2009 21:50:10 +0100, Jan Janak  wrote:
> I get errors about missing files too. There are several reasons why
> that can happen:
> 
>  1) A message is moved from one folder to another in other mail
> clients that work with the Maildir spool.

Not a problem in my case because I currently have everything in one big
maildir (100k in one directory is a lot, but not too painful at 0.3s for
ls and 2s to stat everything).

>  2) A client changes the flags on a message, for example, when you
> read a message or mark it as deleted. Maildir stores flags in
> filenames.

This seems like a problem.  I'm not familiar with xapian, is it
necessarily an expensive operation to correct these inconsistencies?
Matching by thread id ought to be cheap.

>  3) Message flags are updated on the IMAP server (for example when you
> mark a message as read in gmail). Offlineimap keeps message flags
> synchronized.  If you mark a local message as read then the change is
> propagated to the IMAP server and vice versa.

Do you know if Offlineimap (or some similar tool) can be told not to
bother keeping flags synchronized?

Jed


[notmuch] Asynchronous tagging

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 20:45:30 +0100, Jed Brown  wrote:
> Actually, this popped up again.  I have a workaround, but here's the
> story if you are interested.

Hmmm... we definitely want to fix this, so let's figure this out.

> After changing a flag in Gmail and syncing with offlineimap, I get this
> in my inbox
> 
>  Today 19:18 [1/2] (null)   (null) (inbox 
> unread)
> 
> And when I try to open it, the buffer is full of stderr.
> 
>   Error opening 
> /home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
>  No such file or directory

Ah, OK. So you made a change on the Gmail side and that caused a file to
be renamed locally.

And yes, this currently makes notmuch very confused. That's a known
issue that needs to be documented better. And even better needs to be
fixed, (I just added a note for this to TODO).

> Explicitly archiving the null message removes it from these queries so
> the clutter is gone now, but it has to be done manually because the null
> message doesn't match any search terms.

Manually? All tag manipulation is done by search terms, so there's no
other way to remove a tag.

Or did you mean you removed the tag from within emacs? In that case, the
search term used to find the message is the message id itself. (Try
running "M-x visible-mode" from a notmuch-search view in emacs to see
what those look like.)

Meanwhile, just archiving the message won't make things perfect for
you. The document in the database point to the broken file is still
there. And it should still have all of its terms, so will likely show up
if you do more searches. (The "(null)" stuff you're seeing isn't because
the message is NULL---for example, notmuch was able to find the date,
etc. It's just that notmuch couldn't find the subject and authors when
it went to look for the file.)

So if GMail+offlineimap continues to shuffle your files around, you're
going to keep seeing more and more confusion like this buildup.

So we really just need to teach notmuch how to handle an unstable file
store in order to be able to use it in this kind of setup.

-Carl


[notmuch] Asynchronous tagging

2009-11-21 Thread Jan Janak
On Sat, Nov 21, 2009 at 9:01 PM, Carl Worth  wrote:
>> 3. I had initially put 'notmuch new' in a cron job (instead of
>> offlineimap postsync hook) and new/search would sometimes complain about
>> missing files in the maildir. ?The first time this happened, it did not
>> correct itself and I ended up reimporting the database (I had moved some
>> things around so I could have been at fault). ?Since then I have seen
>> these errors on a couple occasions, but they always go away upon
>> rerunning 'notmuch new'. ?My guess is that it has to do with offlineimap
>> changing flags so I moved 'notmuch new' to the postsync hook and have
>> not seen the errors since. ?If it is important that notmuch never runs
>> concurrently with an offlineimap sync, it should eventually go in the
>> docs.
>
> Thanks for the pointer.
>
> Does offlineimap use tmp while it's delivering message and then move
> things to new? If so, then maybe all we need to do to fix notmuch to not
> look into tmp directories?

Yes, it does. I think all delivery agents work this way, IIRC the
reason why messages are first written in tmp and then moved to new is
to make sure that clients do not see partially written messages.
Maildir has been designed to be lock-less so this is needed.

I get errors about missing files too. There are several reasons why
that can happen:

 1) A message is moved from one folder to another in other mail
clients that work with
the Maildir spool.

 2) A client changes the flags on a message, for example, when you
read a message or
mark it as deleted. Maildir stores flags in filenames.

 3) Message flags are updated on the IMAP server (for example when you
mark a message
as read in gmail). Offlineimap keeps message flags synchronized.
If you mark a
local message as read then the change is propagated to the IMAP
server and vice
versa.

 -- Jan


[notmuch] Asynchronous tagging

2009-11-21 Thread Jed Brown
On Sat, 21 Nov 2009 21:01:20 +0100, Carl Worth  wrote:
> Yes, this is a known bug in Xapian (it rewrites all of the indexed terms
> for the email message even though you're just trying to add/remove one
> term). The Xapian ticket for this is here:
> 
>   replace_document should make minimal changes to database file
>   http://trac.xapian.org/ticket/250

This bug report is concerned that it could require an API change, it
sounds like you think this is unnecessary.  Thanks for the detailed
explanation.

> Chris Wilson just contributed a patch to enable read-only usage of
> notmuch while another notmuch process holds the write lock.

I'm running it.

> Does offlineimap use tmp while it's delivering message and then move
> things to new? If so, then maybe all we need to do to fix notmuch to not
> look into tmp directories?

Yes, that's how maildir is supposed to work.  Deliver to tmp, hard link
from new, then unlink in tmp.  The client should never look in tmp.
Should be very quick to fix in notmuch.

Jed


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 20:36:06 +0100, Stefan Schmidt  wrote:
> Yup, I had the repo on my disk a week before Keith blogged about it. Just nice
> that it was going crazy that fast and people start using it and contributing 
> to
> it.

Yes, it's quite fun.

> > Though, frankly, I think we need to fix "notmuch new" to do much better
> > than 40 files/sec.
> 
> As a sidenote. That one is on a notebook with a slow 5400 disk and crypt + 
> lvm +
> ext3 on top. Perhaps I should put some money back for an X25 SSD. ;)

Sure. But I think we can still do a lot better even on your machine. :-)

> I have to admit it took me some time. Something like below should help?

Thanks so much! I committed this, (and then added a bit more
documentation on top of it).

> I think that's what I will try to get working here. Sounds the nearest 
> solution
> to my problem. That in combination with the just merged tags-based-on-folders
> patch should make me a lot happier. :)

Well, do note that I just reverted that patch too. :-/

So you might want to cherry-pick it back (or even add the configuration
option that will let us push it back out again).

-Carl


[notmuch] Asynchronous tagging

2009-11-21 Thread Karl Wiberg
On Sat, Nov 21, 2009 at 9:01 PM, Carl Worth  wrote:

> Does offlineimap use tmp while it's delivering message and then move
> things to new? If so, then maybe all we need to do to fix notmuch to not
> look into tmp directories?

That's probably the right thing to do regardless---IIRC, the tmp
directory exists so that processes can put messages there while they
are writing them, and then do an atomic rename to the new (or cur)
directory.

-- 
Karl Wiberg, kha at treskal.com
   subrabbit.wordpress.com
   www.treskal.com/kalle


[notmuch] [PATCH] notmuch: Add Maildir directory name as tag name for messages

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 19:39:22 +0100, Carl Worth  wrote:
> On Wed, 18 Nov 2009 21:25:28 +0530, aneesh.kumar at linux.vnet.ibm.com 
> (Aneesh Kumar K.V) wrote:
> A fantastic patch, Aneesh! I think this will be handy for a lot of
> people importing mail from all kinds of systems. This is pushed now.

And sadly, I just pulled it out again.

I realized that I actually don't want my mail tagged based on the
maildir directories I'm using, (they are arbitrarily-named directories
used only to keep the per-directory number of files below about 10
thousand).

So we'll probably need to make this an opt-in feature from the
configuration file.

-Carl


[notmuch] notmuch vim support

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 14:14:02 -0500, Bart Trojanowski  wrote:
> Perfect.  How would you like to proceed with the initial integration?
> Pulling my tree, or me squashing all the changes into a single patch?
> Also, are you OK with where the files live (ie under a vim directory).

I'm glad to pull (it will be nice for you to have the accurate
history). And a vim directory sounds perfect.

Or should we do interfaces/emacs and interfaces/vim ? I don't know.

> After that, do you want me to submit patches by email, via pull
> requests, or giving me push access to a branch at your repo?

Email or pull requests are fine. You're clearly the "owner" of the vim
stuff, so don't expect any refusals to pull from me. (And feel free to
ping me out of band if you don't want to clutter the list with pull
requests---but it's more likely that you'll want to let people on the
list know about changes anyway.)

> BTW, have you considered using github or gitorious so branch management
> becomes a distributed problem?

I don't see any advantage to have the main notmuch repository
there. We've already got a server with a nice name. The nice thing about
git, of course, is that branches and repositories can be distributed
arbitrarily and there's no problem.

-Carl


[notmuch] hijacked threads can be confusing in notmuch

2009-11-21 Thread Bart Trojanowski
I'd like to see how other people feel about a problem I just ran into.

I wanted to look at the 'notmuch count' patch that Keith Packard posted
a couple of days ago (as I am porting his folder mode to notmuch.vim).

I was unable to find it using notmuch because he happened to post 3
patches in one go.  It's pretty common git practice, and by no means to
I hold Keith responsible :)

In mutt it looked like this:

  [notmuch] [PATCH 1/3] Make mouse-1 click in search view show thread
  ??>[notmuch] [PATCH 2/3] Add 'notmuch count' command to show the count of 
matching messages
??>[notmuch] [PATCH 3/3] Add notmuch-index mode to display message 
counts

... and in the mutt threaded display the relationship between the three
messages is pretty clear.

Now consider what happens when I run notmuch search 'notmuch count'.  I
get this:

  Today 02:15 [2/3] Keith Packard; [notmuch] [PATCH 1/3] Make mouse-1 click in 
search view show thread (inbox unread)

This thread happens to look completely unrelated to my search at first
glance.  So naturally, I dismissed it.  I finally clued in what was
happening and came back to it after.

On a related note, one mail related pet peeve I have is when people
reply to a random email in their mailbox when they actually intend to
start a new thread.  Doing that would totally mess up someone using
notmuch.  They could get search results with threads which have no
relevance to their actual search... at least at first glance.

So is there something better that we could do when detecting hijacked
threads like this?  Is it safe to cut threads when you notice a topic
change?  Or maybe it would be better to just mark such threads in the
output of notmuch-search (either a boolean flag, or a count of topic
changes).

Anyone know how Gmail deals with this?

Cheers,
-Bart

-- 
WebSig: http://www.jukie.net/~bart/sig/


[notmuch] notmuch vim support

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 13:39:09 -0500, Bart Trojanowski  wrote:
> I've spent a few days on porting notmuch.el to vim.  It's not as feature
> rich as notmuch.el yet, but it's coming along nicely.  The results of my
> work are here:

Oh, hey! That's fantastic. I didn't expect to see a second notmuch
interface so soon.

> There are some screenshots in my blog:
> 
> http://www.jukie.net/~bart/blog/notmuch.vim

It looks nice. I'm watching Chris Wilson play with it now, (in spite of
the fact that Chris sent patches already---he wasn't actually using
notmuch until he saw this vim interface appear).

I'm actually surprised as to how faithfully it replicates the notmuch.el
experience. And some of that includes misfeatures that I just hadn't
figured out how to do better, (like the global toggling of signature
visibility).

So, well done!

I think the only thing we noticed is that "filter" doesn't add the "and"
between the old and new search terms. But Chris may have more to comment
soon.

> I'd love for this to be included in the official project, but I am not
> sure if you'd prefer to merge at this point or wait will I am done.
> Naturally, if anyone is interested I'd love some help.

Let's definitely merge it into the repository. As things are changing
quickly now, it will be nice to be able to update both interfaces in
concert. (Like with my patch that changes --max-threads to --max, etc.)

I don't know how realistic it will be to keep both interfaces totally
synchronized in terms of look-and-feel, colors, keybindings, etc. But it
might be nice to do that as much as possible. I think it would be
helpful if we could have interface discussions as an entire community
and not split based on choice of editor.

-Carl


[notmuch] Asynchronous tagging

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 19:35:39 +0100, Jed Brown  wrote:
> I'm really enjoying notmuch, thanks!  I have a minor issue and a couple
> observations worth noting.

Thanks, Jed! And welcome to notmuch.

> 1. Changing tags (like removing inbox/unread) has really high latency.

Yes, this is a known bug in Xapian (it rewrites all of the indexed terms
for the email message even though you're just trying to add/remove one
term). The Xapian ticket for this is here:

replace_document should make minimal changes to database file
http://trac.xapian.org/ticket/250

I've looked at the code, and it looks like it's going to be easy to
fix. If anyone wants to try, here's the file to change:

xapian-core/backends/flint/flint_database.cc

And look for:

// FIXME - in the case where there is overlap between the new
// termlist and the old termlist, it would be better to compare the
// two lists, and make the minimum set of modifications required.
// This would lead to smaller changesets for replication, and
// probably be faster overall

So I think this might be as easy as just walking over two sorted lists
looking for differences.

Note that this is in the currently default "flint" backend, but the
Xapian folks are probably more interested in fixing the in-development
"chert" backend. So the patch to get upstreamed there will probably also
fix:

xapian-core/backends/chert/chert_database.cc

(I'm hoping the fix will be the same---an identical comment exists
there.)

Also, if you want to experiment with the chert backend, compile current
Xapian source and run notmuch with XAPIAN_PREFER_CHERT=1. I haven't
tried that yet, but there are claims that a chert database can be 40%
smaller than an equivalent flint database.

> 2. I have 'notmuch new' in an offlineimap postsync hook, but
> notmuch-search-refresh-view occasionally complains that another process
> has the lock (since I might press '=' when 'notmuch new' is running).
> Waiting a moment and trying again works fine, but it would be nice to
> clean this up eventually.

Chris Wilson just contributed a patch to enable read-only usage of
notmuch while another notmuch process holds the write lock. This should
be very nice, (and means that new users will be able to start playing
with notmuch even while the initial index creation is happening).

> 3. I had initially put 'notmuch new' in a cron job (instead of
> offlineimap postsync hook) and new/search would sometimes complain about
> missing files in the maildir.  The first time this happened, it did not
> correct itself and I ended up reimporting the database (I had moved some
> things around so I could have been at fault).  Since then I have seen
> these errors on a couple occasions, but they always go away upon
> rerunning 'notmuch new'.  My guess is that it has to do with offlineimap
> changing flags so I moved 'notmuch new' to the postsync hook and have
> not seen the errors since.  If it is important that notmuch never runs
> concurrently with an offlineimap sync, it should eventually go in the
> docs.

Thanks for the pointer.

Does offlineimap use tmp while it's delivering message and then move
things to new? If so, then maybe all we need to do to fix notmuch to not
look into tmp directories?

-Carl






[notmuch] Greetings and suggestions

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 13:08:42 -0500, Brett Viren  
wrote:
> I recently heard about notmuch from a reddit.com posting.  Immediately
> I knew I needed to try it.

Hi Brett, welcome to Notmuch!

> I've been a long time GNUS user and with VM before that.  GNUS is
> great but I have never been very successful getting it to play well
> with an indexer.

Excellent. It's great to have a long-time GNUS user here. I really
wanted to poke around in GNUS for ideas, (and code), but have so far
entirely failed to even _run_ in to view any email messages. I'm clearly
not smart enough to be a GNUS user.

But I know that if I *could* run it, I'd see lots of great stuff in
it. And integrating nicely would be great. (The emacs code we have for
notmuch is already integrating with the message-mode stuff from GNUS).

> So far, after a little playing with notmuch I think it looks very
> good!

Thanks so much.

>  - posting styles (ie, set From: to non standard value) based on
> search pattern.  This is really important when one subscribes to
> different mailing lists using different addresses and the ML only
> allows mail from registered addresses.

Yes. That's a good idea. We already set From: based on how a message
being replied-to was addressed, but not yet based on the new To:
address.

>  - apply tags based on search pattern as new mail is processed.  This
> would be like GNUS split-rules, but even better as one can place more
> than one tag.

This is definitely on the feature list and is something we'll be
implementing soon. In the meantime, those of us that are using notmuch
in earnest are using shell scripts to load our mail like this:

notmuch new
notmuch tag +notmuch to:notmuchmail.org and not tag:notmuch
...

But just moving that configuration to the configuration file will be a
lot better.

> Anyways, great project and I look forward to contributing where I can.

Thanks Brett,

I'll look forward to any contributions from you.

-Carl


[notmuch] Asynchronous tagging

2009-11-21 Thread Jed Brown
On Sat, 21 Nov 2009 19:35:39 +0100, Jed Brown  wrote:

[...]

> 3. I had initially put 'notmuch new' in a cron job (instead of
> offlineimap postsync hook) and new/search would sometimes complain about
> missing files in the maildir.  The first time this happened, it did not
> correct itself and I ended up reimporting the database (I had moved some
> things around so I could have been at fault).  Since then I have seen
> these errors on a couple occasions, but they always go away upon
> rerunning 'notmuch new'.  My guess is that it has to do with offlineimap
> changing flags so I moved 'notmuch new' to the postsync hook and have
> not seen the errors since.  If it is important that notmuch never runs
> concurrently with an offlineimap sync, it should eventually go in the
> docs.

Actually, this popped up again.  I have a workaround, but here's the story if 
you are interested.

After changing a flag in Gmail and syncing with offlineimap, I get this
in my inbox

 Today 19:18 [1/2] (null)   (null) (inbox 
unread)

And when I try to open it, the buffer is full of stderr.

  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory

It is present in any searches that contain the problem files

  $ notmuch search tag:inbox | wc
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258826583_1.20705.kunyang,U=174235,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d088d057aa1b32ac1fbc67b56:2,:
 No such file or directory
  Error opening 
/home/jed/.mail-archive/gmail-all/new/1258827595_0.20705.kunyang,U=174288,FMD5=844bb96d

[notmuch] notmuch vim support

2009-11-21 Thread Stefan Schmidt
Hello.

[Resend, first one missed the list]

On Sat, 2009-11-21 at 14:14, Bart Trojanowski wrote:
> * Stefan Schmidt  [091121 14:06]:
> > IMHO it would be best to do the development inside the main tree. You could 
> > use
> > branches there too and it would be easier to find and test. And as notmuch 
> > is
> > still under heavy development its not a problem that the vim interface is as
> > well.
> 
> Perfect.  How would you like to proceed with the initial integration?
> Pulling my tree, or me squashing all the changes into a single patch?
> Also, are you OK with where the files live (ie under a vim directory).

Just started to use it. Its Carls trun to decide such things. :)

regards
Stefan Schmidt


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Stefan Schmidt
Hello.

On Sat, 2009-11-21 at 18:07, Carl Worth wrote:
> On Sat, 21 Nov 2009 15:51:11 +0100, Stefan Schmidt  datenfreihafen.org> wrote:
> > Disclaimer: I'm using vim, in combination with mutt for email, for years, 
> > but
> > never dealt with emacs. Please have this in mind and spot any emacs user 
> > errors
> > in this report. :)
> 
> Hi Stefan, welcome to Notmuch! And don't worry, we don't discriminate
> (too much) against non-emacs users around here.

:)

> > I have first seen notmuch several weeks ago as it seems a silent project. 
> > Being
> > more then happy now that it envolves quickly and a real developer community
> > builds around it.
> 
> Yes. Notmuch was a silent project since it was just something that I was
> doing for myself. I was always writing it as free software, and even had
> a public git repository available, but hadn't advertised it at all yet.

Yup, I had the repo on my disk a week before Keith blogged about it. Just nice
that it was going crazy that fast and people start using it and contributing to
it.

> > But now to my problem. Getting m mail indexed was easy enough:
> > 
> > stefan at excalibur:~$ du -chs not-much-mail/
> > 1.5Gnot-much-mail/
> > 1.5Gtotal
> > stefan at excalibur:~$ time notmuch new
> > Found 103677 total files.
> > Processed 103677 total files in 42m 30s (40 files/sec.).
> > Added 100899 new messages to the database (not much, really).
> 
> Good. I'm glad that went fairly smoothly for you.
> 
> Though, frankly, I think we need to fix "notmuch new" to do much better
> than 40 files/sec.

As a sidenote. That one is on a notebook with a slow 5400 disk and crypt + lvm +
ext3 on top. Perhaps I should put some money back for an X25 SSD. ;)

> > I put (require  'notmuch) in my ~/.emacs ans start emacs with the -f notmuch
> > option to enter the notmuch mode.
> 
> I'm glad you've figured that much out. I feel bad that that's not even
> in the documentation anywhere yet.

I have to admit it took me some time. Something like below should help?

> > What happends then is that a notmuch process gets started and emacs
> > waits for the return.
> 
> OK. This is a known shortcoming. As Bdale supposes, this problem is from
> notmuch trying to load and construct every thread in your
> database. There are actually several different bugs/missing features
> here that should be addressed:
> 
>   * "notmuch new" should look at the R flag in maildir files to
> determine that they are read and do not need to be marked as "inbox"
> and "unread"

I think that's what I will try to get working here. Sounds the nearest solution
to my problem. That in combination with the just merged tags-based-on-folders
patch should make me a lot happier. :)


>From 8f95e039e98addd0f4be7c31e41e534f1b519a5d Mon Sep 17 00:00:00 2001
From: Stefan Schmidt 
Date: Sat, 21 Nov 2009 20:31:55 +0100
Subject: [PATCH] INSTALL: emacs install dokumentation.

Write down the steps needed to install and actuall use notmuch in emacs. Should
help emacs newbies.

Signed-off-by: Stefan Schmidt 
---
 INSTALL |8 
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/INSTALL b/INSTALL
index de268b6..64b8e36 100644
--- a/INSTALL
+++ b/INSTALL
@@ -14,6 +14,14 @@ Notmuch are satisfied. If they are not, the configure script 
will
 notice that and provide instructions on where to obtain the necessary
 dependencies.

+notmuch.el installation
+---
+Installing the notmuch.el emacs lisp function systemwide:
+
+   sudo make install-emacs
+
+Each user needs to add (require 'notmuch) in his ~/.emacs to activate it.
+
 Dependencies
 
 Notmuch depends on three libraries: Xapian, GMime 2.4, and Talloc
-- 
1.6.5.3

regards
Stefan Schmidt


[notmuch] [PATCH] Makefile: Magic silent rules.

2009-11-21 Thread Chris Wilson
Use the facilities of GNU make to create a magic function that will
on the first invocation print a description of how to enable verbose
compile lines and then print the quiet rule.

Signed-off-by: Chris Wilson 
Signed-off-by: Carl Worth 
Cc: Mikhail Gusarov 

[ickle: Rebased, and duplicate command string eliminated.]
[ickle: Fixed verbose bug pointed out by Mikhail]
---
 Makefile   |   23 ---
 Makefile.local |4 ++--
 lib/Makefile.local |2 +-
 3 files changed, 23 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 3fedcf1..5d8f321 100644
--- a/Makefile
+++ b/Makefile
@@ -2,6 +2,10 @@
 WARN_FLAGS=-Wall -Wextra -Wmissing-declarations -Wwrite-strings -Wswitch-enum
 CFLAGS=-O2

+# Additional programs that are used during the compilation process.
+EMACS ?= emacs
+GZIP ?= gzip
+
 # Additional flags that we will append to whatever the user set.
 # These aren't intended for the user to manipulate.
 extra_cflags := $(shell pkg-config --cflags glib-2.0 gmime-2.4 talloc)
@@ -31,14 +35,27 @@ include lib/Makefile.local
 # And get user settings from the output of configure
 include Makefile.config

+# The user has not set any verbosity, default to quiet mode and inform the
+# user how to enable verbose compiles.
+ifeq ($(V),)
+quiet_DOC := "Use \"$(MAKE) V=1\" to see the verbose compile lines.\n"
+quiet = @echo $(quiet_DOC)$(eval quiet_DOC:=)"  $1 $@"; $($1)
+endif
+# The user has explicitly enabled quiet compilation.
+ifeq ($(V),0)
+quiet = @echo "  $1$@"; $($1)
+endif
+# Otherwise, print the full command line.
+quiet ?= $($1)
+
 %.o: %.cc $(all_deps)
-   $(CXX) -c $(CXXFLAGS) $< -o $@
+   $(call quiet,CXX) -c $(CXXFLAGS) $< -o $@

 %.o: %.c $(all_deps)
-   $(CC) -c $(CFLAGS) $< -o $@
+   $(call quiet,CC) -c $(CFLAGS) $< -o $@

 %.elc: %.el
-   emacs -batch -f batch-byte-compile $<
+   $(call quiet,EMACS) -batch -f batch-byte-compile $<

 .deps/%.d: %.c $(all_deps)
@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
diff --git a/Makefile.local b/Makefile.local
index 3c99624..5fd5d4f 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -20,10 +20,10 @@ notmuch_client_srcs =   \

 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
 notmuch: $(notmuch_client_modules) lib/notmuch.a
-   $(CXX) $^ $(LDFLAGS) -o $@
+   $(call quiet,CXX) $^ $(LDFLAGS) -o $@

 notmuch.1.gz: notmuch.1
-   gzip --stdout notmuch.1 > notmuch.1.gz
+   $(call quiet,GZIP) --stdout $^ > $@

 install: all notmuch.1.gz
for d in $(DESTDIR)$(prefix)/bin/ $(DESTDIR)$(prefix)/share/man/man1 \
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 79f7b0b..a7562c9 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -18,7 +18,7 @@ libnotmuch_cxx_srcs = \

 libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 $(dir)/notmuch.a: $(libnotmuch_modules)
-   $(AR) rcs $@ $^
+   $(call quiet,AR) rcs $@ $^

 SRCS  := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs)
 CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a
-- 
1.6.5.3



[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Stefan Schmidt
Hello.

On Sat, 2009-11-21 at 18:26, Carl Worth wrote:
> On Sat, 21 Nov 2009 16:36:55 +0100, Stefan Schmidt  datenfreihafen.org> wrote:
> 
> > In my case only 80 messages were printed before the gap. All of them had a 
> > wrong
> > year in the timestamp. 1900 and 1970. Maybe notmuch just comes into a bad 
> > state
> > with this dates?
> 
> I don't think the bogus dates are throwing anything off. It's more
> likely that you just have a number of messages with no Date header on
> them at all. And for such messages, notmuch just chooses a time_t value
> of 0 so you'll see whatever that 0 maps to on your system---a date of
> 1970 there is not surprising. :-)

Yeah, I figured that removing the offending messages and re-run it brought
nothing. Time to look at the source. :)

regards
Stefan Schmidt


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Stefan Schmidt
Hello.

On Sat, 2009-11-21 at 18:16, Carl Worth wrote:
> On Sat, 21 Nov 2009 08:12:52 -0700, Bdale Garbee  wrote:
> > I haven't figured out how to quickly tag everything as already read or
> > archived or whatever .. can someone who knows more about what's going on
> > confirm my hypothesis and if so, suggest the best approach to getting to
> > a happier state?
> 
> See my message up-thread. The only reasonable ways all really do involve
> at least a little bit of C-code hacking to either prevent those tags
> from getting put there by "notmuch new" or to make it easier to get them
> off afterwards.

Let's see if I come up with something here.

> And I can't help but apologize. I've known about all these issues, and
> wouldn't have invited people to try things out in the current state. But
> it was nice of Keith to share this with everyone. And it's nice of all
> you to come take a look at things.

Getting it out now was a good move. It had enough code to actually do omething
usefull and many people waited for something like this. The increasing number of
contributors in such a short time shows it very well. :)

regards
Stefan Schmidt


[notmuch] [PATCH] Mac OS X compatibility fixes

2009-11-21 Thread Alexander Botero-Lowry
On Sun, 22 Nov 2009 11:07:36 +0800, Jjgod Jiang  wrote:
> Add missing GNU extensions strdup() and getline(). The C library
> shipped with Mac OS X does not include them (though it does support
> some GNU extensions when _GNU_SOURCE is defined), so we have to
> add these two. The getline() implementation is a modified version
> of getdelim() from GNU Libc.
Awesome!

> diff --git a/lib/xutil.c b/lib/xutil.c
> index 6fa5eb0..805b236 100644
> --- a/lib/xutil.c
> +++ b/lib/xutil.c
> @@ -79,6 +79,105 @@ xstrdup (const char *s)
>  return ret;
>  }
>  
> +#ifdef __APPLE__
Not awesome.

This should be done in a capabilites way, for example strndup was added
to FreeBSD in 7.2 (which is this current release of the 7 line), and so
for older versions of FreeBSD strndup will be needed. getdelim() and
getline() came in FreeBSD 8, so they'll be needed for the entire 7 line.
So Instead of just assuming __APPLE__ this should be done by determing
if the symbols are generally needed.

alex


[notmuch] [PATCH] Permit opening the notmuch database in read-only mode.

2009-11-21 Thread Chris Wilson
We only rarely need to actually open the database for writing, but we
always create a Xapian::WritableDatabase. This has the effect of
preventing searches and like whilst updating the index.

Signed-off-by: Chris Wilson 
Acked-by: Carl Worth 
[ickle: Updated to add WRITABLE mode to notmuch-new]
---
 lib/database-private.h |3 ++-
 lib/database.cc|   31 ---
 lib/message.cc |   15 +--
 lib/notmuch-private.h  |1 +
 lib/notmuch.h  |   13 +++--
 notmuch-dump.c |3 ++-
 notmuch-new.c  |4 +++-
 notmuch-reply.c|3 ++-
 notmuch-restore.c  |3 ++-
 notmuch-search.c   |3 ++-
 notmuch-show.c |3 ++-
 notmuch-tag.c  |3 ++-
 12 files changed, 66 insertions(+), 19 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 76e26ce..79c7916 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -27,7 +27,8 @@

 struct _notmuch_database {
 char *path;
-Xapian::WritableDatabase *xapian_db;
+notmuch_database_mode_t mode;
+Xapian::Database *xapian_db;
 Xapian::QueryParser *query_parser;
 Xapian::TermGenerator *term_gen;
 };
diff --git a/lib/database.cc b/lib/database.cc
index 207246c..fb38664 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -172,6 +172,8 @@ notmuch_status_to_string (notmuch_status_t status)
return "No error occurred";
 case NOTMUCH_STATUS_OUT_OF_MEMORY:
return "Out of memory";
+case NOTMUCH_STATUS_READONLY_DATABASE:
+   return "The database is read-only";
 case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
return "A Xapian exception occurred";
 case NOTMUCH_STATUS_FILE_ERROR:
@@ -438,7 +440,8 @@ notmuch_database_create (const char *path)
goto DONE;
 }

-notmuch = notmuch_database_open (path);
+notmuch = notmuch_database_open (path,
+NOTMUCH_DATABASE_MODE_WRITABLE);

   DONE:
 if (notmuch_path)
@@ -448,7 +451,8 @@ notmuch_database_create (const char *path)
 }

 notmuch_database_t *
-notmuch_database_open (const char *path)
+notmuch_database_open (const char *path,
+  notmuch_database_mode_t mode)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL, *xapian_path = NULL;
@@ -481,9 +485,14 @@ notmuch_database_open (const char *path)
 if (notmuch->path[strlen (notmuch->path) - 1] == '/')
notmuch->path[strlen (notmuch->path) - 1] = '\0';

+notmuch->mode = mode;
 try {
-   notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path,
-  
Xapian::DB_CREATE_OR_OPEN);
+   if (mode == NOTMUCH_DATABASE_MODE_WRITABLE) {
+   notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path,
+  
Xapian::DB_CREATE_OR_OPEN);
+   } else {
+   notmuch->xapian_db = new Xapian::Database (xapian_path);
+   }
notmuch->query_parser = new Xapian::QueryParser;
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
@@ -521,7 +530,8 @@ notmuch_database_open (const char *path)
 void
 notmuch_database_close (notmuch_database_t *notmuch)
 {
-notmuch->xapian_db->flush ();
+if (notmuch->mode == NOTMUCH_DATABASE_MODE_WRITABLE)
+   (static_cast  (notmuch->xapian_db))->flush 
();

 delete notmuch->term_gen;
 delete notmuch->query_parser;
@@ -567,11 +577,18 @@ notmuch_database_set_timestamp (notmuch_database_t 
*notmuch,
const char *key, time_t timestamp)
 {
 Xapian::Document doc;
+Xapian::WritableDatabase *db;
 unsigned int doc_id;
 notmuch_private_status_t status;
 notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
 char *db_key = NULL;

+if (notmuch->mode == NOTMUCH_DATABASE_MODE_READONLY) {
+   fprintf (stderr, "Attempted to update a read-only database.\n");
+   return NOTMUCH_STATUS_READONLY_DATABASE;
+}
+
+db = static_cast  (notmuch->xapian_db);
 db_key = timestamp_db_key (key);

 try {
@@ -586,9 +603,9 @@ notmuch_database_set_timestamp (notmuch_database_t *notmuch,
doc.add_term (term);
talloc_free (term);

-   notmuch->xapian_db->add_document (doc);
+   db->add_document (doc);
} else {
-   notmuch->xapian_db->replace_document (doc_id, doc);
+   db->replace_document (doc_id, doc);
}

 } catch (Xapian::Error &error) {
diff --git a/lib/message.cc b/lib/message.cc
index e0b8a8e..7ba06c9 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -168,9 +168,15 @@ _notmuch_message_create_for_message_id (notmuch_database_t 
*notmuch,
 {
 notmuch_message_t *message;
 Xapian::Document doc;
+Xapian::WritableDatabase *db;
 unsigned int doc_id;
 char *term;

+if (notmuch->mode == NO

[notmuch] notmuch vim support

2009-11-21 Thread Stefan Schmidt
Hello.

On Sat, 2009-11-21 at 13:39, Bart Trojanowski wrote:
> 
> I've spent a few days on porting notmuch.el to vim.  It's not as feature
> rich as notmuch.el yet, but it's coming along nicely.  The results of my
> work are here:

Great stuff! I hoped for someone stepping up early to provide alternative
interfaces.

> I'd love for this to be included in the official project, but I am not
> sure if you'd prefer to merge at this point or wait will I am done.

IMHO it would be best to do the development inside the main tree. You could use
branches there too and it would be easier to find and test. And as notmuch is
still under heavy development its not a problem that the vim interface is as
well.

regards
Stefan Schmidt


[notmuch] [PATCH] Permit opening the notmuch database in read-only mode.

2009-11-21 Thread Chris Wilson
We only rarely need to actually open the database for writing, but we
always create a Xapian::WritableDatabase. This has the effect of
preventing searches and like whilst updating the index.

Signed-off-by: Chris Wilson 
Acked-by: Carl Worth 
---
 lib/database-private.h |3 ++-
 lib/database.cc|   31 ---
 lib/message.cc |   15 +--
 lib/notmuch-private.h  |1 +
 lib/notmuch.h  |   13 +++--
 notmuch-dump.c |3 ++-
 notmuch-new.c  |4 +++-
 notmuch-reply.c|3 ++-
 notmuch-restore.c  |3 ++-
 notmuch-search.c   |3 ++-
 notmuch-show.c |3 ++-
 notmuch-tag.c  |3 ++-
 12 files changed, 66 insertions(+), 19 deletions(-)

diff --git a/lib/database-private.h b/lib/database-private.h
index 76e26ce..79c7916 100644
--- a/lib/database-private.h
+++ b/lib/database-private.h
@@ -27,7 +27,8 @@

 struct _notmuch_database {
 char *path;
-Xapian::WritableDatabase *xapian_db;
+notmuch_database_mode_t mode;
+Xapian::Database *xapian_db;
 Xapian::QueryParser *query_parser;
 Xapian::TermGenerator *term_gen;
 };
diff --git a/lib/database.cc b/lib/database.cc
index 207246c..fb38664 100644
--- a/lib/database.cc
+++ b/lib/database.cc
@@ -172,6 +172,8 @@ notmuch_status_to_string (notmuch_status_t status)
return "No error occurred";
 case NOTMUCH_STATUS_OUT_OF_MEMORY:
return "Out of memory";
+case NOTMUCH_STATUS_READONLY_DATABASE:
+   return "The database is read-only";
 case NOTMUCH_STATUS_XAPIAN_EXCEPTION:
return "A Xapian exception occurred";
 case NOTMUCH_STATUS_FILE_ERROR:
@@ -438,7 +440,8 @@ notmuch_database_create (const char *path)
goto DONE;
 }

-notmuch = notmuch_database_open (path);
+notmuch = notmuch_database_open (path,
+NOTMUCH_DATABASE_MODE_WRITABLE);

   DONE:
 if (notmuch_path)
@@ -448,7 +451,8 @@ notmuch_database_create (const char *path)
 }

 notmuch_database_t *
-notmuch_database_open (const char *path)
+notmuch_database_open (const char *path,
+  notmuch_database_mode_t mode)
 {
 notmuch_database_t *notmuch = NULL;
 char *notmuch_path = NULL, *xapian_path = NULL;
@@ -481,9 +485,14 @@ notmuch_database_open (const char *path)
 if (notmuch->path[strlen (notmuch->path) - 1] == '/')
notmuch->path[strlen (notmuch->path) - 1] = '\0';

+notmuch->mode = mode;
 try {
-   notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path,
-  
Xapian::DB_CREATE_OR_OPEN);
+   if (mode == NOTMUCH_DATABASE_MODE_WRITABLE) {
+   notmuch->xapian_db = new Xapian::WritableDatabase (xapian_path,
+  
Xapian::DB_CREATE_OR_OPEN);
+   } else {
+   notmuch->xapian_db = new Xapian::Database (xapian_path);
+   }
notmuch->query_parser = new Xapian::QueryParser;
notmuch->term_gen = new Xapian::TermGenerator;
notmuch->term_gen->set_stemmer (Xapian::Stem ("english"));
@@ -521,7 +530,8 @@ notmuch_database_open (const char *path)
 void
 notmuch_database_close (notmuch_database_t *notmuch)
 {
-notmuch->xapian_db->flush ();
+if (notmuch->mode == NOTMUCH_DATABASE_MODE_WRITABLE)
+   (static_cast  (notmuch->xapian_db))->flush 
();

 delete notmuch->term_gen;
 delete notmuch->query_parser;
@@ -567,11 +577,18 @@ notmuch_database_set_timestamp (notmuch_database_t 
*notmuch,
const char *key, time_t timestamp)
 {
 Xapian::Document doc;
+Xapian::WritableDatabase *db;
 unsigned int doc_id;
 notmuch_private_status_t status;
 notmuch_status_t ret = NOTMUCH_STATUS_SUCCESS;
 char *db_key = NULL;

+if (notmuch->mode == NOTMUCH_DATABASE_MODE_READONLY) {
+   fprintf (stderr, "Attempted to update a read-only database.\n");
+   return NOTMUCH_STATUS_READONLY_DATABASE;
+}
+
+db = static_cast  (notmuch->xapian_db);
 db_key = timestamp_db_key (key);

 try {
@@ -586,9 +603,9 @@ notmuch_database_set_timestamp (notmuch_database_t *notmuch,
doc.add_term (term);
talloc_free (term);

-   notmuch->xapian_db->add_document (doc);
+   db->add_document (doc);
} else {
-   notmuch->xapian_db->replace_document (doc_id, doc);
+   db->replace_document (doc_id, doc);
}

 } catch (Xapian::Error &error) {
diff --git a/lib/message.cc b/lib/message.cc
index e0b8a8e..7ba06c9 100644
--- a/lib/message.cc
+++ b/lib/message.cc
@@ -168,9 +168,15 @@ _notmuch_message_create_for_message_id (notmuch_database_t 
*notmuch,
 {
 notmuch_message_t *message;
 Xapian::Document doc;
+Xapian::WritableDatabase *db;
 unsigned int doc_id;
 char *term;

+if (notmuch->mode == NOTMUCH_DATABASE_MODE_READONLY) {
+   *status_ret =

[notmuch] Problem Building on OSX leopard 10.5.8

2009-11-21 Thread Carl Worth
On Wed, 18 Nov 2009 12:47:47 -0500, Saptarshi Guha  wrote:
> Having given up on most email clients yet trapped by them, I came  
> across notmuch.
> Quite eager to build,

Hi Saptarshi, Welcome to notmuch!

I hope you find it helpful.

> In the INSTALL/README file, mention that xapian-config should be on path
> and that it be called xapian-config.
> When I installed the latest, it was installed in /usr/local/bin (not  
> in my path)
> at xapian-config-1.1

OK. I've added that now.

> 2. download gmime (no problem here)

Good news.

> 3. Install talloc (the makefile has errors in it)

What's the error here? (Not that it's our bug, but it would be nice to
report it to the talloc folks or perhaps warn notmuch users about it).

> lib/index.cc: In function void  
> _index_address_mailbox(notmuch_message_t*, const char*,  
> InternetAddress*):
> lib/index.cc:46: error: strndup was not declared in this scope

Some systems (including yours) don't have a strndup function. One
proposed plan is to put an implementation of strndup into notmuch, (it's
not a complex function). But then again, we could just change the caller
to use talloc_strndup instead.

Portability patches are always welcome, (otherwise I'll try to do things
when I can but I won't be able to test things well).

-Carl


[notmuch] [PATCH] notmuch new: Support for conversion of spool subdirectories into tags

2009-11-21 Thread Carl Worth
On Wed, 18 Nov 2009 05:57:03 +0100, Jan Janak  wrote:
> This patch makes it possible to convert subdirectory names inside the
> spool directory into message tags. Messages stored in subdirectory
> "foo" will be marked with tag "foo". Message duplicates found in several
> subdirectories will be given one tag per subdirectory.

Hi Jan,

I just pushed out a patch from Aneesh for similar functionality, but
with a simpler implementation.

One thing that patch loses is the addition of multiple tags when a
message appears in multiple directories. Would you like to code up that
feature on top of the current code?

-Carl


[notmuch] [PATCH] notmuch: Add Maildir directory name as tag name for messages

2009-11-21 Thread Carl Worth
On Wed, 18 Nov 2009 21:25:28 +0530, aneesh.kumar at linux.vnet.ibm.com (Aneesh 
Kumar K.V) wrote:
> This patch adds maildir directory name as the tag name for
> messages. This helps in adding tags using filtering already
> provided by procmail.

A fantastic patch, Aneesh! I think this will be handy for a lot of
people importing mail from all kinds of systems. This is pushed now.

Thanks again,

-Carl


[notmuch] Asynchronous tagging

2009-11-21 Thread Jed Brown
I'm really enjoying notmuch, thanks!  I have a minor issue and a couple
observations worth noting.

1. Changing tags (like removing inbox/unread) has really high latency.
For example, notmuch-show-advance-marking-read-and-archiving takes 2 to
4 seconds (by comparison, this is as long as a vague search returning
1000+ threads).  I have about 100k messages in a maildir on
linux-2.6.31-6, ext4, xapian-1.0.17.  I tried switching to the
development version of xapian, but the notmuch configure didn't pick it
up (maybe it would still work though).  Is this a known issue?  Is it
worth making certain notmuch.el operations asynchronous to hide this
latency?

2. I have 'notmuch new' in an offlineimap postsync hook, but
notmuch-search-refresh-view occasionally complains that another process
has the lock (since I might press '=' when 'notmuch new' is running).
Waiting a moment and trying again works fine, but it would be nice to
clean this up eventually.

3. I had initially put 'notmuch new' in a cron job (instead of
offlineimap postsync hook) and new/search would sometimes complain about
missing files in the maildir.  The first time this happened, it did not
correct itself and I ended up reimporting the database (I had moved some
things around so I could have been at fault).  Since then I have seen
these errors on a couple occasions, but they always go away upon
rerunning 'notmuch new'.  My guess is that it has to do with offlineimap
changing flags so I moved 'notmuch new' to the postsync hook and have
not seen the errors since.  If it is important that notmuch never runs
concurrently with an offlineimap sync, it should eventually go in the
docs.


Thanks again,

Jed


[notmuch] [PATCH] RFC: quiet make

2009-11-21 Thread Mikhail Gusarov

Twas brillig at 14:16:52 21.11.2009 UTC+01 when cworth at cworth.org did gyre 
and gimble:

 CW> Meanwhile, the feature I've always wanted with systems like this is
 CW> to document to the user that there's the possibility of adding a
 CW> "V=1" to the command line to get the full output.

I have had another idea in mind which might be useful: a "header" for
silent make mode which explains what commands are run exactly, including
placeholders like $@ $^, so it is not necessary to run with make V=1 to
figure out what's going on.

-- 
  http://fossarchy.blogspot.com/
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 834 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/b6cd5864/attachment.pgp>


[notmuch] Perftool run

2009-11-21 Thread Brett Viren
Well, the result is attached.  There is nothing glaringly obvious and
I'll need to learn the internals better before suggesting anything.

-Brett.
-- next part --
A non-text attachment was scrubbed...
Name: notmuch-dbg.ps.gz
Type: application/x-gzip
Size: 24494 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/5cbf046d/attachment-0001.bin>


[notmuch] [PATCH] Makefile: Magic silent rules.

2009-11-21 Thread Chris Wilson
Use the facilities of GNU make to create a magic function that will
on the first invocation print a description of how to enable verbose
compile lines and then print the quiet rule.

Signed-off-by: Chris Wilson 
Signed-off-by: Carl Worth 
Cc: Mikhail Gusarov 
---
 Makefile   |   22 +++---
 Makefile.local |4 ++--
 lib/Makefile.local |2 +-
 3 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/Makefile b/Makefile
index 3fedcf1..77e70ed 100644
--- a/Makefile
+++ b/Makefile
@@ -2,6 +2,10 @@
 WARN_FLAGS=-Wall -Wextra -Wmissing-declarations -Wwrite-strings -Wswitch-enum
 CFLAGS=-O2

+# Additional programs that are used during the compilation process.
+EMACS ?= emacs
+GZIP ?= gzip
+
 # Additional flags that we will append to whatever the user set.
 # These aren't intended for the user to manipulate.
 extra_cflags := $(shell pkg-config --cflags glib-2.0 gmime-2.4 talloc)
@@ -31,14 +35,26 @@ include lib/Makefile.local
 # And get user settings from the output of configure
 include Makefile.config

+# The user has not set any verbosity, default to quiet mode and inform the
+# user how to enable verbose compiles.
+ifeq ($(V),)
+quiet_DOC := "Use \"$(MAKE) V=1\" to see the verbose compile lines.\n"
+quiet = @echo $(quiet_DOC)$(eval quiet_DOC:=)"  $1 $@"; $($1)
+endif
+# The user has explicitly enabled quiet compilation.
+ifeq ($(V),0)
+quiet = @echo "  $1$@"; $($1)
+endif
+# Otherwise, print the full command line.
+
 %.o: %.cc $(all_deps)
-   $(CXX) -c $(CXXFLAGS) $< -o $@
+   $(call quiet,CXX) -c $(CXXFLAGS) $< -o $@

 %.o: %.c $(all_deps)
-   $(CC) -c $(CFLAGS) $< -o $@
+   $(call quiet,CC) -c $(CFLAGS) $< -o $@

 %.elc: %.el
-   emacs -batch -f batch-byte-compile $<
+   $(call quiet,EMACS) -batch -f batch-byte-compile $<

 .deps/%.d: %.c $(all_deps)
@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
diff --git a/Makefile.local b/Makefile.local
index 3c99624..5fd5d4f 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -20,10 +20,10 @@ notmuch_client_srcs =   \

 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
 notmuch: $(notmuch_client_modules) lib/notmuch.a
-   $(CXX) $^ $(LDFLAGS) -o $@
+   $(call quiet,CXX) $^ $(LDFLAGS) -o $@

 notmuch.1.gz: notmuch.1
-   gzip --stdout notmuch.1 > notmuch.1.gz
+   $(call quiet,GZIP) --stdout $^ > $@

 install: all notmuch.1.gz
for d in $(DESTDIR)$(prefix)/bin/ $(DESTDIR)$(prefix)/share/man/man1 \
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 79f7b0b..a7562c9 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -18,7 +18,7 @@ libnotmuch_cxx_srcs = \

 libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 $(dir)/notmuch.a: $(libnotmuch_modules)
-   $(AR) rcs $@ $^
+   $(call quiet,AR) rcs $@ $^

 SRCS  := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs)
 CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a
-- 
1.6.5.3



[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 16:36:55 +0100, Stefan Schmidt  wrote:
> I executed "/usr/local/bin/notmuch search --sort=oldest-first tag:inbox" by 
> hand
> and from the 21 minutes it took it stayed around 20 in a state where no new
> message where printed and then sudenly all the rest comes up.

That's actually the expected behavior currently.

It used to be that "notmuch search" on the command line wouldn't present
any results until everything was available.

I recently threw in a hack to present the first 100 thread results
quickly and only then does it sit and spin before all the results are
available. I suppose it wouldn't be any harder for it to keep returning
chunks of 100 threads at a time, (though this will slow down the final
result a bit---perhaps not significantly).

And I wouldn't really mind any slowdown there anyway, since any *real*
interface should be calling "notmuch search" in small chunks anyway.

So I'll go ahead and do that.

> In my case only 80 messages were printed before the gap. All of them had a 
> wrong
> year in the timestamp. 1900 and 1970. Maybe notmuch just comes into a bad 
> state
> with this dates?

I don't think the bogus dates are throwing anything off. It's more
likely that you just have a number of messages with no Date header on
them at all. And for such messages, notmuch just chooses a time_t value
of 0 so you'll see whatever that 0 maps to on your system---a date of
1970 there is not surprising. :-)

> I will remove these mails and re-generate the notmuch index to test this out
> after dinner later today.

See my other mail. You may want to tweak the behavior of "notmuch new"
before running it again. (I would not expect the results to be any
different from running it again with no change.)

-Carl


[notmuch] [PATCH] notmuch: Add Maildir directory name as

2009-11-21 Thread mich...@michielbuddingh.net
 =?UTF-8?Q?tag=09name=20for=20messages?=
MIME-Version: 1.0
Date: Sun, 22 Nov 2009 00:25:43 +0100
From: Michiel Buddingh' 
Cc: Carl Worth , notmuch 
In-Reply-To: <20091121221207.GB17268 at jukie.net>
References: <87fx8bygi7.fsf at linux.vnet.ibm.com> <87bpiv4t9h.fsf at 
yoom.home.cworth.org> <87y6lz39nd.fsf at yoom.home.cworth.org> 
<20091121221207.GB17268 at jukie.net>
Message-ID: <9cce5525b093b87fe74d427954ffad89 at localhost>
X-Sender: michiel at michielbuddingh.net
User-Agent: RoundCube Webmail/0.1
Content-Type: multipart/mixed;
boundary="=_365db8948ca90006e4bd3511ef958271"


--=_365db8948ca90006e4bd3511ef958271
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset=UTF-8

On Sat, 21 Nov 2009 17:12:07 -0500, Bart Trojanowski 
wrote:
>> I realized that I actually don't want my mail tagged based on the
>> maildir directories I'm using, (they are arbitrarily-named directories
>> used only to keep the per-directory number of files below about 10
>> thousand).
>> 
>> So we'll probably need to make this an opt-in feature from the
>> configuration file.
> 
> I think notmuch needs something that will add tags based on the
> attributes of a message (headers or body), as it imports data from a
> maildir.
> 
> I am currently considering having procmail deliver to date based
> (-MM) folders and have notmuch determine what tags they should get.

(hi, new here, just subscribed today.  Wanted to reply to Carl's 
 earlier message I read in the archives, but since I don't have that,
 I'm replying to Bart's reply to that message)

Any attempt to match tags up to directories will eventually have 
to deal with with the fact that tags can't be neatly mapped onto 
them.  If I remove a directory-tag from a message, does this 
mean the message is removed from that directory?  What if a 
message has two directory-tags, does it mean it's present in both
directories?

At the same time, this kind of interoperability would be highly
desirable to those of us who access their mail using other  
clients (webmail, mobile phones, etc.) that expect hierarchical
ordering.

In the mean time, I've made a smaller, hopefully more harmless 
patch to let 'notmuch new' mark messages stored in a Maildir 'cur'
folder as 'read' rather than 'unread'.

via webmail,
Michiel Buddingh'
--=_365db8948ca90006e4bd3511ef958271
Content-Transfer-Encoding: base64
Content-Type: application/octet-stream; charset=UTF-8;
 name=0001-Derive-read-status-from-Maildir-cur-folder.patch;
Content-Disposition: attachment;
 filename=0001-Derive-read-status-from-Maildir-cur-folder.patch;

RnJvbSA4YTJkZTEwNjk2YzBlOGVjYWI3ZmU1NDQxYjk5YTMzYTkwNGRiNzY5IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBNaWNoaWVsIEJ1ZGRpbmdoIDxtaWNoaWVsQG1pY2hpZWxidWRk
aW5naC5uZXQ+CkRhdGU6IFN1biwgMjIgTm92IDIwMDkgMDA6MTQ6MTkgKzAxMDAKU3ViamVjdDog
W1BBVENIXSBEZXJpdmUgcmVhZCBzdGF0dXMgZnJvbSBNYWlsZGlyIGN1ciBmb2xkZXIKCi0tLQog
bm90bXVjaC1uZXcuYyB8ICAgNDIgKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0t
LS0tLS0tCiAxIGZpbGVzIGNoYW5nZWQsIDMwIGluc2VydGlvbnMoKyksIDEyIGRlbGV0aW9ucygt
KQoKZGlmZiAtLWdpdCBhL25vdG11Y2gtbmV3LmMgYi9ub3RtdWNoLW5ldy5jCmluZGV4IGJjMzVi
NGUuLmEzNjY0NTMgMTAwNjQ0Ci0tLSBhL25vdG11Y2gtbmV3LmMKKysrIGIvbm90bXVjaC1uZXcu
YwpAQCAtMzksMTIgKzM5LDE3IEBAIGhhbmRsZV9zaWdpbnQgKHVudXNlZCAoaW50IHNpZykpCiB9
CiAKIHN0YXRpYyB2b2lkCi10YWdfaW5ib3hfYW5kX3VucmVhZCAobm90bXVjaF9tZXNzYWdlX3Qg
Km1lc3NhZ2UpCit0YWdfYXNfaW5ib3ggKG5vdG11Y2hfbWVzc2FnZV90ICptZXNzYWdlLCBjb25z
dCBpbnQgcmVhZCkKIHsKICAgICBub3RtdWNoX21lc3NhZ2VfYWRkX3RhZyAobWVzc2FnZSwgImlu
Ym94Iik7Ci0gICAgbm90bXVjaF9tZXNzYWdlX2FkZF90YWcgKG1lc3NhZ2UsICJ1bnJlYWQiKTsK
KyAgICBpZiAocmVhZCA9PSBUUlVFKSB7CisJbm90bXVjaF9tZXNzYWdlX2FkZF90YWcgKG1lc3Nh
Z2UsICJyZWFkIik7CisgICAgfSBlbHNlIHsKKwlub3RtdWNoX21lc3NhZ2VfYWRkX3RhZyAobWVz
c2FnZSwgInVucmVhZCIpOworICAgIH0KIH0KIAorCiBzdGF0aWMgdm9pZAogYWRkX2ZpbGVzX3By
aW50X3Byb2dyZXNzIChhZGRfZmlsZXNfc3RhdGVfdCAqc3RhdGUpCiB7CkBAIC0xMTEsMTkgKzEx
NiwyNCBAQCBhZGRfZmlsZXNfcmVjdXJzaXZlIChub3RtdWNoX2RhdGFiYXNlX3QgKm5vdG11Y2gs
CiAgICAgbm90bXVjaF9zdGF0dXNfdCBzdGF0dXMsIHJldCA9IE5PVE1VQ0hfU1RBVFVTX1NVQ0NF
U1M7CiAgICAgbm90bXVjaF9tZXNzYWdlX3QgKm1lc3NhZ2UgPSBOVUxMOwogICAgIHN0cnVjdCBk
aXJlbnQgKipuYW1lbGlzdCA9IE5VTEw7CisgICAgaW50IGRpcl9oYXNfb25seV9yZWFkX21lc3Nh
Z2VzID0gRkFMU0U7CiAgICAgaW50IG51bV9lbnRyaWVzOwogCi0gICAgLyogSWYgd2UncmUgdG9s
ZCB0bywgd2UgYmFpbCBvdXQgb24gZW5jb3VudGVyaW5nIGEgcmVhZC1vbmx5Ci0gICAgICogZGly
ZWN0b3J5LCAod2l0aCB0aGlzIGJlaW5nIGEgY2xlYXIgY2x1ZSBmcm9tIHRoZSB1c2VyIHRvCi0g
ICAgICogTm90bXVjaCB0aGF0IG5ldyBtYWlsIHdvbid0IGJlIGFycml2aW5nIHRoZXJlIGFuZCB3
ZSBuZWVkIG5vdAotICAgICAqIGxvb2suICovCi0gICAgaWYgKHN0YXRlLT5pZ25vcmVfcmVhZF9v
bmx5X2RpcmVjdG9yaWVzICYmCi0JKHN0LT5zdF9tb2RlICYgU19JV1VTUikgPT0gMCkKLSAgICB7
Ci0Jc3RhdGUtPnNhd19yZWFkX29ubHlfZGlyZWN0b3J5ID0gVFJVRTsKLQlnb3RvIERPTkU7CisK
KyAgICBpZiAoKHN0LT5zdF9tb2RlICYgU19JV1VTUikgPT0gMCkgeworCS8qIElmIHdlJ3JlIHRv
bGQgdG8sIHdlIGJhaWwgb3V0IG9uIGVuY291bnRlcmluZyBhIHJlYWQtb25seQorCSAqIGRpcmVj
dG9yeSwgKHdpdGggdGhpcyBiZWluZyBhIGNsZWFyIGNsdWUgZnJvbSB0aGUgdXNlciB0bworCSAq
IE5vdG11Y2ggdGhhdCBuZXcg

[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 08:12:52 -0700, Bdale Garbee  wrote:
> I haven't figured out how to quickly tag everything as already read or
> archived or whatever .. can someone who knows more about what's going on
> confirm my hypothesis and if so, suggest the best approach to getting to
> a happier state?

See my message up-thread. The only reasonable ways all really do involve
at least a little bit of C-code hacking to either prevent those tags
from getting put there by "notmuch new" or to make it easier to get them
off afterwards.

I'm hoping everyone with this problem will happen to choose a different
solution and we'll get a nice flood of patches to improve things. :-)

And I can't help but apologize. I've known about all these issues, and
wouldn't have invited people to try things out in the current state. But
it was nice of Keith to share this with everyone. And it's nice of all
you to come take a look at things.

So, I'll just ask for a little patience, and we'll hopefully have a nice
system soon.

-Carl


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 15:51:11 +0100, Stefan Schmidt  wrote:
> Disclaimer: I'm using vim, in combination with mutt for email, for years, but
> never dealt with emacs. Please have this in mind and spot any emacs user 
> errors
> in this report. :)

Hi Stefan, welcome to Notmuch! And don't worry, we don't discriminate
(too much) against non-emacs users around here.

> I have first seen notmuch several weeks ago as it seems a silent project. 
> Being
> more then happy now that it envolves quickly and a real developer community
> builds around it.

Yes. Notmuch was a silent project since it was just something that I was
doing for myself. I was always writing it as free software, and even had
a public git repository available, but hadn't advertised it at all yet.

And Keith did rather catch me off guard by announcing it. But I can't
complain as we have gotten a nice community started already, and it's
great to have other people writing the code that I intended to
write. :-)

But it's also true that some obvious problems just aren't taken care of
yet.

> But now to my problem. Getting m mail indexed was easy enough:
> 
> stefan at excalibur:~$ du -chs not-much-mail/
> 1.5Gnot-much-mail/
> 1.5Gtotal
> stefan at excalibur:~$ time notmuch new
> Found 103677 total files.
> Processed 103677 total files in 42m 30s (40 files/sec.).
> Added 100899 new messages to the database (not much, really).

Good. I'm glad that went fairly smoothly for you.

Though, frankly, I think we need to fix "notmuch new" to do much better
than 40 files/sec. One plan I have for this is to not use the database
to search for message IDs when adding many messages---but to instead
just use a hash-table (seeded from any messages already in the
database). This would allow us to do all thread resolution before
indexing messages, without having to do the N different searches, and
also means we'd avoid continually rewriting documents when merging
thread IDs.

> I put (require  'notmuch) in my ~/.emacs ans start emacs with the -f notmuch
> option to enter the notmuch mode.

I'm glad you've figured that much out. I feel bad that that's not even
in the documentation anywhere yet.

> What happends then is that a notmuch process gets started and emacs
> waits for the return.

OK. This is a known shortcoming. As Bdale supposes, this problem is from
notmuch trying to load and construct every thread in your
database. There are actually several different bugs/missing features
here that should be addressed:

  * "notmuch new" should look at the R flag in maildir files to
determine that they are read and do not need to be marked as "inbox"
and "unread"

  * "notmuch setup" should prompt for some date range, ("last 2 months"
by default?) before which no messages will be considered unread.

Either of those two fixes would have prevented your particular
problem. But it's still easy to generate searches that return large
numbers of results. So there's some more to do:

  * The emacs code needs to call "notmuch search" with the --first and
--max-threads options to get a limited set of results, (one or two
screenfuls). You should be able to test this at the command line and
see that it returns results quickly. Then, of course, we'd like the
emacs code to fill in subsequent screenfuls as you page.

But none of that helps you right now. What you need is to retroactively
remove all of the "inbox" and "unread" tags from messages older than
some time period. So then there's another missing feature:

  * We need to support date-range-based searches. If we had that you
could just do:

notmuch tag -inbox -unread until:"2 months ago"

But we don't quite have this yet. Xapian does have support for a
slightly less convenient date range specification:

1970-01-01..2009-09-21

but it turns out that we can't even use that just yet, since to make
that work we would have to have dates saved as MMDD strings for
each message, (where instead we have time_t values stored serialized
into a string that will sort correctly.). So we need a new
ValueRangeProcessor class to map to timestamps, and then we'll need
some fancy parsing to do things like "2 months ago".

So, what's the best thing to do today if you want to start playing with
notmuch? I think you could pick one of the above to work on, (a quick
hack to "notmuch new" and a re-import might do the trick). Or you might
just remove the inbox and unread tags from all messages and then just
let messages that are actually *new* in the future get tagged into the
inbox by "notmuch new". Oh, but then there's another missing feature:

  * We need a syntax to specify a search string that should match all
messages. Then you could do:

notmuch tag -inbox -unread 

Yikes! So many bugs and missing features. How is anyone actually using
this system? Well, Keith and I were able to get past all this by simply
doing a "notmuch restore" based on tags we g

[notmuch] suggestion/request about notmuch config

2009-11-21 Thread Jameson Graef Rollins
Hi, folks.  I just recently heard about notmuch and am very excited by
it.  It looks like exactly what I've been waiting years for.  Also
excited to see so much good development work happening.

I started playing around with notmuch today and I have a suggestion
about the notmuch config files and databases.

I would prefer to see all notmuch files and data in the same
directory, preferably:

~/.notmuch

The config file could be:

~/.notmuch/config

and the database could be:

~/.notmuch/db

or something to that affect.  This would make it much easier to keep
track of in my opinion.  I would also prefer to keep the database
completely separate from my mail directory itself.

I think these changes would be fairly trivial.  I would be happy to
take a crack at them if there aren't any major concerns about this
proposal.  Obviously moving the default config might impact current
users, but I think it's fairly minor.  One reason why it's probably
better to make this change now...

Let me know what people think.  Looking forward to working with
notmuch!

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/a6a113ce/attachment.pgp>


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Brett Viren
On Sat, Nov 21, 2009 at 5:40 PM, Jed Brown  wrote:
> On Sat, 21 Nov 2009 17:36:18 -0500, Brett Viren  
> wrote:
>> How can I purge the index? ?I can't locate it.
>
> I believe you can just remove /path/to/maildir/.notmuch

Doh!  Thanks.  I have a dovecot controlled Maildir so this
dot-directory got lost in the forest of all the others.

-Brett.


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Brett Viren
On Sat, Nov 21, 2009 at 12:07 PM, Carl Worth  wrote:

> Though, frankly, I think we need to fix "notmuch new" to do much better
> than 40 files/sec.

Just a "me too".

Processed 130871 total files in 38m 7s (57 files/sec.).
Added 102723 new messages to the database (not much, really).

This was ~2GB of mail on a 2.5GHz CPU.  That seems pretty reasonable
to me but I'd like to rerun the "notmuch new" under google perftools
to see if there are any obvious bottlenecks that might be cleaned up.

How can I purge the index?  I can't locate it.

-Brett.


[notmuch] [PATCH 2/2] Add a new "notmuch search-messages" command.

2009-11-21 Thread Carl Worth
This allows for searching for individual messages, rather than threads
as "notmuch search" does. Currently just prints out the message id for
each message.
---
 Makefile.local|   29 +-
 notmuch-client.h  |3 +
 notmuch-search-messages.c |  138 +
 notmuch.c |   29 -
 4 files changed, 181 insertions(+), 18 deletions(-)
 create mode 100644 notmuch-search-messages.c

diff --git a/Makefile.local b/Makefile.local
index 3c99624..288d5e9 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -2,20 +2,21 @@ all: notmuch notmuch.1.gz

 emacs: notmuch.elc

-notmuch_client_srcs =  \
-   notmuch.c   \
-   notmuch-config.c\
-   notmuch-dump.c  \
-   notmuch-new.c   \
-   notmuch-reply.c \
-   notmuch-restore.c   \
-   notmuch-search.c\
-   notmuch-setup.c \
-   notmuch-show.c  \
-   notmuch-tag.c   \
-   notmuch-time.c  \
-   gmime-filter-reply.c\
-   query-string.c  \
+notmuch_client_srcs =  \
+   notmuch.c   \
+   notmuch-config.c\
+   notmuch-dump.c  \
+   notmuch-new.c   \
+   notmuch-reply.c \
+   notmuch-restore.c   \
+   notmuch-search.c\
+   notmuch-search-messages.c   \
+   notmuch-setup.c \
+   notmuch-show.c  \
+   notmuch-tag.c   \
+   notmuch-time.c  \
+   gmime-filter-reply.c\
+   query-string.c  \
show-message.c

 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
diff --git a/notmuch-client.h b/notmuch-client.h
index b65aa77..c4ca687 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -106,6 +106,9 @@ int
 notmuch_search_command (void *ctx, int argc, char *argv[]);

 int
+notmuch_search_messages_command (void *ctx, int argc, char *argv[]);
+
+int
 notmuch_setup_command (void *ctx, int argc, char *argv[]);

 int
diff --git a/notmuch-search-messages.c b/notmuch-search-messages.c
new file mode 100644
index 000..b01d566
--- /dev/null
+++ b/notmuch-search-messages.c
@@ -0,0 +1,138 @@
+/* notmuch - Not much of an email program, (just index and search)
+ *
+ * Copyright ?? 2009 Carl Worth
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 3 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program.  If not, see http://www.gnu.org/licenses/ .
+ *
+ * Author: Carl Worth 
+ */
+
+#include "notmuch-client.h"
+
+/* If the user asks for a *lot* of results, lets give some results as
+ * quickly as possible and let the user read those while we compute
+ * the remainder. */
+#define NOTMUCH_SHOW_INITIAL_BURST 100
+
+static void
+do_search_messages (notmuch_query_t *query,
+   int first, int max)
+{
+notmuch_messages_t *messages;
+notmuch_message_t *message;
+
+for (messages = notmuch_query_search_messages (query, first, max);
+notmuch_messages_has_more (messages);
+notmuch_messages_advance (messages))
+{
+   message = notmuch_messages_get (messages);
+
+   printf ("id:%s\n", notmuch_message_get_message_id (message));
+
+   notmuch_message_destroy (message);
+}
+}
+
+int
+notmuch_search_messages_command (void *ctx, int argc, char *argv[])
+{
+notmuch_config_t *config;
+notmuch_database_t *notmuch;
+notmuch_query_t *query;
+char *query_str;
+int i, first = 0, max = -1;
+char *opt, *end;
+notmuch_sort_t sort = NOTMUCH_SORT_NEWEST_FIRST;
+
+for (i = 0; i < argc && argv[i][0] == '-'; i++) {
+   if (strcmp (argv[i], "--") == 0) {
+   i++;
+   break;
+   }
+   if (STRNCMP_LITERAL (argv[i], "--first=") == 0) {
+   opt = argv[i] + sizeof ("--first=") - 1;
+   first = strtoul (opt, &end, 10);
+   if (*opt == '\0' || *end != '\0') {
+   fprintf (stderr, "Invalid value for --first: %s\n", opt);
+   return 1;
+   }
+   } else if (STRNCMP_LITERAL (argv[i], "--max=") == 0) {
+   opt = argv[i] + sizeof ("--max=") - 1;
+   max = strtoul (opt, &end, 10);
+   if (*opt == '\0' || *end != '\0') {
+   fprintf (stderr, "Invalid value for --max: %s\n", opt);
+   return 1;
+   

[notmuch] [PATCH 1/2] notmuch search: Rename the --max-threads option to --max.

2009-11-21 Thread Carl Worth
The idea is to make this a little more general so that different
commands can use identical option names.
---
 notmuch-search.c |   22 +++---
 notmuch.c|2 +-
 2 files changed, 12 insertions(+), 12 deletions(-)

diff --git a/notmuch-search.c b/notmuch-search.c
index 2b1c0fe..845321d 100644
--- a/notmuch-search.c
+++ b/notmuch-search.c
@@ -29,7 +29,7 @@ static void
 do_search_threads (const void *ctx,
   notmuch_query_t *query,
   notmuch_sort_t sort,
-  int first, int max_threads)
+  int first, int max)
 {
 notmuch_thread_t *thread;
 notmuch_threads_t *threads;
@@ -37,7 +37,7 @@ do_search_threads (const void *ctx,
 time_t date;
 const char *relative_date;

-for (threads = notmuch_query_search_threads (query, first, max_threads);
+for (threads = notmuch_query_search_threads (query, first, max);
 notmuch_threads_has_more (threads);
 notmuch_threads_advance (threads))
 {
@@ -83,7 +83,7 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
 notmuch_database_t *notmuch;
 notmuch_query_t *query;
 char *query_str;
-int i, first = 0, max_threads = -1;
+int i, first = 0, max = -1;
 char *opt, *end;
 notmuch_sort_t sort = NOTMUCH_SORT_NEWEST_FIRST;

@@ -99,11 +99,11 @@ notmuch_search_command (void *ctx, int argc, char *argv[])
fprintf (stderr, "Invalid value for --first: %s\n", opt);
return 1;
}
-   } else if (STRNCMP_LITERAL (argv[i], "--max-threads=") == 0) {
-   opt = argv[i] + sizeof ("--max-threads=") - 1;
-   max_threads = strtoul (opt, &end, 10);
+   } else if (STRNCMP_LITERAL (argv[i], "--max=") == 0) {
+   opt = argv[i] + sizeof ("--max=") - 1;
+   max = strtoul (opt, &end, 10);
if (*opt == '\0' || *end != '\0') {
-   fprintf (stderr, "Invalid value for --max-threads: %s\n", opt);
+   fprintf (stderr, "Invalid value for --max: %s\n", opt);
return 1;
}
} else if (STRNCMP_LITERAL (argv[i], "--sort=") == 0) {
@@ -151,17 +151,17 @@ notmuch_search_command (void *ctx, int argc, char *argv[])

 notmuch_query_set_sort (query, sort);

-if (max_threads < 0 || max_threads > NOTMUCH_SHOW_INITIAL_BURST)
+if (max < 0 || max > NOTMUCH_SHOW_INITIAL_BURST)
 {
do_search_threads (ctx, query, sort,
   first, NOTMUCH_SHOW_INITIAL_BURST);

first += NOTMUCH_SHOW_INITIAL_BURST;
-   if (max_threads > 0)
-   max_threads -= NOTMUCH_SHOW_INITIAL_BURST;
+   if (max > 0)
+   max -= NOTMUCH_SHOW_INITIAL_BURST;
 }

-do_search_threads (ctx, query, sort, first, max_threads);
+do_search_threads (ctx, query, sort, first, max);

 notmuch_query_destroy (query);
 notmuch_database_close (notmuch);
diff --git a/notmuch.c b/notmuch.c
index 5cc8e4c..14d4865 100644
--- a/notmuch.c
+++ b/notmuch.c
@@ -139,7 +139,7 @@ command_t commands[] = {
   "\n"
   "\t\tSupported options for search include:\n"
   "\n"
-  "\t\t--max-threads=\n"
+  "\t\t--max=\n"
   "\n"
   "\t\t\tRestricts displayed search results to a subset\n"
   "\t\t\tof the results that would match the terms.\n"
-- 
1.6.5.2



[notmuch] RFC: "notmuch search-messages"

2009-11-21 Thread Carl Worth
Here's a quick proposal for a new "notmuch search-messages" command,
from which one can generate a list of Message IDs matching a search
string.

Beyond this, I'd like to add some --format options to both "search"
and "search-messages" so that users can get out the information they
want, (such as thread-id only from a thread search, filenames only
from a message search, etc.).

Any thoughts from others on this kind of thing?

I know people have been talking about doing thing like modifying
message files based on tags, and this seems like the beginnings of
support for doing scripted operations like that.

-Carl



[notmuch] [PATCH] notmuch: Add Maildir directory name as tag name for messages

2009-11-21 Thread Bart Trojanowski
* Carl Worth  [091121 15:28]:
> And sadly, I just pulled it out again.
> 
> I realized that I actually don't want my mail tagged based on the
> maildir directories I'm using, (they are arbitrarily-named directories
> used only to keep the per-directory number of files below about 10
> thousand).
> 
> So we'll probably need to make this an opt-in feature from the
> configuration file.

I think notmuch needs something that will add tags based on the
attributes of a message (headers or body), as it imports data from a
maildir.

I am currently considering having procmail deliver to date based
(-MM) folders and have notmuch determine what tags they should get.

-Bart

-- 
WebSig: http://www.jukie.net/~bart/sig/


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Stefan Schmidt
Hello.

On Sat, 2009-11-21 at 08:12, Bdale Garbee wrote:
> On Sat, 2009-11-21 at 15:51 +0100, Stefan Schmidt wrote:
> 
> > Sadly that takes around 25 minutes here on an Intel Core2Duo notbeook 
> > (Thinkpad
> > X200s). I tried this several times now. CPU load was low (~10%) during this 
> > time
> > so it is mostly IO bound.
> 
> I see the same behavior on my notebook.  
> 
> I gather from talking to keithp that things like the 'state of already
> being read' aren't being picked up from the file names in the local
> Maildir yet.  Thus I suspect it's a fairly unusual / worst case scenario
> trying to start up with 178k (in my case) supposedly-unread messages
> tagged inbox.

Using the read flag during notmuch new would indeed be nice. But some further
testing brings some doubts that it is an overload due to to many unread
messages.

I executed "/usr/local/bin/notmuch search --sort=oldest-first tag:inbox" by hand
and from the 21 minutes it took it stayed around 20 in a state where no new
message where printed and then sudenly all the rest comes up.

In my case only 80 messages were printed before the gap. All of them had a wrong
year in the timestamp. 1900 and 1970. Maybe notmuch just comes into a bad state
with this dates?

Bdale, can you confirm this for your case?

I will remove these mails and re-generate the notmuch index to test this out
after dinner later today.

regards
Stefan Schmidt


[notmuch] [PATCH 1/2] lib/message: Add function to get maildir flags.

2009-11-21 Thread Keith Packard
On Sun, 22 Nov 2009 01:11:00 +0100, Stefan Schmidt  wrote:

> +const char *
> +notmuch_message_get_flags (notmuch_message_t *message)

This function should interpret the flags that it finds and return a
suitable set of notmuch tags. I'd suggest that 'unread' messages get
both 'unread' and 'inbox' tags, as Maildir doesn't distinguish between
'don't show this to be by default again please' and 'I've read this
message'. It seems best to hide the maildir-specific details inside the
library instead of exposing them.

Also, we have only the 'unread' tag; we don't have a 'read' tag, which
would simply be the inverse of 'unread'.

-- 
keith.packard at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/394e59b9/attachment.pgp>


[notmuch] Asynchronous tagging

2009-11-21 Thread Keith Packard
On Sat, 21 Nov 2009 23:46:44 +0100, Carl Worth  wrote:

> So some investigation is needed to see how important that optimization
> is, and if it's important to see whether there's another way to keep the
> performance while being able to support renames. (Or alternately,
> allowing the user to configure an option saying, "I need to support
> renames even if that means that notmuch new is a bit slower.").

I'd suggest that the best way to make this more efficient would be to
capture directory contents (along with the directory mtime) and use that
to detect changes. If we assume that mail messages are never changed, we
could use that to avoid stat'ing files in directories too.

-- 
keith.packard at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/469948c7/attachment.pgp>


[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Stefan Schmidt
Hello.

Disclaimer: I'm using vim, in combination with mutt for email, for years, but
never dealt with emacs. Please have this in mind and spot any emacs user errors
in this report. :)

I have first seen notmuch several weeks ago as it seems a silent project. Being
more then happy now that it envolves quickly and a real developer community
builds around it.

But now to my problem. Getting m mail indexed was easy enough:

stefan at excalibur:~$ du -chs not-much-mail/
1.5Gnot-much-mail/
1.5Gtotal
stefan at excalibur:~$ time notmuch new
Found 103677 total files.
Processed 103677 total files in 42m 30s (40 files/sec.).
Added 100899 new messages to the database (not much, really).

Tip: If you have any sub-directories that are archives (that is,
they will never receive new mail), marking these directories as
read-only (chmod u-w /path/to/dir) will make "notmuch new"
much more efficient (it won't even look in those directories).

real43m0.943s
user22m46.513s
sys 0m39.418s


I put (require  'notmuch) in my ~/.emacs ans start emacs with the -f notmuch
option to enter the notmuch mode. What happends then is that a notmuch process
gets started and emacs waits for the return.

23649 pts/1SN+0:00  |   \_ emacs -f notmuch
23651 ?RNs0:03  |   \_ /usr/local/bin/notmuch search
--sort=oldest-first tag:inbox

Sadly that takes around 25 minutes here on an Intel Core2Duo notbeook (Thinkpad
X200s). I tried this several times now. CPU load was low (~10%) during this time
so it is mostly IO bound.

I checked that I don't have any big files like mutt header caches left and all
my mail is stored in maildir format diretcly from offlineimap. I'm more then
happy to test any patches on this issue or do some debugging myself if I get
some hints where to look.

regards
Stefan Schmidt


[notmuch] [PATCH] RFC: quiet make

2009-11-21 Thread Carl Worth
On Fri, 20 Nov 2009 19:18:27 +0600, Mikhail Gusarov  wrote:
> I don't entirely like duplicating every command line in makefile,
> so this patch is RFC. Someone with bigger Make-fu than mine probably
> knows a better way.

Hi Mikhail,

Getting a quieter compile out is a great idea.

But, you're right that duplicating the command line invocations is a
problem. That sets up a fragile system where we'll end up with "make"
and "make V=1" actually doing different things.

Meanwhile, the feature I've always wanted with systems like this is to
document to the user that there's the possibility of adding a "V=1" to
the command line to get the full output.

Fortunately, I'm sitting next to Chris Wilson right now, and I know that
all I need to do is to mention the idea to him, and he'll give us a
solution that doesn't duplicate the commands, documents V=1, and will
just be a tiny change to our Makefiles.

-Carl


[notmuch] notmuch vim support

2009-11-21 Thread Bart Trojanowski
* Stefan Schmidt  [091121 14:06]:
> IMHO it would be best to do the development inside the main tree. You could 
> use
> branches there too and it would be easier to find and test. And as notmuch is
> still under heavy development its not a problem that the vim interface is as
> well.

Perfect.  How would you like to proceed with the initial integration?
Pulling my tree, or me squashing all the changes into a single patch?
Also, are you OK with where the files live (ie under a vim directory).

After that, do you want me to submit patches by email, via pull
requests, or giving me push access to a branch at your repo?

BTW, have you considered using github or gitorious so branch management
becomes a distributed problem?

Cheers,
-Bart

-- 
WebSig: http://www.jukie.net/~bart/sig/


[notmuch] [PATCH] Makefile: Magic silent rules.

2009-11-21 Thread Chris Wilson
Use the facilities of GNU make to create a magic function that will
on the first invocation print a description of how to enable verbose
compile lines and then print the quiet rule.

Signed-off-by: Chris Wilson 
Signed-off-by: Carl Worth 
Cc: Mikhail Gusarov 
---
 Makefile   |   18 +++---
 Makefile.local |2 +-
 lib/Makefile.local |2 +-
 3 files changed, 17 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 0411cbb..96fcdd5 100644
--- a/Makefile
+++ b/Makefile
@@ -28,14 +28,26 @@ include lib/Makefile.local
 # And get user settings from the output of configure
 include Makefile.config

+# The user has not set any verbosity, default to quiet mode and inform the
+# user how to enable verbose compiles.
+ifeq ($(V),)
+quiet_DOC := "Use \"$(MAKE) V=1\" to see the verbose compile lines.\n"
+quiet = @echo $(quiet_DOC)$(eval quiet_DOC:=)"  $1 $@";
+endif
+# The user has explicitly enabled quiet compilation.
+ifeq ($(V),0)
+quiet = @echo "  $1$@";
+endif
+# Otherwise, print the full command line.
+
 %.o: %.cc
-   $(CXX) -c $(CFLAGS) $(CXXFLAGS) $< -o $@
+   $(call quiet,CXX) $(CXX) -c $(CFLAGS) $(CXXFLAGS) $< -o $@

 %.o: %.c
-   $(CC) -c $(CFLAGS) $< -o $@
+   $(call quiet,CC) $(CC) -c $(CFLAGS) $< -o $@

 %.elc: %.el
-   emacs -batch -f batch-byte-compile $<
+   $(call quiet,EMACS) emacs -batch -f batch-byte-compile $<

 .deps/%.d: %.c
@set -e; rm -f $@; mkdir -p $$(dirname $@) ; \
diff --git a/Makefile.local b/Makefile.local
index ecd4ceb..115de0e 100644
--- a/Makefile.local
+++ b/Makefile.local
@@ -20,7 +20,7 @@ notmuch_client_srcs = \

 notmuch_client_modules = $(notmuch_client_srcs:.c=.o)
 notmuch: $(notmuch_client_modules) lib/notmuch.a
-   $(CXX) $^ $(LDFLAGS) -o $@
+   $(call quiet,CXX) $(CXX) $^ $(LDFLAGS) -o $@

 notmuch.1.gz: notmuch.1
gzip --stdout notmuch.1 > notmuch.1.gz
diff --git a/lib/Makefile.local b/lib/Makefile.local
index 79f7b0b..f635da7 100644
--- a/lib/Makefile.local
+++ b/lib/Makefile.local
@@ -18,7 +18,7 @@ libnotmuch_cxx_srcs = \

 libnotmuch_modules = $(libnotmuch_c_srcs:.c=.o) $(libnotmuch_cxx_srcs:.cc=.o)
 $(dir)/notmuch.a: $(libnotmuch_modules)
-   $(AR) rcs $@ $^
+   $(call quiet,AR) $(AR) rcs $@ $^

 SRCS  := $(SRCS) $(libnotmuch_c_srcs) $(libnotmuch_cxx_srcs)
 CLEAN := $(CLEAN) $(libnotmuch_modules) $(dir)/notmuch.a
-- 
1.6.5.3



[notmuch] notmuch vim support

2009-11-21 Thread Bart Trojanowski
Hi list,

I've spent a few days on porting notmuch.el to vim.  It's not as feature
rich as notmuch.el yet, but it's coming along nicely.  The results of my
work are here:

git://git.jukie.net/notmuch.git (on the vim branch)
http://git.jukie.net/notmuch.git?a=shortlog;h=refs/heads/vim

There are some screenshots in my blog:

http://www.jukie.net/~bart/blog/notmuch.vim

List of supported features are in the README:

http://git.jukie.net/notmuch.git?a=blob;f=vim/README;hb=refs/heads/vim

I'd love for this to be included in the official project, but I am not
sure if you'd prefer to merge at this point or wait will I am done.
Naturally, if anyone is interested I'd love some help.

Cheers,

-Bart

-- 
WebSig: http://www.jukie.net/~bart/sig/


[notmuch] interesting project!

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 11:01:46 +0200, Dirk-Jan C. Binnema  wrote:
> Hi all,

Hi, Dirk. Welcome to notmuch!

> Wow, 'notmuch' looks like a very interesting project. In 2008, I wrote an
> e-mail (Maildir) search tool called 'mu'[1], also using Xapian and GMime; my
> plan was at some point to turn it into a mail reader (use
> offlineimap/fetchmail etc. for getting the mail, and something else for
> sending it), but I never got that far. Search works pretty well
> though. Anyhow, it seems notmuch is getting there quickly.

Ah, how ignorant I was. I probably could have saved myself a bunch of
work if I had just started with mu. Oh, well.

> Anyhow, I'll study the notmuch code and see if there are some useful bits in
> my code that might make sense there, e.g., various dir scanning optimizations,
> see [2].

That sounds great. It's also good to have people with experience in this
area join and help out. I'll look forward to any ideas or other
contributions you will have.

> [2] http://djcbflux.blogspot.com/2008/10/seek-destroy.html

Thanks. Stewart Smith contributed a patch to notmuch a couple of days
ago that added inode sorting, (which I was totally unaware of as an
optimization idea):

Read mail directory in inode number order
http://git.notmuchmail.org/git/notmuch?a=commitdiff;h=a45ff8c36112a2f17c1ad5c20a16c30a47759797

-Carl


[notmuch] Greetings and suggestions

2009-11-21 Thread Brett Viren
Hi,

I recently heard about notmuch from a reddit.com posting.  Immediately
I knew I needed to try it.

I've been a long time GNUS user and with VM before that.  GNUS is
great but I have never been very successful getting it to play well
with an indexer.

So far, after a little playing with notmuch I think it looks very good!

I think integrating notmuch into GNUS or incorporating GNUS features
into notmuch.el would be good direction to go.  The two features I see
missing most are:

 - posting styles (ie, set From: to non standard value) based on
search pattern.  This is really important when one subscribes to
different mailing lists using different addresses and the ML only
allows mail from registered addresses.

 - apply tags based on search pattern as new mail is processed.  This
would be like GNUS split-rules, but even better as one can place more
than one tag.

Anyways, great project and I look forward to contributing where I can.

-Brett.


[notmuch] [PATCH] Permit opening the notmuch database in read-only mode.

2009-11-21 Thread Keith Packard
On Sat, 21 Nov 2009 19:54:25 +, Chris Wilson  
wrote:

> +typedef enum {
> +NOTMUCH_DATABASE_MODE_READONLY = 0,
> +NOTMUCH_DATABASE_MODE_WRITABLE
> +} notmuch_database_mode_t;

Wondering if MODE_READWRITE would be more descriptive?

-- 
keith.packard at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/442e8395/attachment.pgp>


[notmuch] [PATCH] Add notmuch-folder mode to provide an overview of search matches

2009-11-21 Thread Keith Packard
Folder mode takes a (user-configurable) list of search patterns and
produces a list of those patterns and the count of messages that they
match. When an entry in this list is selected, a search window with
the defined search is opened.  The set of folders is defined as a
list, each element contains the name of the folder and the query string
to count.

Signed-off-by: Keith Packard 
---
 notmuch.el |   81 
 1 files changed, 81 insertions(+), 0 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index 4f369de..7cab3c1 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -1046,4 +1046,85 @@ current search results AND that are tagged with the 
given tag."

 (setq mail-user-agent 'message-user-agent)

+(defvar notmuch-folder-mode-map
+  (let ((map (make-sparse-keymap)))
+(define-key map "n" 'next-line)
+(define-key map "p" 'previous-line)
+(define-key map "x" 'kill-this-buffer)
+(define-key map "q" 'kill-this-buffer)
+(define-key map "s" 'notmuch-search)
+(define-key map (kbd "RET") 'notmuch-folder-show-search)
+(define-key map "<" 'beginning-of-buffer)
+(define-key map "=" 'notmuch-folder)
+(define-key map "?" 'describe-mode)
+(define-key map [mouse-1] 'notmuch-folder-show-search)
+map)
+  "Keymap for \"notmuch folder\" buffers.")
+
+(fset 'notmuch-folder-mode-map notmuch-folder-mode-map)
+
+(defcustom notmuch-folders (quote (("inbox" . "tag:inbox") ("unread" . 
"tag:unread")))
+  "List of searches for the notmuch folder view"
+  :type '(alist :key-type (string) :value-type (string))
+  :group 'notmuch)
+  
+(defun notmuch-folder-mode ()
+  "Major mode for showing notmuch 'folders'.
+
+This buffer contains a list of messages counts returned by a
+customizable set of searches of your email archives. Each line
+in the buffer shows the search terms and the resulting message count.
+
+Pressing RET on any line opens a search window containing the
+results for the search terms in that line.
+
+\\{notmuch-folder-mode-map}"
+  (interactive)
+  (kill-all-local-variables)
+  (use-local-map 'notmuch-folder-mode-map)
+  (setq truncate-lines t)
+  (hl-line-mode 1)
+  (setq major-mode 'notmuch-folder-mode
+   mode-name "notmuch-folder")
+  (setq buffer-read-only t))
+
+(defun notmuch-folder-add (folders)
+  (if folders
+  (let ((name (car (car folders)))
+   (inhibit-read-only t)
+   (search (cdr (car folders
+   (insert name)
+   (indent-to 16 1)
+   (call-process notmuch-command nil t nil "count" search)
+   (notmuch-folder-add (cdr folders)
+
+(defun notmuch-folder-find-name ()
+  (save-excursion
+(beginning-of-line)
+(let ((beg (point)))
+  (forward-word)
+  (filter-buffer-substring beg (point)
+
+(defun notmuch-folder-show-search (&optional folder)
+  "Show a search window for the search related to the specified folder."
+  (interactive)
+  (if (null folder)
+  (setq folder (notmuch-folder-find-name)))
+  (let ((search (assoc folder notmuch-folders)))
+(if search
+   (notmuch-search (cdr search) t
+
+(defun notmuch-folder ()
+  "Show the notmuch folder view and update the displayed counts."
+  (interactive)
+  (let ((buffer (get-buffer-create "*notmuch-folders*")))
+(switch-to-buffer buffer)
+(let ((inhibit-read-only t)
+ (n (line-number-at-pos)))
+  (erase-buffer)
+  (notmuch-folder-mode)
+  (notmuch-folder-add notmuch-folders)
+  (goto-char (point-min))
+  (goto-line n
+
 (provide 'notmuch)
-- 
1.6.5.2



[notmuch] interesting project!

2009-11-21 Thread Jameson Greaf Rollins
On Sat, Nov 21, 2009 at 01:10:42PM +0100, Carl Worth wrote:
> On Sat, 21 Nov 2009 11:01:46 +0200, Dirk-Jan C. Binnema  gmail.com> wrote:
> > Anyhow, I'll study the notmuch code and see if there are some useful bits in
> > my code that might make sense there, e.g., various dir scanning 
> > optimizations,
> > see [2].
> 
> That sounds great. It's also good to have people with experience in this
> area join and help out. I'll look forward to any ideas or other
> contributions you will have.

I've been using mu for a while now and have found it incredibly
useful.  I just heard about notmuch and it seems like the mail
processing system I've been waiting for, so I'm incredibly excited.
The idea of the mu and notmuch folks working together sounds
incredibly awesome.  I am really encouraged.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/717c6765/attachment.pgp>


[notmuch] [PATCH] notmuch.el: Add face support to search and show mode

2009-11-21 Thread Carl Worth
On Sat, 21 Nov 2009 09:37:30 +0530, "Aneesh Kumar K.V"  wrote:
> To verify whether i have some setting in .emacs i ran emacs -q and still
> i am able see the colors. So not sure what extra is needed. I also tried
> emacs23 and that also worked. Are you able to see this faces via 
> customize-face
> command ? And what is the color they display in customize-face ?

I think this was user-error on my part. So sorry about the noise.

For one thing, I recently broke the Makefiles to install
notmuch.el/notmuch.elc into the wrong directory.

Fixes shortly...

-Carl


[notmuch] interesting project!

2009-11-21 Thread Dirk-Jan C. Binnema
Hi all,

Wow, 'notmuch' looks like a very interesting project. In 2008, I wrote an
e-mail (Maildir) search tool called 'mu'[1], also using Xapian and GMime; my
plan was at some point to turn it into a mail reader (use
offlineimap/fetchmail etc. for getting the mail, and something else for
sending it), but I never got that far. Search works pretty well
though. Anyhow, it seems notmuch is getting there quickly.

Anyhow, I'll study the notmuch code and see if there are some useful bits in
my code that might make sense there, e.g., various dir scanning optimizations,
see [2].

Good luck!
Dirk.


[1] http://www.djcbsoftware.nl/code/mu/
[2] http://djcbflux.blogspot.com/2008/10/seek-destroy.html

-- 
Dirk-Jan C. Binnema  Helsinki, Finland
e:djcb at djcbsoftware.nl   w:www.djcbsoftware.nl
pgp: D09C E664 897D 7D39 5047 A178 E96A C7A1 017D DA3C


[notmuch] [PATCH] notmuch.el: Add face support to search and show mode

2009-11-21 Thread Aneesh Kumar K.V
On Sat, Nov 21, 2009 at 03:59:12AM +0100, Carl Worth wrote:
> On Fri, 20 Nov 2009 20:57:24 +0530, "Aneesh Kumar K.V"  linux.vnet.ibm.com> wrote:
> > This add two faces, notmuch-show-subject-face and
> > notmuch-tag-unread-face. The first face is used to show the subject
> > line in the notmuch-show-mode and the second one the unread tag in
> > the notmuch-search-mode.
> > 
> > The changes are done looking at message.el in emacs source
> 
> Hi Aneesh,
> 
> This looks like very interesting stuff, but I wasn't able to get it to
> work. Initially I was running emacs within a terminal (emacs -nw) so I
> thought that might be preventing me from seeing the face differences or
> so.
> 
> But I can't seem to see anything in graphical emacs either.
> 
> Is there something more I need to do to make this work?

To verify whether i have some setting in .emacs i ran emacs -q and still
i am able see the colors. So not sure what extra is needed. I also tried
emacs23 and that also worked. Are you able to see this faces via customize-face
command ? And what is the color they display in customize-face ?

-aneesh



[notmuch] [PATCH] Missing final semi-colon in .desktop's Categories.

2009-11-21 Thread James Rowe
"Those keys which have several values should have a semicolon as the trailing
character."
  -- http://standards.freedesktop.org/desktop-entry-spec/1.0/ar01s03.html

Signed-off-by: James Rowe 
---
 notmuch.desktop |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/notmuch.desktop b/notmuch.desktop
index d29dff6..819cd1e 100644
--- a/notmuch.desktop
+++ b/notmuch.desktop
@@ -4,4 +4,4 @@ Exec=emacs -f notmuch
 Icon=emblem-mail
 Terminal=false
 Type=Application
-Categories=Network;Email
+Categories=Network;Email;
-- 
1.6.4.4



[notmuch] 25 minutes load time with emacs -f notmuch

2009-11-21 Thread Bdale Garbee
On Sat, 2009-11-21 at 15:51 +0100, Stefan Schmidt wrote:

> Sadly that takes around 25 minutes here on an Intel Core2Duo notbeook 
> (Thinkpad
> X200s). I tried this several times now. CPU load was low (~10%) during this 
> time
> so it is mostly IO bound.

I see the same behavior on my notebook.  

I gather from talking to keithp that things like the 'state of already
being read' aren't being picked up from the file names in the local
Maildir yet.  Thus I suspect it's a fairly unusual / worst case scenario
trying to start up with 178k (in my case) supposedly-unread messages
tagged inbox.

I haven't figured out how to quickly tag everything as already read or
archived or whatever .. can someone who knows more about what's going on
confirm my hypothesis and if so, suggest the best approach to getting to
a happier state?

Bdale




[notmuch] [PATCH] notmuch.el: Add face support to search and show mode

2009-11-21 Thread Carl Worth
On Fri, 20 Nov 2009 20:57:24 +0530, "Aneesh Kumar K.V"  wrote:
> This add two faces, notmuch-show-subject-face and
> notmuch-tag-unread-face. The first face is used to show the subject
> line in the notmuch-show-mode and the second one the unread tag in
> the notmuch-search-mode.
> 
> The changes are done looking at message.el in emacs source

Hi Aneesh,

This looks like very interesting stuff, but I wasn't able to get it to
work. Initially I was running emacs within a terminal (emacs -nw) so I
thought that might be preventing me from seeing the face differences or
so.

But I can't seem to see anything in graphical emacs either.

Is there something more I need to do to make this work?

-Carl


[notmuch] [PATCH] Quote file names passed to the shell

2009-11-21 Thread Jed Brown

Prior to this, notmuch-show-pipe-message could not handle file names
with spaces and similar.
---
 notmuch.el |3 ++-
 1 files changed, 2 insertions(+), 1 deletions(-)

-- next part --
A non-text attachment was scrubbed...
Name: 0001-Quote-file-names-passed-to-the-shell.patch
Type: text/x-patch
Size: 796 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20091121/3e6fa1e5/attachment.bin>


[notmuch] Segfault searching for tags

2009-11-21 Thread Carl Worth
On Fri, 20 Nov 2009 20:03:00 +0100, Adrian Perez de Castro  wrote:
> Well, of course you are right, it is an overloaded operator, which
> (unfortunately, IMHO) looks like a pointer dereference. That is exactly
> one of the things that I find more confusing about C++: it has features
> like operator overloading which look cool initially, but that in the end
> imply more complexity than needed. I can understand why you decided to
> wrap Xapian with a plain C API :)

I'm glad you agree.

Though I should mention that I earned my summer's salary during an
internship once by solving a performance problem that had dodged the
engineers on the project, (since they overlooked an overloaded array
subscript operator on a std::string class as something that could be
expensive---profiling made it obvious, and a temporary copy to a real
array with a real subscript fixed the bug).

So I can't say that operator overloading never helped me. But I know I
left that internship determined not to use it myself.

> I can confirm that this patch avoids the segfault in my case, too. Thanks
> a lot for the quick fix.

Excellent. I'm glad to hear it worked for you.

I'm sorry that the bug was there, since this was a regression that's
come back once or twice now. The project is overdue for a test suite
already...

-Carl


[notmuch] [PATCH] Drop redundant CFLAGS, was already included in CXXFLAGS

2009-11-21 Thread Carl Worth
On Fri, 20 Nov 2009 16:58:15 +0100, Jed Brown  wrote:
>  %.o: %.cc $(all_deps)
> - $(CXX) -c $(CFLAGS) $(CXXFLAGS) $< -o $@
> + $(CXX) -c $(CXXFLAGS) $< -o $@

Thanks, Jed!

It was nice to get the patch, (which is pushed now).

But it was even nicer to get the encouragement to display text
attachments *and* to include quoted text attachments in replies. I've
often been frustrated by email programs that wouldn't quote patch
attachments in replies for sake of review. So now at least my email
program no longer has that bug.

-Carl

PS. GPG signatures are no longer silently dropped when presenting
messages. We don't yet present the full signature yet, (and in
particular do not yet verify it), but you can at least now see whether a
signature is present.


[notmuch] [PATCH] Support for printing file paths in new command

2009-11-21 Thread Adrian Perez
For very large mail boxes, it is desirable to know which files are being
processed e.g. when a crash occurs to know which one was the cause. Also,
it may be interesting to have a better idea of how the operation is
progressing when processing mailboxes with big messages.

This patch adds support for printing messages as they are processed by
"notmuch new":

* The "new" command now supports a "--verbose" flag.

* When running in verbose mode, the file path of the message about to be
  processed is printed in the following format:

current/total: /path/to/message/file

  Where "current" is the number of messages processed so far and "total" is
  the total count of files to be processed.

  The status line is erased using an ANSI sequence "\033[K" (erase current
  line from the cursor to the end of line) each time it is refreshed. This
  should not pose a problem because nearly every terminal supports it.

* The signal handler for SIGALRM and the timer are not enabled when running
  in verbose mode, because we are already printing progress with each file,
  periodical reports are not neccessary.
---
 notmuch-client.h |1 +
 notmuch-new.c|   73 +++--
 notmuch.c|   11 ++-
 3 files changed, 58 insertions(+), 27 deletions(-)

diff --git a/notmuch-client.h b/notmuch-client.h
index b65aa77..6908844 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -74,6 +74,7 @@ typedef void (*add_files_callback_t) (notmuch_message_t 
*message);
 typedef struct {
 int ignore_read_only_directories;
 int saw_read_only_directory;
+int verbose;

 int total_files;
 int processed_files;
diff --git a/notmuch-new.c b/notmuch-new.c
index 1b05584..6b4b83c 100644
--- a/notmuch-new.c
+++ b/notmuch-new.c
@@ -177,6 +177,14 @@ add_files_recursive (notmuch_database_t *notmuch,
if (path_dbtime == 0 || st->st_mtime > path_dbtime) {
state->processed_files++;

+   if (state->verbose) {
+   printf ("\r\033[K%i/%i: %s\r",
+   state->processed_files,
+   state->total_files,
+   next);
+   fflush (stdout);
+   }
+
status = notmuch_database_add_message (notmuch, next, &message);
switch (status) {
/* success */
@@ -270,32 +278,36 @@ add_files (notmuch_database_t *notmuch,
return NOTMUCH_STATUS_FILE_ERROR;
 }

-/* Setup our handler for SIGALRM */
-memset (&action, 0, sizeof (struct sigaction));
-action.sa_handler = handle_sigalrm;
-sigemptyset (&action.sa_mask);
-action.sa_flags = SA_RESTART;
-sigaction (SIGALRM, &action, NULL);
-
-/* Then start a timer to send SIGALRM once per second. */
-timerval.it_interval.tv_sec = 1;
-timerval.it_interval.tv_usec = 0;
-timerval.it_value.tv_sec = 1;
-timerval.it_value.tv_usec = 0;
-setitimer (ITIMER_REAL, &timerval, NULL);
+if (!state->verbose) {
+   /* Setup our handler for SIGALRM */
+   memset (&action, 0, sizeof (struct sigaction));
+   action.sa_handler = handle_sigalrm;
+   sigemptyset (&action.sa_mask);
+   action.sa_flags = SA_RESTART;
+   sigaction (SIGALRM, &action, NULL);
+
+   /* Then start a timer to send SIGALRM once per second. */
+   timerval.it_interval.tv_sec = 1;
+   timerval.it_interval.tv_usec = 0;
+   timerval.it_value.tv_sec = 1;
+   timerval.it_value.tv_usec = 0;
+   setitimer (ITIMER_REAL, &timerval, NULL);
+}

 status = add_files_recursive (notmuch, path, &st, state);

-/* Now stop the timer. */
-timerval.it_interval.tv_sec = 0;
-timerval.it_interval.tv_usec = 0;
-timerval.it_value.tv_sec = 0;
-timerval.it_value.tv_usec = 0;
-setitimer (ITIMER_REAL, &timerval, NULL);
-
-/* And disable the signal handler. */
-action.sa_handler = SIG_IGN;
-sigaction (SIGALRM, &action, NULL);
+if (!state->verbose) {
+   /* Now stop the timer. */
+   timerval.it_interval.tv_sec = 0;
+   timerval.it_interval.tv_usec = 0;
+   timerval.it_value.tv_sec = 0;
+   timerval.it_value.tv_usec = 0;
+   setitimer (ITIMER_REAL, &timerval, NULL);
+
+   /* And disable the signal handler. */
+   action.sa_handler = SIG_IGN;
+   sigaction (SIGALRM, &action, NULL);
+}

 return status;
 }
@@ -370,8 +382,7 @@ count_files (const char *path, int *count)
 }

 int
-notmuch_new_command (void *ctx,
-unused (int argc), unused (char *argv[]))
+notmuch_new_command (void *ctx, int argc, char *argv[])
 {
 notmuch_config_t *config;
 notmuch_database_t *notmuch;
@@ -383,6 +394,18 @@ notmuch_new_command (void *ctx,
 const char *db_path;
 char *dot_notmuch_path;
 struct sigaction action;
+int i;
+
+add_files_state.verbose = 0;
+
+for (i = 0; i < argc && argv[i][0] == '-'; i++) {
+   if (STRNCMP_L

[notmuch] Recommended Coding Style?

2009-11-21 Thread Carl Worth
On Fri, 20 Nov 2009 15:15:16 +, Chris Wilson  
wrote:
> Excerpts from Jan Janak's message of Fri Nov 20 15:08:31 + 2009:
> > Is there any recommended coding style for the C/C++ code in notmuch?
> 
> My prediction is that Carl will go with something close to what he wrote
> for Cairo:
> 
>   http://cgit.freedesktop.org/cairo/tree/CODING_STYLE
> 
> He may or may not change to a more kernel style, but he definitely will
> not mandate 3-space indents. ;-)

I think Chris is right on this.

I'll adapt that CODING_STYLE document at some point and bring it into
the repository. And I probably will convert the indentation from
4-column indents (with mixed spaces and tabs) to 8-column indents (with
tabs only).

One thing that people might find a bit different are that I like to put
a space between a function name and the left parenthesis introducing
it's arguments.

That is, just as we have a space here:

if (condition) {
...
}

I like one here as well:

foo = some_function (value);

Most of the patches coming in have been fine so far. (Some have used
tabs only for indentation and I've just silently cleaned those up for
now, since I'm considering switching to that style anyway at some
point.)

One style point I would like to insist on is that commit messages should
always have more than the first, single-line summary.

The single line summary is good about saying *what* the commit does, but
I always want to see at least one sentence about the *why* as well.

I'll likely start bouncing patches that don't have that.

Thanks for the question, Jan.

-Carl




[notmuch] [PATCH] When a search query triggers a Xapian exception, log what the query was.

2009-11-21 Thread Carl Worth
On Fri, 20 Nov 2009 14:00:42 +0100, Eric Anholt  wrote:
> In my script containing a series of queries to be run on new mail for
> setting up tags, it's nice to see which query I typed wrong.

Good point. Pushed.

Thanks,

-Carl


[notmuch] Gentoo ebuild for notmuch

2009-11-21 Thread Carl Worth
On Fri, 20 Nov 2009 16:09:54 +, James Rowe  wrote:
>   The patch that follows makes it easier to use the install target
> within the ebuild. If this sort of change is acceptable then the ebuild
> will be simpler, and it is more likely to stay working without me
> kicking heaps of patches at you. Given that I'd say "yes, please include
> it."

That change looks just fine to me. Pushed now.

Feel free to send me more if you'd like it integrated to our repository.

-Carl