[notmuch] Introducing notmuchsync

2010-01-19 Thread Sebastian Spaeth
On Tue, 19 Jan 2010 11:04:49 -0500, Jameson Rollins  wrote:
> I should have mentioned in my previous mail that I think this tool is
> a great idea, and I plan on using it.  I just hope that all of it's
> functionality will be integrated directly into notmuch itself.

I fully agree. 
[Snip more stuff I fully agree with]
[snip workflow]

> Do you do another notmuchsync after the final notmuch new, to get any
> new flags in the maildir synced with the database?

Ohh, I had never considered that, but true, if another mail client
modifies maildir flags, we probably want to have that reflected as notmuch
tags as well...

So, yes that should work even better (my real script is a bit fancier,
changing output options ans stuff):

notmuch new   # make db consistent (earlier deleted mails etc)
notmuchsync -s -n # MailDir flags update and "cur" dir moving
offlineimap   # sync with IMAP server
notmuch new   # incorporate new mails in notmuch db
notmuchsync -r -n # reflect changed MailDir flags in notmuch tags

Sorry for all the spam today about my toys. I'll try to keep a bit
more quiet in the future in order to not start annoying you. :-)

spaetz


[notmuch] Introducing notmuchsync

2010-01-19 Thread Sebastian Spaeth
On Tue, 19 Jan 2010 10:24:27 -0500, Jameson Rollins  wrote:
> Actually, I don't think this is exactly correct.  I believe that the
> move from new to cur should be done by the MUA when the MUA actually
> processes the message, but before the user views the message.  The S
> flag should then be added by the MUA when the user actually views the
> message.  The MTA should be delivering messages to new, and the MUA
> should move it to cur.  That is at least how's it's described by DJB
> for qmail (see THE MAILDIR STRUCTURE section in the qmail man page):

I have done it now as suggested by Marten: move the mail to "cur" when the
user has actually seen it (as per lack of the 'unread' tag). While I
agree that it probably should move it earlier (when it is "processed"
whatever that means for notmuch), this was just the most convenient for
me. In addition, I have also not found a maildir specification, so the
behavior seems to be a bit undefined.

> Currently notmuch itself does not really conform to what I believe is
> the maildir spec, which makes it a little difficult to use with other
> MUAs.  Notmuch does not move messages from new to cur, or modify the
> flags.  

That is why I have coded my notmuchsync tool. It does all that. While I
agree that notmuch should probably (and faster) do all that itself, the
current design seems to be to keep notmuch flexible, small and to never touch
your mailstore. Until that changes, surrounding scripts will have to
perform these tasks.

My current synchronization script looks basically like this:

notmuch new   # make db consistent (earlier deleted mails etc)
notmuchsync -s -n # MailDir flags update and "cur" dir moving
offlineimap   # sync with IMAP server
notmuch new   # incorporate new mails in notmuch db

spaetz


[notmuch] Introducing notmuchsync

2010-01-19 Thread Sebastian Spaeth
On Tue, 19 Jan 2010 16:00:06 +0100, Marten Veldthuis  
wrote:
> On Tue, 19 Jan 2010 14:37:03 +0100, "Sebastian Spaeth"  SSpaeth.de> wrote:
> >  - Move read files from 'new' to 'cur' folder. At what point is that
> >moving typically done in Maildir? When the user has actually looked
> >at the mail?
> 
> Yes, exactly that.

Thanks Marten. Implemented moving from "new" to "cur" now.

spaetz


[notmuch] Introducing notmuchsync

2010-01-19 Thread Marten Veldthuis
On Tue, 19 Jan 2010 14:37:03 +0100, "Sebastian Spaeth"  wrote:
>  - Move read files from 'new' to 'cur' folder. At what point is that
>moving typically done in Maildir? When the user has actually looked
>at the mail?

Yes, exactly that.

-- 
- Marten


[notmuch] Introducing notmuchsync

2010-01-19 Thread Sebastian Spaeth

> What needs improvment?
>  - Support for the "T" (delete), "F" (flagged), "D" (draft) tags/flags.

Having just finished this feature, I herewith declare notmuchsync to be version
0.0.1 :). It works for fine for me, I use it to sync the "FTDS" Maildir flags
(corresponding notmuch flags: 'unread','flagged','draft','delete'), as
well as for periodically pruning mail files that are marked as 'delete'.

Let me know if you have issues with it.

The 2 most urgent issues are: 
 - Move read files from 'new' to 'cur' folder. At what point is that
   moving typically done in Maildir? When the user has actually looked
   at the mail?
 - chunked output parsing (9k messages went through fine on my laptop,
   but I doubt that 200k msg would)

Sebastian


[notmuch] Introducing notmuchsync

2010-01-19 Thread Jameson Rollins
On Tue, Jan 19, 2010 at 11:29:10AM -0500, Servilio Afre Puentes wrote:
> For what I've read so far from messages of the authors in this mailing
> list and the programmed behaviour, I think that is the intention
> (though not fully implemented yet) and in my opinion the right one.
> Maybe if the notmuch.el code would have been written as a set of hooks
> into Gnus for customizing views this would have been more evident
> (though I haven't neither used or looked at the vim front-end code),
> or maybe I am missing the point entirely and the long-term goal of the
> authors is to have a full-fledged MUA.

I don't think that the intention is to be a full-fledged MUA, but I'm
not sure it should just be an indexer either.  I think notmuch should
make itself basically a MUA minus mail reader, so that custom mail
readers can be built on top of it.  But like you, this is mostly just
my opinion of what notmuch should be doing.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100119/e06a9dd8/attachment.pgp>


[notmuch] Introducing notmuchsync

2010-01-19 Thread Servilio Afre Puentes
2010/1/19 Jameson Rollins :
[...]
> That said, I have vasilated just a bit on this, as to whether notmuch
> should touch the mail at all, or just process it. ?But having thought
> about it a bit, I think that notmuch really *is* an MUA, or at least
> the mail processing part of a MUA (MUA minus message reader), and
> should therefore do the appropriate things with the maildir.

I on the other hand think that it is just the indexer, and that the
right way to integrate it into a MUA is by making this MUA call
notmuch to update the DB/index with the actions just performed on a
message/group of messages (moving, deleting, tagging, etc.).

For what I've read so far from messages of the authors in this mailing
list and the programmed behaviour, I think that is the intention
(though not fully implemented yet) and in my opinion the right one.
Maybe if the notmuch.el code would have been written as a set of hooks
into Gnus for customizing views this would have been more evident
(though I haven't neither used or looked at the vim front-end code),
or maybe I am missing the point entirely and the long-term goal of the
authors is to have a full-fledged MUA.

Regards,

Servilio


[notmuch] Introducing notmuchsync

2010-01-19 Thread Jameson Rollins
On Tue, Jan 19, 2010 at 04:52:42PM +0100, Sebastian Spaeth wrote:
> That is why I have coded my notmuchsync tool. It does all that. While I
> agree that notmuch should probably (and faster) do all that itself, the
> current design seems to be to keep notmuch flexible, small and to never touch
> your mailstore. Until that changes, surrounding scripts will have to
> perform these tasks.

I should have mentioned in my previous mail that I think this tool is
a great idea, and I plan on using it.  I just hope that all of it's
functionality will be integrated directly into notmuch itself.

That said, I have vasilated just a bit on this, as to whether notmuch
should touch the mail at all, or just process it.  But having thought
about it a bit, I think that notmuch really *is* an MUA, or at least
the mail processing part of a MUA (MUA minus message reader), and
should therefore do the appropriate things with the maildir.

> My current synchronization script looks basically like this:
> 
> notmuch new   # make db consistent (earlier deleted mails etc)
> notmuchsync -s -n # MailDir flags update and "cur" dir moving
> offlineimap   # sync with IMAP server
> notmuch new   # incorporate new mails in notmuch db

Do you do another notmuchsync after the final notmuch new, to get any
new flags in the maildir synced with the database?

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100119/4e1b6b5a/attachment.pgp>


[notmuch] Introducing notmuchsync

2010-01-19 Thread Jameson Rollins
On Tue, Jan 19, 2010 at 04:00:06PM +0100, Marten Veldthuis wrote:
> On Tue, 19 Jan 2010 14:37:03 +0100, "Sebastian Spaeth"  SSpaeth.de> wrote:
> >  - Move read files from 'new' to 'cur' folder. At what point is that
> >moving typically done in Maildir? When the user has actually looked
> >at the mail?
> 
> Yes, exactly that.

Actually, I don't think this is exactly correct.  I believe that the
move from new to cur should be done by the MUA when the MUA actually
processes the message, but before the user views the message.  The S
flag should then be added by the MUA when the user actually views the
message.  The MTA should be delivering messages to new, and the MUA
should move it to cur.  That is at least how's it's described by DJB
for qmail (see THE MAILDIR STRUCTURE section in the qmail man page):

http://qmail.org/man/man5/maildir.html

I can't find an official "spec", though, and the cr.yp.to maildir page
is a little vague on the issue:

http://cr.yp.to/proto/maildir.html

Currently notmuch itself does not really conform to what I believe is
the maildir spec, which makes it a little difficult to use with other
MUAs.  Notmuch does not move messages from new to cur, or modify the
flags.  I've been arguing that notmuch should conform to this spec
directly, by moving messages from new to cur when it processes the
maildir, and by keeping tags synced with the maildir message flags.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100119/20e4a28e/attachment.pgp>


[notmuch] xapian errors and notmuch return codes

2010-01-19 Thread Sebastian Spaeth
When I perform a "notmuch search" while notmuch is updating tags in a
different shell, I get:
"A Xapian exception occurred performing query: The revision being read
has been discarded - you should call Xapian::Database::reopen() and
retry the operation"

but the notmuch return code is still "0" (indicating success?!).

It would be great to have a return code reflect the validity of returned
results, especially when the main means of interacting are piping from
the notmuch app at the moment.

Also, could the "man" page indicate which return codes notmuch delivers
in which cases? I would find that pretty helpful.

Sorry for not delivering patches, but I think there was a patch for
return codes on the list recently.


Sebastian


[notmuch] xapian errors and notmuch return codes

2010-01-19 Thread Sebastian Spaeth
When I perform a notmuch search while notmuch is updating tags in a
different shell, I get:
A Xapian exception occurred performing query: The revision being read
has been discarded - you should call Xapian::Database::reopen() and
retry the operation

but the notmuch return code is still 0 (indicating success?!).

It would be great to have a return code reflect the validity of returned
results, especially when the main means of interacting are piping from
the notmuch app at the moment.

Also, could the man page indicate which return codes notmuch delivers
in which cases? I would find that pretty helpful.

Sorry for not delivering patches, but I think there was a patch for
return codes on the list recently.


Sebastian
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Introducing notmuchsync

2010-01-19 Thread Jameson Rollins
On Tue, Jan 19, 2010 at 04:00:06PM +0100, Marten Veldthuis wrote:
 On Tue, 19 Jan 2010 14:37:03 +0100, Sebastian Spaeth sebast...@sspaeth.de 
 wrote:
   - Move read files from 'new' to 'cur' folder. At what point is that
 moving typically done in Maildir? When the user has actually looked
 at the mail?
 
 Yes, exactly that.

Actually, I don't think this is exactly correct.  I believe that the
move from new to cur should be done by the MUA when the MUA actually
processes the message, but before the user views the message.  The S
flag should then be added by the MUA when the user actually views the
message.  The MTA should be delivering messages to new, and the MUA
should move it to cur.  That is at least how's it's described by DJB
for qmail (see THE MAILDIR STRUCTURE section in the qmail man page):

http://qmail.org/man/man5/maildir.html

I can't find an official spec, though, and the cr.yp.to maildir page
is a little vague on the issue:

http://cr.yp.to/proto/maildir.html

Currently notmuch itself does not really conform to what I believe is
the maildir spec, which makes it a little difficult to use with other
MUAs.  Notmuch does not move messages from new to cur, or modify the
flags.  I've been arguing that notmuch should conform to this spec
directly, by moving messages from new to cur when it processes the
maildir, and by keeping tags synced with the maildir message flags.

jamie.


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Introducing notmuchsync

2010-01-19 Thread Sebastian Spaeth
On Tue, 19 Jan 2010 16:00:06 +0100, Marten Veldthuis mar...@veldthuis.com 
wrote:
 On Tue, 19 Jan 2010 14:37:03 +0100, Sebastian Spaeth sebast...@sspaeth.de 
 wrote:
   - Move read files from 'new' to 'cur' folder. At what point is that
 moving typically done in Maildir? When the user has actually looked
 at the mail?
 
 Yes, exactly that.

Thanks Marten. Implemented moving from new to cur now.

spaetz
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Introducing notmuchsync

2010-01-19 Thread Jameson Rollins
On Tue, Jan 19, 2010 at 04:52:42PM +0100, Sebastian Spaeth wrote:
 That is why I have coded my notmuchsync tool. It does all that. While I
 agree that notmuch should probably (and faster) do all that itself, the
 current design seems to be to keep notmuch flexible, small and to never touch
 your mailstore. Until that changes, surrounding scripts will have to
 perform these tasks.

I should have mentioned in my previous mail that I think this tool is
a great idea, and I plan on using it.  I just hope that all of it's
functionality will be integrated directly into notmuch itself.

That said, I have vasilated just a bit on this, as to whether notmuch
should touch the mail at all, or just process it.  But having thought
about it a bit, I think that notmuch really *is* an MUA, or at least
the mail processing part of a MUA (MUA minus message reader), and
should therefore do the appropriate things with the maildir.

 My current synchronization script looks basically like this:
 
 notmuch new   # make db consistent (earlier deleted mails etc)
 notmuchsync -s -n # MailDir flags update and cur dir moving
 offlineimap   # sync with IMAP server
 notmuch new   # incorporate new mails in notmuch db

Do you do another notmuchsync after the final notmuch new, to get any
new flags in the maildir synced with the database?

jamie.


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Introducing notmuchsync

2010-01-19 Thread Sebastian Spaeth
On Tue, 19 Jan 2010 11:04:49 -0500, Jameson Rollins 
jroll...@finestructure.net wrote:
 I should have mentioned in my previous mail that I think this tool is
 a great idea, and I plan on using it.  I just hope that all of it's
 functionality will be integrated directly into notmuch itself.

I fully agree. 
[Snip more stuff I fully agree with]
[snip workflow]

 Do you do another notmuchsync after the final notmuch new, to get any
 new flags in the maildir synced with the database?

Ohh, I had never considered that, but true, if another mail client
modifies maildir flags, we probably want to have that reflected as notmuch
tags as well...

So, yes that should work even better (my real script is a bit fancier,
changing output options ans stuff):

notmuch new   # make db consistent (earlier deleted mails etc)
notmuchsync -s -n # MailDir flags update and cur dir moving
offlineimap   # sync with IMAP server
notmuch new   # incorporate new mails in notmuch db
notmuchsync -r -n # reflect changed MailDir flags in notmuch tags

Sorry for all the spam today about my toys. I'll try to keep a bit
more quiet in the future in order to not start annoying you. :-)

spaetz
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Introducing notmuchsync

2010-01-19 Thread Jameson Rollins
On Tue, Jan 19, 2010 at 11:29:10AM -0500, Servilio Afre Puentes wrote:
 For what I've read so far from messages of the authors in this mailing
 list and the programmed behaviour, I think that is the intention
 (though not fully implemented yet) and in my opinion the right one.
 Maybe if the notmuch.el code would have been written as a set of hooks
 into Gnus for customizing views this would have been more evident
 (though I haven't neither used or looked at the vim front-end code),
 or maybe I am missing the point entirely and the long-term goal of the
 authors is to have a full-fledged MUA.

I don't think that the intention is to be a full-fledged MUA, but I'm
not sure it should just be an indexer either.  I think notmuch should
make itself basically a MUA minus mail reader, so that custom mail
readers can be built on top of it.  But like you, this is mostly just
my opinion of what notmuch should be doing.

jamie.


signature.asc
Description: Digital signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] [PATCH] Simplify unread tag handling in emacs UI.

2010-01-19 Thread Jameson Rollins
This patch is intended to greatly simplify the handling of the
unread tag in the emacs UI.  This patch adds a new function
'notmuch-show-mark-read', that removes the unread tag in
notmuch-show-mode.  This function is then executed as a
notmuch-show-hook, and by notmuch-show-next-message.  All of the
functions that explicitly marked messages as unread are removed or
renamed.

The idea here is that the user should never have to worry about
manipulating the unread tag.  The tag should persist until the user
actually views a message, at which point it should be automatically
removed.  This patch simplifies the notmuch.el quite a bit, removing a
lot of somewhat redundant functions, and reducing clutter in the name
and key-mapping space.
---
 notmuch.el |   95 +++
 1 files changed, 31 insertions(+), 64 deletions(-)

diff --git a/notmuch.el b/notmuch.el
index 97914f2..2da3c06 100644
--- a/notmuch.el
+++ b/notmuch.el
@@ -69,15 +69,12 @@
 (define-key map v 'notmuch-show-view-all-mime-parts)
 (define-key map - 'notmuch-show-remove-tag)
 (define-key map + 'notmuch-show-add-tag)
-(define-key map X 'notmuch-show-mark-read-then-archive-then-exit)
 (define-key map x 'notmuch-show-archive-thread-then-exit)
-(define-key map A 'notmuch-show-mark-read-then-archive-thread)
 (define-key map a 'notmuch-show-archive-thread)
 (define-key map p 'notmuch-show-previous-message)
-(define-key map N 'notmuch-show-mark-read-then-next-open-message)
 (define-key map n 'notmuch-show-next-message)
 (define-key map (kbd DEL) 'notmuch-show-rewind)
-(define-key map   'notmuch-show-advance-marking-read-and-archiving)
+(define-key map   'notmuch-show-advance-and-archive)
 map)
   Keymap for \notmuch show\ buffers.)
 (fset 'notmuch-show-mode-map notmuch-show-mode-map)
@@ -226,13 +223,22 @@ Unlike builtin `previous-line' this version accepts no 
arguments.
 (cons (notmuch-show-get-message-id) nil)))
  (notmuch-show-set-tags (sort (set-difference tags toremove :test 
'string=) 'string))
 
-(defun notmuch-show-archive-thread-maybe-mark-read (markread)
+(defun notmuch-show-archive-thread ()
+  Archive each message in thread, then show next thread from search.
+
+Archive each message currently shown by removing the \inbox\
+tag from each. Then kill this buffer and show the next thread
+from the search from which this thread was originally shown.
+
+Note: This command is safe from any race condition of new messages
+being delivered to the same thread. It does not archive the
+entire thread, but only the messages shown in the current
+buffer.
+  (interactive)
   (save-excursion
 (goto-char (point-min))
 (while (not (eobp))
-  (if markread
- (notmuch-show-remove-tag unread inbox)
-   (notmuch-show-remove-tag inbox))
+  (notmuch-show-remove-tag inbox)
   (if (not (eobp))
  (forward-char))
   (if (not (re-search-forward notmuch-show-message-begin-regexp nil t))
@@ -245,47 +251,12 @@ Unlike builtin `previous-line' this version accepts no 
arguments.
  (forward-line)
  (notmuch-search-show-thread)
 
-(defun notmuch-show-mark-read-then-archive-thread ()
-  Remove unread tags from thread, then archive and show next thread.
-
-Archive each message currently shown by removing the \unread\
-and \inbox\ tag from each. Then kill this buffer and show the
-next thread from the search from which this thread was originally
-shown.
-
-Note: This command is safe from any race condition of new messages
-being delivered to the same thread. It does not archive the
-entire thread, but only the messages shown in the current
-buffer.
-  (interactive)
-  (notmuch-show-archive-thread-maybe-mark-read t))
-
-(defun notmuch-show-archive-thread ()
-  Archive each message in thread, then show next thread from search.
-
-Archive each message currently shown by removing the \inbox\
-tag from each. Then kill this buffer and show the next thread
-from the search from which this thread was originally shown.
-
-Note: This command is safe from any race condition of new messages
-being delivered to the same thread. It does not archive the
-entire thread, but only the messages shown in the current
-buffer.
-  (interactive)
-  (notmuch-show-archive-thread-maybe-mark-read nil))
-
 (defun notmuch-show-archive-thread-then-exit ()
   Archive each message in thread, then exit back to search results.
   (interactive)
   (notmuch-show-archive-thread)
   (kill-this-buffer))
 
-(defun notmuch-show-mark-read-then-archive-then-exit ()
-  Remove unread tags from thread, then archive and exit to search results.
-  (interactive)
-  (notmuch-show-mark-read-then-archive-thread)
-  (kill-this-buffer))
-
 (defun notmuch-show-view-raw-message ()
   View the raw email of the current message.
   (interactive)
@@ -440,7 +411,8 @@ Returns nil if already on the last message in the buffer.
 (while 

[notmuch] Joining

2010-01-19 Thread Servilio Afre Puentes
Maybe a bit late for saying Hi, glad to join! since I have already
sent another message, but yesterday was a long day at work.

I also knew about notmuch from the LWN article back in December and
have been following it through the git update and the mail archive
since then (though not reading all messages neither all the code).

I am also very glad of something like this having been created and
look forward to help enhance it and use it more extensively (right now
I am trying to get the Gnus bit in place).

Regards,

Servilio
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch