[notmuch] Some thoughts about notmuch sync with other agents

2010-02-01 Thread martin f krafft
also sprach Paul R  [2010.01.28.0316 +1300]:
> As you see, I advocate a NotMuch <-> IMAP synchronisation ASAP :)

Given the limitations of IMAP (non-transactional, non-standard
keywords implementation, ?), I think chances of this are rapidly
diminishing, but I'd love to be proven wrong.

> First of all, processing mail with MUA involves some simple logic that
> is shared by most MUA. This is about receiving *new* mails, *reading*
> mail, *replying* to mail and so on... IMHO, this really belongs to the
> mail processing logic and not to the user logic. Hence my first
> request :
> 
>   1: Don't use user tags space to store MUA flags.
> 
>  That means no more "seen", "unread", "replied" as tags. These are
>  MUA processing *flags*, that must belong to an established set.
>  Tags, on the other hand, are user-land information. So no more
>  [seen, replied, grandma, important] tag sets :)

I disagree.

The MUA actually doesn't (shouldn't) care at all about any of these
flags, at least not for core functionality. Sure, a MUA should
probably hide messages tagged 'deleted', and it would be nice if it
could be configured to highlight messages tagged 'important', but
none of the others ? "seen", "unread", "replied", ? ? have any role
in the core functionality. They are purely user-specific.

They are leftovers from days when some MUA designer decided that
having these flags would be a useful way to organise e-mail
handling, but that person probably dealt with a dozen messages
a day, and didn't have half his/her life organised through
electronic mail.

Point being, times and needs have changed. And while we're in the
process of finding the technology that can suit those needs (in the
most generic way), we might just as well (and should) rid ourselves
from these leftovers.

Any solution must be generic enough so that if you rely on the
functionality provided by these tags, you can trivially make it
happen, e.g. with hooks to add/remove flags on certain actions (such
as sending a reply, or reading a message).

Neither IMAP nor Maildir is capable of storing an extensible,
freely-configurable set of tags (keywords). Therefore, we need a new
method anyway.

> Once this is done, notmuch will know, in a robust a predictable
> way, what happened to a mail. Simply put, NotMuch will store and
> expose MUA flags (Passed, Replied, Seen, Trashed, Draft, and
> Flagged [1]). For each , notmuch should associate
> a _synced flag. When changing  from notmuch, it should
> set the _synced bit to 0. These are MUA mail flags.

I think the semantics would be clearer the other way around: setting
*_changed when a flag is changed.

> Additionally, in a third ? space ?, notmuch should store its ? new ?
> bit, as well as a ? missing ? bit probably. Again, this is neither MUA
> logic or user logic, so this should not interfer with user
> classification facility provided by tags, nor with MUA flags. It,
> really, is something else, let's name it "status".

Or "lifetime".

> Once this is done, the 'notmuch new' command should find new mails
> and set the 'new' bit for them, and find the missing mails and set
> the 'missing' bit for them. This will allow for robust external
> processing.

Why would I want to keep around a record in the database when the
physical file is no longer present?

> # 1/ Sync from NotMuch to MailDir
> 
> notmuch list flags:(seen and not seen_synced) 
>   | notmuch-maildir --mark-mail seen
>   | notmuch move --stdin
>   | notmuch set flags:+seen_synced --stdin

Have you seen/look at notmuchsync?

> The output of the first command would be a list of filenames for mails
> 'seen' since last sync. The second one, an external notmuch--maildir
> helper, would propagate this logic to the MailDir store (easy, this is
> simply a rename), and output the list of (old-name ! new-name). Then
> notmuch would use this information, via a generic 'move' switch, to know
> that mail has been moved, and would output the list of the new places.
> Finaly, notmuch would set the seen_synced flag.

What happens if notmuch-move gets killed due to out-of-memory half
way through?

> As we have seen, this would allow most IMAP <-> MailDir <-> NotMuch
> synchronisation, without having to implement any kind of
> MailDir-specific logic inside notmuch.

It would not take care of any tags beyond the very strictly defined
and limited set of tags you listed in your mail. My point is that we
need to solve this problem more generally anyway ? why should
a "replied" tag be synchronised, but a "no-need-to-reply" tag not?

-- 
martin | http://madduck.net/ | http://two.sentenc.es/

#include 

spamtraps: madduck.bogus at madduck.net
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature (see http://martin-krafft.net/gpg/)
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100201/8ffddb4f/attachment.pgp>


[notmuch] Emacs paned UI

2010-02-01 Thread Tad Fisher

Hello everyone!

I've been using notmuch and following its development for a little while
now, and I've come to like it very much. One thing that has been lacking
is an easy-to-use Emacs interface, and I believe Keith and Carl had a
little discussion a while back about how we should go about making a
paned UI.

So, I dusted off my Elisp manual and hacked up notmuch.el to support a
3-paned UI. This involved the creation of a minor mode (I call it
notmuch-browse-mode) that manages window state, along with several
modifications of existing functions to "do the right thing" when we're
in browse-mode (such as close the "show" window, switch to the "folder"
window, etc.).

The UI is thus far usable, but not bug-free. For example,
advancing-and-archiving using SPC sometimes "advances" to the same
message (but I am always able to advance with "a"). Also, the search
window does not update after archiving, but I believe this is an
existing bug.

The code is in the "emacs-ui" branch of the following repo:

git://github.com/tadfisher/notmuch-tad.git

Suggestions, criticism, and a code review are both needed and desired.

-- 
Tad Fisher
tadfisher at gmail.com

sent via something other than notmuch: http://notmuchmail.org/


[notmuch] Some thoughts about notmuch sync with other agents

2010-02-01 Thread Sebastian Spaeth
REPOSTING, as the previous reply went to Gmane

> One may
> or may not like IMAP for good reasons, the fact is that it is here and
> has allowed users to read mails from various places and terminals,
> keeping important information synced. So I think that notmuch will have
> to live with that, and provide what is required to integrate smoothly
> into this environment.

agreed

> First of all, processing mail with MUA involves some simple logic that
> is shared by most MUA. This is about receiving *new* mails, *reading*
> mail, *replying* to mail and so on... IMHO, this really belongs to the
> mail processing logic and not to the user logic. Hence my first
> request :
>   1: Don't use user tags space to store MUA flags.

I don't really see what you gain by this proposal. I don't necessarily
think it's bad either, but there is no advantage IMHO.

What you propose is basically that those tags are not directly seen by
the user and not modifiable by the user. What's the advantage over
having a small set of "reserved tag keywords"? Your notmuch-capable
editor could still chose to not display "unread" and "deleted" tags but
simply hide deleted mails and show unread mails in bold or so. I simply
don't think it's important how tags are stored and the current scheme is
dead simple which makes it pretty nice.

>  That means no more "seen", "unread", "replied" as tags. These are
>  MUA processing *flags*, that must belong to an established set.
>  Tags, on the other hand, are user-land information. So no more
>  [seen, replied, grandma, important] tag sets :)

See above, the mail editor might chose to hide thos MUA processing
flags, but I don't think it is important whether they are hidden by the
user or now. 

Other than that, I agree that "notmuch new" should set a "new" and/or
"modified"tag if it detects a new mail (or a modified one), rather than
the current set of fixed tags that aren't too helpful for 3rd party
scripts.

Sebastian 
-- next part --
Date: Mon, 01 Feb 2010 15:54:57 +0100
Message-ID: <871vh557lq.fsf at SSpaeth.de>


[notmuch] Request for high-priority improvements to notmuch

2010-02-01 Thread Scott Robinson
Excerpts from sebastian's message of Mon Feb 01 14:29:44 -0600 2010:
> 2)
> > JSON output for "notmuch search/show" with ability to filter output fields
> > "search" --> "search --output=thread_id,date,number,author,subject,tags"
> >  "show"  --> "search
> > --output=message_id,tags,path,header,body,attachments"
> 
> YES PLEASE :-). notmuch seems designed to work in an ecosystem of
> surrounding scripts, feeding data in and out. But we are all currently
> limited to regexes for that. And heck, I hard a hard time understanding
> why all hell broke out until I found that i had added a tag containing
> parentheses which made my regex fail. :-). XML, JSON, any structured
> output would be nice.
> 
> And as for filtering: YES, PLEASE :-). notmuchsync and many other 3rd
> party apps would love that. As father of notmuchsync, I can tell you my
> little script hickups very badly when slurping in 200k mails (including
> text bodies) just to find out the maildir tags of those mails.
> 

There's been a JSON patch waiting for a month now.


[notmuch] Request for high-priority improvements to notmuch

2010-02-01 Thread Jameson Rollins
nd to seeing them in near-future releases.

jamie.
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20100201/7ba85655/attachment.pgp>


[notmuch] Request for high-priority improvements to notmuch

2010-02-01 Thread sebast...@sspaeth.de
Let me second those thoughts. I'll even put forward 2 more pet suggestions
:).

1)
> Ability to apply tags based on folder paths in "notmuch new"

This will let me sync tags with notmuchsync much more easily than going
through all mails and detecting their IMAP path from the filename. Support
for searching/tagging mails in certain folders would be really nice. I
would appreciate but do not urgently need a [tags] section in notmuch
proper, as I'll extend notmuchsync to do that for me.

2)
> JSON output for "notmuch search/show" with ability to filter output fields
> "search" --> "search --output=thread_id,date,number,author,subject,tags"
>  "show"  --> "search
> --output=message_id,tags,path,header,body,attachments"

YES PLEASE :-). notmuch seems designed to work in an ecosystem of
surrounding scripts, feeding data in and out. But we are all currently
limited to regexes for that. And heck, I hard a hard time understanding
why all hell broke out until I found that i had added a tag containing
parentheses which made my regex fail. :-). XML, JSON, any structured
output would be nice.

And as for filtering: YES, PLEASE :-). notmuchsync and many other 3rd
party apps would love that. As father of notmuchsync, I can tell you my
little script hickups very badly when slurping in 200k mails (including
text bodies) just to find out the maildir tags of those mails.

> * Proper maildir sync ("search --output=message_id,tags,path" ...)
[snip many sensible proposals]

Sebastian

P.S. as a bonus: My very ugly way of getting a distribution of existing
tags (extra points for beautified versions):

notmuch dump|sed -e 's/^.*(//'|sed -e 's/)$//'|sed 's/ /\n/g'|sort|uniq
-c|sort -r




[notmuch] patchwork test instance (was: Git feature branch)

2010-02-01 Thread martin f krafft
I investigated some patch/issue trackers over the weekend. Here's my
summary/reply.

The executive summary is that
http://patchwork.madduck.net/project/notmuch/list/ now exists.
I have not really used it for anything real, so if some of you feel
inclined to give it a shot, sign up and triage away! Feedback
welcome.

also sprach James Rowe jnr...@gmail.com [2010.01.28.2005 +1300]:
   Roundup has command line and email interfaces.  The email interface is
 quite similar to debian's.  I've never used a launchpad hosted project
 so I can't compare it.

Roundup is an issue tracker, while Patchwork is a patch tracker.
They are fundamentally distinct, but there are overlaps. What led me
to go the Patchwork-path is that projects like the kernel and Git
don't use issue trackers but work entirely patch-based.

I don't know if that is the right way to do things, but having an
issue tracker that fills up with bugs and wishlist items lacking
patches is no better in the long run than not having an issue
tracker.

Arguably, being patch-centric means that a project has a higher
barrier of entry, but it also means that if someone wants something,
they know that they'll have to somehow end up with a patch. The way
this happens on Git is that you either write it yourself and bring
it up to discussion (which is what patchwork facilitates), or
constructively theorise the functionality until someone else
submits a patch.

   Google's codereview tool has a nice interface for collecting and
 commenting on patches, but I suspect that suggestion will also meet with
 a degree of friction.  To me codereview feels like patchwork with
 polish.

Maybe you could take some ideas from codereview and inform the
patchwork people about them?

   Both gitorious and github have commenting functionality built in.
 Commenting on commits in a fork is as easy as opening the commit in
 a browser.  I use something along the lines of the following script to
 open commits on github:
 
 #! /bin/sh
 BASE=$(git config remote.${2:-origin}.url | sed 
 's,git\(@\|://\)\([^:/]*\)[:/]\(.*\).git,http://\2/\3/commit,')
 COMMIT=$(git rev-parse ${1:-HEAD})
 sensible-browser ${BASE}/${COMMIT}
 
   Using github or gitorious you can easily find and track forks from one
 place as well, which makes discovering new work much easier.  Github
 even provides a pretty single page interface to the work going on in
 other forks, gitorious requires a little more leg work to do the same
 but not much.

Git now has commit notes, but it doesn't seem like that's integrated
with Github/Gitorious.

Mind you, patchwork isn't integrated at all with Git. It should be
possible to set it up to automatically flag patches that are
accepted into mainline, next, or pu.

The benefit of patchwork is that discussion isn't moved to the web,
but patchwork hooks into the mailing list, so discussion can stay
where it should IMHO be.

   For a couple of hosted projects we use at the office we email the
 individual entries from http://github.com/$user/$project/comments.atom
 to the mailing list so they're /forcibly/ seen by everybody :)

Right, but replying requires them to open a browser and be online at
the time, right?



Anyway, I suggest we give patchwork a try. It occurs to me that
notmuch can pretty much do all of what patchwork is doing — after
all, it's just tagging patches/threads, but until we have
synchronisable tags and a mailing list archive based on notmuch
(which could then replace patchwork), I think we'll need to employ
a third tool.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
what's your conceptual continuity? --
 well, it should be easy to see:
 the crux of the bisquit is the apopstrophe!
-- frank zappa
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[notmuch] Emacs paned UI

2010-02-01 Thread Tad Fisher

Hello everyone!

I've been using notmuch and following its development for a little while
now, and I've come to like it very much. One thing that has been lacking
is an easy-to-use Emacs interface, and I believe Keith and Carl had a
little discussion a while back about how we should go about making a
paned UI.

So, I dusted off my Elisp manual and hacked up notmuch.el to support a
3-paned UI. This involved the creation of a minor mode (I call it
notmuch-browse-mode) that manages window state, along with several
modifications of existing functions to do the right thing when we're
in browse-mode (such as close the show window, switch to the folder
window, etc.).

The UI is thus far usable, but not bug-free. For example,
advancing-and-archiving using SPC sometimes advances to the same
message (but I am always able to advance with a). Also, the search
window does not update after archiving, but I believe this is an
existing bug.

The code is in the emacs-ui branch of the following repo:

git://github.com/tadfisher/notmuch-tad.git

Suggestions, criticism, and a code review are both needed and desired.

-- 
Tad Fisher
tadfis...@gmail.com

sent via something other than notmuch: http://notmuchmail.org/
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [notmuch] Request for high-priority improvements to notmuch

2010-02-01 Thread martin f krafft
also sprach sebast...@sspaeth.de sebast...@sspaeth.de [2010.02.02.0929 +1300]:
 YES PLEASE :-). notmuch seems designed to work in an ecosystem of
 surrounding scripts, feeding data in and out. But we are all currently
 limited to regexes for that. And heck, I hard a hard time understanding
 why all hell broke out until I found that i had added a tag containing
 parentheses which made my regex fail. :-). XML, JSON, any structured
 output would be nice.

Shoot me, but I'd say mbox output would be nice too.

-- 
martin | http://madduck.net/ | http://two.sentenc.es/
 
time flies like an arrow. fruit flies like a banana.
   -- groucho marx
 
spamtraps: madduck.bo...@madduck.net


digital_signature_gpg.asc
Description: Digital signature (see http://martin-krafft.net/gpg/)
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch