Re: folder and path completely broken in HEAD?

2014-05-05 Thread Carl Worth
dm-list-email-notm...@scs.stanford.edu writes:
 First, thanks for the response.  The responsiveness and friendliness of
 the notmuch mailing list goes a long way towards compensating for any
 missing features / customizability one might want.

I'm delighted to hear that the notmuch community maintains such a
helpful and friendly atmosphere.

 However, currently it seems strange that there are *two* different
 search terms (folder and path), and that neither one lets you search for
 a portion of your folder name.

For what it's worth, I totally agree. I'm guilty as far as sitting out
of the detailed design discussions, (I don't use any sort of
folder-based searching, so I don't care too much). I was aware of the
problems of the original folder: code I wrote, and I was happy to hear
that people were addressing those problems.

But it's terribly strange to me that notmuch now has two different
search terms that overlap so much in functionality. I know that I will
never trust myself to be able to distinguish/describe the folder:
vs. path: semantics without consulting the documentation. And that's
discouraging.

I think the original folder: shortcomings could have been addressed
without adding two terms, and also without losing some functionality,
(as shown in David's use case).

I would have liked to have seen some explicit syntax for anchoring the
beginning and end of the directory name, (which could have then been
re-used for anchoring subject: or even some future header: prefix).

 First, are there people out there who do not use a collection of maildir
 directories, with all mail in cur and new?

When I started notmuch, my entire mail archive was in non-maildir
directories. Since then, the bulk of my mail has been converted to
maildir (by accident more than anything). But I still do create
non-maildir directories for mail occasionally.

One use case for this is when someone forwards me an email (or a thread
of emails) to demonstrate a bug in notmuch. In a case like this, I will
do something like:

mkdir ~/mail/dm-folder-bug-2014-05

and then copy the mail files into that directory. I see no benefit nor
point to creating cur and new directories in a case like this.

 If not, why does notmuch try to find mail in non-mail-directories, and
 why do you need search terms differentiating new and cur?

I've always thought that the cur and new directories were somewhere
on the spectrum between pointless and annoying. The idea with the
original folder: indexing was to implicitly ignore these, (when both
existed).

It seems that the new folder: continues this idea, while the new
path: does not. Meanwhile, the new folder: anchors the search to the
beginning of the directory, while path: does not. And finally, path:
adds a magic syntax to do hierarchical matching while folder: does
not.

That's an odd hodgepodge of non-orthogonal distinction in functionality.

 Conversely, I find it particularly weird that there's no convenient
 way to say stop trying to index stuff that isn't in a maildir (cur or
 new).  You can do the inverse and blacklist files, but then I end up
 with stuff like this in my .notmuch-config:

Like I said, I started with an assumption of a hierarchy of directories
containing nothing but mail files, (that's exactly what I had at the
time).

 
 ignore=dovecot-keywords;dovecot-uidlist;dovecot-uidvalidity;dovecot.index;dovecot.index.cache;dovecot.index.log;dovecot.index.log.2;dovecot.index.search;dovecot.index.search.uids;maildirfolder;

I really do like explicitly describing the things you want to
ignore. One of the worst things that notmuch could do is to silently
fail to index legitimate emails. So I would not be in favor of features
that made that easier.

So the above looks fairly reasonable to me. Though it would also be
reasonable to allow for some wildcards there so that you could have
simply:

 ignore=dovecot*;maildirfolder;

 Second, does anyone out there have a collection with more than a few
 thousand maildirs?

I certainly don't. I have just short of one million messages, but only
841 directories in my mail store. And since that's counting .,
./tmp, ./cur, and ./new for every maildir, I probably only have
about 200 maildirs.

-Carl

-- 
carl.d.wo...@intel.com


pgpVzdGIUyAKY.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: folder and path completely broken in HEAD?

2014-05-05 Thread Jani Nikula

Hi Carl -

On Tue, 06 May 2014, Carl Worth cwo...@cworth.org wrote:
 dm-list-email-notm...@scs.stanford.edu writes:
 However, currently it seems strange that there are *two* different
 search terms (folder and path), and that neither one lets you search for
 a portion of your folder name.

 For what it's worth, I totally agree. I'm guilty as far as sitting out
 of the detailed design discussions, (I don't use any sort of
 folder-based searching, so I don't care too much). I was aware of the
 problems of the original folder: code I wrote, and I was happy to hear
 that people were addressing those problems.

 But it's terribly strange to me that notmuch now has two different
 search terms that overlap so much in functionality. I know that I will
 never trust myself to be able to distinguish/describe the folder:
 vs. path: semantics without consulting the documentation. And that's
 discouraging.

The discussions about this were lengthy and tedious, and I was glad we
eventually reached some consensus about what we wanted. It's always
disappointing to find out one hasn't found the solution to satisfy
everyone, but in the end I think I'm happier we were able to reach a
decision, do something about the real issues people were facing with
folder: and move on, rather than just grind to a halt. I think we were
pretty close to everyone just dropping the ball and letting the folder:
prefix be, warts and all.

The idea of path: is that it's the exact filesystem directory, relative
from the maildir root, with an rsync like ** syntax for recursive
matching. Turns out people want folder: to hide maildir implementation
details like cur and new. These are not compatible, or you need to add a
syntax that's not easy or discoverable.

path: is now pretty much complete, and allows one to do robust scripting
that won't break in surprising ways. folder: is something we can still
add new functionality into, for example fancier interpretations of
maildir++, or anchoring if we ever get the custom query parser.

 I think the original folder: shortcomings could have been addressed
 without adding two terms, and also without losing some functionality,
 (as shown in David's use case).

 I would have liked to have seen some explicit syntax for anchoring the
 beginning and end of the directory name, (which could have then been
 re-used for anchoring subject: or even some future header: prefix).

As I understood it, that would have required writing a custom query
parser, a significant effort. At least nobody came up with a scheme to
do the anchoring without the parser while addressing the other issues
with the old folder: prefix.

 I've always thought that the cur and new directories were somewhere
 on the spectrum between pointless and annoying. The idea with the
 original folder: indexing was to implicitly ignore these, (when both
 existed).

 It seems that the new folder: continues this idea, while the new
 path: does not.

Correct.

 Meanwhile, the new folder: anchors the search to the beginning of
 the directory, while path: does not.

Incorrect (or I don't understand you).

 And finally, path: adds a magic syntax to do hierarchical matching
 while folder: does not.

Correct.

 That's an odd hodgepodge of non-orthogonal distinction in
 functionality.

I'm sorry to hear you think that way. One is verbatim filesystem path,
the other hides mail store folder implementation details as a
convenience.


BR,
Jani.
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: folder and path completely broken in HEAD?

2014-05-05 Thread Carl Worth
Jani Nikula j...@nikula.org writes:
 The discussions about this were lengthy and tedious, and I was glad we
 eventually reached some consensus about what we wanted.

This much I do understand. And I apologize if I seem to be raising anew
issues that should finally be behind us.

 Meanwhile, the new folder: anchors the search to the beginning of
 the directory, while path: does not.

 Incorrect (or I don't understand you).
...
 That's an odd hodgepodge of non-orthogonal distinction in
 functionality.

 I'm sorry to hear you think that way. One is verbatim filesystem path,
 the other hides mail store folder implementation details as a
 convenience.

It may be that I simply misunderstood one aspect of path:. If it's
effectively ''anchored'' the same way as folder: then perhaps these
are more similar than I thought. In which case, I apologize for the
noise.

-Carl


pgpepWFsMp21N.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch