Re: Inconsistencies in handling command flags: `--flag=value` different than `--flag value`

2020-05-05 Thread Carl Worth
On Wed, Apr 29 2020, Jameson Graef Rollins wrote:
> I dare say there are few people that have muscle memory for the notmuch
> command line (especially for notmuch show), and fewer that aren't
> themselves developers...

Here, the concern isn't just "muscle memory" (forcing users to learn how
to retype commands at the command line) but also that users can have
developed scripts that call notmuch, and we would need to have an
extremely significant reason to break those.

And the notmuch CLI, from the beginning, was intentionally designed to
be comfortable for users to use directly (by typing at an interactive
shell), and for users to also incorporate into various scripts.

And even if they _are_ developers, they don't deserve to have their
environments broken either.

That said, it is unfortunate that some confusion has crept into the API.

I have to admit that I do not understand what this sentence of the
documentation means:

For the cases where it's not ambiguous (in particular excluding
boolean options), a space can also be used.

What are the possible cases where it could be ambiguous?

I'm personally opposed to adding any new option like --strict that
changes how command-line options are parsed.

I'm also opposed to renaming any of our existing command-line options.

It seems clear we could at least fix the wording of the above
documentation to make it clear that a space separator cannot be used for
Boolean options.

The only further change that could possibly make sense (to improve
consistency) would be to back out support for using a space separator
for _any_ options. But that introduces all the opportunities to break
users' environments, so it's probably a non-starter as well.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Weird tagging issue

2020-03-29 Thread Carl Worth
On Mon, Mar 30 2020, Brian May wrote:
> This applies to all 4 of those messages. Yes, they all look like they
> have the same Message-ID

4 different messages all with the same message ID definitely violates a
core assumption of notmuch (that each message as a unique message ID).

I think it would be reasonable to state that the behavior of notmuch is
officially "undefined" in that case.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: email recipient not taken into account when replying?

2020-01-08 Thread Carl Worth
On Wed, Jan 08 2020, Alan Schmitt wrote:
> Is there another way to tell notmuch to use as From address the address
> the email was sent to, if it's one of my emails?

Yes. The user.other_email configuration entry should do that.

For example, in ~/.notmuch-config:

[user]
name=Alan Schmitt
primary_email=alan.schm...@polytechnique.org
other_email=alan.schm...@inria.fr,alan.schm...@irisa.fr,presid...@lemondedujeu.org"

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: script to move messages according to tags

2019-12-11 Thread Carl Worth
Yes,

Notmuch considers any email files with the same message-id header as
being the same message so won't show you multiple versions of the same
email, (unless you're specifically asking for it to with something like
--output=files).

In normal usage I never worry about the duplicates I get from mailing
lists, etc.

I hope you'll find notmuch very useful.

Have fun!

-Carl

On Wed, Dec 11 2019, Alan Schmitt wrote:
> On 2019-12-11 16:33, David Edmondson  writes:
>
>> How you decide to deal with multiple results is really up to you 
>> - in my
>> case I move/delete all of them (except that I don't actually 
>> delete
>> anything).
>
> This is probably the key: I'm used to file-based MUA that will 
> show me
> duplicates. If notmuch hides them from me, I could just ignore 
> that they
> exist (it's not like it's taking too much space anyway…)
>
> Best,
>
> Alan
> ___
> notmuch mailing list
> notmuch@notmuchmail.org
> https://notmuchmail.org/mailman/listinfo/notmuch


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] Display extra headers for emacs-mua - db config option

2019-11-22 Thread Carl Worth
On Thu, Nov 21 2019, Johan Parin wrote:
> Johan Parin  writes:
>> Daniel Kahn Gillmor  writes:
>>>
>>> Is it that much worse to pass around the notmuch_database_t *?

My opinion is that we should not use the static notmuch_database_t
pointer. I think any desire to reach for that is, at best, papering over
some other problem that would be better fixed itself.

>> It has a lot of code impact, it really propagates to a lot of
>> places.
...
> Here is a taste (not fully tested, but seems to work).
...
>  static int
> -do_reply (notmuch_config_t *config,
> +do_reply (notmuch_database_t *notmuch,
> +   notmuch_config_t *config,

This passing around of the pair of notmuch_database_t* and a
notmuch_config_t* all over the place looks like an anti-pattern to me,
(and something that was just being hidden by the static
notmuch_database_t* that would be better to be fixed properly).

One option for reducing that pair to a single pointer would be to move
the configuration into the database, (as Daniel has been arguing for a
while).

I've argued against that in the past, (and obviously, I came up with the
current approach originally). But the historical situation has already
led to some unpleasant mixing of configuration state, (where some state
is in the configuration file and other state is in the database and it's
hard to predict which is where and which things can be controlled in the
configuration file).

And I think a patch like the above with the "notmuch, config" all over
the place would be another unpleasant result of the historical
convention.

So, I wouldn't be opposed to having the configuration move into the
database entirely. I would definitely like to see a tool that allows for
a dump/restore operation of the configuration state, (so users can still
edit configuration parameters with a text editor and with
fully-documented parameters). Bonus points if the users can also capture
their own commentary/explanation of values they are setting (as they can
do with the current configuration file).

And if things do move, existing configuration files should be updated
with a comment describing what that new dump/restore mechanism is and
that the current configuration file will no longer be used, (though, at
the transition point, obviously configuration parameters should be
copied from the configuration file into the database).

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Workaround for Exchange-corrupted PGP/Mime structure

2019-11-20 Thread Carl Worth
On Wed, Nov 20 2019, Carl Worth wrote:
> I'll update my notmuch and give this a try.

Just a "git pull; make; make install" and my problem went away.

Thanks again for the fixes, Daniel.

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Workaround for Exchange-corrupted PGP/Mime structure

2019-11-20 Thread Carl Worth
On Wed, Nov 20 2019, David Bremner wrote:
> There's some recent work by dkg in (currrently unreleased) commits on
> master. "git log -Smangled"

Thanks, David, and Daniel!

Those commit messages look very useful to me.

I'll update my notmuch and give this a try.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Workaround for Exchange-corrupted PGP/Mime structure

2019-11-20 Thread Carl Worth
I recently received a GPG-encrypted message that notmuch was not able to
handle transparently as I'm used to. (That is, instead of decrypting the
message and verifying the signature inside, etc.—instead I was just
presented with an attachment that contained the encrypted message.)

After some poking around, it appears that the problem is that the
message was sent through Microsoft Exchange which corrupts PGP/mime
structure by changing the outer content-type from "multipart/encrypted"
to "multipart/mixed".

I've verified this bug by sending mail from notmuch through Exchange as
the MTA and it indeed does corrupt the mail in exactly this fashion.

It looks like this bug in Exchange is known[1] and there's evidence of
other MUAs working around the bug[2].

Have other notmuch users encountered this problem?

What would anyone think about a workaround where notmuch-emacs could
grub around inside a "multipart/mixed" structure looking for child parts
that look like they really should have been a part of a
"multipart/encrypted" container?

-Carl

[1] 
https://social.technet.microsoft.com/Forums/en-US/c5059734-5031-4d82-96eb-c74fe60b5e9e/quotthis-is-a-broken-pgpmime-message-from-msexchangequot

[2] https://gitlab.com/mikecardwell/gpgit/blob/master/gpgit.pl#L108
which also refers to Enigmail having a workaround for thi.


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [FEATURE] indexing arbitrary headers

2019-05-30 Thread Carl Worth
On Thu, May 30 2019, Daniel Kahn Gillmor wrote:
> I just wanted to point out that this specific request has now been added
> to notmuch, thanks to David Bremner!  It will hopefully be part of the
> forthcoming 0.29 release:

Woo-hoo! Thanks so much!!

This has been on my I-wish-I-would-have-done-that-from-the-beginning
list for notmuch for a very long time.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: how to search for hyphenated words? (was: how to search for Morse code?)

2019-03-12 Thread Carl Worth
On Tue, Mar 12 2019, Gregor Zattler wrote:
> what I do not understand is that it dosn't matter if I search for
>
> org-notmuch
>
> or
>
> "org-notmuch"
>
> '"org-notmuch"'
>
> or even
>
> org ADJ/1 notmuch

Correct. All four of those forms are giving you phrase searches, (so a
term "org" followed immediately by a term "notmuch").

> a typical example of a matched message is the attached one.
> Somehow the search matches the address of this very mailing list
> in the body of the email (I assume).

No, I don't think you are seeing a match on the mailing-list address
itself, (which has "notmuch" two terms before "org").

> Therefore I wonder why notmuch matches 581 messages, not 16795
> messages or 77 messages.

David showed you one example from the message you copied:

> To: David Edmondson , notmuch@notmuchmail.org

And I showed one earlier in the thread.

In each case, the message includes "org" followed (after some amount of
punctuation and whitespace, perhaps including newlines) by "notmuch".

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: how to search for hyphenated words? (was: how to search for Morse code?)

2019-03-08 Thread Carl Worth
Hi Gregor,

The trick here is that when notmuch is indexing body text it feeds it
into a Xapian function that parses the text by finding "terms" in the
text. And this parser considers both punctuation and whitespace as
separators between terms.

So your messages are not being indexed in a way to let you distinguish
between "org notmuch" and "org-notmuch".

(Of note, the query parser applies the same parsing to your query---so
that even when you think you're typing an exact phrase like
"org-notmuch" that gets parsed into separate terms "org" and "notmuch"
for searching.)

> all these resulted in very many hits most or all of which do not
> contain the string "org-notmuch", one found email was e.g.
>
> id:20180904105723.15564-3-da...@tethera.net

That message does contain the following:

   +test_emacs '(notmuch-tree "id:000-real-r...@example.org")
   +   (notmuch-test-wait)

Where you will notice that there's a term "org" followed (after some
punctuation and whitespace separators) by a term "notmuch".

> How would one search for hyphenated words with notmuch?

You would need to arrange to have the indexer consider the hyphen as a
letter-like character to be made part of terms. Or be extra clever and
index something like "notmuch-test-wait" in multiple ways (such as a
single term "notmuch-test-wait" as well as three adjacent terms
"notmuch", "test", and "wait" as notmuch is doing currently).

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: travis build env switched to xenial

2019-03-06 Thread Carl Worth
On Wed, Mar 06 2019, David Bremner wrote:
> Those of you about to tell me how much better FreedomAvocadoCI [2] is
> than Travis are encouraged to set up instances and we can try it out. As
> long as Carl doesn't object, I would be interested in a setup where we
> run e.g. webhooks directly on git.notmuchmail.org to remove our
> dependence on github for CI.

I'm all in favor of removing a dependence on a third-party service here,
so would not object to adding some webhooks to our server.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Emacs and the missing 'g'

2019-02-13 Thread Carl Worth
On Tue, Feb 12 2019, Matt Armstrong wrote:
> I'm returning to notmuch after a hiatus.  It is still nice!

Glad to hear it! Welcome back.

> Most notmuch modes are "special" modes -- i.e. their text is generated
> by Emacs and does not reflect a file.  In these modes 'g' has
> traditionally been bound to revert the buffer [1].
...
> Any objection to also binding 'g'?

I have no opposition to this. I wasn't aware of the convention around
'g' when I first did the thing with '=', but I'd be happy to match that
convention to have notmuch feel more like a well-behaved citizen of the
emacs universe.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Release signatures

2019-02-11 Thread Carl Worth
On Sun, Feb 10 2019, David Bremner wrote:
> Adam Majer  writes:
>> The releases are signed in a funny way. The .asc file are not detached
>> signatures of the checksum, but actually contain it inside the .asc file.
>>
>> # gpg -v --verify notmuch-0.28.1.tar.gz.sha256.asc
>> ...
>> gpg: binary signature, digest algorithm SHA256, key algorithm rsa3072
>> gpg: WARNING: not a detached signature; file
>> 'notmuch-0.28.1.tar.gz.sha256' was NOT verified!
>>
>> A much better way of signing this would have been as a detached
>> signature of the tarball itself. Why sign a hash of a hash? ;)
>
> I'm not sure why Carl did it that way 10 years ago. Perhaps Carl
> remembers?  Offhand, I don't see any reason not to go with a more
> standard detached signature, other than it needs someone to do the
> relevant work.

If I did something non-standard here it certainly wasn't intentional.

I certainly would not oppose moving to a more standard (and obvious to
us) means of signing the releases.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: question about deletion and counts

2018-11-07 Thread Carl Worth
On Sat, Nov 03 2018, Jeff Templon wrote:
> No new mail. Removed 577 messages. Detected 89 file renames.
>
> 577 + 89 = 666 ... my guess is that there were 577 messages and 89 files
> that represented duplicates of messages.

Yes. Among the messages you had tagged as deleted you had 577 unique
message IDs. In addition, you had another 89 files with message IDs that
were duplicates of one of the 577.

> But I didn't rename the files, I deleted them.  Should I worry?

Nope. Nothing to worry about here.

> Why is the message inaccurate?

Because notmuch has a more narrow view of what a "rename" is than you
do.

A file rename is a high-level operation that will be seen by notmuch as
multiple operations seen over the course of a single run of notmuch
new:

  1. A new file is added with a message ID that already exists in the
 database

  2. A file is removed with a message ID for which there are multiple
 files in the database

But notmuch doesn't detect whether both of these operations are seen in
a single pass in order to detect a rename. Instead, what it is doing is
counting every occurence of (2) above as a rename. Here's what the code
looks like (notmuch-new.c:remove_filename):

status = notmuch_database_remove_message (notmuch, path);
if (status == NOTMUCH_STATUS_DUPLICATE_MESSAGE_ID) {
add_files_state->renamed_messages++;
if (add_files_state->synchronize_flags == true)
notmuch_message_maildir_flags_to_tags (message);
status = NOTMUCH_STATUS_SUCCESS;
} else if (status == NOTMUCH_STATUS_SUCCESS) {
add_files_state->removed_messages++;
}

So, whenever removing a filename, it will either get counted as a rename
(if there is still at least one other filename in the database with the
same message ID), or it will get counted as a removal (if this was the
last filename for message ID).

I suppose you could come up with some other name for what it is
counting, such as "removals of duplicate messages" instead of "rename",
but that's what's happening.

I hope that helps.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: finding file by size

2018-11-07 Thread Carl Worth
On Tue, Nov 06 2018, Ralph Seichter wrote:
> I'm not sure about using Notmuch itself,

Right. Notmuch doesn't currently index (as far as I'm aware) anything
that would be useful for sorting by size.

> but this should work:
>
>   find /path/to/maildir -type f -size +50M | xargs rm

Hmm... I imagine that Mark would be more interested in viewing these
files to ensure they are what he thinks they are before deleting them.

So, capturing the results of that with a notmuch tag would be
a reasonable thing to do. The only trick there is that I don't see any
existing search term to find a message associated with a particular file
name, (we have "path:" and "folder:" to find messages in a specific
directory, but nothing I see for finding the message corresponding to a
specific file).

So, then we could extract the message-id from each file and do a search
based on that I guess?

Here's a (bash) command I just ran on my mail store of over a million
messages that tagged the 8 messages larger than 50MB. It took about a
minute to run (with a warm cache):

  for msg in $(
   for file in $(find . -type f -size +50M);
   do
 grep -i ^Message-Id $file | sed -e 's/^.*<\(.*\)>.*/\1/';
   done
 );
  do
notmuch tag +large id:$msg;
  done

With that, I'm able to go through the list from:

  notmuch search tag:large

to investigate whether these large emails are worth keeping.

So, that's obviously not extremely elegant, but it's at least possible.

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: tell me how to do this (mass tagging of messages)

2018-10-17 Thread Carl Worth
On Wed, Oct 17 2018, Jeff Templon wrote:
> Indeed, as long as I am satisfied that all the messages in the view are
> OK to delete / tag / whatever ... I usually look at such at view to make
> sure that eg. the SPAM filter is doing the right thing.

I have not found any case in the notmuch-emacs UI where this has been an
issue for me, personally. If you do find a use case that's problematic,
let us know and we could certainly expand the interface to accommodate
that in some way.

Meanwhile, at the command line this distinction is made very clear. Try
each of these commands, for example:

notmuch search some-search-pattern

notmuch search --output threads some-search-pattern

notmuch search --output messages some-search-pattern

The second and third options there are very appropriate for generating a
list of threads (or messages) that you could then iterate over and
perform operations on.

And of course, if you're wanting to do an operation such as tagging or
un-tagging, then you can do that at that command-line directly without
needing an external loop over the results:

notmuch tag +some-tag some-search-pattern

notmuch tag -some-tag some-search-pattern

Both of these operations will act on matched messages, (no threads are
involved).

So hopefully playing with the above operations makes it clear how
notmuch itself is operating.

And the notmuch-emacs stuff is a pretty simple layer on top of that, so
again, we could easily improve things as necessary.

> My opinion : the best option would be to have some flag / toggle
> variable / whatever that would cause a search results screen to display
> only matching messages and not the whole thread.

That would certainly be an option.

But again, what operation does notmuch-emacs allow you to do on the
thread that you are really only wanting to do on the matching messages?

For me, a frequent operation is to just _see_ the matching messages. And
for this particular operation, just pressing "Enter" on a thread does
exactly that. It brings up a view of the whole thread, but with only the
matching messages open (and non-matching messages collapsed).

-Carl




signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: tell me how to do this right (mail sent to lists)

2018-10-12 Thread Carl Worth
On Wed, Oct 10 2018, Jeff Templon wrote:
>> The tag is not associated with the file in Sent, it is associated
>> with the message-id.
>
> I guess I didn't make myself clear enough, again.  I didn't mean that
> the tag is associated with the file.  What I am guessing is something
> like this:

Hi Jeff,

Thanks for persevering so that we can all try to understand what's
happening. I appreciate the patience on all sides. :-)

> for message in new_messages:
>if message.id not in database:
>   process message and determine list of tags
>   appy those tags to the messageID

Well, there are actually a couple of different processing loops that you
might be describing with the above. Let me try to walk through things:

First, there's a loop where "notmuch new" finds previously-unseen files
and indexes the content, adding it to the database:

for message_file in new_files:
  message_id = get_headers_message_id (message_file)
  (message,is_new) = database_lookup (message_id)
  index_file (message, message_file)
  if is_new:
add_new_message_tags (message)

The above pseudo-code is based on the loop in notmuch-new.c:add_files(),
add_file() as well as lib/add-message.cc:notmuch_database_index_file()
and more or less trying to use naming consistent with the code.

Something to not in the above loop: The database_lookup above, (which is
actually _notmuch_message_create_for_message_id), can either create a
new message object in the database or return an existing object. But,
either way, the content of the message will be indexed. So, the
significant feature is that notmuch will always be able to search the
content it indexes for any message file, (regardless of the order it was
encountered given any duplication).

However, as can also be seen in the above loop, tags that are added
to new messages, (these are as configured in the "new.tags" entry in
~/.notmuch-config) will only be added if the message is new in this pass
of notmuch-new.

I'm not looking into the code for afew right now, but I can guess a
couple of places the undesired bug could be happening:

1. It could be looping over all messages with the "new" tag. And if your
   sent message gets tagged "new" in a pass before the mailing-list
   duplicate is present, then afew will not have access to the
   mailing-list version when it does its processing. Then, later, when a
   pass does have the mailing-list duplicate present, it won't be
   considered a "new" message so would not get picked processed in a
   loop considering messages tagged "new".

2. It's possible that both message files are present at the time that
   afew does its processing, but that it only opens one of the files to
   go looking for the List-Id header, (which it must be doing
   somewhere---as David mentioned, the List-Id header is not ever
   indexed by notmuch itself).

And in the above discussion, I'm assuming that it's even notmuch-new
that's doing the detection of new files. Some people use mail flows that
have some external mechanism for processing new incoming mail and then
calling "notmuch insert" for each one.

In conclusion, you have a few different options to get reliable
behavior:

One option is to use notmuch-based searches to find the mailing-list
mail that is of interest for you. To do this you would want to key off
of a header that is indexed by notmuch. For example, you could do
something like:

notmuch tag +my-list-tag to:my-list-recipient-address

Another option is to continue to tag messages by inspecting the file
(outside of notmuch) to look for a header like List-Id (like you are
apparently doing now). To make this reliably, you would simply want to
ensure that that processing happens on every new file that is added. And
note that the "new" tag as added by "notmuch new" is not reliable for
that. That tag _is_ reliable for learning that a new message ID has
become available in the database, but is not reliably for know that a
new message file has appeared, (for a message ID that was present
previously).

Does that help explain things?

-Carl







signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: tell me how to do this right (mail sent to lists)

2018-10-09 Thread Carl Worth
On Tue, Oct 09 2018, Jeff Templon wrote:
> So here’s the deal : I am tagging with afew, it has a special mailing
> list filter, which does the right thing - it tags the copy that was
> delivered from the mailing list with the list name.

I think you may be mistaken about the above details.

If the two messages have the same message ID, then as far as notmuch is
concerned that is just a single message, (that happens to be backed by
multiple files).

> However (and maybe this is just the Emacs interface?) when I view the
> message in notmuch-emacs, it only shows one of them, and this seems to
> be (always?  at least usually) the copy that is Fcc’d to my sent
> folder and tagged only with “sent” because it doesn’t have a list-id
> yet.

It is true that if a single message is backed by multiple files, the
notmuch-emacs interface will only display one of them. But even if you
changed which one it chooses (which would require some new code as far
as I know) shouldn't change what tags you are seeing.

> So I’m still looking for how to do this “right”.  I must not be the
> only one that sends mails to mailing lists?

You're certainly not the only one. Hopefully, what you really want
doesn't require the ability to maintain separate tags for these two
very-similar messages, (since notmuch cannot do that currently).

I guess the question I have is: What has given you the idea that afew is
tagging one message but not the other?

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: notmuch shows tag which is not on any email

2018-06-04 Thread Carl Worth
On Mon, Jun 04 2018, Gregor Zattler wrote:
> Dear notmuch developers,

Hi, Gregor!

> does notmuch remember tags even if at time of query there is no
> message tagged with the respective tag?:

No. At least, from when I originally invented the notmuch schema, tags
only existed on messages and there was no separate store of a list of
known tags anywhere.

If something in the schema has changed since, I would appreciate someone
correcting me.

So, I'm not sure what's going on in with your notmuch database.

I tried tagging a couple of messages with the same tags you are seeing
in case there was something in one of these tag names specifically that
was ticking a problem. But things look fine for me.

Here's the output I'm now getting on my notmuch store with the same
commands you gave, (and note that everything is working for me):

$ notmuch search --output=tags '*' | grep telegraph
EA%3Dtelegraph%40gmx%2Enet
EA=telegr...@gmx.net
$ notmuch count  -- is:EA%3Dtelegraph%40gmx%2Enet
1
$ notmuch count  -- is:/EA.*telegraph/
2
$ notmuch search --output=tags '*' | grep telegraph | head -n 1 | wc -c
27

Anyone ave a theory for what could explain what Gregor is seeing?

-Carl



signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: a temporary nmweb view of notmuch mailing list archive

2018-02-12 Thread Carl Worth
On Sun, Feb 04 2018, Daniel Kahn Gillmor wrote:
> I did a bit of experimenting with Brian Sniffen's proposed notmuch-web
> branch, and it's now (temporarily) publishing a view of the notmuch
> mailing list here:
>
> https://nmbug.notmuchmail.org/btsmail/

Thanks, Brian and Daniel!

When I first started on notmuch, I was hoping that one immediate benefit
would be that free-software projects could finally have working web
archives of their mailing lists, (without thread views breaking at
arbitrary boundaries such as months or years).

I guess the "immediate" aspect of my hope didn't play out so much, but I
am very excited to see this now. So, thanks again!

At first glance I saw "terms ... after ... before" and thought that the
after/before fields were for some sort of contextual context, but I
finally figured out what those were actually for. So maybe it would help
to add some text along the lines of "Limit to date range" or so.

And now that we have real date searching, it might make sense to use
that in the searching instead of the time stamp.

Another thing that might be nice for selecting dates ranges is the
ability to select something less precise than a particular day.

Searching for all the messages in a month with something like:

notmuch search date:2018-01

or even a year with:

notmuch search date:2018

is pretty darn convenient in notmuch, or even to search across two
months with:

notmuch search date:2018-01..2018-02

It would be nice to not lose that kind of convenience in a web
interface.

Anyway, thanks again. This is looking great, and I will look forward to
entirely removing the pipermail-based archives from the website as soon
as we can.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Fetching from the git repositories over https?

2018-02-09 Thread Carl Worth
On Sun, Jan 28 2018, Adam Plaice wrote:
> I apologise if I'm asking in the wrong place.

Not at all. This is the right place.

> Is it possible to clone/fetch from the notmuch git repositories
> (particularly https://git.notmuchmail.org/git/notmuch) over https
> rather than with the `git://' protocol?  (None of the likely
> alternatives seem to work.)

It wasn't possible when you asked, but I just configured this, and it
seems to work.

Specifically, I have tested that I can point my browser at:

https://git.notmuchmail.org/git/notmuch

to see the gitweb view of the git history on the web, and I can also use
that same URL for a git clone:

git clone https://git.notmuchmail.org/git/notmuch

and that works.

I also verified that I a "git push" from such a clone results in a 403
error as desired.

So give that a try, and anyone, let me know if you see anything that I
may have broken or setup incorrectly.

Also, I haven't yet updated any documentation to point to this new
mechanism, so that's something that could still be done.

> Thank you for your time and thank you for notmuch,

You're quite welcome. And thank you for your contribution!

-Carl
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] debian: drop export-dir from gbp.conf

2018-02-09 Thread Carl Worth
On Tue, Feb 06 2018, Daniel Kahn Gillmor wrote:
> No one knows why this invocation is here.  it does weird things during
> the build.  If a developer really wants this behavior, they can
> specify it in their own ~/.gbp.conf

I imagine I'm to blame for this one?

One of the things that has always bugged me about a bunch of that Debian
build utilities is that they spit out results into the parent directory
of the current working directory from where they are invoked.

I've always found that be really mystifying, (no well-behaved program
does such a thing).

I _think_ the export-dir directive here was something I found that made
git-buildpackage behave more sane in this regard.

But I'm not the one doing frequent package builds of notmuch these days,
so feel free to do what you think best.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Possible bug with named queries

2017-12-13 Thread Carl Worth
On Wed, Dec 13 2017, David Bremner wrote:
> STORED IN DATABASE, thanks to some doc cleanup by dkg. In general I'd
> recommend using using "notmuch config" for all configuration changes,
> then you don't have to keep track of which ones are stored in
> .notmuch-config.

Really? We're deprecating the ability to just edit the configuration
file by hand? That's unfortunate.

> There are technical reasons for certain items being in
> the database, and that's likely to continue.

I won't dispute the technical reasons, (I'm out of the loop on that).

But if running "notmuch config" to add a saved search stores some
necessary data in the database, couldn't the same code be invoked when
loading the configuration file and noticing a new saved search there in
the file?

We've gone out of our way to make the configuration file as useful as
possible, (such as by maintaining comments in the file even when using a
tool such as "notmuch config" to edit it). Let's please not throw out
all the benefits by just saying that manually editing the configuration
file is unreliable and people shouldn't do it.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: notmuch.el: controlling what does and doesn't get expanded in searches

2016-08-04 Thread Carl Worth
On Thu, Aug 04 2016, Matt Armstrong wrote:
> Yes, I find the query semantics with respect to tags and threads a bit
> confusing at times.  This is not a problem specific to notmuch, as I
> find the same kinds of issues in GMail.  Usually the problem occurs at
> the semantic border between per-message tags and thread-based
> operations.
>
> I can now explain what I am seeing.

Hi Matt,

I haven't been very active on the notmuch mailing list in quite some
time, but I wanted to poke my head in quickly to welcome you and to
thank you for your contribution.

I really like detailed discussions like this about coming up with good
workflows, and trying to figure out how notmuch could better accommodate
those workflows. I think this is one of the most valuable aspects of
notmuch, (that it lets us ask these kinds of questions).

So, please, keep it up!

>  i) notmuch could have an "also expand tags" feature, where thread based
> results would auto expand matching tags.  I would set this to
> "unread".

This approach makes a lot of sense to me based on how notmuch.el works.

>  ii) notmuch could have an "expand query" feature, where thread based
>  results would use an entirely different query to decide, within a
>  thread, which messages to expand.  I would set this query to
>  "tag:unread".

This approach would necessarily be quite a bit more complex in the
implementation without much difference in the user-visible behavior. So
I don't think we would want to pursue this.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Push to wiki fails with "cannot run hooks/pre-receive" error

2016-03-23 Thread Carl Worth
On Wed, Mar 23 2016, J. Lewis Muir wrote:
> I just tried to push a minor fix to the wiki, but I get the following
> error:
...
> remote: error: cannot run hooks/pre-receive: No such file or directory

Oops. My mistake.

I recently converted all user-space packages on the server from 32-bit
to 64-bit, but was inadvertently over-zealous and removed the packages
necessary to still run a 32-bit binary.

Of course, there aren't any packaged 32-binaries left around, but this
pre-receive hook is a binary that ikiwiki compiles, so it is still
currently a 32-bit binary.

I just installed the support for running 32-bit binaries again, so
hopefully this will work now.

Let me know if you run into any further problems.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Notmuch -- Just an email system

2016-03-15 Thread Carl Worth
On Sat, Mar 12 2016, David Bremner wrote:
> I'm not carl, but you can find information about licensing of the wiki
> material here
>
>   https://notmuchmail.org/COPYING/
>
> We'd be happy to host a russian version of some of the pages on the
> wiki; see
>
>   https://notmuchmail.org/wikiwriteaccess/
>
> for how to add pages.

And I am Carl, and I approve of the above message!

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: Stolen email list?

2016-03-03 Thread Carl Worth
On Thu, Mar 03 2016, David Bremner wrote:
> Johnny Utahh  writes:
>
>> I recently received email sent to my notmuch-based email address that 
>> was clearly spam. I suspect that notmuch@notmuchmail.org email-address 
>> list may have been stolen. If you're not already aware.
>
> I'm not aware of any such event. I've CC'd Carl, who runs the lists.

I'm certainly not aware of anything either.

But I'd be happy to investigate anything I can, (if you have data to
point me to where to look).

In the meantime, if the list was leaked, are there any mitigating steps
you would recommend we perform at this point?

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: NOTMUCH_STATUS_LAST_STATUS problematic across additive library upgrades

2015-12-07 Thread Carl Worth
On Mon, Dec 07 2015, Daniel Kahn Gillmor wrote:
> I note that it's exposed in the go bindings -- maybe it shouldn't be
> exposed in any of the bindings that can avoid exposing it?

Quite likely. It's only there in C because there's no other way to know
the bounds for iterating over all possible values for a particular
enum. If other languages don't have this deficiency, then they shouldn't
need this silly thing.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: elisp + company completion patches, v7

2015-10-26 Thread Carl Worth
On Sun, Oct 25 2015, Tomi Ollila wrote:
> I, if this does not sound nitpicking, would amend those 'This patch' and
> 'With this patch' texts in commit messages away by some rewording -- those
> pose as a bad example for someone reading the code -- and diminish the
> effect when I complain that again in future patched ;)

What's the motivation here?

It's often the case that a good commit message needs to describe the way
things were working prior to a commit, (describing the bug or missing
feature or what have you), and also needs to describe the change that is
affected by the commit itself.

So I actually like seeing commit messages that say things like:

Prior to this commit... (some bug existed...)

In this commit... (some code is reworked to fix the bug...)

But maybe I've missed the point of your message.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
https://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] doc: update copyright year

2015-10-15 Thread Carl Worth
On Thu, Oct 15 2015, Tomi Ollila wrote:
> -copyright = u'2014, Carl Worth and many others'
> +copyright = u'2015, Carl Worth and many others'

I think the correct approach to copyright here is to expand the range of
years, (so "2014-2015") rather than increment it.

Presumably that means the starting date should be 2009, too.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: correct way to search for only PDF attachments

2015-09-28 Thread Carl Worth
On Mon, Sep 28 2015, Xu Wang wrote:
> I would look to look for all emails from a colleague jongho. I tried:
>
> from:jongho attachment:pdf
>
> which seems to do as I wanted.

Good. That should work.

> To understand more, what does the following search for?
>
> from:jongho attachment:.*pdf

Uhm, probably only strange things. There are some mechanisms for getting
notmuch to emit some debugging information on what the final search
terms end up being, (but I don't recall if they still require
recompilation or not).

I'm not testing now, but I wouldn't be surprised if that ended up doing
something like searching for a phrase like "attachment pdf" anywhere
within a message. (The Xapian parser can be somewhat unpredictable when
you give it unexpected input.)

> Also, how does the first one above know that I want only PDF
> attachments and not an attachment called "pdformula.txt" ?

It doesn't know that you want only PDF attachments. The key part is that
the indexing is performed by breaking text up into individual terms, (at
punctuation boundaries usually). So a search specification like
"attachment:pdf" is searching for things that were indexed with the
"pdf" term within the attachment prefix. So that won't match a filename
like pdformula.txt, (which would be indexed as two terms, "pdformula"
and "txt"), but it would match pdf.ormula.txt, (which would be indexed
as three terms, "pdf", "ormula" and "txt").

The Xapian documentation can be examined if you want more details.

-Carl


signature.asc
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Website and mailing list are hopefully back now

2015-04-03 Thread Carl Worth
On Fri, Apr 03 2015, Carl Worth wrote:
> The notmuchmail.org server experienced a catastrophic failure a few days
> ago. I gave up on getting access to that server anymore, and have
> instead brought things back up on a separate server that I had
> available, (and to which I have actual, physical access this time).

Oh, and by the way, when you use ssh to the new server from an account
that had connected to the old server, you'll likely get the scary
warning that the ssh host key has changed.

Here is the fingerprint of the RSA key that you should expect to get:

1f:64:19:21:50:8f:1a:72:bc:69:45:8c:25:c5:71:7a

With that fingerprint, you can assume that the key change isn't due to a
man-in-the-middle attack and follow the instructions on removing the
old key.

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20150403/ae983f46/attachment.sig>


Website and mailing list are hopefully back now

2015-04-03 Thread Carl Worth
Hello all,

The notmuchmail.org server experienced a catastrophic failure a few days
ago. I gave up on getting access to that server anymore, and have
instead brought things back up on a separate server that I had
available, (and to which I have actual, physical access this time).

I restored the git repository and website from a nightly backup that was
taken late on March 30. (The server seems to have failed sometime
between then and when the nightly backup on March 31 was attempted.)

There are likely things that are not quite properly setup on the new
server, as well as data that I may not have copied from the backup.
If you encounter anything, please let me know and I'll see what I can
do.

Thanks for all your patience,

-Carl

PS. When the new server came up, the mail server wasn't properly
handling mailing-list traffic. So a few messages came to the server,
(and landed in the web archives), but didn't get sent out to subscribers
of the list. Obviously, the messages hopefully still went out to
individuals directly addressed by the mails.

Here are the list of message-ids for the mails that didn't go out. If
you sent any of these, you might want to check whether they are
important enough to resend to the list:

1428047832-14922-1-git-send-email-jani at nikula.org
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5.fsf at maritornes.cs.unb.ca
1428047832-14922-1-git-send-email-jani at nikula.org
yq65pp7l9a6f.fsf at jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5.fsf at maritornes.cs.unb.ca
87sichmoaa.fsf at hirscho.lama.univ-savoie.fr





















-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 818 bytes
Desc: not available
URL: 



Website and mailing list are hopefully back now

2015-04-03 Thread Carl Worth
Hello all,

The notmuchmail.org server experienced a catastrophic failure a few days
ago. I gave up on getting access to that server anymore, and have
instead brought things back up on a separate server that I had
available, (and to which I have actual, physical access this time).

I restored the git repository and website from a nightly backup that was
taken late on March 30. (The server seems to have failed sometime
between then and when the nightly backup on March 31 was attempted.)

There are likely things that are not quite properly setup on the new
server, as well as data that I may not have copied from the backup.
If you encounter anything, please let me know and I'll see what I can
do.

Thanks for all your patience,

-Carl

PS. When the new server came up, the mail server wasn't properly
handling mailing-list traffic. So a few messages came to the server,
(and landed in the web archives), but didn't get sent out to subscribers
of the list. Obviously, the messages hopefully still went out to
individuals directly addressed by the mails.

Here are the list of message-ids for the mails that didn't go out. If
you sent any of these, you might want to check whether they are
important enough to resend to the list:

1428047832-14922-1-git-send-email-j...@nikula.org
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5@maritornes.cs.unb.ca
1428047832-14922-1-git-send-email-j...@nikula.org
yq65pp7l9a6f@jinwoo-macbookair.roam.corp.google.com
87wq1s4yz5@maritornes.cs.unb.ca
87sichmoaa@hirscho.lama.univ-savoie.fr























pgpfOqZrlixHH.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Website and mailing list are hopefully back now

2015-04-03 Thread Carl Worth
On Fri, Apr 03 2015, Carl Worth wrote:
 The notmuchmail.org server experienced a catastrophic failure a few days
 ago. I gave up on getting access to that server anymore, and have
 instead brought things back up on a separate server that I had
 available, (and to which I have actual, physical access this time).

Oh, and by the way, when you use ssh to the new server from an account
that had connected to the old server, you'll likely get the scary
warning that the ssh host key has changed.

Here is the fingerprint of the RSA key that you should expect to get:

1f:64:19:21:50:8f:1a:72:bc:69:45:8c:25:c5:71:7a

With that fingerprint, you can assume that the key change isn't due to a
man-in-the-middle attack and follow the instructions on removing the
old key.

-Carl


pgpi1PXbXAk0R.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 07/11] lib: Implement ghost-based thread linking

2014-10-03 Thread Carl Worth
On Fri, Oct 03 2014, Austin Clements wrote:
 This updates the thread linking code to use ghost messages instead of
 user metadata to link messages into threads.

 In contrast with the old approach, this is actually correct.

Hi Austin,

I've read through your commit messages, (not the code), and I just
wanted to say a big thank you to you!

I'm really looking forward to the improved robustness in notmuch with
this support.

I really appreciate your diligence in working through a fairly large,
low-level change here.

-Carl


pgpavnlvD49KS.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] cli: Be more helpful when .notmuch-config does not exist

2014-09-03 Thread Carl Worth
Austin Clements amdra...@mit.edu writes:
 Since this condition is expected for new users, this patch adds
 specific handling for the file-not-found case to give a message that
 is friendly and actionable.
...
 + fprintf (stderr, Configuration file %s not found.\n
 +  Try running 'notmuch setup' to create a 
 configuration.\n,
 +  config-filename);

Thanks, Austin!

I've always wanted notmuch to provide as gentle an introduction as
possible, (since switching MUA and indexing all email[*] is already a
fairly invasive thing).

So I really appreciate improvements like this.

 +test_begin_subtest Notmuch new without a config suggests notmuch setup
 +output=$(notmuch --config=new-notmuch-config new 21)
 +test_expect_equal $output \
 +Configuration file new-notmuch-config not found.
 +Try running 'notmuch setup' to create a configuration.

And a test to boot. That really warms my heart.

-Carl

[*] A further possible improvement here would be some mechanism for
initially indexing only very recent email, (to let the user start
playing with notmuch right away), and then in some sort of ''offline''
fashion, completing the full indexing.


pgpf52pf6_z_w.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH v2] emacs: jump: sort-order bugfix

2014-09-03 Thread Carl Worth
Mark Walters markwalters1...@gmail.com writes:
 The symptom is that if you are in a search buffer and use notmuch jump
 to run a saved search which does not have an explicitly set sort order
 then the sort order of the saved-search is inherited from the current
 search buffer rather than being the default search order.

 Thanks to Jani for finding the bug.

Thanks, Mark. The description above is just what I was looking for.

 (Part of the reason I did not send more before is I had not checked
 what the exact outcome of the buggy code was: it was obvious what the
 code was intended to do, and that with the extra quote it
 would do what it was intended to do.)

Yes. I understand now how the bug-fix was more obvious than the bug.

But thanks for doing the extra investigation nonetheless.

-Carl


pgpUyACD1apZc.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] emacs: jump: sort-order bugfix

2014-09-02 Thread Carl Worth
Mark Walters markwalters1...@gmail.com writes:
 default-value needs its argument to be quoted.
 ---

Hi Mark,

I really appreciate you sending in a bug fix. Thanks!

I'd prefer to see the commit message describing the bug and the fix a
bit.

You do have some text here that would be a great addition to the commit
message:

 Slightly strangely default-value of 't or nil is 't or nil
 respectively so the code didn't give an error but just did the wrong
 thing.

But even beyond that. What is the wrong thing that the code did here?

Imagine someone (me!) trying to read and review the patch without ever
encountering the bug before. How can I test the bug both before and
after applying the patch?

I'd really like to see that information in the commit message.

Thanks,

-Carl


pgpkfzqsTF18i.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH v1] emacs: `notmuch-search-find-stable-query-region' should never return an empty query.

2014-05-22 Thread Carl Worth
Carl Worth cwo...@cworth.org writes:
 Mark Walters markwalters1...@gmail.com writes:
 Ok I now understand some of why it is a bug and it is a pretty gross
 mix: emacs does something odd, then notmuch-tag.c then xapian. It only
 happens in some cases:
...
 Anyway, I'm glad to see the notmuch emacs interface getting patched for
 this problem. We should ensure that the Xapian parsing bug is reported
 as well.

*And*, since Mark identified bugs in three different layers, we should
 fix that third layer as well.

It seems we could make notmuch-tag.c more robust by using native Xapian
interfaces to construct a query AND query data structure rather
than using string manipulation in an attempt to construct this same
query. That would prevent this ambiguity, and avoid any other cases of
emacs (or other users) running across a similar bug here.

-Carl

-- 
carl.d.wo...@intel.com


pgpqcsHyu7MQy.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH v1] emacs: `notmuch-search-find-stable-query-region' should never return an empty query.

2014-05-21 Thread Carl Worth
Mark Walters markwalters1...@gmail.com writes:
 Ok I now understand some of why it is a bug and it is a pretty gross
 mix: emacs does something odd, then notmuch-tag.c then xapian. It only
 happens in some cases:

 WARNING the following will archive some random messages: have a dump or
 backup or something!

Thank you!!

I'm quite sure that I've been bitten by this bug a couple of
times. (Actually, it's likely bitten me more than the couple of times I
happened to notice it.) When I noticed it, I had only a few messages in
my inbox, (had just been reading/archiving some), pressed '=' to
refresh, and several of them disappeared.

It was extremely disconcerting that notmuch would lose data like that on
me. I hadn't reported the bug because I hadn't found any reliable way to
replicate it, nor could I be sure that it wasn't something unique to my
setup, (such that I had actually archived those messages, but from some
other script or view, etc.).

But given your description, I'm confident that the data loss I watched
before my eyes fits that description.

 ( () ) and ( tag:inbox )

 for reasons which aren't clear to me this matches some but not all
 of the messages with tag:inbox. This is despite the fact that searching for
 ( () )  or () by itself yields no results.

You can use NOTMUCH_DEBUG_QUERY to get some insight into this.

So the query that works as expected:

$ NOTMUCH_DEBUG_QUERY=1 notmuch count (())
Query string is:
(())
Exclude query is:
Xapian::Query()
Final query is:
Xapian::Query()
0

And then the query that is misbehaving:

$ NOTMUCH_DEBUG_QUERY=1 notmuch count (()) and tag:inbox
Query string is:
(()) and tag:inbox
Exclude query is:
Xapian::Query()
Final query is:
Xapian::Query((Tmail AND (Zand:(pos=1) FILTER Kinbox)))
14

So you can see that the query is actually matching a term Zand, (that
is, matching on emails for which the word and has been indexed).

It's a funny bug. Note that using () and tag:inbox does not trigger
the same behavior:

$ notmuch count () and tag:inbox
A Xapian exception occurred: Syntax: expression AND expression
Query string was: () and tag:inbox
0

I would have much preferred getting that exception instead of the query
we were getting before.

Anyway, I'm glad to see the notmuch emacs interface getting patched for
this problem. We should ensure that the Xapian parsing bug is reported
as well.

-Carl


pgpDHzyKTi8qh.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 0/5] nmbug-status fixups from notmuch-to-html

2014-05-11 Thread Carl Worth
W. Trevor King wk...@tremily.us writes:
 I've cherry-picked the three patches I liked from Carl's master [1,2]
 and added two cleanup commits of my own.  I've left Carl's commit
 messages unchanged.  I'm fine squashing my fixups into Carl's
 originals (leaving Carl as the author) if that's ok with him.

Please feel free to do whatever you like with any of these.

Definitely update the commit messages to match the reality of what
you're committing. You can do that by directly editing the text, or
adding a clarifying paragraph after the fact.

And if you make any substantial changes, I certainly won't mind (or even
notice) if my authorship doesn't remain.

Thanks for ensuring the code doesn't get lost. I appreciate that.

-Carl


pgpsVqm6UK2YI.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] nmbug-status: Make the footer's hr gray

2014-05-11 Thread Carl Worth
W. Trevor King wk...@tremily.us writes:
 On Sun, May 11, 2014 at 12:08:10AM +0300, Tomi Ollila wrote:
 The rendered hr/ irritates me a bit, though. Maybe it could be
 colored light gray ?

 I can't reroll that one myself, unless Carl gives the ok.  But here's
 some light-gray styling as an additional patch.

To be totally clear. Yes, hack it up like crazy however you see fit!

-Carl


pgpM1HTI69LYl.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: folder and path completely broken in HEAD?

2014-05-05 Thread Carl Worth
dm-list-email-notm...@scs.stanford.edu writes:
 First, thanks for the response.  The responsiveness and friendliness of
 the notmuch mailing list goes a long way towards compensating for any
 missing features / customizability one might want.

I'm delighted to hear that the notmuch community maintains such a
helpful and friendly atmosphere.

 However, currently it seems strange that there are *two* different
 search terms (folder and path), and that neither one lets you search for
 a portion of your folder name.

For what it's worth, I totally agree. I'm guilty as far as sitting out
of the detailed design discussions, (I don't use any sort of
folder-based searching, so I don't care too much). I was aware of the
problems of the original folder: code I wrote, and I was happy to hear
that people were addressing those problems.

But it's terribly strange to me that notmuch now has two different
search terms that overlap so much in functionality. I know that I will
never trust myself to be able to distinguish/describe the folder:
vs. path: semantics without consulting the documentation. And that's
discouraging.

I think the original folder: shortcomings could have been addressed
without adding two terms, and also without losing some functionality,
(as shown in David's use case).

I would have liked to have seen some explicit syntax for anchoring the
beginning and end of the directory name, (which could have then been
re-used for anchoring subject: or even some future header: prefix).

 First, are there people out there who do not use a collection of maildir
 directories, with all mail in cur and new?

When I started notmuch, my entire mail archive was in non-maildir
directories. Since then, the bulk of my mail has been converted to
maildir (by accident more than anything). But I still do create
non-maildir directories for mail occasionally.

One use case for this is when someone forwards me an email (or a thread
of emails) to demonstrate a bug in notmuch. In a case like this, I will
do something like:

mkdir ~/mail/dm-folder-bug-2014-05

and then copy the mail files into that directory. I see no benefit nor
point to creating cur and new directories in a case like this.

 If not, why does notmuch try to find mail in non-mail-directories, and
 why do you need search terms differentiating new and cur?

I've always thought that the cur and new directories were somewhere
on the spectrum between pointless and annoying. The idea with the
original folder: indexing was to implicitly ignore these, (when both
existed).

It seems that the new folder: continues this idea, while the new
path: does not. Meanwhile, the new folder: anchors the search to the
beginning of the directory, while path: does not. And finally, path:
adds a magic syntax to do hierarchical matching while folder: does
not.

That's an odd hodgepodge of non-orthogonal distinction in functionality.

 Conversely, I find it particularly weird that there's no convenient
 way to say stop trying to index stuff that isn't in a maildir (cur or
 new).  You can do the inverse and blacklist files, but then I end up
 with stuff like this in my .notmuch-config:

Like I said, I started with an assumption of a hierarchy of directories
containing nothing but mail files, (that's exactly what I had at the
time).

 
 ignore=dovecot-keywords;dovecot-uidlist;dovecot-uidvalidity;dovecot.index;dovecot.index.cache;dovecot.index.log;dovecot.index.log.2;dovecot.index.search;dovecot.index.search.uids;maildirfolder;

I really do like explicitly describing the things you want to
ignore. One of the worst things that notmuch could do is to silently
fail to index legitimate emails. So I would not be in favor of features
that made that easier.

So the above looks fairly reasonable to me. Though it would also be
reasonable to allow for some wildcards there so that you could have
simply:

 ignore=dovecot*;maildirfolder;

 Second, does anyone out there have a collection with more than a few
 thousand maildirs?

I certainly don't. I have just short of one million messages, but only
841 directories in my mail store. And since that's counting .,
./tmp, ./cur, and ./new for every maildir, I probably only have
about 200 maildirs.

-Carl

-- 
carl.d.wo...@intel.com


pgpVzdGIUyAKY.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: folder and path completely broken in HEAD?

2014-05-05 Thread Carl Worth
Jani Nikula j...@nikula.org writes:
 The discussions about this were lengthy and tedious, and I was glad we
 eventually reached some consensus about what we wanted.

This much I do understand. And I apologize if I seem to be raising anew
issues that should finally be behind us.

 Meanwhile, the new folder: anchors the search to the beginning of
 the directory, while path: does not.

 Incorrect (or I don't understand you).
...
 That's an odd hodgepodge of non-orthogonal distinction in
 functionality.

 I'm sorry to hear you think that way. One is verbatim filesystem path,
 the other hides mail store folder implementation details as a
 convenience.

It may be that I simply misunderstood one aspect of path:. If it's
effectively ''anchored'' the same way as folder: then perhaps these
are more similar than I thought. In which case, I apologize for the
noise.

-Carl


pgpepWFsMp21N.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Thanks for the nmbug-status program

2014-04-23 Thread Carl Worth
W. Trevor King wk...@tremily.us writes:
  I think that making nmbug and nmbug-status generally applicable for
  other projects is a good target.  I don't see any need to pull them
  out into a stand-alone project to do that though.
 
 For what it's worth, I don't disagree.

 Does that mean your patches are aimed upstream at the notmuch
 repository?

I didn't really intend them that way. What I meant was that I'd be happy
to switch to a script within the notmuch repository if it did what I
wanted, (which is basically just create an HTML file from a notmuch
query).

 (or their shell's approriate environment variable syntax), and I'd
 version the config file in the same repo as the associated tags.  I
 don't see the point of keeping the config file somewhere else, when
 you'll already need a repository to store the tags.

Ah, then you've missed something different about my usage. I don't have
any repository storing tags. My tags exist only in my notmuch
database. The tool I want here is a simpler one, (think, allow HTML
export from HTML). Heck, what I want could almost be nothing more than:

notmuch search --format=html

Other than that I want the other bits like multiple defined views with
their own queries and descriptions.

I see the part that has a repository holding tags as totally orthogonal
functionality. That orthogonal functionality is tag synchronization or
tag collaboration and is independent from HTML export to my way of
thinking.

So it makes sense to me that the HTML export just talks to the notmuch
database and doesn't need to assume anything else.

 * d456390 Add a comment describing the program

 I like the idea, but I'd prefer a docstring just after the leading
 comment.  Then we could reuse the docstring as the description for the
 ArgumentParser.

Sure. That's better.

 * 336411f Add notmuch search to the advertised queries.

 Folks can already copy-paste the terms into Emacs etc.  You only need
 the 'notmuch search' prefix if you're pasting into a shell, in which
 case you'll also want to escape shell-sensitive characters like
 parenthesis.  I'm fine without this one.

This is more about advertising notmuch to people not familiar with
it. It's more important for something like mesa stable queue than it
is for the notmuch bug list, of course.

 * 396869c Add support for a --query option (along with a default 
 configuration)

 I don't have an opinion about this one ;).  The default config doesn't
 look very useful (especially with it's dummy metadata settings), but
 I'd only be using the in-Git-config approach, and I don't mind if
 others want something like this.

This is about making this script self-documenting.

When I first tried nmbug-status I got just a stack trace[*], not even a
usage message. Even after running nmbug-status --help I wasn't any
closer to knowing how to run the thing. It wasn't obvious that --config
was the option I needed to get things to work, (from the stack trace,
clearly this script wants some git repository somewhere), and even if I
had know that --config was what I wanted, I had no idea how to construct
a config file.

I tried the documentation for nmbug which had a git clone command in
it, but even then I couldn't get nmbug-status to work, (I tried to
replicate this bug just now, but couldn't---it does seem to be working
after the documented git clone). But even then, all I had was a bare
repository and no clear indication that I needed a config file nor where
to find it.

By the time I had read the code and understood it well enough to know
where to find the config file, I also knew enough to fix the script to
be self-documenting as far as configuration. So that's what I did.

If you run notmuch-to-html with no arguments it will naturally walk
you through everything you need. It will give you functional sample
output before you've even constructed a configuration file. And then it
will give you the exact syntax for that example configuration file.

I can't see how these changes could be anything other than an
improvement.

-Carl

[*] Here it is, for reference:

(master)$ ./nmbug-status 
fatal: Not a git repository: '/home/cworth/.nmbug'
fatal: Not a git repository: '/home/cworth/.nmbug'
Traceback (most recent call last):
  File ./nmbug-status, line 254, in module
config = read_config(path=args.config)
  File ./nmbug-status, line 73, in read_config
return json.load(fp)
  File /usr/lib/python2.7/json/__init__.py, line 290, in load
**kw)
  File /usr/lib/python2.7/json/__init__.py, line 338, in loads
return _default_decoder.decode(s)
  File /usr/lib/python2.7/json/decoder.py, line 366, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File /usr/lib/python2.7/json/decoder.py, line 384, in raw_decode
raise ValueError(No JSON object could be decoded)
ValueError: No JSON object could be decoded



pgpPcVgTDlY6t.pgp
Description: PGP signature
___
notmuch mailing list

Re: Thanks for the nmbug-status program

2014-04-23 Thread Carl Worth
Jani Nikula j...@nikula.org writes:
 On Wed, 23 Apr 2014, W. Trevor King wk...@tremily.us wrote:
 * f55ff7e Drop the --get-query option

 I'd like to keep the in-Git config, so I'm -1 here.

 FWIW I added the --get-query option as a convenience for accessing the
 queries from a nmbug companion shell script (which isn't in the notmuch
 repo though). Without it, one needs to duplicate the code digging the
 config from the nmbug repo and parsing the json (which includes the json
 schema). Doing that basically leads to a python snippet that is a
 subset of what nmbug-status already does.

OK. Color me convinced. I added the option back to the version of the
script that I'm hosting.

-Carl

-- 
carl.d.wo...@intel.com


pgpthKLlFrYQi.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Thanks for the nmbug-status program

2014-04-23 Thread Carl Worth
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

W. Trevor King wk...@tremily.us writes:
 So, “here's a goal to shoot for, and I'm happy with my fork.  If
 someone else wants to land patches like these upstream, I'm also happy
 dropping my fork.” ;). 

More just, Thanks for your code. I used it and happened to change it a
bit while using it. Take a look if you want.

 I think it would be easier to just mail your patches to the list and
 see what sticks.

Easier than what? I might be spending more time on emails now than I did
on any of my changes. And I'm not sure what we're talking about. :-)

I don't have any real plans for hacking on notmuch-to-html more (unless
I run into some problem). And if I do, I'll probably do just what I did
this time, (take a look the latest nmbug stuff and go from there).

But sure, next time I can try to send patches to the list first.

It's just that I never set out to make any changes this time around. I
was just trying to create/publish a web page, and some changes happened
as a side effect. Then I found myself unwilling to publish a page
without a pointer to the actual code that made it (so I had to publish
my changed version). Then I found myself unwilling to publish code
without history (so I had to publish a git repository). Then I found
myself unwilling to publish a git repository without at least some
guidance for a user who tried to run the code, (so I made a few more
changes beyond what I had strictly needed to make my page). So, yes,
some yak-shaving there, but git tells me it was only about one hour.

 That makes sense, but I'd prefer something like:
...
 Heh, that should probably be fixed ;)
...
 I'd prefer a clearer error message when we can't find a config file (I'm
 working up a patch now ;)

All sounds good to me. Thanks for everything you do. Have fun!

- -Carl

- -- 
carl.d.wo...@intel.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBCgAGBQJTV/ruAAoJEGACM7qeVNxhwYkP+wTufLg1PvUCJMqHxFAMQCdM
gy81dXAaJvSSOWkp/A+fsE7i8JNZ96dIrPMHTKRp0dgpM/M89sXT1DHr4w7hNS67
sVmMawlsBElze19E0uTyWrcJuxk9A5QOaweCAWtLwrgacsZBfJ4mdZqWxXWdp8aw
ue8lELuVr3u66GCNYJqDWMgZNT0WPrGVz/3x3SShf+taWKlYHqnO0ESd+nDkU0WW
PxoGWxqdVptbZCLeJ8xuQN73KMiF7rkvuzYiFBqgWi2SZr/4heKP2VRa0G02KRjC
zgJVhKt7W3I/tV4qCOimaVCPL1sqRTNISh3sIJokW3IDLNzObK17/+sdYanxFEhI
RI6iM/hPy9iQ5wKSFa0M69FAk5+T0LQARYMxbzKogjXo55IanTK5gm4Mnh1uG86Z
205b5CzUL1Xetaw10luU9aIn1NAbE5VbNPIn0okNzpV8EkJOYZOFQXfxEjv5ntUQ
fin4n2XpBb4NTenQgSHCO/L8Hl2d6e6HmSSufRgbhBoR3OTmnv94sMhgOBnT/7AM
j5rJ6IHxHLQSvV5uPHUpSpY1xIasNnNgLnwZDYMtULxxhLQAQ4ikaKntDHLi8/+B
rbpIa5JFo4NqBuPs/MZDP4P7XCveLM8PWPJFwf1fh8Q6pVi2K0gbmlXZKq51DrcB
YPLzfFAr422ar7yjGVAl
=a4iT
-END PGP SIGNATURE-
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Thanks for the nmbug-status program

2014-04-22 Thread Carl Worth
Hi David,

I just wanted to give you a quick thank you for the nmbug-status
program. It's really handy to be able to publish decent-looking HTML
pages based on notmuch search results with this.

The script is quite general, (only a tiny portion was tied directly to
nmbug), so I renamed it notmuch-to-html, and hosted it here after a tiny
bit of generalization and a couple of minor user-interface changes:

http://git.cworth.org/git/notmuch-to-html

Feel free to re-adopt this as part of nmbug if you want or to suck it
back into notmuch/contrib or whatever.

Again, thanks!

-Carl

PS. For reference, here are the results I'm currently generating with
the script (for the Mesa project):

http://cworth.org/~cworth/mesa-stable-queue/

You'll see that I'm doing something very similar to what nmbug does,
(even with a compatible tag-naming scheme). In my case, I'm not (yet)
doing any collaborative maintenance of the patch queue, so I don't need
any of the tag-synchronization features that nmbug provides.

I would like to be able to do collaborative tag maintenance in the
future. And I think for that what I'd really like is a standalone tool
that allowed me to synchronize a set of tags (such as, anything with a
msq:: prefix). Of course, that's really close to what nmbug is. I'd
just like to see it be a bit more general, and walk the user more gently
through the setup.

(I'll confess that I tried to hack up nmbug a bit, but failed. There's
some mismatch between me and perl that is apparently not going to be
resolved.)

-- 
carl.d.wo...@intel.com


pgpQX1OhV5isz.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Thanks for the nmbug-status program

2014-04-22 Thread Carl Worth
W. Trevor King wk...@tremily.us writes:
  http://git.cworth.org/git/notmuch-to-html

 Is there a clonable URL for this?

Yes. Sorry I didn't provide that, (and clearly I need to reconfigure my
git web pages to mention the URL):

git clone http://git.cworth.org/git/notmuch-to-html

 I think that making nmbug and nmbug-status generally applicable for
 other projects is a good target.  I don't see any need to pull them
 out into a stand-alone project to do that though.

For what it's worth, I don't disagree.

-- 
carl.d.wo...@intel.com


pgpseKquuCLLa.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: Thanks for the nmbug-status program

2014-04-22 Thread Carl Worth
W. Trevor King wk...@tremily.us writes:
   $ git clone http://git.cworth.org/git/notmuch-to-html

Sorry about that. What I had meant to say is:

  git clone git://git.cworth.org/git/notmuch-to-html

(This time I actually copy-pasted from a terminal where I tested that
the command actually works.)

And now, just in case you say you can talk http, but not git protocol,
I've made this work too:

  git clone http://http-git.cworth.org/git/notmuch-to-html

-Carl

PS. If someone knows magic so that git clients could clone from the same
URL that web browsers can use to get to the web pages, it would be nice
to have that set up.


pgpM0yuwJkWEf.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [RFC PATCH] Re: excessive thread fusing

2014-04-21 Thread Carl Worth
Austin Clements amdra...@mit.edu writes:
 But let me propose an idea I've been kicking around for a while: ghost
 message documents.  Rather than using user metadata for tracking these
 missing messages, use regular documents with the exact same terms we
 use now for message IDs and thread IDs, but with a Tghost term instead
 of a Tmail term to distinguish their type.

For what it's worth. I like this proposed fix very much. It's obviously
what I should have done in the first place.

 In effect, the database becomes truly monotonic.

Yes. All of these benefits are very compelling.

 [1] Curious?
 yes n | xapian-inspect postlist.DB | \
 awk '!/Key/ {next} /Key: \\x00\\xc0thread_id_/ {N++} /Key: \\x00\\xd0/ 
 {exit} END {print N}'

16725 for me.

-Carl



pgpuS8ZLLxizB.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [RFC PATCH] Re: excessive thread fusing

2014-04-20 Thread Carl Worth
Mark Walters markwalters1...@gmail.com writes:
 I have done dome debugging of this.

Thanks for looking closely, Mark!

 There is a patch below which fixes this test case but who knows what
 it breaks! Please DO NOT apply unless someone who knows this code says
 it's OK.

I wrote much of the original code being patched here, so hopefully I
understand it and can say something useful.

I agree that the patch should not be applied. I don't like to see one
piece of code not trusting another in the same code base. If the
parse_references() function doesn't deal well with a malformed header,
then we should fix it, not step around it.

Meanwhile, not treating all potential referenced message IDs
consistently could definitely make the notmuch algorithm more fragile
and sensitive to the order of message indexing, etc. So let's not do
that.

Instead, let's track down and fix the actual bug.

Thanks for the idea of using two-digit names for these messages. That
makes it much easier to inspect the relevant headers.

Below, I've grepped out the actual References and In-Reply-To headers
From the messages, and then simply substituted minimal, and
easy-to-understand values for the message IDs.

With these minimally modified headers, it's easy to manually inspect the
relationships and see that messages 17 and 18 belong in one thread, and
messages 32-52 belong in a separate thread.

It's also quite easy to see the potential source of the bug. The
In-Reply-To headers for messages 18, 32, and 52 all share a common
string (an email address) formatted to look like a message-id,
e.fr...@ucl.ac.uk. If notmuch looks at those headers, and treats
that string as a message-id, then all of theses messages will be
connected into a single thread.

And since that's the reported behavior, it seems likely that
e.fr...@ucl.ac.uk is the cause of this bug.

 I put some debug stuff in _notmuch_database_link_message_to_parents and
 I think that the problem comes from the call to parse_references on line
 1767 which adds the malformed in-reply-to header to the hash table, so
 this malformed line gets added as a potential parent. 

Am I correct that your debugging showed that e.fr...@ucl.ac.uk is
being added to the hash table?

My inspection of _parse_references() and parse_message_id() suggests
that that's exactly what notmuch is doing, (treating both of the
angle-bracketed portions (e.fr...@ucl.ac.uk as well as the actual
message-ID, ID17 or ID31 or ID39) as message IDs.

So it seems like we need a new _parse_in_reply_to() function to use in
place of _parse_references() and the new function will need a better
heuristic for dealing with the unpredictability of In-Reply-To.

The only real reason that we are trying to grab multiple message ID
values from an In-Reply-To header is that RFC 2822 explicitly allows
that, (to support a message simultaneously replying to multiple
messages). I don't believe that that's common, but we might as well
support it. At the same time, RFC 2822 also explicitly specifies that
the In-Reply-To header will consist of nothing but message IDs.

So perhaps the heuristic here could be to notice any characters outside
of angle brackets, (like Message in the headers below), and in that
case go to a strictly not RFC 2822 mode and look for exactly one
message ID. At that point, JWZ would recommend the first -bracketed
text found therein, but that would give precisely the wrong answer in
this particular case. Here the correct Message ID appears in the last
-bracketed text. I have not surveyed a large email corpus to determine
how often last -bracketed text would fail as a heuristic.

Another idea would be to trigger specifically on common forms. Judging
From the samples in this particular thread, it seems like a workable
heuristic would be:

If the In-Reply-To header begins with '':

Parse that initial portion as a message ID

Else if it ends with '':

Parse that final portion as a message ID

Else

Ignore this garbage-valued header.

That's probably the best and most reliably thing to do here.

Does anyone have any better ideas?

 As a clear example that I don't understand this code I don't know why
 this no longer causes a problem if message 17 gets added too.

I wanted to test my own knowledge of the code to see if I could explain
this. But I didn't precisely follow your explanation of the behavior you
saw. In cases (1) and (2) of your description, what order are you using
to add all messages or add all apart from 52?

Then, for cases (3) and (4), what is done before adding the messages
mentioned in these cases? Add all other messages? Again, in what order?

I haven't tracked through all the logic of the existing algorithm for
this case. But I don't like hearing that notmuch constructs different
threads for the same messages presented in different orders. This sounds
like a bug separate from what we've discussed above. 

-Carl

18:References: ID17

[PATCH] emacs: notmuch-hello: Add variable to control order or saved-search results

2014-02-06 Thread Carl Worth
This allows for the user to control the order that results are
returned from saved-searches, (and independently from the order of all
other searches which is controlled by notmuch-search-oldest-first).
---

This allows me to get back to the behavior that notmuch had long ago,
(the documentation for the changed behavior was updated in commit
c1a42652a173a4bb70ab72388e6ad150d19a2b06). I'm not sure when the
actual behavioral change was made.

A fanicer change might allow for a configurable sort order for each
saved search, but this more minimal change at least lets me use the
latest notmuch once again.

 emacs/notmuch-hello.el | 18 +-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/emacs/notmuch-hello.el b/emacs/notmuch-hello.el
index 7b3d76b..a9f8230 100644
--- a/emacs/notmuch-hello.el
+++ b/emacs/notmuch-hello.el
@@ -59,6 +59,14 @@ alist to be used.
   :value notmuch-sort-saved-searches))
   :group 'notmuch-hello)
 
+(defcustom notmuch-saved-searches-oldest-first t
+  Show the oldest mail first for saved-search results.
+
+This variable defines the default sort order for displaying
+saved-search results.
+  :type 'boolean
+  :group 'notmuch-hello)
+
 (defvar notmuch-hello-indent 4
   How much to indent non-headers.)
 
@@ -327,6 +335,14 @@ diagonal.
  :notmuch-search-terms)
  notmuch-search-oldest-first))
 
+; Like notmuch-hello-widget-search, but with the
+; notmuch-saved-searches-oldest-first variable controlling the sort
+; order.
+(defun notmuch-hello-do-saved-search (widget rest ignore)
+  (notmuch-search (widget-get widget
+ :notmuch-search-terms)
+ notmuch-saved-searches-oldest-first))
+
 (defun notmuch-saved-search-count (search)
   (car (process-lines notmuch-command count search)))
 
@@ -460,7 +476,7 @@ Such a list can be computed with 
`notmuch-hello-query-counts'.
(widget-insert (format %8s 
   (notmuch-hello-nice-number msg-count)))
(widget-create 'push-button
-  :notify #'notmuch-hello-widget-search
+  :notify #'notmuch-hello-do-saved-search
   :notmuch-search-terms query
   name)
(setq column-indent
-- 
1.9.rc1

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 0/5] lib: make folder: prefix literal

2014-02-04 Thread Carl Worth
Austin Clements amdra...@mit.edu writes:
 Agreed.  I believe this will also support MH, if I understand MH
 correctly (does anyone actually use MH?)

When I started notmuch, I had all of my mail in one-message-per-file in
various directories, (without these silly cur and new directories
that maildir uses).

At some point, I did a mass conversion of all of my directories to be
roughly:

~/mail//MM

So that I keep directories small by just delivering a month's worth of
mail to each directory. This conversion, (and the delivery agent I am
currently using, maildrop), happen to create the silly cur and new
directories.

So most of my mail still is in maildir format now.

But I do have a few messages in non-maildir directories. These have
generally come into being in cases such as someone providing me a
message to demonstrate a notmuch bug or use case. So in cases like this
I did things like:

mkdir ~/mail/bug-description
cp example-file ~/mail/bug-description

I also have a few directories created similarly when I've copied some
downloaded archives from a mailing list into my mail storage. (But often
I've used mb2md for those so the conversion has accidentally created
maildir directories).

I don't know if the non-maildir directories I have are strict mh
format, (did it have filenames with sequential numbers? I don't
recall).

But my intention with notmuch from the beginning was to support any
one-message-per-file layout without enforcing any particular naming of
directories or files. And I would like to see that preserved.

Since then, we have also supported various semantics when people do
encode information in directories and filenames, (such as ignoring
cur/new and interpreting maildir flags). This kind of thing does
seem good.

-Carl


pgpZXkuCQyczA.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 0/5] lib: make folder: prefix literal

2014-01-29 Thread Carl Worth
Jani Nikula j...@nikula.org writes:
 Unfortunately, I haven't had the time to experiment with this. But it
 bugs me that the probabilistic folder: prefix has stemming and it's case
 insensitive. It's possible to work around the stemming with the anchors
 you suggest or by quoting, but is there a way to have case sensitive
 probabilistic prefixes?

The stemming and case insensitivity just has to do with which terms are
shoved into the database, (you have to add extra terms to get these
features). If we're getting those features for folder now, (and I agree
that we don't want them), it's because we're calling some Xapian
convenience function along the lines of create a bunch of terms for
this chunk of text.

The fix for that is to do the simple thing and simply break the path at
each '/' and add a term for each component. Then these problems all go
away.

So fixes for this should not require switching from a probabilistic to a
Boolean prefix.

-Carl

-- 
carl.d.wo...@intel.com


pgpO4oiDDTjcx.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH 0/5] lib: make folder: prefix literal

2014-01-25 Thread Carl Worth
Jani Nikula j...@nikula.org writes:
 Here's a thought. With boolean prefix folder:, we can devise a scheme
 where the folder: query defines what is to be matched.

I like the idea, but I tried to infer the rules from the examples, and I
failed. It looks like there are two new symbols, / and /. but I
couldn't decipher the exact semantics of each.

I think a proposal like this should not re-use the '/' symbol as we
already have that as a path divider. (See rsync for lots of user
confusion with a significant trailing '/').

I propose a similar, but slightly different approach, where we add two
additional symbols:

  '^'   Matches the beginning of a path

  '$'   Matches the end of a path

[Obviously, I chose these symbols from regular expressions. I would be
OK with alternate symbols, ('$' seems like it might be problematic in
the shell, but perhaps not too much if it's always at the end of a
phrase.)]

This way, one could search for:

  folder:fooWorks like folder: historically

  folder:^full/path$Works like Jani's proposal

  folder:^path/prefix   Satisfies Tomi's use case, (as well as anyone
who doesn't want to have to specify or
distinguish between /cur or /new.

Any extra '/' at the beginning or end of a search string, (such as
folder:^/full/path/$) would not change the semantics.

Further, I think we can implement this with less database bloat by
leaving folder as probabilistic and simply indexing two new terms to
indicate the beginning of the path and the end of the path.

Finally, we could also extend the scheme to other things like subject:
to allow for an exact subject search like:

subject:^lib: make folder: prefix literal$

It was with an eye toward something like this that I chose to make
folder: probabilistic in the first place. (I probably would have indexed
things appropriately in the first place as well, but at the time doing
the necessary query parsing for '^' and '$' seemed daunting).

-Carl

-- 
carl.d.wo...@intel.com


pgpGvAffMislw.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] debian: package ruby bindings

2013-04-29 Thread Carl Worth
Felipe Contreras felipe.contre...@gmail.com writes:
 My hypothetical laziness is not the reason. It was at the strong
 insistence of Carl that things are organized this way.

 I still don't see why we need to do that. The notmuch project
 shouldn't be bound to the debian project.

I'm not sure what your specific question is here.

My strong insistence was that the Debian packaging stuff, (like
debian/rules), be hosted in the main notmuch repository.

If that were separated to a separate repository (or a separate branch),
then nothing in the current situation would change. Your proposed
change, (which is to debian/rules if I read it correctly), would instead
be directed at that separate repository/branch.

And either way, David's currently the person to decide what lands there.

Is there anything I'm missing here?

-Carl


pgpbehbODblKH.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] bitmap:improve memory usage using CHAR_BITS and unsigned CHAR

2013-02-13 Thread Carl Worth
Robert Mast  writes:
> Re-submission for comment-reasons.

Very nicely done, Robert. Thanks.

-Carl (who's not quite merging patches again yet, but likely will be soon)

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Search using email headers does not work

2013-02-12 Thread Carl Worth
Michal Vyskocil  writes:
> I like such idea - but how hard would be to add a config options to
> extend a list of default headers used for indexing?

Not hard.

Obviously, notmuch is already indexing some headers, (From, Date, To,
Subject), and it wouldn't be hard to index more. [*]

List-Id has been proposed often enough that it should probably be on by
default.

Then, there are other, slightly more special-case headers, (like headers
added by spam filtering software), that probably would be best added on
an opt-in basis based on a configuration option.

-Carl

[*] The existing set of indexed headers is really just historical
accident based on two things:

1. I was originally writing index code to match sup's index building, so
   I focused on the headers sup indexed.

2. I wrote code to index enough headers to satisfy my use case.

It's embarrassing to me that I added this note to the notmuch TODO file
in October 2009 (and it's still there):

Add support for the user to specify custom headers to be indexed.

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Reply all - issue

2013-02-12 Thread Carl Worth
Jameson Graef Rollins  writes:
> Just a thought: what if messages with a given tag (e.g. "new-thread")
> were always treated as the source of a new thread?

It's a good start. And an approach like that would have the advantage
that one could undo a thread-split by just removing the tag. (That's not
an explicit thread-join feature, but I don't think anyone has ever asked
for that.)

> A message with the given tag could just be (re)indexed with any
> In-Reply-To/References headers stripped before indexing.

It would require a little more than that. Imagine this thread:

  A: Subject: An original thread
  ??B: Subject: Thread hijacking is fun (tag:new-thread)
??C: Subject: Re: Thread hijacking is fun

In this case, message C is likely to have a References header that
mentions both A and B. So the thread stitching logic in notmuch will
want to merge threads A and B when indexing C. So special care will have
to be taken here as well, (not just when indexing B).

And that special care may not be cheap if it requires additional
database lookups for each unique thread ID encountered among references
of a message.

Though, I don't mean to dissuade anyone from thinking this through and
coding it up. The relevant code for the pieces I'm referring to starts
in _notmuch_database_link_message in lib/database.cc.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



RE: Reply all - issue

2013-02-12 Thread Carl Worth
Jameson Graef Rollins jroll...@finestructure.net writes:
 Just a thought: what if messages with a given tag (e.g. new-thread)
 were always treated as the source of a new thread?

It's a good start. And an approach like that would have the advantage
that one could undo a thread-split by just removing the tag. (That's not
an explicit thread-join feature, but I don't think anyone has ever asked
for that.)

 A message with the given tag could just be (re)indexed with any
 In-Reply-To/References headers stripped before indexing.

It would require a little more than that. Imagine this thread:

  A: Subject: An original thread
  └─B: Subject: Thread hijacking is fun (tag:new-thread)
└─C: Subject: Re: Thread hijacking is fun

In this case, message C is likely to have a References header that
mentions both A and B. So the thread stitching logic in notmuch will
want to merge threads A and B when indexing C. So special care will have
to be taken here as well, (not just when indexing B).

And that special care may not be cheap if it requires additional
database lookups for each unique thread ID encountered among references
of a message.

Though, I don't mean to dissuade anyone from thinking this through and
coding it up. The relevant code for the pieces I'm referring to starts
in _notmuch_database_link_message in lib/database.cc.

-Carl

-- 
carl.d.wo...@intel.com


pgptx_atmDvLv.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] convert bitmap to unsigned char

2013-02-11 Thread Carl Worth
Robert Mast  writes:
> ---

Hi Robert,

It looks like the git exercise is proving useful. Keep it up!

If you look through the git logs a bit you'll find that the overwhelming
convention is for a commit message to have a single-line summary
followed by a (potentially longer) expanded comment.

The convention I like best is for the single-line summary to
efficiently  describe everything about "what" the patch does. If it's
hard to fit this into a single line there's a good chance your patch
should be split up. With your commit message here, the one-line summary
is great, (and much better than in your first submission).

The expanded comment should describe the "why" of the change. And here,
your commit message doesn't have anything. So I'm left wondering why
your commit exists. Does this save memory? Does this fix some type error
or expected alignment somewhere? etc.

Please re-submit your patch with a little more explanation of the
motivation behind the patch.

-Carl

PS. I do know that you did have more in the commit message originally,
and Austin recommended removing "snide" issues, (doubts and
meta-questions about the patch---things that don't belong in the commit
history).

Your previous commit message was:

  Reading it in detail I thought it allocated way too much memory and
  didn't use the full size of the allocated unsigned ints for storing
  bits.

  Am I right, and is this the right way to patch code to notmuch?

I'm not actually looking at the code in context now, so I can't render a
correct commit message, but this attempted rewording should hopefully
give you the idea:

  Using char instead of int allows for simpler definitions of the
  DOCIDSET macros so the code is easier to understand.

  ---

  Am I reading that correctly? Or is there also some space saving
  happening here as well? Please re-submit a new patch with a commit
  message that makes things clear one way or the other.

Do you see how this "why" part of the commit message is ready to stand
alone in our commit history (assuming it's correct and
accepted). Meta-questions like "is this even a sane way of doing
things?" are great to ask, and very helpful for code reviewers, but best
placed after the "---" so that they don't become part of the commit
message.

Thanks for playing with notmuch!

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Re: [PATCH] convert bitmap to unsigned char

2013-02-11 Thread Carl Worth
Robert Mast beheer...@tekenbeetziekten.nl writes:
 ---

Hi Robert,

It looks like the git exercise is proving useful. Keep it up!

If you look through the git logs a bit you'll find that the overwhelming
convention is for a commit message to have a single-line summary
followed by a (potentially longer) expanded comment.

The convention I like best is for the single-line summary to
efficiently  describe everything about what the patch does. If it's
hard to fit this into a single line there's a good chance your patch
should be split up. With your commit message here, the one-line summary
is great, (and much better than in your first submission).

The expanded comment should describe the why of the change. And here,
your commit message doesn't have anything. So I'm left wondering why
your commit exists. Does this save memory? Does this fix some type error
or expected alignment somewhere? etc.

Please re-submit your patch with a little more explanation of the
motivation behind the patch.

-Carl

PS. I do know that you did have more in the commit message originally,
and Austin recommended removing snide issues, (doubts and
meta-questions about the patch---things that don't belong in the commit
history).

Your previous commit message was:

  Reading it in detail I thought it allocated way too much memory and
  didn't use the full size of the allocated unsigned ints for storing
  bits.

  Am I right, and is this the right way to patch code to notmuch?

I'm not actually looking at the code in context now, so I can't render a
correct commit message, but this attempted rewording should hopefully
give you the idea:

  Using char instead of int allows for simpler definitions of the
  DOCIDSET macros so the code is easier to understand.

  ---

  Am I reading that correctly? Or is there also some space saving
  happening here as well? Please re-submit a new patch with a commit
  message that makes things clear one way or the other.

Do you see how this why part of the commit message is ready to stand
alone in our commit history (assuming it's correct and
accepted). Meta-questions like is this even a sane way of doing
things? are great to ask, and very helpful for code reviewers, but best
placed after the --- so that they don't become part of the commit
message.

Thanks for playing with notmuch!

-Carl

-- 
carl.d.wo...@intel.com


pgp_QJvrmT3C4.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


It's good to be "back"!

2013-01-29 Thread Carl Worth
Elsewhere, in a non-hijacked thread, Carl Worth wrote:
> Is there any existing thread-breaking? There wasn't the last time I
> looked at the code closely, (but admittedly, that was a while ago).

Hi folks,

I've been away from any active involvement in the notmuch community for
far too long.

It turns out that even with an email system as sophisticated as notmuch,
it's possible to slip into bad habits that allow one to get behind with
email. So I got behind.

I also had some difficulty in accepting the fact that I wasn't able to
be as directly involved with notmuch as I was in the beginning. Early on
I would read every message sent to the list and try to ensure that every
question received at least some answer, (whether from me or for someone
else). But of course, I never succeeded at actually keeping up with
that, so my queues kept getting bigger and more intimidating, etc.

Anyway, it's been about 18 months since I've done anything real with
notmuch. I'm so delighted to see that the community that built up around
notmuch has been so independent, doing great things in my absence. I'm
excited to start looking close at the new features that I've not played
with (nmbug, date parsing, etc.), and maybe start implementing some of
those ancient features that I never got around to implementing in the
first place, (indexing of List-Id headers?).

So a huge thank you to all of you!

When I got over the psychological hurdles and decided to just archive
away the old, neglected mail queues, notmuch at least did make that
really easy. And I've now got an email workflow going really well for
me, that I think will be really useful. Notmuch allows the workflow
quite well, but doesn't guide me through it as smoothly as I would
like. More on that in a later message.

But for now, I should be reading the list much more regularly going
forward, and I'll be happy to have people CC me directly when they want
my particular attention for some specific reason.

And of course, all of the 12,000+ [*] messages I missed in the last
18 months[*] are still present in my archives. So new replies to old
messages might bring things to my attention. People can use that
intentionally if it will be useful.

I don't know how much code I'll be committing on a regular basis. But I
will definitely be "around" now much more.

Thanks again for all you've done over the last while. And thanks in
advance for what I know will be a warm welcome.

-Carl

[*] Hurrah for date searches! It's wonderful to have this feature!!
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20130129/3705c869/attachment.pgp>


Reply all - issue

2013-01-29 Thread Carl Worth
Robert Mast  writes:
> Your point on patch-breaking related to gmail and my proposal isn't
> completely clear to me, but I've probably addressed it well with my new
> approach.

The issue here is that many developers tend to develop a patch series
(perhaps with dozens of patches) as a single conceptual unit. When these
are emailed out, they are often sent as one thread with a new subject
for every patch. In particular, users of git and "git send-email" often
send patches this way. For what it's worth, it's my preferred way to
send and receive patches via email.

It's extremely useful for messages like this to be presented as a
single thread. This means that the dozens of messages don't clutter the
inbox, and it also allows for an operation to act on all of the messages
at once, (for example, notmuch provides "C-u |" which can apply all of
the received patches to a code repository in a single operation).

So, those of us accustomed to sending, receiving, reviewing, and
applying patches emailed in this way would be basically unable to use an
email program that split threads unconditionally on subject changes.

So it may be tricky to find a single behavior that would make everyone
happy. Perhaps a configuration option for splitting threads on subject
changes.

> I'll study the code for adding the option of unconditional (stripped)
> subject breaking on top of the existing thread-breaking.

Is there any existing thread-breaking? There wasn't the last time I
looked at the code closely, (but admittedly, that was a while ago).

-Carl

-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



RE: Reply all - issue

2013-01-28 Thread Carl Worth
Robert Mast beheer...@tekenbeetziekten.nl writes:
 Your point on patch-breaking related to gmail and my proposal isn't
 completely clear to me, but I've probably addressed it well with my new
 approach.

The issue here is that many developers tend to develop a patch series
(perhaps with dozens of patches) as a single conceptual unit. When these
are emailed out, they are often sent as one thread with a new subject
for every patch. In particular, users of git and git send-email often
send patches this way. For what it's worth, it's my preferred way to
send and receive patches via email.

It's extremely useful for messages like this to be presented as a
single thread. This means that the dozens of messages don't clutter the
inbox, and it also allows for an operation to act on all of the messages
at once, (for example, notmuch provides C-u | which can apply all of
the received patches to a code repository in a single operation).

So, those of us accustomed to sending, receiving, reviewing, and
applying patches emailed in this way would be basically unable to use an
email program that split threads unconditionally on subject changes.

So it may be tricky to find a single behavior that would make everyone
happy. Perhaps a configuration option for splitting threads on subject
changes.

 I'll study the code for adding the option of unconditional (stripped)
 subject breaking on top of the existing thread-breaking.

Is there any existing thread-breaking? There wasn't the last time I
looked at the code closely, (but admittedly, that was a while ago).

-Carl



pgpsTAUNjRnwq.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


It's good to be back!

2013-01-28 Thread Carl Worth
Elsewhere, in a non-hijacked thread, Carl Worth wrote:
 Is there any existing thread-breaking? There wasn't the last time I
 looked at the code closely, (but admittedly, that was a while ago).

Hi folks,

I've been away from any active involvement in the notmuch community for
far too long.

It turns out that even with an email system as sophisticated as notmuch,
it's possible to slip into bad habits that allow one to get behind with
email. So I got behind.

I also had some difficulty in accepting the fact that I wasn't able to
be as directly involved with notmuch as I was in the beginning. Early on
I would read every message sent to the list and try to ensure that every
question received at least some answer, (whether from me or for someone
else). But of course, I never succeeded at actually keeping up with
that, so my queues kept getting bigger and more intimidating, etc.

Anyway, it's been about 18 months since I've done anything real with
notmuch. I'm so delighted to see that the community that built up around
notmuch has been so independent, doing great things in my absence. I'm
excited to start looking close at the new features that I've not played
with (nmbug, date parsing, etc.), and maybe start implementing some of
those ancient features that I never got around to implementing in the
first place, (indexing of List-Id headers?).

So a huge thank you to all of you!

When I got over the psychological hurdles and decided to just archive
away the old, neglected mail queues, notmuch at least did make that
really easy. And I've now got an email workflow going really well for
me, that I think will be really useful. Notmuch allows the workflow
quite well, but doesn't guide me through it as smoothly as I would
like. More on that in a later message.

But for now, I should be reading the list much more regularly going
forward, and I'll be happy to have people CC me directly when they want
my particular attention for some specific reason.

And of course, all of the 12,000+ [*] messages I missed in the last
18 months[*] are still present in my archives. So new replies to old
messages might bring things to my attention. People can use that
intentionally if it will be useful.

I don't know how much code I'll be committing on a regular basis. But I
will definitely be around now much more.

Thanks again for all you've done over the last while. And thanks in
advance for what I know will be a warm welcome.

-Carl

[*] Hurrah for date searches! It's wonderful to have this feature!!


pgpC4BQgkXm04.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] Restore original keybinding ('r' = reply-to-all)

2012-06-28 Thread Carl Worth
Philip Hands  writes:
> I find the change to the new (only reply to sender) behaviour serously
> irritating, because it seems I cannot train myself to hit R all the time
> (which is pretty much what I always want).

I'm in a similar camp. I tried (and failed) to adopt to the current
mode, (once I realized that the change had happened and that I had been
mis-replying).

I think part of the problem with training here is that if your mental
model is "I generally want to reply to all, and exceptionally want to
reply to only some" then the 'r == reply-to-sender' often does do what
you want. That is, when the CC list is empty, reply-to-sender is no
different than reply-to-all. So there's some mis-training that occurs,
('r' seems to do what I want). Worse, the results of the mis-training
are hidden from the user, (since if the user didn't pay attention to the
CC list before hitting 'r', the unintentional culling of recipients is
hidden within the composition buffer).

For me, personally, I tend to do a final examination of my message just
before sending. This is a quick scan to look for typos, make sure my
message is clear, etc. During part of this scan, it's also natural for
me to ensure that there isn't anyone in the recipient list that
shouldn't be receiving the message. If I do decide to remove some
recipients from my message, this is a natural time for me to do it, (or
perhaps earlier, during composition, when first writing something
sensitive).

So, for me, making a decision *before* writing anything doesn't fit my
mental model, (I'll often change the tone of my message while composing,
and in ways that the recipient list might change).

I also find reply-to-sender-only too limiting of an operation, (compared
to reply-to-all followed by header editing). For example, sometimes I
might want to drop some smaller subset of recipients from my reply.

My workflow is definitely influenced my the habits of coworkers in my
corporate environment. While there are many mailing-list addresses,
there are often large threads involving ad-hoc recipient lists. And as
conversations go, some groups of individuals needs to be added or
removed from the discussion. Header editing works for that, in a way
that reply-to-sender doesn't help.

> On the other hand, I'm perfectly capable of customising this, but have
> something of a fetish for at least trying to live with defaults for a
> period, so it's my own fault for putting up with it.

I'm obviously capable of making a customization as well, (and have done
so). The current mechanism[*] I'm using for this customization is
particularly clumsy, (it's not exposed in the customize buffer, it
requires the user to know the names of internal objects like
notmuch-show-mode-map and notmuch-search-reply-to-thread-sender), and it
requires the user to change 4 settings, (in two separate places), in
order to get a consistent experience, (so it would be easy to
accidentally make search-mode and show-mode behave differently).

There's clearly difference of opinion on what the defaults should be. So
at the very least, we should make it easier to customize this.

How about the following:

  With no customization in place, the first time the user hits 'r',
  notmuch prompts with something like:

Reply to all or sender only? [asAS?]:

  Hitting '?' would the provide more instructions:

'a': Reply to all recipients for this reply.
's': Reply to sender-only for this reply.
'A': Reply to all recipients now and in the future (no questions asked)
'S': Reply to sender-only now and in the future (no questions asked)

 Note: After setting a default behavior with 'A' or 'S' here, the
 alternate behavior can still be obtained by initiating a reply with
 'R' rather than 'r'.

That would satisfy me as being sufficiently easy-to-use and sufficiently
self-documenting.

It also as the advantage of letting us make a change now without
tripping up any users.

I think the implementation should function by setting a single
customize-based variable. But care should be taken such that the notmuch
help modes still correctly describe what 'r' and 'R' do depending on how
this variable is configured. My current approach of setting a preference
by changing the keybindings yields correct documentation "for
free". It's probably a little trickier to get that correct documentation
with a single variable controlling things, but I hope it's not too hard.

Anyone care to attempt an implementation of this?

-Carl

[*] Here's what I'm using now:

(define-key notmuch-show-mode-map "r" 'notmuch-show-reply)
(define-key notmuch-show-mode-map "R" 'notmuch-show-reply-sender)
(define-key notmuch-search-mode-map "r" 'notmuch-search-reply-to-thread)
(define-key notmuch-search-mode-map "R" 'notmuch-search-reply-to-thread-sender)
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 

Re: [PATCH] Restore original keybinding ('r' = reply-to-all)

2012-06-28 Thread Carl Worth
Philip Hands p...@hands.com writes:
 I find the change to the new (only reply to sender) behaviour serously
 irritating, because it seems I cannot train myself to hit R all the time
 (which is pretty much what I always want).

I'm in a similar camp. I tried (and failed) to adopt to the current
mode, (once I realized that the change had happened and that I had been
mis-replying).

I think part of the problem with training here is that if your mental
model is I generally want to reply to all, and exceptionally want to
reply to only some then the 'r == reply-to-sender' often does do what
you want. That is, when the CC list is empty, reply-to-sender is no
different than reply-to-all. So there's some mis-training that occurs,
('r' seems to do what I want). Worse, the results of the mis-training
are hidden from the user, (since if the user didn't pay attention to the
CC list before hitting 'r', the unintentional culling of recipients is
hidden within the composition buffer).

For me, personally, I tend to do a final examination of my message just
before sending. This is a quick scan to look for typos, make sure my
message is clear, etc. During part of this scan, it's also natural for
me to ensure that there isn't anyone in the recipient list that
shouldn't be receiving the message. If I do decide to remove some
recipients from my message, this is a natural time for me to do it, (or
perhaps earlier, during composition, when first writing something
sensitive).

So, for me, making a decision *before* writing anything doesn't fit my
mental model, (I'll often change the tone of my message while composing,
and in ways that the recipient list might change).

I also find reply-to-sender-only too limiting of an operation, (compared
to reply-to-all followed by header editing). For example, sometimes I
might want to drop some smaller subset of recipients from my reply.

My workflow is definitely influenced my the habits of coworkers in my
corporate environment. While there are many mailing-list addresses,
there are often large threads involving ad-hoc recipient lists. And as
conversations go, some groups of individuals needs to be added or
removed from the discussion. Header editing works for that, in a way
that reply-to-sender doesn't help.

 On the other hand, I'm perfectly capable of customising this, but have
 something of a fetish for at least trying to live with defaults for a
 period, so it's my own fault for putting up with it.

I'm obviously capable of making a customization as well, (and have done
so). The current mechanism[*] I'm using for this customization is
particularly clumsy, (it's not exposed in the customize buffer, it
requires the user to know the names of internal objects like
notmuch-show-mode-map and notmuch-search-reply-to-thread-sender), and it
requires the user to change 4 settings, (in two separate places), in
order to get a consistent experience, (so it would be easy to
accidentally make search-mode and show-mode behave differently).

There's clearly difference of opinion on what the defaults should be. So
at the very least, we should make it easier to customize this.

How about the following:

  With no customization in place, the first time the user hits 'r',
  notmuch prompts with something like:

Reply to all or sender only? [asAS?]:

  Hitting '?' would the provide more instructions:

'a': Reply to all recipients for this reply.
's': Reply to sender-only for this reply.
'A': Reply to all recipients now and in the future (no questions asked)
'S': Reply to sender-only now and in the future (no questions asked)

 Note: After setting a default behavior with 'A' or 'S' here, the
 alternate behavior can still be obtained by initiating a reply with
 'R' rather than 'r'.

That would satisfy me as being sufficiently easy-to-use and sufficiently
self-documenting.

It also as the advantage of letting us make a change now without
tripping up any users.

I think the implementation should function by setting a single
customize-based variable. But care should be taken such that the notmuch
help modes still correctly describe what 'r' and 'R' do depending on how
this variable is configured. My current approach of setting a preference
by changing the keybindings yields correct documentation for
free. It's probably a little trickier to get that correct documentation
with a single variable controlling things, but I hope it's not too hard.

Anyone care to attempt an implementation of this?

-Carl

[*] Here's what I'm using now:

(define-key notmuch-show-mode-map r 'notmuch-show-reply)
(define-key notmuch-show-mode-map R 'notmuch-show-reply-sender)
(define-key notmuch-search-mode-map r 'notmuch-search-reply-to-thread)
(define-key notmuch-search-mode-map R 'notmuch-search-reply-to-thread-sender)


pgp2bTFGturSn.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch

[PATCH] Restore original keybinding ('r' = reply-to-all)

2012-06-27 Thread Carl Worth
Since the beginning of time, the emacs interface provided a keybinding
of 'r' to reply to a message, (and originally, all recipients).

Then, before release 0.12 the emacs interface acquired a new
reply-to-sender only feature. In commit
f02b475fa781bb5df3358c73213e7633a99f016e the new feature was put onto
the original keybinding, (and reply-to-all was moved to 'R').

This restores the original keybinding and uses the new keybinding for
the new feature.
---

The commit message above motivates this change from the point of view
of interface compatibility. That argument would have been stronger if
I had made it in January, (at the time of the change), rather than
now, (since there's now a new incompatiblity made by changing things
back).

That said, I still think the compatibility argument is sound.

I have other feelings as to which reply behavior should be treated
preferentially. But those are potentially more controversial, so I'll
leave the bikeshedding out of our commit history.

Thanks to everyone keeping notmuch alive and well during my extended
absence from maintainership. What a tremendous free-software success
story this all is. I'm so grateful to all of you.

-Carl

 emacs/notmuch-show.el |4 ++--
 emacs/notmuch.el  |4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 4349836..399a8bd 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1145,8 +1145,8 @@ reset based on the original query."
(define-key map "s" 'notmuch-search)
(define-key map "m" 'notmuch-mua-new-mail)
(define-key map "f" 'notmuch-show-forward-message)
-   (define-key map "r" 'notmuch-show-reply-sender)
-   (define-key map "R" 'notmuch-show-reply)
+   (define-key map "r" 'notmuch-show-reply)
+   (define-key map "R" 'notmuch-show-reply-sender)
(define-key map "|" 'notmuch-show-pipe-message)
(define-key map "w" 'notmuch-show-save-attachments)
(define-key map "V" 'notmuch-show-view-raw-message)
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index c6236db..3a6b025 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -215,8 +215,8 @@ For a mouse binding, return nil."
 (define-key map ">" 'notmuch-search-last-thread)
 (define-key map "p" 'notmuch-search-previous-thread)
 (define-key map "n" 'notmuch-search-next-thread)
-(define-key map "r" 'notmuch-search-reply-to-thread-sender)
-(define-key map "R" 'notmuch-search-reply-to-thread)
+(define-key map "r" 'notmuch-search-reply-to-thread)
+(define-key map "R" 'notmuch-search-reply-to-thread-sender)
 (define-key map "m" 'notmuch-mua-new-mail)
 (define-key map "s" 'notmuch-search)
 (define-key map "o" 'notmuch-search-toggle-order)
-- 
1.7.10



[PATCH] Restore original keybinding ('r' = reply-to-all)

2012-06-27 Thread Carl Worth
Since the beginning of time, the emacs interface provided a keybinding
of 'r' to reply to a message, (and originally, all recipients).

Then, before release 0.12 the emacs interface acquired a new
reply-to-sender only feature. In commit
f02b475fa781bb5df3358c73213e7633a99f016e the new feature was put onto
the original keybinding, (and reply-to-all was moved to 'R').

This restores the original keybinding and uses the new keybinding for
the new feature.
---

The commit message above motivates this change from the point of view
of interface compatibility. That argument would have been stronger if
I had made it in January, (at the time of the change), rather than
now, (since there's now a new incompatiblity made by changing things
back).

That said, I still think the compatibility argument is sound.

I have other feelings as to which reply behavior should be treated
preferentially. But those are potentially more controversial, so I'll
leave the bikeshedding out of our commit history.

Thanks to everyone keeping notmuch alive and well during my extended
absence from maintainership. What a tremendous free-software success
story this all is. I'm so grateful to all of you.

-Carl

 emacs/notmuch-show.el |4 ++--
 emacs/notmuch.el  |4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 4349836..399a8bd 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -1145,8 +1145,8 @@ reset based on the original query.
(define-key map s 'notmuch-search)
(define-key map m 'notmuch-mua-new-mail)
(define-key map f 'notmuch-show-forward-message)
-   (define-key map r 'notmuch-show-reply-sender)
-   (define-key map R 'notmuch-show-reply)
+   (define-key map r 'notmuch-show-reply)
+   (define-key map R 'notmuch-show-reply-sender)
(define-key map | 'notmuch-show-pipe-message)
(define-key map w 'notmuch-show-save-attachments)
(define-key map V 'notmuch-show-view-raw-message)
diff --git a/emacs/notmuch.el b/emacs/notmuch.el
index c6236db..3a6b025 100644
--- a/emacs/notmuch.el
+++ b/emacs/notmuch.el
@@ -215,8 +215,8 @@ For a mouse binding, return nil.
 (define-key map  'notmuch-search-last-thread)
 (define-key map p 'notmuch-search-previous-thread)
 (define-key map n 'notmuch-search-next-thread)
-(define-key map r 'notmuch-search-reply-to-thread-sender)
-(define-key map R 'notmuch-search-reply-to-thread)
+(define-key map r 'notmuch-search-reply-to-thread)
+(define-key map R 'notmuch-search-reply-to-thread-sender)
 (define-key map m 'notmuch-mua-new-mail)
 (define-key map s 'notmuch-search)
 (define-key map o 'notmuch-search-toggle-order)
-- 
1.7.10

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


notmuch git's disk is full! ERROR pushing.

2012-01-08 Thread Carl Worth
On Sun, 08 Jan 2012 13:55:53 +0100, Sebastian Spaeth  
wrote:
> Actually trying to push the above 2 patches, I get 
> 
> Writing objects: 100% (8/8), 1.25 KiB, done.
> Total 8 (delta 5), reused 0 (delta 0)
> error: unable to create temporary sha1 filename : No space left on
> device

My apologies!

I've relieved the immediate problem for now. (And I'm hoping to soon
move notmuchmail.org to a different server with a disk that's a couple
of orders of magnitude larger.)

-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Re: notmuch git's disk is full! ERROR pushing.

2012-01-08 Thread Carl Worth
On Sun, 08 Jan 2012 13:55:53 +0100, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 Actually trying to push the above 2 patches, I get 
 
 Writing objects: 100% (8/8), 1.25 KiB, done.
 Total 8 (delta 5), reused 0 (delta 0)
 error: unable to create temporary sha1 filename : No space left on
 device

My apologies!

I've relieved the immediate problem for now. (And I'm hoping to soon
move notmuchmail.org to a different server with a disk that's a couple
of orders of magnitude larger.)

-Carl


pgpiSNTzVnsTM.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Rethinking *_destroy()

2011-09-19 Thread Carl Worth
On Mon, 19 Sep 2011 16:22:39 -0400, Ben Gamari  
wrote:
> As many might have noticed, there was recently a bit of a discussion on
> this list concerning the state of memory management in libnotmuch,
> especially regarding some classes of garbage collectors.
> 
> To summarize (someone correct me if I get something wrong),

Thanks for the summary, Ben. As many might have noticed, I'm somewhat
behind on my reading of the notmuch mailing list right now. So I
appreciate you bringing this issue to my attention.

> Overall, this means that languages with cyclical garbage collectors
> (Python, Haskell, and I'm sure others) can not bind libnotmuch
> correctly.

That certainly sound unappealing.

> I can see at least two ways of doing this,
> 
>   1) Acknowledging that we use talloc and allowing users to use
>  talloc_ref and talloc_unlink directly

I like this option, myself. I think talloc has been a wonderful boon to
my programming. So I don't have a problem with the notmuch API
documentation committing the implementation to using talloc. I'm also
quite glad to let the notmuch documentation advertise talloc to any
readers.

>   2) Wrapping talloc by adding a *_ref() and *_unref() to each object

That looks like a lot of extra API, but with no substantial benefit. (We
would get the freedom to switch to some other implementation of talloc,
but, I don't think we need that.)

> Additionally, we need to decide to what extent we want to break the
> libnotmuch API. While strictly speaking we could keep *_destroy() around
> without breaking existing code, this will mean we will have two ways of
> freeing an object. Perhaps a scheduled deprecation in a release or two
> is in order here?

Actually, I would prefer to leave *_destroy around, (as long as it's
doing nothing other than tall_free (or talloc_unlink? [*]) which does
appear to be the case for all existing functions).

To me, it doesn't really look like two ways of freeing an object.
Anyone who calls notmuch_query_create, (for example), should call
notmuch_query_destroy and those two calls pair nicely.

Then, *some* callers will want to take advantage of talloc. Those users
can add calls to talloc_reference and talloc_unlink (which pair nicely
themselves).

What I don't want is the unnatural pairing of something like
"notmuch_query_create" with "talloc_unlink". That just seems uselessly
harder to learn and remember.

> Any and all feedback is desired.

Thanks for pointing out the issue. Let me know if I've missed
anything. And, happy binding!

-Carl

[*] These destroy() functions are all currently calling talloc_free()
which won't work if the user has any remaining parents still present due
to calls to talloc_reference(). Perhaps we can just document that all
talloc_reference()/talloc_unlink() pairs must be complete before the
caller calls _destroy()?

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Re: Rethinking *_destroy()

2011-09-19 Thread Carl Worth
On Mon, 19 Sep 2011 16:22:39 -0400, Ben Gamari bgamari.f...@gmail.com wrote:
 As many might have noticed, there was recently a bit of a discussion on
 this list concerning the state of memory management in libnotmuch,
 especially regarding some classes of garbage collectors.
 
 To summarize (someone correct me if I get something wrong),

Thanks for the summary, Ben. As many might have noticed, I'm somewhat
behind on my reading of the notmuch mailing list right now. So I
appreciate you bringing this issue to my attention.

 Overall, this means that languages with cyclical garbage collectors
 (Python, Haskell, and I'm sure others) can not bind libnotmuch
 correctly.

That certainly sound unappealing.

 I can see at least two ways of doing this,
 
   1) Acknowledging that we use talloc and allowing users to use
  talloc_ref and talloc_unlink directly

I like this option, myself. I think talloc has been a wonderful boon to
my programming. So I don't have a problem with the notmuch API
documentation committing the implementation to using talloc. I'm also
quite glad to let the notmuch documentation advertise talloc to any
readers.

   2) Wrapping talloc by adding a *_ref() and *_unref() to each object

That looks like a lot of extra API, but with no substantial benefit. (We
would get the freedom to switch to some other implementation of talloc,
but, I don't think we need that.)

 Additionally, we need to decide to what extent we want to break the
 libnotmuch API. While strictly speaking we could keep *_destroy() around
 without breaking existing code, this will mean we will have two ways of
 freeing an object. Perhaps a scheduled deprecation in a release or two
 is in order here?

Actually, I would prefer to leave *_destroy around, (as long as it's
doing nothing other than tall_free (or talloc_unlink? [*]) which does
appear to be the case for all existing functions).

To me, it doesn't really look like two ways of freeing an object.
Anyone who calls notmuch_query_create, (for example), should call
notmuch_query_destroy and those two calls pair nicely.

Then, *some* callers will want to take advantage of talloc. Those users
can add calls to talloc_reference and talloc_unlink (which pair nicely
themselves).

What I don't want is the unnatural pairing of something like
notmuch_query_create with talloc_unlink. That just seems uselessly
harder to learn and remember.

 Any and all feedback is desired.

Thanks for pointing out the issue. Let me know if I've missed
anything. And, happy binding!

-Carl

[*] These destroy() functions are all currently calling talloc_free()
which won't work if the user has any remaining parents still present due
to calls to talloc_reference(). Perhaps we can just document that all
talloc_reference()/talloc_unlink() pairs must be complete before the
caller calls _destroy()?

-- 
carl.d.wo...@intel.com


pgpZlK4NrApVm.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Encodings

2011-07-11 Thread Carl Worth
On Mon, 11 Jul 2011 16:04:17 +0200, Sebastian Spaeth  
wrote:
> The answer is that things are very implicit. notmuch.h speaks of
> strings but never mentions encodings

Much of this was intentional on my part.

For example, I intentionally avoided restrictions on what could be
stored as a tag in the database, (other than the terminating character
implied by "string" of course).

> So, can be document what encoding we are expected to pass in the various
> APIs

Yes, let's clarify documentation wherever we need to.

> For some of the stuff we read directly from the files, eg
> arbitrary headers, we can probably be least sure

The headers should be decoded to utf-8, (via
g_mime_utils_header_decode_text), before being stored in the database.

> but are e.g. the returned tags always utf-8?

No. The tag data is returned exactly as the user presented it.

> I would love to make the python bindings use unicode() instances in
> cases where we can be sure to actually receive utf-8 encoded strings.
> 
> Encodings make my brain hurt. Unfortunately one cannot simply ignore
> them.

I think a lot of the pain here is due to some bad design decisions in
python itself. Of course, my saying that doesn't make things any easier
for you.

But do tell me what more we can do to clarify behavior or documentation.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Re: Encodings

2011-07-11 Thread Carl Worth
On Mon, 11 Jul 2011 16:04:17 +0200, Sebastian Spaeth sebast...@sspaeth.de 
wrote:
 The answer is that things are very implicit. notmuch.h speaks of
 strings but never mentions encodings

Much of this was intentional on my part.

For example, I intentionally avoided restrictions on what could be
stored as a tag in the database, (other than the terminating character
implied by string of course).

 So, can be document what encoding we are expected to pass in the various
 APIs

Yes, let's clarify documentation wherever we need to.

 For some of the stuff we read directly from the files, eg
 arbitrary headers, we can probably be least sure

The headers should be decoded to utf-8, (via
g_mime_utils_header_decode_text), before being stored in the database.

 but are e.g. the returned tags always utf-8?

No. The tag data is returned exactly as the user presented it.

 I would love to make the python bindings use unicode() instances in
 cases where we can be sure to actually receive utf-8 encoded strings.
 
 Encodings make my brain hurt. Unfortunately one cannot simply ignore
 them.

I think a lot of the pain here is due to some bad design decisions in
python itself. Of course, my saying that doesn't make things any easier
for you.

But do tell me what more we can do to clarify behavior or documentation.

-Carl

-- 
carl.d.wo...@intel.com


pgpa9xhIIcXO4.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] NEWS: Add notes for (imminent) notmuch 0.6 release

2011-07-01 Thread Carl Worth
By skimming through "git log 0.5..origin/release" late at
night. Hopefully everything here is accurate.
---

I've already pushed this out to master.

David, obviously this is intended to be cherry-picked over to the
release branch.

And everyone, please review what I've written and provide any
clarfications or corections necessary. Let me know if you think I've
included more detail than necessary or if I've missed anything
important.

Thanks,

-Carl


 NEWS |  344 ++
 1 files changed, 344 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index dae7832..d55453b 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,347 @@
+Notmuch 0.6 (2011-07-XX)
+===
+New, general features
+-
+Folder-based searching
+
+  Notmuch queries can now include a search term to match the
+  directories in which mail files are stored (within the mail
+  storage). The syntax is as follows:
+
+   folder:
+
+  For example, one might use things such as:
+
+   folder:spam
+   folder:2011/06
+
+  or anything else that matches directories within your mail storage.
+
+  This feature is particularly useful for users of delivery-agent
+  software (such as procmail or maildrop) that is filtering mail and
+  delivering it to particular folders, or users of systems such as
+  Gmail that use filesystem directories to indicate message tags.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will be searchable with folder: terms. In order to enable
+  this feature for all mail, the entire notmuch index will need to be
+  rebuilt as follows:
+
+   notmuch dump > notmuch.dump
+   # Backup, then remove notmuch database ($MAIL/.notmuch)
+   notmuch new
+   notmuch restore notmuch.dump
+
+New, automatic tags: "signed" and "encrypted"
+
+  These tags will automatically be applied to messages containing
+  multipart/signed and multipart/encrypted parts.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will receive these tags. In order to enable this feature for
+  all mail, the entire notmuch index will need to be rebuilt (see
+  above).
+
+New command-line features
+-
+Add new "notmuch show --verify" option for signature verification
+
+  This option instruct notmuch to verify the signature of
+  PGP/MIME-signed parts.
+
+Add new "notmuch show --decrypt" and "notmuch reply --decrypt" options
+
+  This option instructs notmuch to decrypt PGP/MIME-encrypted parts.
+
+Proper nesting of multipart parts in "notmuch show" output
+
+  MIME parts are now display with proper nesting to reflect original
+  MIME hierarchy of a message. This allows clients to correctly
+  analyze the MIME structure, (such as, for example, determining to
+  which parts a signature part applies).
+
+Add new "notmuch show --part" option
+
+  This is a replacement for the older "notmuch part" command, (which
+  is now deprecated?it should still work as always, but is no longer
+  documented). Putting part output under "notmuch show" allows for all
+  of the "notmuch show" options to be applied when extracting a single
+  part, (such as --format=json for extracting a message part with JSON
+  formatting).
+
+Deprecate "notmuch search-tags", (in favor of "notmuch search --output=tags *")
+
+  The "notmuch search-tags" sub-command has been redundant since the
+  addition of the --output=tags option to "notmuch search". We now
+  make that more clear by deprecating "notmuch search-tags", (dropping
+  it from the documentation). We do continue to support the old syntax
+  by translating it internally to the new call.
+
+Performance improvements
+
+Faster searches (by doing fewer serches to construct threads)
+
+  Whenever a user asks for search results as threads, notmuch first
+  performs a search for messages matching the query, then performs
+  additional searches to find other messages in the resulting threads.
+
+  Removing inefficiences and redundancies in these secondary searches
+  results in a measured speedups of 1.5x for a typical search.
+
+Faster searches (by doing fewer passes to gather message data)
+
+  Optimizing Xapian data access patterns (using a single pass to get
+  all message-document data rather than a ps for each data type)
+  results in a measured speedup of 1.7x for a typical search.
+
+  The benefits of this optimization combine with the preceding
+  optimization. With both in place, Austin Clements measured a speedup
+  of 2.5x for a search of all messages in his inbox (was 4.5s, now
+  1.8s). Thanks, Austin!
+
+Faster initial indexing
+
+  More efficient indexing of new messages results in a measured
+  speedup of 1.4x for the initial indexing of 3 GB of mail (1h 14m
+  rather than 1h 46m). Thanks to Austin Clements and Michal Sojka.
+
+Make "notmuch new" faster for unchanged directories
+
+  Optimizing to not do any further 

[PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

2011-07-01 Thread Carl Worth
On Fri,  1 Jul 2011 02:01:08 -0700, Carl Worth  wrote:
> Rather than fixing the name of the variable and changing its default
> value, here we remove the condition entirely, such that the feature is
> enabled unconditionally.

I've just pushed this change out.

I emailed it here just to be able to comment on it.

David, I implemented this fix while writing up the NEWS[*] notes for the
0.6 release (and noticing that this feature which was implemented wasn't
actually usable as documented so I couldn't really add a NEWS item for
it).

If you want to sneak it into 0.6 I think it's safe, (it even hits one
existing test case---though an additional test case to show an actual
subject change appearing would be even better).

It's your call of course, and if the change doesn't go, then that's fine
too. In that case, I'll just not comment on this feature in NEWS until
our next release.

-Carl

[*] Yes, I'm working away at this?almost done now...
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110701/8bb02d6b/attachment-0001.pgp>


[PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

2011-07-01 Thread Carl Worth
The feature to show subject changes in the collapsed thread view was
originally added (8ab433607) with an option
(notmuch-show-always-show-subject) to display the subject
for all messages, even when there was no change.

The subsequent commit (4f04d273) changed the sense of the test (or to
and) and the name of the controlling variable
(notmuch-show-elide-same-subject).

But this commit is broken in a few ways:

  1. The original definition of notmuch-show-always-show-subject was
 left around

 But the variable isn't actually used in the code at all, so it
 just adds clutter and confusion to the customization interface.

  2. The name and description of the controlling variable doesn't
 match the implementation

 The name suggests that setting the variable to t will cause
 repeated subjects to be elided, (suggesting that when it is nil
 all subjects will be shown).

 However, when the variable is nil, no subjects are shown. So a
 correct name for the variable in this sense would be
 notmuch-show-subject-changes.

Showing subject changes is a useful feature, and should be on by
default. (We don't want to bury generally useful features behind
customizations that users have to find).

Rather than fixing the name of the variable and changing its default
value, here we remove the condition entirely, such that the feature is
enabled unconditionally.

So both the currently-used variable and the stale definition of the
formerly-used are removed.

Also, the one relevant test-suite result is updated, (showing the
intial subject of a collapsed thread, and no subject display for later
messages that do not change the subject).
---
 emacs/notmuch-show.el  |   16 ++--
 .../notmuch-show-thread-with-hidden-messages   |1 +
 2 files changed, 3 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6685717..f96743b 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -65,17 +65,6 @@ any given message."
   :group 'notmuch
   :type 'boolean)

-(defcustom notmuch-show-elide-same-subject nil
-  "Do not show the subject of a collapsed message if it is the
-same as that of the previous message."
-  :group 'notmuch
-  :type 'boolean)
-
-(defcustom notmuch-show-always-show-subject t
-  "Should a collapsed message show the `Subject:' line?"
-  :group 'notmuch
-  :type 'boolean)
-
 (defvar notmuch-show-markup-headers-hook '(notmuch-show-colour-headers)
   "A list of functions called to decorate the headers listed in
 `notmuch-message-headers'.")
@@ -727,9 +716,8 @@ current buffer, if possible."
   ;; If the subject of this message is the same as that of the
   ;; previous message, don't display it when this message is
   ;; collapsed.
-  (when (and notmuch-show-elide-same-subject
-(not (string= notmuch-show-previous-subject
-  bare-subject)))
+  (when (not (string= notmuch-show-previous-subject
+ bare-subject))
(forward-line 1))
   (setq headers-start (point-marker)))
 (setq headers-end (point-marker))
diff --git 
a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages 
b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
index 5df6606..8a0660f 100644
--- a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
+++ b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
@@ -1,3 +1,4 @@
 Jan Janak  (2009-11-17) (inbox unread)
+Subject: [notmuch] What a great idea!
  Jan Janak  (2009-11-17) (inbox)
  Carl Worth  (2009-11-18) (inbox unread)
-- 
1.7.5.4



[PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

2011-07-01 Thread Carl Worth
The feature to show subject changes in the collapsed thread view was
originally added (8ab433607) with an option
(notmuch-show-always-show-subject) to display the subject
for all messages, even when there was no change.

The subsequent commit (4f04d273) changed the sense of the test (or to
and) and the name of the controlling variable
(notmuch-show-elide-same-subject).

But this commit is broken in a few ways:

  1. The original definition of notmuch-show-always-show-subject was
 left around

 But the variable isn't actually used in the code at all, so it
 just adds clutter and confusion to the customization interface.

  2. The name and description of the controlling variable doesn't
 match the implementation

 The name suggests that setting the variable to t will cause
 repeated subjects to be elided, (suggesting that when it is nil
 all subjects will be shown).

 However, when the variable is nil, no subjects are shown. So a
 correct name for the variable in this sense would be
 notmuch-show-subject-changes.

Showing subject changes is a useful feature, and should be on by
default. (We don't want to bury generally useful features behind
customizations that users have to find).

Rather than fixing the name of the variable and changing its default
value, here we remove the condition entirely, such that the feature is
enabled unconditionally.

So both the currently-used variable and the stale definition of the
formerly-used are removed.

Also, the one relevant test-suite result is updated, (showing the
intial subject of a collapsed thread, and no subject display for later
messages that do not change the subject).
---
 emacs/notmuch-show.el  |   16 ++--
 .../notmuch-show-thread-with-hidden-messages   |1 +
 2 files changed, 3 insertions(+), 14 deletions(-)

diff --git a/emacs/notmuch-show.el b/emacs/notmuch-show.el
index 6685717..f96743b 100644
--- a/emacs/notmuch-show.el
+++ b/emacs/notmuch-show.el
@@ -65,17 +65,6 @@ any given message.
   :group 'notmuch
   :type 'boolean)
 
-(defcustom notmuch-show-elide-same-subject nil
-  Do not show the subject of a collapsed message if it is the
-same as that of the previous message.
-  :group 'notmuch
-  :type 'boolean)
-
-(defcustom notmuch-show-always-show-subject t
-  Should a collapsed message show the `Subject:' line?
-  :group 'notmuch
-  :type 'boolean)
-
 (defvar notmuch-show-markup-headers-hook '(notmuch-show-colour-headers)
   A list of functions called to decorate the headers listed in
 `notmuch-message-headers'.)
@@ -727,9 +716,8 @@ current buffer, if possible.
   ;; If the subject of this message is the same as that of the
   ;; previous message, don't display it when this message is
   ;; collapsed.
-  (when (and notmuch-show-elide-same-subject
-(not (string= notmuch-show-previous-subject
-  bare-subject)))
+  (when (not (string= notmuch-show-previous-subject
+ bare-subject))
(forward-line 1))
   (setq headers-start (point-marker)))
 (setq headers-end (point-marker))
diff --git 
a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages 
b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
index 5df6606..8a0660f 100644
--- a/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
+++ b/test/emacs.expected-output/notmuch-show-thread-with-hidden-messages
@@ -1,3 +1,4 @@
 Jan Janak j...@ryngle.com (2009-11-17) (inbox unread)
+Subject: [notmuch] What a great idea!
  Jan Janak j...@ryngle.com (2009-11-17) (inbox)
  Carl Worth cwo...@cworth.org (2009-11-18) (inbox unread)
-- 
1.7.5.4

___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: [PATCH] emacs: Fix to unconditionally display subject changes in collapsed thread view

2011-07-01 Thread Carl Worth
On Fri,  1 Jul 2011 02:01:08 -0700, Carl Worth cwo...@cworth.org wrote:
 Rather than fixing the name of the variable and changing its default
 value, here we remove the condition entirely, such that the feature is
 enabled unconditionally.

I've just pushed this change out.

I emailed it here just to be able to comment on it.

David, I implemented this fix while writing up the NEWS[*] notes for the
0.6 release (and noticing that this feature which was implemented wasn't
actually usable as documented so I couldn't really add a NEWS item for
it).

If you want to sneak it into 0.6 I think it's safe, (it even hits one
existing test case---though an additional test case to show an actual
subject change appearing would be even better).

It's your call of course, and if the change doesn't go, then that's fine
too. In that case, I'll just not comment on this feature in NEWS until
our next release.

-Carl

[*] Yes, I'm working away at this—almost done now...


pgpXiFxJ4yJoe.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


[PATCH] NEWS: Add notes for (imminent) notmuch 0.6 release

2011-07-01 Thread Carl Worth
By skimming through git log 0.5..origin/release late at
night. Hopefully everything here is accurate.
---

I've already pushed this out to master.

David, obviously this is intended to be cherry-picked over to the
release branch.

And everyone, please review what I've written and provide any
clarfications or corections necessary. Let me know if you think I've
included more detail than necessary or if I've missed anything
important.

Thanks,

-Carl


 NEWS |  344 ++
 1 files changed, 344 insertions(+), 0 deletions(-)

diff --git a/NEWS b/NEWS
index dae7832..d55453b 100644
--- a/NEWS
+++ b/NEWS
@@ -1,3 +1,347 @@
+Notmuch 0.6 (2011-07-XX)
+===
+New, general features
+-
+Folder-based searching
+
+  Notmuch queries can now include a search term to match the
+  directories in which mail files are stored (within the mail
+  storage). The syntax is as follows:
+
+   folder:path
+
+  For example, one might use things such as:
+
+   folder:spam
+   folder:2011/06
+
+  or anything else that matches directories within your mail storage.
+
+  This feature is particularly useful for users of delivery-agent
+  software (such as procmail or maildrop) that is filtering mail and
+  delivering it to particular folders, or users of systems such as
+  Gmail that use filesystem directories to indicate message tags.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will be searchable with folder: terms. In order to enable
+  this feature for all mail, the entire notmuch index will need to be
+  rebuilt as follows:
+
+   notmuch dump  notmuch.dump
+   # Backup, then remove notmuch database ($MAIL/.notmuch)
+   notmuch new
+   notmuch restore notmuch.dump
+
+New, automatic tags: signed and encrypted
+
+  These tags will automatically be applied to messages containing
+  multipart/signed and multipart/encrypted parts.
+
+  NOTE: Only messages that are newly indexed with this version of
+  notmuch will receive these tags. In order to enable this feature for
+  all mail, the entire notmuch index will need to be rebuilt (see
+  above).
+
+New command-line features
+-
+Add new notmuch show --verify option for signature verification
+
+  This option instruct notmuch to verify the signature of
+  PGP/MIME-signed parts.
+
+Add new notmuch show --decrypt and notmuch reply --decrypt options
+
+  This option instructs notmuch to decrypt PGP/MIME-encrypted parts.
+
+Proper nesting of multipart parts in notmuch show output
+
+  MIME parts are now display with proper nesting to reflect original
+  MIME hierarchy of a message. This allows clients to correctly
+  analyze the MIME structure, (such as, for example, determining to
+  which parts a signature part applies).
+
+Add new notmuch show --part option
+
+  This is a replacement for the older notmuch part command, (which
+  is now deprecated—it should still work as always, but is no longer
+  documented). Putting part output under notmuch show allows for all
+  of the notmuch show options to be applied when extracting a single
+  part, (such as --format=json for extracting a message part with JSON
+  formatting).
+
+Deprecate notmuch search-tags, (in favor of notmuch search --output=tags *)
+
+  The notmuch search-tags sub-command has been redundant since the
+  addition of the --output=tags option to notmuch search. We now
+  make that more clear by deprecating notmuch search-tags, (dropping
+  it from the documentation). We do continue to support the old syntax
+  by translating it internally to the new call.
+
+Performance improvements
+
+Faster searches (by doing fewer serches to construct threads)
+
+  Whenever a user asks for search results as threads, notmuch first
+  performs a search for messages matching the query, then performs
+  additional searches to find other messages in the resulting threads.
+
+  Removing inefficiences and redundancies in these secondary searches
+  results in a measured speedups of 1.5x for a typical search.
+
+Faster searches (by doing fewer passes to gather message data)
+
+  Optimizing Xapian data access patterns (using a single pass to get
+  all message-document data rather than a ps for each data type)
+  results in a measured speedup of 1.7x for a typical search.
+
+  The benefits of this optimization combine with the preceding
+  optimization. With both in place, Austin Clements measured a speedup
+  of 2.5x for a search of all messages in his inbox (was 4.5s, now
+  1.8s). Thanks, Austin!
+
+Faster initial indexing
+
+  More efficient indexing of new messages results in a measured
+  speedup of 1.4x for the initial indexing of 3 GB of mail (1h 14m
+  rather than 1h 46m). Thanks to Austin Clements and Michal Sojka.
+
+Make notmuch new faster for unchanged directories
+
+  Optimizing to not do any further examinations of sub-directories
+  

Re: [PATCH] remove prefixes from `--output={threads, messages}' results

2011-07-01 Thread Carl Worth
On Thu, 30 Jun 2011 10:19:49 +0200, Pieter Praet pie...@praet.org wrote:
 Alter `do_search_threads()' and `do_search_messages()'
 to not prepend each result with `thread:' respectively `id:'.

My one concern here is that I've sometimes had a message-id without the
id prefix and run a command like this:

notmuch search 1309421989-22410-1-git-send-email-pie...@praet.org

And I've gotten confused when I've received no output, (didn't I receive
that mail? what happened?).

But I think this is a separate bug where the right fix is to make any
search terms with no prefix search through *all* prefixed terms
generated from email content. This would allow us to also avoid indexing
some content twice, (currently we store subject, from, and to both with
a prefix and without a prefix).

-Carl

-- 
carl.d.wo...@intel.com


pgpidP7oSWd6S.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


Re: patchwork

2011-07-01 Thread Carl Worth
On Thu, 30 Jun 2011 11:01:14 +0200, martin f krafft madd...@madduck.net wrote:
 Are people using http://patchwork.notmuchmail.org at all? The reason
 of my asking is that it receives quite a lot of spam and I do not
 need to invest this time for a service noone uses.

I have not been using it. I dos appreciate your concern for keeping the
notmuch community vibrant and your volunteer efforts to setup this
service.

But if it's costing more time now than it's worth, please feel free to
drop it (unless someone else says differently of course).

-Carl

-- 
carl.d.wo...@intel.com


pgpnDhMlqesfi.pgp
Description: PGP signature
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch


patchwork

2011-06-30 Thread Carl Worth
On Thu, 30 Jun 2011 11:01:14 +0200, martin f krafft  
wrote:
> Are people using http://patchwork.notmuchmail.org at all? The reason
> of my asking is that it receives quite a lot of spam and I do not
> need to invest this time for a service noone uses.

I have not been using it. I dos appreciate your concern for keeping the
notmuch community vibrant and your volunteer efforts to setup this
service.

But if it's costing more time now than it's worth, please feel free to
drop it (unless someone else says differently of course).

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH] remove prefixes from `--output={threads, messages}' results

2011-06-30 Thread Carl Worth
On Thu, 30 Jun 2011 10:19:49 +0200, Pieter Praet  wrote:
> Alter `do_search_threads()' and `do_search_messages()'
> to not prepend each result with `thread:' respectively `id:'.

My one concern here is that I've sometimes had a message-id without the
id prefix and run a command like this:

notmuch search 1309421989-22410-1-git-send-email-pieter at praet.org

And I've gotten confused when I've received no output, (didn't I receive
that mail? what happened?).

But I think this is a separate bug where the right fix is to make any
search terms with no prefix search through *all* prefixed terms
generated from email content. This would allow us to also avoid indexing
some content twice, (currently we store subject, from, and to both with
a prefix and without a prefix).

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Preventing the user shooting themself in the foot

2011-06-29 Thread Carl Worth
On Thu, 30 Jun 2011 13:04:23 +1000, Brian May  wrote:
> On 30 June 2011 08:40, Carl Worth  wrote:
> > The 'a' keybinding, (in turn), was designed for cases when you *know*
> > you don't want to read the rest of the thread.
> 
> ... in which case it should also mark everything as read. IMHO.

I know the current behavior only catches my opinion (and only an opinion
I had at one particular point in time). So I won't say this is Right,
but I will at least explain what I was thinking:

The "unread" tag is distinct from the "inbox" tag. Why two tags? Don't
they normally change at the same time? If a key like 'a' got rid of the
"unread" tag as well as the "inbox" then there would be almost no need
for having two tags.

The idea I had is that "inbox" is fully under explicit control by the
user. The user must make an intentional decision to "archive" a message
in order for that tag to be removed.

Distinct from that is "unread" which is handled automatically by the
mail client (as well as it can tell what you've actually read or
not). So this tag is removed only implicitly, (we don't have specific
commands to manipulate the "unread" tag). When the client displays a
message as the "current" message it immediately removes the "unread"
tag.

 Whenever it displays a message to the
user, (as the "current" message), it removes the unread tag from that
message.

This means that messages can lose the "unread" tag while still remaining
tagged "inbox", (you read a message, but don't archive it), and that
messages can lose the "archive" tag while still remaining tagged
"unread", (you archive a thread before reading all messages in the
thread).

The distinction ends up being useful to me. If at some point someone
points me to a specific message, and when I search for it I see the
"unread" tag, then this highlights to me that I never even looked at the
message.

> Are there any keyboard bindings to go forwards to the next message or
> backwards to the last message without marking anything as archived?

As mentioned by someone else, you can navigate the messages in a thread
with 'n' and 'p'.

One of the obviously missing keybindings is a way to easily navigate


[PATCH] new: Improved workaround for mistaken new directories

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 19:00:01 -0400, Austin Clements  wrote:
> We work around this by setting the new directory's database mtime to
> -1 before scanning anything in the new directory.  This also obviates
> the need for the workaround used in 8c39e8d6.

Thanks. This is pushed.

> +/* XXX This is a temporary workaround.  If we don't update the
> + * database mtime until after processing messages in this
...
> + * A better solution would be for notmuch_database_get_directory
> + * to indicate if it really created a new directory or not, either
> + * by a new out-argument, or by recording this information and
> + * providing an accessor.

A much better "XXX" comment! It tells the potential reader of the future
what to do to be able to remove the comment. Thanks so much.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



Preventing the user shooting themself in the foot

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 20:42:01 +0100, Robin Green  wrote:
> It's really dangerous to use the 'a' key in notmuch-mode in an inbox
> thread which has multiple unread replies! Yes, the other unread replies
> will still be tagged unread, but the user might not immediately be aware
> of them. It would be really useful to have an optional warning ("More
> unread messages in this thread, are you sure?") for this situation!

That was why I originally designed the space bar keybinding for reading
through each message in turn, (and then it archives the thread only when you
page past the last message).

The 'a' keybinding, (in turn), was designed for cases when you *know*
you don't want to read the rest of the thread.

That was the original idea behind these two, at least.  In any case,
it's doing exactly what it's defined to do.

Perhaps you're just missing a key for archiving the current message (and
advancing to the next)?

I know there are various problems with the current keybindings?and
several obviously-missing operations. And it's funny how long I've put
up with shortcomings rather than just fixing them.

I'll be glad to look at proposals for improving things.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH] test: document test_expect_equal_file

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 10:06:32 -0700, Jameson Graef Rollins  wrote:
> This test was not properly documented when it was originally added (my
> bad).

Thanks. Merged!

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH] lib/Makefile.local: remove leftover debugging output.

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 07:42:02 -0300, david at tethera.net wrote:
> From: David Bremner 
> 
> The removed "echo $(libnotmuch_modules)" was strictly for debugging.

Merged, thanks.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH 1/2] new: Don't update DB mtime if FS mtime equals wall-clock time.

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 03:10:54 -0400, Austin Clements  wrote:
> This fixes a race where multiple message deliveries in the same second
> with an intervening notmuch new could result in messages being ignored
> by notmuch (at least, until a later delivery forced a rescan).

Excellent fix, Austin.

I've merged this, (and the following commit to drop the ugly
increment_mtime).

I think that even fixes an outstanding portability bug (where someone
previously sent patches to change our implementation of
increment_mtime).

It's sure nice to see notmuch become more robust all the time.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH] test: revert non-intentional changes introduced in eb4cf465

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 22:23:47 +0200, Pieter Praet  wrote:
> eb4cf465 introduces changes which weren't part of the submitted
> patch (id:"87liwlip2j.fsf at gmail.com"), presumably made during
> resolving merge conflicts.

Oops, you caught me.

> This commit reverts both changes, as they were clearly accidental.

Thanks. This is merged (automatically, so hopefully without errors!) and
pushed out.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH] Fix folder: coherence issue

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 14:04:45 -0600, Mark Anderson  wrote:
> Add removal of all ZXFOLDER terms to removal of all XFOLDER terms for
> each message filename removal.
> 
> The existing filename-list reindexing will put all the needed terms
> back in.  Test search-folder-coherence now passes.

Thanks so much, Mark!

I've now pushed the new test (followed by a quick fix to turn sleep into
increment_mtime) and the fix.

David, I recommend these three commits for the release branch. This is
an important fix before we first make a release with the folder:
feature.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH 1/2] new: Don't update DB mtime if FS mtime equals wall-clock time.

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 06:47:37 -0700, Carl Worth  wrote:
> On Wed, 29 Jun 2011 03:10:54 -0400, Austin Clements  
> wrote:
> > + * XXX Bug workaround: If this is a new directory, we *must*
> > + * update the mtime; otherwise the next run will see the 0 mtime
...
> I like to reserve "XXX" as an indication that some further work is
> necessary.

Reading your other mail now, I see that there are bugs here and that you
do want to eliminate the new_directory optimization. That wasn't clear
to me from the comment above.

So the XXX is probably fine, but could perhaps give a little more
indication of what could be done to eliminate the bug.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 
<http://notmuchmail.org/pipermail/notmuch/attachments/20110629/4f487e2a/attachment.pgp>


[PATCH 10/10] test: use emacsclient(1) for Emacs tests

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 03:34:58 -0400, Austin Clements  wrote:
> Or, really, any file system, since notmuch doesn't currently take
> advantage of sub-second mtimes.

Oh, ouch. I hadn't realized that I'd codified the 1-second granularity
mtime into the notmuch API.

Oh well, we could extend that with a floating-point API as needed.

> > Austin has a plan to fix the use of the mtime timestamp in notmuch,
> > (never storing the current mtime in the database if it's the same as
> > the current time), that should hopefully close this race window.
> 
> Posted.

Thanks!

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



[PATCH 1/2] new: Don't update DB mtime if FS mtime equals wall-clock time.

2011-06-29 Thread Carl Worth
On Wed, 29 Jun 2011 03:10:54 -0400, Austin Clements  wrote:
> This fixes a race where multiple message deliveries in the same second
> with an intervening notmuch new could result in messages being ignored
> by notmuch (at least, until a later delivery forced a rescan).

Thanks for the fix, Austin!

> + * XXX Bug workaround: If this is a new directory, we *must*
> + * update the mtime; otherwise the next run will see the 0 mtime
> + * and think this is still a new directory containing no files or
> + * subdirs (which is unsound in general).  If fs_mtime ==
> + * stat_time, we set the database mtime to a bogus (but non-zero!)
> + * value to force a rescan.

I like to reserve "XXX" as an indication that some further work is
necessary.

That doesn't seem to be the case here, (instead, you seem to have
thought this issue out quite fully).

Other than that, I'm quite happy with the patch.

-Carl

-- 
carl.d.worth at intel.com
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: 



  1   2   3   4   5   6   7   8   9   10   >