[notmuch] strange behavior of indexing of and searching for strings containing '[]'
On 2010-02-05, Jameson Rollins wrote: > Hey, folks. I've been noticing some strange behavior of notmuch search > results for strings containing '[]'. Here are some searches for some > exact strings in messages subjects: The '[]' is a red herring. Xapian's TermGenerator and QueryParser classes treat these two characters pretty much as if they were spaces. > servo:~ 0$ notmuch search subject:'emacs paned UI' Note that the '' is quoting for the shell only here. So Xapian sees: subject:emacs paned UI Assuming you are defaulting to an AND search, that's `emacs in the subject' AND `paned anywhere in the indexed text' AND `UI anywhere in the indexed text'. To specify a quoted phrase you want "" anyway (not ''), so the command matching what I think you intended to search for is: notmuch search 'subject:"emacs paned UI"' > servo:~ 0$ notmuch search subject:'[notmuch] emacs paned UI' notmuch search 'subject:"[notmuch] emacs paned UI"' Which should return identical results to: notmuch search 'subject:"notmuch emacs paned UI"' > thread:5f2cb4b108773a39161b33c86e54f7fd 4 mins. ago [1/1] Jameson Rollins;= > [notmuch] loss of duplicate messages (inbox) > servo:~ 0$=20 > > Not only did it not turn up the message that *does* match that exact > string in it's subject line, it actually turns up a completely different > message that doesn't match the search term at all! It matches the notmuch in the subject, and presumably emacs, paned, and UI in the body. > [snip the rest - the same explanations apply] Cheers, Olly
[notmuch] loss of duplicate messages
On Fri, 05 Feb 2010 11:31:34 -0500, Jameson Rollins wrote: > I'm noticing that notmuch is either not syncing, or not returning in > searches, duplicate messages that have identical bodies but different > headers. This is indeed the correct behaviour of notmuch. There has been some discussion on it in the past, I believe with proposals to track both messages and show only one; but I don't think I've seen proponents of showing both duplicate messages. Personally I'd find it rather annoying if I'd see messages twice. But I do see the value in being sure that your mail gets delivered through the list. I believe the solution I've seen discussed was for notmuch to somehow determine which of the duplicates holds the most information (which would be the one through the list, not the one directly to you). On the other hand, enough mailing lists destroy this behaviour, see: the thread [notmuch] [PATCH (rebased)] Handle message renames in mail spool, around id:87y6lir37f.fsf at vertex.dottedmag -- - Marten
[notmuch] strange behavior of indexing of and searching for strings containing '[]'
On Fri, Feb 5, 2010 at 4:44 PM, Jameson Rollins wrote: > Does anyone have any idea what's going on here? ?I think I saw mention > of this issue on IRC somewhere, but I thought I should bring it up > explicitly here. ?This is definitely some buggy behavior. Afaik, stuff in between [] is not indexed, but that doesn't quite explain the other weird results. -- Oliver Charles / aCiD2
[notmuch] hack to retag a directory
On Thu, 03 Dec 2009 15:26:56 -0400, david at tethera.net wrote: > > I think this will be obsolete pretty soon when the equivalent is > > built-in to notmuch, but in the mean time, here is a script that > > somebody might find useful: retag a whole directory (recursively). I > > don't claim it is nice in any way, but it seems usable for me, taking > > about 5 seconds to retag a directory containing 1000 messages. > > Sigh. And of course the version I posted was broken. I put a fixed version at > > > http://pivot.cs.unb.ca/git/?p=notmuch-scripts.git;a=blob_plain;f=tagdir;hb=HEAD Thanks for sharing that, David. Obviously, we now have outstanding patches to provide search capabilities based on the folder containing messages. So once that gets merged, you shouldn't need this script anymore. > You might, or might not also be interested in > > > http://pivot.cs.unb.ca/git/?p=notmuch-scripts.git;a=blob_plain;f=gitmuch;hb=HEAD > > which is the beginnings of how to keep tags in git (for syncing > between machines). But this one looks quite interesting. Obviously, it's not a complex script, but it looks pretty handy to me. I might start using this to have a little history of my tag changes, (rather than just including the dump output in occasional backups like I have been doing). And it's interesting that this script might be just good enough for the synchronization needs of some people. It's not integrated, and might require manual fixup of any resulting git conflicts, but it might be handy for some. The biggest problem I see is that if I were to read some messages locally, and then run "gitmuch restore" then this would wipe out the local changes I had made. So we'll definitely want a more integrated solution to eliminate the chance of problems like this. > Right now the notmuch restore step is the > bottleneck, but Carl apparently knows how to speed 'notmuch restore' > up. One easy answer is to just make "notmuch restore" do nothing for messages where the existing tags are the same as the tags mentioned in the input file. I just pushed a change to implement this, (along with new tests for "notmuch dump" and "notmuch restore" of course). For me, this takes a "notmuch restore" right after a "notmuch dump" from about 10 minutes down to 1 minute, (and it was about 2 hours before the Xapian Defect #250 fix). The other idea that I didn't do yet is to change "notmuch restore" to do a single search for all messages rather than N searches each resulting in 1 message. But the 1-minute time I'm getting for "notmuch restore" now is basically the same time required for a "notmuch dump", (which is already doing a single global search). So perhaps Xapian is just plain fast enough that a change like that won't help at all. -Carl -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/6e107d10/attachment.pgp>
[notmuch] patchwork now auto-updates patches from Git (was: Git feature branch)
also sprach martin f krafft [2010.02.04.1650 +1300]: > - ? which brings me to my second point: there are certain things > that patchwork can do, at least in theory: > > * mark patches accepted when they hit your (canonical) master > branch > * mark patches rfc when they hit e.g. my (canonical) next branch > * mark patches "under review" when they hit the all-patches (or > pu) branch. > > I have not yet tried any of these, and I am basing this theory > only on the idea that git-patch-id can come to the rescue, for > there is no other linkage between the patch on the mailing list > (and thus known to patchwork), and the commit in the repo. Patchwork now marks patches Accepted once they hit Carl's master branch (up to 10 minutes delay due to Cron). It uses an algorithm similar (but not equal) to git-patch-id in that it hashes the diff. This means that the commit message can be amended when patches are applied/cherry-picked, but the patch itself must be verbatim. I ran it on all history thus far and it found 99 patches. It'll be trivial to set it up to mark other states when the corresponding commits hit another branch. Let me know if there are any problems, and feedback welcome. -- martin | http://madduck.net/ | http://two.sentenc.es/ "if they can get you asking the wrong questions, they don't have to worry about answers." -- thomas pynchon spamtraps: madduck.bogus at madduck.net -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: Digital signature (see http://martin-krafft.net/gpg/) URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/a05b53e1/attachment.pgp>
[notmuch] loss of duplicate messages
On Fri, 05 Feb 2010 14:47:03 -0500, Jameson Rollins wrote: > On Fri, 05 Feb 2010 14:39:22 -0500, micah anderson > wrote: > > Welcome to how gmail does it. > > Why welcome me to "how gmail does it"?! I never wanted to go there! Because the perception of the internet by the third larges mail provider on the internet, with over 146 million users[0] is not a trivial number that can be ignored. Maybe I should have said, "welcome to the place where you do not wish to tread, the place where 1/3rd of the people using the internet have their perceptions shaped. be careful, there is a lot of stuff here you do not want to step in!" micah 0. Arrington, Michael (2009-07-09). "Bing Comes to Hotmail". Techcrunch. http://www.techcrunch.com/2009/07/09/bing-comes-to-hotmail/. Retrieved 2009-07-11. "Hotmail is still by far the largest web mail provider on the Internet, with 343 million monthly users according to Comscore. Second and third are Yahoo (285 million) and Gmail (146 million)." -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/7af9e309/attachment.pgp>
[notmuch] loss of duplicate messages
On Fri, 05 Feb 2010 14:39:22 -0500, micah anderson wrote: > Welcome to how gmail does it. Why welcome me to "how gmail does it"?! I never wanted to go there! -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/267a1efc/attachment.pgp>
[notmuch] loss of duplicate messages
On Fri, 05 Feb 2010 12:49:21 -0500, Jameson Graef Rollins wrote: > A policy of only returning one is going to be problematic for folks who > want or expect to see the other. And in fact think I want to see both. > I have both, and I've asked notmuch to index both, so why shouldn't it > return both in a search? Welcome to how gmail does it. When they first hit the scene, as an operator of a large mailing list service, I was *constantly* being bugged with support issues from people who were expecting this very behavior, "I sent a message to the list, but I never got it, did it get posted to the list?!". Soon I found out that gmail did exactly what you are reporting notmuch as doing. The frightening thing is that over the last few years of gmail's existence, those complaints and support issues have totally gone away. Does that mean that gmail has trained people to no longer expect this behavior? micah -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/f91d5aec/attachment.pgp>
[notmuch] loss of duplicate messages
On Fri, 05 Feb 2010 18:25:59 +0100, Marten Veldthuis wrote: > This is indeed the correct behaviour of notmuch. There has been some > discussion on it in the past, I believe with proposals to track both > messages and show only one; but I don't think I've seen proponents of > showing both duplicate messages. > > Personally I'd find it rather annoying if I'd see messages twice. But I > do see the value in being sure that your mail gets delivered through the > list. I believe the solution I've seen discussed was for notmuch to > somehow determine which of the duplicates holds the most information > (which would be the one through the list, not the one directly to you). Hey, Marten. Thanks for the reply. The problem I have with only returning one of the redundant messages is that I don't think anyone could ever really convince me that notmuch has the ability to decide which of the redundant messages is the *right* one to return. I think notmuch is currently just returning the first one it indexes, but why is that better than returning the one most recently indexed? A policy of only returning one is going to be problematic for folks who want or expect to see the other. And in fact think I want to see both. I have both, and I've asked notmuch to index both, so why shouldn't it return both in a search? jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/ff825716/attachment.pgp>
[notmuch] [PATCH -v2] notmuch.el: Support for customizing search result display
On Wed, 2 Dec 2009 18:19:38 +0530, "Aneesh Kumar K.V" wrote:
> From: Aneesh Kumar K.V
>
> This patch helps in customizing search result display
> similar to mutt's index_format. The customization is done
> by defining an alist as below
>
> (setq notmuch-search-result-format '(("date" . "%s ")
>("authors" . "%-40s ")
>("subject" . "%s ")
>("tags" . "(%s)")))
>
> The supported keywords are date, count, authors, subject and tags.
>
> Signed-off-by: Aneesh Kumar K.V
Hi Aneesh,
I'm sorry this patch has lingered so long without comment. There's
really only one problem I see with it:
> +(defcustom notmuch-search-result-format nil
> + "Search result formating. Supported fields are
> + date, count, authors, subject, tags
> +ex: (setq notmuch-search-result-format \(\(\"authors\" . \"%-40s\"\)
> + \(\"subject\" . \"%s\"\)\)\)"
> +:type '(alist :key-type (string) :value-type (string))
> +:group 'notmuch)
...
> - (insert (format "%s %-7s %-40s %s (%s)\n" date count
> authors subject tags))
> + (if (not notmuch-search-result-format)
> + (progn (insert (format "%s %-7s %-40s %s" date
> count authors subject))
> +;; insert the fontified tag
> +(insert-tags (format "%s" tags))
> +(insert "\n"))
> + (notmuch-search-show-result date count authors
> subject tags))
I don't like that the new format variable is nil by default and then
there's an open-coded implementation of the default formatting. This has
a couple of problems:
1. The new code is not being exercised by default, so it would be easy
to break it without realizing.
2. The system is not very self-documenting.
If a new user wants to tweak the default format, that will be a lot
easier if they find a customizable variable that describes the
current format. *That* will be a lot easier to modify rather than
trying to learn what should be used instead of "nil".
-Carl
-- next part --
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
URL:
<http://notmuchmail.org/pipermail/notmuch/attachments/20100205/487d054d/attachment.pgp>
[notmuch] [PATCH 3/3] notmuch.el: Use emacs built-in forward-button and backward-button
On Tue, 1 Dec 2009 14:13:53 +0800, Kan-Ru Chen wrote: > There are built-ins, so why not use them? Naturally. Thanks again---this is pushed. -Carl -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/7e37caf0/attachment.pgp>
[notmuch] [PATCH 2/3] notmuch.el: Add collapse all and expand all to notmuch-show
On Tue, 1 Dec 2009 14:17:32 +0800, Kan-Ru Chen wrote: > These two functions behave like gmail's collapse all and expand all > commands. notmuch-show-collapse-all is bound to 'B' but > notmuch-show-expand-all has no keybindig because I thought it is not often > used. I haven't applied this one for a few reasons: 1. I don't know what the functions are support to do. The commit log says "behave like gmail" but I don't know what that means personally. And the functions don't have any documentation strings. I assume that these functions are either showing or hiding all message bodies in the current thread? 2. You provided a binding for collapse, but not expand. I don't like functionality which lets a user hide a bunch of data, and then not be able to get it back---that's really annoying if the user hits the key accidentally. Personally, I think I'd be much more likely to use expand before collapse, (for example, to easily see the context when a new message arrives in a thread that I'd previously read so comes up collapsed by default). Should we perhaps make something which temporarily makes all hidden messages visible but then toggles back to the previous subset of some show messages? Once again, I'm worried about providing trapdoor interfaces where the user can't get back to the previous state easily. -Carl -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/b60dae32/attachment.pgp>
[notmuch] strange behavior of indexing of and searching for strings containing '[]'
Hey, folks. I've been noticing some strange behavior of notmuch search results for strings containing '[]'. Here are some searches for some exact strings in messages subjects: servo:~ 0$ notmuch search subject:'emacs paned UI' thread:533da424197bb6ba61a42b667d5d8d8f Wed. 14:12 [2/2] Tad Fisher, Jameson Rollins; [notmuch] Emacs paned UI () servo:~ 0$ So that's fine and expected. This however is not: servo:~ 0$ notmuch search subject:'[notmuch] emacs paned UI' thread:5f2cb4b108773a39161b33c86e54f7fd 4 mins. ago [1/1] Jameson Rollins; [notmuch] loss of duplicate messages (inbox) servo:~ 0$ Not only did it not turn up the message that *does* match that exact string in it's subject line, it actually turns up a completely different message that doesn't match the search term at all! This search actually turns up both: servo:~ 0$ notmuch search subject:'notmuch emacs paned UI' thread:5f2cb4b108773a39161b33c86e54f7fd 5 mins. ago [1/1] Jameson Rollins; [notmuch] loss of duplicate messages (inbox) thread:533da424197bb6ba61a42b667d5d8d8f Wed. 14:12 [2/2] Tad Fisher, Jameson Rollins; [notmuch] Emacs paned UI () servo:~ 0$ Which is again strange, because the second message does not at all match that search term. Does anyone have any idea what's going on here? I think I saw mention of this issue on IRC somewhere, but I thought I should bring it up explicitly here. This is definitely some buggy behavior. jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/5ea39095/attachment.pgp>
[notmuch] [PATCH 1/3] notmuch.el: Add keybinding to toggle display of message body and headers.
On Tue, 1 Dec 2009 14:13:51 +0800, Kan-Ru Chen wrote: > I really missed this feature. Added notmuch-show-toggle-current-body and > notmuch-show-toggle-current-header and bind them to 'b' and 'h'. Thanks! I pushed this out, (along with a fix to use forward-line instead of next-line, and also adding the missing documentation for the notmuch-show-toggle-current-header function). -Carl PS. Sorry I took so long on an obviously-useful patch! -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/44fd7d47/attachment.pgp>
[notmuch] loss of duplicate messages
Hey, folks. I'm noticing a somewhat problematic behavior of notmuch that I was wondering if anyone could comment on. I'm noticing that notmuch is either not syncing, or not returning in searches, duplicate messages that have identical bodies but different headers. This comes up when I send messages to lists to which I am subscribed. I have copies of my sent mail saved locally, so I generally have two copies of emails that I send to such lists: the one that I sent, and the one that I receive back from the list. Here's an example: servo:~ 0$ notmuch search subject:"emacs paned UI" thread:533da424197bb6ba61a42b667d5d8d8f Wed. 14:12 [2/2] Tad Fisher, Jameson Rollins; [notmuch] Emacs paned UI () servo:~ 0$ notmuch count subject:"emacs paned UI" 2 servo:~ 0$ grep -r -i "emacs paned UI" .mail-notmuch/inbox .mail-notmuch/sent .mail-notmuch/inbox/new/1265078715_3.20270.servo,U=249614,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,:Subject: [notmuch] Emacs paned UI .mail-notmuch/inbox/new/1265224417_0.1998.servo,U=250039,FMD5=7e33429f656f1e6e9d79b29c3f82c57e:2,:Subject: Re: [notmuch] Emacs paned UI .mail-notmuch/sent/cur/1265224340.M66544P992Q1.servo:2,S:Subject: Re: [notmuch] Emacs paned UI servo:~ 0$ As you can see, notmuch returns two messages matching the search term, where as a simple grep turns up three, the original message, my response, and my response from the list, the latter two being exact duplicates except for the headers. The message that notmuch is returning is the one in my sent mail, presumably because it showed up in the index first, and the second identical one was just dropped. I'm not exactly sure what the correct behavior is here, but I would actually like to see my messages sent to the list returned to me. It's a way of verifying that they did go to the list, as well as getting a feeling for the round trip time. I personally wouldn't mind just seeing both copies of the message returned by notmuch, as I can just delete one of them if I don't want it to turn up again. Would this behavior be problematic in any way? Do folks have suggestions of other behaviors that might get around this problem? jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/4f054a21/attachment.pgp>
[notmuch] notmuch new: Memory problem (with uuencoded content)
On Thu, 26 Nov 2009 11:16:21 -0800, Carl Worth wrote: > Clearly, some experimenting is needed. Dominik, if you can share the > large file, (with either me alone or with the whole list), a pointer to > where we could download it would be appreciated. Dominik replied to me privately and described a way for me to create a file that replicates the bug. Here's a recipe I came up with from his description: mkdir tmp cd tmp/ echo [database]$'\n'path=mail > notmuch-config mkdir mail echo From: Me$'\n'To: You$'\n'Subject: uuencode$'\n' > mail/msg dd if=/dev/urandom of=blob bs=1024 count=10240 uuencode blob < blob >> mail/msg NOTMUCH_CONFIG=notmuch-config notmuch new So that's a 10MB blob of random data which uuencodes to a ~14MB mail file. And notmuch (before a patch I just pushed) chews on it for quite a while, consuming several hundred MB of memory and resulting finally in a 76MB Xapian database (with chert). I'm not sure if there is a Xapian bug there or not, (or perhaps a bug in how notmuch is using Xapian to generate the terms for this large of an email message). But the thing that's obvious to me is that indexing encoded data like this doesn't make any sense at all. So I've just pushed a set of patches to notmuch to make it detect uuencoded data within a mail message and ignore it. Of course, I also pushed a set of tests to the test suite for this, (and some new "notmuch search" tests while I was at it). -Carl -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/46bd9f2a/attachment-0001.pgp>
[notmuch] [PATCHv2] notmuch.el: colorize lines in notmuch-search based on thread tags.
I realize I should have mentioned that this patch is meant to supercede the previous patch. There are just a couple of minor improvements, including turning on highlighting for deleted threads by default. jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20100205/65b775d7/attachment.pgp>
[notmuch] [PATCH 2/2] notmuch-reply.c: Handle munged `Reply-To' headers.
On Thu, 04 Feb 2010 12:54:20 -0800, Carl Worth wrote: > And I'm glad I did because that turned up a bug in the patch, (using > == instead of != for the return value of strcasestr resulted in *all* > messages with a Reply-To header being considered as munged). Yikes, I've been using this thing for two months and hadn't noticed. > Here's one cleanup I made which you might find interesting as a style > issue (where I prefer naming a function based on what it *does* rather > than on what it's being *used* for): Yup, I do the same, but must have been too lazy to think of a decent name. Jed
Re: [notmuch] [PATCHv2] notmuch.el: colorize lines in notmuch-search based on thread tags.
I realize I should have mentioned that this patch is meant to supercede the previous patch. There are just a couple of minor improvements, including turning on highlighting for deleted threads by default. jamie. pgpXjXWTROZ0t.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
