new "crypto" branch providing full PGP/MIME support
On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins wrote: > Please test and provide feedback. I would really like to see this > series merged into the mainline for the next release, if at all > possible. Fan. Tastic. Thanks so much for this, incredible work! Grabbed and testing now, the only slight niggle I see is that when I reply to an encrypted mail I still get: On Thu, 03 Feb 2011 19:17:45 +, Someone wrote: Non-text part: application/pgp-encrypted Non-text part: application/octet-stream Can we pass the decrypted text to message mode on a reply? -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/ad85cf74/attachment.pgp>
notmuch-show-get-filename and maildir location
I recently upgraded to the newer notmuch to test out the new folder patch (its great!) and the new decryption/signature verification (its fantastic!), and noticed something odd start happening. I think that notmuch moves a new message to the maildir cur directory,
[PATCH 2/2] notmuch.el:notmuch-search-process-filter: Rewritten. Cope with incomplete lines.
Hallo! On Thu, 3 Feb 2011 12:06:20 -0500, Austin Clements wrote: > Is there a reason you keep the remaining data in a string instead of > taking the more idiomatic elisp approach of leaving it in the process > buffer? In fact, the code would probably be simpler if you > immediately appended the string to the process buffer like a normal > process-filter and then peeled things away using buffer-oriented > regexp functions like looking-at. Elisp is a lot better at > manipulating buffers than it is at manipulating strings. Ha, I hear you -- this is what I meant to do originally. But then, the save-in-string approach (even though I always considered keeping state in the string a bit ugly) seemed more simple to me. As I said: writing elisp code is not my primary profession... :-) (Perhaps I should buy a book about it, or something.) Now that you confirmed my original idea, I'll see about re-writing the code accordingly, so thanks for the input! Gr??e, Thomas -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 489 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/4b4df877/attachment.pgp>
always encrypting messages to self [was: Re: new "crypto" branch providing full PGP/MIME support]
On 02/03/2011 03:34 PM, Jameson Rollins wrote: > On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor fifthhorseman.net> wrote: >> You either want to fix this in your emacs config by putting your >> fingerprint into mml2015-signers and setting mml2015-encrypt-to-self >> >> Or you want to set gpg's default-recipient-self option (and >> default-recipient option if you hold more than one secret key and want >> to be sure it chooses the right one) > > Actually, I think the gpg option we're looking for here is > "encrypt-to". "default-recipient-self" sets the recipient only if none > other is specified. I just set "encrypt-to " in my gpg.conf > and it seems to do as expected (all encrypted messages are also > encrypted to myself). Yes, this is correct. thanks for figuring that out, Jamie. --dkg -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1030 bytes Desc: OpenPGP digital signature URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/65bc54a1/attachment.pgp>
new "crypto" branch providing full PGP/MIME support
Hi Micah-- just wanted to follow up on your points/questions: On 02/03/2011 11:25 AM, micah anderson wrote: > 1. I personally think notmuch-show-process-pgpmime should default to > true note that with it set to false, you can still M-RET (instead of RET) on an item in the summary window to have it set for that particular view. > 2. in-line pgp messages don't have any processing done on them. getting > the mime-encoded processing work is a huge step and I'm happy that > works, in-line can (and IMHO, should) come later yeah, inline is a whole different thing, and much more difficult to manage programmatically in the notmuch backend, i think. I dealing with inline signatures and encryption should be a job for the emacs (or vim or whatever) frontend. > 3. i'm not sure expired/revoked keys are handled properly - tested on a > message that was encrypted by a key that was revoked and got "End of > file during parsing" when you say "encrypted by" do you mean "encrypted to"? do you have access to the corresponding secret key? > 4. messages that I sent encrypted to someone are not also encrypted to > myself, which means that a thread which contains my replies isn't able > to decrypt my messages in that thread and results in a purple > 'decryption error'. Perhaps this is an emacs UI tweak that needs to be > made to get messages also encrypted to my own key? this is an issue for the emacs message modes (or maybe for your gpg configuration), not for notmuch. You either want to fix this in your emacs config by putting your fingerprint into mml2015-signers and setting mml2015-encrypt-to-self Or you want to set gpg's default-recipient-self option (and default-recipient option if you hold more than one secret key and want to be sure it chooses the right one) > 5. unknown keys are represented in a long format, > eg. '0x5585F58CC827A062' when most tools represent them just with their > shortened keyid (in this case this one would be: 0xC827A062), is there a > particular reason for this? Short keyIDs are easily spoofable; believing anything based on a matched short keyID is a mistake. "unknown keys" themselves may or may not have properly signed a message -- since we don't have the key handy, we don't have a way of checking. note that "unknown key" is different from "good signature from known key but we do not know who controls the key" > I recognize some people's keyids in the > short form, but do not in the longform. You can derive the short form from the long form by ignoring everything but the last 8 hex characters. But if you actually recognize the short form, i'd expect you to have the relevant key in your keyring; is this really a use case worth bothering with? do you have a suggestion for how you think it should behave differently? --dkg -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1030 bytes Desc: OpenPGP digital signature URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/8c4b917c/attachment.pgp>
new "crypto" branch providing full PGP/MIME support
On Thu, 03 Feb 2011 20:42:38 +, Darren McGuicken wrote: > On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins finestructure.net> wrote: > > Please test and provide feedback. I would really like to see this > > series merged into the mainline for the next release, if at all > > possible. > > Fan. Tastic. Thanks so much for this, incredible work! Thanks! I hope people find it useful. > Grabbed and testing now, the only slight niggle I see is that when I > reply to an encrypted mail I still get: > > On Thu, 03 Feb 2011 19:17:45 +, Someone wrote: > Non-text part: application/pgp-encrypted > Non-text part: application/octet-stream > > Can we pass the decrypted text to message mode on a reply? Yeah, some folks pointed this out on #notmuch this morning. The decryption is only happening in notmuch-show, but we need to do it in notmuch-reply as well for the decrypted text to show up in the reply. I'm working on it now. jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/9b51ebd4/attachment.pgp>
new "crypto" branch providing full PGP/MIME support
On 02/02/2011 08:18 PM, Jameson Rollins wrote: > Hi, all. I have pushed a new branch called "crypto" to my notmuch > repository [0]. This branch provides full support for PGP/MIME signed > and encrypted messages, including emacs UI support. I have tested this, and am now using it. I'm very happy with it. I support its inclusion in the mainline. Thanks for doing this, Jamie. This is excellent! less important stuff follows I want to raise one (non-blocking) question about the decryption to see if anyone has any suggestions: If you do "notmuch show --format=json" on a PGP/MIME-encrypted plaintext message, it emits the base message, which is structured like this: 1 ???multipart/encrypted 2 ??application/pgp-encrypted attachment 3 ??application/octet-stream inline [msg.asc] with these patches, if you do "notmuch show --format=json --decrypt", it emits this instead: 1 ???multipart/encrypted 2 ??text/plain inline and it attaches an encstatus (and possibly sigstatus, if the message was signed) to part 1. I'll call this "method A". There are other methods that could be used as well, and it's worth making sure we've chosen one that we think is what we'll want in the future. here are two other proposals: Method B: 1 ???multipart/encrypted 2 ??application/pgp-encrypted attachment 3 ??text/plain inline That is, just replace part 3 (the actual encrypted body) with the decrypted material. This has the advantage that for single-part encrypted messages, the structure and part numbers of the message remains the same as without --decrypt. Method C: 1 ??text/plain inline That is, replace the entire multipart/encrypted with the decrypted material. This avoids having an explicitly-labeled "multipart/encrypted" wrapper around cleartext (which might be considered odd). It would mean attaching the encstatus and sigstatus directly to the decrypted part, though. I don't actually see any of these methods as being significantly better than the others -- i think they all have some inherent ugliness. So i'm fine with going with method A as Jamie chose it and has it working. But i wanted to see if anyone had strong arguments in favor of the other methods (or if there are other --decrypt methods we could use, for that matter) --dkg -- next part -- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 1030 bytes Desc: OpenPGP digital signature URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/7c936d02/attachment.pgp>
new "crypto" branch providing full PGP/MIME support
On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor wrote: > On 02/03/2011 11:25 AM, micah anderson wrote: > > 1. I personally think notmuch-show-process-pgpmime should default to > > true > > note that with it set to false, you can still M-RET (instead of RET) on > an item in the summary window to have it set for that particular view. This is also useful if you set notmuch-show-process-pgpmime and ever come across a message that is causing crypto problems. M-RET will return you to the normal view. > > 3. i'm not sure expired/revoked keys are handled properly - tested on a > > message that was encrypted by a key that was revoked and got "End of > > file during parsing" > > when you say "encrypted by" do you mean "encrypted to"? do you have > access to the corresponding secret key? I also seem to be noticing issues with revoked keys. I'm looking in to the issue. If anyone else notices something similar, please do relay your experience. > > 4. messages that I sent encrypted to someone are not also encrypted to > > myself, which means that a thread which contains my replies isn't able > > to decrypt my messages in that thread and results in a purple > > 'decryption error'. Perhaps this is an emacs UI tweak that needs to be > > made to get messages also encrypted to my own key? > > this is an issue for the emacs message modes (or maybe for your gpg > configuration), not for notmuch. > > You either want to fix this in your emacs config by putting your > fingerprint into mml2015-signers and setting mml2015-encrypt-to-self > > Or you want to set gpg's default-recipient-self option (and > default-recipient option if you hold more than one secret key and want > to be sure it chooses the right one) Actually, I think the gpg option we're looking for here is "encrypt-to". "default-recipient-self" sets the recipient only if none other is specified. I just set "encrypt-to " in my gpg.conf and it seems to do as expected (all encrypted messages are also encrypted to myself). jamie. -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/491434d0/attachment.pgp>
[PATCH 2/2] notmuch.el:notmuch-search-process-filter: Rewritten. Cope with incomplete lines.
Nice catch. Is there a reason you keep the remaining data in a string instead of taking the more idiomatic elisp approach of leaving it in the process buffer? In fact, the code would probably be simpler if you immediately appended the string to the process buffer like a normal process-filter and then peeled things away using buffer-oriented regexp functions like looking-at. Elisp is a lot better at manipulating buffers than it is at manipulating strings. On Wed, Feb 2, 2011 at 6:56 PM, Thomas Schwinge wrote: > This issue has been lying in ambush as of 2009-11-24's commit > 93af7b574598637c2766dd1f8ef343962c9a8efb. > > Signed-off-by: Thomas Schwinge > --- > ?emacs/notmuch.el | ? 70 +++-- > ?1 files changed, 41 insertions(+), 29 deletions(-) > > diff --git a/emacs/notmuch.el b/emacs/notmuch.el > index 3d82f0d..35ccee6 100644 > --- a/emacs/notmuch.el > +++ b/emacs/notmuch.el > @@ -641,9 +641,6 @@ non-authors is found, assume that all of the authors > match." > ? ? (propertize authors 'face 'notmuch-search-matching-authors))) > > ?(defun notmuch-search-insert-authors (format-string authors) > - ?;; Save the match data to avoid interfering with > - ?;; `notmuch-search-process-filter'. > - ?(save-match-data > ? ? (let* ((formatted-authors (format format-string authors)) > ? ? ? ? ? (formatted-sample (format format-string "")) > ? ? ? ? ? (visible-string formatted-authors) > @@ -709,7 +706,7 @@ non-authors is found, assume that all of the authors > match." > ? ? ? ? ?(setq overlay (make-overlay start (point))) > ? ? ? ? ?(overlay-put overlay 'invisible invis-spec) > ? ? ? ? ?(overlay-put overlay 'isearch-open-invisible > #'notmuch-search-isearch-authors-show))) > - ? ? ?(insert padding > + ? ? ?(insert padding))) > > ?(defun notmuch-search-insert-field (field date count authors subject tags) > ? (cond > @@ -736,6 +733,10 @@ non-authors is found, assume that all of the authors > match." > ? ? ? ? ?do (notmuch-search-insert-field field date count authors subject > tags))) > ? (insert "\n")) > > +(defvar notmuch-search-process-filter-data nil > + ?"Data that has not yet been processed.") > +(make-variable-buffer-local 'notmuch-search-process-filter-data) > + > ?(defun notmuch-search-process-filter (proc string) > ? "Process and filter the output of \"notmuch search\"" > ? (let ((buffer (process-buffer proc)) > @@ -743,31 +744,41 @@ non-authors is found, assume that all of the authors > match." > ? ? (if (buffer-live-p buffer) > ? ? ? ?(with-current-buffer buffer > ? ? ? ? ?(save-excursion > - ? ? ? ? ? (let ((line 0) > - ? ? ? ? ? ? ? ? (more t) > - ? ? ? ? ? ? ? ? (inhibit-read-only t)) > - ? ? ? ? ? ? (while more > - ? ? ? ? ? ? ? (if (string-match "^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) > \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$" string line) > - ? ? ? ? ? ? ? ? ? (let* ((thread-id (match-string 1 string)) > - ? ? ? ? ? ? ? ? ? ? ? ? ?(date (match-string 2 string)) > - ? ? ? ? ? ? ? ? ? ? ? ? ?(count (match-string 3 string)) > - ? ? ? ? ? ? ? ? ? ? ? ? ?(authors (match-string 4 string)) > - ? ? ? ? ? ? ? ? ? ? ? ? ?(subject (match-string 5 string)) > - ? ? ? ? ? ? ? ? ? ? ? ? ?(tags (match-string 6 string)) > - ? ? ? ? ? ? ? ? ? ? ? ? ?(tag-list (if tags (save-match-data (split-string > tags) > - ? ? ? ? ? ? ? ? ? ? (goto-char (point-max)) > - ? ? ? ? ? ? ? ? ? ? (let ((beg (point-marker))) > - ? ? ? ? ? ? ? ? ? ? ? (notmuch-search-show-result date count authors > subject tags) > - ? ? ? ? ? ? ? ? ? ? ? (notmuch-search-color-line beg (point-marker) > tag-list) > - ? ? ? ? ? ? ? ? ? ? ? (put-text-property beg (point-marker) > 'notmuch-search-thread-id thread-id) > - ? ? ? ? ? ? ? ? ? ? ? (put-text-property beg (point-marker) > 'notmuch-search-authors authors) > - ? ? ? ? ? ? ? ? ? ? ? (put-text-property beg (point-marker) > 'notmuch-search-subject subject) > - ? ? ? ? ? ? ? ? ? ? ? (if (string= thread-id notmuch-search-target-thread) > - ? ? ? ? ? ? ? ? ? ? ? ? ? (progn > - ? ? ? ? ? ? ? ? ? ? ? ? ? ? (set 'found-target beg) > - ? ? ? ? ? ? ? ? ? ? ? ? ? ? (set 'notmuch-search-target-thread "found" > - ? ? ? ? ? ? ? ? ? ? (set 'line (match-end 0))) > - ? ? ? ? ? ? ? ? (set 'more nil) > + ? ? ? ? ? (let ((inhibit-read-only t) > + ? ? ? ? ? ? ? ? ;; We may have a partial line saved from the last iteration. > + ? ? ? ? ? ? ? ? (string (concat notmuch-search-process-filter-data string)) > + ? ? ? ? ? ? ? ? (start 0)) > + ? ? ? ? ? ? (goto-char (point-max)) > + ? ? ? ? ? ? ;; Split `string' into lines. > + ? ? ? ? ? ? (while (string-match "\n" string start) > + ? ? ? ? ? ? ? (let ((line (substring string start (match-beginning 0 > + ? ? ? ? ? ? ? ? ;; Save the beginning of the next line already here, so that > + ? ? ? ? ? ? ? ? ;; we can mangle the match data later on. > + ? ? ? ? ? ? ? ? (setq start (match-end 0)) > + ? ? ? ? ? ? ? ? (if (string-match > + ? ? ? ? ? ? ? ? ? ? ?"^\\(thread:[0-9A-Fa-f]*\\)
notmuch's idea of concurrency / failing an invocation
On Thu, 27 Jan 2011 12:35:16 -0800, Jameson Rollins wrote: > On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson > wrote: > > Due to my harddisk in my laptop being slow (5400RPM), my notmuch > > database growing, and perhaps some fragmentation somewhere, this has > > become *incredibly* annoying for me. I am checking email every 30 > > minutes, and I'm nicing and ionicing the processes so I can use my > > machine, but while those processes are running, I'm effectively locked > > out of a good portion of my email. > > I also have a very slow disk, but this is very rarely a problem for me. > I retrieve mail every 10 minutes, and the corresponding notmuch new > usually takes a minute or so. I really haven't found it to be much of a > bother to just wait it out. Sometimes I can have several thousand messages to add/remove from the database. I know this is probably unusal, but for me its not due to system emails. I suppose if I checked my email more frequently, I'd have less to process on each run, but thats more side-stepping the concurrency issue. micah -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/e1123cff/attachment.pgp>
new "crypto" branch providing full PGP/MIME support
On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins wrote: > Hi, all. I have pushed a new branch called "crypto" to my notmuch > repository [0]. This branch provides full support for PGP/MIME signed > and encrypted messages, including emacs UI support. It has been applied > on top of cworth's current master (21e97c50). It includes the > following: > > * David Edmondson's improved multipart handling patch series (cherry-picked) > * Daniel Gillmor's PGP/MIME signature verification patch series > (cherry-picked) > * my PGP/MIME decryption+verification patch series > * a test suite for signature verification and decryption > * emacs support for the above Don't forget that you also included man page changes! > Please test and provide feedback. I would really like to see this > series merged into the mainline for the next release, if at all > possible. I've really really really wanted this functionality, so I pulled this right away and have been testing it, its really slick! I like how the emacs UI gives you good visual feedback for different signature states (I had red for a signature from Sebastian Spaeth because I did not have the key; orange for when I obtained that key; and green for Jameson and dkg's mails because I have exchanged keys with them and have full validity for them; and purple for a decryption error). The minor delay in opening a thread with signatures is not bad, and is to be expected. And messages that are PGP/MIME encrypted are decrypted automatically, wow, this is amazing. I enthusiastically support merging this into mainline for the next release. I have a couple points of feedback that I do not think should hold up merging this work: 1. I personally think notmuch-show-process-pgpmime should default to true 2. in-line pgp messages don't have any processing done on them. getting the mime-encoded processing work is a huge step and I'm happy that works, in-line can (and IMHO, should) come later 3. i'm not sure expired/revoked keys are handled properly - tested on a message that was encrypted by a key that was revoked and got "End of file during parsing" 4. messages that I sent encrypted to someone are not also encrypted to myself, which means that a thread which contains my replies isn't able to decrypt my messages in that thread and results in a purple 'decryption error'. Perhaps this is an emacs UI tweak that needs to be made to get messages also encrypted to my own key? 5. unknown keys are represented in a long format, eg. '0x5585F58CC827A062' when most tools represent them just with their shortened keyid (in this case this one would be: 0xC827A062), is there a particular reason for this? I recognize some people's keyids in the short form, but do not in the longform. 6. this is awesome, huge thanks to everyone who has worked on this! micah -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 835 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/e0066155/attachment.pgp>
[PATCH 4/4] ruby: Add generated files to gitignore
On Mon, 31 Jan 2011 23:48:57 +0100, Thomas Schwinge wrote: > Hallo! > > On Mon, 10 Jan 2011 16:39:28 +0200, Ali Polatel wrote: > > --- a/.gitignore > > +++ b/.gitignore > > @@ -13,3 +13,7 @@ libnotmuch.so* > > .*.swp > > *.elc > > releases > > + > > +bindings/ruby/mkmf.log > > +bindings/ruby/notmuch.so > > +bindings/ruby/Makefile > > These should rather be put into bindings/ruby/.gitignore, I'd say. Good idea! Just pushed ca69ce2397c0bb9b0e590a80b898ab4d58bb > > Gr??e, > Thomas -- Regards, Ali Polatel -- next part -- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: <http://notmuchmail.org/pipermail/notmuch/attachments/20110203/a4a9be91/attachment.pgp>
Folder search semantics (was Re: [RFC PATCH v2 0/8] Custom query parser, date search, folder search, and more)
Quoth Carl Worth on Feb 02 at 2:48 pm: > Restricting my reply to one tiny bit of your mail: > > You wrote: > > non-recursive is the only thing that makes sense for Maildir++ folders > > Either I'm not understanding Maildir++ folders, or I don't agree with > you. > > I might have an email archive that looks like this: > > Maildir > .work > .project1 > .project2 > .etc... > .family > .dad > .mom > .brother > .etc... > > With the above setup, what would be unreasonable about wanting to search > for all work-related messages (across all projects, say) with a string > like "folder:work" ? > > Now, a person might definitely want to search for messages in the > ".work" folder directly, (not including the sub-folders), so we should > provide support for users to get at that behavior as well, (such as a > proposed "folder:work$" or so). > > To me, both cases are perfectly legitimate, and I don't understand an > argument that claims that only one makes sense. (Or again, I may be > misunderstanding something.) (Somebody with more first-hand Maildir++ experience should jump in here. I stopped using Maildir++ a long time ago, so I may have no idea what I'm talking about.) Both cases are perfectly legitimate. However, the issue with Maildir++ is that the inbox is stored in the top-level directory: Maildir cur new tmp .work .work.project1 As a consequence, all folders are subfolders of the inbox. With recursive search, a search for your inbox folder returns *all* of your messages. I wasn't trying to say that we shouldn't support recursive search (I'm all for flexibility), but it's a confusing default for Maildir++ because of this. Maildir++ has the added twist that the inbox folder has no name. As a result, currently notmuch can't search for a Maildir++ inbox folder, which needs to be addressed somehow. The least surprising approach would compatibility with the Maildir++ convention of calling the top-level folder INBOX, the subfolder INBOX.work, etc. Maildir++ issues aside, I submit that rooted, non-recursive folder searches are a more natural default with a more conventional syntactic extension to non-rooted/recursive searches. In id:87aaiy3u65.fsf at yoom.home.cworth.org, you mentioned that you implemented non-rooted folder search to mimic subject search. But file system paths are not natural language like subject lines. File system paths are hierarchical and rooted. Of course, special query operators like ^ and $ can mitigate this, but these queries *aren't* regexps and, furthermore, people don't usually apply regexps to file names. They apply globs. Glob syntax has the added benefit of congruity with Xapian wildcard syntax. This naturally leads to a rooted, non-recursive syntax by default (like globs), where a * at the end means recursive and a * at the beginning means non-rooted. In fact, we could easily generalize this to arbitrary shell globs. Here's a proposal that, I think, addresses Maildir++ inboxes and subfolders; rooted, non-rooted, recursive, and non-recursive queries; and then some. Plus, it wouldn't require many code changes; you've already done the hard work. Switch XFOLDER from a probabilistic prefix with word-splitting to a boolean prefix without word-splitting. When indexing, strip off the cur or new and examine the resulting directory name. If it's the mail root, this is a Maildir++ inbox, so add the term XFOLDERINBOX. If it starts with a dot, it's a Maildir++ subfolder, so add the term XFOLDERINBOX<.dirname>. Otherwise, add the term XFOLDER. Then, using a custom query transform for the "folder:" prefix, enumerate XFOLDER terms and form a synonym query out of those that fnmatch the user's folder query.
[PATCH 2/2] notmuch.el:notmuch-search-process-filter: Rewritten. Cope with incomplete lines.
This issue has been lying in ambush as of 2009-11-24's commit 93af7b574598637c2766dd1f8ef343962c9a8efb. Signed-off-by: Thomas Schwinge --- emacs/notmuch.el | 70 +++-- 1 files changed, 41 insertions(+), 29 deletions(-) diff --git a/emacs/notmuch.el b/emacs/notmuch.el index 3d82f0d..35ccee6 100644 --- a/emacs/notmuch.el +++ b/emacs/notmuch.el @@ -641,9 +641,6 @@ non-authors is found, assume that all of the authors match." (propertize authors 'face 'notmuch-search-matching-authors))) (defun notmuch-search-insert-authors (format-string authors) - ;; Save the match data to avoid interfering with - ;; `notmuch-search-process-filter'. - (save-match-data (let* ((formatted-authors (format format-string authors)) (formatted-sample (format format-string "")) (visible-string formatted-authors) @@ -709,7 +706,7 @@ non-authors is found, assume that all of the authors match." (setq overlay (make-overlay start (point))) (overlay-put overlay 'invisible invis-spec) (overlay-put overlay 'isearch-open-invisible #'notmuch-search-isearch-authors-show))) - (insert padding + (insert padding))) (defun notmuch-search-insert-field (field date count authors subject tags) (cond @@ -736,6 +733,10 @@ non-authors is found, assume that all of the authors match." do (notmuch-search-insert-field field date count authors subject tags))) (insert "\n")) +(defvar notmuch-search-process-filter-data nil + "Data that has not yet been processed.") +(make-variable-buffer-local 'notmuch-search-process-filter-data) + (defun notmuch-search-process-filter (proc string) "Process and filter the output of \"notmuch search\"" (let ((buffer (process-buffer proc)) @@ -743,31 +744,41 @@ non-authors is found, assume that all of the authors match." (if (buffer-live-p buffer) (with-current-buffer buffer (save-excursion - (let ((line 0) - (more t) - (inhibit-read-only t)) - (while more - (if (string-match "^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$" string line) - (let* ((thread-id (match-string 1 string)) - (date (match-string 2 string)) - (count (match-string 3 string)) - (authors (match-string 4 string)) - (subject (match-string 5 string)) - (tags (match-string 6 string)) - (tag-list (if tags (save-match-data (split-string tags) - (goto-char (point-max)) - (let ((beg (point-marker))) - (notmuch-search-show-result date count authors subject tags) - (notmuch-search-color-line beg (point-marker) tag-list) - (put-text-property beg (point-marker) 'notmuch-search-thread-id thread-id) - (put-text-property beg (point-marker) 'notmuch-search-authors authors) - (put-text-property beg (point-marker) 'notmuch-search-subject subject) - (if (string= thread-id notmuch-search-target-thread) - (progn - (set 'found-target beg) - (set 'notmuch-search-target-thread "found" - (set 'line (match-end 0))) - (set 'more nil) + (let ((inhibit-read-only t) + ;; We may have a partial line saved from the last iteration. + (string (concat notmuch-search-process-filter-data string)) + (start 0)) + (goto-char (point-max)) + ;; Split `string' into lines. + (while (string-match "\n" string start) + (let ((line (substring string start (match-beginning 0 + ;; Save the beginning of the next line already here, so that + ;; we can mangle the match data later on. + (setq start (match-end 0)) + (if (string-match + "^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$" + line) + (let* ((thread-id (match-string 1 line)) +(date (match-string 2 line)) +(count (match-string 3 line)) +(authors (match-string 4 line)) +(subject (match-string 5 line)) +(tags (match-string 6 line)) +(tag-list (if tags (split-string tags + (let ((beg (point-marker))) + (notmuch-search-show-result date count authors subject tags) +
[PATCH 1/2] New test: Emacs' forgetfulness.
Signed-off-by: Thomas Schwinge --- test/emacs-forgetfulness | 38 ++ test/notmuch-test|1 + 2 files changed, 39 insertions(+), 0 deletions(-) create mode 100755 test/emacs-forgetfulness diff --git a/test/emacs-forgetfulness b/test/emacs-forgetfulness new file mode 100755 index 000..e17b26f --- /dev/null +++ b/test/emacs-forgetfulness @@ -0,0 +1,38 @@ +#!/bin/bash + +test_description=Emacs\'\ forgetfulness + +. test-lib.sh + +# RFC822 imposes a 998 character limit per line. +x=0123456789 # 10 +x=$x$x$x$x$x$x$x$x$x$x # 100 +x=$x$x$x$x$x$x$x$x$x # 900 + +# If setting this ``too high'' (TODO: yet to be determined), Emacs will crash +# with a segmentation fault. +n=20 +for i in $(seq 1 $n); do + # Roughly 2 KiB per message. That is, we need two messages in order to + # exceed the typical size of the pipe buffer (4 KiB on commodity systems). + generate_message [subject]=$i-$x [from]=$i-$x at x.x +done +# With 20 messages ?? 2 KiB, we have about 10 full pipe buffers, which should be +# enough to trigger the erroneous behavior. + +notmuch new > /dev/null + +test_begin_subtest 'Search for all messages' +output=$(exec 2>&1; \ + diff -wu \ + <(notmuch search \* \ + | sed \ + -e 's%^thread:[0-9a-f]*\ %%' \ + -e 's%;%%'; \ + echo 'End of search results.'; \ + echo) \ + <(test_emacs 2>&1 \ + '(notmuch-search "*") (notmuch-test-wait) (message (buffer-string))')) +test_expect_equal "$output" '' + +test_done diff --git a/test/notmuch-test b/test/notmuch-test index 9d77c0f..2f11eac 100755 --- a/test/notmuch-test +++ b/test/notmuch-test @@ -36,6 +36,7 @@ TESTS=" encoding emacs maildir-sync + emacs-forgetfulness " # Clean up any results from a previous run -- 1.7.1
[BUG] Emacs UI dropping every 25th line, roughly
Hallo! Here is the problem, as suspected. On Wed, 02 Feb 2011 17:12:16 +0100, I wrote: > At this / in the middle of this point, the 4 KiB size is hit. After > accumulating some more (notice the 3.6 seconds delay), Emacs read()s the > first chunk (line breaks inserted for clarity): > > 16:26:57.928798 read(8, "[first two dozen results]" > "thread:06c7 2009-12-16 [2/2] > Thomas Schwinge; [subject] ([flags])\n" > "t", 4096) = 4095 > > The last result line is obviously incomplete, only the `t' so far. notmuch.el:notmuch-search-process-filter will be called for processing these lines. It'll do fine up to the single `t' -- which simply isn't a conforming line and thus will be dropped (which is questionable behavior anyway, I would think?). > There is more to be read, so Emacs quickly goes on with the next chunk: > > 16:26:57.966247 read(8, "hread:0ac4 2009-12-16 [1/1] jsm28 > at sourceware.org; [subject] ([flags])\n" > "[more results]", 4096) = 4095 Same thing; this time ``hread:[...]'' isn't a confirming line, and will be dropped. After that, regular processing continues. This problem has been around as of the beginning of the incremental / asynchronous search results changes, as documented in id:87aayatnw4.fsf at yoom.home.cworth.org on 2009-11-25; 2009-11-24's commit 93af7b574598637c2766dd1f8ef343962c9a8efb. Emacs LISP is not my speciality, neither is any other LISP dialect, so someone please carefully review this. Gr??e (und gute Nacht...), Thomas
[PATCH] Shell programming: directories are ``executable'' (that is, searchable), too.
Signed-off-by: Thomas Schwinge tho...@schwinge.name
---
test/test-lib.sh |5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/test/test-lib.sh b/test/test-lib.sh
index f536172..3471ead 100755
--- a/test/test-lib.sh
+++ b/test/test-lib.sh
@@ -815,13 +815,14 @@ EOF
}
+# Locate the directory containing the `notmuch' executable we are to use.
find_notmuch_path ()
{
dir=$1
while [ -n $dir ]; do
bin=$dir/notmuch
- if [ -x $bin ]; then
+ if [ -f $bin -a -x $bin ]; then
echo $dir
return
fi
@@ -858,7 +859,7 @@ then
make_valgrind_symlink () {
# handle only executables
- test -x $1 || return
+ test -f $1 -a -x $1 || return
base=$(basename $1)
symlink_target=$TEST_DIRECTORY/../$base
--
1.7.1
___
notmuch mailing list
notmuch@notmuchmail.org
http://notmuchmail.org/mailman/listinfo/notmuch
Re: [RFC PATCH v2 0/8] Custom query parser, date search, folder search, and more
Restricting my reply to one tiny bit of your mail: You wrote: non-recursive is the only thing that makes sense for Maildir++ folders Either I'm not understanding Maildir++ folders, or I don't agree with you. I might have an email archive that looks like this: Maildir .work .project1 .project2 .etc... .family .dad .mom .brother .etc... With the above setup, what would be unreasonable about wanting to search for all work-related messages (across all projects, say) with a string like folder:work ? Now, a person might definitely want to search for messages in the .work folder directly, (not including the sub-folders), so we should provide support for users to get at that behavior as well, (such as a proposed folder:work$ or so). To me, both cases are perfectly legitimate, and I don't understand an argument that claims that only one makes sense. (Or again, I may be misunderstanding something.) -Carl -- carl.d.wo...@intel.com pgpdyeeZ7h1Pw.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
[PATCH 2/2] notmuch.el:notmuch-search-process-filter: Rewritten. Cope with incomplete lines.
This issue has been lying in ambush as of 2009-11-24's commit 93af7b574598637c2766dd1f8ef343962c9a8efb. Signed-off-by: Thomas Schwinge tho...@schwinge.name --- emacs/notmuch.el | 70 +++-- 1 files changed, 41 insertions(+), 29 deletions(-) diff --git a/emacs/notmuch.el b/emacs/notmuch.el index 3d82f0d..35ccee6 100644 --- a/emacs/notmuch.el +++ b/emacs/notmuch.el @@ -641,9 +641,6 @@ non-authors is found, assume that all of the authors match. (propertize authors 'face 'notmuch-search-matching-authors))) (defun notmuch-search-insert-authors (format-string authors) - ;; Save the match data to avoid interfering with - ;; `notmuch-search-process-filter'. - (save-match-data (let* ((formatted-authors (format format-string authors)) (formatted-sample (format format-string )) (visible-string formatted-authors) @@ -709,7 +706,7 @@ non-authors is found, assume that all of the authors match. (setq overlay (make-overlay start (point))) (overlay-put overlay 'invisible invis-spec) (overlay-put overlay 'isearch-open-invisible #'notmuch-search-isearch-authors-show))) - (insert padding + (insert padding))) (defun notmuch-search-insert-field (field date count authors subject tags) (cond @@ -736,6 +733,10 @@ non-authors is found, assume that all of the authors match. do (notmuch-search-insert-field field date count authors subject tags))) (insert \n)) +(defvar notmuch-search-process-filter-data nil + Data that has not yet been processed.) +(make-variable-buffer-local 'notmuch-search-process-filter-data) + (defun notmuch-search-process-filter (proc string) Process and filter the output of \notmuch search\ (let ((buffer (process-buffer proc)) @@ -743,31 +744,41 @@ non-authors is found, assume that all of the authors match. (if (buffer-live-p buffer) (with-current-buffer buffer (save-excursion - (let ((line 0) - (more t) - (inhibit-read-only t)) - (while more - (if (string-match ^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$ string line) - (let* ((thread-id (match-string 1 string)) - (date (match-string 2 string)) - (count (match-string 3 string)) - (authors (match-string 4 string)) - (subject (match-string 5 string)) - (tags (match-string 6 string)) - (tag-list (if tags (save-match-data (split-string tags) - (goto-char (point-max)) - (let ((beg (point-marker))) - (notmuch-search-show-result date count authors subject tags) - (notmuch-search-color-line beg (point-marker) tag-list) - (put-text-property beg (point-marker) 'notmuch-search-thread-id thread-id) - (put-text-property beg (point-marker) 'notmuch-search-authors authors) - (put-text-property beg (point-marker) 'notmuch-search-subject subject) - (if (string= thread-id notmuch-search-target-thread) - (progn - (set 'found-target beg) - (set 'notmuch-search-target-thread found - (set 'line (match-end 0))) - (set 'more nil) + (let ((inhibit-read-only t) + ;; We may have a partial line saved from the last iteration. + (string (concat notmuch-search-process-filter-data string)) + (start 0)) + (goto-char (point-max)) + ;; Split `string' into lines. + (while (string-match \n string start) + (let ((line (substring string start (match-beginning 0 + ;; Save the beginning of the next line already here, so that + ;; we can mangle the match data later on. + (setq start (match-end 0)) + (if (string-match + ^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$ + line) + (let* ((thread-id (match-string 1 line)) +(date (match-string 2 line)) +(count (match-string 3 line)) +(authors (match-string 4 line)) +(subject (match-string 5 line)) +(tags (match-string 6 line)) +(tag-list (if tags (split-string tags + (let ((beg (point-marker))) + (notmuch-search-show-result date count authors subject tags) +
Re: [BUG] Emacs UI dropping every 25th line, roughly
Hallo! Here is the problem, as suspected. On Wed, 02 Feb 2011 17:12:16 +0100, I wrote: At this / in the middle of this point, the 4 KiB size is hit. After accumulating some more (notice the 3.6 seconds delay), Emacs read()s the first chunk (line breaks inserted for clarity): 16:26:57.928798 read(8, [first two dozen results] thread:06c7 2009-12-16 [2/2] Thomas Schwinge; [subject] ([flags])\n t, 4096) = 4095 The last result line is obviously incomplete, only the `t' so far. notmuch.el:notmuch-search-process-filter will be called for processing these lines. It'll do fine up to the single `t' -- which simply isn't a conforming line and thus will be dropped (which is questionable behavior anyway, I would think?). There is more to be read, so Emacs quickly goes on with the next chunk: 16:26:57.966247 read(8, hread:0ac4 2009-12-16 [1/1] js...@sourceware.org; [subject] ([flags])\n [more results], 4096) = 4095 Same thing; this time ``hread:[...]'' isn't a confirming line, and will be dropped. After that, regular processing continues. This problem has been around as of the beginning of the incremental / asynchronous search results changes, as documented in id:87aayatnw4@yoom.home.cworth.org on 2009-11-25; 2009-11-24's commit 93af7b574598637c2766dd1f8ef343962c9a8efb. Emacs LISP is not my speciality, neither is any other LISP dialect, so someone please carefully review this. Grüße (und gute Nacht...), Thomas ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
new crypto branch providing full PGP/MIME support
Hi, all. I have pushed a new branch called crypto to my notmuch repository [0]. This branch provides full support for PGP/MIME signed and encrypted messages, including emacs UI support. It has been applied on top of cworth's current master (21e97c50). It includes the following: * David Edmondson's improved multipart handling patch series (cherry-picked) * Daniel Gillmor's PGP/MIME signature verification patch series (cherry-picked) * my PGP/MIME decryption+verification patch series * a test suite for signature verification and decryption * emacs support for the above I have signed-off on all cherry-picked commits, as I have reviewed and tested them extensively (I have been using them in my daily notmuch use for many months now). The multipart and sig-verification patches were cherry-picked to get around a couple of accidental spurious commits in their originating branches that would have needed to be reverted. Please test and provide feedback. I would really like to see this series merged into the mainline for the next release, if at all possible. jamie. [0] git://finestructure.net/notmuch pgppeOSNllxAM.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Folder search semantics (was Re: [RFC PATCH v2 0/8] Custom query parser, date search, folder search, and more)
Quoth Carl Worth on Feb 02 at 2:48 pm: Restricting my reply to one tiny bit of your mail: You wrote: non-recursive is the only thing that makes sense for Maildir++ folders Either I'm not understanding Maildir++ folders, or I don't agree with you. I might have an email archive that looks like this: Maildir .work .project1 .project2 .etc... .family .dad .mom .brother .etc... With the above setup, what would be unreasonable about wanting to search for all work-related messages (across all projects, say) with a string like folder:work ? Now, a person might definitely want to search for messages in the .work folder directly, (not including the sub-folders), so we should provide support for users to get at that behavior as well, (such as a proposed folder:work$ or so). To me, both cases are perfectly legitimate, and I don't understand an argument that claims that only one makes sense. (Or again, I may be misunderstanding something.) (Somebody with more first-hand Maildir++ experience should jump in here. I stopped using Maildir++ a long time ago, so I may have no idea what I'm talking about.) Both cases are perfectly legitimate. However, the issue with Maildir++ is that the inbox is stored in the top-level directory: Maildir cur new tmp .work .work.project1 As a consequence, all folders are subfolders of the inbox. With recursive search, a search for your inbox folder returns *all* of your messages. I wasn't trying to say that we shouldn't support recursive search (I'm all for flexibility), but it's a confusing default for Maildir++ because of this. Maildir++ has the added twist that the inbox folder has no name. As a result, currently notmuch can't search for a Maildir++ inbox folder, which needs to be addressed somehow. The least surprising approach would compatibility with the Maildir++ convention of calling the top-level folder INBOX, the subfolder INBOX.work, etc. Maildir++ issues aside, I submit that rooted, non-recursive folder searches are a more natural default with a more conventional syntactic extension to non-rooted/recursive searches. In id:87aaiy3u65@yoom.home.cworth.org, you mentioned that you implemented non-rooted folder search to mimic subject search. But file system paths are not natural language like subject lines. File system paths are hierarchical and rooted. Of course, special query operators like ^ and $ can mitigate this, but these queries *aren't* regexps and, furthermore, people don't usually apply regexps to file names. They apply globs. Glob syntax has the added benefit of congruity with Xapian wildcard syntax. This naturally leads to a rooted, non-recursive syntax by default (like globs), where a * at the end means recursive and a * at the beginning means non-rooted. In fact, we could easily generalize this to arbitrary shell globs. Here's a proposal that, I think, addresses Maildir++ inboxes and subfolders; rooted, non-rooted, recursive, and non-recursive queries; and then some. Plus, it wouldn't require many code changes; you've already done the hard work. Switch XFOLDER from a probabilistic prefix with word-splitting to a boolean prefix without word-splitting. When indexing, strip off the cur or new and examine the resulting directory name. If it's the mail root, this is a Maildir++ inbox, so add the term XFOLDERINBOX. If it starts with a dot, it's a Maildir++ subfolder, so add the term XFOLDERINBOX.dirname. Otherwise, add the term XFOLDERdirname. Then, using a custom query transform for the folder: prefix, enumerate XFOLDER terms and form a synonym query out of those that fnmatch the user's folder query. ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 4/4] ruby: Add generated files to gitignore
On Mon, 31 Jan 2011 23:48:57 +0100, Thomas Schwinge tho...@schwinge.name wrote: Hallo! On Mon, 10 Jan 2011 16:39:28 +0200, Ali Polatel a...@exherbo.org wrote: --- a/.gitignore +++ b/.gitignore @@ -13,3 +13,7 @@ libnotmuch.so* .*.swp *.elc releases + +bindings/ruby/mkmf.log +bindings/ruby/notmuch.so +bindings/ruby/Makefile These should rather be put into bindings/ruby/.gitignore, I'd say. Good idea! Just pushed ca69ce2397c0bb9b0e590a80b898ab4d58bb Grüße, Thomas -- Regards, Ali Polatel pgpwScQVgJMNS.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: notmuch's idea of concurrency / failing an invocation
On Thu, 27 Jan 2011 12:35:16 -0800, Jameson Rollins jroll...@finestructure.net wrote: On Thu, 27 Jan 2011 13:40:25 -0500, micah anderson mi...@riseup.net wrote: Due to my harddisk in my laptop being slow (5400RPM), my notmuch database growing, and perhaps some fragmentation somewhere, this has become *incredibly* annoying for me. I am checking email every 30 minutes, and I'm nicing and ionicing the processes so I can use my machine, but while those processes are running, I'm effectively locked out of a good portion of my email. I also have a very slow disk, but this is very rarely a problem for me. I retrieve mail every 10 minutes, and the corresponding notmuch new usually takes a minute or so. I really haven't found it to be much of a bother to just wait it out. Sometimes I can have several thousand messages to add/remove from the database. I know this is probably unusal, but for me its not due to system emails. I suppose if I checked my email more frequently, I'd have less to process on each run, but thats more side-stepping the concurrency issue. micah pgp62R9y86RGi.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [PATCH 2/2] notmuch.el:notmuch-search-process-filter: Rewritten. Cope with incomplete lines.
Nice catch. Is there a reason you keep the remaining data in a string instead of taking the more idiomatic elisp approach of leaving it in the process buffer? In fact, the code would probably be simpler if you immediately appended the string to the process buffer like a normal process-filter and then peeled things away using buffer-oriented regexp functions like looking-at. Elisp is a lot better at manipulating buffers than it is at manipulating strings. On Wed, Feb 2, 2011 at 6:56 PM, Thomas Schwinge tho...@schwinge.name wrote: This issue has been lying in ambush as of 2009-11-24's commit 93af7b574598637c2766dd1f8ef343962c9a8efb. Signed-off-by: Thomas Schwinge tho...@schwinge.name --- emacs/notmuch.el | 70 +++-- 1 files changed, 41 insertions(+), 29 deletions(-) diff --git a/emacs/notmuch.el b/emacs/notmuch.el index 3d82f0d..35ccee6 100644 --- a/emacs/notmuch.el +++ b/emacs/notmuch.el @@ -641,9 +641,6 @@ non-authors is found, assume that all of the authors match. (propertize authors 'face 'notmuch-search-matching-authors))) (defun notmuch-search-insert-authors (format-string authors) - ;; Save the match data to avoid interfering with - ;; `notmuch-search-process-filter'. - (save-match-data (let* ((formatted-authors (format format-string authors)) (formatted-sample (format format-string )) (visible-string formatted-authors) @@ -709,7 +706,7 @@ non-authors is found, assume that all of the authors match. (setq overlay (make-overlay start (point))) (overlay-put overlay 'invisible invis-spec) (overlay-put overlay 'isearch-open-invisible #'notmuch-search-isearch-authors-show))) - (insert padding + (insert padding))) (defun notmuch-search-insert-field (field date count authors subject tags) (cond @@ -736,6 +733,10 @@ non-authors is found, assume that all of the authors match. do (notmuch-search-insert-field field date count authors subject tags))) (insert \n)) +(defvar notmuch-search-process-filter-data nil + Data that has not yet been processed.) +(make-variable-buffer-local 'notmuch-search-process-filter-data) + (defun notmuch-search-process-filter (proc string) Process and filter the output of \notmuch search\ (let ((buffer (process-buffer proc)) @@ -743,31 +744,41 @@ non-authors is found, assume that all of the authors match. (if (buffer-live-p buffer) (with-current-buffer buffer (save-excursion - (let ((line 0) - (more t) - (inhibit-read-only t)) - (while more - (if (string-match ^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$ string line) - (let* ((thread-id (match-string 1 string)) - (date (match-string 2 string)) - (count (match-string 3 string)) - (authors (match-string 4 string)) - (subject (match-string 5 string)) - (tags (match-string 6 string)) - (tag-list (if tags (save-match-data (split-string tags) - (goto-char (point-max)) - (let ((beg (point-marker))) - (notmuch-search-show-result date count authors subject tags) - (notmuch-search-color-line beg (point-marker) tag-list) - (put-text-property beg (point-marker) 'notmuch-search-thread-id thread-id) - (put-text-property beg (point-marker) 'notmuch-search-authors authors) - (put-text-property beg (point-marker) 'notmuch-search-subject subject) - (if (string= thread-id notmuch-search-target-thread) - (progn - (set 'found-target beg) - (set 'notmuch-search-target-thread found - (set 'line (match-end 0))) - (set 'more nil) + (let ((inhibit-read-only t) + ;; We may have a partial line saved from the last iteration. + (string (concat notmuch-search-process-filter-data string)) + (start 0)) + (goto-char (point-max)) + ;; Split `string' into lines. + (while (string-match \n string start) + (let ((line (substring string start (match-beginning 0 + ;; Save the beginning of the next line already here, so that + ;; we can mangle the match data later on. + (setq start (match-end 0)) + (if (string-match + ^\\(thread:[0-9A-Fa-f]*\\) \\([^][]*\\) \\(\\[[0-9/]*\\]\\) \\([^;]*\\); \\(.*\\) (\\([^()]*\\))$ +
Re: [PATCH 2/2] notmuch.el:notmuch-search-process-filter: Rewritten. Cope with incomplete lines.
Hallo! On Thu, 3 Feb 2011 12:06:20 -0500, Austin Clements amdra...@mit.edu wrote: Is there a reason you keep the remaining data in a string instead of taking the more idiomatic elisp approach of leaving it in the process buffer? In fact, the code would probably be simpler if you immediately appended the string to the process buffer like a normal process-filter and then peeled things away using buffer-oriented regexp functions like looking-at. Elisp is a lot better at manipulating buffers than it is at manipulating strings. Ha, I hear you -- this is what I meant to do originally. But then, the save-in-string approach (even though I always considered keeping state in the string a bit ugly) seemed more simple to me. As I said: writing elisp code is not my primary profession... :-) (Perhaps I should buy a book about it, or something.) Now that you confirmed my original idea, I'll see about re-writing the code accordingly, so thanks for the input! Grüße, Thomas pgpkcG4W4yhJj.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: new crypto branch providing full PGP/MIME support
On 02/02/2011 08:18 PM, Jameson Rollins wrote: Hi, all. I have pushed a new branch called crypto to my notmuch repository [0]. This branch provides full support for PGP/MIME signed and encrypted messages, including emacs UI support. I have tested this, and am now using it. I'm very happy with it. I support its inclusion in the mainline. Thanks for doing this, Jamie. This is excellent! less important stuff follows I want to raise one (non-blocking) question about the decryption to see if anyone has any suggestions: If you do notmuch show --format=json on a PGP/MIME-encrypted plaintext message, it emits the base message, which is structured like this: 1 └┬╴multipart/encrypted 2 ├╴application/pgp-encrypted attachment 3 └╴application/octet-stream inline [msg.asc] with these patches, if you do notmuch show --format=json --decrypt, it emits this instead: 1 └┬╴multipart/encrypted 2 └╴text/plain inline and it attaches an encstatus (and possibly sigstatus, if the message was signed) to part 1. I'll call this method A. There are other methods that could be used as well, and it's worth making sure we've chosen one that we think is what we'll want in the future. here are two other proposals: Method B: 1 └┬╴multipart/encrypted 2 ├╴application/pgp-encrypted attachment 3 └╴text/plain inline That is, just replace part 3 (the actual encrypted body) with the decrypted material. This has the advantage that for single-part encrypted messages, the structure and part numbers of the message remains the same as without --decrypt. Method C: 1 └╴text/plain inline That is, replace the entire multipart/encrypted with the decrypted material. This avoids having an explicitly-labeled multipart/encrypted wrapper around cleartext (which might be considered odd). It would mean attaching the encstatus and sigstatus directly to the decrypted part, though. I don't actually see any of these methods as being significantly better than the others -- i think they all have some inherent ugliness. So i'm fine with going with method A as Jamie chose it and has it working. But i wanted to see if anyone had strong arguments in favor of the other methods (or if there are other --decrypt methods we could use, for that matter) --dkg signature.asc Description: OpenPGP digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: new crypto branch providing full PGP/MIME support
Hi Micah-- just wanted to follow up on your points/questions: On 02/03/2011 11:25 AM, micah anderson wrote: 1. I personally think notmuch-show-process-pgpmime should default to true note that with it set to false, you can still M-RET (instead of RET) on an item in the summary window to have it set for that particular view. 2. in-line pgp messages don't have any processing done on them. getting the mime-encoded processing work is a huge step and I'm happy that works, in-line can (and IMHO, should) come later yeah, inline is a whole different thing, and much more difficult to manage programmatically in the notmuch backend, i think. I dealing with inline signatures and encryption should be a job for the emacs (or vim or whatever) frontend. 3. i'm not sure expired/revoked keys are handled properly - tested on a message that was encrypted by a key that was revoked and got End of file during parsing when you say encrypted by do you mean encrypted to? do you have access to the corresponding secret key? 4. messages that I sent encrypted to someone are not also encrypted to myself, which means that a thread which contains my replies isn't able to decrypt my messages in that thread and results in a purple 'decryption error'. Perhaps this is an emacs UI tweak that needs to be made to get messages also encrypted to my own key? this is an issue for the emacs message modes (or maybe for your gpg configuration), not for notmuch. You either want to fix this in your emacs config by putting your fingerprint into mml2015-signers and setting mml2015-encrypt-to-self Or you want to set gpg's default-recipient-self option (and default-recipient option if you hold more than one secret key and want to be sure it chooses the right one) 5. unknown keys are represented in a long format, eg. '0x5585F58CC827A062' when most tools represent them just with their shortened keyid (in this case this one would be: 0xC827A062), is there a particular reason for this? Short keyIDs are easily spoofable; believing anything based on a matched short keyID is a mistake. unknown keys themselves may or may not have properly signed a message -- since we don't have the key handy, we don't have a way of checking. note that unknown key is different from good signature from known key but we do not know who controls the key I recognize some people's keyids in the short form, but do not in the longform. You can derive the short form from the long form by ignoring everything but the last 8 hex characters. But if you actually recognize the short form, i'd expect you to have the relevant key in your keyring; is this really a use case worth bothering with? do you have a suggestion for how you think it should behave differently? --dkg signature.asc Description: OpenPGP digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: new crypto branch providing full PGP/MIME support
On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins jroll...@finestructure.net wrote: Please test and provide feedback. I would really like to see this series merged into the mainline for the next release, if at all possible. Fan. Tastic. Thanks so much for this, incredible work! Grabbed and testing now, the only slight niggle I see is that when I reply to an encrypted mail I still get: On Thu, 03 Feb 2011 19:17:45 +, Someone foo@bar wrote: Non-text part: application/pgp-encrypted Non-text part: application/octet-stream Can we pass the decrypted text to message mode on a reply? pgpvaIMNrRYvK.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: new crypto branch providing full PGP/MIME support
On Thu, 03 Feb 2011 20:42:38 +, Darren McGuicken mailing-notm...@fernseed.info wrote: On Wed, 02 Feb 2011 17:18:45 -0800, Jameson Rollins jroll...@finestructure.net wrote: Please test and provide feedback. I would really like to see this series merged into the mainline for the next release, if at all possible. Fan. Tastic. Thanks so much for this, incredible work! Thanks! I hope people find it useful. Grabbed and testing now, the only slight niggle I see is that when I reply to an encrypted mail I still get: On Thu, 03 Feb 2011 19:17:45 +, Someone foo@bar wrote: Non-text part: application/pgp-encrypted Non-text part: application/octet-stream Can we pass the decrypted text to message mode on a reply? Yeah, some folks pointed this out on #notmuch this morning. The decryption is only happening in notmuch-show, but we need to do it in notmuch-reply as well for the decrypted text to show up in the reply. I'm working on it now. jamie. pgpU7r4oBBd3U.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
always encrypting messages to self [was: Re: new crypto branch providing full PGP/MIME support]
On 02/03/2011 03:34 PM, Jameson Rollins wrote: On Thu, 03 Feb 2011 14:52:01 -0500, Daniel Kahn Gillmor d...@fifthhorseman.net wrote: You either want to fix this in your emacs config by putting your fingerprint into mml2015-signers and setting mml2015-encrypt-to-self Or you want to set gpg's default-recipient-self option (and default-recipient option if you hold more than one secret key and want to be sure it chooses the right one) Actually, I think the gpg option we're looking for here is encrypt-to. default-recipient-self sets the recipient only if none other is specified. I just set encrypt-to mykeyid in my gpg.conf and it seems to do as expected (all encrypted messages are also encrypted to myself). Yes, this is correct. thanks for figuring that out, Jamie. --dkg signature.asc Description: OpenPGP digital signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
notmuch-show-get-filename and maildir location
I recently upgraded to the newer notmuch to test out the new folder patch (its great!) and the new decryption/signature verification (its fantastic!), and noticed something odd start happening. I think that notmuch moves a new message to the maildir cur directory, From the new directory when you first read it. Is that right? This is actually what I would expect with maildirs, but notmuch wasn't doing this before. The reason why I care is because if I open a message, notmuch-show-get-filename tells me that the file is located in the 'new/' path, but it is actually not there at all, looking around it appears to be in the 'cur/' directory. This is because I made a function to let me do bayes training by putting mistaken emails in my Mistakes folder, so that they can be retrained: (defun notmuch-mark-as-mistake () Moves the current message into the Mistakes folder (interactive) (let* ((fullpath (notmuch-show-get-filename)) (filename (file-name-nondirectory fullpath))) (save-excursion (rename-file fullpath (concat /home/micah/Maildir/INBOX.Mistakes/new/ filename))) (notmuch-show-archive-thread-then-exit) (notmuch-search-refresh-view) (hl-line-mode 1)(hl-line-mode 1))) (define-key notmuch-show-mode-map (kbd S) 'notmuch-mark-as-mistake) if I hit the 'S' key on a message now, it complains that it cannot rename the file because it doesn't exist. Incidentally, does anyone know how I can do this without having to actually do a notmuch-show on the message? Most spam I can recognize by the subject in my inbox, and I dont need to open it to see that. micah pgpHTT9EUb8zD.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
