[O] Email - Org-mode: charset problem
Hi! Is there somebody who managed to develop an email to Org-mode bridge without having charset problems? (No, I do not use Emacs as a MUA[4]) Disclaimer: This is not directly related to Org-mode but I guess there are people here with the very same problem. I am using a procmail[1] with an entry to forward emails containing a keyword in the subject into my inbox.org (where also MobileOrg[2] entries are written to). This is quite handy since some tasks arrive at emails and I want to capture this everywhere (smartphone, webmail, ...) using a simply email forward. Unfortunately I get weird stuff like uuencoded things, UTF-8 in ASCII, ... which messes up my inbox.org and I am not able to read it afterwards.[5] I guess this is because procmail and formail[3] – the tools I am using to extract mail infos to append to the org-file – are 7-bit-only or similar. If this is the case, I guess I'll have to find a different approach for this purpose. Any ideas? 1. http://www.procmail.org/ 2. https://github.com/matburt/mobileorg-android/wiki/ 3. http://www.linuxcommand.org/man_pages/formail1.html 4. http://en.wikipedia.org/wiki/Mail_user_agent 5. I can give examples if you want. -- Karl Voit
Re: [O] Email - Org-mode: charset problem
At Mon, 27 Jun 2011 10:30:06 +0200, Karl Voit wrote: Hi! Is there somebody who managed to develop an email to Org-mode bridge without having charset problems? (No, I do not use Emacs as a MUA[4]) Disclaimer: This is not directly related to Org-mode but I guess there are people here with the very same problem. I am using a procmail[1] with an entry to forward emails containing a keyword in the subject into my inbox.org (where also MobileOrg[2] entries are written to). This is quite handy since some tasks arrive at emails and I want to capture this everywhere (smartphone, webmail, ...) using a simply email forward. Unfortunately I get weird stuff like uuencoded things, UTF-8 in ASCII, ... which messes up my inbox.org and I am not able to read it afterwards.[5] I guess this is because procmail and formail[3] – the tools I am using to extract mail infos to append to the org-file – are 7-bit-only or similar. If this is the case, I guess I'll have to find a different approach for this purpose. Any ideas? As far as I know a lot, if not most, email messages are transfered in 7bit for backward compatibility reasons and getting things other than text/plain; charset=us-ascii across the net via email requires a lot of things to consider (RFC2045-49 in all their glory). And if you receive message from others, their mua might get things wrong and you end up with things like utf-8 in ascii. So dumping the raw message won't help, you need functions for processing the raw message informaton. Here's an idea (or two). I do use an Emacs base mua (Wanderlust) and recently started to file reference notes for certain messages. Because in cases I want to keept the message (e.g. an interesting usenet post) I save the raw message to a temporary file and add it as an attachment. This works nice, although I'd really like to have an automated way of save+attache when capturing. So, what about org-capture? You could use a script that serializes the raw message to disk and -- somehow -- calls capture with a reference to this file. Emacs opens the mail file and extracts information required for the template using the build-in MIME libaries (mml-* IIRC) which are capable of correctly parsing and if necessary decoding (e.g. quoted printable encoded letters in the subject). After extracting the information Emacs creates the appropriate capture entry, somehow adds the message file as attachment and finishes the capture process. -or- Don't dump the message in the org buffer as is, but wrap it in a #+begin_src MIME MODE where MIME-MODE is the build in mode for decoding MIME messages. You could then view the original message with C-c ' -- only task left is dealing with the subject line for the entry headline. Best, -- David -- OpenPGP... 0x99ADB83B5A4478E6 Jabber dmj...@jabber.org Email. dm...@ictsoc.de pgp5PBOeMPntH.pgp Description: PGP signature
Re: [O] Email - Org-mode: charset problem
At Mon, 27 Jun 2011 10:30:06 +0200, Karl Voit wrote: Is there somebody who managed to develop an email to Org-mode bridge without having charset problems? (No, I do not use Emacs as a MUA[4]) David Maus dm...@ictsoc.de writes: I do use an Emacs base mua (Wanderlust) and recently started to file reference notes for certain messages. Because in cases I want to keept the message (e.g. an interesting usenet post) I save the raw message to a temporary file and add it as an attachment. This works nice, although I'd really like to have an automated way of save+attache when capturing. So, what about org-capture? You could use a script that serializes the raw message to disk and -- somehow -- calls capture with a reference to this file. Emacs opens the mail file and extracts information required for the template using the build-in MIME libaries (mml-* IIRC) which are capable of correctly parsing and if necessary decoding (e.g. quoted printable encoded letters in the subject). After extracting the information Emacs creates the appropriate capture entry, somehow adds the message file as attachment and finishes the capture process. I do something like this, though I also use Emacs as an MUA (Gnus). My approach was to write a little parser that binds the various parts of a message in a pre-determined format to variables that I grab in an Org capture template, and dynamically scope those variables into a call to org-capture. I have included my code below. But to address your original question: unfortunately, I do not have a solution to the encoding problem. Despite my pre-determined format, the emails I got (student paper submissions) were often in strange encodings that I ended up fixing by hand during the capture process. However, as David points out, I believe that it would be possible to use Emacs' MIME libraries to do the dirty work. I just haven't had time (or desire, honestly) to figure out how. If you figure it out, though, I'd love to hear about it! My (quite simplistic) code is below. I hope you find it useful if you decide to go this route! Best, Richard simple working example: Gnus side ; handling paper submissions (defun capture-paper-part () Call parse-paper and capture the result using org-capture (interactive) (let ((paper (parse-paper *phil100-paper-header-re*))) (if paper (org-capture nil ap) ; ap is Org-capture key combo for template shown below (message No paper found in this part (if (not (boundp 'gnus-mime-action-alist)) (setq gnus-mime-action-alist '())) (add-to-list 'gnus-mime-action-alist '(capture paper . capture-paper-part)) (defconst *phil100-paper-header-re* ^\\(?1:.*\\)\n\\(?2:.*\\)\n\\(?3:[Pp]aper *\\(?31:[0-9]+\\).*\\)) (defun parse-paper (header-re) Return an alist representing a paper, by parsing the buffer using header-re (save-excursion (goto-char (point-min)) (if (re-search-forward header-re nil t) (list `(header . ,(or (match-string 0) )) `(author . ,(or (match-string 2) )) `(title . ,(or (match-string 1) )) `(date . ,(or (match-string 3) )) `(num . ,(or (match-string 31) )) `(body . ,(buffer-substring (match-end 0) (point-max nil))) (defun paper-property (key) ; paper is dynamically scoped in by capture-paper-part (or (cdr (assoc key paper)) )) (defun string-replace-downcase (from to in) Replace FROM with TO in string IN, and also downcase the result (with-temp-buffer (insert in) (goto-char (point-min)) (while (search-forward from nil t) (replace-match to nil t)) (downcase-region (point-min) (point-max)) (buffer-substring (point-min) (point-max simple working example: Org capture template ** %(paper-property 'author) :PROPERTIES: :EXPORT_TITLE: %(paper-property 'title) :EXPORT_AUTHOR: %(paper-property 'author) :EXPORT_DATE: Paper %(paper-property 'num) :EXPORT_FILE_NAME: %(concat (paper-property 'num) - (string-replace-downcase - (paper-property 'author)) .tex) :END: %(paper-property 'body)
