Re: [notmuch] RFC: output json from notmuch?
On Sun, 13 Dec 2009 19:21:02 -0400, David Bremner da...@tethera.net wrote: It would be nice to have more structured output from notmuch-show. I decided to investigate sexp (i.e. lisp) and json output. Thanks, David! Then I found that json parsing is provided by the library json.el shipped with emacs23, so I decided to play with json a bit. That sounds very promising. I wasn't aware this existed inside emacs already. I settled on the jansson library (http://www.digip.org/jansson/) because it seemed to have a sane api and documentation, but there are many choices. I'll follow up with the actual patch, but the idea is to replace the printfs in show_message with calls to set (key,value) pairs in a json object, and output it at the end. I'm still not sure of the need to depend on a library just to *generate* any particular format. I would expect that job to be so constrained as to be almost trivial. I won't necessarily block a patch based on that, but I think we should at least look at how easy it would be to just emit the syntax manually. So, do people think this is a reasonable idea to persue? I do, yes. The current lack of full quoting is an obnoxious bug. 1) it adds a dependency, but not a heavy one. jansson is about 300k installed on debian/i386 That might be acceptable. And we might rewrite the patch to avoid it as well. I'd like to see what that would take. 2) It might mean that people using emacs22 might have to score json.el from somewhere; I'm not sure. Currently, it sounds like notmuch is entirely unusable from emacs22, so I'm really not too concerned about one more thing that requires emacs 23. 3) There is some increase in memory use since the whole thread has to be built as a json object before being output. Right. That would be another reason to just stream the output manually. The syntax doesn't require things like knowing the size of an array before emitting the first element, right? 4) Of course jansson is doing it's own reference counting memory managment, and not using talloc. But we already have glib... It's not evil to not use talloc. So not an issue there. -Carl pgpf3vKQgsEeq.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] RFC: output json from notmuch?
On Sun, 13 Dec 2009 16:05:07 -0800, Scott Robinson sc...@quadhome.com wrote: I have a patch for a --output=(text|json) for both notmuch-show and notmuch-search. I mentioned it earlier on the list, and no one seemed to have any interest. Hi Scott, I remember you mentioning this earlier, but I didn't remember seeing the actual patch? Did I miss it? I also didn't find it again when searching and re-reading the thread. hopefully this isn't a case of my mail client's search feature failing me... ;-) Is it worth updating to HEAD and trying again? If you'd like to, please feel free. I'd be glad to evaluate different options here. -Carl pgpJXn1ZOgOKy.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] RFC: output json from notmuch?
On Sun, 13 Dec 2009 20:42:07 -0400, David Bremner brem...@unb.ca wrote: On a different topic (and in response to Carl and your earlier discussion), as a counter-weight to the desire to avoid dependencies (which I agree with), I also think we should be careful about how many embedded copies of code there are, from the point of keeping up with security problems. This is probably a bias I picked up from Debian. Embedded copies of code already packaged in Debian is something to avoid, definitely. I'm not actively against having dependencies for functionality that makes sense. For example, the current libsha1 code in notmuch has been the subject of a debate about embedded code copies here on the list already. If somebody would like to maintain that code as a Debian package, then I would be happy to depend on it that way rather than having an embedded copy inside notmuch. For something like JSON output, I really can't see how we need an external library. The job we're talking about is changing our current delimiters and then fixing our code to properly quote delimiters appearing in the content. That doesn't sound like a job that needs a library. (Meanwhile, if we were parsing JSON, then I'd be happy to depend on a library for that.) -Carl pgpUv5TfbA3jw.pgp Description: PGP signature ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] RFC: output json from notmuch?
On Mon, 14 Dec 2009 14:30:45 -0800, Carl Worth cwo...@cworth.org wrote: I'm still not sure of the need to depend on a library just to *generate* any particular format. I would expect that job to be so constrained as to be almost trivial. I won't necessarily block a patch based on that, but I think we should at least look at how easy it would be to just emit the syntax manually. OK, I'll wait for Scott's patch, and see how he is using cJSON. Just stealing a few functions might be the way to go. The cJSON code is less horrifying after running through indent. Now all we need is indent --carl :) d ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] RFC: output json from notmuch?
On Sun, 13 Dec 2009 19:21:02 -0400, David Bremner da...@tethera.net wrote: Attachments: json output of a message from carl, and the equivalent sexpr as parsed by json.el. uh, right. Second time lucky. msg.json Description: json encoded message msg.sexpr Description: msg converted to sexpr by json.el -- David Bremner Professor, UNB Computer Science brem...@unb.ca http://www.cs.unb.ca/~bremner Cross Appointment, UNB Mathematics http://www.mitacs.ca/ MITACS Atlantic Scientific Director ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] RFC: output json from notmuch?
Excerpts from David Bremner's message of Sun Dec 13 15:26:02 -0800 2009: On Sun, 13 Dec 2009 19:21:02 -0400, David Bremner da...@tethera.net wrote: Attachments: json output of a message from carl, and the equivalent sexpr as parsed by json.el. uh, right. Second time lucky. I have a patch for a --output=(text|json) for both notmuch-show and notmuch-search. I mentioned it earlier on the list, and no one seemed to have any interest. Is it worth updating to HEAD and trying again? -- Scott Robinson | http://quadhome.com/ Q: Why are my replies five sentences or less? A: http://five.sentenc.es/ ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] RFC: output json from notmuch?
Excerpts from David Bremner's message of Mon Dec 14 00:21:02 +0100 2009: So, do people think this is a reasonable idea to persue? JSON was actually the first thing I thought of when I first saw the output of notmuch-show. I think it's a much more natural fit for notmuch than say, sexps, since most languages will have libraries for JSON, which will be nicer to interfaces other than the emacs one. -- - Marten ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
Re: [notmuch] RFC: output json from notmuch?
On Sun, 13 Dec 2009 16:05:07 -0800, Scott Robinson sc...@quadhome.com wrote: I have a patch for a --output=(text|json) for both notmuch-show and notmuch-search. I mentioned it earlier on the list, and no one seemed to have any interest. Ahh, I missed that. I think it was just before I subscribed to the list, and I skipped over the subject about perl and python. Is it worth updating to HEAD and trying again? Sure, it sounds more complete than what I have. You could also wait and see what feedback this thread gathers. On a different topic (and in response to Carl and your earlier discussion), as a counter-weight to the desire to avoid dependencies (which I agree with), I also think we should be careful about how many embedded copies of code there are, from the point of keeping up with security problems. This is probably a bias I picked up from Debian. -- David Bremner Professor, UNB Computer Science brem...@unb.ca http://www.cs.unb.ca/~bremner Cross Appointment, UNB Mathematics http://www.mitacs.ca/ MITACS Atlantic Scientific Director ___ notmuch mailing list notmuch@notmuchmail.org http://notmuchmail.org/mailman/listinfo/notmuch
