Re: [Orgmode] [babel] Painless integration of source blocks with language
On Sun, Jan 16, 2011 at 9:31 AM, Eric Schulte wrote: > > #+source: wrappable > #+begin_src emacs-lisp > (setq x (+ 4 x)) > #+end_src > > #+begin_src emacs-lisp :comments noweb :noweb yes :tangle yes > (let ((x 1)) >(message "x=%s" x) ><> >(message "x=%s" x)) > #+end_src > > which tangles out the following emacs-lisp > #+begin_src emacs-lisp > ;; [[file:~/src/babel-dev/scraps.org: > :*wrap%20noweb%20references%20in%20comments][wrap-noweb-references-in-comments:2]] > (let ((x 1)) >(message "x=%s" x) >;; [[file:~/src/babel-dev/scraps.org: > :*wrap%20noweb%20references%20in%20comments][wrappable]] >(setq x (+ 4 x)) >;; wrappable ends here >(message "x=%s" x)) > ;; wrap-noweb-references-in-comments:2 ends here > #+end_src > > Cheers -- Eric > Cool! Does it yet support detangling? I tried it out and it said i wasnt in a tangle code. ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
Seth Burleigh writes: >>As for how to trace back through noweb links, the best option seem to be >>using the existing jump function to navigate from raw source to the >>embedded block, keeping track of the point's offset form the beginning >>of the block, then using `org-babel-expand-src-block' to expand the body >>of the embedded code block marking noweb references with text properties >>as they are inserted into the expanding body, and then using the point >>offset to place the point into the appropriate noweb reference. This >>process could then recurse on the embedded noweb code block until it >>ends up in a non-noweb portion of an expanded code block body. > > If you dont have any comment anchors in the text, how would you know which > noweb block you are in? If your org file and raw files are synched, then > you could, but if they become unsynched, which is the way it should be > (change raw code until done, then detangle to org), i dont exactly see how > you can do it. > Yea, I think you're right. I can't come up with any way of marking or remembering the boundaries of noweb reference sections without inserting comment wrappers around such sections. I've just pushed up a new header argument combination ":comments noweb" which will wrap all embedded noweb sections in link comments, as demonstrated in the following example. Hopefully this should be sufficient for a complete mapping from a pure code file back to the original org-mode file. #+source: wrappable #+begin_src emacs-lisp (setq x (+ 4 x)) #+end_src #+begin_src emacs-lisp :comments noweb :noweb yes :tangle yes (let ((x 1)) (message "x=%s" x) <> (message "x=%s" x)) #+end_src which tangles out the following emacs-lisp #+begin_src emacs-lisp ;; [[file:~/src/babel-dev/scraps.org::*wrap%20noweb%20references%20in%20comments][wrap-noweb-references-in-comments:2]] (let ((x 1)) (message "x=%s" x) ;; [[file:~/src/babel-dev/scraps.org::*wrap%20noweb%20references%20in%20comments][wrappable]] (setq x (+ 4 x)) ;; wrappable ends here (message "x=%s" x)) ;; wrap-noweb-references-in-comments:2 ends here #+end_src Cheers -- Eric ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
>As for how to trace back through noweb links, the best option seem to be >using the existing jump function to navigate from raw source to the >embedded block, keeping track of the point's offset form the beginning >of the block, then using `org-babel-expand-src-block' to expand the body >of the embedded code block marking noweb references with text properties >as they are inserted into the expanding body, and then using the point >offset to place the point into the appropriate noweb reference. This >process could then recurse on the embedded noweb code block until it >ends up in a non-noweb portion of an expanded code block body. If you dont have any comment anchors in the text, how would you know which noweb block you are in? If your org file and raw files are synched, then you could, but if they become unsynched, which is the way it should be (change raw code until done, then detangle to org), i dont exactly see how you can do it. The only option which allows someone to change the raw text file is to use comment anchors - or you could save marks in a separate file, but this wouldn't allow for offline editing unless you open it with a program that understands this format. With the comment method, you get the raw file advantage, plus emacs can hide these comments if one makes a minor mode to do so - a win win situation, i think, except for the aforementioned shortcomings. ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
I briefly mentioned the approach you describe below earlier in this thread [1], and while I see the appeal of this approach, I think that the second approach I described in that same message (using offsets instead of comments) is preferable -- although I am of course open to being persuaded otherwise. Best -- Eric Seth Burleigh writes: > I would just like to throw in a quick idea. > The easiest way to support noweb tangling is to get org-babel-tangle to > create nested tags and change detangle to take these into account. > > for example, i have the forex_user source block that is tangled. > > ;; [[file:~/Dropbox/.rep/clj-forex/clj-forex.org::*User][forex_user]] > my code... > ;; [[file:~/Dropbox/.rep/clj-forex/clj-forex.org::*User][embedded_block]] > my embedded code ... > ;; embedded_block ends here > ;; forex_user ends here > > corresponding to a forex_user block of > #+begin_src > my code ... > <> > #+end_src > > This would only work for noweb on separate lines and preferably only when > one block is noweb embedded into one other source block, but i believe that > this is the most common case anyways. This could be enforced in the > detangle/tangle code. > > Then, a separate minor mode could make these file: markers invisible and > create a colored overlay pattern per source block code. Saving the file > would still save the markers, you just wouldnt see them if the minor mode > was on. Footnotes: [1] http://thread.gmane.org/gmane.emacs.orgmode/35863/focus=35971 ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
I would just like to throw in a quick idea. The easiest way to support noweb tangling is to get org-babel-tangle to create nested tags and change detangle to take these into account. for example, i have the forex_user source block that is tangled. ;; [[file:~/Dropbox/.rep/clj-forex/clj-forex.org::*User][forex_user]] my code... ;; [[file:~/Dropbox/.rep/clj-forex/clj-forex.org::*User][embedded_block]] my embedded code ... ;; embedded_block ends here ;; forex_user ends here corresponding to a forex_user block of #+begin_src my code ... <> #+end_src This would only work for noweb on separate lines and preferably only when one block is noweb embedded into one other source block, but i believe that this is the most common case anyways. This could be enforced in the detangle/tangle code. Then, a separate minor mode could make these file: markers invisible and create a colored overlay pattern per source block code. Saving the file would still save the markers, you just wouldnt see them if the minor mode was on. ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
"Eric Schulte" writes: > Eric S Fraga writes: [...] >> management issue: I find it difficult (a) to remember what all my source >> code snippets are called and (b) to navigate to any given snippet. I >> would love to see a babel table of contents popup (a la the table of >> contents popup with reftex implements for latex files). Is something >> like this already available? If not, how difficult would it be to >> implement (I'm happy to try given a pointer in the right >> direction(s)...). >> > > There was some talk of merging imenu with Babel which would provide the > functionality you describe, I don't believe this ever resulting in > working code however. There are a couple of options... > > If you know the name of the code block you want to find you can use > `org-babel-goto-named-src-block' (bound to C-c C-v g) to jump to a named > code block (∃ a similar function for finding named results). This > function provides completion on the block names, the function > `org-babel-src-block-names' returns a list of all named blocks in the > current buffer, so it could be used to built up such a table. In fact > the following code block will insert a table of such names in the > current buffer. Note: you will need to pull the latest as I had to fix > a small bug in `org-babel-src-block-names'. > > #+begin_src emacs-lisp :results list > (mapcar #'list (reverse (org-babel-src-block-names))) > #+end_src This is brilliant. Thanks. Having the table of contents, together with being able to jump to any block, is half the battle won! Using it already. eric -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1 : using Org-mode version 7.4 (release_7.4.166.gf7a7) ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
[Orgmode] [babel] Painless integration of source blocks with language
woops, what i actually meant is that noweb doesnt work. I was thinking about it, and it might be possible to (a) automatically create overlays around begin_src blocks using auto-overlays (b) have a custom syntax parser that parses noweb syntax in those blocks. Sort of like what is done for syntax highlighting. Im gradually converting my code to literate programming and am just on the verge of splitting up my one clojure ns to one-block org file into multiple blocks per ns, so i will soon need the noweb functionality. When i get it ill make sure to post and update! ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
Eric S Fraga writes: > "Eric Schulte" writes: > > [...] > >> A crude version of the above is already possible using the >> `org-babel-detangle' function. For example, follow the instructions in >> the attached org-mode file (which uses elisp rather than clojure code >> blocks simply for wider portability to non-clojure users). > > Detangling, as currently implemented, doesn't do the job for me as it > doesn't understand noweb. My current mode of operation with org and > babel is to have various snippets of code throughout a file and then > combine these in different ways using noweb syntax which I then tangle > to create different source files (each bringing together different > pieces in different configurations). > I agree, this is an area ripe for improvement. > > In any case, and please excuse me for hijacking this thread a little, > the increasing use of babel (a good thing!) especially with noweb syntax > and tangling (as this thread is about) is bringing up a document > management issue: I find it difficult (a) to remember what all my source > code snippets are called and (b) to navigate to any given snippet. I > would love to see a babel table of contents popup (a la the table of > contents popup with reftex implements for latex files). Is something > like this already available? If not, how difficult would it be to > implement (I'm happy to try given a pointer in the right > direction(s)...). > There was some talk of merging imenu with Babel which would provide the functionality you describe, I don't believe this ever resulting in working code however. There are a couple of options... If you know the name of the code block you want to find you can use `org-babel-goto-named-src-block' (bound to C-c C-v g) to jump to a named code block (∃ a similar function for finding named results). This function provides completion on the block names, the function `org-babel-src-block-names' returns a list of all named blocks in the current buffer, so it could be used to built up such a table. In fact the following code block will insert a table of such names in the current buffer. Note: you will need to pull the latest as I had to fix a small bug in `org-babel-src-block-names'. #+begin_src emacs-lisp :results list (mapcar #'list (reverse (org-babel-src-block-names))) #+end_src Hope that helps, Best -- Eric > > Thanks, > eric ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
"Eric Schulte" writes: [...] > A crude version of the above is already possible using the > `org-babel-detangle' function. For example, follow the instructions in > the attached org-mode file (which uses elisp rather than clojure code > blocks simply for wider portability to non-clojure users). Detangling, as currently implemented, doesn't do the job for me as it doesn't understand noweb. My current mode of operation with org and babel is to have various snippets of code throughout a file and then combine these in different ways using noweb syntax which I then tangle to create different source files (each bringing together different pieces in different configurations). In any case, and please excuse me for hijacking this thread a little, the increasing use of babel (a good thing!) especially with noweb syntax and tangling (as this thread is about) is bringing up a document management issue: I find it difficult (a) to remember what all my source code snippets are called and (b) to navigate to any given snippet. I would love to see a babel table of contents popup (a la the table of contents popup with reftex implements for latex files). Is something like this already available? If not, how difficult would it be to implement (I'm happy to try given a pointer in the right direction(s)...). Thanks, eric -- : Eric S Fraga (GnuPG: 0xC89193D8FFFCF67D) in Emacs 23.2.1 : using Org-mode version 7.4 (release_7.4.153.ga0b80) ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
Re: [Orgmode] [babel] Painless integration of source blocks with language
Hi Seth, Thanks for the thoughtful comments. I especially like the method of literate programming described in your second proposal. Over the last months I have switched from working mainly in code blocks to working mainly in pure source files due to issues along the lines of those mentioned in your first proposal. It seems to me that working in two frames as you suggest -- e.g. writing code in the pure code buffer, documentation in the Org-mode buffer, and maintaining a constant mapping between the two -- would resolve the issues mentioned in your first, and should be sufficient. With some elisp code it should be possible to support no-web in such a scheme allowing the code buffer to be automatically re-organized based on changes to the .org buffer. A crude version of the above is already possible using the `org-babel-detangle' function. For example, follow the instructions in the attached org-mode file (which uses elisp rather than clojure code blocks simply for wider portability to non-clojure users). Ultimately I think that more code support (possibly implemented using a minor mode) allowing things like easy navigation, interactive tangling/arrangement, noweb support, and (possibly) removing the need for comment markers would be the preferred solution. Cheers -- Eric #+Title: Detangle Example #+Babel: :comments link :tangle detangle.el This file demonstrates detangling of source files with Org-mode. 1. Evaluate this elisp code block to customize the behavior of link creation on your system -- in the future this could be folded into ob-tangle. #+begin_src emacs-lisp :results silent :tangle no (setq org-link-to-org-use-id nil) #+end_src 2. Tangle out this file by calling `[[elisp:(org-babel-tangle)]]' bound to =C-c C-v t=, notice the buffer-wide =:comments= and =:tangle= header arguments at the top of the file. 3. Open up the [[file:detangle.el][detangle.el]] file to find the tangled source code. Notice the comments which are used to associate parts of the source file with this org file. 4. Call the `org-babel-tangle-jump-to-org' function from one of the =message= lines in the elisp file to jump back to the related portion of this org file. 5. Navigate back to [[file:detangle.el][detangle.el]] and edit part of the elisp code (e.g. change the text of one of the messages), then call the `org-babel-detangle' function from inside of the elisp code buffer. Notice that your edits have now been propagated back to the original Org-mode buffer. * first A first section. #+source: a-named-block #+begin_src emacs-lisp (message "this is the contents of the first code block.") #+end_src * second A second section. #+begin_src emacs-lisp (message "this block has no name -- it is the second code block") #+end_src Seth Burleigh writes: > Preface: I hope attachments show up, i dont know if they are allowed > The attachments did make it through. > > Im currently interested in using babel for a medium size clojure project. I > think the below propositions would greatly benefit babel in accomplishing > literate programming. > > First part of the proposal to make this painless: > > In a literate document, you might very well have small chunks of code for > one ns scattered around and then finally combined using noweb into one file > and tangle the output. Currently, keys like > compile-file (C-c C-k), goto function definition (M-.), do not work at all > in the source blocks. This is unfortunate, and makes life very painful. So, > first we need to get these keys to work. > Heres how it can possibly be done. > > Lets say we want to compile the code in block A. Block B uses noweb syntax > to import block A, and then tangles to file src/B.clj. So we need to search > for the presence of <> in a tangled code chunk , tangle the chunk to a > file, go to the file buffer and position the cursor at the correct relative > position, and then invoke the appropriate function (in this case, > slime-compile-and-load-file). > > Ive attached two files which do this, the first one is an example org file > in which you can use the compile keystrokes on (after opening a slime server > for clojure). However, it doesnt position the cursor in the tangled file > correctly, but this is not needed for compile/load file. Its not a very good > implementation, but it works for this case. Also, it doesn't work when the > indirect buffer created by C-c ' is open. The advantage of this method is > that it can easily be generalized to any language, since the compiler gets > what it expects - a file of code to operate on. > > Second part of proposal: > > Literate documents are good for documentation, but if you're the author, you > dont need the documentation and it will certainly get in the way of you > writing code to keep having to type C-c ' to open various chunks of code. > Ideally, we would like to have our file of code (that is, the tangled file > output for one ns in clojure)
[Orgmode] [babel] Painless integration of source blocks with language
Preface: I hope attachments show up, i dont know if they are allowed Im currently interested in using babel for a medium size clojure project. I think the below propositions would greatly benefit babel in accomplishing literate programming. First part of the proposal to make this painless: In a literate document, you might very well have small chunks of code for one ns scattered around and then finally combined using noweb into one file and tangle the output. Currently, keys like compile-file (C-c C-k), goto function definition (M-.), do not work at all in the source blocks. This is unfortunate, and makes life very painful. So, first we need to get these keys to work. Heres how it can possibly be done. Lets say we want to compile the code in block A. Block B uses noweb syntax to import block A, and then tangles to file src/B.clj. So we need to search for the presence of <> in a tangled code chunk , tangle the chunk to a file, go to the file buffer and position the cursor at the correct relative position, and then invoke the appropriate function (in this case, slime-compile-and-load-file). Ive attached two files which do this, the first one is an example org file in which you can use the compile keystrokes on (after opening a slime server for clojure). However, it doesnt position the cursor in the tangled file correctly, but this is not needed for compile/load file. Its not a very good implementation, but it works for this case. Also, it doesn't work when the indirect buffer created by C-c ' is open. The advantage of this method is that it can easily be generalized to any language, since the compiler gets what it expects - a file of code to operate on. Second part of proposal: Literate documents are good for documentation, but if you're the author, you dont need the documentation and it will certainly get in the way of you writing code to keep having to type C-c ' to open various chunks of code. Ideally, we would like to have our file of code (that is, the tangled file output for one ns in clojure) as the top buffer, and our documentation at the bottom. Changes to the code will automatically reflect itself in the documentation (after a save). In order to accomplish this, there has to be a method to map from tangled file line number, to the correct chunk name and line number in the .org file. This might be accomplished by tangling code like this: ;;#chunk-name ..code ;;# I think this would be much more natural than editing each chunk separately. Of course, the ;;# might become an annoyance, but im sure there might be a better way. With this mapping , we can develop the code until we get it right. Then we can hit a keystroke and have our documentation jump to the correct line number in the .org file, and then do all the documentation for that. With the line mapping ability also comes the ability to map errors in the line number to numbers in the org file. Although, this probably wouldn't be necessary, since we would be looking at the combined chunk file. Any thoughts/ideas how to implement this? test.org Description: Binary data seth-obj-clj.el Description: Binary data ___ Emacs-orgmode mailing list Please use `Reply All' to send replies to the list. Emacs-orgmode@gnu.org http://lists.gnu.org/mailman/listinfo/emacs-orgmode
