Re: [O] Org babel does not work properly with included files
I would say we can close this bug report as INVALID. AFAICT, the actual problem was happening simply because of the incorrect treatment of missing links during export, which is now fixed. When that got fixed, my issue went away. best, r
Re: [O] Org babel does not work properly with included files
Robert Goldman rpgold...@sift.info writes: Looking over this some more, I see that the challenge is to: 1. read the file parameters (whatever they are) from the original file (hence opening the file from the link) and 2. read the header parameters from the export buffer, since the header may not actually be contained in the original file. The above is a good summary. Babel ensures that code blocks will be evaluated in the original buffer, so that they can e.g., reference a code block outside of the exported subtree when only exporting a subtree. This seems like a substantial reorganization from the original, which attempts to do both tasks in the original file (and fails for included files). I'm not sure that the current behavior is a bug. Is it reasonable to place code block parameters into an included file? These parameters would not be successfully found during interactive evaluation, and could only plausibly be used during export as you anticipated. Also, this would seem to break Babel's current guarantee that code blocks will be evaluated in the original Org-mode file. If this were to be implemented, I think one option would be to perform the following on export 1. make a copy of the original buffer 2. call `org-export-handle-include-files' in this copy 3. resolve all parameters in this copied buffer The above shouldn't be difficult to implement, and creating the copy shouldn't add too much computational time to the export process. My concern is that expansion of included files may be more likely to cause confusing behavior (e.g., by change the evaluation environment between interactive evaluation and export) than it would to be used constructively. Maybe I have missed your initial use case. Thanks -- Eric p.s. Sorry about the delayed reply I've had very little mailing list connectivity lately best, r -- Eric Schulte http://cs.unm.edu/~eschulte/
Re: [O] Org babel does not work properly with included files
On 5/5/11 May 5 -11:56 AM, Eric Schulte wrote: Robert Goldman rpgold...@sift.info writes: Looking over this some more, I see that the challenge is to: 1. read the file parameters (whatever they are) from the original file (hence opening the file from the link) and 2. read the header parameters from the export buffer, since the header may not actually be contained in the original file. The above is a good summary. Babel ensures that code blocks will be evaluated in the original buffer, so that they can e.g., reference a code block outside of the exported subtree when only exporting a subtree. This seems like a substantial reorganization from the original, which attempts to do both tasks in the original file (and fails for included files). I'm not sure that the current behavior is a bug. Is it reasonable to place code block parameters into an included file? These parameters would not be successfully found during interactive evaluation, and could only plausibly be used during export as you anticipated. Aren't the code block parameters supposed to appear /with/ the code block? So here's the use case: I have a file chapter.org. This contains a full draft of a chapter of my manual. I finish it and circulate it for comments, then get it ready for inclusion. Now I have manual.org and I want to include the main body of chapter.org (typically there's some front matter I leave off). When I put the #include in manual.org, the source code snippets in chapter.org, which used to work, no longer do. This doesn't seem like /such/ a crazy use case that it shouldn't work, does it? [btw, I am not entirely sure I know what header parameters are --- are these the parameters that come from the #+begin_src line? If so, shouldn't they definitely be read from the #+begin_src line? They can't very well be read from manual.org, which doesn't contain the #+begin_src line.] Sorry if I wasn't clear in my original message. Best, r
Re: [O] Org babel does not work properly with included files
I'm not sure that the current behavior is a bug. Is it reasonable to place code block parameters into an included file? These parameters would not be successfully found during interactive evaluation, and could only plausibly be used during export as you anticipated. Aren't the code block parameters supposed to appear /with/ the code block? So here's the use case: I have a file chapter.org. This contains a full draft of a chapter of my manual. I finish it and circulate it for comments, then get it ready for inclusion. Now I have manual.org and I want to include the main body of chapter.org (typically there's some front matter I leave off). When I put the #include in manual.org, the source code snippets in chapter.org, which used to work, no longer do. This doesn't seem like /such/ a crazy use case that it shouldn't work, does it? Oh, my apologies, apparently in scanning this email thread I mis-understood your use case. I've just tried to re-create the situation you've described above (including a file which contains code blocks). I was unable to reproduce your problem locally (the results of exporting both to html and tex are included). Could you modify the attached example sufficiently to demonstrate the problem you're experiencing? example.tar.bz2 Description: Binary data [btw, I am not entirely sure I know what header parameters are --- see http://orgmode.org/manual/Header-arguments.html are these the parameters that come from the #+begin_src line? If so, shouldn't they definitely be read from the #+begin_src line? They can't very well be read from manual.org, which doesn't contain the #+begin_src line.] These could appear with the code block, or as a property in an enclosing heading. Another example of a case where the original file is needed rather than the exported file would be the case of exporting a subtree which includes a block which references a variable defined elsewhere in the file. Sorry if I wasn't clear in my original message. No problem, I believe I misread this email thread. Thanks -- Eric Best, r -- Eric Schulte http://cs.unm.edu/~eschulte/
Re: [O] Org babel does not work properly with included files
On 5/5/11 May 5 -4:27 PM, Eric Schulte wrote: I'm not sure that the current behavior is a bug. Is it reasonable to place code block parameters into an included file? These parameters would not be successfully found during interactive evaluation, and could only plausibly be used during export as you anticipated. Aren't the code block parameters supposed to appear /with/ the code block? So here's the use case: I have a file chapter.org. This contains a full draft of a chapter of my manual. I finish it and circulate it for comments, then get it ready for inclusion. Now I have manual.org and I want to include the main body of chapter.org (typically there's some front matter I leave off). When I put the #include in manual.org, the source code snippets in chapter.org, which used to work, no longer do. This doesn't seem like /such/ a crazy use case that it shouldn't work, does it? Oh, my apologies, apparently in scanning this email thread I mis-understood your use case. I've just tried to re-create the situation you've described above (including a file which contains code blocks). I was unable to reproduce your problem locally (the results of exporting both to html and tex are included). Could you modify the attached example sufficiently to demonstrate the problem you're experiencing? I will work on it --- I have a somewhat complicated perl script that is pulling stuff out for me and exporting results as source. Getting it to work in a small example is tricky! [btw, I am not entirely sure I know what header parameters are --- see http://orgmode.org/manual/Header-arguments.html That's what I thought --- I was just confused because the comment in the code flips from using the term argument to parameter, and I considered the possibility that there were two different mechanisms. Thanks, r
[O] Org babel does not work properly with included files
Here's the problem: when org-babel goes to look for parameters when executing a source block, it looks for them in the parent source file, and not in the parent source file with the included materials. Here is the function that goes awry: (defmacro org-babel-exp-in-export-file (lang rest body) (declare (indent 1)) `(let* ((lang-headers (intern (concat org-babel-default-header-args: ,lang))) (heading (nth 4 (ignore-errors (org-heading-components (link (when org-current-export-file (org-make-link-string (if heading (concat org-current-export-file :: heading) org-current-export-file (export-buffer (current-buffer)) results) (when link ;; resolve parameters in the original file so that ;; headline and file-wide parameters are included, attempt ;; to go to the same heading in the original file (set-buffer (get-file-buffer org-current-export-file)) (save-restriction (condition-case nil ** (org-open-link-from-string link) (error (when heading (goto-char (point-min)) (re-search-forward (regexp-quote heading) nil t (setq results ,@body)) (set-buffer export-buffer) results))) The line that blows up, AFAICT is the one marked with an asterisk. It goes to look in the file *without* incorporated materials, instead of the file *with* incorporated materials. I believe that what needs to happen is that the link must be searched for in the export-buffer, rather than in org-current-export-file. Unfortunately, I don't understand the code well enough to figure out how to make this modification. I believe that the first trick is to NOT concatenate org-current-export-file onto the link string. The second would be to change org-open-link-from-string to something that looks for the link match in the current buffer. To see the bug, it should be sufficient to install the attached to .org files into a temporary directory and then try to export foo.org to latex. You may need to tweak your configuration so that evaluation is enabled for sh scripts (require 'ob-sh) The error messages one gets are deeply cryptic. To debug this I modified org-link-search as follows. I added a DEBUG call when the link is unfound. This is, of course, not generally good practice, but you will want this debug call in place when you test the export per my instructions above. You will get a debug window with a backtrace that should make things relatively clear. Here is the modified org-link-search --- you can just evaluate the definition before testing the export. To see what I've changed, just search for debug... Cheers, r (defun org-link-search (s optional type avoid-pos) Search for a link search option. If S is surrounded by forward slashes, it is interpreted as a regular expression. In org-mode files, this will create an `org-occur' sparse tree. In ordinary files, `occur' will be used to list matches. If the current buffer is in `dired-mode', grep will be used to search in all files. If AVOID-POS is given, ignore matches near that position. (let ((case-fold-search t) (s0 (mapconcat 'identity (org-split-string s [ \t\r\n]+) )) (markers (concat \\(?: (mapconcat (lambda (x) (regexp-quote (car x))) (append '(() ( ) (\t) (\n)) org-emphasis-alist) \\|) \\))) (pos (point)) (pre nil) (post nil) words re0 re1 re2 re3 re4_ re4 re5 re2a re2a_ reall) (cond ;; First check if there are any special search functions ((run-hook-with-args-until-success 'org-execute-file-search-functions s)) ;; Now try the builtin stuff ((and (equal (string-to-char s0) ?#) ( (length s0) 1) (save-excursion (goto-char (point-min)) (and (re-search-forward (concat ^[ \t]*:CUSTOM_ID:[ \t]+ (regexp-quote (substring s0 1)) [ \t]*$) nil t) (setq type 'dedicated pos (match-beginning 0 ;; There is an exact target for this (goto-char pos) (org-back-to-heading t))) ((save-excursion (goto-char (point-min)) (and (re-search-forward (concat (regexp-quote s0) ) nil t) (setq type 'dedicated pos (match-beginning 0 ;; There is an exact target for this (goto-char pos)) ((and (string-match ^(\\(.*\\))$ s0) (save-excursion (goto-char (point-min)) (and (re-search-forward (concat [^[] (regexp-quote (format org-coderef-label-format (match-string 1 s0
Re: [O] Org babel does not work properly with included files
Robert Goldman rpgold...@sift.info wrote: foo.org: * Purpose This document is intended to demonstrate that src buffer evaluation in subsidiary, included files, does not work. * Demo #+include ~/src/org-test/bar.org bar.org: * Here's the demo #+begin_src sh :exports results :results output echo foo! #+end_src As a workaround, try making the headings identical, i.e. bar.org looks like this: --8---cut here---start-8--- * Demo #+begin_src sh :exports results :results output echo foo! #+end_src --8---cut here---end---8--- Nick
Re: [O] Org babel does not work properly with included files
Sorry, I think I have created a red herring here by leaving that code block in both files. To see what the problem really is, consider the case where the source code block appears ONLY in the included file. (I tested the source block in the master file to make sure it worked before I copied it into the included file and forgot to remove it from the master file.) Best, R -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Nick Dokos nicholas.do...@hp.com wrote: Robert Goldman rpgold...@sift.info wrote: foo.org: * Purpose This document is intended to demonstrate that src buffer evaluation in subsidiary, included files, does not work. * Demo #+include ~/src/org-test/bar.org bar.org: * Here's the demo #+begin_src sh :exports results :results output echo foo! #+end_src As a workaround, try making the headings identical, i.e. bar.org looks like this: --8---cut here---start-8--- * Demo #+begin_src sh :exports results :results output echo foo! #+end_src --8---cut here---end---8--- Nick
Re: [O] Org babel does not work properly with included files
Also note that if you have an included file with MULTIPLE code blocks, this approach won't generalize Best, -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Nick Dokos nicholas.do...@hp.com wrote: Robert Goldman rpgold...@sift.info wrote: foo.org: * Purpose This document is intended to demonstrate that src buffer evaluation in subsidiary, included files, does not work. * Demo #+include ~/src/org-test/bar.org bar.org: * Here's the demo #+begin_src sh :exports results :results output echo foo! #+end_src As a workaround, try making the headings identical, i.e. bar.org looks like this: --8---cut here---start-8--- * Demo #+begin_src sh :exports results :results output echo foo! #+end_src --8---cut here---end---8--- Nick
Re: [O] Org babel does not work properly with included files
Looking over this some more, I see that the challenge is to: 1. read the file parameters (whatever they are) from the original file (hence opening the file from the link) and 2. read the header parameters from the export buffer, since the header may not actually be contained in the original file. This seems like a substantial reorganization from the original, which attempts to do both tasks in the original file (and fails for included files). best, r
Re: [O] Org babel does not work properly with included files
Robert P. Goldman rpgold...@sift.info wrote: Sorry, I think I have created a red herring here by leaving that code blo= ck in both files. To see what the problem really is, consider the case wh= ere the source code block appears ONLY in the included file. (I tested the source block in the master file to make sure it worked befo= re I copied it into the included file and forgot to remove it from the ma= ster file.) ?? There was no code block in foo.org: just the include. Only bar.org had the code block. As for multiple code blocks in the included file (your next message), yeah, probably. But my suggestion was just a (possible) workaround: it's definitely not a fix. Nick
Re: [O] Org babel does not work properly with included files
Yes. That's right, the single source block was a simplification for the example. It was abstracted from a more complicated case with a dozen or so source blocks in the subordinate filter. But thanks for the suggestion. Hope we can find a fix! Best R -- Sent from my Android phone with K-9 Mail. Please excuse my brevity. Nick Dokos nicholas.do...@hp.com wrote: Robert P. Goldman rpgold...@sift.info wrote: Sorry, I think I have created a red herring here by leaving that code blo= ck in both files. To see what the problem really is, consider the case wh= ere the source code block appears ONLY in the included file. (I tested the source block in the master file to make sure it worked befo= re I copied it into the included file and forgot to remove it from the ma= ster file.) ?? There was no code block in foo.org: just the include. Only bar.org had the code block. As for multiple code blocks in the included file (your next message), yeah, probably. But my suggestion was just a (possible) workaround: it's definitely not a fix. Nick
