Re: [O] [PATCH] inline src block results can be removed
On 2014-11-24 12:12, Daniele Pizzolli wrote:
On 2014-11-24 11:18, Andreas Leha wrote:
Hi Daniele,
I think your wishlist is somewhere further down the road. I usually
implement some of your points in the src_language. I see that it
would
be nice if org supported these use cases, but I would see them as part
of the LOB or maybe in some package in contrib rather than in core
org/babel.
For that, I'd argue removable inline results that can be exported
without special formatting would be all that is needed to support your
wishlist.
Hi Andreas,
sure, I think I will start getting the string I needed from
src_language. But in some language is really hard to get facilities
for text manipulation while the output as table generally supported.
And I also think that the fine grained control of the layout of the
inline sentence pertains to the org layer.
Hi Andreas,
Yesterday I forgot to mention another reason because I think that the
formatting of the results pertains to the org layer and not to the
original language.
It's about proper boundaries and untrusted or limited trust of the
babel code. By default I wish to run untrusted code and display in a
safely manner the result. This is already possible by remote code
evaluation (but I not really checked if the output is always safely
escaped before the insertion in the org buffer).
Using the source language for outputting org-code is a viable
workaround for my needs but in the long run I will prefer to not
include some potential harmful generated org code.
I double checked about the output escaping right now. I guessed that
the output in the raw mode was safely escaped. But it really seems
the opposite: [[info:org#Results]]. However it seems that the RESULTS
code are not subsequently evaluated even with:
#+BEGIN_SRC elisp
(org-babel-do-load-languages
'org-babel-load-languages
'((sh . t)))
(setq org-export-babel-evaluate t)
#+END_SRC
Example:
#+NAME: is-this-safe-or-not
#+BEGIN_SRC sh :result output raw
printf 'Is nested code generation always safe? src_sh{rm -Rf /tmp/123
echo true; echo false}'
#+END_SRC
# TODO: the pipe causes a bug...
#+RESULTS: is-this-safe-or-not
: Is nested code generation always safe? src_sh{rm -Rf /tmp/123 echo
true; echo false}
So now I am more confused than before... Has anybody a proof or a
counterexample that is (im)possible to run safely untrusted code on
(untrusted) remote host and still have the local buffer trusted?
By the way I really wanted to write:
#+NAME: problem-with-pipes...
#+BEGIN_SRC sh :result output raw
printf 'Is nested code generation always safe? src_sh{rm -Rf /tmp/123
echo true || echo false}'
#+END_SRC
#+RESULTS: problem-with-pipes...
| Is nested code generation always safe? src_sh{rm -Rf /tmp/123 echo
true | | echo false} |
I guess that something is wrong here:
Org-mode version 8.3beta (release_8.3beta-485-gf70439 @
/home/user/.emacs.d/el-get/org-mode/lisp/)
Maybe we need a new result format for safe output or am I missing
something?
Thanks in advance,
Daniele
Re: [O] [PATCH] inline src block results can be removed
Hi Daniele, Daniele Pizzolli d...@toel.it writes: On 2014-11-24 12:12, Daniele Pizzolli wrote: On 2014-11-24 11:18, Andreas Leha wrote: Hi Daniele, I think your wishlist is somewhere further down the road. I usually implement some of your points in the src_language. I see that it would be nice if org supported these use cases, but I would see them as part of the LOB or maybe in some package in contrib rather than in core org/babel. For that, I'd argue removable inline results that can be exported without special formatting would be all that is needed to support your wishlist. Hi Andreas, sure, I think I will start getting the string I needed from src_language. But in some language is really hard to get facilities for text manipulation while the output as table generally supported. And I also think that the fine grained control of the layout of the inline sentence pertains to the org layer. Hi Andreas, Yesterday I forgot to mention another reason because I think that the formatting of the results pertains to the org layer and not to the original language. It's about proper boundaries and untrusted or limited trust of the babel code. By default I wish to run untrusted code and display in a safely manner the result. This is already possible by remote code evaluation (but I not really checked if the output is always safely escaped before the insertion in the org buffer). I agree in principle. But there are limits. Org cannot be expected to know how to format any object I am creating to my liking. So, in my opinion there are 'standard objects' (like tables) that can be passed to org and org can format them appropriately. For more special and targeted things I have to provide the formatting myself. I also agree that it is much cleaner to separate the formatting from the generation. And whenever that is possible that's the way to go. For instance, I can pass my object as a table and have some custom code to do the formatting. That custom code does not have to be the same language that generated the object, but can be any language with babel support. I can then even make this very transparent by using the :post header argument. Simple (non-inline) example with a function to add hlines to tables (the function is old and ugly, though...) --8---cut here---start-8--- * Example Add hlines to tables at arbitrary rows ignoring already existing hlines. #+name: add-hlines #+begin_src emacs-lisp :var x=(quote ((parameter value) (amount 1) (margin 12))) :var below=1 (labels ((p-cumsum (boolean-list) (if (not boolean-list) 0 (if (not (sequencep boolean-list)) 1 (if (equal 1 (length boolean-list)) (list (if (car boolean-list) 1 0)) (let ((firstsum (p-cumsum (car boolean-list))) (lastsum (p-cumsum (cdr boolean-list (cons firstsum (mapcar '(lambda (x) (+ firstsum x)) lastsum (let* ((h (mapcar (lambda(a) (not (equal a 'hline))) x)) (hs (p-cumsum h)) (below-list (if (listp below) below (list below))) (below-rows (reverse (sort below-list ' (message %S x) (message %S h) (message %S hs) (dolist (below-row below-rows) (let ((after-row (apply #'+ (mapcar '(lambda (x) (if (= x below-row) 1 0)) hs (message %S after-row) (setq x (append (cl-subseq x 0 after-row) (list 'hline) (cl-subseq x after-row) x)) #+end_src #+begin_src R :results table replace :post add-hlines[:results table](below='(2 3),x=*this*) data.frame(a=1:10, b=11:20) #+end_src #+results: | 1 | 11 | | 2 | 12 | |+| | 3 | 13 | |+| | 4 | 14 | | 5 | 15 | | 6 | 16 | | 7 | 17 | | 8 | 18 | | 9 | 19 | | 10 | 20 | --8---cut here---end---8--- Using the source language for outputting org-code is a viable workaround for my needs but in the long run I will prefer to not include some potential harmful generated org code. I double checked about the output escaping right now. I guessed that the output in the raw mode was safely escaped. But it really seems the opposite: [[info:org#Results]]. However it seems that the RESULTS code are not subsequently evaluated even with: I do not think that style of 'recursive' evaluation is possible. But there is the :post header argument, see above. #+BEGIN_SRC elisp (org-babel-do-load-languages 'org-babel-load-languages '((sh . t))) (setq org-export-babel-evaluate t) #+END_SRC Example: #+NAME: is-this-safe-or-not #+BEGIN_SRC sh :result output raw printf 'Is nested code generation always safe?
Re: [O] [PATCH] inline src block results can be removed
On 2014-11-17 00:23, Nicolas Goaziou wrote:
Charles C. Berry writes:
For now, I'd be willing to make patches that will allow removal of the
inline src block results that do *not* involve these header args:
[]
IMO, we're too much focused on the implementation details. We ought to
agree on what should be done first. For example, considering
`org-babel-insert-result' and its RESULT-PARAMS argument, I think the
following makes sense:
| Param | Example output|
|-+---|
| default | {{{results(42)}}} |
| file| {{{results(file:something.pdf)}}} |
| list| |
| raw | 42|
| drawer | |
| org | {{{results(src_org{...})}}} |
| html| {{{results(@@html:...@@)}}} |
| latex | {{{results(@@latex:...@@)}}} |
| code| {{{results(src_xxx{...})}}} |
Basically, it should be possible to remove any kind of result using
results macro, with the exception of raw. list and drawer can
be ignored since there is no inline equivalent.
[]
There's no rush, however. Non-removable results from inline source
blocks have been there for a long time.
Hello all,
I jump into this conversation because in the next months I will try to
write a supplementary information annex using org-mode with Gnu R, so
my idea on what should be done follows...
I guess I will start my project using those patches. Thanks!
A brief review on how how results are presented (or usually
anticipated) inline led me to the following wish list:
* provide some facilities to keep track of the 'unit of measurement'
Few numbers are pure numbers. There is a little value added to the
correctness of the work as a whole if I get the correct number from
babel but I have hard coded the wrong unit of measure in the text.
Please follow the examples.
# Load used languages
#+BEGIN_SRC emacs-lisp
(org-babel-do-load-languages
'org-babel-load-languages
'((emacs-lisp . nil)
(octave . t)
(R . t)))
#+END_SRC
Ideally, I think that reasonable solution would take table like the
following:
#+TBLNAME:speed-1
| speed | speed_unit |
|---+|
|12 | km/s |
and output by default or through an easy to call syntax:
#+BEGIN_EXAMPLE
12 km/s
#+END_EXAMPLE
I am not aware of any facilities provided by languages like R or
octave to keep track of the unit of measure, If you have some please
share.
A more space wise representation, that add another layer of
indirection would be to have a look-up table like the following:
#+TBLNAME:speed-1-units-lookup
| name | unit |
|+--|
| speed | km/s |
| length | km |
| time | s|
But I am not sure that it's worth it, at least for those that are
working with small data sets.
* support for inline rendering of tables
The most complex inline output I think I will need at some point
follows:
#+TBLNAME:data-set-1
#+NAME:data-set-1
| position | item1 | author1 | item2 | author2 | correlation |
|--+---+-+---+-+-|
|1 | i1| a1 | j1| b1 | c1 |
|2 | i2| a2 | j2| b2 | c2 |
|3 | i3| a3 | j3| b3 | c3 |
#+NAME: block-1
#+BEGIN_SRC R :session :colnames yes :exports none :var
data_set_1=data-set-1
# Just to show that we use R
data_set_2 - data_set_1
#+END_SRC
#+RESULTS: block-1
| position | item1 | author1 | item2 | author2 | correlation |
|--+---+-+---+-+-|
|1 | i1| a1 | j1| b1 | c1 |
|2 | i2| a2 | j2| b2 | c2 |
|3 | i3| a3 | j3| b3 | c3 |
And I really like to be easily rendered as an inline string through
some formatting specs e.g.:
#+BEGIN_EXAMPLE
#+PROPERTY: header-args: R :fmt '(%(position)i) “%(item1)s” by
/%(author1)s/ correlate to “%(item2)s” by /%(author2)s/ with a
correlation coefficient of %(correlation).2f'
#+PROPERTY: header-args: R :rowsep (', ', ' and ')
The preliminary results show that: src_R[:session :results
replace]{data_set_2}.
#+END_EXAMPLE
Will be rendered as:
#+BEGIN_org
The preliminary results show that:
(1) “i1” by /a1/ correlate to “j1” by /b1/ with a correlation
coefficient of c1,
(2) “i2” by /a2/ correlate to “j2” by /b2/ with a correlation
coefficient of c2 and
(3) “i3” by /a3/ correlate to “j3” by /b3/ with a correlation
coefficient of c3.
#+END_org
For languages that does not support tables with headers we can stick
with the positional formatting:
#+NAME: block-2
#+BEGIN_SRC octave :session :colnames yes :exports none :var
data_set_1=data-set-1
ans = data_set_1;
#+END_SRC
# TODO: figure out why this does not work
#+RESULTS: block-2
: iii123aaa123jjj123bbb123ccc123
#+BEGIN_EXAMPLE
#+PROPERTY: header-args: octave :fmt '(%i)
Re: [O] [PATCH] inline src block results can be removed
Hi Daniele,
I think your wishlist is somewhere further down the road. I usually
implement some of your points in the src_language. I see that it would
be nice if org supported these use cases, but I would see them as part
of the LOB or maybe in some package in contrib rather than in core
org/babel.
For that, I'd argue removable inline results that can be exported
without special formatting would be all that is needed to support your
wishlist.
A few inline comments below.
Daniele Pizzolli d...@toel.it writes:
On 2014-11-17 00:23, Nicolas Goaziou wrote:
Charles C. Berry writes:
For now, I'd be willing to make patches that will allow removal of the
inline src block results that do *not* involve these header args:
[]
IMO, we're too much focused on the implementation details. We ought to
agree on what should be done first. For example, considering
`org-babel-insert-result' and its RESULT-PARAMS argument, I think the
following makes sense:
| Param | Example output|
|-+---|
| default | {{{results(42)}}} |
| file| {{{results(file:something.pdf)}}} |
| list| |
| raw | 42|
| drawer | |
| org | {{{results(src_org{...})}}} |
| html| {{{results(@@html:...@@)}}} |
| latex | {{{results(@@latex:...@@)}}} |
| code| {{{results(src_xxx{...})}}} |
Basically, it should be possible to remove any kind of result using
results macro, with the exception of raw. list and drawer can
be ignored since there is no inline equivalent.
[]
There's no rush, however. Non-removable results from inline source
blocks have been there for a long time.
Hello all,
I jump into this conversation because in the next months I will try to
write a supplementary information annex using org-mode with Gnu R, so
my idea on what should be done follows...
I guess I will start my project using those patches. Thanks!
A brief review on how how results are presented (or usually
anticipated) inline led me to the following wish list:
* provide some facilities to keep track of the 'unit of measurement'
Few numbers are pure numbers. There is a little value added to the
correctness of the work as a whole if I get the correct number from
babel but I have hard coded the wrong unit of measure in the text.
Please follow the examples.
# Load used languages
#+BEGIN_SRC emacs-lisp
(org-babel-do-load-languages
'org-babel-load-languages
'((emacs-lisp . nil)
(octave . t)
(R . t)))
#+END_SRC
Ideally, I think that reasonable solution would take table like the
following:
#+TBLNAME:speed-1
| speed | speed_unit |
|---+|
|12 | km/s |
and output by default or through an easy to call syntax:
#+BEGIN_EXAMPLE
12 km/s
#+END_EXAMPLE
I am not aware of any facilities provided by languages like R or
octave to keep track of the unit of measure, If you have some please
share.
A more space wise representation, that add another layer of
indirection would be to have a look-up table like the following:
#+TBLNAME:speed-1-units-lookup
| name | unit |
|+--|
| speed | km/s |
| length | km |
| time | s|
I usually have an even more elaborate legend table like
| name | display | unit |
|--+-+--|
| sp | speed | km/s |
| le | length | km |
| ti | time| s|
And sometimes even (some) translation support:
| name | display | de | unit |
|--+-+-+--|
| sp | speed | Geschwindigkeit | km/s |
| le | length | Länge | km |
| ti | time| Zeit| s|
Only the table is in org, the handling is done not by org, but in R.
But I am not sure that it's worth it, at least for those that are
working with small data sets.
* support for inline rendering of tables
The most complex inline output I think I will need at some point
follows:
#+TBLNAME:data-set-1
#+NAME:data-set-1
| position | item1 | author1 | item2 | author2 | correlation |
|--+---+-+---+-+-|
|1 | i1| a1 | j1| b1 | c1 |
|2 | i2| a2 | j2| b2 | c2 |
|3 | i3| a3 | j3| b3 | c3 |
#+NAME: block-1
#+BEGIN_SRC R :session :colnames yes :exports none :var
data_set_1=data-set-1
# Just to show that we use R
data_set_2 - data_set_1
#+END_SRC
#+RESULTS: block-1
| position | item1 | author1 | item2 | author2 | correlation |
|--+---+-+---+-+-|
|1 | i1| a1 | j1| b1 | c1 |
|2 | i2| a2 | j2| b2 | c2 |
|3 | i3| a3 | j3| b3 | c3 |
And I really
Re: [O] [PATCH] inline src block results can be removed
On 2014-11-24 11:18, Andreas Leha wrote: Hi Daniele, I think your wishlist is somewhere further down the road. I usually implement some of your points in the src_language. I see that it would be nice if org supported these use cases, but I would see them as part of the LOB or maybe in some package in contrib rather than in core org/babel. For that, I'd argue removable inline results that can be exported without special formatting would be all that is needed to support your wishlist. Hi Andreas, sure, I think I will start getting the string I needed from src_language. But in some language is really hard to get facilities for text manipulation while the output as table generally supported. And I also think that the fine grained control of the layout of the inline sentence pertains to the org layer. A few inline comments below. Thanks again, they are really a nice help to start with! Best, Daniele
Re: [O] [PATCH] inline src block results can be removed
Charles C. Berry ccbe...@ucsd.edu writes:
For now, I'd be willing to make patches that will allow removal of the
inline src block results that do *not* involve these header args:
- :file fn
- :wrap wrapper
- :results latex html drawer org code
which I can do barely touching `org-babel-insert-result' and this
simplifies matters a lot.
IMO, we're too much focused on the implementation details. We ought to
agree on what should be done first. For example, considering
`org-babel-insert-result' and its RESULT-PARAMS argument, I think the
following makes sense:
| Param | Example output|
|-+---|
| default | {{{results(42)}}} |
| file| {{{results(file:something.pdf)}}} |
| list| |
| raw | 42|
| drawer | |
| org | {{{results(src_org{...})}}}|
| html| {{{results(@@html:...@@)}}} |
| latex | {{{results(@@latex:...@@)}}} |
| code| {{{results(src_xxx{...})}}}|
Basically, it should be possible to remove any kind of result using
results macro, with the exception of raw. list and drawer can
be ignored since there is no inline equivalent.
Another option for drawer is to also use export snippets. So,
basically, drawer something would generate
{{{results(@@something:...@@)}}}.
I propose to do this by using the patches of ox.el and ob-exp.el from
my last post. For ob-core.el, I would
- leave defcustom org-babel-inline-wrap as =%s= (or use defconst - I
do not have a strong opinion either way).
`org-babel-inline-wrap' as a defcustom is fine if you hardcode
{{{results(...)}}} wrapping at a higher level.
- modify `org-babel-examplify-region' along these lines
#+BEGIN_SRC emacs-lisp
(insert
(replace-regexp-in-string
, \\,
(format
(concat {{{results(
org-babel-inline-result-wrap
)}}}
(prog1 (buffer-substring beg end)
(delete-region beg end
nil t))
#+END_SRC
But this is unrelated to examplify. Wrapping should probably be
a dedicated function systematically called on results from an inline
block.
If it is felt that more retooling of `org-babel-insert-results' is really
needed, I can get to it early next year. In fact, I'll be out of email
range from late this month till then, so any problems I create now will
have to wait till then for me to work on them.
It's going to be difficult not to alter `org-babel-insert-results' since
the plan is to change completely how inline source blocks are handled.
There's no rush, however. Non-removable results from inline source
blocks have been there for a long time.
Regards,
Re: [O] [PATCH] inline src block results can be removed
On Fri, 14 Nov 2014, Nicolas Goaziou wrote:
Charles C. Berry ccbe...@ucsd.edu writes:
More patches (as you can see). Now ox.el, ob-core.el, and ob-exp.el
are patched.
Thanks.
[skipping to the bottom - omitting useful critiques of code and
opinions about strategy and tactics from Nicolas]
WDYT?
After staring at `org-babel-insert-result' for too long, I am beginning to
feel like Alice in Wonderland. As currently implemented, inline src blocks
are somewhere between fragile and broken. I worry about making them even
more fragile, the logic in `org-babel-insert-result' has plenty of twists
ans turns, and I cannot commit the effort to dig deeply into them. So I am
looking for an easy way out.
For now, I'd be willing to make patches that will allow removal of the
inline src block results that do *not* involve these header args:
- :file fn
- :wrap wrapper
- :results latex html drawer org code
which I can do barely touching `org-babel-insert-result' and this
simplifies matters a lot.
I propose to do this by using the patches of ox.el and ob-exp.el from
my last post. For ob-core.el, I would
- leave defcustom org-babel-inline-wrap as =%s= (or use defconst - I
do not have a strong opinion either way).
- allow replacement of exising results in `org-babel-insert-result'
like so
#+BEGIN_EXAMPLE
- (existing-result (unless inlinep
-(org-babel-where-is-src-block-result
+ (existing-result (if inlinep
+(org-babel-delete-results-macro)
+(org-babel-where-is-src-block-result
#+END_EXAMPLE
and defun `org-babel-delete-results-macro' per Nicolas' suggestions
for `org-babel-delete-babel-snippet' (from earlier patch) to use
`org-element-context' and friends (and not mess with export
snippets).
- modify `org-babel-examplify-region' along these lines
#+BEGIN_SRC emacs-lisp
(insert
(replace-regexp-in-string
, \\,
(format
(concat {{{results(
org-babel-inline-result-wrap
)}}}
(prog1 (buffer-substring beg end)
(delete-region beg end
nil t))
#+END_SRC
I believe that this is simple enough to avoid breaking idioms that
folks might use now.
As for the choice between =%s= and %s, the latter was hard coded
until f285b7ed3d097dd1cbb55fa3c31bc92aa0149054 in February 2013 and
has been the default since. It also parallels what happens with
handling src block results. Going forward I do not think this
behavior should change.
I have too litle experience with #+MACROs to know if Aaron's
suggestion to let the user customize the macro is opening up potential
issues when users get `creative'.
I can do what I've outlined in the coming days.
If it is felt that more retooling of `org-babel-insert-results' is really
needed, I can get to it early next year. In fact, I'll be out of email
range from late this month till then, so any problems I create now will
have to wait till then for me to work on them.
Thanks for the critique of my earlier patches and your thoughts.
Best,
Chuck
Re: [O] [PATCH] inline src block results can be removed
Hello,
Aaron Ecay aarone...@gmail.com writes:
This is a step back from the present situation, where a user can get
a custom format applied by default to all inline results by setting
‘org-babel-inline-result-wrap’. I think it’s reasonable for users to
want to set off babel results in a special font (to indicate that
they are automatically generated, e.g.) This proposal would add an
additional overhead for generating this formatting to every inline
call.
I expect Babel to provide a parameter to generate verbatim output, and
a way to use this parameter globally. If it doesn't, then we can keep
`org-babel-inline-result-wrap' and simply wrap {{{results(...)}}} macro
around the output.
I think the method I sketched above will allow users to redefine the
results macro in the same way as any other macro (via #+MACRO), thus
allowing the possibility of user-specified special formatting.
Babel output is expected to be modified by Babel functions/variables.
I wouldn't want to see macro used as the main tool to modify inline
Babel output.
Of course, you can still do it. But you're on your own.
{{{results(...)}}} is just a wrapper.
Indeed, this seems correct. (The “const” designation is just a hint to
users/developers – it doesn’t affect the ability of people who really
want to customize the value to setq it to something else in their init
file.)
Of course, users are free to break their local copy of Org.
With {{{results(@@backend:...@@)}}}, we don't need the extra trick. Even
if it is more verbose, I think a regular syntax is better.
This doesn’t mesh with the suggestion to allow the results macro to
supply formatting.
See above.
Regards,
--
Nicolas Goaziou
Re: [O] [PATCH] inline src block results can be removed
Nicolas,
More patches (as you can see). Now ox.el, ob-core.el, and ob-exp.el are
patched.
A few examples of how they render various src_lang[headers]{code} setups
are also attached.
Discussion inline below.
On Thu, 13 Nov 2014, Nicolas Goaziou wrote:
Hello,
Charles C. Berry ccbe...@ucsd.edu writes:
I like the flexibility that macros would allow.
I like it too. Macros are much better than export snippets for the task.
I don't think the usual #+MACRO works here, as the definition would be
found in `org-macro-templates' by the first call and existing stuff
would be expanded instead of being left for babel to remove it. But
setting it up as a document keyword should work, right?
Don't know if there are other gotchas.
Maybe a limited collection of formats could be set up to support basic
markup options and the macro could choose amongst them with a second
arg set by a babel header arg.
I think {{{results()}}} should remain a dumb wrapper itself and not try
to do some formatting (i.e., a simple, hard-coded macro). Formatting
should be on the side of Babel and, possibly, its arguments. Let's not
duplicate features.
Point taken.
Also, the user can customize org-babel-inline-result-wrap to always get
verbatim or otherwise wrap the contents of the macro.
I am not quite sure how to marry this to header args. Maybe the :wrap
header arg should be hijacked for inline src blocks to specify a macro
for the results.
Macro can be the default output. If you don't want a macro, use raw
header. IOW, there is no need for a specific header arg.
I mean, does anyone actually use stuff like src_R[:wrap latex]{1+2}?
The current result cannot be parsed as an export block, AFAICS.
It could evaluate to @@latex:3@@. Parsing can also be solved if
necessary.
`:wrap latex' results in @@latex: ... @@.
`:results latex' results in
: @@LaTeX:
: results@@
which is a bit unsightly, but can be parsed and removed.
I have not touched
- :RESULTS drawers
- lists
- tables
---
I appreciate your coaching/feedback.
Aaron Ecay's suggestion to use a macro was a good one. Thanks Aron.
Best,
Chuck
From b369b0a1e69fd2b91c8f4eb7d824dcd18232917b Mon Sep 17 00:00:00 2001
From: chasberry ccbe...@ucsd.edu
Date: Thu, 13 Nov 2014 20:45:01 -0800
Subject: [PATCH 1/3] lisp/ob-core.el: Replace inline `results' macro call or
export-snippet
* lisp/ob-core.el (org-babel-inline-result-wrap): Default is
{{{results(%s)}}}.
* lisp/ob-core.el (org-babel-insert-result): Delete any `results'
macro or export-snippet immediately following inline src block;
insert current value in 'results' macro or export snippet if :wrap
header argument is given. Escape commas in the result with
backslashes if the macro form is used.
* lisp/ob-core.el (org-babel-delete-babel-snippet): Add function to
delete {{{results(.*)}}} macro calls or @@backend:.*@@ provided
they follow inline src blocks. Adding extra spaces between an
export snippet and its inline src block will protect it from
removal.
---
lisp/ob-core.el | 49 +
1 file changed, 41 insertions(+), 8 deletions(-)
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 6c38677..227c8f0 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -156,7 +156,7 @@ See also `org-babel-noweb-wrap-start'.
:group 'org-babel
:type 'string)
-(defcustom org-babel-inline-result-wrap =%s=
+(defcustom org-babel-inline-result-wrap {{{results(%s)}}}
Format string used to wrap inline results.
This string must include a \%s\ which will be replaced by the results.
:group 'org-babel
@@ -2136,8 +2136,9 @@ code the results are extracted in the syntax of the
source
(goto-char (match-end 0))
(insert (if (listp result) \n ))
(point
-(existing-result (unless inlinep
- (org-babel-where-is-src-block-result
+(existing-result (if inlinep
+ (org-babel-delete-babel-snippet)
+(org-babel-where-is-src-block-result
t info hash indent)))
(results-switches
(cdr (assoc :results_switches (nth 2 info
@@ -2216,6 +2217,12 @@ code the results are extracted in the syntax of the
source
((member file result-params)
(when inlinep (goto-char inlinep))
(insert result))
+;; escape commas, e.g. {{{results(a\,b)}}}
+((and inlinep
+ (equal '(replace) result-params)
+ (not (assoc :wrap (nth 2 info
+ (goto-char beg)
+ (insert (replace-regexp-in-string , \\, result nil t)))
(t (goto-char beg) (insert result)))
(when (funcall proper-list-p result) (goto-char
(org-table-end)))
(setq end (point-marker))
@@
Re: [O] [PATCH] inline src block results can be removed
Charles C. Berry ccbe...@ucsd.edu writes:
More patches (as you can see). Now ox.el, ob-core.el, and ob-exp.el
are patched.
Thanks.
Also, the user can customize org-babel-inline-result-wrap to always
get verbatim or otherwise wrap the contents of the macro.
I don't think this is a good idea.
If we rely on the macro recognition to properly inline results, setting
to anything else would break Org. It should be a defconst or we are
bound to shoot ourselves in the foot.
If a user wants verbatim output, he will have to somehow generate
{{{results:=output=}}}, e.g., through appropriate code or parameters.
`:wrap latex' results in @@latex: ... @@.
I was thinking to {{{results(@@latex:...@@)}}}. A bit more verbose, but
you can also replace :wrap latex results.
`:results latex' results in
: @@LaTeX:
: results@@
which is a bit unsightly, but can be parsed and removed.
It is important to remove the spurious newline character, as it could
break the surrounding structure.
I have not touched
- :RESULTS drawers
- lists
- tables
These have to be ignored. An inline element cannot generate a non-inline
element. At least, we shouldn't make it easy to do that. There are
plenty other ways to generate those.
From b369b0a1e69fd2b91c8f4eb7d824dcd18232917b Mon Sep 17 00:00:00 2001
From: chasberry ccbe...@ucsd.edu
Date: Thu, 13 Nov 2014 20:45:01 -0800
Subject: [PATCH 1/3] lisp/ob-core.el: Replace inline `results' macro call or
export-snippet
* lisp/ob-core.el (org-babel-inline-result-wrap): Default is
{{{results(%s)}}}.
As written earlier, I highly suggest to make it a defconst.
* lisp/ob-core.el (org-babel-insert-result): Delete any `results'
macro or export-snippet immediately following inline src block;
insert current value in 'results' macro or export snippet if :wrap
header argument is given. Escape commas in the result with
backslashes if the macro form is used.
You also need to escape backslash characters before commas, per
(info (org) Macro replacement)
* lisp/ob-core.el (org-babel-delete-babel-snippet): Add function to
delete {{{results(.*)}}} macro calls or @@backend:.*@@ provided
they follow inline src blocks. Adding extra spaces between an
export snippet and its inline src block will protect it from
removal.
With {{{results(@@backend:...@@)}}}, we don't need the extra trick. Even
if it is more verbose, I think a regular syntax is better.
+ ;; escape commas, e.g. {{{results(a\,b)}}}
Missing capital and final dot.
+ ((and inlinep
+(equal '(replace) result-params)
Spurious space.
+(not (assoc :wrap (nth 2 info
`assq'
(cond
((assoc :wrap (nth 2 info))
(let ((name (or (cdr (assoc :wrap (nth 2 info))) RESULTS)))
- (funcall wrap (concat #+BEGIN_ name)
- (concat #+END_ (car (org-split-string name))
+ (if inlinep
+ (funcall wrap (concat @@ name :) @@ nil t)
+ (funcall wrap (concat #+BEGIN_ name)
+(concat #+END_ (car (org-split-string
name)))
((member html result-params)
- (funcall wrap #+BEGIN_HTML #+END_HTML))
- ((member latex result-params)
- (funcall wrap #+BEGIN_LaTeX #+END_LaTeX))
+ (if inlinep
+ (funcall wrap @@HTML: @@)
+ (funcall wrap #+BEGIN_HTML #+END_HTML)))
+ ((member latex result-params)
+ (if inlinep
+ (funcall wrap @@LaTeX: @@)
I'd rather have back-end names lowercase @@html:...@@ and @@latex:...@@
even though they are case insensitive anyway.
Also, you can avoid repeating funcall wrap:
(apply wrap (if inlinep '(@@latex: @@) '(#+begin_latex #+end_latex)))
+(defun org-babel-delete-babel-snippet (optional info)
+ When point is in an inline src block, delete an export-snippet
+or `results' macro call just after it. To protect export snippets
+from removal, add extra spaces between the src block and the
+snippet.
+ (let ((location (org-babel-where-is-src-block-result nil info)))
+(when location
+ (save-excursion
+ (goto-char location)
+ (cond
+ ((looking-at \\([ ]\\{1,2\\}\\)\\(@\\))
+ (goto-char (match-end 1))
+ (let ((export-snippet (org-element-export-snippet-parser)))
Don't call dedicated parser functions directly. The public functions are
`org-element-at-point' and `org-element-context'.
Anyway, this is not useful if we don't write directly export snippets.
+(if export-snippet
+(let ((start (org-element-property :begin export-snippet))
+ (end (org-element-property :end export-snippet)))
+ (delete-region start end)
+ ((looking-at
Re: [O] [PATCH] inline src block results can be removed
Hi Chuck, Hi Nicolas,
I had a response to Chuck’s earlier message that was sitting around
waiting to be finished...time marches on however. Apologies. I think
the following bit is the only part that’s potentially still relevant:
I don't think the usual #+MACRO works here, as the definition would be
found in `org-macro-templates' by the first call and existing stuff would
be expanded instead of being left for babel to remove it.
I see what you mean. One option might be to do the following in
org-export-as (untested pseudocode):
(org-macro-initialize-templates)
(let (results-macro)
(setq results-macro (assoc results org-macro-templates))
(setq org-macro-templates (remove results-macro org-macro-templates))
(org-macro-replace-all org-macro-templates)
(org-export-execute-babel-code)
(org-macro-replace-all (list (or results-macro
(cons results
org-default-result-macro) ;; new
defcustom or defvar
Back to the present:
2014ko azaroak 14an, Nicolas Goaziou-ek idatzi zuen:
Charles C. Berry ccbe...@ucsd.edu writes:
More patches (as you can see). Now ox.el, ob-core.el, and ob-exp.el
are patched.
Thanks.
Also, the user can customize org-babel-inline-result-wrap to always
get verbatim or otherwise wrap the contents of the macro.
I don't think this is a good idea.
If we rely on the macro recognition to properly inline results, setting
to anything else would break Org. It should be a defconst or we are
bound to shoot ourselves in the foot.
If a user wants verbatim output, he will have to somehow generate
{{{results:=output=}}}, e.g., through appropriate code or parameters.
This is a step back from the present situation, where a user can get
a custom format applied by default to all inline results by setting
‘org-babel-inline-result-wrap’. I think it’s reasonable for users to
want to set off babel results in a special font (to indicate that
they are automatically generated, e.g.) This proposal would add an
additional overhead for generating this formatting to every inline
call.
I think the method I sketched above will allow users to redefine the
results macro in the same way as any other macro (via #+MACRO), thus
allowing the possibility of user-specified special formatting.
[...]
* lisp/ob-core.el (org-babel-inline-result-wrap): Default is
{{{results(%s)}}}.
As written earlier, I highly suggest to make it a defconst.
Indeed, this seems correct. (The “const” designation is just a hint to
users/developers – it doesn’t affect the ability of people who really
want to customize the value to setq it to something else in their init
file.)
With {{{results(@@backend:...@@)}}}, we don't need the extra trick. Even
if it is more verbose, I think a regular syntax is better.
This doesn’t mesh with the suggestion to allow the results macro to
supply formatting.
--
Aaron Ecay
Re: [O] [PATCH] inline src block results can be removed
Hello,
Charles C. Berry ccbe...@ucsd.edu writes:
I like the flexibility that macros would allow.
I like it too. Macros are much better than export snippets for the task.
I don't think the usual #+MACRO works here, as the definition would be
found in `org-macro-templates' by the first call and existing stuff
would be expanded instead of being left for babel to remove it. But
setting it up as a document keyword should work, right?
Don't know if there are other gotchas.
Maybe a limited collection of formats could be set up to support basic
markup options and the macro could choose amongst them with a second
arg set by a babel header arg.
I think {{{results()}}} should remain a dumb wrapper itself and not try
to do some formatting (i.e., a simple, hard-coded macro). Formatting
should be on the side of Babel and, possibly, its arguments. Let's not
duplicate features.
I am not quite sure how to marry this to header args. Maybe the :wrap
header arg should be hijacked for inline src blocks to specify a macro
for the results.
Macro can be the default output. If you don't want a macro, use raw
header. IOW, there is no need for a specific header arg.
I mean, does anyone actually use stuff like src_R[:wrap latex]{1+2}?
The current result cannot be parsed as an export block, AFAICS.
It could evaluate to @@latex:3@@. Parsing can also be solved if
necessary.
Thanks for your work.
Regards,
--
Nicolas Goaziou
Re: [O] [PATCH] inline src block results can be removed
Hi,
Nicolas Goaziou m...@nicolasgoaziou.fr writes:
Hello,
Charles C. Berry ccbe...@ucsd.edu writes:
I like the flexibility that macros would allow.
I like it too. Macros are much better than export snippets for the task.
I don't think the usual #+MACRO works here, as the definition would be
found in `org-macro-templates' by the first call and existing stuff
would be expanded instead of being left for babel to remove it. But
setting it up as a document keyword should work, right?
Don't know if there are other gotchas.
Maybe a limited collection of formats could be set up to support basic
markup options and the macro could choose amongst them with a second
arg set by a babel header arg.
I think {{{results()}}} should remain a dumb wrapper itself and not try
to do some formatting (i.e., a simple, hard-coded macro). Formatting
should be on the side of Babel and, possibly, its arguments. Let's not
duplicate features.
I am not quite sure how to marry this to header args. Maybe the :wrap
header arg should be hijacked for inline src blocks to specify a macro
for the results.
Macro can be the default output. If you don't want a macro, use raw
header. IOW, there is no need for a specific header arg.
I mean, does anyone actually use stuff like src_R[:wrap latex]{1+2}?
The current result cannot be parsed as an export block, AFAICS.
It could evaluate to @@latex:3@@. Parsing can also be solved if
necessary.
Without too much value to add to this thread at this point, I just want
to say, that I love the direction this thread has taken. There is
good reason now to hope for better inline results handling in org.
Thanks for your work.
I second that! Thanks, Chuck!
Regards,
Andreas
Re: [O] [PATCH] inline src block results can be removed
Hi Chuck,
2014ko azaroak 12an, Charles C. Berry-ek idatzi zuen:
Inline src blocks cannot update their results --- causing some of us
heaadaches [1].
These patches fix that by placing the result of an inline src block in an
export snippet with a faux :back-end called 'babel'.
So C-c C-c with point on src_R{1+2} will insert `@@babel:3@@'. Updating
the contents of the inline src block and retyping C-c C-c will update the
snippet. On export, these snippets are rendered using the verbatim
transcoder, e.g. \texttt{3} for latex backends.
Support for most backends is provided.
org-babel-execute-buffer will also update such snippets.
Instead of using an export snippet, which requires per-backend changes,
you could wrap results in a macro, e.g. {{{results(2)}}}.
Users could customize this macro per-buffer (with the usual #+macro
keyword) to provide their own formatting of inline results. You
could provide the fallback interpretation in the second call to
‘org-macro-replace-all’ in ‘org-export-as’ (currently responsible
for expanding a few macros like {{{author}}} and {{{date}}}).
What do you think of this idea?
--
Aaron Ecay
Re: [O] [PATCH] inline src block results can be removed
On Wed, 12 Nov 2014, Aaron Ecay wrote:
Hi Chuck,
2014ko azaroak 12an, Charles C. Berry-ek idatzi zuen:
Inline src blocks cannot update their results --- causing some of us
heaadaches [1].
These patches fix that by placing the result of an inline src block in an
export snippet with a faux :back-end called 'babel'.
So C-c C-c with point on src_R{1+2} will insert `@@babel:3@@'. Updating
the contents of the inline src block and retyping C-c C-c will update the
snippet. On export, these snippets are rendered using the verbatim
transcoder, e.g. \texttt{3} for latex backends.
Support for most backends is provided.
org-babel-execute-buffer will also update such snippets.
Instead of using an export snippet, which requires per-backend changes,
you could wrap results in a macro, e.g. {{{results(2)}}}.
Users could customize this macro per-buffer (with the usual #+macro
keyword) to provide their own formatting of inline results. You
could provide the fallback interpretation in the second call to
‘org-macro-replace-all’ in ‘org-export-as’ (currently responsible
for expanding a few macros like {{{author}}} and {{{date}}}).
What do you think of this idea?
Aaron,
I like the flexibility that macros would allow.
Some care needs to be taken with commas, but not a big deal.
I don't think the usual #+MACRO works here, as the definition would be
found in `org-macro-templates' by the first call and existing stuff would
be expanded instead of being left for babel to remove it. But setting it
up as a document keyword should work, right?
Don't know if there are other gotchas.
Maybe a limited collection of formats could be set up to support basic
markup options and the macro could choose amongst them with a second arg
set by a babel header arg.
I am not quite sure how to marry this to header args. Maybe the :wrap
header arg should be hijacked for inline src blocks to specify a macro for
the results.
I mean, does anyone actually use stuff like src_R[:wrap latex]{1+2}? The
current result cannot be parsed as an export block, AFAICS.
Chuck
Re: [O] [PATCH] inline src block results can be removed
Hi Chuck,
Charles C. Berry ccbe...@ucsd.edu writes:
Inline src blocks cannot update their results --- causing some of us
heaadaches [1].
These patches fix that by placing the result of an inline src block in
an export snippet with a faux :back-end called 'babel'.
So C-c C-c with point on src_R{1+2} will insert
`@@babel:3@@'. Updating the contents of the inline src block and
retyping C-c C-c will update the snippet. On export, these snippets
are rendered using the verbatim transcoder, e.g. \texttt{3} for latex
backends.
Support for most backends is provided.
org-babel-execute-buffer will also update such snippets.
Please try these patches out.
Remember to recompile the files that these patches modify or you will
get wrong results when you restart emacs.
HTH,
Chuck
Footnote: [1] http://thread.gmane.org/gmane.emacs.orgmode/92481
p.s. If these patchesa are (eventually) acceptable, FSF has a
copyright assignment for me on file.
First of all: Thanks a lot! I'll (try to find time to) test these
patches.
Reading your description, I already have a first further feature
request, though ;-) Or rather a question.
It sounds as if your patches turn inline source into limited source
blocks in terms of adherence to header arguments. Given that most
likely there are not too many header arguments on inline source blocks,
this might not be a huge problem.
But my first question would be about ':results raw', as I never had a
case where I wanted the results of my inline source block to be
\texttt{}.
I guess, my question is: Do your patches restrict the use of babel
headers on inline source blocks. And if so, is that just a matter of
'not implemented yet' or is there a fundamental issue here?
Thanks,
Andreas
Re: [O] [PATCH] inline src block results can be removed
On Wed, 12 Nov 2014, Andreas Leha wrote:
Hi Chuck,
Charles C. Berry ccbe...@ucsd.edu writes:
Inline src blocks cannot update their results --- causing some of us
heaadaches [1].
[deleted announcement of fix]
First of all: Thanks a lot! I'll (try to find time to) test these
patches.
Yes. Please try them.
Reading your description, I already have a first further feature
request, though ;-) Or rather a question.
It sounds as if your patches turn inline source into limited source
blocks in terms of adherence to header arguments. Given that most
likely there are not too many header arguments on inline source blocks,
this might not be a huge problem.
See below :results latex and friends seem to work as before.
It would be nice if others could point out any hiccups.
But my first question would be about ':results raw', as I never had a
case where I wanted the results of my inline source block to be
\texttt{}.
If you start with src_R{1+2} and evaluate it, you get the `@@babel:3@@'.
If you modify that to src_R[:results raw]{1+2} and evaluate it, the
`@@...@@' goes away and is replaced with the raw result `3'. Of course,
you are then stuck with that and cannot easily make further revisions.
However, with modest tooling, you can set up the :results header
bufferwide so that when you want to export, the '@@...@@' will be removed
from the temp buffer and the export will use `raw' results for inline src
blocks.
Or you can make a custom version of org-latex-export-snippet that uses a
different transcoder. Or you can just return the raw text and the
`@@babel:3@@' will end up in the *.tex as 3. Just change the last line to
(org-element-property :value export-snippet)[more parens] to get the raw
value all the time.
I guess, my question is: Do your patches restrict the use of babel
headers on inline source blocks. And if so, is that just a matter of
'not implemented yet' or is there a fundamental issue here?
I do not think things are worse with the patches.
src_R[:results latex]{1+2} is the same with the patches, I think. But
ideally, that would generate something that acts like @@latex:3@@ and that
could be safely removed.
In terms of implementation, if one wanted fine control over each inline
src block, more tooling will be needed. Export snippets do not have lots
of stops and whistles to play with, just a :back-end and a :value and
location info. One could use `org-export-get-previous-element' to look up
the header args and figure out what to do next. But that can get hairy if
the header arg needs further processing. Considering the limited use to
which inline src blocks are put, it probably is not coding up loads of
tricky features.
Best,
Chuck
