Re: [O] org tables into R?
Aaron Ecay aarone...@gmail.com writes: Hi Rainer and Andreas, 2015ko urtarrilak 6an, Rainer M Krug-ek idatzi zuen: What would definitely solve the issue is if the string containing the tabs would use \t instead - but I have no idea how this could be achieved easily. This could be achieved by modifying the call to orgtbl-to-csv in org-babel-R-assign-elisp: diff --git i/lisp/ob-R.el w/lisp/ob-R.el index 6f76aa5..ec3f110 100644 --- i/lisp/ob-R.el +++ w/lisp/ob-R.el @@ -239,7 +239,10 @@ This function is called by `org-babel-execute-src-block'. (min (if lengths (apply 'min lengths) 0))) ;; Ensure VALUE has an orgtbl structure (depth of at least 2). (unless (listp (car value)) (setq value (list value))) - (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field))) + (let ((file (replace-regexp-in-string +\t \\t +(orgtbl-to-tsv value '(:raw t :fmt org-babel-R-quote-tsv-field)) +nil t)) (header (if (or (eq (nth 1 value) 'hline) colnames-p) TRUE FALSE)) (row-names (if rownames-p 1 NULL))) (This is just a quick hack; if it works then the code should actually use orgtbl-to-generic to do the conversion in a single step.) That looks like it would work. , | ... |sep = \\, ; was before sep = \\\t\ | ... ` This will set the separator to whitespace, i.e. that is one or more spaces, tabs, newlines or carriage returns (from R help). This was the approach that Michael took in his original patch. It introduces new bugs related to the handling of quotes, though – see the “Details” section of ?scan in R for the gory details (beginning with “If ‘sep’ is the default”...). I agree completely - it would not be a final solution but possibly solve the problem temporarily in an emacs.org file. But the real question is: why are on some setups the tabs converted to spaces? Andreas raised a bug a while ago that indicates to me that this code should switch to csv rather than a whitespace-delimited format. But there are lots of corner cases, and I have not had time to work on a patch that I am confident could cover them all. Might be a solution - but I do not know how commas are working under different locales. In MS Office, a csv file is a semicolon (;) separated file when in the chosen language settings a comma is used as decimal separator - and and I have no idea if language settings affect emacs in the same way (I actually don't hope so). This makes me think - why not use | as a separator (as it is the case in org)? In this way, all tables which are in org can be directly given to R as strings without the hassle of to much of conversion? Rainer Andreas’s original report (also referenced earlier in this thread): http://news.gmane.org/find-root.php?message_id=olulhmmtei5.fsf%40med.uni%2dgoettingen.de Thanks, -- Rainer M. Krug email: Raineratkrugsdotde PGP: 0x0F52F982 signature.asc Description: PGP signature
Re: [O] org tables into R?
Hi Aaron and Nicolas,
Thanks for partly fixing this issue. Unfortunately, when working with
source block interactively the issue persists for me. If in the same
sample file I view the source block via 'C-c C-v v' and step through the
generated code line-by-line, the table is again not split into columns.
This time I immediately tried with 'emacs -Q' and I can reproduce the
issue.
Here is the ecm again:
--8---cut here---start-8---
* test
#+NAME: data
| A | B | C |
|-++|
| 115 | 76 | 60 |
| 124 | 78 | 55 |
| 118 | 73 | 65 |
| 114 | 75 | 61 |
| 108 | 74 | 82 |
and pass it into R like this, then evaluation (C-c C-c) works, but
stepping through the code generated for viewing (C-c C-v v) does not:
#+BEGIN_SRC R :results output :exports both :session :var data.table=data
names(data.table)
head(data.table)
#+END_SRC
#+results:
: [1] A B C
: A B C
: 1 115 76 60
: 2 124 78 55
: 3 118 73 65
: 4 114 75 61
: 5 108 74 82
#+begin_src emacs-lisp
(concat
(replace-regexp-in-string on .* (emacs-version))
\n
(replace-regexp-in-string @.* ) (org-version nil t))
\n
(replace-regexp-in-string ].* ] (ess-version)))
#+end_src
#+results:
: GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version
10.9.4 (Build 13E28))
: of 2014-09-02
: Org-mode version 8.3beta (release_8.3beta-717-ga8096c)
: ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd]
--8---cut here---end---8---
Here is what I see in the org babel preview:
--8---cut here---start-8---
data.table - local({
con - textConnection(
\A\ \B\ \C\
\115\ \76\ \60\
\124\ \78\ \55\
\118\ \73\ \65\
\114\ \75\ \61\
\108\ \74\ \82\
)
res - utils::read.table(
con,
header= TRUE,
row.names = NULL,
sep = \t,
as.is = TRUE
)
close(con)
res
})
names(data.table)
head(data.table)
--8---cut here---end---8---
And finally, here is my R session, when I step through the code:
--8---cut here---start-8---
R version 3.1.2 (2014-10-31) -- Pumpkin Helmet
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin10.8.0 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
options(STERM='iESS', str.dendrogram.last=', editor='emacsclient',
show.error.locations=TRUE)
data.table - local({
+ con - textConnection(
+\A\ \B\ \C\
+ \115\ \76\ \60\
+ \124\ \78\ \55\
+ \118\ \73\ \65\
+ \114\ \75\ \61\
+ \108\ \74\ \82\
+ )
+ res - utils::read.table(
+con,
+header= TRUE,
+row.names = NULL,
+sep = \t,
+as.is = TRUE
+ )
+ close(con)
+ res
+})
names(data.table)
[1] A...B...C
head(data.table)
A...B...C
1 115 76 60
2 124 78 55
3 118 73 65
4 114 75 61
5 108 74 82
--8---cut here---end---8---
Regards,
Andreas
Re: [O] org tables into R?
Andreas Leha andreas.l...@med.uni-goettingen.de writes:
Hi Aaron and Nicolas,
Sorry for coming so late to this topic - I should try to figure out what
is causing the problem as I am the one responsible for this code...
Thanks for partly fixing this issue. Unfortunately, when working with
source block interactively the issue persists for me. If in the same
sample file I view the source block via 'C-c C-v v' and step through the
generated code line-by-line, the table is again not split into columns.
This time I immediately tried with 'emacs -Q' and I can reproduce the
issue.
Here is the ecm again:
* test
#+NAME: data
| A | B | C |
|-++|
| 115 | 76 | 60 |
| 124 | 78 | 55 |
| 118 | 73 | 65 |
| 114 | 75 | 61 |
| 108 | 74 | 82 |
and pass it into R like this, then evaluation (C-c C-c) works, but
stepping through the code generated for viewing (C-c C-v v) does not:
#+BEGIN_SRC R :results output :exports both :session :var data.table=data
names(data.table)
head(data.table)
#+END_SRC
#+results:
: [1] A B C
: A B C
: 1 115 76 60
: 2 124 78 55
: 3 118 73 65
: 4 114 75 61
: 5 108 74 82
#+begin_src emacs-lisp
(concat
(replace-regexp-in-string on .* (emacs-version))
\n
(replace-regexp-in-string @.* ) (org-version nil t))
\n
(replace-regexp-in-string ].* ] (ess-version)))
#+end_src
#+results:
: GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version
10.9.4 (Build 13E28))
: of 2014-09-02
: Org-mode version 8.3beta (release_8.3beta-717-ga8096c)
: ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd]
,
| #+RESULTS:
| : GNU Emacs 24.4.1 (x86_64-apple-darwin14.0.0, Carbon Version 157 AppKit
1343.16)
| : of 2014-11-24
| : Org-mode version 8.3beta (release_8.3beta-695-g390015)
| : ess-version: 14.09 [svn: 6041 (2014-09-13)]
`
I get the same in the preview, but it works for me. I do not use -q.
,
| data.table - local({
| + con - textConnection(
| +\A\ \B\ \C\
| + \115\ \76\ \60\
| + \124\ \78\ \55\
| + \118\ \73\ \65\
| + \114\ \75\ \61\
| + \108\ \74\ \82\
| + )
| + res - utils::read.table(
| +con,
| +header= TRUE,
| +row.names = NULL,
| +sep = \t,
| +as.is = TRUE
| + )
| + close(con)
| + res
| +})
|names(data.table)
| [1] A B C
|head(data.table)
| A B C
| 1 115 76 60
| 2 124 78 55
| 3 118 73 65
| 4 114 75 61
| 5 108 74 82
|
`
Here is what I see in the org babel preview:
This might be due to temail, but you don't have tabs (\t) between your
entries below - could you verify if this is the case? If yes, this might
be a locale issue (would be strange)?
data.table - local({
con - textConnection(
\A\ \B\ \C\
\115\ \76\ \60\
\124\ \78\ \55\
\118\ \73\ \65\
\114\ \75\ \61\
\108\ \74\ \82\
)
res - utils::read.table(
con,
header= TRUE,
row.names = NULL,
sep = \t,
as.is = TRUE
)
close(con)
res
})
names(data.table)
head(data.table)
One solution would be to insert (instead of the tab character) \t to
separate the fields?
You could try and modify the ob-R-transfer-variable-table-with-header -
From the help:
,
| ob-R-transfer-variable-table-with-header is a variable defined in `ob-R.el'.
| Its value is
| %s - local({\n con - textConnection(\n %S\n )\n res -
utils::read.table(\n con,\n header= %s,\n row.names =
%s,\n sep = \\\t\,\n as.is = TRUE\n )\n
close(con)\n res\n })
|
| This variable may be risky if used as a file-local variable.
|
| Documentation:
| R code used to transfer a table defined as a variable from org to R.
|
| This function is used when the table contains a header.
`
This is passed to the (format) function and one might be able to change
something there.
Cheers,
Rainer
And finally, here is my R session, when I step through the code:
R version 3.1.2 (2014-10-31) -- Pumpkin Helmet
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin10.8.0 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
options(STERM='iESS', str.dendrogram.last=', editor='emacsclient',
show.error.locations=TRUE)
data.table - local({
+ con - textConnection(
+\A\ \B\ \C\
Re: [O] org tables into R?
Hi Rainer,
Thanks for chiming in.
Rainer M Krug rai...@krugs.de writes:
Andreas Leha andreas.l...@med.uni-goettingen.de writes:
Hi Aaron and Nicolas,
Sorry for coming so late to this topic - I should try to figure out what
is causing the problem as I am the one responsible for this code...
Thanks for partly fixing this issue. Unfortunately, when working with
source block interactively the issue persists for me. If in the same
sample file I view the source block via 'C-c C-v v' and step through the
generated code line-by-line, the table is again not split into columns.
This time I immediately tried with 'emacs -Q' and I can reproduce the
issue.
Here is the ecm again:
* test
#+NAME: data
| A | B | C |
|-++|
| 115 | 76 | 60 |
| 124 | 78 | 55 |
| 118 | 73 | 65 |
| 114 | 75 | 61 |
| 108 | 74 | 82 |
and pass it into R like this, then evaluation (C-c C-c) works, but
stepping through the code generated for viewing (C-c C-v v) does not:
#+BEGIN_SRC R :results output :exports both :session :var data.table=data
names(data.table)
head(data.table)
#+END_SRC
#+results:
: [1] A B C
: A B C
: 1 115 76 60
: 2 124 78 55
: 3 118 73 65
: 4 114 75 61
: 5 108 74 82
#+begin_src emacs-lisp
(concat
(replace-regexp-in-string on .* (emacs-version))
\n
(replace-regexp-in-string @.* ) (org-version nil t))
\n
(replace-regexp-in-string ].* ] (ess-version)))
#+end_src
#+results:
: GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version
10.9.4 (Build 13E28))
: of 2014-09-02
: Org-mode version 8.3beta (release_8.3beta-717-ga8096c)
: ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd]
,
| #+RESULTS:
| : GNU Emacs 24.4.1 (x86_64-apple-darwin14.0.0, Carbon Version 157 AppKit
1343.16)
| : of 2014-11-24
| : Org-mode version 8.3beta (release_8.3beta-695-g390015)
| : ess-version: 14.09 [svn: 6041 (2014-09-13)]
`
I get the same in the preview, but it works for me. I do not use -q.
,
| data.table - local({
| + con - textConnection(
| +\A\ \B\ \C\
| + \115\ \76\ \60\
| + \124\ \78\ \55\
| + \118\ \73\ \65\
| + \114\ \75\ \61\
| + \108\ \74\ \82\
| + )
| + res - utils::read.table(
| +con,
| +header= TRUE,
| +row.names = NULL,
| +sep = \t,
| +as.is = TRUE
| + )
| + close(con)
| + res
| +})
|names(data.table)
| [1] A B C
|head(data.table)
| A B C
| 1 115 76 60
| 2 124 78 55
| 3 118 73 65
| 4 114 75 61
| 5 108 74 82
|
`
Here is what I see in the org babel preview:
This might be due to temail, but you don't have tabs (\t) between your
entries below - could you verify if this is the case? If yes, this might
be a locale issue (would be strange)?
Yes, there seem to be no tabs. If there were tabs, R should parse the
table correctly, I guess.
(BTW: My custom filter replacing tabs with spaces is still disabled,
even though I pulled from master, which includes the recent
commit 70f965535b07c03a40792237de3a970d1cb57551
Author: Nicolas Goaziou m...@nicolasgoaziou.fr
Date: Mon Jan 5 12:39:26 2015 +0100
org-table: Ignore filters and hooks in radio tables
)
data.table - local({
con - textConnection(
\A\ \B\ \C\
\115\ \76\ \60\
\124\ \78\ \55\
\118\ \73\ \65\
\114\ \75\ \61\
\108\ \74\ \82\
)
res - utils::read.table(
con,
header= TRUE,
row.names = NULL,
sep = \t,
as.is = TRUE
)
close(con)
res
})
names(data.table)
head(data.table)
One solution would be to insert (instead of the tab character) \t to
separate the fields?
You could try and modify the ob-R-transfer-variable-table-with-header -
From the help:
,
| ob-R-transfer-variable-table-with-header is a variable defined in `ob-R.el'.
| Its value is
| %s - local({\n con - textConnection(\n %S\n )\n res -
| utils::read.table(\n con,\n header = %s,\n row.names = %s,\n sep =
| \\\t\,\n as.is = TRUE\n )\n close(con)\n res\n })
|
| This variable may be risky if used as a file-local variable.
|
| Documentation:
| R code used to transfer a table defined as a variable from org to R.
|
| This function is used when the table contains a header.
`
This is passed to the (format) function and one might be able to change
something there.
Thanks for the hint. Do I assume correctly, that this variable is used
when the code block is evaluated as well? In that case, I am reluctant
to change it, as the evaluation now works. The surprising thing is the
different behaviour for preview and evaluation.
I am extremely tight with my time right now, so that I won't be able to
do much experimentation/investigation right now, I am afraid. But I
will try to still be helpful...
Thanks,
Andreas
Cheers,
Rainer
And finally,
Re: [O] org tables into R?
Andreas Leha andreas.l...@med.uni-goettingen.de writes:
Hi Rainer,
Thanks for chiming in.
Hope I can help...
Rainer M Krug rai...@krugs.de writes:
Andreas Leha andreas.l...@med.uni-goettingen.de writes:
Hi Aaron and Nicolas,
Sorry for coming so late to this topic - I should try to figure out what
is causing the problem as I am the one responsible for this code...
Thanks for partly fixing this issue. Unfortunately, when working with
source block interactively the issue persists for me. If in the same
sample file I view the source block via 'C-c C-v v' and step through the
generated code line-by-line, the table is again not split into columns.
This time I immediately tried with 'emacs -Q' and I can reproduce the
issue.
Here is the ecm again:
* test
#+NAME: data
| A | B | C |
|-++|
| 115 | 76 | 60 |
| 124 | 78 | 55 |
| 118 | 73 | 65 |
| 114 | 75 | 61 |
| 108 | 74 | 82 |
and pass it into R like this, then evaluation (C-c C-c) works, but
stepping through the code generated for viewing (C-c C-v v) does not:
#+BEGIN_SRC R :results output :exports both :session :var data.table=data
names(data.table)
head(data.table)
#+END_SRC
#+results:
: [1] A B C
: A B C
: 1 115 76 60
: 2 124 78 55
: 3 118 73 65
: 4 114 75 61
: 5 108 74 82
#+begin_src emacs-lisp
(concat
(replace-regexp-in-string on .* (emacs-version))
\n
(replace-regexp-in-string @.* ) (org-version nil t))
\n
(replace-regexp-in-string ].* ] (ess-version)))
#+end_src
#+results:
: GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version
10.9.4 (Build 13E28))
: of 2014-09-02
: Org-mode version 8.3beta (release_8.3beta-717-ga8096c)
: ess-version: 14.05 [git: 4283f1304a54502c42707b6a4ba347703f0992dd]
,
| #+RESULTS:
| : GNU Emacs 24.4.1 (x86_64-apple-darwin14.0.0, Carbon Version 157 AppKit
1343.16)
| : of 2014-11-24
| : Org-mode version 8.3beta (release_8.3beta-695-g390015)
| : ess-version: 14.09 [svn: 6041 (2014-09-13)]
`
I get the same in the preview, but it works for me. I do not use -q.
,
| data.table - local({
| + con - textConnection(
| +\A\\B\ \C\
| + \115\ \76\ \60\
| + \124\ \78\ \55\
| + \118\ \73\ \65\
| + \114\ \75\ \61\
| + \108\ \74\ \82\
| + )
| + res - utils::read.table(
| +con,
| +header= TRUE,
| +row.names = NULL,
| +sep = \t,
| +as.is = TRUE
| + )
| + close(con)
| + res
| +})
|names(data.table)
| [1] A B C
|head(data.table)
| A B C
| 1 115 76 60
| 2 124 78 55
| 3 118 73 65
| 4 114 75 61
| 5 108 74 82
|
`
Here is what I see in the org babel preview:
This might be due to temail, but you don't have tabs (\t) between your
entries below - could you verify if this is the case? If yes, this might
be a locale issue (would be strange)?
Yes, there seem to be no tabs. If there were tabs, R should parse the
table correctly, I guess.
Exactly. So somehow the tabs get replaced in your system by spaces.
(BTW: My custom filter replacing tabs with spaces is still disabled,
even though I pulled from master, which includes the recent
commit 70f965535b07c03a40792237de3a970d1cb57551
Author: Nicolas Goaziou m...@nicolasgoaziou.fr
Date: Mon Jan 5 12:39:26 2015 +0100
org-table: Ignore filters and hooks in radio tables
)
data.table - local({
con - textConnection(
\A\ \B\ \C\
\115\ \76\ \60\
\124\ \78\ \55\
\118\ \73\ \65\
\114\ \75\ \61\
\108\ \74\ \82\
)
res - utils::read.table(
con,
header= TRUE,
row.names = NULL,
sep = \t,
as.is = TRUE
)
close(con)
res
})
names(data.table)
head(data.table)
One solution would be to insert (instead of the tab character) \t to
separate the fields?
You could try and modify the ob-R-transfer-variable-table-with-header -
From the help:
,
| ob-R-transfer-variable-table-with-header is a variable defined in
`ob-R.el'.
| Its value is
| %s - local({\n con - textConnection(\n %S\n )\n res -
| utils::read.table(\n con,\n header = %s,\n row.names = %s,\n sep =
| \\\t\,\n as.is = TRUE\n )\n close(con)\n res\n })
|
| This variable may be risky if used as a file-local variable.
|
| Documentation:
| R code used to transfer a table defined as a variable from org to R.
|
| This function is used when the table contains a header.
`
This is passed to the (format) function and one might be able to change
something there.
Thanks for the hint. Do I assume correctly, that this variable is used
when the code block is evaluated as well? In that case, I am reluctant
to change it, as the evaluation now works. The surprising thing is the
different behaviour for preview and evaluation.
Yup - you are right. These variables
Re: [O] org tables into R?
Hi Nicolas,
2015ko urtarrilak 6an, Nicolas Goaziou-ek idatzi zuen:
Aaron Ecay aarone...@gmail.com writes:
You are correct about the silent dropping of macro-like text. However,
with current master that case gives an undefined macro error, which is
even worse.
I disagree. If we allow to insert macro-like text in the table, it will
become active in the generated table and will, sooner or later, generate
an undefined macro error anyway.
Try this (in emacs -Q with org and ESS in the load-path) to
see it:
#+name: foo
#+begin_src R
c(foo,{{{bar}}})
#+end_src
Yes, an error is thrown, but
#+RESULTS: foo
|foo|
| {{{bar}}} |
is as cheesy. See above.
Macros are a very special kind of datum in Org. Luckily, their syntax is
awkward so you're unlikely to create one unwillingly. How common is your
example?
Nothing unwilling about it – I had been deliberately trying to generate a
table with macros in it as the result of a babel block. These macros are
defined in the document, in order to encapsulate some fiddly typesetting
which recurs very commonly.
Shouldn’t this workflow be supported?
In any case, the point is broader: the orgtbl-* functions cannot cope
with macros in their input at all (not just in the context of babel). I
think the programmatic interface to org tables ought to be composable
with macros.
Thanks,
--
Aaron Ecay
Re: [O] org tables into R?
Aaron Ecay aarone...@gmail.com writes: Nothing unwilling about it – I had been deliberately trying to generate a table with macros in it as the result of a babel block. These macros are defined in the document, in order to encapsulate some fiddly typesetting which recurs very commonly. Shouldn’t this workflow be supported? Note that Babel blocks are evaluated after macro expansion. As a consequence, the workflow above is hardly useful, as it requires its user to manually evaluate all blocks generating macros /before/ exporting the document. If the workflow is not used and doesn't have intrinsic value, I don't think we need to invent kludges to support it. In any case, the point is broader: the orgtbl-* functions cannot cope with macros in their input at all (not just in the context of babel). I think the programmatic interface to org tables ought to be composable with macros. orgtbl-* are not an API for tables, but converters. My point is still valid: you don't have a real use-case, so just don't bother. Regards,
Re: [O] org tables into R?
Aaron Ecay aarone...@gmail.com writes:
You are correct about the silent dropping of macro-like text. However,
with current master that case gives an undefined macro error, which is
even worse.
I disagree. If we allow to insert macro-like text in the table, it will
become active in the generated table and will, sooner or later, generate
an undefined macro error anyway.
Try this (in emacs -Q with org and ESS in the load-path) to
see it:
#+name: foo
#+begin_src R
c(foo,{{{bar}}})
#+end_src
Yes, an error is thrown, but
#+RESULTS: foo
|foo|
| {{{bar}}} |
is as cheesy. See above.
Macros are a very special kind of datum in Org. Luckily, their syntax is
awkward so you're unlikely to create one unwillingly. How common is your
example?
OTOH ob-R.el should consider using :raw t parameter for its table
conversion function.
I think :raw is needed in ‘org-babel-insert-result’ in addition to my
previous patch. New patch attached.
Actually, my advice doesn't stand. :raw t is not sufficient to prevent
macro expansion, which happens at the beginning of the export process.
Regards,
Re: [O] org tables into R?
Hi Rainer and Andreas, 2015ko urtarrilak 6an, Rainer M Krug-ek idatzi zuen: What would definitely solve the issue is if the string containing the tabs would use \t instead - but I have no idea how this could be achieved easily. This could be achieved by modifying the call to orgtbl-to-csv in org-babel-R-assign-elisp: diff --git i/lisp/ob-R.el w/lisp/ob-R.el index 6f76aa5..ec3f110 100644 --- i/lisp/ob-R.el +++ w/lisp/ob-R.el @@ -239,7 +239,10 @@ This function is called by `org-babel-execute-src-block'. (min (if lengths (apply 'min lengths) 0))) ;; Ensure VALUE has an orgtbl structure (depth of at least 2). (unless (listp (car value)) (setq value (list value))) - (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field))) + (let ((file (replace-regexp-in-string +\t \\t +(orgtbl-to-tsv value '(:raw t :fmt org-babel-R-quote-tsv-field)) +nil t)) (header (if (or (eq (nth 1 value) 'hline) colnames-p) TRUE FALSE)) (row-names (if rownames-p 1 NULL))) (This is just a quick hack; if it works then the code should actually use orgtbl-to-generic to do the conversion in a single step.) , | ... |sep = \\, ; was before sep = \\\t\ | ... ` This will set the separator to whitespace, i.e. that is one or more spaces, tabs, newlines or carriage returns (from R help). This was the approach that Michael took in his original patch. It introduces new bugs related to the handling of quotes, though – see the “Details” section of ?scan in R for the gory details (beginning with “If ‘sep’ is the default”...). Andreas raised a bug a while ago that indicates to me that this code should switch to csv rather than a whitespace-delimited format. But there are lots of corner cases, and I have not had time to work on a patch that I am confident could cover them all. Andreas’s original report (also referenced earlier in this thread): http://news.gmane.org/find-root.php?message_id=olulhmmtei5.fsf%40med.uni%2dgoettingen.de Thanks, -- Aaron Ecay
Re: [O] org tables into R?
Hello,
Aaron Ecay aarone...@gmail.com writes:
Recently(-ish), Nicolas updated the org-table functions to use the
export framework. One of the drawbacks of this approach (as you have
brought to our attention) is that export filters are invoked. This
should probably be disabled.
Indeed. I removed user-defined filters and hooks. Thank you.
Another drawback is the detection of unexpanded macros (which will be
triggered e.g. if a tabular babel result has macro-like {{{text}}} in
it). A proof-of-concept patch for this is attached.
I don't think it needs to be fixed: macro would then be silently
dropped, which is a step backwards. OTOH ob-R.el should consider using
:raw t parameter for its table conversion function.
Regards,
--
Nicolas Goaziou
Re: [O] org tables into R?
Hi Nicolas,
2015ko urtarrilak 5an, Nicolas Goaziou-ek idatzi zuen:
Hello,
Aaron Ecay aarone...@gmail.com writes:
Recently(-ish), Nicolas updated the org-table functions to use the
export framework. One of the drawbacks of this approach (as you have
brought to our attention) is that export filters are invoked. This
should probably be disabled.
Indeed. I removed user-defined filters and hooks. Thank you.
Another drawback is the detection of unexpanded macros (which will be
triggered e.g. if a tabular babel result has macro-like {{{text}}} in
it). A proof-of-concept patch for this is attached.
I don't think it needs to be fixed: macro would then be silently
dropped, which is a step backwards.
You are correct about the silent dropping of macro-like text. However,
with current master that case gives an undefined macro error, which is
even worse. Try this (in emacs -Q with org and ESS in the load-path) to
see it:
#+name: foo
#+begin_src R
c(foo,{{{bar}}})
#+end_src
OTOH ob-R.el should consider using :raw t parameter for its table
conversion function.
I think :raw is needed in ‘org-babel-insert-result’ in addition to my
previous patch. New patch attached.
From 4d2985ba88d2ba0c35ff715a7285469e8040d4b0 Mon Sep 17 00:00:00 2001
From: Aaron Ecay aarone...@gmail.com
Date: Sun, 4 Jan 2015 18:14:26 -0500
Subject: [PATCH] [babel] fix macros in tabular output
* lisp/ox.el (org-export-as): Support :sloppy-macros plist entry.
* lisp/org-table.el (orgtbl-to-generic): Use it.
* lisp/ob-R.el (org-babel-R-assign-elisp):
* lisp/ob-core.el (org-babel-insert-result): Use :raw argument to
org-table conversion.
---
lisp/ob-R.el | 2 +-
lisp/ob-core.el | 2 +-
lisp/org-table.el | 2 +-
lisp/ox.el| 3 ++-
4 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/lisp/ob-R.el b/lisp/ob-R.el
index 2470b4f..6f76aa5 100644
--- a/lisp/ob-R.el
+++ b/lisp/ob-R.el
@@ -239,7 +239,7 @@ This function is called by `org-babel-execute-src-block'.
(min (if lengths (apply 'min lengths) 0)))
;; Ensure VALUE has an orgtbl structure (depth of at least 2).
(unless (listp (car value)) (setq value (list value)))
- (let ((file (orgtbl-to-tsv value '(:fmt org-babel-R-quote-tsv-field)))
+ (let ((file (orgtbl-to-tsv value '(:raw t :fmt org-babel-R-quote-tsv-field)))
(header (if (or (eq (nth 1 value) 'hline) colnames-p)
TRUE FALSE))
(row-names (if rownames-p 1 NULL)))
diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 93fcb2a..9ff83f2 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -2213,7 +2213,7 @@ code the results are extracted in the syntax of the source
(lambda (el) (or (listp el) (eq el 'hline)))
result)
result (list result))
- '(:fmt (lambda (cell) (format %s cell \n))
+ '(:raw t :fmt (lambda (cell) (format %s cell \n))
(goto-char beg) (when (org-at-table-p) (org-table-align)))
((and (listp result) (not (funcall proper-list-p result)))
(insert (format %s\n result)))
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6b33eda..7a53d7a 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -4834,7 +4834,7 @@ This may be either a string or a function of two arguments:
(table-cell . ,(org-table--to-generic-cell params))
;; Section. Return contents to avoid garbage around table.
(section . (lambda (s c i) c
- 'body-only (org-combine-plists params '(:with-tables t))
+ 'body-only (org-combine-plists params '(:with-tables t :sloppy-macros t))
(defun org-table--generic-apply (value name optional with-cons rest args)
(cond ((null value) nil)
diff --git a/lisp/ox.el b/lisp/ox.el
index f47baef..0fcfc04 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -2885,7 +2885,8 @@ Return code as a string.
(cons email (or (plist-get info :email) ))
(cons title
(org-element-interpret-data (plist-get info :title
- 'finalize)
+ (unless (plist-get info :sloppy-macros)
+ 'finalize))
;; Parse buffer.
(setq tree (org-element-parse-buffer nil visible-only))
;; Handle left-over uninterpreted elements or objects in
--
2.2.1
There are several usages of orgtbl-to-* functions in babel (R, awk,
gnuplot, shell, sqlite, sql). Org-plot also uses one. The attached
patch adds :raw to the ob-R call, but if I understand correctly probably
all of them should add it. Should I do that?
Thanks,
--
Aaron Ecay
Re: [O] org tables into R?
Hi Vikas, Thanks for following up on this issue! Vikas Rawal vikasli...@agrarianresearch.org writes: Michael Gauland mikely...@gmail.com writes: I want to use an R block to manipulate data from an org table, but I'm not getting what I expect. What am I doing wrong? I believe, that this is a bug that has been reported already [1] and is still open. The code sent by Michael works fine for me (that it, the org table is read as a data.frame). It may help to upgrade your org-mode. What version are you on? Vikas It turns out I cannot reproduce it either with emacs -Q. I investigated and found that the culprit is an export filter I have defined that replaces tabs with spaces: --8---cut here---start-8--- (defun my-e-beamer-final-filter (contents backend info) (replace-regexp-in-string \t contents)) (add-to-list 'org-export-filter-final-output-functions 'my-e-beamer-final-filter) --8---cut here---end---8--- Now, I consider that a bug as well. I am not exporting anything, here. But I am merely evaluating a code block. Export filters should not interfere. Plus, it used to work (unfortunately, I do not have the time to bisect right now). So, why are export filters involved? And can that be disabled? Regards, Andreas PS: Here is my environment: #+begin_src emacs-lisp (concat (replace-regexp-in-string on .* (emacs-version)) \n (replace-regexp-in-string @.* (org-version nil t))) #+end_src #+results: : GNU Emacs 24.4.50.1 (x86_64-apple-darwin13.3.0, NS appkit-1265.21 Version 10.9.4 (Build 13E28)) : of 2014-09-02 : Org-mode version 8.3beta (release_8.3beta-709-g3e64f6
Re: [O] org tables into R?
Hi Andreas,
Thanks for following up on this problem.
2015ko urtarrilak 4an, Andreas Leha-ek idatzi zuen:
I investigated and found that the culprit is an export filter I have
defined that replaces tabs with spaces:
--8---cut here---start-8---
(defun my-e-beamer-final-filter (contents backend info)
(replace-regexp-in-string \t contents))
(add-to-list 'org-export-filter-final-output-functions
'my-e-beamer-final-filter)
--8---cut here---end---8---
Now, I consider that a bug as well. I am not exporting anything, here.
But I am merely evaluating a code block. Export filters should not
interfere. Plus, it used to work (unfortunately, I do not have the time
to bisect right now).
So, why are export filters involved? And can that be disabled?
Recently(-ish), Nicolas updated the org-table functions to use the
export framework. One of the drawbacks of this approach (as you have
brought to our attention) is that export filters are invoked. This
should probably be disabled.
Another drawback is the detection of unexpanded macros (which will be
triggered e.g. if a tabular babel result has macro-like {{{text}}} in
it). A proof-of-concept patch for this is attached.
From 50ce44be96f446272d1e465c01265632f0ed488f Mon Sep 17 00:00:00 2001
From: Aaron Ecay aarone...@gmail.com
Date: Sun, 4 Jan 2015 18:14:26 -0500
Subject: [PATCH] [export] allow sloppy usage of macros for internal APIs
* lisp/ox.el (org-export-as): Support :sloppy-macros plist entry.
* lisp/org-table.el (orgtbl-to-generic): Use it.
---
lisp/org-table.el | 2 +-
lisp/ox.el| 3 ++-
2 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/lisp/org-table.el b/lisp/org-table.el
index 6b33eda..7a53d7a 100644
--- a/lisp/org-table.el
+++ b/lisp/org-table.el
@@ -4834,7 +4834,7 @@ This may be either a string or a function of two arguments:
(table-cell . ,(org-table--to-generic-cell params))
;; Section. Return contents to avoid garbage around table.
(section . (lambda (s c i) c
- 'body-only (org-combine-plists params '(:with-tables t))
+ 'body-only (org-combine-plists params '(:with-tables t :sloppy-macros t))
(defun org-table--generic-apply (value name optional with-cons rest args)
(cond ((null value) nil)
diff --git a/lisp/ox.el b/lisp/ox.el
index f47baef..0fcfc04 100644
--- a/lisp/ox.el
+++ b/lisp/ox.el
@@ -2885,7 +2885,8 @@ Return code as a string.
(cons email (or (plist-get info :email) ))
(cons title
(org-element-interpret-data (plist-get info :title
- 'finalize)
+ (unless (plist-get info :sloppy-macros)
+ 'finalize))
;; Parse buffer.
(setq tree (org-element-parse-buffer nil visible-only))
;; Handle left-over uninterpreted elements or objects in
--
2.2.1
I wonder, though, whether the various exceptions like this that will
stack up will tip the balance towards its being simpler for org-table to
have its own set of functions that operate on the org-element parse
tree, but don’t invoke org-export code. For csv this would be something
like (pseudocode):
(org-element-mapconcat
(lambda (table-row)
(org-element-mapconcat
(lambda (table-cell)
(quote-for-csv (org-element-interpret table-cell))
table-row
,))
table
\n)
Nicolas, WDYT? If you prefer to continue using the export framework, I
can add some commentary to the attached patch and install it.
Thanks,
--
Aaron Ecay
Re: [O] org tables into R?
I modified the definition of ob-R-transfer-variable-table-with-header in ob-R.el by changing 'sep' from \\\t\ to \\. Now #+BEGIN_SRC R :results output :exports both :session :var data.table=data :colnames yes names(data.table) head(data.table) #+END_SRC (note the addition of :colnames=yes) give me: #+RESULTS: : [1] A B C : A B C : 1 115 76 60 : 2 124 78 55 : 3 118 73 65 : 4 114 75 61 : 5 108 74 82 as I expect. What is the current procedure to submit a patch? Thanks, Michael
Re: [O] org tables into R?
Hi Michael, Michael Gauland mikely...@gmail.com writes: I want to use an R block to manipulate data from an org table, but I'm not getting what I expect. If I define a table like this: #+NAME: data | A | B | C | |-+-+| | 115 | 76 | 60 | | 124 | 78 | 55 | | 118 | 73 | 65 | | 114 | 75 | 61 | | 108 | 74 | 82 | and pass it into R like this: #+BEGIN_SRC R :results output :exports both :session :var data.table=data names(data.table) head(data.table) #+END_SRC I expect to get a table with three columns (A, B, and C), but instead I seem to get a single column named A.B.C: #+RESULTS: : [1] A.B.C : A.B.C : 1 115 76 60 : 2 124 78 55 : 3 118 73 65 : 4 114 75 61 : 5 108 74 82 What am I doing wrong? I believe, that this is a bug that has been reported already [1] and is still open. Regards, Andreas [1] http://news.gmane.org/find-root.php?message_id=olulhmmtei5.fsf%40med.uni%2dgoettingen.de
[O] org tables into R?
I want to use an R block to manipulate data from an org table, but I'm not getting what I expect. If I define a table like this: #+NAME: data | A | B | C | |-+-+| | 115 | 76 | 60 | | 124 | 78 | 55 | | 118 | 73 | 65 | | 114 | 75 | 61 | | 108 | 74 | 82 | and pass it into R like this: #+BEGIN_SRC R :results output :exports both :session :var data.table=data names(data.table) head(data.table) #+END_SRC I expect to get a table with three columns (A, B, and C), but instead I seem to get a single column named A.B.C: #+RESULTS: : [1] A.B.C : A.B.C : 1 115 76 60 : 2 124 78 55 : 3 118 73 65 : 4 114 75 61 : 5 108 74 82 What am I doing wrong? Thanks, Michael Gauland
Re: [O] org tables into R?
Michael Gauland mikely...@gmail.com mailto:mikely...@gmail.com writes: I want to use an R block to manipulate data from an org table, but I'm not getting what I expect. What am I doing wrong? I believe, that this is a bug that has been reported already [1] and is still open. The code sent by Michael works fine for me (that it, the org table is read as a data.frame). It may help to upgrade your org-mode. What version are you on? Vikas
