Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
Achim Gratz strom...@nexgo.de writes: On my 32-bit Arch machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321.0 Since that latter number is not representable as an integer in Emacs (it loses 4 bit due to the way Lisp represents these things), it takes the next best thing (FP in that case). Since you are on Arch, I believe there is a way to compile Emacs to use large integers even on 32bit systems - it may have other problems, but should solve this. I've just compiled Emacs again using --with-wide-int as an additional (or sole) configure option and get: (string-to-number 987654321): 987654321 (#o7267464261, #x3ade68b1) So, no need to upgrade the whole system to 64bit if that was the only problem... :-) Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ SD adaptations for Waldorf Q V3.00R3 and Q+ V3.54R2: http://Synth.Stromeko.net/Downloads.html#WaldorfSDada
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
Just to be safe, I nuked my org-mode directory and re-cloned from the git repository: I'm now at release_7.8.03.386.g2239d (I was at release_7.8.03-351-g47eb3 previously). Is there a more recent version? I also removed the org directory that came with the packaged version of emacs, just in case. This sounds just like my setup. I get the following from `org-version' which should be equivalent. Org-mode version 7.8.03 (release_7.8.03.419.gdeb6e) I start emacs with the following options: $ emacs -Q -eval '(setq load-path (cons ~/share/emacs/site-lisp/org-mode/lisp load-path))' -eval (require ob-R) test.org I just did the same using the following command line to launch Emacs, and I still get no decimal place in my results. $ emacs -Q -eval '(setq laod-path (cons ~/.emacs.d/src/org/lisp/ load-path))' -eval (require 'ob-R) /tmp/dan.org I get the same behavior as before. I was a little puzzled when I saw this happen, as I've been working on this study for over a year in org and it's the first time I've seen this behavior. I went back and tried previous versions (e.g., 7.4), and now I see the same thing. I wonder if Arch (a rolling release linux distribution that tries to be close to the leading edge) has updated an underlying library that impacts the babel code in some odd way. (Granted, I don't see how this could be, as the emacs environment seems pretty well insulated from the underlying OS; PEBKAC just as likely.) I doubt this is the case. I also use Arch (just ran pacman -Syu this morning) so we likely have the largely same underlying versions of libraries installed. Maybe the issue could lay somewhere in your R config? Sorry I can't be of more help. Best, In any event, since you can't reproduce the behavior (thanks again for trying), I don't expect it's anything you can fix. Maybe it will show up in other distributions as they update to more current versions. In the mean time, I'll try to improve my elisp skills and figure out where the extraneous formatting occurs. Best, Dan - GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info - Org-mode version 7.8.03 (release_7.8.03.386.g2239d) -- Eric Schulte http://cs.unm.edu/~eschulte/
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
On 02/19/2012 08:48 AM, Eric Schulte wrote: Just to be safe, I nuked my org-mode directory and re-cloned from the git repository: I'm now at release_7.8.03.386.g2239d (I was at release_7.8.03-351-g47eb3 previously). Is there a more recent version? I also removed the org directory that came with the packaged version of emacs, just in case. This sounds just like my setup. I get the following from `org-version' which should be equivalent. Org-mode version 7.8.03 (release_7.8.03.419.gdeb6e) I start emacs with the following options: $ emacs -Q -eval '(setq load-path (cons ~/share/emacs/site-lisp/org-mode/lisp load-path))' -eval (require ob-R) test.org I just did the same using the following command line to launch Emacs, and I still get no decimal place in my results. $ emacs -Q -eval '(setq laod-path (cons ~/.emacs.d/src/org/lisp/ load-path))' -eval (require 'ob-R) /tmp/dan.org I get the same behavior as before. I was a little puzzled when I saw this happen, as I've been working on this study for over a year in org and it's the first time I've seen this behavior. I went back and tried previous versions (e.g., 7.4), and now I see the same thing. I wonder if Arch (a rolling release linux distribution that tries to be close to the leading edge) has updated an underlying library that impacts the babel code in some odd way. (Granted, I don't see how this could be, as the emacs environment seems pretty well insulated from the underlying OS; PEBKAC just as likely.) I doubt this is the case. I also use Arch (just ran pacman -Syu this morning) so we likely have the largely same underlying versions of libraries installed. Maybe the issue could lay somewhere in your R config? Sorry I can't be of more help. Best, At first I thought it was R as well, but the fact that there is no decimal point in the output file plus the fact that (outside of emacs) I can use read.table to pull in the table and the result has no decimal formatting makes me think otherwise. That you're running the same setup as me, though, gives me hope. I can backup, re-install Arch, and see if the problem goes away. Thanks for your help. I'm not certain when I'll have time to try this, but I'll follow up on this thread when I do. Best, Dan In any event, since you can't reproduce the behavior (thanks again for trying), I don't expect it's anything you can fix. Maybe it will show up in other distributions as they update to more current versions. In the mean time, I'll try to improve my elisp skills and figure out where the extraneous formatting occurs. Best, Dan - GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info - Org-mode version 7.8.03 (release_7.8.03.386.g2239d)
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
On 02/19/2012 09:40 AM, Daniel Drake wrote: On 02/19/2012 08:48 AM, Eric Schulte wrote: Just to be safe, I nuked my org-mode directory and re-cloned from the git repository: I'm now at release_7.8.03.386.g2239d (I was at release_7.8.03-351-g47eb3 previously). Is there a more recent version? I also removed the org directory that came with the packaged version of emacs, just in case. This sounds just like my setup. I get the following from `org-version' which should be equivalent. Org-mode version 7.8.03 (release_7.8.03.419.gdeb6e) I start emacs with the following options: $ emacs -Q -eval '(setq load-path (cons ~/share/emacs/site-lisp/org-mode/lisp load-path))' -eval (require ob-R) test.org I just did the same using the following command line to launch Emacs, and I still get no decimal place in my results. $ emacs -Q -eval '(setq laod-path (cons ~/.emacs.d/src/org/lisp/ load-path))' -eval (require 'ob-R) /tmp/dan.org I get the same behavior as before. I was a little puzzled when I saw this happen, as I've been working on this study for over a year in org and it's the first time I've seen this behavior. I went back and tried previous versions (e.g., 7.4), and now I see the same thing. I wonder if Arch (a rolling release linux distribution that tries to be close to the leading edge) has updated an underlying library that impacts the babel code in some odd way. (Granted, I don't see how this could be, as the emacs environment seems pretty well insulated from the underlying OS; PEBKAC just as likely.) I doubt this is the case. I also use Arch (just ran pacman -Syu this morning) so we likely have the largely same underlying versions of libraries installed. Maybe the issue could lay somewhere in your R config? Sorry I can't be of more help. Best, At first I thought it was R as well, but the fact that there is no decimal point in the output file plus the fact that (outside of emacs) I can use read.table to pull in the table and the result has no decimal formatting makes me think otherwise. That you're running the same setup as me, though, gives me hope. I can backup, re-install Arch, and see if the problem goes away. Thanks for your help. I'm not certain when I'll have time to try this, but I'll follow up on this thread when I do. Best, Dan I'm following up on my last post, just to have it in the record: I've boiled down the behavior to these two examples: output the results by value vs output. R seems not to have anything to do with it. Somehow the by value code is intervening. I'll try to debug. command line: $ emacs -Q -eval '(setq load-path (cons ~/share/emacs/site-lisp/org-mode/lisp load-path))' -eval (require 'ob-sh) test.org In the buffer: *** by output #+name: by-output #+begin_src sh :results output echo 987654321 #+end_src #+RESULTS: by-output : 987654321 *** by value #+name: by-value #+begin_src sh :results value echo 987654321 #+end_src #+RESULTS: by-value : 987654321.0 In any event, since you can't reproduce the behavior (thanks again for trying), I don't expect it's anything you can fix. Maybe it will show up in other distributions as they update to more current versions. In the mean time, I'll try to improve my elisp skills and figure out where the extraneous formatting occurs. Best, Dan - GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info - Org-mode version 7.8.03 (release_7.8.03.386.g2239d)
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
On 02/19/2012 10:28 AM, Daniel Drake wrote: On 02/19/2012 09:40 AM, Daniel Drake wrote: On 02/19/2012 08:48 AM, Eric Schulte wrote: Just to be safe, I nuked my org-mode directory and re-cloned from the git repository: I'm now at release_7.8.03.386.g2239d (I was at release_7.8.03-351-g47eb3 previously). Is there a more recent version? I also removed the org directory that came with the packaged version of emacs, just in case. This sounds just like my setup. I get the following from `org-version' which should be equivalent. Org-mode version 7.8.03 (release_7.8.03.419.gdeb6e) I start emacs with the following options: $ emacs -Q -eval '(setq load-path (cons ~/share/emacs/site-lisp/org-mode/lisp load-path))' -eval (require ob-R) test.org I just did the same using the following command line to launch Emacs, and I still get no decimal place in my results. $ emacs -Q -eval '(setq laod-path (cons ~/.emacs.d/src/org/lisp/ load-path))' -eval (require 'ob-R) /tmp/dan.org I get the same behavior as before. I was a little puzzled when I saw this happen, as I've been working on this study for over a year in org and it's the first time I've seen this behavior. I went back and tried previous versions (e.g., 7.4), and now I see the same thing. I wonder if Arch (a rolling release linux distribution that tries to be close to the leading edge) has updated an underlying library that impacts the babel code in some odd way. (Granted, I don't see how this could be, as the emacs environment seems pretty well insulated from the underlying OS; PEBKAC just as likely.) I doubt this is the case. I also use Arch (just ran pacman -Syu this morning) so we likely have the largely same underlying versions of libraries installed. Maybe the issue could lay somewhere in your R config? Sorry I can't be of more help. Best, At first I thought it was R as well, but the fact that there is no decimal point in the output file plus the fact that (outside of emacs) I can use read.table to pull in the table and the result has no decimal formatting makes me think otherwise. That you're running the same setup as me, though, gives me hope. I can backup, re-install Arch, and see if the problem goes away. Thanks for your help. I'm not certain when I'll have time to try this, but I'll follow up on this thread when I do. Best, Dan I'm following up on my last post, just to have it in the record: I've boiled down the behavior to these two examples: output the results by value vs output. R seems not to have anything to do with it. Somehow the by value code is intervening. I'll try to debug. command line: $ emacs -Q -eval '(setq load-path (cons ~/share/emacs/site-lisp/org-mode/lisp load-path))' -eval (require 'ob-sh) test.org In the buffer: *** by output #+name: by-output #+begin_src sh :results output echo 987654321 #+end_src #+RESULTS: by-output : 987654321 *** by value #+name: by-value #+begin_src sh :results value echo 987654321 #+end_src #+RESULTS: by-value : 987654321.0 A further followup, at the risk of descending into minutia. The culprit seems to be the emacs function string-to-number. On my 32-bit Arch machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321.0 GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info On a 64-bit Ubuntu (11.10) machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321 (#o7267464261, #x3ade68b1) GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.5) of 2011-08-14 on allspice, modified by Debian The extra 3 bits required to represent the larger number apparently push my version of emacs into floating point territory.(log2(987654321)=29.87943; log2(123456789)=26.87943) Don't know if the difference is between emacs versions or 32- vs. 64-bit implementations, but I can probably figure it out from here. In any event, this doesn't seem to be an org-mode problem. Thanks again for all your help and for org-mode/babel in general! Dan In any event, since you can't reproduce the behavior (thanks again for trying), I don't expect it's anything you can fix. Maybe it will show up in other distributions as they update to more current versions. In the mean time, I'll try to improve my elisp skills and figure out where the extraneous formatting occurs. Best, Dan - GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info - Org-mode version 7.8.03 (release_7.8.03.386.g2239d)
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
A further followup, at the risk of descending into minutia. The culprit seems to be the emacs function string-to-number. On my 32-bit Arch machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321.0 GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info On a 64-bit Ubuntu (11.10) machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321 (#o7267464261, #x3ade68b1) GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.5) of 2011-08-14 on allspice, modified by Debian Ah, nice debugging! That must be the issue, as I'm working on a 64-bit machine. Maybe one workaround could be to prefix the numbers with a letter in R-land so that emacs interprets them as strings. Cheers, -- Eric Schulte http://cs.unm.edu/~eschulte/
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
On Sun 19 Feb 2012 12:46:15 PM PST, Eric Schulte wrote: A further followup, at the risk of descending into minutia. The culprit seems to be the emacs function string-to-number. On my 32-bit Arch machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321.0 GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info On a 64-bit Ubuntu (11.10) machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321 (#o7267464261, #x3ade68b1) GNU Emacs 23.3.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.5) of 2011-08-14 on allspice, modified by Debian Ah, nice debugging! That must be the issue, as I'm working on a 64-bit machine. Maybe one workaround could be to prefix the numbers with a letter in R-land so that emacs interprets them as strings. Cheers, Poking around the web I found that emacs guarantees 28 bits as a minimum precision. For a while I did as you suggested, prefixing all subjects ids with 'id'. But it's cleaner if I reinstall a 64-bit version of Arch. From your experiments, I already know its 64-bit version of emacs supports at least 30 bits. Thanks again for your help. Dan
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
Eric Schulte eric.schu...@gmx.com writes: A further followup, at the risk of descending into minutia. The culprit seems to be the emacs function string-to-number. On my 32-bit Arch machine: (string-to-number 123456789): 123456789 (#o726746425, #x75bcd15) (string-to-number 987654321): 987654321.0 Since that latter number is not representable as an integer in Emacs (it loses 4 bit due to the way Lisp represents these things), it takes the next best thing (FP in that case). Since you are on Arch, I believe there is a way to compile Emacs to use large integers even on 32bit systems - it may have other problems, but should solve this. Regards, Achim. -- +[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]+ Wavetables for the Waldorf Blofeld: http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
Daniel Drake silopho...@gmail.com writes:
Hi All,
I'm using R in org-mode/babel to analyze data from a psychological
study. The subjects in this study are identified by nine digit integers
(e.g., 987654321) that I treat as strings (or factors) in my R data
frames.
Tables output by an R code block that contain these subject IDs are
not formatted properly: the subject IDs seem to be treated as numbers
and a decimal point and a trailing zero are appended. For example,
what should be
| subj.id |
|---|
| 987654321 |
becomes
| subj.id |
| 987654321.0 |
(I've included real, self-contained code below.)
When I write the data frames directly to a file from within the R code
block (using a write.table function call that mimics the one in
ob-R.el), the integer IDs are preserved; but when code from ob-R.el
writes them out, the values get reformated as floats.
I've noticed that eight digit integers are not modified in the same
way, which makes me wonder if there is a 'digits' threshold I could
modify to prevent this from happening.
I've pretty much ruled out the possibility that this transformation
occurs in R; however I'm not proficient enough in elisp to follow the
operations that happen after the data frame is written to a file in
the org-babel-R-write-object-command.
Any pointers to help me figure this out would be very appreciated!
(I've seen this thread:
http://comments.gmane.org/gmane.emacs.orgmode/31373, but do not know
if the conversion to calc has been made already or if it is the source
of the problem.)
Hi Dan,
When I launch Emacs without any personal configuration (emacs -Q),
evaluate ob-R.el to add R support, and then execute your code blocks I
get the following behavior.
* Test
** table as generated by org-mode/babel
#+name: make
#+begin_src R :results value :colnames yes
temp - data.frame('A'=c('987654321'),'B'=c('98765432'))
## -- this call mimics the one in ob-R.el
write.table(temp,file='test_r.tsv',sep='\t',na='nil',row.names=FALSE,col.names=TRUE,quote=FALSE)
temp
#+end_src
#+RESULTS: make
| A |B |
|---+--|
| 987654321 | 98765432 |
** table as generated by R directly
#+name: read
#+begin_src sh :results output
cat test_r.tsv
#+end_src
#+RESULTS: read
: A B
: 987654321 98765432
I would recommend installing the latest version of Org-mode from source
as that tends to fix many problems, and will at least ensure that we are
both working from the same code base moving forward. See
http://orgmode.org/worg/org-faq.html#Keeping-current
Best,
--
Eric Schulte
http://cs.unm.edu/~eschulte/
Re: [O] [0][babel][R] Undesired conversion of integers to floats in R code block output
On 02/18/2012 08:23 AM, Eric Schulte wrote: Daniel Drakesilopho...@gmail.com writes: Hi All, I'm using R in org-mode/babel to analyze data from a psychological study. The subjects in this study are identified by nine digit integers (e.g., 987654321) that I treat as strings (or factors) in my R data frames. Tables output by an R code block that contain these subject IDs are not formatted properly: the subject IDs seem to be treated as numbers and a decimal point and a trailing zero are appended. For example, what should be | subj.id | |---| | 987654321 | becomes | subj.id | | 987654321.0 | (I've included real, self-contained code below.) When I write the data frames directly to a file from within the R code block (using a write.table function call that mimics the one in ob-R.el), the integer IDs are preserved; but when code from ob-R.el writes them out, the values get reformated as floats. I've noticed that eight digit integers are not modified in the same way, which makes me wonder if there is a 'digits' threshold I could modify to prevent this from happening. I've pretty much ruled out the possibility that this transformation occurs in R; however I'm not proficient enough in elisp to follow the operations that happen after the data frame is written to a file in the org-babel-R-write-object-command. Any pointers to help me figure this out would be very appreciated! (I've seen this thread: http://comments.gmane.org/gmane.emacs.orgmode/31373, but do not know if the conversion to calc has been made already or if it is the source of the problem.) Hi Dan, When I launch Emacs without any personal configuration (emacs -Q), evaluate ob-R.el to add R support, and then execute your code blocks I get the following behavior. I would recommend installing the latest version of Org-mode from source as that tends to fix many problems, and will at least ensure that we are both working from the same code base moving forward. See http://orgmode.org/worg/org-faq.html#Keeping-current Best, Hi Eric, First of all, thanks for your reply. Just to be safe, I nuked my org-mode directory and re-cloned from the git repository: I'm now at release_7.8.03.386.g2239d (I was at release_7.8.03-351-g47eb3 previously). Is there a more recent version? I also removed the org directory that came with the packaged version of emacs, just in case. I start emacs with the following options: $ emacs -Q -eval '(setq load-path (cons ~/share/emacs/site-lisp/org-mode/lisp load-path))' -eval (require 'ob-R) test.org I get the same behavior as before. I was a little puzzled when I saw this happen, as I've been working on this study for over a year in org and it's the first time I've seen this behavior. I went back and tried previous versions (e.g., 7.4), and now I see the same thing. I wonder if Arch (a rolling release linux distribution that tries to be close to the leading edge) has updated an underlying library that impacts the babel code in some odd way. (Granted, I don't see how this could be, as the emacs environment seems pretty well insulated from the underlying OS; PEBKAC just as likely.) In any event, since you can't reproduce the behavior (thanks again for trying), I don't expect it's anything you can fix. Maybe it will show up in other distributions as they update to more current versions. In the mean time, I'll try to improve my elisp skills and figure out where the extraneous formatting occurs. Best, Dan - GNU Emacs 23.4.1 (i686-pc-linux-gnu, GTK+ Version 2.24.9) of 2012-02-01 on shirley.hoetzel.info - Org-mode version 7.8.03 (release_7.8.03.386.g2239d)
[O] [0][babel][R] Undesired conversion of integers to floats in R code block output
Hi All,
I'm using R in org-mode/babel to analyze data from a psychological
study. The subjects in this study are identified by nine digit integers
(e.g., 987654321) that I treat as strings (or factors) in my R data
frames.
Tables output by an R code block that contain these subject IDs are
not formatted properly: the subject IDs seem to be treated as numbers
and a decimal point and a trailing zero are appended. For example,
what should be
| subj.id |
|---|
| 987654321 |
becomes
| subj.id |
| 987654321.0 |
(I've included real, self-contained code below.)
When I write the data frames directly to a file from within the R code
block (using a write.table function call that mimics the one in
ob-R.el), the integer IDs are preserved; but when code from ob-R.el
writes them out, the values get reformated as floats.
I've noticed that eight digit integers are not modified in the same
way, which makes me wonder if there is a 'digits' threshold I could
modify to prevent this from happening.
I've pretty much ruled out the possibility that this transformation
occurs in R; however I'm not proficient enough in elisp to follow the
operations that happen after the data frame is written to a file in
the org-babel-R-write-object-command.
Any pointers to help me figure this out would be very appreciated!
(I've seen this thread:
http://comments.gmane.org/gmane.emacs.orgmode/31373, but do not know
if the conversion to calc has been made already or if it is the source
of the problem.)
Thanks,
Dan
I'm running:
- Arch Linux (32 bit)
- GNU Emacs 23.4.1
- Org-mode version release_7.8.03-351-g47eb3
- R version 2.14.1 (2011-12-22)
* Test
** table as generated by org-mode/babel
#+name: make
#+begin_src R :results value :colnames yes
temp - data.frame('A'=c('987654321'),'B'=c('98765432'))
## -- this call mimics the one in ob-R.el
write.table(temp,file='test_r.tsv',sep='\t',na='nil',row.names=FALSE,col.names=TRUE,quote=FALSE)
temp
#+end_src
#+RESULTS: make
| A |B |
|-+--|
| 987654321.0 | 98765432 |
** table as generated by R directly
#+name: read
#+begin_src sh :results output
cat test_r.tsv
#+end_src
#+RESULTS: read
: A B
: 987654321 98765432
