[R] Limitations of R
Dear all, I have a few binary files like 9Gb or even of 50Gb. I would like to ask you what are the known limits of the R for the data processing part, I have a system with a lot of RAM (500Gb) but I am not sure about the internal limitations of the R. How long for example a vector can be? Could you please inform me for these internal R limitations? I would like to thank you in advance for your help B.R Alex [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Limitations of R
On 06.10.2011 16:49, Alaios wrote: Dear all, I have a few binary files like 9Gb or even of 50Gb. I would like to ask you what are the known limits of the R for the data processing part, I have a system with a lot of RAM (500Gb) Really accessible from one core? Amazing. but I am not sure about the internal limitations of the R. How long for example a vector can be? See ?Memory-limits Uwe Ligges Could you please inform me for these internal R limitations? I would like to thank you in advance for your help B.R Alex [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Limitations of R
On 10/06/2011 08:33 AM, Uwe Ligges wrote: On 06.10.2011 16:49, Alaios wrote: Dear all, I have a few binary files like 9Gb or even of 50Gb. Hi Alaios -- Maybe you have a particular domain you are interested in (e.g., high-throughput sequence analysis) and there are packages (e.g., at http://bioconductor.org) that make it easier to work with this size and format of data. Martin I would like to ask you what are the known limits of the R for the data processing part, I have a system with a lot of RAM (500Gb) Really accessible from one core? Amazing. but I am not sure about the internal limitations of the R. How long for example a vector can be? See ?Memory-limits Uwe Ligges Could you please inform me for these internal R limitations? I would like to thank you in advance for your help B.R Alex [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Computational Biology Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109 Location: M1-B861 Telephone: 206 667-2793 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Limitations on R code file
Thanks, Jorge. Yes, I can also run the code. But I would like to know the limitations on the lengths of variable names, formulas/equations, and files. Steve has pointed out a limitation on the length of a variable name is 256kb. So abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz0123456789012345678901 - 1 is OK and abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz0123456789012345678901 + abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz0123456789012345678902 OK too as long as both are claimed. Any ideas on lengths of formulas/equations? -james Hi James, I ran all your code on a fresh R session (64 bit, OS X 10.6.0) and got no errors. I suspect what Steve suggests is true: your computer is running out of memory. Have you tried running it on a 64 bit machine? Here is my sessionInfo(): R version 2.11.0 (2010-04-22) x86_64-apple-darwin9.8.0 locale: [1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] grid_2.11.0lattice_0.18-5 tools_2.11.0 HTH, Jorge On Wed, May 5, 2010 at 5:28 PM, wrote: I would like to run R code from a file that is generated from other source. The file consists of some variables and formulas/equations such as Variables = log(100) ##Other variable definitions VariablesWithLongNames = 2*3*log(12345678)+10^4 + Variables Equations=log(VariablesWithLongNames)+Variables^2 ##Other equations, formulas Result = Equations^2+rnorm(1) hist(Result,breaks=100,freq=FALSE) lines(density(Result),col = blue) The file may be very long and some variable names may be very long too. We know that objects 100Mb can cause R to run out of memory. Not sure if there are any other limitations on variable length and source file length. So my question is what are the limitations of variable name length and the source file length in R? Could you please drop a few lines or point me know where I can find them? Thanks, -james __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Limitations on R code file
I would like to run R code from a file that is generated from other source. The file consists of some variables and formulas/equations such as Variables = log(100) ##Other variable definitions VariablesWithLongNames = 2*3*log(12345678)+10^4 + Variables Equations=log(VariablesWithLongNames)+Variables^2 ##Other equations, formulas Result = Equations^2+rnorm(1) hist(Result,breaks=100,freq=FALSE) lines(density(Result),col = blue) The file may be very long and some variable names may be very long too. We know that objects 100Mb can cause R to run out of memory. Not sure if there are any other limitations on variable length and source file length. So my question is what are the limitations of variable name length and the source file length in R? Could you please drop a few lines or point me know where I can find them? Thanks, -james __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Limitations on R code file
Hi James, On Wed, May 5, 2010 at 5:28 PM, g...@ucalgary.ca wrote: I would like to run R code from a file that is generated from other source. The file consists of some variables and formulas/equations such as Variables = log(100) ##Other variable definitions VariablesWithLongNames = 2*3*log(12345678)+10^4 + Variables Equations=log(VariablesWithLongNames)+Variables^2 ##Other equations, formulas Result = Equations^2+rnorm(1) hist(Result,breaks=100,freq=FALSE) lines(density(Result),col = blue) The file may be very long and some variable names may be very long too. We know that objects 100Mb can cause R to run out of memory. Sorry, I don't follow. I routinely work with objects in my workspace that are much larger than 100MB, so I don't think I understand what you mean here. Is R running out of memory, or is your computer running out of memory? If it's the former, are you running 32bit or 64bit R? Not sure if there are any other limitations on variable length and source file length. So my question is what are the limitations of variable name length and the source file length in R? R - 10 Error: variable names are limited to 256 bytes I don't have an answer about source file size, but I wouldn't imagine there's a limit that any reasonably sized file would break. -- Steve Lianoglou Graduate Student: Computational Systems Biology | Memorial Sloan-Kettering Cancer Center | Weill Medical College of Cornell University Contact Info: http://cbio.mskcc.org/~lianos/contact __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Limitations on R code file
Hi James, I ran all your code on a fresh R session (64 bit, OS X 10.6.0) and got no errors. I suspect what Steve suggests is true: your computer is running out of memory. Have you tried running it on a 64 bit machine? Here is my sessionInfo(): R version 2.11.0 (2010-04-22) x86_64-apple-darwin9.8.0 locale: [1] en_US.UTF-8/en_US.UTF-8/C/C/en_US.UTF-8/en_US.UTF-8 attached base packages: [1] stats graphics grDevices utils datasets methods base loaded via a namespace (and not attached): [1] grid_2.11.0lattice_0.18-5 tools_2.11.0 HTH, Jorge On Wed, May 5, 2010 at 5:28 PM, wrote: I would like to run R code from a file that is generated from other source. The file consists of some variables and formulas/equations such as Variables = log(100) ##Other variable definitions VariablesWithLongNames = 2*3*log(12345678)+10^4 + Variables Equations=log(VariablesWithLongNames)+Variables^2 ##Other equations, formulas Result = Equations^2+rnorm(1) hist(Result,breaks=100,freq=FALSE) lines(density(Result),col = blue) The file may be very long and some variable names may be very long too. We know that objects 100Mb can cause R to run out of memory. Not sure if there are any other limitations on variable length and source file length. So my question is what are the limitations of variable name length and the source file length in R? Could you please drop a few lines or point me know where I can find them? Thanks, -james __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
