Re: [R] Error: cannot allocate vector of size X.0 Mb
Thank you all for the suggestions. We do intend to get more RAM space. Meanwhile I shall have a look at the ShortRead package features. -- View this message in context: http://r.789695.n4.nabble.com/Error-cannot-allocate-vector-of-size-X-0-Mb-tp2539031p2540518.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: cannot allocate vector of size X.0 Mb
On Sep 14, 2010, at 9:09 AM, John1983 wrote: Hi, I am working with a file (900MB in size) that has around 10 million records (in particular FASTQ records). I am able to read in the file as an object of BStringSet. When I start to manipulate the data, after almost 4 hours, I get the error message as Error: cannot allocate vector of size X.0 Mb (where X was once 160MB and then 180MB). The R version used is 2.11.1. I am not sure how to check if this is 64-bit or not so that I can use 4GB of RAM. I typed 'version' at the R prompt and see this: platform x86_64-unknown-linux-gnu arch x86_64 os linux-gnu system x86_64, linux-gnu. Does this mean it is already a 64-bit version of R I use? Please advice. In an R session, see what the result of: .Machine$sizeof.pointer gets you. If it returns 4, you are using 32 bit R and you will need to install 64 bit R. If it returns 8, you are using 64 bit R. The information above only tells us/you that you are running a 64 bit OS. HTH, Marc Schwartz __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: cannot allocate vector of size X.0 Mb
Yes I see. So I typed as you mentioned and I get an 8 (therefore this is a 64-bit R). Is there anything else I need to check to remove this error? -- View this message in context: http://r.789695.n4.nabble.com/Error-cannot-allocate-vector-of-size-X-0-Mb-tp2539031p2539078.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: cannot allocate vector of size X.0 Mb
On 14.09.2010 16:47, John1983 wrote: Yes I see. So I typed as you mentioned and I get an 8 (therefore this is a 64-bit R). Is there anything else I need to check to remove this error? Yes: If the amount of RAM in your machine is sufficient Best, Uwe __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: cannot allocate vector of size X.0 Mb
On Sep 14, 2010, at 9:47 AM, John1983 wrote: Yes I see. So I typed as you mentioned and I get an 8 (therefore this is a 64-bit R). Is there anything else I need to check to remove this error? 1. Add more RAM. 2. Depending upon what you are doing relative to data management/analysis, you might want to see if there are more efficient approaches. 3. Free up RAM that may be in use by other processes, etc. 4. Look at R packages such as biglm, ff/bit, biganalytics, etc. 5. Consider using a backend data base application (eg. MySQL, Postgres, etc.) to manage the data sets and use R only for analyses, as may be appropriate relative to splitting the tasks. You did not indicate just how much RAM you have on the machine in question, but if it is within your budget/capability, adding more RAM would be the most transparent approach. HTH, Marc Schwartz __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error: cannot allocate vector of size X.0 Mb
On 09/14/2010 08:02 AM, Marc Schwartz wrote: On Sep 14, 2010, at 9:47 AM, John1983 wrote: Yes I see. So I typed as you mentioned and I get an 8 (therefore this is a 64-bit R). Is there anything else I need to check to remove this error? 1. Add more RAM. 2. Depending upon what you are doing relative to data management/analysis, you might want to see if there are more efficient approaches. 3. Free up RAM that may be in use by other processes, etc. 4. Look at R packages such as biglm, ff/bit, biganalytics, etc. 5. Consider using a backend data base application (eg. MySQL, Postgres, etc.) to manage the data sets and use R only for analyses, as may be appropriate relative to splitting the tasks. You did not indicate just how much RAM you have on the machine in question, but if it is within your budget/capability, adding more RAM would be the most transparent approach. Before spending the big bucks you might try asking on the Bioconductor mailing list http://bioconductor.org/help/mailing-list/ home of the Biostrings package and BStringSet, andwhere for instance the ShortRead package readFastq would be a natural way of parsing your FASTQ file, and most operations you'd likely perform on a fastq file (given sufficient memory) would take more than a few (tens of, at the most) seconds. Martin HTH, Marc Schwartz __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
