Thanks for all the input. Now to go further off topic.. Does anyone have any comments regarding running 64 BIT R on cluster/grid systems? Given an (almost) unlimited amount of memory, can R hypotheticaly handle Very Large Datasets?
I'm finding that even small sub sets of this data come in at 1 GB (1-5 million rows), which no R 32 BIT workstation (at least in this lab) can handle. This type of stuff is done effortlessly in genomic research, mapping DNA, etc.... Tom Colson Center for Earth Observation North Carolina State University Raleigh, NC 27695 (919) 515 3434 (919) 673 8023 [EMAIL PROTECTED] Online Calendar: http://www4.ncsu.edu/~tpcolson -----Original Message----- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Graham Jones Sent: Wednesday, February 16, 2005 5:08 AM To: Prof Brian Ripley Cc: r-help@stat.math.ethz.ch Subject: Re: Off topic -- large data sets. Was RE: [R] 64 Bit R BackgroundQuestion In message <[EMAIL PROTECTED]>, Prof Brian Ripley <[EMAIL PROTECTED]> writes >But Bert's caveats apply: you have 200 problems of size 20,000 since in >QDA each class's distribution is estimated separately, and a single >pass will give you the sufficient statistics however large the dataset is. > I think we've interpreted Bert's question differently. I am not saying I need to have vast amounts of data in RAM, or in a single data structure, or anything like that, and I am not saying I need a 64-bit version of R. What I am saying is that if I had 40 million cases for a problem like the one I described, I'd want to use all of them when designing a classifier. Patrick Burns, if you're reading: OCR = optical character recognition. -- Graham Jones, author of SharpEye Music Reader http://www.visiv.co.uk 21e Balnakeil, Durness, Lairg, Sutherland, IV27 4PT, Scotland, UK ______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
______________________________________________ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
