On Thu, 29 Jan 2004, Yun-Fang Juan wrote: > > Here is the exact error I got > ---------------------- > Read 73 items > Error: cannot allocate vector of size 1953 Kb > Execution halted > ----------------------- > I am running R on Freebsd 4.3 > with double CPU and 2 GB memory > Is that sufficient?
Clearly not. What is the structure of your `attributes'? As Andy Liaw said, the design matrix may be bigger than that if there are factors involved. (And you need several copies of the design matrix.) I would try a 10% sample of the rows to get a measure of what will fit into your memory. I have never seen a regression problem for which 600k cases were needed, and would be interested to know the context. (It is hard to imagine that the cases are from a single homogeneous population and that a linear model fits so well that the random error is not dominated by systematic error.) > > Yun-Fang > ----- Original Message ----- > From: "Yun-Fang Juan" <[EMAIL PROTECTED]> > To: <[EMAIL PROTECTED]> > Sent: Thursday, January 29, 2004 7:03 PM > Subject: [R] memory problem for R > > > > Hi, > > I try to use lm to fit a linear model with 600k rows and 70 attributes. > > But I can't even load the data into the R environment. > > The error message says the vector memory is used up. > > > > Is there anyone having experience with large datasets in R? (I bet) > > > > Please advise. > > > > > > thanks, > > > > > > Yun-Fang > > > > [[alternative HTML version deleted]] > > > > ______________________________________________ > > [EMAIL PROTECTED] mailing list > > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > > PLEASE do read the posting guide! > http://www.R-project.org/posting-guide.html > > > > > > ______________________________________________ > [EMAIL PROTECTED] mailing list > https://www.stat.math.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html > > -- Brian D. Ripley, [EMAIL PROTECTED] Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UK Fax: +44 1865 272595 ______________________________________________ [EMAIL PROTECTED] mailing list https://www.stat.math.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
