From: "Gilpin, Scott" <[EMAIL PROTECTED]>
>-----Original Message-----
>> From: [EMAIL PROTECTED] [mailto:r-help-
>> [EMAIL PROTECTED] On Behalf Of Søren Højsgaard
>> Sent: Friday, August 27, 2004 11:46 AM
>> To: [EMAIL PROTECTED]
>> Cc: Søren Højsgaard
>> Subject: [R] Reading SAS data into R
>>
>> Dear all,
>> One of my students (whom I am trying to convince to use R) wants to get a
>> fairly large SAS dataset into R (about 150mB). An obvious and simple thing
>> she tried was to write the dataset as a .csv-file and then read that into
>> R, but that takes forever (or something close to that..). The dataset is
>> so large, that exporting it as an Excel file from SAS is not feasible
>> (more than 65000 lines). I am reluctant to ask her to go through all the
>> data base steps (then she'll just stick to SAS...). Can anyone help me out
>> on that one?
What platform are you on, and how much memory do you have? 150mb isn't *that* large - but it will depend on your system. See the FAQs regarding memory issues, as well as ?mem.limits and ?gc
It also depends on the type of data you're working with (integers take half the space of numerics) and what type of analysis you want to do. The CSV approach should work fine - but you'll want to use scan instead of read.table. You can use scan to read data in chunks (using skip and nlines), do something useful with this chunk of data, run gc(), and then read in another chunk.
In general, users I've seen who try to go from SAS to R don't seem to realize that R is not a data manipulation language, and hence just try to shove their entire dataset into R and manipulate it (which is what they would do in SAS). Perl, awk, cut, etc. (not to mention DBMSes) are all very useful for processing data before putting it into R.
-----------End quoted text
R is also a data manipulation language and once you get used to it it is better than SAS at data manipulation, for non-huge datasets. We have many examples of data manipulation with R on our web site http://biostat.mc.vanderbilt.edu (see especially the Alzola and Harrell text).
What we do for importing SAS datasets is described at http://biostat.mc.vanderbilt.edu/twiki/bin/view/Main/SASexportHowto . This will preserve labels and value labels, handle dates, times, date/times, and get around problems we've faced in importing SAS V5 transport files using the foreign package, by having SAS run PROC EXPORT to create csv files.
--
Frank E Harrell Jr Professor and Chair School of Medicine
Department of Biostatistics Vanderbilt University______________________________________________ [EMAIL PROTECTED] mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide! http://www.R-project.org/posting-guide.html
