[R] Row limit for read.table
I have been trying to read in a large data set using read.table, but I've only been able to grab the first 50,871 rows of the total 122,269 rows. f - read.table(http://www.cs.odu.edu/~fmccown/R/Tchange_rates_crawled.dat;, header=TRUE, nrows=123000, comment.char=, sep=\t) length(f$change_rate) [1] 50871 From searching the email archives, I believe this is due to size limits of a data frame. So... 1) Why doesn't read.table give a proper warning when it doesn't place every read item into a data frame? 2) Why isn't there a parameter to read.table that allows the user to specify which columns s/he is interested in? This functionality would allow extraneous columns to be ignored which would improve memory usage. I've already made a work-around by loading the table into mysql and doing a select on the 2 columns I need. I just wonder why the above 2 points aren't implemented. Maybe they are and I'm totally missing it. Thanks, Frank -- Frank McCown Old Dominion University http://www.cs.odu.edu/~fmccown/ __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Row limit for read.table
In your case, read.table behaves as documented. The ' - character is one of the standard quoting characters. Some (but very few) of the entrys contain single ' chars, so sometimes more than ten thousand lines are just treated as a single entry. Try using quote= to disable quoting, as documented on the help page: f-read.table(http://www.cs.odu.edu/~fmccown/R/Tchange_rates_crawled.dat;, header=TRUE, nrows=123000, comment.char=, sep=\t,quote=) length(f$change_rate) [1] 122271 So either adding quote= works or removing sep=\t (and not using quote) works. It seems an odd side-effect that specifying the separator changes the default behavior of quoting (because of the ' character). I don't see that association made in the help file. There is (colClasses, works as documented). Try f-read.table(http://www.cs.odu.edu/~fmccown/R/Tchange_rates_crawled.dat;, + header=TRUE, nrows=123000, comment.char=, sep=\t,quote=,colClasses=c(character,NULL,NULL,NULL,NULL)) dim(f) [1] 122271 1 Did you read the help page? Of course I did. For me the definition of colClasses wasn't clear... A vector of classes to be assumed for the columns didn't seem to be the same thing as the columns you would like to be read. I may have made the association if the help page had contained a simple example of using colClasses. Thanks for the help, Frank -- Frank McCown Old Dominion University http://www.cs.odu.edu/~fmccown/ __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Install RMySQL with R 2.4.0
Joe Byers wrote: All, After staring at this error message for an hour or so yesterday and this morning. I decided to try something else. Low and behold trying to build the package in cygwin causes R to try and build under linux/unix not windows. I went to the command prompt and was able to build the package. Download the RMySQL...tar.gz file and unzip somewhere like drive:/projects Several notes 1. make sure you have mysql directories on your computer somewhere with the subdirs of include, bin, and lib. You can just copy these from you actual server unless you want to install them. I used d:/mysql/... 2. Modify configure.win in RMySQL and Makevars.win ins RMySQL/src to have the mysql directories from (1) 3. Copy and paste this script to a batch file and execute ** Rem build without --docs=normal tries to build chm help on windows this bombs Rem if a zip program not installed the zip file will not be built Rem go find the temp directory where R built the package and copy to ../R/library Rem temp directory will look something like C:\Temp\Rinst32098657\RMySQL Rem if R bin directory in the path this will run otherwise add the drive:\Dir1\R\bin to the command Rcmd build --binary \projects\RMySQL --docs=normal *** 4. Note that I have --docs=normal in the command line. This is needed to get the package built. Windows packages now default to chm files and RMySQL does not have any windows chm help files. All txt, html, and latex help are built with this option. 5. I am not sure where the RMySQL...zip file is stored, I think in ...R\Bin. I just copied the files from the temp\RinstXX\to the ...\R\library to install. This may or may not work for you, it did for me. I will try and update my website www.cba.utulsa.edu/byersj Research and Analytics section to include a link to the RMySQL zip file for others to download. Good Luck Joe Joe, Thanks for telling us how you got RMySQL installed. Would you mind posting the dll files so the rest of us wouldn't have to recompile anything? Thanks, Frank __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Install RMySQL with R 2.4.0
I noticed that RMySQL is now at version 0.5-10, but the libMySQL.dll and RMySQL.dll files that are required for Windows are missing from the distribution. These files were present in previous versions of RMySQL, and the installation instructions still state they are needed. I have an older version of RMySQL working with R-2.3.0 on Windows XP, but when I try to run RMySQL_0.5-10 with R-2.4.0 I get the following error message: library (RMySQL); Error in library(RMySQL) : 'RMySQL' is not a valid package -- installed 2.0.0? This is after updating the DBI package and installing RMySQL on R-2.4.0. Of course I wouldn't expect RMySQL_0.5-10 to work since the DLLs have not been included in the distribution. Does anyone have an updated email address for David A. James, the maintainer for RMySQL? The address included in the distribution appears to be no longer valid. Thanks, Frank -- Frank McCown Old Dominion University http://www.cs.odu.edu/~fmccown/ __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Adding to user-contributed documentation
I was wondering if anyone knows who should be contacted to add to the R user-contributed documentation at http://www.r-project.org/other-docs.html There doesn't appear to be any information on the page as to how one would add their new resource. I'd like to add a guide I created called Producing Simple Graphs with R http://www.cs.odu.edu/~fmccown/R/ I plan on adding additional graphs (like scatterplots) as time permits. Thanks, Frank __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Turn off border on barplot
I was wondering if there was any way to completely remove the borders around each bar when using the barplot function. I understand how to change the color of the border (border=white), but I need the border to be invisible so I can plot many skinny bars right next to each other with only the bar's color visible. Thanks, Frank -- Frank McCown Old Dominion University http://www.cs.odu.edu/~fmccown/ __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Turn off border on barplot
barplot(1:10, border=NA) That's it! Thanks a ton, guys. Frank __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Install RMySQL with R 2.4.0
I just installed RMySQL 0.5-9 with R 2.4.0 on Windows XP and got the following error message when trying to run a script with RMySQL: Error in library(RMySQL) : 'RMySQL' is not a valid package -- installed 2.0.0? Any ideas? Thanks, Frank __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Misunderstanding escape (backslash)
Forgive my ignorance, but shouldn't '\\' be converted into '\' in my string? In my output (below), you can see that '\\' remains '\\'. term = mother\'s day term [1] mother's day term = mother\\\'s day term [1] mother\\'s day --- should be mother\'s day Thanks, Frank __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Misunderstanding escape (backslash)
And check out the new FAQ 7.37 http://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-does-backslash-behave-strangely-inside-strings_003f Thanks, guys, for the help. Frank __ R-help@stat.math.ethz.ch mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
