[R] lm-step: use train-data model to validate test-data
I have model-data named as: model that is split as model.T(train) and model.V(test or validation). The least square model (from lm to step) is built withmodel.T and I like to see how model.T is robust by comparing predicted model.V toactual model.V. How do I get score for model.V based on model built on model.T? The code highlighted below does not get what I expected.Please advise! Thanks! # score the model score.T <- data.frame(predict(step, model.T)) # get predicted score for train data score.V <- data.frame(predict(step, model.V)) # for test data but seems incorrect # get the actual values actual.T <- data.frame(model.T$sales) actual.V <- data.frame(model.V$sales) # comparison for model.T comp.T=cbind(actual.T,round(score.T,digit=2)) plot(comp.T) # comparison for model.V (use Model.T to predict Model.V for true validation comp.V=cbind(actual.V,round(score.V,digit=2)) plot(comp.V) [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] MACRO-LOOP in R
I am doing the data transpose with rename as shown below (step1 ~ step4) 1. Is any way in R similar to PROC TRANSPOSE used in SAS?2. How to use MACRO-LOOP to simplify the following procedure? THANK YOU FOR HELPS! # create data for test x<-data.frame( a=c(1,2,3), b=c("1","2","3")); x; str(x)# step1: parse out to 3 tabs x1<-x[x$a == 1,]; x1 x2<-x[x$a == 2,]; x2 x3<-x[x$a == 3,]; x3# step2: remove column a in each tab x1$a<-NULL; x1 x2$a<-NULL; x2 x3$a<-NULL; x3# step3: rename column b to b1, b2 and b3 by y1, y2 and y3 names(x1)[names(x1)=="b"]<-"b_1"; x1 names(x2)[names(x2)=="b"]<-"b_2"; x2 names(x3)[names(x3)=="b"]<-"b_3"; x3# setp4: set x1, x3 and x3 together x123=cbind(x1,x2,x3); x123 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Create macro_var in R
One more question (see below). I cannot use macro-var, mvar, for creating new name, as shown below. Any advice is highly appreciated! > mvar<-"pop" > new.pop<-tab[[mvar]]; new.pop [1] 698 1214 1003 1167 2549 824 944 1937 935 570 0 > new.tab[[mvar]]<-d$pop; Error in new.tab[[mvar]] <- d$pop : object 'new.tab' not found On Thursday, February 4, 2016 11:02 AM, Amoy Yang <amo...@yahoo.com> wrote: This works although it looks rare by using min(",key,"). Don't know why but just have to remember it. This is a tough part in R. Thanks for helps! Amoy On Wednesday, February 3, 2016 5:25 PM, Gabor Grothendieck <ggrothendi...@gmail.com> wrote: See Example 5. Insert Variables on the sqldf home page. https://github.com/ggrothendieck/sqldf On Wed, Feb 3, 2016 at 2:16 PM, Amoy Yang via R-help <r-help@r-project.org> wrote: > First, MVAR<-c("population) should be the same as "population'". Correct? > You use tab[[MVAR]] to refer to "population" where double [[...]] removes > double quotes "...", which seemingly work for r-code although it is tedious > in comparison direct application in SAS %let MVAR=population. But it does not > work for sqldef in R as shown below. > >> key<-"pop" >> library(sqldf) >> sqldf("select grade, count(*) as cnt, min(tab[[key]]) as min, > + max(pop) as max, avg(pop) as mean, median(pop) as median, > + stdev(pop) as stdev from tab group by grade") > Error in sqliteSendQuery(con, statement, bind.data) : > error in statement: near "[[key]": syntax error > > > > > On Wednesday, February 3, 2016 12:40 PM, "ruipbarra...@sapo.pt" ><ruipbarra...@sapo.pt> wrote: > > > Hello, > > You can't use tab$MVAR but you can use tab[[MVAR]] if you do MVAR <- > "population" (no need for c()). > > Hope this helps, > > Rui Barradas > Citando Amoy Yang via R-help <r-help@r-project.org>: > population is the field-name in data-file (say, tab). MVAR<-population takes > data (in the column of population) rather than field-name as done in SAS: > %let MVAR=population; > In the following r-program, for instance, I cannot use ... tab$MVAR...or > simply MVAR itself since MVAR is defined as "population" with double quotes > if using MVAR<-c("population") > > On Wednesday, February 3, 2016 11:54 AM, Duncan Murdoch ><murdoch.dun...@gmail.com> wrote: > > > On 03/02/2016 12:41 PM, Amoy Yang via R-help wrote: > There is a %LET statement in SAS: %let MVAR=population; Thus, MVAR can be >used through entire program. > In R, I tried MAVR<-c("population"). The problem is that MAVR comes with > double quote "" that I don't need. But MVAR<-c(population) did NOT work > out. Any way that double quote can be removed as done in SAS when creating > macro_var? > Thanks in advance for helps! > R doesn't have a macro language, and you usually don't need one. > > If you are only reading the value of population, then > > MAVR <- population > > is fine. This is sometimes the same as c(population), but in general > it's different: c() will remove some attributes, such as > the dimensions on arrays. > > If you need to modify it in your program, it's likely more complicated. > The normal way to go would be to put your code in a function, and have > it return the modified version. For example, > > population <- doModifications(population) > > where doModifications is a function with a definition like > > doModifications <- function(MAVR) { > # do all your calculations on MAVR > # then return it at the end using > MAVR > } > > Duncan Murdoch > > > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.r-project.org/posting-guide.htmlandprovide commented, minimal, > self-contained, reproducible code. > > > > > [[alternative HTML version deleted]] > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. -- Statistics & Software Consulting GKX Group, GKX Associates Inc. tel: 1-877-GKX-GROUP email: ggrothendieck at gmail.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Create macro_var in R
There is a %LET statement in SAS: %let MVAR=population; Thus, MVAR can be used through entire program. In R, I tried MAVR<-c("population"). The problem is that MAVR comes with double quote "" that I don't need. But MVAR<-c(population) did NOT work out. Any way that double quote can be removed as done in SAS when creating macro_var? Thanks in advance for helps! Amoy [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Create macro_var in R
First, MVAR<-c("population) should be the same as "population'". Correct? You use tab[[MVAR]] to refer to "population" where double [[...]] removes double quotes "...", which seemingly work for r-code although it is tedious in comparison direct application in SAS %let MVAR=population. But it does not work for sqldef in R as shown below. > key<-"pop" > library(sqldf) > sqldf("select grade, count(*) as cnt, min(tab[[key]]) as min, + max(pop) as max, avg(pop) as mean, median(pop) as median, + stdev(pop) as stdev from tab group by grade") Error in sqliteSendQuery(con, statement, bind.data) : error in statement: near "[[key]": syntax error On Wednesday, February 3, 2016 12:40 PM, "ruipbarra...@sapo.pt" <ruipbarra...@sapo.pt> wrote: Hello, You can't use tab$MVAR but you can use tab[[MVAR]] if you do MVAR <- "population" (no need for c()). Hope this helps, Rui Barradas Citando Amoy Yang via R-help <r-help@r-project.org>: population is the field-name in data-file (say, tab). MVAR<-population takes data (in the column of population) rather than field-name as done in SAS: %let MVAR=population; In the following r-program, for instance, I cannot use ... tab$MVAR...or simply MVAR itself since MVAR is defined as "population" with double quotes if using MVAR<-c("population") On Wednesday, February 3, 2016 11:54 AM, Duncan Murdoch <murdoch.dun...@gmail.com> wrote: On 03/02/2016 12:41 PM, Amoy Yang via R-help wrote: There is a %LET statement in SAS: %let MVAR=population; Thus, MVAR can be used through entire program. In R, I tried MAVR<-c("population"). The problem is that MAVR comes with double quote "" that I don't need. But MVAR<-c(population) did NOT work out. Any way that double quote can be removed as done in SAS when creating macro_var? Thanks in advance for helps! R doesn't have a macro language, and you usually don't need one. If you are only reading the value of population, then MAVR <- population is fine. This is sometimes the same as c(population), but in general it's different: c() will remove some attributes, such as the dimensions on arrays. If you need to modify it in your program, it's likely more complicated. The normal way to go would be to put your code in a function, and have it return the modified version. For example, population <- doModifications(population) where doModifications is a function with a definition like doModifications <- function(MAVR) { # do all your calculations on MAVR # then return it at the end using MAVR } Duncan Murdoch [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Create macro_var in R
population is the field-name in data-file (say, tab). MVAR<-population takes data (in the column of population) rather than field-name as done in SAS: %let MVAR=population; In the following r-program, for instance, I cannot use ... tab$MVAR...or simply MVAR itself since MVAR is defined as "population" with double quotes if using MVAR<-c("population") On Wednesday, February 3, 2016 11:54 AM, Duncan Murdoch <murdoch.dun...@gmail.com> wrote: On 03/02/2016 12:41 PM, Amoy Yang via R-help wrote: > There is a %LET statement in SAS: %let MVAR=population; Thus, MVAR can be >used through entire program. > In R, I tried MAVR<-c("population"). The problem is that MAVR comes with > double quote "" that I don't need. But MVAR<-c(population) did NOT work > out. Any way that double quote can be removed as done in SAS when creating > macro_var? > Thanks in advance for helps! R doesn't have a macro language, and you usually don't need one. If you are only reading the value of population, then MAVR <- population is fine. This is sometimes the same as c(population), but in general it's different: c() will remove some attributes, such as the dimensions on arrays. If you need to modify it in your program, it's likely more complicated. The normal way to go would be to put your code in a function, and have it return the modified version. For example, population <- doModifications(population) where doModifications is a function with a definition like doModifications <- function(MAVR) { # do all your calculations on MAVR # then return it at the end using MAVR } Duncan Murdoch [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Create macro_var in R
Right! the following works to r but not sqldf. MVAR <- "population" tab[[ MVAR ]] sqldf("select tab[[MVAR]] from tab") On Wednesday, February 3, 2016 1:18 PM, Amoy Yang via R-help <r-help@r-project.org> wrote: First, MVAR<-c("population) should be the same as "population'". Correct? You use tab[[MVAR]] to refer to "population" where double [[...]] removes double quotes "...", which seemingly work for r-code although it is tedious in comparison direct application in SAS %let MVAR=population. But it does not work for sqldef in R as shown below. > key<-"pop" > library(sqldf) > sqldf("select grade, count(*) as cnt, min(tab[[key]]) as min, + max(pop) as max, avg(pop) as mean, median(pop) as median, + stdev(pop) as stdev from tab group by grade") Error in sqliteSendQuery(con, statement, bind.data) : error in statement: near "[[key]": syntax error On Wednesday, February 3, 2016 12:40 PM, "ruipbarra...@sapo.pt" <ruipbarra...@sapo.pt> wrote: Hello, You can't use tab$MVAR but you can use tab[[MVAR]] if you do MVAR <- "population" (no need for c()). Hope this helps, Rui Barradas Citando Amoy Yang via R-help <r-help@r-project.org>: population is the field-name in data-file (say, tab). MVAR<-population takes data (in the column of population) rather than field-name as done in SAS: %let MVAR=population; In the following r-program, for instance, I cannot use ... tab$MVAR...or simply MVAR itself since MVAR is defined as "population" with double quotes if using MVAR<-c("population") On Wednesday, February 3, 2016 11:54 AM, Duncan Murdoch <murdoch.dun...@gmail.com> wrote: On 03/02/2016 12:41 PM, Amoy Yang via R-help wrote: There is a %LET statement in SAS: %let MVAR=population; Thus, MVAR can be used through entire program. In R, I tried MAVR<-c("population"). The problem is that MAVR comes with double quote "" that I don't need. But MVAR<-c(population) did NOT work out. Any way that double quote can be removed as done in SAS when creating macro_var? Thanks in advance for helps! R doesn't have a macro language, and you usually don't need one. If you are only reading the value of population, then MAVR <- population is fine. This is sometimes the same as c(population), but in general it's different: c() will remove some attributes, such as the dimensions on arrays. If you need to modify it in your program, it's likely more complicated. The normal way to go would be to put your code in a function, and have it return the modified version. For example, population <- doModifications(population) where doModifications is a function with a definition like doModifications <- function(MAVR) { # do all your calculations on MAVR # then return it at the end using MAVR } Duncan Murdoch [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.r-project.org/posting-guide.htmlandprovide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error opening SHP file
Let me just make this case simple and quick to address what I need. Giving a data-file that includes tow columns: zip5 and population. How do I put pop (colored with different segments) by zips (with boundary) on the USA map with R? Amoy On Friday, January 22, 2016 12:45 PM, David Winsemius <dwinsem...@comcast.net> wrote: > On Jan 22, 2016, at 9:24 AM, Amoy Yang via R-help <r-help@r-project.org> > wrote: > > This is the results that addresses David's advice. >> library(maptools) >> library(maps) >> state.map <- readShapeSpatial("maps/st24_d00.shp") > Error in getinfo.shape(fn) : Error opening SHP file >> # David question: What does list.files('maps') return? Is there a >> 'st24_d00.shp' value in there? >> list.files('maps') > character(0) > I actually use the link below to learn how to load state/zip-data on USA map. That link says: "For example (assumes you have the maryland shapefiles in the map subdirectory):" So it did not purport to tell you how to put shape files in that subdirectory. That statement assumed you understood basic OS path naming conventions in your unstated OS and how to move files around, which is not a topic for rhelp. Try working through the examples in the documents that come with the `sp`-package. -- David. > > # > http://stackoverflow.com/questions/1441717/plotting-color-map-with-zip-codes-in-r-or-python > Plotting color map with zip codes in R or Python > | | > | | | | | | | | > | Plotting color map with zip codes in R or PythonI have some US demographic > and firmographic data. I would like to plot zipcode areas in a state or a > smaller region (e.g. city). Each area would be annotated by col... | > | | > | View on stackoverflow.com | Preview by Yahoo | > | | > | | > > On Friday, January 22, 2016 10:03 AM, Barry Rowlingson ><b.rowling...@lancaster.ac.uk> wrote: > > > We can duplicate the error by giving a path to a non-existent > shapefile, which is probably the original problem: > >> require(maptools) > Loading required package: maptools > Loading required package: sp > Checking rgeos availability: TRUE >> foo=readShapeSpatial("fnord.shp") > Error in getinfo.shape(fn) : Error opening SHP file > > The error message there isn't totally explicit, and might cover a > range of other possibilities such as a corrupted shapefile, or a > missing .shx component of the shapefile or whatever. > > BUT you probably shouldn't be using readShapeSpatial anyway, as it has > a habit of not reading the coordinate system in the .prj file. I find > it much easier to use `raster::shapefile` which *does* read the > coordinate system *and* gives a more explicit error message for a > missing shapefile: > >> require(raster) > Loading required package: raster >> foo=shapefile("fnord.shp") > Error in normalizePath(x, winslash = "/", mustWork = TRUE) : > path[1]="fnord.shp": No such file or directory > > "No such file or directory" > > Barry > > > > On Fri, Jan 22, 2016 at 2:25 AM, boB Rudis <b...@rudis.net> wrote: >> Agreed with the others. After finding that shapefile and getting it to >> work you are definitely not in the proper working directory. >> >> On Thu, Jan 21, 2016 at 8:40 PM, David Winsemius <dwinsem...@comcast.net> >> wrote: >>> >>>> On Jan 21, 2016, at 4:39 PM, Amoy Yang via R-help <r-help@r-project.org> >>>> wrote: >>>> >>>> Any advice for the following errors? >>>> state.map <- readShapeSpatial("maps/st24_d00.shp") >>>> Error in getinfo.shape(fn) : Error opening SHP file >>> >>> What does list.files('maps') return? Is there a 'st24_d00.shp' value in >>> there? >>> >>> >>> -- >>> David. >>> >>>> > > > > [[alternative HTML version deleted]] ^^ > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. David Winsemius Alameda, CA, USA [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Error opening SHP file
This is the results that addresses David's advice. > library(maptools) > library(maps) > state.map <- readShapeSpatial("maps/st24_d00.shp") Error in getinfo.shape(fn) : Error opening SHP file > # David question: What does list.files('maps') return? Is there a > 'st24_d00.shp' value in there? > list.files('maps') character(0) I actually use the link below to learn how to load state/zip-data on USA map. # http://stackoverflow.com/questions/1441717/plotting-color-map-with-zip-codes-in-r-or-python Plotting color map with zip codes in R or Python | | | | | | | | | | | Plotting color map with zip codes in R or PythonI have some US demographic and firmographic data. I would like to plot zipcode areas in a state or a smaller region (e.g. city). Each area would be annotated by col... | | | | View on stackoverflow.com | Preview by Yahoo | | | | | On Friday, January 22, 2016 10:03 AM, Barry Rowlingson <b.rowling...@lancaster.ac.uk> wrote: We can duplicate the error by giving a path to a non-existent shapefile, which is probably the original problem: > require(maptools) Loading required package: maptools Loading required package: sp Checking rgeos availability: TRUE > foo=readShapeSpatial("fnord.shp") Error in getinfo.shape(fn) : Error opening SHP file The error message there isn't totally explicit, and might cover a range of other possibilities such as a corrupted shapefile, or a missing .shx component of the shapefile or whatever. BUT you probably shouldn't be using readShapeSpatial anyway, as it has a habit of not reading the coordinate system in the .prj file. I find it much easier to use `raster::shapefile` which *does* read the coordinate system *and* gives a more explicit error message for a missing shapefile: > require(raster) Loading required package: raster > foo=shapefile("fnord.shp") Error in normalizePath(x, winslash = "/", mustWork = TRUE) : path[1]="fnord.shp": No such file or directory "No such file or directory" Barry On Fri, Jan 22, 2016 at 2:25 AM, boB Rudis <b...@rudis.net> wrote: > Agreed with the others. After finding that shapefile and getting it to > work you are definitely not in the proper working directory. > > On Thu, Jan 21, 2016 at 8:40 PM, David Winsemius <dwinsem...@comcast.net> > wrote: >> >>> On Jan 21, 2016, at 4:39 PM, Amoy Yang via R-help <r-help@r-project.org> >>> wrote: >>> >>> Any advice for the following errors? >>> state.map <- readShapeSpatial("maps/st24_d00.shp") >>> Error in getinfo.shape(fn) : Error opening SHP file >> >> What does list.files('maps') return? Is there a 'st24_d00.shp' value in >> there? >> >> >> -- >> David. >> >>> >>> [[alternative HTML version deleted]] >>> >>> __ >>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >>> and provide commented, minimal, self-contained, reproducible code. >> >> David Winsemius >> Alameda, CA, USA >> >> __ >> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html >> and provide commented, minimal, self-contained, reproducible code. > > __ > R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Error opening SHP file
Any advice for the following errors? state.map <- readShapeSpatial("maps/st24_d00.shp") Error in getinfo.shape(fn) : Error opening SHP file [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Use SQL in R environment
Hi All, I am new here and a beginner for R. Can I use SQL procedure in R environment as it can be done in SAS starting with PROC SQL; Thanks for helps! Amoy [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.