Re: [R] Convert json data to an r dataframe
Thank you very much, Rui! The problem with your script is that data is now in a string format, while I need to obtain a data.frame instead. Do you really think that I cannot do anything else with this json data? Thank you very much again Duccio 2012/12/27 Rui Barradas ruipbarra...@sapo.pt Hello, Your dataset is not in tabular form, so I find it difficult to transform it into a data.frame, but you can see it in R with the following. install.packages(RJSONIO, dependencies = TRUE) library(RJSONIO) url - http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=;http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te= dat - fromJSON(url) str(dat) # List with 3 components str(dat[[1]][[1]]) dat[[1]][[1]] str(dat[[1]][[2]]) unlist(dat[[1]][[2]]) str(dat[[1]][[3]]) dat[[1]][[3]] Hope this helps, Rui Barradas Em 27-12-2012 11:55, sp.duccio escreveu: Hello to everybody, I need to convert a json dataset in an R dataframe. I suppose that I'd need to use rjson or rjsonio package. The json dataset is:http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te= It would be nice if someone can help me to create a function like the one below:http://lamages.blogspot.it/2011/09/accessing-and-plotting-world-bank-data.html that can applied to my json dataset and with all the parameter values that can be specifiable. Is there anyone who can solve this problem? Thanx in advance for any help! Duccio __r-h...@r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Il messaggio trasmesso può contenere informazioni di carattere confidenziale rivolte esclusivamente al destinatario. Ne è vietato l'uso, la diffusione, la distribuzione o la riproduzione da parte di altre persone e/o entità diverse da quelle specificate. Nel caso aveste ricevuto questo messaggio per errore siete pregati di segnalarlo immediatamente al mittente e cancellare quanto ricevuto. This electronic mail transmission may contain confidential information addressed only to the person (s) named. Any use, distribution, copying or disclosure by any other person and/or entities other than the intended recipient is prohibited. If you received this transmission in error, please inform the sender immediately and delete the material. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Conjunction and disjunction in pubmed query
On Dec 27, 2012, at 3:10 PM, email wrote: Hi: I am trying to query pubmed abstracts using the following syntax: url= http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?; search = paste(url, db=pubmedterm=, queryTerm1, +AND+, queryTerm2,+OR+,queryTerm3, +OR+, queryTerm4, [abstract]retmax=100usehistory=y, sep=) I get an error here: Error in paste(url, db=pubmedterm=, queryTerm1, +AND+, queryTerm2, : object 'queryTerm1' not found docId - xmlTreeParse(getURL(paste(url, search, sep=)), useInternalNodes=TRUE) I want to fetch abstracts containing queryTerm1 AND queryTerm2 Or queryTerm3 OR queryTerm4. The code runs without error, but from the result I find that conjunction and disjunction is not working. Not working is a phrase that conveys very little. What exactly are you seeing, and how is it different than what you expected to see. Can anyone suggest a correct syntax for doing AND and OR pubmed query? Reviewing an earlier question [Subject: Re: [R] query multiple terms in PubMed abstract Date: December 11, 2012] I find that you did not reply to my earlier response where I said your code was returning expected results and asked what problems were prompting your questions. If you do not respond to questions and fail to include complete code, there is little people will be able to do. You need to provide complete code and a more complete problem description. If I were interested in articles the had MeSH headings for mortality and leukemia I would first attempt constructing a search phrase that looked like mortality[mh] AND leukemia[mh] rather than mortality[mh]+AND+leukemia[mh]. But I see that the + symbol seemed to succeed last time. You may need to use parentheses to get the intended grouping of PubMed search expressions. -- David Winsemius, MD Alameda, CA, USA __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Convert json data to an r dataframe
It really seems to me that what you ought to be doing is carefully studying the code provided at the lamages link that you referenced until you can modify it to do what you want. It really isn't fair for you to ask people to do your thinking for you. This is the R-help mailing list, not R-do-my-work-for-me mailing list. JSON, like XML, is a far more flexible data format than a data frame. That means it is up to you to decide what data or relationships you want to ignore so you can squeeze some useful subset of the information into a data frame. There is no way to write a do-it-all function that does this for all cases. Your goals must be part of each extraction function you use, so you really need to write it yourself. --- Jeff NewmillerThe . . Go Live... DCN:jdnew...@dcn.davis.ca.usBasics: ##.#. ##.#. Live Go... Live: OO#.. Dead: OO#.. Playing Research Engineer (Solar/BatteriesO.O#. #.O#. with /Software/Embedded Controllers) .OO#. .OO#. rocks...1k --- Sent from my phone. Please excuse my brevity. sp.duccio sp.duc...@gmail.com wrote: Thank you very much, Rui! The problem with your script is that data is now in a string format, while I need to obtain a data.frame instead. Do you really think that I cannot do anything else with this json data? Thank you very much again Duccio 2012/12/27 Rui Barradas ruipbarra...@sapo.pt Hello, Your dataset is not in tabular form, so I find it difficult to transform it into a data.frame, but you can see it in R with the following. install.packages(RJSONIO, dependencies = TRUE) library(RJSONIO) url - http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=;http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te= dat - fromJSON(url) str(dat) # List with 3 components str(dat[[1]][[1]]) dat[[1]][[1]] str(dat[[1]][[2]]) unlist(dat[[1]][[2]]) str(dat[[1]][[3]]) dat[[1]][[3]] Hope this helps, Rui Barradas Em 27-12-2012 11:55, sp.duccio escreveu: Hello to everybody, I need to convert a json dataset in an R dataframe. I suppose that I'd need to use rjson or rjsonio package. The json dataset is:http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te= It would be nice if someone can help me to create a function like the one below:http://lamages.blogspot.it/2011/09/accessing-and-plotting-world-bank-data.html that can applied to my json dataset and with all the parameter values that can be specifiable. Is there anyone who can solve this problem? Thanx in advance for any help! Duccio __r-h...@r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Convert json data to an r dataframe
Sorry, I really didn't want to push anyone to make my work. Sorry if my words sounded like that. I appreciated your help very much. Thank you D. 2012/12/28 Jeff Newmiller jdnew...@dcn.davis.ca.us It really seems to me that what you ought to be doing is carefully studying the code provided at the lamages link that you referenced until you can modify it to do what you want. It really isn't fair for you to ask people to do your thinking for you. This is the R-help mailing list, not R-do-my-work-for-me mailing list. JSON, like XML, is a far more flexible data format than a data frame. That means it is up to you to decide what data or relationships you want to ignore so you can squeeze some useful subset of the information into a data frame. There is no way to write a do-it-all function that does this for all cases. Your goals must be part of each extraction function you use, so you really need to write it yourself. --- Jeff NewmillerThe . . Go Live... DCN:jdnew...@dcn.davis.ca.usBasics: ##.#. ##.#. Live Go... Live: OO#.. Dead: OO#.. Playing Research Engineer (Solar/BatteriesO.O#. #.O#. with /Software/Embedded Controllers) .OO#. .OO#. rocks...1k --- Sent from my phone. Please excuse my brevity. sp.duccio sp.duc...@gmail.com wrote: Thank you very much, Rui! The problem with your script is that data is now in a string format, while I need to obtain a data.frame instead. Do you really think that I cannot do anything else with this json data? Thank you very much again Duccio 2012/12/27 Rui Barradas ruipbarra...@sapo.pt Hello, Your dataset is not in tabular form, so I find it difficult to transform it into a data.frame, but you can see it in R with the following. install.packages(RJSONIO, dependencies = TRUE) library(RJSONIO) url - http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te= http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te= dat - fromJSON(url) str(dat) # List with 3 components str(dat[[1]][[1]]) dat[[1]][[1]] str(dat[[1]][[2]]) unlist(dat[[1]][[2]]) str(dat[[1]][[3]]) dat[[1]][[3]] Hope this helps, Rui Barradas Em 27-12-2012 11:55, sp.duccio escreveu: Hello to everybody, I need to convert a json dataset in an R dataframe. I suppose that I'd need to use rjson or rjsonio package. The json dataset is: http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te= It would be nice if someone can help me to create a function like the one below: http://lamages.blogspot.it/2011/09/accessing-and-plotting-world-bank-data.html that can applied to my json dataset and with all the parameter values that can be specifiable. Is there anyone who can solve this problem? Thanx in advance for any help! Duccio __r-h...@r-project.org mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Il messaggio trasmesso può contenere informazioni di carattere confidenziale rivolte esclusivamente al destinatario. Ne è vietato l'uso, la diffusione, la distribuzione o la riproduzione da parte di altre persone e/o entità diverse da quelle specificate. Nel caso aveste ricevuto questo messaggio per errore siete pregati di segnalarlo immediatamente al mittente e cancellare quanto ricevuto. This electronic mail transmission may contain confidential information addressed only to the person (s) named. Any use, distribution, copying or disclosure by any other person and/or entities other than the intended recipient is prohibited. If you received this transmission in error, please inform the sender immediately and delete the material. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] R crashing inconsistently within for loops
You are not wrong to expect R to not crash. However, R (as most people use it) is not monolithic, and you have provided neither reproducible code nor sessionInfo() with the relevant packages loaded to help anyone interested in investigating the problem. You are the most likely person to be able to generate sample code that reproduces your problem, even if imperfectly. --- Jeff NewmillerThe . . Go Live... DCN:jdnew...@dcn.davis.ca.usBasics: ##.#. ##.#. Live Go... Live: OO#.. Dead: OO#.. Playing Research Engineer (Solar/BatteriesO.O#. #.O#. with /Software/Embedded Controllers) .OO#. .OO#. rocks...1k --- Sent from my phone. Please excuse my brevity. Steve Powers power...@nd.edu wrote: Hello, This one has been bugging me for a long time and I have never found a solution. I am using R version 2.15.1 but it has come up in older versions of R I have used over the past 2-3 years. Q: Am I wrong to expect that R should handle hundreds of iterations of the base model or statistical functions, embedded within for loops, in one script run? I have found that when I write scripts that do this, sometimes they have a tendency to crash, seemingly unpredictably. For example, one problem script of mine employs glm and gls about a hundred different times, and output files are being written at the end of each iteration. I have used my output files to determine that the crash cause is not consistent (R never fails at the same iteration). Note that the data are fixed here (no data generation or randomization steps, so that is not the issue). But it is clear that scripts with larger numbers of iterations are more likely to produce a crash. And a year or two ago, I had a seemingly stable R script again with for looped model fits, but discovered this script was prone to crashing when I ran it on a newer PC. Because the new PC also seemed to be blazing through R code absurdly fast, I tried adding a short fluff procedure at the end of each iteration that required a few seconds of processing time. Low and behold, when I added that, the script stopped crashing (and each iteration of course took longer). I still don't understand why that fixed things. What is going on? Solutions? Thanks.---steve __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] R crashing inconsistently within for loops
Note though that the posting guide asked you not to use the word 'crash', as your audience has no idea what you mean by it. In some of the senses people use (e.g. when R reports an error in your code), you should expect R to 'crash'. On 28/12/2012 09:04, Jeff Newmiller wrote: You are not wrong to expect R to not crash. However, R (as most people use it) is not monolithic, and you have provided neither reproducible code nor sessionInfo() with the relevant packages loaded to help anyone interested in investigating the problem. You are the most likely person to be able to generate sample code that reproduces your problem, even if imperfectly. --- Jeff NewmillerThe . . Go Live... DCN:jdnew...@dcn.davis.ca.usBasics: ##.#. ##.#. Live Go... Live: OO#.. Dead: OO#.. Playing Research Engineer (Solar/BatteriesO.O#. #.O#. with /Software/Embedded Controllers) .OO#. .OO#. rocks...1k --- Sent from my phone. Please excuse my brevity. Steve Powers power...@nd.edu wrote: Hello, This one has been bugging me for a long time and I have never found a solution. I am using R version 2.15.1 but it has come up in older versions of R I have used over the past 2-3 years. Q: Am I wrong to expect that R should handle hundreds of iterations of the base model or statistical functions, embedded within for loops, in one script run? I have found that when I write scripts that do this, sometimes they have a tendency to crash, seemingly unpredictably. For example, one problem script of mine employs glm and gls about a hundred different times, and output files are being written at the end of each iteration. I have used my output files to determine that the crash cause is not consistent (R never fails at the same iteration). Note that the data are fixed here (no data generation or randomization steps, so that is not the issue). But it is clear that scripts with larger numbers of iterations are more likely to produce a crash. And a year or two ago, I had a seemingly stable R script again with for looped model fits, but discovered this script was prone to crashing when I ran it on a newer PC. Because the new PC also seemed to be blazing through R code absurdly fast, I tried adding a short fluff procedure at the end of each iteration that required a few seconds of processing time. Low and behold, when I added that, the script stopped crashing (and each iteration of course took longer). I still don't understand why that fixed things. What is going on? Solutions? Thanks.---steve __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Merging data tables
Hi all, I am trying to merge several data sets and end up with a long data format by date time so I can run correlations and plots. I am using Deducer as an R GUI but can just use the R console if easier. The data sets are weather with wind speed, relative humidity and temperatures by date and minute and bat activity with date, time, label, and an activity index number. The bat activity is only during the nocturnal time frames while the weather data was recorded for 24 hours. Therefore a lot of weather data with no related activity for bats. I have failed so far to achieve what I need and tried plyr and reshape2. There are many Null data rows with no data or 0 (zero) for wind speed.. What other tools steps would be more appropriate short of manually in Excel cutting and pasting for a day or more? Data formats are: bat activity DateTimeLabel Number 6/3/201110:01 Tadbra 2 6/3/201110:02 Tadbra 4 6/3/201110:08 Tadbra 1 6/3/201110:13 Tadbra 2 6/3/201110:49 Tadbra 2 6/3/201110:51 Tadbra 2 6/3/201110:52 Tadbra 4 Weather: datetimeTemp_I Temp_E RH mph_a mph_x 6/3/20110:0015 15.730.44.4 15.5 6/3/20110:3015 15.231.65.7 11.2 6/3/20111:0015 15.131.310.317.5 6/3/20111:3014 13.644.54.7 12.1 6/3/20112:0012.513.237.92.1 6.5 6/3/20112:3012.513.535.36.3 10.1 6/3/20113:0012 12.137.73 7.4 6/3/20113:3011.511.538.73.4 6 6/3/20114:0010 9.9 52.71.4 4.2 6/3/20114:309.5 9.1 43.21.3 3.8 6/3/20115:008 8.7 59.21.2 3.1 6/3/20115:307 8 62.51.1 4.2 6/3/20116:006 7.8 47.80.5 2.2 6/3/20116:307.5 11.537.51.7 3.8 6/3/20117:0010.514 33.10.6 2.2 6/3/20117:3014 17.332.11.6 3.6 6/3/20118:0017.520.323.90.4 2 6/3/20118:3021.522.820.70.4 1.8 6/3/20119:0024.524.914.10.3 2.2 6/3/20119:3026 26.920.31.7 5.6 6/3/201110:00 27.527.420.72.5 6.5 6/3/201110:30 28.529.810 1.6 4.2 6/3/201111:00 29.529.49.8 1.8 4 6/3/201111:30 30 29.99.2 2.7 7.8 Tnx for any suggestions, Bruce [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Dock graphs when using R in Eclipse with StatET plug In
Anyone have a solution to this by now? Unfortunately, I made some changes, and now the graphic window pops up and canât be docked anymore. Thanks for any hints! [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Merging data tables
see ?merge, for your dataframes, the following should work: merge(bat_activity, weather, by=c(date, time), all=T) 2012/12/28 Neotropical bat risk assessments neotropical.b...@gmail.com Hi all, I am trying to merge several data sets and end up with a long data format by date time so I can run correlations and plots. I am using Deducer as an R GUI but can just use the R console if easier. The data sets are weather with wind speed, relative humidity and temperatures by date and minute and bat activity with date, time, label, and an activity index number. The bat activity is only during the nocturnal time frames while the weather data was recorded for 24 hours. Therefore a lot of weather data with no related activity for bats. I have failed so far to achieve what I need and tried plyr and reshape2. There are many Null data rows with no data or 0 (zero) for wind speed.. What other tools steps would be more appropriate short of manually in Excel cutting and pasting for a day or more? Data formats are: bat activity DateTimeLabel Number 6/3/201110:01 Tadbra 2 6/3/201110:02 Tadbra 4 6/3/201110:08 Tadbra 1 6/3/201110:13 Tadbra 2 6/3/201110:49 Tadbra 2 6/3/201110:51 Tadbra 2 6/3/201110:52 Tadbra 4 Weather: datetimeTemp_I Temp_E RH mph_a mph_x 6/3/20110:0015 15.730.44.4 15.5 6/3/20110:3015 15.231.65.7 11.2 6/3/20111:0015 15.131.310.317.5 6/3/20111:3014 13.644.54.7 12.1 6/3/20112:0012.513.237.92.1 6.5 6/3/20112:3012.513.535.36.3 10.1 6/3/20113:0012 12.137.73 7.4 6/3/20113:3011.511.538.73.4 6 6/3/20114:0010 9.9 52.71.4 4.2 6/3/20114:309.5 9.1 43.21.3 3.8 6/3/20115:008 8.7 59.21.2 3.1 6/3/20115:307 8 62.51.1 4.2 6/3/20116:006 7.8 47.80.5 2.2 6/3/20116:307.5 11.537.51.7 3.8 6/3/20117:0010.514 33.10.6 2.2 6/3/20117:3014 17.332.11.6 3.6 6/3/20118:0017.520.323.90.4 2 6/3/20118:3021.522.820.70.4 1.8 6/3/20119:0024.524.914.10.3 2.2 6/3/20119:3026 26.920.31.7 5.6 6/3/201110:00 27.527.420.72.5 6.5 6/3/201110:30 28.529.810 1.6 4.2 6/3/201111:00 29.529.49.8 1.8 4 6/3/201111:30 30 29.99.2 2.7 7.8 Tnx for any suggestions, Bruce [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Irreproducible exception in R extension
Dear R-Users, I am having some trouble running an R extension on our cluster (linux). I call C++ code in which I use a) the Scythe Statistical Library and b) OpenMP. Most of the jobs run without a problem, but some arbitrary jobs throw an exception of the kind printed below while running in a parallel loop. The behavior is to me not reproducible, although at every run of 729 Jobs it at least happens once. I already tried to enforce the error by using the Intel Compiler instead of the GCC 4.7.2 and using the Options '-checkinit' for possibly uninitialized values and '-ftrapuv' for initialization of stack variables to unusual values to aid error detection. But I cannot enforce the exception, which makes it impossible to debug. I already googled this error, but there only 2-3 small forum discussions about it, and so I hope I find some suggestions here. 1. What do you think this error comes from? 2. What do you advice me to do to detect the error? I am excited for your answers. Simon *** glibc detected *** /home/simo_109//opt/R/lib64/R/bin/exec/R: double free or corruption (!prev): 0x01355610 *** === Backtrace: = /lib64/libc.so.6[0x3845a75916] /lib64/libc.so.6[0x3845a78443] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdE10deallocateEv+0x7b)[0x7f4645d5d6db] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdED1Ev+0x47)[0x7f4645d5d5bb] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdE17withdrawReferenceEv+0xaa)[0x7f4645d83442] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED1Ev+0x94)[0x7f4645d82cc8] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED2Ev+0x47)[0x7f4645d82d9f] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe6MatrixIdLNS_12matrix_orderE0ELNS_12matrix_styleE1EED1Ev+0x5d)[0x7f4645d8b26d] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_Z11MCgmmS_implIN6scythe8mersenneEEvRNS0_3rngIT_EERP7SEXPRECS8_jjRNS0_6MatrixIdLNS0_12matrix_orderE0ELNS0_12matrix_styleE0EEERKSC_jS8_S8_+0x478f)[0x7f4645d6d6af] /opt/intel/Compiler/12.1/6.361/rwthlnk/compiler/lib/intel64/libiomp5.so(__kmp_invoke_microtask+0x93)[0x7f4645aabee3] === Memory map: 0040-00401000 r-xp 00:21 32550383 /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R 0060-00601000 rw-p 00:21 32550383 /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R 00601000-029cc000 rw-p 00:00 0 [heap] 369ac0-369ac3a000 r-xp 08:01 263771 /lib64/libreadline.so.6.0 369ac3a000-369ae3a000 ---p 0003a000 08:01 263771 /lib64/libreadline.so.6.0 369ae3a000-369ae42000 rw-p 0003a000 08:01 263771 /lib64/libreadline.so.6.0 369ae42000-369ae43000 rw-p 00:00 0 369bc0-369bc16000 r-xp 08:01 263757 /lib64/libgcc_s-4.4.6-20120305.so.1 369bc16000-369be15000 ---p 00016000 08:01 263757 /lib64/libgcc_s-4.4.6-20120305.so.1 369be15000-369be16000 rw-p 00015000 08:01 263757 /lib64/libgcc_s-4.4.6-20120305.so.1 369c00-369c0e8000 r-xp 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c0e8000-369c2e8000 ---p 000e8000 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c2e8000-369c2ef000 r--p 000e8000 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c2ef000-369c2f1000 rw-p 000ef000 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c2f1000-369c306000 rw-p 00:00 0 369d00-369d00d000 r-xp 08:01 671072 /usr/lib64/libgomp.so.1.0.0 369d00d000-369d20c000 ---p d000 08:01 671072 /usr/lib64/libgomp.so.1.0.0 369d20c000-369d20d000 rw-p c000 08:01 671072 /usr/lib64/libgomp.so.1.0.0 369fa0-369faf r-xp 08:01 660772 /usr/lib64/libgfortran.so.3.0.0 369faf-369fcef000 ---p 000f 08:01 660772 /usr/lib64/libgfortran.so.3.0.0 369fcef000-369fcf1000 rw-p 000ef000 08:01 660772 /usr/lib64/libgfortran.so.3.0.0 369fcf1000-369fcf2000 rw-p 00:00 0 36a360-36a373f000 r-xp 08:01 675210 /usr/lib64/libicuuc.so.42.1 36a373f000-36a393f000 ---p 0013f000 08:01 675210 /usr/lib64/libicuuc.so.42.1 36a393f000-36a395 rw-p 0013f000 08:01 675210 /usr/lib64/libicuuc.so.42.1 36a395-36a3952000 rw-p 00:00 0 36a3a0-36a3b88000 r-xp
[R] Irreproducible exception in R extension
Dear R-Users, I am having some trouble running an R extension on our cluster (linux). I call C++ code in which I use a) the Scythe Statistical Library and b) OpenMP. Most of the jobs run without a problem, but some arbitrary jobs throw an exception of the kind printed below while running in a parallel loop. The behavior is to me not reproducible, although at every run of 729 Jobs it at least happens once. I already tried to enforce the error by using the Intel Compiler instead of the GCC 4.7.2 and using the Options '-checkinit' for possibly uninitialized values and '-ftrapuv' for initialization of stack variables to unusual values to aid error detection. But I cannot enforce the exception, which makes it impossible to debug. I already googled this error, but there only 2-3 small forum discussions about it, and so I hope I find some suggestions here. 1. What do you think this error comes from? 2. What do you advice me to do to detect the error? I am excited for your answers. Simon *** glibc detected *** /home/simo_109//opt/R/lib64/R/bin/exec/R: double free or corruption (!prev): 0x01355610 *** === Backtrace: = /lib64/libc.so.6[0x3845a75916] /lib64/libc.so.6[0x3845a78443] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdE10deallocateEv+0x7b)[0x7f4645d5d6db] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdED1Ev+0x47)[0x7f4645d5d5bb] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdE17withdrawReferenceEv+0xaa)[0x7f4645d83442] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED1Ev+0x94)[0x7f4645d82cc8] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED2Ev+0x47)[0x7f4645d82d9f] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe6MatrixIdLNS_12matrix_orderE0ELNS_12matrix_styleE1EED1Ev+0x5d)[0x7f4645d8b26d] /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_Z11MCgmmS_implIN6scythe8mersenneEEvRNS0_3rngIT_EERP7SEXPRECS8_jjRNS0_6MatrixIdLNS0_12matrix_orderE0ELNS0_12matrix_styleE0EEERKSC_jS8_S8_+0x478f)[0x7f4645d6d6af] /opt/intel/Compiler/12.1/6.361/rwthlnk/compiler/lib/intel64/libiomp5.so(__kmp_invoke_microtask+0x93)[0x7f4645aabee3] === Memory map: 0040-00401000 r-xp 00:21 32550383 /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R 0060-00601000 rw-p 00:21 32550383 /rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R 00601000-029cc000 rw-p 00:00 0 [heap] 369ac0-369ac3a000 r-xp 08:01 263771 /lib64/libreadline.so.6.0 369ac3a000-369ae3a000 ---p 0003a000 08:01 263771 /lib64/libreadline.so.6.0 369ae3a000-369ae42000 rw-p 0003a000 08:01 263771 /lib64/libreadline.so.6.0 369ae42000-369ae43000 rw-p 00:00 0 369bc0-369bc16000 r-xp 08:01 263757 /lib64/libgcc_s-4.4.6-20120305.so.1 369bc16000-369be15000 ---p 00016000 08:01 263757 /lib64/libgcc_s-4.4.6-20120305.so.1 369be15000-369be16000 rw-p 00015000 08:01 263757 /lib64/libgcc_s-4.4.6-20120305.so.1 369c00-369c0e8000 r-xp 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c0e8000-369c2e8000 ---p 000e8000 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c2e8000-369c2ef000 r--p 000e8000 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c2ef000-369c2f1000 rw-p 000ef000 08:01 656025 /usr/lib64/libstdc++.so.6.0.13 369c2f1000-369c306000 rw-p 00:00 0 369d00-369d00d000 r-xp 08:01 671072 /usr/lib64/libgomp.so.1.0.0 369d00d000-369d20c000 ---p d000 08:01 671072 /usr/lib64/libgomp.so.1.0.0 369d20c000-369d20d000 rw-p c000 08:01 671072 /usr/lib64/libgomp.so.1.0.0 369fa0-369faf r-xp 08:01 660772 /usr/lib64/libgfortran.so.3.0.0 369faf-369fcef000 ---p 000f 08:01 660772 /usr/lib64/libgfortran.so.3.0.0 369fcef000-369fcf1000 rw-p 000ef000 08:01 660772 /usr/lib64/libgfortran.so.3.0.0 369fcf1000-369fcf2000 rw-p 00:00 0 36a360-36a373f000 r-xp 08:01 675210 /usr/lib64/libicuuc.so.42.1 36a373f000-36a393f000 ---p 0013f000 08:01 675210 /usr/lib64/libicuuc.so.42.1 36a393f000-36a395 rw-p 0013f000 08:01 675210 /usr/lib64/libicuuc.so.42.1 36a395-36a3952000 rw-p 00:00 0 36a3a0-36a3b88000 r-xp
Re: [R] help with reshaping wide to long format
Hi, DId you tried copy and paste the code in my reply? If not, please do. In your previous replies, you forgot to close the bracket()) or adding # in between the codes. Please check your codes. It would be also helpful to read the posting guide (http://www.r-project.org/posting-guide.html) and An introduction to R (the link will be in the posting guide). A.K. - Original Message - From: usha2013 usha.nat...@gmail.com To: r-help@r-project.org Cc: Sent: Friday, December 28, 2012 2:02 AM Subject: Re: [R] help with reshaping wide to long format Hi, Sorry, but how did you bring it out? Thanks On Fri, Dec 28, 2012 at 8:48 AM, arun kirshna [via R] ml-node+s789695n4654093...@n4.nabble.com wrote: Hi, bp.sub- structure(list(CODEA = c(1L, 3L, 4L, 7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 16L, 17L), C45 = c(NA, 2L, 2L, 2L, 2L, 1L, NA, 1L, 1L, 2L, 1L, 2L, 1L), ragek = c(3L, 3L, 3L, 4L, 4L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L), ra80 = c(4L, 3L, 6L, 6L, 4L, 6L, 3L, 4L, 4L, 4L, 4L, 4L, 4L), ra98 = c(1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L, 2L, 2L, 2L, 3L, 1L), CBCLAggressionAt1410 = c(NA, 0L, NA, 0L, 0L, 0L, NA, 0L, NA, NA, 0L, 0L, NA), CBCLInternalisingAt1410 = c(NA, 0L, NA, 0L, 0L, 0L, NA, 0L, NA, NA, 0L, 0L, NA), Obese14 = c(NA, 0L, NA, 0L, 0L, 0L, NA, NA, NA, NA, 0L, 0L, NA), Obese21 = c(NA, 0L, NA, 1L, 0L, 0L, NA, 0L, NA, NA, 0L, 0L, NA), Overweight14 = c(NA, 0L, NA, 0L, 0L, 0L, NA, NA, NA, NA, 0L, 0L, NA), Overweight21 = c(NA, 1L, NA, 1L, 0L, 0L, NA, 0L, NA, NA, 1L, 0L, NA), hibp14 = c(NA, 0L, NA, 0L, 0L, 0L, NA, NA, NA, NA, 1L, 1L, NA), hibp21 = c(NA, 0L, NA, 0L, 0L, 0L, NA, 0L, NA, NA, 1L, NA, NA)), .Names = c(CODEA, C45, ragek, ra80, ra98, CBCLAggressionAt1410, CBCLInternalisingAt1410, Obese14, Obese21, Overweight14, Overweight21, hibp14, hibp21), row.names = c(NA, 13L), class = data.frame) BP.stack1- reshape(bp.sub,sep=,timevar= time,direction=long,varying=list(names(bp.sub[8:9]),names(bp.sub[10:11]),names(bp.sub[12:13])),v.names=c(Obese,Overweight,HiBP),idvar=CODEA) BP.stack1$time - recode(BP.stack1$time,(1=14;2=21)) BP.stack1 # CODEA C45 ragek ra80 ra98 CBCLAggressionAt1410 CBCLInternalisingAt1410 #1.1 1 NA 3 4 1 NA NA #3.1 3 2 3 3 1 0 0 #4.1 4 2 3 6 1 NA NA #7.1 7 2 4 6 1 0 0 #8.1 8 2 4 4 1 0 0 #9.1 9 1 3 6 2 0 0 #10.1 10 NA 3 3 1 NA NA #11.1 11 1 3 4 1 0 0 #12.1 12 1 3 4 2 NA NA #13.1 13 2 3 4 2 NA NA #14.1 14 1 3 4 2 0 0 #16.1 16 2 3 4 3 0 0 #17.1 17 1 3 4 1 NA NA #1.2 1 NA 3 4 1 NA NA #3.2 3 2 3 3 1 0 0 #4.2 4 2 3 6 1 NA NA #7.2 7 2 4 6 1 0 0 #8.2 8 2 4 4 1 0 0 #9.2 9 1 3 6 2 0 0 #10.2 10 NA 3 3 1 NA NA #11.2 11 1 3 4 1 0 0 #12.2 12 1 3 4 2 NA NA #13.2 13 2 3 4 2 NA NA #14.2 14 1 3 4 2 0 0 #16.2 16 2 3 4 3 0 0 #17.2 17 1 3 4 1 NA NA # time Obese Overweight HiBP #1.1 14 NA NA NA #3.1 14 0 0 0 #4.1 14 NA NA NA #7.1 14 0 0 0 #8.1 14 0 0 0 #9.1 14 0 0 0 #10.1 14 NA NA NA #11.1 14 NA NA NA #2.1 14 NA NA NA #13.1 14 NA NA NA #14.1 14 0 0 1 #16.1 14 0 0 1 #17.1 14 NA NA NA #1.2 21 NA NA NA #3.2 21 0 1 0 #4.2 21 NA NA NA #7.2 21 1 1 0 #8.2 21 0 0 0 #9.2 21 0 0 0 #10.2 21 NA NA NA #11.2 21 0 0 0 #12.2 21 NA NA NA #13.2 21 NA NA NA #14.2 21 0 1 1 #16.2 21 0 0 NA #17.2 21 NA NA NA names(bp.sub)[grep(\\d+,names(bp.sub))]-gsub((\\D+)(\\d+),\\1_\\2,names(bp.sub)[grep(\\d+,names(bp.sub))]) BP.stack2-reshape(bp.sub,dir=long,varying=8:13,sep=_) Bp.stack2 CODEA C_45 ragek ra_80 ra_98 CBCLAggressionAt_1410 1.14 1 NA 3 4 1 NA 2.14 3 2 3 3 1 0 3.14 4 2 3 6 1
Re: [R] Merging data tables
HI, YOu may try ?merge() or ?join() from library(plyr) bat_activity-read.table(text= Date Time Label Number 6/3/2011 10:01 Tadbra 2 6/3/2011 10:02 Tadbra 4 6/3/2011 10:08 Tadbra 1 6/3/2011 10:13 Tadbra 2 6/3/2011 10:49 Tadbra 2 6/3/2011 10:51 Tadbra 2 6/3/2011 10:52 Tadbra 4 ,sep=,header=TRUE,stringsAsFactors=FALSE) Weather-read.table(text= date time Temp_I Temp_E RH mph_a mph_x 6/3/2011 0:00 15 15.7 30.4 4.4 15.5 --- 6/3/2011 11:00 29.5 29.4 9.8 1.8 4 6/3/2011 11:30 30 29.9 9.2 2.7 7.8 ,sep=,header=TRUE,stringsAsFactors=FALSE) bat_activity1-data.frame(dateTime=as.POSIXct(paste(bat_activity[,1],bat_activity[,2]),format=%m/%d/%Y %H:%M),bat_activity[,3:4]) Weather1-data.frame(dateTime=as.POSIXct(paste(Weather[,1],Weather[,2]),format=%m/%d/%Y %H:%M),Weather[,3:7]) res- merge(bat_activity1,Weather1,by=dateTime,all=TRUE) head(res) #here there are NAs because there are no corresponding values in one of the dataset # dateTime Label Number Temp_I Temp_E RH mph_a mph_x #1 2011-06-03 00:00:00 NA NA 15.0 15.7 30.4 4.4 15.5 #2 2011-06-03 00:30:00 NA NA 15.0 15.2 31.6 5.7 11.2 #3 2011-06-03 01:00:00 NA NA 15.0 15.1 31.3 10.3 17.5 #4 2011-06-03 01:30:00 NA NA 14.0 13.6 44.5 4.7 12.1 #5 2011-06-03 02:00:00 NA NA 12.5 13.2 37.9 2.1 6.5 #6 2011-06-03 02:30:00 NA NA 12.5 13.5 35.3 6.3 10.1 Also, you mentioned about merging more than 2 datasets. In that case, use: Reduce(function(...) merge(...,by=dateTime),list(bat_activity1,Weather1,dat3,dat4)) #or library(reshape) merge_recurse(list(bat_activity1,Weather,dat3,dat4), by=dateTime) Hope this helps. A.K. - Original Message - From: Neotropical bat risk assessments neotropical.b...@gmail.com To: r-help@r-project.org; dedu...@googlegroups.com Cc: Sent: Friday, December 28, 2012 6:46 AM Subject: [R] Merging data tables Hi all, I am trying to merge several data sets and end up with a long data format by date time so I can run correlations and plots. I am using Deducer as an R GUI but can just use the R console if easier. The data sets are weather with wind speed, relative humidity and temperatures by date and minute and bat activity with date, time, label, and an activity index number. The bat activity is only during the nocturnal time frames while the weather data was recorded for 24 hours. Therefore a lot of weather data with no related activity for bats. I have failed so far to achieve what I need and tried plyr and reshape2. There are many Null data rows with no data or 0 (zero) for wind speed.. What other tools steps would be more appropriate short of manually in Excel cutting and pasting for a day or more? Data formats are: bat activity Date Time Label Number 6/3/2011 10:01 Tadbra 2 6/3/2011 10:02 Tadbra 4 6/3/2011 10:08 Tadbra 1 6/3/2011 10:13 Tadbra 2 6/3/2011 10:49 Tadbra 2 6/3/2011 10:51 Tadbra 2 6/3/2011 10:52 Tadbra 4 Weather: date time Temp_I Temp_E RH mph_a mph_x 6/3/2011 0:00 15 15.7 30.4 4.4 15.5 6/3/2011 0:30 15 15.2 31.6 5.7 11.2 6/3/2011 1:00 15 15.1 31.3 10.3 17.5 6/3/2011 1:30 14 13.6 44.5 4.7 12.1 6/3/2011 2:00 12.5 13.2 37.9 2.1 6.5 6/3/2011 2:30 12.5 13.5 35.3 6.3 10.1 6/3/2011 3:00 12 12.1 37.7 3 7.4 6/3/2011 3:30 11.5 11.5 38.7 3.4 6 6/3/2011 4:00 10 9.9 52.7 1.4 4.2 6/3/2011 4:30 9.5 9.1 43.2 1.3 3.8 6/3/2011 5:00 8 8.7 59.2 1.2 3.1 6/3/2011 5:30 7 8 62.5 1.1 4.2 6/3/2011 6:00 6 7.8 47.8 0.5 2.2 6/3/2011 6:30 7.5 11.5 37.5 1.7 3.8 6/3/2011 7:00 10.5 14 33.1 0.6 2.2 6/3/2011 7:30 14 17.3 32.1 1.6 3.6 6/3/2011 8:00 17.5 20.3 23.9 0.4 2 6/3/2011 8:30 21.5 22.8 20.7 0.4 1.8 6/3/2011 9:00 24.5 24.9 14.1 0.3 2.2 6/3/2011 9:30 26 26.9 20.3 1.7 5.6 6/3/2011 10:00 27.5 27.4 20.7 2.5 6.5 6/3/2011 10:30 28.5 29.8 10 1.6 4.2 6/3/2011 11:00 29.5 29.4 9.8 1.8 4 6/3/2011 11:30 30 29.9 9.2 2.7 7.8 Tnx for any suggestions, Bruce [[alternative HTML version deleted]] __
Re: [R] efficiently multiply different matrices in 3-d array with different vectors?
On Thu, 27 Dec 2012 22:03:19 -0800 arun smartpink...@yahoo.com wrote: HI, Not sure if this is what you wanted: set.seed(14) Z-array(sample(1:100,80,replace=TRUE),dim=c(5,2,8)) set.seed(21) Y-matrix(sample(1:40,40,replace=TRUE),nrow=8) do.call(cbind,lapply(seq_len(dim(Z)[1]),function(i) Z[i,,]%*%Y[,i])) # [,1] [,2] [,3] [,4] [,5] #[1,] 5065 6070 12328 11536 13678 #[2,] 6152 9878 11161 13777 7991 Yes, this does it, many thanks! Ranjan - Original Message - From: Ranjan Maitra maitra.mbox.igno...@inbox.com To: r-help@r-project.org Cc: Sent: Friday, December 28, 2012 12:33 AM Subject: [R] efficiently multiply different matrices in 3-d array with different vectors? Hello, I have been wondering of an efficient way to do this: I have an n x m x p array Z and a p x n matrix Y. I want to multiply each of the n matrices with the corresponding column vector of Y. In other words, I am wanting to matrix multiply: Z[i, ,] %*% Y[, i] which will give me a (two-dimensional) array or matrix of dimension n x p with the i'th row storing the above. Any pointers on an efficient way of doing this? I considered using apply, but was not successful. Many thanks again and best wishes, Ranjan -- Important Notice: This mailbox is ignored: e-mails are set to be deleted on receipt. For those needing to send personal or professional e-mail, please use appropriate addresses. FREE ONLINE PHOTOSHARING - Share your photos online with your friends and family! Visit http://www.inbox.com/photosharing to find out more! __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Important Notice: This mailbox is ignored: e-mails are set to be deleted on receipt. For those needing to send personal or professional e-mail, please use appropriate addresses. GET FREE 5GB EMAIL - Check out spam free email with many cool features! Visit http://www.inbox.com/email to find out more! __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Merging data tables
HI, It is related to the memory. Check these links for discussions regarding this matter: http://r.789695.n4.nabble.com/Memory-Problem-td853869.html http://r.789695.n4.nabble.com/Memory-issues-td3220252.html A.K. From: Neotropical bat risk assessments neotropical.b...@gmail.com To: arun smartpink...@yahoo.com Sent: Friday, December 28, 2012 9:19 AM Subject: Re: [R] Merging data tables Hi, I was able to get the merge to work OK, I think... Bats_weather-merge(sewage.Wind_Temp.temp,Tabr_all2.temp,by.x=row.names,by.y=row.names,incomparables = NA,all.x =F,all.y =T) rm(list=c(sewage.Wind_Temp.temp,Tabr_all2.temp)) dput(Bats_weather,'C:/=Bat data working/Rainey/Sewage/Tadbra_weather.robj') Bats_weather[,9]-as.character(Bats_weather[,9]) dput(Bats_weather,'C:/=Bat data working/Rainey/Sewage/Tadbra_weather.robj') obj_a - t.test(x = Bats_weather$Number,y = Bats_weather$Temp_E,paired = TRUE) print(obj_a) But I can not run the correlations... ggcorplot(cor.mat=corr.mat,data=Bats_weather, + cor_text_limits=c(5,20), + line.method=lm) Error: cannot allocate vector of size 1.6 Gb In addition: Warning message: In get(x, envir = this, inherits = inh) : restarting interrupted promise evaluation rm('corr.mat') -- Bruce W. Miller, PhD. Neotropical bat risk assessments If we lose the bats, we may lose much of the tropical vegetation and the lungs of the planet Using acoustic sampling to map species distributions for 15 years. Providing Interactive identification keys to the vocal signatures of New World Bats For various project details see: https://sites.google.com/site/batsoundservices/ __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Question about Linear Regression
Dear All, A semi-trivial question: suppose you want to carry out a linear regression of the kind y~x1+x2+x3 where x3 is a dichotomous variable assuming only 0 and 1 values (x1 and x2 are continuous variables). Is there any particular caveat I should be aware of? Can I code this as a simple multiple linear regression into R or is there anything else I should bear in mind? Many thanks Lorenzo __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Question about Linear Regression
You can run that as it is. The term to search for on Google is 'dummy coding'. Jeremy On 28 December 2012 07:45, Lorenzo Isella lorenzo.ise...@gmail.com wrote: where x3 is a dichotomous variable assuming only 0 and 1 values (x1 and x2 are continuous variables). Is there any particular caveat I should be aware of? Can I code this as a simple multiple linear regression into R or is there anything else I should bear in mind? [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Help reading matrixmarket files
Could someone please approve this message? Thanks! On Thu, Dec 27, 2012 at 4:38 PM, Brad Cox bradj...@gmail.com wrote: I'm trying to read data a program produces in matrixmarket array format into R and its giving me fits. I've tried read.MM (below) and readMM (from the Matrix package) but neither works. One of them says array format isn't supported, the other reports something indecipherable about Fortran. Here's the file contents. Can't get much simpler than that! %%MatrixMarket matrix array real general %This file contains LIBFM global mean which is required for computing predictions. 1 1 1.786238830716e-02 Can someone please help? Seems such a simple problem. library(spam) Spam version 0.29-2 (2012-08-17) is loaded. Type 'help( Spam)' or 'demo( spam)' for a short introduction and overview of this package. Help for individual functions is also obtained by adding the suffix '.spam' to the function name, e.g. 'help( chol.spam)'. Attaching package: âspamâ The following object(s) are masked from âpackage:baseâ: backsolve, forwardsolve x=read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm) Error in read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm) : invalid mode (list) to pass to Fortran (arg 8) -- Guns kill people... the way forks make people fat. Dr. Brad J. CoxCell: 703-594-1883 Blog: http://bradjcox.blogspot.comWeb: http://virtualschool.edu [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] vectorization modifying globals in functions
In current versions of R the apply functions do not gain much (if any) in speed over a well written for loop (the for loops are much more efficient than they used to be). Using global variables could actually slow things down a little for what you are doing, if you use `-` then it has to search through multiple environments to find which to replace. In general you should avoid using global variables. It is best to pass all needed variables into a function as arguments, do any modifications internally inside the function on local copies, then return the modified local copy from the function (you can use a list if you want to return multiple variables). Since each iteration of your code depends on the previous iteration, vectorizing is not going to help (or even be reasonable). If you want to speed up the code then you might consider a compiled option, see the inline or rcpp packages (or others). On Thu, Dec 27, 2012 at 1:38 PM, Sam Steingold s...@gnu.org wrote: I have the following code: --8---cut here---start-8--- d - rep(10,10) for (i in 1:100) { a - sample.int(length(d), size = 2) if (d[a[1]] = 1) { d[a[1]] - d[a[1]] - 1 d[a[2]] - d[a[2]] + 1 } } --8---cut here---end---8--- it does what I want, i.e., modified vector d 100 times. Now, if I want to repeat this 1e6 times instead of 1e2 times, I want to vectorize it for speed, so I do this: --8---cut here---start-8--- update - function (i) { a - sample.int(n.agents, size = 2) if (d[a[1]] = delta) { d[a[1]] - d[a[1]] - 1 d[a[2]] - d[a[2]] + 1 } entropy(d, unit=log2) } system.time(entropy.history - sapply(1:1e6,update)) --8---cut here---end---8--- however, the global d is not modified, apparently update modifies the local copy. so, 1. is there a way for a function to modify a global variable? 2. how would you vectorize this loop? thanks! -- Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X 11.0.11103000 http://www.childpsy.net/ http://honestreporting.com http://pmw.org.il http://www.PetitionOnline.com/tap12009/ A number problem solved with floats turns into 1.9998 problems. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Gregory (Greg) L. Snow Ph.D. 538...@gmail.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Help reading matrixmarket files
On 28.12.2012 17:28, Brad Cox wrote: Could someone please approve this message? Thanks! Approve? On Thu, Dec 27, 2012 at 4:38 PM, Brad Cox bradj...@gmail.com wrote: I'm trying to read data a program produces in matrixmarket array format into R and its giving me fits. I've tried read.MM (below) and readMM (from the Matrix package) but neither works. One of them says array format isn't supported, the other reports something indecipherable about Fortran. Here's the file contents. Can't get much simpler than that! %%MatrixMarket matrix array real general %This file contains LIBFM global mean which is required for computing predictions. 1 1 1.786238830716e-02 Can someone please help? Seems such a simple problem. library(spam) Spam version 0.29-2 (2012-08-17) is loaded. Type 'help( Spam)' or 'demo( spam)' for a short introduction and overview of this package. Help for individual functions is also obtained by adding the suffix '.spam' to the function name, e.g. 'help( chol.spam)'. Attaching package: ‘spam’ The following object(s) are masked from ‘package:base’: backsolve, forwardsolve x=read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm) Error in read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm) : invalid mode (list) to pass to Fortran (arg 8) If read.MM does not work and you are sure about the file location and it is in the right format, send an example file that reproduced the error to the maintainer of the package with the broken import function (or right away patched version of the broken function). Uwe Ligges __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] R crashing inconsistently within for loops
Sometimes these intermittent crashes come from memory misuse, e.g., not allocating enough scratch space. You can sometimes make those coding errors cause more consistent problems by calling gctorture(TRUE) before running your code. Here is an example in which it looks like package:gam's lo() is misusing memory when its degree argument is 2 - sometimes it can do 10 iterations, sometimes 3, sometimes 1: library(gam) Loading required package: splines Loaded gam 1.06.2 v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), data=mtcars)}) 1 2 3 4 5 6 7 8 9 10 v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), data=mtcars)}) 1 2 3 Error in sys.call() : invalid 'which' argument v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), data=mtcars)}) 1 Error in sys.call() : invalid 'which' argument If I call gctorture(TRUE) before calling lo(degree=2) then it hangs on the first call. One could attach a debugger at this point to get a clue about where it is failing. library(gam) Loading required package: splines Loaded gam 1.06.2 gctorture(TRUE) v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), data=mtcars)}) 1 Using valgrind is helpful also. Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Steve Powers Sent: Thursday, December 27, 2012 7:01 PM To: r-help@r-project.org Subject: [R] R crashing inconsistently within for loops Hello, This one has been bugging me for a long time and I have never found a solution. I am using R version 2.15.1 but it has come up in older versions of R I have used over the past 2-3 years. Q: Am I wrong to expect that R should handle hundreds of iterations of the base model or statistical functions, embedded within for loops, in one script run? I have found that when I write scripts that do this, sometimes they have a tendency to crash, seemingly unpredictably. For example, one problem script of mine employs glm and gls about a hundred different times, and output files are being written at the end of each iteration. I have used my output files to determine that the crash cause is not consistent (R never fails at the same iteration). Note that the data are fixed here (no data generation or randomization steps, so that is not the issue). But it is clear that scripts with larger numbers of iterations are more likely to produce a crash. And a year or two ago, I had a seemingly stable R script again with for looped model fits, but discovered this script was prone to crashing when I ran it on a newer PC. Because the new PC also seemed to be blazing through R code absurdly fast, I tried adding a short fluff procedure at the end of each iteration that required a few seconds of processing time. Low and behold, when I added that, the script stopped crashing (and each iteration of course took longer). I still don't understand why that fixed things. What is going on? Solutions? Thanks.---steve -- Steve Powers power...@nd.edu University of Notre Dame Environmental Change Initiative website (http://www.nd.edu/~spowers2/index.htm) __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Using grImport to create a watermark
Hi I want to use grImport to create a watermark on a plot() using the methods Paul Murrell describes here: http://cran.r-project.org/web/packages/grImport/vignettes/import.pdf (page 28). I can essentially reproduce this manually at the R prompt, and independently I can use grid.picture( ) successfully in a R script, but when I attempt to do do this in my script: postscript(outputFilename,paper=us) grid.picture(noaalogo,distort=FALSE,width=0.5,x=0.50,y=0.50) grid.rect(gp=gpar(fill=rgb(1,1,1,0.9))) plot(dat,xaxt=n,type=b,ylim=c(y_range[1],y_range[2]),main=labelStr,xlab=Date,ylab=Elevation, NAVD88 (Ft)) dev.off() The picture noaalogo is drawn, but either the plot() is obscured or is not drawn at all. In either case, the watermarking effect is not happening in my script, but does work as expected, manually, at the R prompt. I'm sure this is a graphics device related issue, but I have not been able to find a solution through my searches. Thank you, Tom -- Thomas E Adams Development Operations Hydrologist National Weather Service Ohio River Forecast Center 1901 South State Route 134 Wilmington, OH 45177 http://www.erh.noaa.gov/er/ohrfc/ EMAIL: thomas.ad...@noaa.gov VOICE: 937-383-0528 FAX:937-383-0033 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] multiple sheet in r
Dear useRs, i want to open an excel file having 59 sheets in r and GOD knows the number of command i tried. unfortunately none worked, obviously error will be on my part. All the sheets contain 1 kind of data and the data starts from A12:M12 (which contains the header of the data), the line immediately under the header is blank. The number of columns of data in each sheet are same while the number of row are different as length of data is different. you help is required.. regardseliza [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Any simple way to make this happen?
Hi, The data is x - c(4,3,5); I need to translate it to y - c(1,2,3,4,1,2,3,1,2,3,4,5); i.e. for each number in x, we need to generate 1:x and put it in y. The program need to evaluate this type of calculation for millions of times in simulation. Is there any elegant way to make this happen without iteration? Many thanks!! Best, Haoda [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Any simple way to make this happen?
x - c(4,3,5) y - c(1,2,3,4,1,2,3,1,2,3,4,5) all.equal(y, sequence(x)) [1] TRUE Bill Dunlap Spotfire, TIBCO Software wdunlap tibco.com -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Haoda Fu Sent: Friday, December 28, 2012 9:43 AM To: r-help@r-project.org Cc: haod...@yahoo.com Subject: [R] Any simple way to make this happen? Hi, The data is x - c(4,3,5); I need to translate it to y - c(1,2,3,4,1,2,3,1,2,3,4,5); i.e. for each number in x, we need to generate 1:x and put it in y. The program need to evaluate this type of calculation for millions of times in simulation. Is there any elegant way to make this happen without iteration? Many thanks!! Best, Haoda [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] multiple sheet in r
If you want help from us mere mortals then it would help us to help you if you could tell us not only the number of commands that you tried but what the actual commands were. Also the phrase none worked does not help us help you, see fortune(324). A description of what happened and how the results (if any) differed from what you expected would help. I would suggest trying the XLConnect package, but that may be what you tried already. On Fri, Dec 28, 2012 at 10:37 AM, eliza botto eliza_bo...@hotmail.comwrote: Dear useRs, i want to open an excel file having 59 sheets in r and GOD knows the number of command i tried. unfortunately none worked, obviously error will be on my part. All the sheets contain 1 kind of data and the data starts from A12:M12 (which contains the header of the data), the line immediately under the header is blank. The number of columns of data in each sheet are same while the number of row are different as length of data is different. you help is required.. regardseliza [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Gregory (Greg) L. Snow Ph.D. 538...@gmail.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Any simple way to make this happen?
It works like a magic~!! Many thanks A.K. Best, Haoda From: arun smartpink...@yahoo.com Cc: R help r-help@r-project.org Sent: Friday, December 28, 2012 12:51 PM Subject: Re: [R] Any simple way to make this happen? Hi, Try: y-paste(c,(,paste(unlist(sapply(x,seq)),collapse=,),),sep=) y #[1] c(1,2,3,4,1,2,3,1,2,3,4,5) A.K. - Original Message - To: r-help@r-project.org r-help@r-project.org Cc: haod...@yahoo.com haod...@yahoo.com Sent: Friday, December 28, 2012 12:43 PM Subject: [R] Any simple way to make this happen? Hi, The data is x - c(4,3,5); I need to translate it to y - c(1,2,3,4,1,2,3,1,2,3,4,5); i.e. for each number in x, we need to generate 1:x and put it in y. The program need to evaluate this type of calculation for millions of times in simulation. Is there any elegant way to make this happen without iteration? Many thanks!! Best, Haoda [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Any simple way to make this happen?
The only way that I can think of that would not use iteration would be to create some type of lookup table with every possible vector you may ever use, then lookup the results in that table. If you are happy with internal iteration then one possibility is: x - c(4,3,5) unlist(lapply(x,seq_len)) [1] 1 2 3 4 1 2 3 1 2 3 4 5 On Fri, Dec 28, 2012 at 10:43 AM, Haoda Fu fu...@yahoo.com.cn wrote: Hi, The data is x - c(4,3,5); I need to translate it to y - c(1,2,3,4,1,2,3,1,2,3,4,5); i.e. for each number in x, we need to generate 1:x and put it in y. The program need to evaluate this type of calculation for millions of times in simulation. Is there any elegant way to make this happen without iteration? Many thanks!! Best, Haoda [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Gregory (Greg) L. Snow Ph.D. 538...@gmail.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] vectorization modifying globals in functions
On Dec 27, 2012, at 12:38 PM, Sam Steingold wrote: I have the following code: --8---cut here---start-8--- d - rep(10,10) for (i in 1:100) { a - sample.int(length(d), size = 2) if (d[a[1]] = 1) { d[a[1]] - d[a[1]] - 1 d[a[2]] - d[a[2]] + 1 } } --8---cut here---end---8--- it does what I want, i.e., modified vector d 100 times. Now, if I want to repeat this 1e6 times instead of 1e2 times, I want to vectorize it for speed, so I do this: You could get some modest improvement by vectorizing the two lookups, additions, and assignments into one: d[a] - d[a]-c(1,-1) In a test with 10 iterations, it yields about a 1.693/1.394 -1 = 21 percent improvement. --8---cut here---start-8--- update - function (i) { a - sample.int(n.agents, size = 2) if (d[a[1]] = delta) { d[a[1]] - d[a[1]] - 1 d[a[2]] - d[a[2]] + 1 } entropy(d, unit=log2) The `unit` seems likely to throw an error since there is no argument for it to match. } system.time(entropy.history - sapply(1:1e6,update)) --8---cut here---end---8--- however, the global d is not modified, apparently update modifies the local copy. You could have returned 'd' and the entropy result as a list. But what would be the point of saving 1e6 copies so, 1. is there a way for a function to modify a global variable? So if you replaced it in the global environment, you would only be seeing the result of the last iteration of the loop. What's the use of that 2. how would you vectorize this loop? thanks! -- David Winsemius, MD Alameda, CA, USA __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] multiple sheet in r
Dear Arun and Greg,extremly sorry for asking you into something which was quite easy...the following command worked library(XLConnect)sheets - list()for(x in 1:59) {sheets[[x]] - readWorksheetFromFile(PAK.xls, sheet=x[i],region=A12:M1)} regardselisa Date: Fri, 28 Dec 2012 10:56:43 -0700 Subject: Re: [R] multiple sheet in r From: 538...@gmail.com To: eliza_bo...@hotmail.com CC: r-help@r-project.org If you want help from us mere mortals then it would help us to help you if you could tell us not only the number of commands that you tried but what the actual commands were. Also the phrase none worked does not help us help you, see fortune(324). A description of what happened and how the results (if any) differed from what you expected would help. I would suggest trying the XLConnect package, but that may be what you tried already. On Fri, Dec 28, 2012 at 10:37 AM, eliza botto eliza_bo...@hotmail.com wrote: Dear useRs, i want to open an excel file having 59 sheets in r and GOD knows the number of command i tried. unfortunately none worked, obviously error will be on my part. All the sheets contain 1 kind of data and the data starts from A12:M12 (which contains the header of the data), the line immediately under the header is blank. The number of columns of data in each sheet are same while the number of row are different as length of data is different. you help is required.. regardseliza [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Gregory (Greg) L. Snow Ph.D. 538...@gmail.com [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Help reading matrixmarket files
On Fri, Dec 28, 2012 at 11:47 AM, Uwe Ligges lig...@statistik.tu-dortmund.de wrote: Approve? Isn't it still being queued for approval? Can't tell from this end. Your mail to 'R-help' with the subject Help reading matrixmarket files Is being held until the list moderator can review it for approval. The reason it is being held: Post to moderated list [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Help reading matrixmarket files
On 28.12.2012 19:49, Brad Cox wrote: On Fri, Dec 28, 2012 at 11:47 AM, Uwe Ligges lig...@statistik.tu-dortmund.de mailto:lig...@statistik.tu-dortmund.de wrote: Approve? Isn't it still being queued for approval? Can't tell from this end. Your mail to 'R-help' with the subject Help reading matrixmarket files Is being held until the list moderator can review it for approval. The reason it is being held: Post to moderated list The approval happened before, obviously, if you were not on the list yourself. There is no need to ask for that separately and generate even more traffic. Best, Uwe Ligges __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Using grImport to create a watermark
I suspect you need a device supporting translucency: PostScript does not and hence postscript() cannot. Try the pdf() device (and convert the output if you need it). On 28/12/2012 17:23, Thomas Adams - NOAA Federal wrote: Hi… I want to use grImport to create a watermark on a plot() using the methods Paul Murrell describes here: http://cran.r-project.org/web/packages/grImport/vignettes/import.pdf (page 28). I can essentially reproduce this manually at the R prompt, and On what OS and what device: see the posting guide independently I can use grid.picture(…) successfully in a R script, but when I attempt to do do this in my script: • • • postscript(outputFilename,paper=us) grid.picture(noaalogo,distort=FALSE,width=0.5,x=0.50,y=0.50) grid.rect(gp=gpar(fill=rgb(1,1,1,0.9))) plot(dat,xaxt=n,type=b,ylim=c(y_range[1],y_range[2]),main=labelStr,xlab=Date,ylab=Elevation, NAVD88 (Ft)) • • • dev.off() The picture noaalogo is drawn, but either the plot() is obscured or is not drawn at all. In either case, the watermarking effect is not happening in my script, but does work as expected, manually, at the R prompt. I'm sure this is a graphics device related issue, but I have not been able to find a solution through my searches. Thank you, Tom __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] multiple sheet in r
HI, Check this link: http://stackoverflow.com/questions/6894922/how-to-read-multiple-excel-sheets-in-r-programming http://theweiluo.wordpress.com/2011/09/07/how-to-read-a-multiple-sheet-excel-file-into-r/ A.K. - Original Message - From: eliza botto eliza_bo...@hotmail.com To: r-help@r-project.org r-help@r-project.org Cc: Sent: Friday, December 28, 2012 12:37 PM Subject: [R] multiple sheet in r Dear useRs, i want to open an excel file having 59 sheets in r and GOD knows the number of command i tried. unfortunately none worked, obviously error will be on my part. All the sheets contain 1 kind of data and the data starts from A12:M12 (which contains the header of the data), the line immediately under the header is blank. The number of columns of data in each sheet are same while the number of row are different as length of data is different. you help is required.. regardseliza [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] comparison of large data set
Hi Jean, thank-you. It was my fault on not ccing R-help on the previous correspondence. With regards to the FALSE/TRUE or 0/1, you are right that they are the same logical identities, but it better suits our original numerical data to display 0s and 1s. I want to thank you for correcting the 2 mistakes, the code works now. Is it possible to label each of the 54 matrices so that justbig lists the 31 approved matrices along with their name [whatever the name is for the 31 matrices out of the 54 total matrices]? For this code to be fully applied to this project we have to know which site locations are successful ( 0.7) and which ones are not ( 0.7). Thank-you Jean. Irucka -Original Message- From: Adams, Jean [jvad...@usgs.gov] Sent: 12/21/2012 1:25:24 PM To: iruc...@mail2world.com Cc: r-help@r-project.org Subject: Re: [R] comparison of large data set Irucka, You should cc R-help on all correspondence so that other readers of the list can follow the conversation. (1) You say that you need 0s and 1s rather than TRUE/FALSE. Since a 0/1 matrix and TRUE/FALSE matrix behave exactly the same way in most applications, I'm not sure why it would matter which one you had. (2) My mistake. I had not noticed that you were eliminating the case where both the observed and the modeled were FALSE.. I have modified my code to give the same results as your f2 function. compare.all - sapply(modeldepth, function(md) mean((md==observeddepth)[md | observeddepth])) (3) My mistake again. There should be only single brackets, not double brackets. justbig - modeldepth[compare.all 0.7] Jean On Fri, Dec 21, 2012 at 11:27 AM, Irucka Embry iruc...@mail2world.com wrote: Hi Jean Adams, how are you? I want to thank you for your response to my request for assistance. I received some assistance yesterday afternoon and I was able to update the code which is posted here: http://r.789695.n4.nabble.com/variable-names-in-numeric-list-and-Bayesi an-inference-td4653674.html. I posted the new code with some new questions that I have with regards to the code that I have written. Can you look over that post and suggest any code revisions for those aspects that do not work? Thank-you. The code that you suggested worked well overall, except for 3 aspects of it: Here I actually needed the binary 0s and 1s rather than a TRUE/FALSE logical matrix # a function to read in the data as a matrix of logicals myreadfun - function(file) { as.matrix(read.ascii.grid(file)$data)!=0 } Here I needed to calculate the f2 probability rather than the mean compare.all - sapply(modeldepths, function(md) mean(md==observeddepth)) str(compare.all) num [1:54] 0.936 0.94 0.944 0.944 0.945 ... Here most of the entries are greater than 0.7, but it should just be 31 of the 54 that are greater than 0.7 justbig - modeldepths[[compare.all 0.7]] Error in modeldepths[[compare.all 0.7]] : recursive indexing failed at level 2 Once again, thank-you for your assistance. Irucka Embry -Original Message- From: Adams, Jean [jvad...@usgs.gov] Sent: 12/21/2012 10:32:54 AM To: iruc...@mail2world.com Cc: r-help@r-project.org Subject: Re: [R] comparison of large data set Irucka, I did not test this code out on any data, but I think it will work. Jean # a function to read in the data as a matrix of logicals myreadfun - function(file) { as.matrix(read.ascii.grid(file)$data)!=0 } # names of the 54 modeled depth files modfiles - paste0(MaxFloodDepth_, 1:54, .txt) # read in the observed and modeled depth files # observeddepth is a matrix observeddepth - myreadfun(MaxFloodDepth_Observed.txt) # modeldepths is a list of matrices modeldepths - lapply(filenames, myreadfun) # calculate the proportion of matrix elements that agree with the observed file # the results is a vector with one number for each modeled depth matrix compare.all - sapply(modeldepths, function(md) mean(md==observeddepth)) # select just those matrices that had a large proportion of agreements # justbig is a list of matrices justbig - modeldepths[[compare.all 0.7]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. span id=m2wTlpfont face=Arial, Helvetica, sans-serif size=2 style=font-size:13.5px___BRGet the Free email that has everyone talking at a href=http://www.mail2world.com target=newhttp://www.mail2world.com/abr font color=#99Unlimited Email Storage #150; POP3 #150; Calendar #150; SMS #150; Translator #150; Much More!/font/font/span [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide
Re: [R] Any simple way to make this happen?
Hi, Try: y-paste(c,(,paste(unlist(sapply(x,seq)),collapse=,),),sep=) y #[1] c(1,2,3,4,1,2,3,1,2,3,4,5) A.K. - Original Message - From: Haoda Fu fu...@yahoo.com.cn To: r-help@r-project.org r-help@r-project.org Cc: haod...@yahoo.com haod...@yahoo.com Sent: Friday, December 28, 2012 12:43 PM Subject: [R] Any simple way to make this happen? Hi, The data is x - c(4,3,5); I need to translate it to y - c(1,2,3,4,1,2,3,1,2,3,4,5); i.e. for each number in x, we need to generate 1:x and put it in y. The program need to evaluate this type of calculation for millions of times in simulation. Is there any elegant way to make this happen without iteration? Many thanks!! Best, Haoda [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Merging data tables
It is also possible to construct the r call in the GUI. See: http://www.deducer.org/pmwiki/index.php?n=Main.MergeData Ian Sent from my iPhone On Dec 28, 2012, at 4:41 AM, Djordje Bajic je.li@gmail.com wrote: see ?merge, for your dataframes, the following should work: merge(bat_activity, weather, by=c(date, time), all=T) 2012/12/28 Neotropical bat risk assessments neotropical.b...@gmail.com Hi all, I am trying to merge several data sets and end up with a long data format by date time so I can run correlations and plots. I am using Deducer as an R GUI but can just use the R console if easier. The data sets are weather with wind speed, relative humidity and temperatures by date and minute and bat activity with date, time, label, and an activity index number. The bat activity is only during the nocturnal time frames while the weather data was recorded for 24 hours. Therefore a lot of weather data with no related activity for bats. I have failed so far to achieve what I need and tried plyr and reshape2. There are many Null data rows with no data or 0 (zero) for wind speed.. What other tools steps would be more appropriate short of manually in Excel cutting and pasting for a day or more? Data formats are: bat activity DateTimeLabel Number 6/3/201110:01 Tadbra 2 6/3/201110:02 Tadbra 4 6/3/201110:08 Tadbra 1 6/3/201110:13 Tadbra 2 6/3/201110:49 Tadbra 2 6/3/201110:51 Tadbra 2 6/3/201110:52 Tadbra 4 Weather: datetimeTemp_I Temp_E RH mph_a mph_x 6/3/20110:0015 15.730.44.4 15.5 6/3/20110:3015 15.231.65.7 11.2 6/3/20111:0015 15.131.310.317.5 6/3/20111:3014 13.644.54.7 12.1 6/3/20112:0012.513.237.92.1 6.5 6/3/20112:3012.513.535.36.3 10.1 6/3/20113:0012 12.137.73 7.4 6/3/20113:3011.511.538.73.4 6 6/3/20114:0010 9.9 52.71.4 4.2 6/3/20114:309.5 9.1 43.21.3 3.8 6/3/20115:008 8.7 59.21.2 3.1 6/3/20115:307 8 62.51.1 4.2 6/3/20116:006 7.8 47.80.5 2.2 6/3/20116:307.5 11.537.51.7 3.8 6/3/20117:0010.514 33.10.6 2.2 6/3/20117:3014 17.332.11.6 3.6 6/3/20118:0017.520.323.90.4 2 6/3/20118:3021.522.820.70.4 1.8 6/3/20119:0024.524.914.10.3 2.2 6/3/20119:3026 26.920.31.7 5.6 6/3/201110:00 27.527.420.72.5 6.5 6/3/201110:30 28.529.810 1.6 4.2 6/3/201111:00 29.529.49.8 1.8 4 6/3/201111:30 30 29.99.2 2.7 7.8 Tnx for any suggestions, Bruce [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Using grImport to create a watermark
Prof Ripley, I see that now; I can get the transparency to work, but even with using pdf() my plot still does not draw if I first use: grid.picture(noaalogo,distort=**FALSE,width=0.5,x=0.50,y=0.50) grid.rect(gp=gpar(fill=rgb(1,**1,1,0.9))) Regards, Tom On Fri, Dec 28, 2012 at 1:55 PM, Prof Brian Ripley rip...@stats.ox.ac.ukwrote: I suspect you need a device supporting translucency: PostScript does not and hence postscript() cannot. Try the pdf() device (and convert the output if you need it). On 28/12/2012 17:23, Thomas Adams - NOAA Federal wrote: Hi I want to use grImport to create a watermark on a plot() using the methods Paul Murrell describes here: http://cran.r-project.org/web/**packages/grImport/vignettes/**import.pdfhttp://cran.r-project.org/web/packages/grImport/vignettes/import.pdf(page 28). I can essentially reproduce this manually at the R prompt, and On what OS and what device: see the posting guide independently I can use grid.picture( ) successfully in a R script, but when I attempt to do do this in my script: postscript(outputFilename,**paper=us) grid.picture(noaalogo,distort=**FALSE,width=0.5,x=0.50,y=0.50) grid.rect(gp=gpar(fill=rgb(1,**1,1,0.9))) plot(dat,xaxt=n,type=b,**ylim=c(y_range[1],y_range[2]),** main=labelStr,xlab=Date,**ylab=Elevation, NAVD88 (Ft)) dev.off() The picture noaalogo is drawn, but either the plot() is obscured or is not drawn at all. In either case, the watermarking effect is not happening in my script, but does work as expected, manually, at the R prompt. I'm sure this is a graphics device related issue, but I have not been able to find a solution through my searches. Thank you, Tom __** R-help@r-project.org mailing list https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/** posting-guide.html http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Brian D. Ripley, rip...@stats.ox.ac.uk Professor of Applied Statistics, http://www.stats.ox.ac.uk/~**ripley/http://www.stats.ox.ac.uk/~ripley/ University of Oxford, Tel: +44 1865 272861 (self) 1 South Parks Road, +44 1865 272866 (PA) Oxford OX1 3TG, UKFax: +44 1865 272595 __** R-help@r-project.org mailing list https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/** posting-guide.html http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Thomas E Adams Development Operations Hydrologist National Weather Service Ohio River Forecast Center 1901 South State Route 134 Wilmington, OH 45177 http://www.erh.noaa.gov/er/ohrfc/ EMAIL: thomas.ad...@noaa.gov VOICE: 937-383-0528 FAX:937-383-0033 [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] comparison of large data set
Irucka, You could assign names to the compare.all list ... for example ... names(compare.all) - paste0(Obs, 1:54) Then, when you create the subset list, justbig, it will have the appropriate names. If you just want to see the indices of the successful locations, you could print (1:54)[compare.all 0.7] Jean On Fri, Dec 28, 2012 at 11:10 AM, Irucka Embry iruc...@mail2world.comwrote: Hi Jean, thank-you. It was my fault on not ccing R-help on the previous correspondence. With regards to the FALSE/TRUE or 0/1, you are right that they are the same logical identities, but it better suits our original numerical data to display 0s and 1s. I want to thank you for correcting the 2 mistakes, the code works now. Is it possible to label each of the 54 matrices so that justbig lists the 31 approved matrices along with their name [whatever the name is for the 31 matrices out of the 54 total matrices]? For this code to be fully applied to this project we have to know which site locations are successful ( 0.7) and which ones are not ( 0.7). Thank-you Jean. Irucka -Original Message- From: Adams, Jean [jvad...@usgs.gov] Sent: 12/21/2012 1:25:24 PM To: iruc...@mail2world.com Cc: r-help@r-project.org Subject: Re: [R] comparison of large data set Irucka, You should cc R-help on all correspondence so that other readers of the list can follow the conversation. (1) You say that you need 0s and 1s rather than TRUE/FALSE. Since a 0/1 matrix and TRUE/FALSE matrix behave exactly the same way in most applications, I'm not sure why it would matter which one you had. (2) My mistake. I had not noticed that you were eliminating the case where both the observed and the modeled were FALSE.. I have modified my code to give the same results as your f2 function. compare.all - sapply(modeldepth, function(md) mean((md==observeddepth)[md | observeddepth])) (3) My mistake again. There should be only single brackets, not double brackets. justbig - modeldepth[compare.all 0.7] Jean On Fri, Dec 21, 2012 at 11:27 AM, Irucka Embry iruc...@mail2world.com wrote: Hi Jean Adams, how are you? I want to thank you for your response to my request for assistance. I received some assistance yesterday afternoon and I was able to update the code which is posted here: http://r.789695.n4.nabble.com/variable-names-in-numeric-list-and-Bayesian-inference-td4653674.html . I posted the new code with some new questions that I have with regards to the code that I have written. Can you look over that post and suggest any code revisions for those aspects that do not work? Thank-you. The code that you suggested worked well overall, except for 3 aspects of it: Here I actually needed the binary 0s and 1s rather than a TRUE/FALSE logical matrix # a function to read in the data as a matrix of logicals myreadfun - function(file) { as.matrix(read.ascii.grid(file)$data)!=0 } Here I needed to calculate the f2 probability rather than the mean compare.all - sapply(modeldepths, function(md) mean(md==observeddepth)) str(compare.all) num [1:54] 0.936 0.94 0.944 0.944 0.945 ... Here most of the entries are greater than 0.7, but it should just be 31 of the 54 that are greater than 0.7 justbig - modeldepths[[compare.all 0.7]] Error in modeldepths[[compare.all 0.7]] : recursive indexing failed at level 2 Once again, thank-you for your assistance. Irucka Embry -Original Message- From: Adams, Jean [jvad...@usgs.gov] Sent: 12/21/2012 10:32:54 AM To: iruc...@mail2world.com Cc: r-help@r-project.org Subject: Re: [R] comparison of large data set Irucka, I did not test this code out on any data, but I think it will work. Jean # a function to read in the data as a matrix of logicals myreadfun - function(file) { as.matrix(read.ascii.grid(file)$data)!=0 } # names of the 54 modeled depth files modfiles - paste0(MaxFloodDepth_, 1:54, .txt) # read in the observed and modeled depth files # observeddepth is a matrix observeddepth - myreadfun(MaxFloodDepth_Observed.txt) # modeldepths is a list of matrices modeldepths - lapply(filenames, myreadfun) # calculate the proportion of matrix elements that agree with the observed file # the results is a vector with one number for each modeled depth matrix compare.all - sapply(modeldepths, function(md) mean(md==observeddepth)) # select just those matrices that had a large proportion of agreements # justbig is a list of matrices justbig - modeldepths[[compare.all 0.7]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. ___ Get the
[R] parallel error message extraction (in mclapply)?
dear R experts---I am looking at a fairly uninformative error in my program: Error in mclapply(1:nrow(opts), solveme) : (converted from warning) all scheduled cores encountered errors in user code the doc on ?mclapply tells me that In addition, each process is running the job inside try(..., silent=TRUE) so if error occur they will be stored as try-error objects in the list. I looked up try, but this manpage creates as many new riddles as it solves. so, it isn't obvious to me how I get to the try-error object list, or how to find out what triggered the abort. of course, I can use lapply to debug this, but this could be slow for some programs. (I know the answer for my problem is that a uniroot deep inside my code complained that my endpoints were not opposite.) in general, is there a way to get some more information from mclapply failures? could this be added to the docs for mclapply? /iaw Ivo Welch (ivo.we...@gmail.com) http://www.ivo-welch.info/ __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] R crashing inconsistently within for loops
Steve Powers powers_s at nd.edu writes: Hello, This one has been bugging me for a long time and I have never found a solution. I am using R version 2.15.1 but it has come up in older versions of R I have used over the past 2-3 years. Q: Am I wrong to expect that R should handle hundreds of iterations of the base model or statistical functions, embedded within for loops, in one script run? I have found that when I write scripts that do this, sometimes they have a tendency to crash, seemingly unpredictably. For example, one problem script of mine employs glm and gls about a hundred different times, and output files are being written at the end of each iteration. I have used my output files to determine that the crash cause is not consistent (R never fails at the same iteration). Note that the data are fixed here (no data generation or randomization steps, so that is not the issue). But it is clear that scripts with larger numbers of iterations are more likely to produce a crash. And a year or two ago, I had a seemingly stable R script again with for looped model fits, but discovered this script was prone to crashing when I ran it on a newer PC. Because the new PC also seemed to be blazing through R code absurdly fast, I tried adding a short fluff procedure at the end of each iteration that required a few seconds of processing time. Low and behold, when I added that, the script stopped crashing (and each iteration of course took longer). I still don't understand why that fixed things. All of the advice given so far is useful (I think), but I thought I would chime in and answer one of your questions, which is that this is indeed surprising, especially if you're using only base packages. I and many other people routinely run thousands of iterations of these types of analyses with no problem. If (as Brian Ripley suggested) crash just means that some of your scripts stop with errors in some cases, then that's *not* surprising -- there are lots of ways to get glm() and gls() to give errors with slightly weird data sets. However, having R actually crash (i.e. the whole R session 'terminates abnormally', in the words of the posting guide) is quite a bit more unusual, and (if you are only using base R, not any contributed packages that may call compiled code in bad ways [or calling your own compiled code]) *always* constitutes a bug. Nondeterministic behavior in a deterministic function (i.e. no random number generation) is also unusual/surprising. The key here is finding a reproducible example, which can be tough for these kinds of problems. But it sounds like you have one, so if you can trim it down to a manageable size (see http://tinyurl.com/reproducible-000 for tips on creating reproducible examples), and give full information about your system (as suggested by another poster), this would be of great interest, especially to the people who hang out on r-de...@r-project.org . My prior probabilities are fairly strongly on the problem being something flaky about your system, but helping to identify these kinds of bugs in R is a community service ... __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Problem to load a shape file
Dear all, I've been puzzled why I not able to load a shapefile from a connection. Does anyone here can give a reasonable answer? When I try the following script I got this error: Error in getinfo.shape(fn) : Error opening SHP file #Reproduction temp - tempfile() download.file(http://metodologiapolitica.com/download/Brasil.zip,temp) shp.br - readShapePoly(unzip(temp, Brasil.shp)) unlink(temp) Just for acknowledgement, I successful can load the file whenever I open it from the disk. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.