Re: [R] Convert json data to an r dataframe

2012-12-28 Thread sp.duccio
Thank you very much, Rui!

The problem with your script is that data is now in a string format, while
I need to obtain a data.frame instead.
Do you really think that I cannot do anything else with this json data?
Thank you very much again

Duccio


2012/12/27 Rui Barradas ruipbarra...@sapo.pt

  Hello,

 Your dataset is not in tabular form, so I find it difficult to transform
 it into a data.frame, but you can see it in R with the following.


 install.packages(RJSONIO, dependencies = TRUE)
 library(RJSONIO)

 url -
 http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=;http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=
 dat - fromJSON(url)

 str(dat)  # List with 3 components

 str(dat[[1]][[1]])
 dat[[1]][[1]]

 str(dat[[1]][[2]])
 unlist(dat[[1]][[2]])

 str(dat[[1]][[3]])
 dat[[1]][[3]]


 Hope this helps,

 Rui Barradas
 Em 27-12-2012 11:55, sp.duccio escreveu:

 Hello to everybody,

 I need to convert a json dataset in an R dataframe.
 I suppose that I'd need to use rjson or rjsonio package.
 The json dataset 
 is:http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=
 It would be nice if someone can help me to create a function like the one
 below:http://lamages.blogspot.it/2011/09/accessing-and-plotting-world-bank-data.html
 that can applied to my json dataset and with all the parameter values that
 can be specifiable.
 Is there anyone who can solve this problem?

 Thanx in advance for any help!
 Duccio




 __r-h...@r-project.org mailing 
 listhttps://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.





-- 
Il messaggio trasmesso può contenere informazioni di carattere
confidenziale rivolte esclusivamente al destinatario. Ne è vietato
l'uso, la diffusione, la distribuzione o la riproduzione da parte di
altre persone e/o entità diverse da quelle specificate. Nel caso aveste
ricevuto questo messaggio per errore siete pregati di segnalarlo
immediatamente al mittente e cancellare quanto ricevuto.

This electronic mail transmission may contain confidential information
addressed only to the person (s) named. Any use, distribution, copying or
disclosure by any other person and/or entities other than the intended
recipient is prohibited. If you received this transmission in error,
please inform the sender immediately and delete the material.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Conjunction and disjunction in pubmed query

2012-12-28 Thread David Winsemius


On Dec 27, 2012, at 3:10 PM, email wrote:


Hi:

I am trying to query pubmed abstracts using the following syntax:

url= http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?;

search = paste(url, db=pubmedterm=, queryTerm1, +AND+,
queryTerm2,+OR+,queryTerm3, +OR+, queryTerm4,
[abstract]retmax=100usehistory=y, sep=)


I get an error here:

Error in paste(url, db=pubmedterm=, queryTerm1, +AND+,  
queryTerm2,  :

  object 'queryTerm1' not found




docId - xmlTreeParse(getURL(paste(url, search, sep=)),
useInternalNodes=TRUE)

I want to fetch abstracts containing queryTerm1 AND queryTerm2
Or queryTerm3 OR queryTerm4. The code runs without error, but from the
result I find that conjunction and disjunction is not working.


Not working is a phrase that conveys very little. What exactly are  
you seeing, and how is it different than what you expected to see.




Can anyone
suggest a correct  syntax for doing AND and OR pubmed query?




Reviewing an earlier question
 [Subject: Re: [R] query multiple terms in PubMed abstract
Date:   December 11, 2012]
 I find that you did not reply to my earlier response where I said  
your code was returning expected results and asked what problems were  
prompting your questions. If you do not respond to questions and fail  
to include complete code, there is little people will be able to do.  
You need to provide complete code and a more complete problem  
description.


 If I were interested in articles the had MeSH headings for  
mortality and leukemia I would first attempt constructing a search  
phrase that looked like mortality[mh] AND leukemia[mh] rather than  
mortality[mh]+AND+leukemia[mh]. But I see that the + symbol seemed  
to succeed last time. You may need to use parentheses to get the  
intended grouping of PubMed  search expressions.


--

David Winsemius, MD
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Convert json data to an r dataframe

2012-12-28 Thread Jeff Newmiller
It really seems to me that what you ought to be doing is carefully studying the 
code provided at the lamages link that you referenced until you can modify it 
to do what you want. It really isn't fair for you to ask people to do your 
thinking for you. This is the R-help mailing list, not R-do-my-work-for-me 
mailing list.

JSON, like XML, is a far more flexible data format than a data frame. That 
means it is up to you to decide what data or relationships you want to ignore 
so you can squeeze some useful subset of the information into a data frame. 
There is no way to write a do-it-all function that does this for all cases. 
Your goals must be part of each extraction function you use, so you really need 
to write it yourself.
---
Jeff NewmillerThe .   .  Go Live...
DCN:jdnew...@dcn.davis.ca.usBasics: ##.#.   ##.#.  Live Go...
  Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
/Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
--- 
Sent from my phone. Please excuse my brevity.

sp.duccio sp.duc...@gmail.com wrote:

Thank you very much, Rui!

The problem with your script is that data is now in a string format,
while
I need to obtain a data.frame instead.
Do you really think that I cannot do anything else with this json data?
Thank you very much again

Duccio


2012/12/27 Rui Barradas ruipbarra...@sapo.pt

  Hello,

 Your dataset is not in tabular form, so I find it difficult to
transform
 it into a data.frame, but you can see it in R with the following.


 install.packages(RJSONIO, dependencies = TRUE)
 library(RJSONIO)

 url -

http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=;http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=
 dat - fromJSON(url)

 str(dat)  # List with 3 components

 str(dat[[1]][[1]])
 dat[[1]][[1]]

 str(dat[[1]][[2]])
 unlist(dat[[1]][[2]])

 str(dat[[1]][[3]])
 dat[[1]][[3]]


 Hope this helps,

 Rui Barradas
 Em 27-12-2012 11:55, sp.duccio escreveu:

 Hello to everybody,

 I need to convert a json dataset in an R dataframe.
 I suppose that I'd need to use rjson or rjsonio package.
 The json dataset
is:http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=
 It would be nice if someone can help me to create a function like the
one

below:http://lamages.blogspot.it/2011/09/accessing-and-plotting-world-bank-data.html
 that can applied to my json dataset and with all the parameter values
that
 can be specifiable.
 Is there anyone who can solve this problem?

 Thanx in advance for any help!
 Duccio




 __r-h...@r-project.org
mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Convert json data to an r dataframe

2012-12-28 Thread sp.duccio
Sorry,

I really didn't want to push anyone to make my work.
Sorry if my words sounded like that.
I appreciated your help very much.
Thank you

D.

2012/12/28 Jeff Newmiller jdnew...@dcn.davis.ca.us

 It really seems to me that what you ought to be doing is carefully
 studying the code provided at the lamages link that you referenced until
 you can modify it to do what you want. It really isn't fair for you to ask
 people to do your thinking for you. This is the R-help mailing list, not
 R-do-my-work-for-me mailing list.

 JSON, like XML, is a far more flexible data format than a data frame. That
 means it is up to you to decide what data or relationships you want to
 ignore so you can squeeze some useful subset of the information into a data
 frame. There is no way to write a do-it-all function that does this for all
 cases. Your goals must be part of each extraction function you use, so you
 really need to write it yourself.
 ---
 Jeff NewmillerThe .   .  Go Live...
 DCN:jdnew...@dcn.davis.ca.usBasics: ##.#.   ##.#.  Live
 Go...
   Live:   OO#.. Dead: OO#..  Playing
 Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
 /Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
 ---
 Sent from my phone. Please excuse my brevity.

 sp.duccio sp.duc...@gmail.com wrote:

 Thank you very much, Rui!
 
 The problem with your script is that data is now in a string format,
 while
 I need to obtain a data.frame instead.
 Do you really think that I cannot do anything else with this json data?
 Thank you very much again
 
 Duccio
 
 
 2012/12/27 Rui Barradas ruipbarra...@sapo.pt
 
   Hello,
 
  Your dataset is not in tabular form, so I find it difficult to
 transform
  it into a data.frame, but you can see it in R with the following.
 
 
  install.packages(RJSONIO, dependencies = TRUE)
  library(RJSONIO)
 
  url -
 
 
 http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=
 
 http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=
 
  dat - fromJSON(url)
 
  str(dat)  # List with 3 components
 
  str(dat[[1]][[1]])
  dat[[1]][[1]]
 
  str(dat[[1]][[2]])
  unlist(dat[[1]][[2]])
 
  str(dat[[1]][[3]])
  dat[[1]][[3]]
 
 
  Hope this helps,
 
  Rui Barradas
  Em 27-12-2012 11:55, sp.duccio escreveu:
 
  Hello to everybody,
 
  I need to convert a json dataset in an R dataframe.
  I suppose that I'd need to use rjson or rjsonio package.
  The json dataset
 is:
 http://apistat.istat.it/?q=getdatajsondataset=DCIS_POPSTRBILdim=1,0,0,0lang=1tr=te=
  It would be nice if someone can help me to create a function like the
 one
 
 below:
 http://lamages.blogspot.it/2011/09/accessing-and-plotting-world-bank-data.html
  that can applied to my json dataset and with all the parameter values
 that
  can be specifiable.
  Is there anyone who can solve this problem?
 
  Thanx in advance for any help!
  Duccio
 
 
 
 
  __r-h...@r-project.org
 mailing listhttps://stat.ethz.ch/mailman/listinfo/r-help
  PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
  and provide commented, minimal, self-contained, reproducible code.
 
 
 




-- 
Il messaggio trasmesso può contenere informazioni di carattere
confidenziale rivolte esclusivamente al destinatario. Ne è vietato
l'uso, la diffusione, la distribuzione o la riproduzione da parte di
altre persone e/o entità diverse da quelle specificate. Nel caso aveste
ricevuto questo messaggio per errore siete pregati di segnalarlo
immediatamente al mittente e cancellare quanto ricevuto.

This electronic mail transmission may contain confidential information
addressed only to the person (s) named. Any use, distribution, copying or
disclosure by any other person and/or entities other than the intended
recipient is prohibited. If you received this transmission in error,
please inform the sender immediately and delete the material.

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] R crashing inconsistently within for loops

2012-12-28 Thread Jeff Newmiller
You are not wrong to expect R to not crash. However, R (as most people use it) 
is not monolithic, and you have provided neither reproducible code nor 
sessionInfo() with the relevant packages loaded to help anyone interested in 
investigating the problem. You are the most likely person to be able to 
generate sample code that reproduces your problem, even if imperfectly.
---
Jeff NewmillerThe .   .  Go Live...
DCN:jdnew...@dcn.davis.ca.usBasics: ##.#.   ##.#.  Live Go...
  Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
/Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
--- 
Sent from my phone. Please excuse my brevity.

Steve Powers power...@nd.edu wrote:

Hello,

This one has been bugging me for a long time and I have never found a
solution. I am using R version 2.15.1 but it has come up in older
versions of R I have used over the past 2-3 years.

Q: Am I wrong to expect that R should handle hundreds of iterations of
the
base model or statistical functions, embedded within for loops, in one
script run? I have found that when I write scripts that do this,
sometimes
they have a tendency to crash, seemingly unpredictably.

For example, one problem script of mine employs glm and gls about a
hundred
different times, and output files are being written at the end of each
iteration. I have used my output files to determine that the crash
cause is
not consistent (R never fails at the same iteration). Note that the
data are
fixed here (no data generation or randomization steps, so that is not
the
issue). But it is clear that scripts with larger numbers of iterations
are
more likely to produce a crash.

And a year or two ago, I had a seemingly stable R script again with for
looped model fits, but discovered this script was prone to crashing
when I
ran it on a newer PC. Because the new PC also seemed to be blazing
through R
code absurdly fast, I tried adding a short fluff procedure at the end
of
each iteration that required a few seconds of processing time. Low and
behold, when I added that, the script stopped crashing (and each
iteration
of course took longer). I still don't understand why that fixed things.

What is going on? Solutions? Thanks.---steve

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] R crashing inconsistently within for loops

2012-12-28 Thread Prof Brian Ripley
Note though that the posting guide asked you not to use the word 
'crash', as your audience has no idea what you mean by it.  In some of 
the senses people use (e.g. when R reports an error in your code), you 
should expect R to 'crash'.


On 28/12/2012 09:04, Jeff Newmiller wrote:

You are not wrong to expect R to not crash. However, R (as most people use it) 
is not monolithic, and you have provided neither reproducible code nor 
sessionInfo() with the relevant packages loaded to help anyone interested in 
investigating the problem. You are the most likely person to be able to 
generate sample code that reproduces your problem, even if imperfectly.
---
Jeff NewmillerThe .   .  Go Live...
DCN:jdnew...@dcn.davis.ca.usBasics: ##.#.   ##.#.  Live Go...
   Live:   OO#.. Dead: OO#..  Playing
Research Engineer (Solar/BatteriesO.O#.   #.O#.  with
/Software/Embedded Controllers)   .OO#.   .OO#.  rocks...1k
---
Sent from my phone. Please excuse my brevity.

Steve Powers power...@nd.edu wrote:


Hello,

This one has been bugging me for a long time and I have never found a
solution. I am using R version 2.15.1 but it has come up in older
versions of R I have used over the past 2-3 years.

Q: Am I wrong to expect that R should handle hundreds of iterations of
the
base model or statistical functions, embedded within for loops, in one
script run? I have found that when I write scripts that do this,
sometimes
they have a tendency to crash, seemingly unpredictably.

For example, one problem script of mine employs glm and gls about a
hundred
different times, and output files are being written at the end of each
iteration. I have used my output files to determine that the crash
cause is
not consistent (R never fails at the same iteration). Note that the
data are
fixed here (no data generation or randomization steps, so that is not
the
issue). But it is clear that scripts with larger numbers of iterations
are
more likely to produce a crash.

And a year or two ago, I had a seemingly stable R script again with for
looped model fits, but discovered this script was prone to crashing
when I
ran it on a newer PC. Because the new PC also seemed to be blazing
through R
code absurdly fast, I tried adding a short fluff procedure at the end
of
each iteration that required a few seconds of processing time. Low and
behold, when I added that, the script stopped crashing (and each
iteration
of course took longer). I still don't understand why that fixed things.

What is going on? Solutions? Thanks.---steve


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.




--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] Merging data tables

2012-12-28 Thread Neotropical bat risk assessments
Hi all,

I am trying to merge several data sets and end up with a long data 
format by date  time so I can run correlations and plots.  I am using 
Deducer as an R GUI but can just use the R console if easier.

The data sets are weather with wind speed, relative humidity and 
temperatures by date and minute and bat activity with date, time, label, 
and an activity index number.  The bat activity is only during the 
nocturnal time frames while the weather data was recorded for 24 hours.  
Therefore a lot of weather data with no related activity for bats.

I have failed so far to achieve what I need and tried plyr and reshape2.
There are many Null data rows with no data or 0 (zero) for wind speed..
What other tools steps would be more appropriate short of manually in 
Excel cutting and pasting for a day or more?


Data formats are:
bat activity
DateTimeLabel   Number
6/3/201110:01   Tadbra  2
6/3/201110:02   Tadbra  4
6/3/201110:08   Tadbra  1
6/3/201110:13   Tadbra  2
6/3/201110:49   Tadbra  2
6/3/201110:51   Tadbra  2
6/3/201110:52   Tadbra  4


Weather:

datetimeTemp_I  Temp_E  RH  mph_a   mph_x
6/3/20110:0015  15.730.44.4 15.5
6/3/20110:3015  15.231.65.7 11.2
6/3/20111:0015  15.131.310.317.5
6/3/20111:3014  13.644.54.7 12.1
6/3/20112:0012.513.237.92.1 6.5
6/3/20112:3012.513.535.36.3 10.1
6/3/20113:0012  12.137.73   7.4
6/3/20113:3011.511.538.73.4 6
6/3/20114:0010  9.9 52.71.4 4.2
6/3/20114:309.5 9.1 43.21.3 3.8
6/3/20115:008   8.7 59.21.2 3.1
6/3/20115:307   8   62.51.1 4.2
6/3/20116:006   7.8 47.80.5 2.2
6/3/20116:307.5 11.537.51.7 3.8
6/3/20117:0010.514  33.10.6 2.2
6/3/20117:3014  17.332.11.6 3.6
6/3/20118:0017.520.323.90.4 2
6/3/20118:3021.522.820.70.4 1.8
6/3/20119:0024.524.914.10.3 2.2
6/3/20119:3026  26.920.31.7 5.6
6/3/201110:00   27.527.420.72.5 6.5
6/3/201110:30   28.529.810  1.6 4.2
6/3/201111:00   29.529.49.8 1.8 4
6/3/201111:30   30  29.99.2 2.7 7.8


Tnx for any suggestions,

Bruce

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Dock graphs when using R in Eclipse with StatET plug In

2012-12-28 Thread Tobias Kisch
Anyone have a solution to this by now?
Unfortunately, I made some changes, and now the graphic window pops up and 
can’t be docked anymore.

Thanks for any hints!


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Merging data tables

2012-12-28 Thread Djordje Bajic
see ?merge,

for your dataframes, the following should work:

merge(bat_activity, weather, by=c(date, time), all=T)


2012/12/28 Neotropical bat risk assessments neotropical.b...@gmail.com

 Hi all,

 I am trying to merge several data sets and end up with a long data
 format by date  time so I can run correlations and plots.  I am using
 Deducer as an R GUI but can just use the R console if easier.

 The data sets are weather with wind speed, relative humidity and
 temperatures by date and minute and bat activity with date, time, label,
 and an activity index number.  The bat activity is only during the
 nocturnal time frames while the weather data was recorded for 24 hours.
 Therefore a lot of weather data with no related activity for bats.

 I have failed so far to achieve what I need and tried plyr and reshape2.
 There are many Null data rows with no data or 0 (zero) for wind speed..
 What other tools steps would be more appropriate short of manually in
 Excel cutting and pasting for a day or more?


 Data formats are:
 bat activity
 DateTimeLabel   Number
 6/3/201110:01   Tadbra  2
 6/3/201110:02   Tadbra  4
 6/3/201110:08   Tadbra  1
 6/3/201110:13   Tadbra  2
 6/3/201110:49   Tadbra  2
 6/3/201110:51   Tadbra  2
 6/3/201110:52   Tadbra  4


 Weather:

 datetimeTemp_I  Temp_E  RH  mph_a   mph_x
 6/3/20110:0015  15.730.44.4 15.5
 6/3/20110:3015  15.231.65.7 11.2
 6/3/20111:0015  15.131.310.317.5
 6/3/20111:3014  13.644.54.7 12.1
 6/3/20112:0012.513.237.92.1 6.5
 6/3/20112:3012.513.535.36.3 10.1
 6/3/20113:0012  12.137.73   7.4
 6/3/20113:3011.511.538.73.4 6
 6/3/20114:0010  9.9 52.71.4 4.2
 6/3/20114:309.5 9.1 43.21.3 3.8
 6/3/20115:008   8.7 59.21.2 3.1
 6/3/20115:307   8   62.51.1 4.2
 6/3/20116:006   7.8 47.80.5 2.2
 6/3/20116:307.5 11.537.51.7 3.8
 6/3/20117:0010.514  33.10.6 2.2
 6/3/20117:3014  17.332.11.6 3.6
 6/3/20118:0017.520.323.90.4 2
 6/3/20118:3021.522.820.70.4 1.8
 6/3/20119:0024.524.914.10.3 2.2
 6/3/20119:3026  26.920.31.7 5.6
 6/3/201110:00   27.527.420.72.5 6.5
 6/3/201110:30   28.529.810  1.6 4.2
 6/3/201111:00   29.529.49.8 1.8 4
 6/3/201111:30   30  29.99.2 2.7 7.8


 Tnx for any suggestions,

 Bruce

 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] Irreproducible exception in R extension

2012-12-28 Thread Simon Zehnder
Dear R-Users,

I am having some trouble running an R extension on our cluster (linux). I call 
C++ code in which I use a) the Scythe Statistical Library and b) OpenMP. Most 
of the jobs run without a problem, but some arbitrary jobs throw an exception 
of the kind printed below while running in a parallel loop. The behavior is to 
me not reproducible, although at every run of 729 Jobs it at least happens 
once. I already tried to enforce the error by using the Intel Compiler instead 
of the GCC 4.7.2 and using the Options '-checkinit' for possibly uninitialized 
values and '-ftrapuv' for initialization of stack variables to unusual values 
to aid error detection. But I cannot enforce the exception, which makes it 
impossible to debug. I already googled this error,  but there only 2-3 small 
forum discussions about it, and so I hope I find some suggestions here. 

1. What do you think this error comes from? 
2. What do you advice me to do to detect the error?

I am excited for your answers.

Simon




*** glibc detected *** /home/simo_109//opt/R/lib64/R/bin/exec/R: double free or 
corruption (!prev): 0x01355610 ***
=== Backtrace: =
/lib64/libc.so.6[0x3845a75916]
/lib64/libc.so.6[0x3845a78443]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdE10deallocateEv+0x7b)[0x7f4645d5d6db]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdED1Ev+0x47)[0x7f4645d5d5bb]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdE17withdrawReferenceEv+0xaa)[0x7f4645d83442]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED1Ev+0x94)[0x7f4645d82cc8]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED2Ev+0x47)[0x7f4645d82d9f]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe6MatrixIdLNS_12matrix_orderE0ELNS_12matrix_styleE1EED1Ev+0x5d)[0x7f4645d8b26d]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_Z11MCgmmS_implIN6scythe8mersenneEEvRNS0_3rngIT_EERP7SEXPRECS8_jjRNS0_6MatrixIdLNS0_12matrix_orderE0ELNS0_12matrix_styleE0EEERKSC_jS8_S8_+0x478f)[0x7f4645d6d6af]
/opt/intel/Compiler/12.1/6.361/rwthlnk/compiler/lib/intel64/libiomp5.so(__kmp_invoke_microtask+0x93)[0x7f4645aabee3]
=== Memory map: 
0040-00401000 r-xp  00:21 32550383   
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R
0060-00601000 rw-p  00:21 32550383   
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R
00601000-029cc000 rw-p  00:00 0  [heap]
369ac0-369ac3a000 r-xp  08:01 263771 
/lib64/libreadline.so.6.0
369ac3a000-369ae3a000 ---p 0003a000 08:01 263771 
/lib64/libreadline.so.6.0
369ae3a000-369ae42000 rw-p 0003a000 08:01 263771 
/lib64/libreadline.so.6.0
369ae42000-369ae43000 rw-p  00:00 0 
369bc0-369bc16000 r-xp  08:01 263757 
/lib64/libgcc_s-4.4.6-20120305.so.1
369bc16000-369be15000 ---p 00016000 08:01 263757 
/lib64/libgcc_s-4.4.6-20120305.so.1
369be15000-369be16000 rw-p 00015000 08:01 263757 
/lib64/libgcc_s-4.4.6-20120305.so.1
369c00-369c0e8000 r-xp  08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c0e8000-369c2e8000 ---p 000e8000 08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c2e8000-369c2ef000 r--p 000e8000 08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c2ef000-369c2f1000 rw-p 000ef000 08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c2f1000-369c306000 rw-p  00:00 0 
369d00-369d00d000 r-xp  08:01 671072 
/usr/lib64/libgomp.so.1.0.0
369d00d000-369d20c000 ---p d000 08:01 671072 
/usr/lib64/libgomp.so.1.0.0
369d20c000-369d20d000 rw-p c000 08:01 671072 
/usr/lib64/libgomp.so.1.0.0
369fa0-369faf r-xp  08:01 660772 
/usr/lib64/libgfortran.so.3.0.0
369faf-369fcef000 ---p 000f 08:01 660772 
/usr/lib64/libgfortran.so.3.0.0
369fcef000-369fcf1000 rw-p 000ef000 08:01 660772 
/usr/lib64/libgfortran.so.3.0.0
369fcf1000-369fcf2000 rw-p  00:00 0 
36a360-36a373f000 r-xp  08:01 675210 
/usr/lib64/libicuuc.so.42.1
36a373f000-36a393f000 ---p 0013f000 08:01 675210 
/usr/lib64/libicuuc.so.42.1
36a393f000-36a395 rw-p 0013f000 08:01 675210 
/usr/lib64/libicuuc.so.42.1
36a395-36a3952000 rw-p  00:00 0 
36a3a0-36a3b88000 r-xp 

[R] Irreproducible exception in R extension

2012-12-28 Thread Simon Zehnder
Dear R-Users,

I am having some trouble running an R extension on our cluster (linux). I call 
C++ code in which I use a) the Scythe Statistical Library and b) OpenMP. Most 
of the jobs run without a problem, but some arbitrary jobs throw an exception 
of the kind printed below while running in a parallel loop. The behavior is to 
me not reproducible, although at every run of 729 Jobs it at least happens 
once. I already tried to enforce the error by using the Intel Compiler instead 
of the GCC 4.7.2 and using the Options '-checkinit' for possibly uninitialized 
values and '-ftrapuv' for initialization of stack variables to unusual values 
to aid error detection. But I cannot enforce the exception, which makes it 
impossible to debug. I already googled this error,  but there only 2-3 small 
forum discussions about it, and so I hope I find some suggestions here. 

1. What do you think this error comes from? 
2. What do you advice me to do to detect the error?

I am excited for your answers.

Simon




*** glibc detected *** /home/simo_109//opt/R/lib64/R/bin/exec/R: double free or 
corruption (!prev): 0x01355610 ***
=== Backtrace: =
/lib64/libc.so.6[0x3845a75916]
/lib64/libc.so.6[0x3845a78443]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdE10deallocateEv+0x7b)[0x7f4645d5d6db]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe9DataBlockIdED1Ev+0x47)[0x7f4645d5d5bb]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdE17withdrawReferenceEv+0xaa)[0x7f4645d83442]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED1Ev+0x94)[0x7f4645d82cc8]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe18DataBlockReferenceIdED2Ev+0x47)[0x7f4645d82d9f]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_ZN6scythe6MatrixIdLNS_12matrix_orderE0ELNS_12matrix_styleE1EED1Ev+0x5d)[0x7f4645d8b26d]
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/library/MCpkg/libs/MCpkg.so(_Z11MCgmmS_implIN6scythe8mersenneEEvRNS0_3rngIT_EERP7SEXPRECS8_jjRNS0_6MatrixIdLNS0_12matrix_orderE0ELNS0_12matrix_styleE0EEERKSC_jS8_S8_+0x478f)[0x7f4645d6d6af]
/opt/intel/Compiler/12.1/6.361/rwthlnk/compiler/lib/intel64/libiomp5.so(__kmp_invoke_microtask+0x93)[0x7f4645aabee3]
=== Memory map: 
0040-00401000 r-xp  00:21 32550383   
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R
0060-00601000 rw-p  00:21 32550383   
/rwthfs/rz/cluster/home/simo_109/opt/R/lib64/R/bin/exec/R
00601000-029cc000 rw-p  00:00 0  [heap]
369ac0-369ac3a000 r-xp  08:01 263771 
/lib64/libreadline.so.6.0
369ac3a000-369ae3a000 ---p 0003a000 08:01 263771 
/lib64/libreadline.so.6.0
369ae3a000-369ae42000 rw-p 0003a000 08:01 263771 
/lib64/libreadline.so.6.0
369ae42000-369ae43000 rw-p  00:00 0 
369bc0-369bc16000 r-xp  08:01 263757 
/lib64/libgcc_s-4.4.6-20120305.so.1
369bc16000-369be15000 ---p 00016000 08:01 263757 
/lib64/libgcc_s-4.4.6-20120305.so.1
369be15000-369be16000 rw-p 00015000 08:01 263757 
/lib64/libgcc_s-4.4.6-20120305.so.1
369c00-369c0e8000 r-xp  08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c0e8000-369c2e8000 ---p 000e8000 08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c2e8000-369c2ef000 r--p 000e8000 08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c2ef000-369c2f1000 rw-p 000ef000 08:01 656025 
/usr/lib64/libstdc++.so.6.0.13
369c2f1000-369c306000 rw-p  00:00 0 
369d00-369d00d000 r-xp  08:01 671072 
/usr/lib64/libgomp.so.1.0.0
369d00d000-369d20c000 ---p d000 08:01 671072 
/usr/lib64/libgomp.so.1.0.0
369d20c000-369d20d000 rw-p c000 08:01 671072 
/usr/lib64/libgomp.so.1.0.0
369fa0-369faf r-xp  08:01 660772 
/usr/lib64/libgfortran.so.3.0.0
369faf-369fcef000 ---p 000f 08:01 660772 
/usr/lib64/libgfortran.so.3.0.0
369fcef000-369fcf1000 rw-p 000ef000 08:01 660772 
/usr/lib64/libgfortran.so.3.0.0
369fcf1000-369fcf2000 rw-p  00:00 0 
36a360-36a373f000 r-xp  08:01 675210 
/usr/lib64/libicuuc.so.42.1
36a373f000-36a393f000 ---p 0013f000 08:01 675210 
/usr/lib64/libicuuc.so.42.1
36a393f000-36a395 rw-p 0013f000 08:01 675210 
/usr/lib64/libicuuc.so.42.1
36a395-36a3952000 rw-p  00:00 0 
36a3a0-36a3b88000 r-xp 

Re: [R] help with reshaping wide to long format

2012-12-28 Thread arun
Hi,
DId you tried copy and paste the code in my reply?  If not, please do.  

In your previous replies, you forgot to close the  bracket()) or adding # 
in between the codes.  Please check your codes.

It would be also helpful to read the posting guide 
(http://www.r-project.org/posting-guide.html) and An introduction to R (the 
link will be in the posting guide).

A.K.




- Original Message -
From: usha2013 usha.nat...@gmail.com
To: r-help@r-project.org
Cc: 
Sent: Friday, December 28, 2012 2:02 AM
Subject: Re: [R] help with reshaping wide to long format

Hi, Sorry, but how did you bring it out?

Thanks

On Fri, Dec 28, 2012 at 8:48 AM, arun kirshna [via R] 
ml-node+s789695n4654093...@n4.nabble.com wrote:

 Hi,
 bp.sub- structure(list(CODEA = c(1L, 3L, 4L, 7L, 8L, 9L, 10L, 11L, 12L,
 13L, 14L, 16L, 17L), C45 = c(NA, 2L, 2L, 2L, 2L, 1L, NA, 1L,
 1L, 2L, 1L, 2L, 1L), ragek = c(3L, 3L, 3L, 4L, 4L, 3L, 3L, 3L,
 3L, 3L, 3L, 3L, 3L), ra80 = c(4L, 3L, 6L, 6L, 4L, 6L, 3L, 4L,
 4L, 4L, 4L, 4L, 4L), ra98 = c(1L, 1L, 1L, 1L, 1L, 2L, 1L, 1L,
 2L, 2L, 2L, 3L, 1L), CBCLAggressionAt1410 = c(NA, 0L, NA, 0L,
 0L, 0L, NA, 0L, NA, NA, 0L, 0L, NA), CBCLInternalisingAt1410 = c(NA,
 0L, NA, 0L, 0L, 0L, NA, 0L, NA, NA, 0L, 0L, NA), Obese14 = c(NA,
 0L, NA, 0L, 0L, 0L, NA, NA, NA, NA, 0L, 0L, NA), Obese21 = c(NA,
 0L, NA, 1L, 0L, 0L, NA, 0L, NA, NA, 0L, 0L, NA), Overweight14 = c(NA,
 0L, NA, 0L, 0L, 0L, NA, NA, NA, NA, 0L, 0L, NA), Overweight21 = c(NA,
 1L, NA, 1L, 0L, 0L, NA, 0L, NA, NA, 1L, 0L, NA), hibp14 = c(NA,
 0L, NA, 0L, 0L, 0L, NA, NA, NA, NA, 1L, 1L, NA), hibp21 = c(NA,
 0L, NA, 0L, 0L, 0L, NA, 0L, NA, NA, 1L, NA, NA)), .Names = c(CODEA,
 C45, ragek, ra80, ra98, CBCLAggressionAt1410,
 CBCLInternalisingAt1410,
 Obese14, Obese21, Overweight14, Overweight21, hibp14,
 hibp21), row.names = c(NA, 13L), class = data.frame)

 BP.stack1- reshape(bp.sub,sep=,timevar=
 time,direction=long,varying=list(names(bp.sub[8:9]),names(bp.sub[10:11]),names(bp.sub[12:13])),v.names=c(Obese,Overweight,HiBP),idvar=CODEA)

 BP.stack1$time - recode(BP.stack1$time,(1=14;2=21))
  BP.stack1
  #    CODEA C45 ragek ra80 ra98 CBCLAggressionAt1410
 CBCLInternalisingAt1410
 #1.1      1  NA     3    4    1                   NA
  NA
 #3.1      3   2     3    3    1                    0
 0
 #4.1      4   2     3    6    1                   NA
  NA
 #7.1      7   2     4    6    1                    0
 0
 #8.1      8   2     4    4    1                    0
 0
 #9.1      9   1     3    6    2                    0
 0
 #10.1    10  NA     3    3    1                   NA
  NA
 #11.1    11   1     3    4    1                    0
 0
 #12.1    12   1     3    4    2                   NA
  NA
 #13.1    13   2     3    4    2                   NA
  NA
 #14.1    14   1     3    4    2                    0
 0
 #16.1    16   2     3    4    3                    0
 0
 #17.1    17   1     3    4    1                   NA
  NA
 #1.2      1  NA     3    4    1                   NA
  NA
 #3.2      3   2     3    3    1                    0
 0
 #4.2      4   2     3    6    1                   NA
  NA
 #7.2      7   2     4    6    1                    0
 0
 #8.2      8   2     4    4    1                    0
 0
 #9.2      9   1     3    6    2                    0
 0
 #10.2    10  NA     3    3    1                   NA
  NA
 #11.2    11   1     3    4    1                    0
 0
 #12.2    12   1     3    4    2                   NA
  NA
 #13.2    13   2     3    4    2                   NA
  NA
 #14.2    14   1     3    4    2                    0
 0
 #16.2    16   2     3    4    3                    0
 0
 #17.2    17   1     3    4    1                   NA
  NA
   #   time Obese Overweight HiBP
 #1.1    14    NA         NA   NA
 #3.1    14     0          0    0
 #4.1    14    NA         NA   NA
 #7.1    14     0          0    0
 #8.1    14     0          0    0
 #9.1    14     0          0    0
 #10.1   14    NA         NA   NA
 #11.1   14    NA         NA   NA
 #2.1   14    NA         NA   NA
 #13.1   14    NA         NA   NA
 #14.1   14     0          0    1
 #16.1   14     0          0    1
 #17.1   14    NA         NA   NA
 #1.2    21    NA         NA   NA
 #3.2    21     0          1    0
 #4.2    21    NA         NA   NA
 #7.2    21     1          1    0
 #8.2    21     0          0    0
 #9.2    21     0          0    0
 #10.2   21    NA         NA   NA
 #11.2   21     0          0    0
 #12.2   21    NA         NA   NA
 #13.2   21    NA         NA   NA
 #14.2   21     0          1    1
 #16.2   21     0          0   NA
 #17.2   21    NA         NA   NA
 names(bp.sub)[grep(\\d+,names(bp.sub))]-gsub((\\D+)(\\d+),\\1_\\2,names(bp.sub)[grep(\\d+,names(bp.sub))])

  BP.stack2-reshape(bp.sub,dir=long,varying=8:13,sep=_)
  Bp.stack2
       CODEA C_45 ragek ra_80 ra_98 CBCLAggressionAt_1410
 1.14      1   NA     3     4     1                    NA
 2.14      3    2     3     3     1                     0
 3.14      4    2     3     6     1                  

Re: [R] Merging data tables

2012-12-28 Thread arun
HI,

YOu may try ?merge() or ?join() from library(plyr)
bat_activity-read.table(text=
Date Time Label Number
6/3/2011 10:01 Tadbra 2
6/3/2011 10:02 Tadbra 4
6/3/2011 10:08 Tadbra 1
6/3/2011 10:13 Tadbra 2
6/3/2011 10:49 Tadbra 2
6/3/2011 10:51 Tadbra 2
6/3/2011 10:52 Tadbra 4
,sep=,header=TRUE,stringsAsFactors=FALSE)

Weather-read.table(text=
date    time Temp_I Temp_E RH mph_a mph_x
6/3/2011 0:00 15 15.7 30.4 4.4 15.5
---

6/3/2011 11:00 29.5 29.4 9.8 1.8 4
6/3/2011 11:30 30 29.9 9.2 2.7 7.8
,sep=,header=TRUE,stringsAsFactors=FALSE)


bat_activity1-data.frame(dateTime=as.POSIXct(paste(bat_activity[,1],bat_activity[,2]),format=%m/%d/%Y
 %H:%M),bat_activity[,3:4])
Weather1-data.frame(dateTime=as.POSIXct(paste(Weather[,1],Weather[,2]),format=%m/%d/%Y
 %H:%M),Weather[,3:7])
res-  merge(bat_activity1,Weather1,by=dateTime,all=TRUE)
head(res)   #here there are NAs because there are no corresponding values in 
one of the dataset
# dateTime Label Number Temp_I Temp_E   RH mph_a mph_x
#1 2011-06-03 00:00:00  NA NA   15.0   15.7 30.4   4.4  15.5
#2 2011-06-03 00:30:00  NA NA   15.0   15.2 31.6   5.7  11.2
#3 2011-06-03 01:00:00  NA NA   15.0   15.1 31.3  10.3  17.5
#4 2011-06-03 01:30:00  NA NA   14.0   13.6 44.5   4.7  12.1
#5 2011-06-03 02:00:00  NA NA   12.5   13.2 37.9   2.1   6.5
#6 2011-06-03 02:30:00  NA NA   12.5   13.5 35.3   6.3  10.1

Also, you mentioned about merging more than 2 datasets. In that case, use:
Reduce(function(...) 
merge(...,by=dateTime),list(bat_activity1,Weather1,dat3,dat4))
#or
library(reshape)
merge_recurse(list(bat_activity1,Weather,dat3,dat4), by=dateTime) 


Hope this helps.

A.K.

- Original Message -
From: Neotropical bat risk assessments neotropical.b...@gmail.com
To: r-help@r-project.org; dedu...@googlegroups.com
Cc: 
Sent: Friday, December 28, 2012 6:46 AM
Subject: [R] Merging data tables

Hi all,

I am trying to merge several data sets and end up with a long data 
format by date  time so I can run correlations and plots.  I am using 
Deducer as an R GUI but can just use the R console if easier.

The data sets are weather with wind speed, relative humidity and 
temperatures by date and minute and bat activity with date, time, label, 
and an activity index number.  The bat activity is only during the 
nocturnal time frames while the weather data was recorded for 24 hours.  
Therefore a lot of weather data with no related activity for bats.

I have failed so far to achieve what I need and tried plyr and reshape2.
There are many Null data rows with no data or 0 (zero) for wind speed..
What other tools steps would be more appropriate short of manually in 
Excel cutting and pasting for a day or more?


Data formats are:
bat activity
Date     Time     Label     Number
6/3/2011     10:01     Tadbra     2
6/3/2011     10:02     Tadbra     4
6/3/2011     10:08     Tadbra     1
6/3/2011     10:13     Tadbra     2
6/3/2011     10:49     Tadbra     2
6/3/2011     10:51     Tadbra     2
6/3/2011     10:52     Tadbra     4


Weather:

date    time     Temp_I     Temp_E     RH     mph_a     mph_x
6/3/2011     0:00     15     15.7     30.4     4.4     15.5
6/3/2011     0:30     15     15.2     31.6     5.7     11.2
6/3/2011     1:00     15     15.1     31.3     10.3     17.5
6/3/2011     1:30     14     13.6     44.5     4.7     12.1
6/3/2011     2:00     12.5     13.2     37.9     2.1     6.5
6/3/2011     2:30     12.5     13.5     35.3     6.3     10.1
6/3/2011     3:00     12     12.1     37.7     3     7.4
6/3/2011     3:30     11.5     11.5     38.7     3.4     6
6/3/2011     4:00     10     9.9     52.7     1.4     4.2
6/3/2011     4:30     9.5     9.1     43.2     1.3     3.8
6/3/2011     5:00     8     8.7     59.2     1.2     3.1
6/3/2011     5:30     7     8     62.5     1.1     4.2
6/3/2011     6:00     6     7.8     47.8     0.5     2.2
6/3/2011     6:30     7.5     11.5     37.5     1.7     3.8
6/3/2011     7:00     10.5     14     33.1     0.6     2.2
6/3/2011     7:30     14     17.3     32.1     1.6     3.6
6/3/2011     8:00     17.5     20.3     23.9     0.4     2
6/3/2011     8:30     21.5     22.8     20.7     0.4     1.8
6/3/2011     9:00     24.5     24.9     14.1     0.3     2.2
6/3/2011     9:30     26     26.9     20.3     1.7     5.6
6/3/2011     10:00     27.5     27.4     20.7     2.5     6.5
6/3/2011     10:30     28.5     29.8     10     1.6     4.2
6/3/2011     11:00     29.5     29.4     9.8     1.8     4
6/3/2011     11:30     30     29.9     9.2     2.7     7.8


Tnx for any suggestions,

Bruce

    [[alternative HTML version deleted]]

__

Re: [R] efficiently multiply different matrices in 3-d array with different vectors?

2012-12-28 Thread Ranjan Maitra
On Thu, 27 Dec 2012 22:03:19 -0800 arun smartpink...@yahoo.com wrote:

 HI,
 
 Not sure if this is what you wanted:
 set.seed(14)
 Z-array(sample(1:100,80,replace=TRUE),dim=c(5,2,8))
 set.seed(21)
 Y-matrix(sample(1:40,40,replace=TRUE),nrow=8)
 do.call(cbind,lapply(seq_len(dim(Z)[1]),function(i) Z[i,,]%*%Y[,i]))
 # [,1] [,2]  [,3]  [,4]  [,5]
 #[1,] 5065 6070 12328 11536 13678
 #[2,] 6152 9878 11161 13777  7991

Yes, this does it, many thanks!

Ranjan


 
 
 
 
 - Original Message -
 From: Ranjan Maitra maitra.mbox.igno...@inbox.com
 To: r-help@r-project.org
 Cc: 
 Sent: Friday, December 28, 2012 12:33 AM
 Subject: [R] efficiently multiply different matrices in 3-d array with 
 different vectors?
 
 Hello,
 
 I have been wondering of an efficient way to do this:
 
 I have an n x m x p array Z and a p x n matrix Y.
 
 I want to multiply each of the n matrices with the corresponding column
 vector of Y.
 
 In other words, I am wanting to matrix multiply: 
 
 Z[i, ,] %*% Y[, i]
 
 which will give me a (two-dimensional) array or matrix of dimension n x
 p with the i'th row storing the above.
 
 Any pointers on an efficient way of doing this? I considered using
 apply, but was not successful. 
 
 Many thanks again and best wishes,
 Ranjan
 
 
 
 -- 
 Important Notice: This mailbox is ignored: e-mails are set to be
 deleted on receipt. For those needing to send personal or professional
 e-mail, please use appropriate addresses.
 
 
 FREE ONLINE PHOTOSHARING - Share your photos online with your friends and 
 family!
 Visit http://www.inbox.com/photosharing to find out more!
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 


-- 
Important Notice: This mailbox is ignored: e-mails are set to be
deleted on receipt. For those needing to send personal or professional
e-mail, please use appropriate addresses.


GET FREE 5GB EMAIL - Check out spam free email with many cool features!
Visit http://www.inbox.com/email to find out more!

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Merging data tables

2012-12-28 Thread arun
HI,

It is related to the memory.

Check these links for discussions regarding this matter:

http://r.789695.n4.nabble.com/Memory-Problem-td853869.html
http://r.789695.n4.nabble.com/Memory-issues-td3220252.html

A.K.



From: Neotropical bat risk assessments neotropical.b...@gmail.com
To: arun smartpink...@yahoo.com 
Sent: Friday, December 28, 2012 9:19 AM
Subject: Re: [R] Merging data tables




Hi, I was able to get the merge to work OK, I think...


Bats_weather-merge(sewage.Wind_Temp.temp,Tabr_all2.temp,by.x=row.names,by.y=row.names,incomparables
= NA,all.x =F,all.y =T)
 rm(list=c(sewage.Wind_Temp.temp,Tabr_all2.temp))
 dput(Bats_weather,'C:/=Bat data
working/Rainey/Sewage/Tadbra_weather.robj')
 Bats_weather[,9]-as.character(Bats_weather[,9])
 dput(Bats_weather,'C:/=Bat data
working/Rainey/Sewage/Tadbra_weather.robj')
 obj_a - t.test(x = Bats_weather$Number,y =
Bats_weather$Temp_E,paired = TRUE)
 print(obj_a) 

But I can not run the correlations...

 ggcorplot(cor.mat=corr.mat,data=Bats_weather,
+ cor_text_limits=c(5,20),
+ line.method=lm)
Error: cannot allocate vector of size 1.6 Gb
In addition: Warning message:
In get(x, envir = this, inherits = inh) :
  restarting interrupted promise evaluation
 rm('corr.mat')


-- 
Bruce W. Miller, PhD.
Neotropical bat risk assessments If we lose the bats, we may lose much of the 
tropical vegetation and the lungs of the planet Using acoustic sampling to map 
species distributions for 15 years. Providing Interactive identification keys 
to the vocal signatures of New World Bats For various project details see: 
https://sites.google.com/site/batsoundservices/

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] Question about Linear Regression

2012-12-28 Thread Lorenzo Isella

Dear All,
A semi-trivial question: suppose you want to carry out a linear regression  
of the kind


y~x1+x2+x3

where x3 is a dichotomous variable assuming only 0 and 1 values (x1 and x2  
are continuous variables).
Is there any particular caveat I should be aware of? Can I code this as a  
simple multiple linear regression into R or is there anything else I  
should bear in mind?

Many thanks

Lorenzo

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Question about Linear Regression

2012-12-28 Thread Jeremy Miles
You can run that as it is. The term to search for on Google is 'dummy
coding'.

Jeremy

On 28 December 2012 07:45, Lorenzo Isella lorenzo.ise...@gmail.com wrote:


 where x3 is a dichotomous variable assuming only 0 and 1 values (x1 and x2
 are continuous variables).
 Is there any particular caveat I should be aware of? Can I code this as a
 simple multiple linear regression into R or is there anything else I should
 bear in mind?


[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Help reading matrixmarket files

2012-12-28 Thread Brad Cox
Could someone please approve this message? Thanks!

On Thu, Dec 27, 2012 at 4:38 PM, Brad Cox bradj...@gmail.com wrote:

 I'm trying to read data a program produces in matrixmarket array format
 into R and its giving me fits. I've tried read.MM (below) and readMM (from
 the Matrix package) but neither works. One of them says array format isn't
 supported, the other reports something indecipherable about Fortran.

 Here's the file contents. Can't get much simpler than that!

 %%MatrixMarket matrix array real general
 %This file contains LIBFM global mean which is required for computing
 predictions.
 1 1
 1.786238830716e-02


 Can someone please help? Seems such a simple problem.


  library(spam)
 Spam version 0.29-2 (2012-08-17) is loaded.
 Type 'help( Spam)' or 'demo( spam)' for a short introduction
 and overview of this package.
 Help for individual functions is also obtained by adding the
 suffix '.spam' to the function name, e.g. 'help( chol.spam)'.
 Attaching package: ‘spam’
 The following object(s) are masked from ‘package:base’:
 backsolve, forwardsolve
  x=read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm)
 Error in read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm)
 :
   invalid mode (list) to pass to Fortran (arg 8)





-- 
Guns kill people... the way forks make people fat.
Dr. Brad J. CoxCell: 703-594-1883
Blog: http://bradjcox.blogspot.comWeb: http://virtualschool.edu

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] vectorization modifying globals in functions

2012-12-28 Thread Greg Snow
In current versions of R the apply functions do not gain much (if any) in
speed over a well written for loop (the for loops are much more efficient
than they used to be).

Using global variables could actually slow things down a little for what
you are doing, if you use `-` then it has to search through multiple
environments to find which to replace.

In general you should avoid using global variables.  It is best to pass all
needed variables into a function as arguments, do any modifications
internally inside the function on local copies, then return the modified
local copy from the function (you can use a list if you want to return
multiple variables).

Since each iteration of your code depends on the previous iteration,
vectorizing is not going to help (or even be reasonable).

If you want to speed up the code then you might consider a compiled option,
see the inline or rcpp packages (or others).


On Thu, Dec 27, 2012 at 1:38 PM, Sam Steingold s...@gnu.org wrote:

 I have the following code:

 --8---cut here---start-8---
 d - rep(10,10)
 for (i in 1:100) {
   a - sample.int(length(d), size = 2)
   if (d[a[1]] = 1) {
 d[a[1]] - d[a[1]] - 1
 d[a[2]] - d[a[2]] + 1
   }
 }
 --8---cut here---end---8---

 it does what I want, i.e., modified vector d 100 times.

 Now, if I want to repeat this 1e6 times instead of 1e2 times, I want to
 vectorize it for speed, so I do this:

 --8---cut here---start-8---
 update - function (i) {
   a - sample.int(n.agents, size = 2)
   if (d[a[1]] = delta) {
 d[a[1]] - d[a[1]] - 1
 d[a[2]] - d[a[2]] + 1
   }
   entropy(d, unit=log2)
 }
 system.time(entropy.history - sapply(1:1e6,update))
 --8---cut here---end---8---

 however, the global d is not modified, apparently update modifies the
 local copy.

 so,
 1. is there a way for a function to modify a global variable?
 2. how would you vectorize this loop?

 thanks!

 --
 Sam Steingold (http://sds.podval.org/) on Ubuntu 12.04 (precise) X
 11.0.11103000
 http://www.childpsy.net/ http://honestreporting.com
 http://pmw.org.il http://www.PetitionOnline.com/tap12009/
 A number problem solved with floats turns into 1.9998
 problems.

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Help reading matrixmarket files

2012-12-28 Thread Uwe Ligges



On 28.12.2012 17:28, Brad Cox wrote:

Could someone please approve this message? Thanks!


Approve?



On Thu, Dec 27, 2012 at 4:38 PM, Brad Cox bradj...@gmail.com wrote:


I'm trying to read data a program produces in matrixmarket array format
into R and its giving me fits. I've tried read.MM (below) and readMM (from
the Matrix package) but neither works. One of them says array format isn't
supported, the other reports something indecipherable about Fortran.

Here's the file contents. Can't get much simpler than that!

%%MatrixMarket matrix array real general

%This file contains LIBFM global mean which is required for computing
predictions.
1 1
1.786238830716e-02



Can someone please help? Seems such a simple problem.



library(spam)

Spam version 0.29-2 (2012-08-17) is loaded.
Type 'help( Spam)' or 'demo( spam)' for a short introduction
and overview of this package.
Help for individual functions is also obtained by adding the
suffix '.spam' to the function name, e.g. 'help( chol.spam)'.
Attaching package: ‘spam’
The following object(s) are masked from ‘package:base’:
 backsolve, forwardsolve

x=read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm)

Error in read.MM(/Users/brad/Dropbox/graphchi/2008.csv_global_mean.mm)
:
   invalid mode (list) to pass to Fortran (arg 8)



If read.MM does not work and you are sure about the file location and it 
is in the right format, send an example file that reproduced the error 
to the maintainer of the package with the broken import function (or 
right away  patched version of the broken function).


Uwe Ligges











__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.



__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] R crashing inconsistently within for loops

2012-12-28 Thread William Dunlap
Sometimes these intermittent crashes come from memory misuse, e.g., not 
allocating enough scratch space.  You can sometimes make those coding errors 
cause more consistent problems by calling gctorture(TRUE) before running your 
code.
Here is an example in which it looks like package:gam's lo() is misusing memory 
when its degree argument is 2 - sometimes it
can do 10 iterations, sometimes 3, sometimes 1:
 
   library(gam)
  Loading required package: splines
  Loaded gam 1.06.2
  
v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), 
data=mtcars)})
  1 2 3 4 5 6 7 8 9 10 
v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), 
data=mtcars)})
  1 2 3 Error in sys.call() : invalid 'which' argument
v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), 
data=mtcars)})
  1 Error in sys.call() : invalid 'which' argument 

If I call gctorture(TRUE) before calling lo(degree=2) then it hangs on the 
first call.  One could attach a debugger at this point to get a clue about 
where it is failing.

   library(gam)
  Loading required package: splines
  Loaded gam 1.06.2
  
  
   gctorture(TRUE)
   v - lapply(1:10,function(i){cat(i,); gam(mpg ~ lo(hp,degree=2), 
data=mtcars)})
  1

Using valgrind is helpful also.

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com


 -Original Message-
 From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
 Behalf
 Of Steve Powers
 Sent: Thursday, December 27, 2012 7:01 PM
 To: r-help@r-project.org
 Subject: [R] R crashing inconsistently within for loops
 
 Hello,
 
 This one has been bugging me for a long time and I have never found a
 solution. I am using R version 2.15.1 but it has come up in older versions of 
 R I have used
 over the past 2-3 years.
 
 Q: Am I wrong to expect that R should handle hundreds of iterations of the
 base model or statistical functions, embedded within for loops, in one
 script run? I have found that when I write scripts that do this, sometimes
 they have a tendency to crash, seemingly unpredictably.
 
 For example, one problem script of mine employs glm and gls about a hundred
 different times, and output files are being written at the end of each
 iteration. I have used my output files to determine that the crash cause is
 not consistent (R never fails at the same iteration). Note that the data are
 fixed here (no data generation or randomization steps, so that is not the
 issue). But it is clear that scripts with larger numbers of iterations are
 more likely to produce a crash.
 
 And a year or two ago, I had a seemingly stable R script again with for
 looped model fits, but discovered this script was prone to crashing when I
 ran it on a newer PC. Because the new PC also seemed to be blazing through R
 code absurdly fast, I tried adding a short fluff procedure at the end of
 each iteration that required a few seconds of processing time. Low and
 behold, when I added that, the script stopped crashing (and each iteration
 of course took longer). I still don't understand why that fixed things.
 
 What is going on? Solutions? Thanks.---steve
 
 --
 Steve Powers
 power...@nd.edu
 University of Notre Dame
 Environmental Change Initiative
 website (http://www.nd.edu/~spowers2/index.htm)
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] Using grImport to create a watermark

2012-12-28 Thread Thomas Adams - NOAA Federal
Hi…

I want to use grImport to create a watermark on a plot() using the methods
Paul Murrell describes here:
http://cran.r-project.org/web/packages/grImport/vignettes/import.pdf (page
28). I can essentially reproduce this manually at the R prompt, and
independently I can use grid.picture(…) successfully in a R script, but
when I attempt to do do this in my script:

•
•
•
postscript(outputFilename,paper=us)

grid.picture(noaalogo,distort=FALSE,width=0.5,x=0.50,y=0.50)
grid.rect(gp=gpar(fill=rgb(1,1,1,0.9)))


plot(dat,xaxt=n,type=b,ylim=c(y_range[1],y_range[2]),main=labelStr,xlab=Date,ylab=Elevation,
NAVD88 (Ft))
•
•
•
dev.off()

The picture noaalogo is drawn, but either the plot() is obscured or is
not drawn at all. In either case, the watermarking effect is not
happening in my script, but does work as expected, manually, at the R
prompt. I'm sure this is a graphics device related issue, but I have not
been able to find a solution through my searches.

Thank you,
Tom


-- 

Thomas E Adams

Development  Operations Hydrologist
National Weather Service
Ohio River Forecast Center
1901 South State Route 134
Wilmington, OH 45177

http://www.erh.noaa.gov/er/ohrfc/

EMAIL:  thomas.ad...@noaa.gov
VOICE:  937-383-0528
FAX:937-383-0033

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] multiple sheet in r

2012-12-28 Thread eliza botto

Dear useRs,
i want to open an excel file having 59 sheets in r and GOD knows the number of 
command i tried. unfortunately none worked, obviously error will be on my part. 
All the sheets contain 1 kind of data and the data starts from  A12:M12 (which 
contains the header of the data), the line immediately under the header is 
blank. The number of columns of data in each sheet are same while the number of 
row are different as length of data is different.
you help is required..
regardseliza  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] Any simple way to make this happen?

2012-12-28 Thread Haoda Fu
Hi,

The data is 

x - c(4,3,5);
I need to translate it to
y - c(1,2,3,4,1,2,3,1,2,3,4,5);
i.e. for each number in x, we need to generate 1:x and put it in y.

The program need to evaluate this type of calculation for millions of times in 
simulation. Is there any elegant way to make this happen without iteration?

Many thanks!!

Best,
Haoda
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Any simple way to make this happen?

2012-12-28 Thread William Dunlap
   x - c(4,3,5)
   y - c(1,2,3,4,1,2,3,1,2,3,4,5)
   all.equal(y, sequence(x))
  [1] TRUE

Bill Dunlap
Spotfire, TIBCO Software
wdunlap tibco.com


 -Original Message-
 From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On 
 Behalf
 Of Haoda Fu
 Sent: Friday, December 28, 2012 9:43 AM
 To: r-help@r-project.org
 Cc: haod...@yahoo.com
 Subject: [R] Any simple way to make this happen?
 
 Hi,
 
 The data is
 
 x - c(4,3,5);
 I need to translate it to
 y - c(1,2,3,4,1,2,3,1,2,3,4,5);
 i.e. for each number in x, we need to generate 1:x and put it in y.
 
 The program need to evaluate this type of calculation for millions of times 
 in simulation.
 Is there any elegant way to make this happen without iteration?
 
 Many thanks!!
 
 Best,
 Haoda
   [[alternative HTML version deleted]]
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] multiple sheet in r

2012-12-28 Thread Greg Snow
If you want help from us mere mortals then it would help us to help you if
you could tell us not only the number of commands that you tried but what
the actual commands were.

Also the phrase none worked does not help us help you, see fortune(324).
 A description of what happened and how the results (if any) differed from
what you expected would help.

I would suggest trying the XLConnect package, but that may be what you
tried already.


On Fri, Dec 28, 2012 at 10:37 AM, eliza botto eliza_bo...@hotmail.comwrote:


 Dear useRs,
 i want to open an excel file having 59 sheets in r and GOD knows the
 number of command i tried. unfortunately none worked, obviously error will
 be on my part. All the sheets contain 1 kind of data and the data starts
 from  A12:M12 (which contains the header of the data), the line immediately
 under the header is blank. The number of columns of data in each sheet are
 same while the number of row are different as length of data is different.
 you help is required..
 regardseliza
 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Any simple way to make this happen?

2012-12-28 Thread Haoda Fu
It works like a magic~!!
Many thanks A.K.

Best,
Haoda





 From: arun smartpink...@yahoo.com

Cc: R help r-help@r-project.org 
Sent: Friday, December 28, 2012 12:51 PM
Subject: Re: [R] Any simple way to make this happen?

Hi,
Try:
y-paste(c,(,paste(unlist(sapply(x,seq)),collapse=,),),sep=)
 y
#[1] c(1,2,3,4,1,2,3,1,2,3,4,5)

A.K.




- Original Message -

To: r-help@r-project.org r-help@r-project.org
Cc: haod...@yahoo.com haod...@yahoo.com
Sent: Friday, December 28, 2012 12:43 PM
Subject: [R] Any simple way to make this happen?

Hi,

The data is 

x - c(4,3,5);
I need to translate it to
y - c(1,2,3,4,1,2,3,1,2,3,4,5);
i.e. for each number in x, we need to generate 1:x and put it in y.

The program need to evaluate this type of calculation for millions of times in 
simulation. Is there any elegant way to make this happen without iteration?

Many thanks!!

Best,
Haoda
    [[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Any simple way to make this happen?

2012-12-28 Thread Greg Snow
The only way that I can think of that would not use iteration would be to
create some type of lookup table with every possible vector you may ever
use, then lookup the results in that table.

If you are happy with internal iteration then one possibility is:

 x - c(4,3,5)
 unlist(lapply(x,seq_len))
 [1] 1 2 3 4 1 2 3 1 2 3 4 5



On Fri, Dec 28, 2012 at 10:43 AM, Haoda Fu fu...@yahoo.com.cn wrote:

 Hi,

 The data is

 x - c(4,3,5);
 I need to translate it to
 y - c(1,2,3,4,1,2,3,1,2,3,4,5);
 i.e. for each number in x, we need to generate 1:x and put it in y.

 The program need to evaluate this type of calculation for millions of
 times in simulation. Is there any elegant way to make this happen without
 iteration?

 Many thanks!!

 Best,
 Haoda
 [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] vectorization modifying globals in functions

2012-12-28 Thread David Winsemius


On Dec 27, 2012, at 12:38 PM, Sam Steingold wrote:


I have the following code:

--8---cut here---start-8---
d - rep(10,10)
for (i in 1:100) {
 a - sample.int(length(d), size = 2)
 if (d[a[1]] = 1) {
   d[a[1]] - d[a[1]] - 1
   d[a[2]] - d[a[2]] + 1
 }
}
--8---cut here---end---8---

it does what I want, i.e., modified vector d 100 times.

Now, if I want to repeat this 1e6 times instead of 1e2 times, I want  
to

vectorize it for speed, so I do this:


You could get some modest improvement by vectorizing the two  
lookups, additions, and assignments into one:


 d[a] - d[a]-c(1,-1)

In a test with 10 iterations, it yields about a  1.693/1.394 -1  =  
21 percent improvement.


--8---cut here---start-8---
update - function (i) {
 a - sample.int(n.agents, size = 2)
 if (d[a[1]] = delta) {
   d[a[1]] - d[a[1]] - 1
   d[a[2]] - d[a[2]] + 1
 }
 entropy(d, unit=log2)


The `unit` seems likely to throw an error since there is no argument  
for it to match.



}
system.time(entropy.history - sapply(1:1e6,update))
--8---cut here---end---8---

however, the global d is not modified, apparently update modifies the
local copy.


You could have returned 'd' and the entropy result as a list. But what  
would be the point of saving 1e6 copies




so,
1. is there a way for a function to modify a global variable?


So if you replaced it in the global environment, you would only be  
seeing the result of the last iteration of the loop. What's the use of  
that



2. how would you vectorize this loop?

thanks!


--
David Winsemius, MD
Alameda, CA, USA

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] multiple sheet in r

2012-12-28 Thread eliza botto

Dear Arun and Greg,extremly sorry for asking you into something which was quite 
easy...the following command worked
library(XLConnect)sheets - list()for(x in 1:59) {sheets[[x]] - 
readWorksheetFromFile(PAK.xls, sheet=x[i],region=A12:M1)}
regardselisa

Date: Fri, 28 Dec 2012 10:56:43 -0700
Subject: Re: [R] multiple sheet in r
From: 538...@gmail.com
To: eliza_bo...@hotmail.com
CC: r-help@r-project.org

If you want help from us mere mortals then it would help us to help you if you 
could tell us not only the number of commands that you tried but what the 
actual commands were. 
Also the phrase none worked does not help us help you, see fortune(324).  A 
description of what happened and how the results (if any) differed from what 
you expected would help.

I would suggest trying the XLConnect package, but that may be what you tried 
already.

On Fri, Dec 28, 2012 at 10:37 AM, eliza botto eliza_bo...@hotmail.com wrote:



Dear useRs,

i want to open an excel file having 59 sheets in r and GOD knows the number of 
command i tried. unfortunately none worked, obviously error will be on my part. 
All the sheets contain 1 kind of data and the data starts from  A12:M12 (which 
contains the header of the data), the line immediately under the header is 
blank. The number of columns of data in each sheet are same while the number of 
row are different as length of data is different.


you help is required..

regardseliza

[[alternative HTML version deleted]]



__

R-help@r-project.org mailing list

https://stat.ethz.ch/mailman/listinfo/r-help

PLEASE do read the posting guide http://www.R-project.org/posting-guide.html

and provide commented, minimal, self-contained, reproducible code.



-- 
Gregory (Greg) L. Snow Ph.D.
538...@gmail.com
  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Help reading matrixmarket files

2012-12-28 Thread Brad Cox
On Fri, Dec 28, 2012 at 11:47 AM, Uwe Ligges 
lig...@statistik.tu-dortmund.de wrote:

 Approve?


Isn't it still being queued for approval? Can't tell from this end.

Your mail to 'R-help' with the subject

Help reading matrixmarket files

Is being held until the list moderator can review it for approval.

The reason it is being held:

Post to moderated list

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Help reading matrixmarket files

2012-12-28 Thread Uwe Ligges



On 28.12.2012 19:49, Brad Cox wrote:

On Fri, Dec 28, 2012 at 11:47 AM, Uwe Ligges
lig...@statistik.tu-dortmund.de
mailto:lig...@statistik.tu-dortmund.de wrote:

Approve?

Isn't it still being queued for approval? Can't tell from this end.

Your mail to 'R-help' with the subject

 Help reading matrixmarket files

Is being held until the list moderator can review it for approval.

The reason it is being held:

 Post to moderated list



The approval happened before, obviously, if you were not on the list 
yourself. There is no need to ask for that separately and generate even 
more traffic.


Best,
Uwe Ligges

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Using grImport to create a watermark

2012-12-28 Thread Prof Brian Ripley
I suspect you need a device supporting translucency: PostScript does not 
and hence postscript() cannot.  Try the pdf() device (and convert the 
output if you need it).


On 28/12/2012 17:23, Thomas Adams - NOAA Federal wrote:

Hi…

I want to use grImport to create a watermark on a plot() using the methods
Paul Murrell describes here:
http://cran.r-project.org/web/packages/grImport/vignettes/import.pdf (page
28). I can essentially reproduce this manually at the R prompt, and


On what OS and what device: see the posting guide 


independently I can use grid.picture(…) successfully in a R script, but
when I attempt to do do this in my script:

•
•
•
postscript(outputFilename,paper=us)

grid.picture(noaalogo,distort=FALSE,width=0.5,x=0.50,y=0.50)
grid.rect(gp=gpar(fill=rgb(1,1,1,0.9)))


plot(dat,xaxt=n,type=b,ylim=c(y_range[1],y_range[2]),main=labelStr,xlab=Date,ylab=Elevation,
NAVD88 (Ft))
•
•
•
dev.off()

The picture noaalogo is drawn, but either the plot() is obscured or is
not drawn at all. In either case, the watermarking effect is not
happening in my script, but does work as expected, manually, at the R
prompt. I'm sure this is a graphics device related issue, but I have not
been able to find a solution through my searches.

Thank you,
Tom




__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.




--
Brian D. Ripley,  rip...@stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford, Tel:  +44 1865 272861 (self)
1 South Parks Road, +44 1865 272866 (PA)
Oxford OX1 3TG, UKFax:  +44 1865 272595

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] multiple sheet in r

2012-12-28 Thread arun
HI,

Check this link:
http://stackoverflow.com/questions/6894922/how-to-read-multiple-excel-sheets-in-r-programming
http://theweiluo.wordpress.com/2011/09/07/how-to-read-a-multiple-sheet-excel-file-into-r/

A.K.



- Original Message -
From: eliza botto eliza_bo...@hotmail.com
To: r-help@r-project.org r-help@r-project.org
Cc: 
Sent: Friday, December 28, 2012 12:37 PM
Subject: [R] multiple sheet in r


Dear useRs,
i want to open an excel file having 59 sheets in r and GOD knows the number of 
command i tried. unfortunately none worked, obviously error will be on my part. 
All the sheets contain 1 kind of data and the data starts from  A12:M12 (which 
contains the header of the data), the line immediately under the header is 
blank. The number of columns of data in each sheet are same while the number of 
row are different as length of data is different.
you help is required..
regardseliza                           
    [[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] comparison of large data set

2012-12-28 Thread Irucka Embry
Hi Jean, thank-you.

It was my fault on not ccing R-help on the previous correspondence. With
regards to the FALSE/TRUE or 0/1, you are right that they are the same
logical identities, but it better suits our original numerical data to
display 0s and 1s. 

I want to thank you for correcting the 2 mistakes, the code works now.

Is it possible to label each of the 54 matrices so that justbig lists
the 31 approved matrices along with their name [whatever the name is for
the 31 matrices out of the 54 total matrices]? For this code to be fully
applied to this project we have to know which site locations are
successful ( 0.7) and which ones are not ( 0.7).

Thank-you Jean.

Irucka



-Original Message- 
From: Adams, Jean [jvad...@usgs.gov]
Sent: 12/21/2012 1:25:24 PM
To: iruc...@mail2world.com
Cc: r-help@r-project.org
Subject: Re: [R] comparison of large data set

Irucka,


You should cc R-help on all correspondence so that other readers of the
list can
follow the conversation.


(1) You say that you need 0s and 1s rather than TRUE/FALSE. Since a 0/1
matrix
and TRUE/FALSE matrix behave exactly the same way in most applications,
I'm not
sure why it would matter which one you had.


(2) My mistake. I had not noticed that you were eliminating the case
where
both the observed and the modeled were FALSE.. I have modified my code
to give
the same results as your f2 function.
compare.all - sapply(modeldepth, function(md)
mean((md==observeddepth)[md |
observeddepth]))



(3) My mistake again. There should be only single brackets, not double
brackets.
justbig - modeldepth[compare.all  0.7]


Jean





On Fri, Dec 21, 2012 at 11:27 AM, Irucka Embry iruc...@mail2world.com
wrote:

Hi Jean Adams, how are you?

I want to thank you for your response to my request for assistance. 

I received some assistance yesterday afternoon and I was able to update
the code
which is posted here:
http://r.789695.n4.nabble.com/variable-names-in-numeric-list-and-Bayesi
an-inference-td4653674.html.
I posted the new code with some new questions that I have with regards
to the
code that I have written. Can you look over that post and suggest any
code
revisions for those aspects that do not work? Thank-you.

The code that you suggested worked well overall, except for 3 aspects
of it:

Here I actually needed the binary 0s and 1s rather than a TRUE/FALSE
logical matrix

# a function to read in the data as a matrix of logicals
myreadfun - function(file) {
as.matrix(read.ascii.grid(file)$data)!=0
}


Here I needed to calculate the f2 probability rather than the mean

compare.all - sapply(modeldepths, function(md)
mean(md==observeddepth))

 str(compare.all)
num [1:54] 0.936 0.94 0.944 0.944 0.945 ...

Here most of the entries are greater than 0.7, but it should just be 31
of the
54 that are greater than 0.7

 justbig - modeldepths[[compare.all  0.7]]

Error in modeldepths[[compare.all  0.7]] : 
recursive indexing failed at level 2

Once again, thank-you for your assistance.

Irucka Embry


-Original Message- 
From: Adams, Jean [jvad...@usgs.gov]
Sent: 12/21/2012 10:32:54 AM
To: iruc...@mail2world.com
Cc: r-help@r-project.org
Subject: Re: [R] comparison of large data set

Irucka,


I did not test this code out on any data, but I think it will work.


Jean




# a function to read in the data as a matrix of logicals
myreadfun - function(file) {
as.matrix(read.ascii.grid(file)$data)!=0
}


# names of the 54 modeled depth files
modfiles - paste0(MaxFloodDepth_, 1:54, .txt)


# read in the observed and modeled depth files
# observeddepth is a matrix
observeddepth - myreadfun(MaxFloodDepth_Observed.txt)

# modeldepths is a list of matrices
modeldepths - lapply(filenames, myreadfun)


# calculate the proportion of matrix elements that agree with the
observed file
# the results is a vector with one number for each modeled depth
matrix
compare.all - sapply(modeldepths, function(md)
mean(md==observeddepth))


# select just those matrices that had a large proportion of agreements
# justbig is a list of matrices
justbig - modeldepths[[compare.all  0.7]]


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide
http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code. 


span id=m2wTlpfont face=Arial, Helvetica, sans-serif size=2 
style=font-size:13.5px___BRGet
 the Free email that has everyone talking at a href=http://www.mail2world.com 
target=newhttp://www.mail2world.com/abr  font color=#99Unlimited 
Email Storage #150; POP3 #150; Calendar #150; SMS #150; Translator #150; 
Much More!/font/font/span
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide 

Re: [R] Any simple way to make this happen?

2012-12-28 Thread arun
Hi,
Try:
y-paste(c,(,paste(unlist(sapply(x,seq)),collapse=,),),sep=)
 y
#[1] c(1,2,3,4,1,2,3,1,2,3,4,5)

A.K.




- Original Message -
From: Haoda Fu fu...@yahoo.com.cn
To: r-help@r-project.org r-help@r-project.org
Cc: haod...@yahoo.com haod...@yahoo.com
Sent: Friday, December 28, 2012 12:43 PM
Subject: [R] Any simple way to make this happen?

Hi,

The data is 

x - c(4,3,5);
I need to translate it to
y - c(1,2,3,4,1,2,3,1,2,3,4,5);
i.e. for each number in x, we need to generate 1:x and put it in y.

The program need to evaluate this type of calculation for millions of times in 
simulation. Is there any elegant way to make this happen without iteration?

Many thanks!!

Best,
Haoda
    [[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Merging data tables

2012-12-28 Thread Ian Fellows
It is also possible to construct the r call in the GUI. See:


http://www.deducer.org/pmwiki/index.php?n=Main.MergeData

Ian


Sent from my iPhone

On Dec 28, 2012, at 4:41 AM, Djordje Bajic je.li@gmail.com wrote:

 see ?merge, 
 
 for your dataframes, the following should work:
 
 merge(bat_activity, weather, by=c(date, time), all=T) 
 
 
 2012/12/28 Neotropical bat risk assessments neotropical.b...@gmail.com
 Hi all,
 
 I am trying to merge several data sets and end up with a long data
 format by date  time so I can run correlations and plots.  I am using
 Deducer as an R GUI but can just use the R console if easier.
 
 The data sets are weather with wind speed, relative humidity and
 temperatures by date and minute and bat activity with date, time, label,
 and an activity index number.  The bat activity is only during the
 nocturnal time frames while the weather data was recorded for 24 hours.
 Therefore a lot of weather data with no related activity for bats.
 
 I have failed so far to achieve what I need and tried plyr and reshape2.
 There are many Null data rows with no data or 0 (zero) for wind speed..
 What other tools steps would be more appropriate short of manually in
 Excel cutting and pasting for a day or more?
 
 
 Data formats are:
 bat activity
 DateTimeLabel   Number
 6/3/201110:01   Tadbra  2
 6/3/201110:02   Tadbra  4
 6/3/201110:08   Tadbra  1
 6/3/201110:13   Tadbra  2
 6/3/201110:49   Tadbra  2
 6/3/201110:51   Tadbra  2
 6/3/201110:52   Tadbra  4
 
 
 Weather:
 
 datetimeTemp_I  Temp_E  RH  mph_a   mph_x
 6/3/20110:0015  15.730.44.4 15.5
 6/3/20110:3015  15.231.65.7 11.2
 6/3/20111:0015  15.131.310.317.5
 6/3/20111:3014  13.644.54.7 12.1
 6/3/20112:0012.513.237.92.1 6.5
 6/3/20112:3012.513.535.36.3 10.1
 6/3/20113:0012  12.137.73   7.4
 6/3/20113:3011.511.538.73.4 6
 6/3/20114:0010  9.9 52.71.4 4.2
 6/3/20114:309.5 9.1 43.21.3 3.8
 6/3/20115:008   8.7 59.21.2 3.1
 6/3/20115:307   8   62.51.1 4.2
 6/3/20116:006   7.8 47.80.5 2.2
 6/3/20116:307.5 11.537.51.7 3.8
 6/3/20117:0010.514  33.10.6 2.2
 6/3/20117:3014  17.332.11.6 3.6
 6/3/20118:0017.520.323.90.4 2
 6/3/20118:3021.522.820.70.4 1.8
 6/3/20119:0024.524.914.10.3 2.2
 6/3/20119:3026  26.920.31.7 5.6
 6/3/201110:00   27.527.420.72.5 6.5
 6/3/201110:30   28.529.810  1.6 4.2
 6/3/201111:00   29.529.49.8 1.8 4
 6/3/201111:30   30  29.99.2 2.7 7.8
 
 
 Tnx for any suggestions,
 
 Bruce
 
 [[alternative HTML version deleted]]
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.
 

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] Using grImport to create a watermark

2012-12-28 Thread Thomas Adams - NOAA Federal
Prof Ripley,

I see that now; I can get the transparency to work, but even with using
pdf() my plot still does not draw if I first use:

grid.picture(noaalogo,distort=**FALSE,width=0.5,x=0.50,y=0.50)
grid.rect(gp=gpar(fill=rgb(1,**1,1,0.9)))

Regards,
Tom


On Fri, Dec 28, 2012 at 1:55 PM, Prof Brian Ripley rip...@stats.ox.ac.ukwrote:

 I suspect you need a device supporting translucency: PostScript does not
 and hence postscript() cannot.  Try the pdf() device (and convert the
 output if you need it).


 On 28/12/2012 17:23, Thomas Adams - NOAA Federal wrote:

 Hi…

 I want to use grImport to create a watermark on a plot() using the methods
 Paul Murrell describes here:
 http://cran.r-project.org/web/**packages/grImport/vignettes/**import.pdfhttp://cran.r-project.org/web/packages/grImport/vignettes/import.pdf(page
 28). I can essentially reproduce this manually at the R prompt, and


 On what OS and what device: see the posting guide 

  independently I can use grid.picture(…) successfully in a R script, but
 when I attempt to do do this in my script:

 •
 •
 •
 postscript(outputFilename,**paper=us)

 grid.picture(noaalogo,distort=**FALSE,width=0.5,x=0.50,y=0.50)
 grid.rect(gp=gpar(fill=rgb(1,**1,1,0.9)))


 plot(dat,xaxt=n,type=b,**ylim=c(y_range[1],y_range[2]),**
 main=labelStr,xlab=Date,**ylab=Elevation,
 NAVD88 (Ft))
 •
 •
 •
 dev.off()

 The picture noaalogo is drawn, but either the plot() is obscured or is
 not drawn at all. In either case, the watermarking effect is not
 happening in my script, but does work as expected, manually, at the R
 prompt. I'm sure this is a graphics device related issue, but I have not
 been able to find a solution through my searches.

 Thank you,
 Tom




 __**
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/**
 posting-guide.html http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.



 --
 Brian D. Ripley,  rip...@stats.ox.ac.uk
 Professor of Applied Statistics,  
 http://www.stats.ox.ac.uk/~**ripley/http://www.stats.ox.ac.uk/~ripley/
 University of Oxford, Tel:  +44 1865 272861 (self)
 1 South Parks Road, +44 1865 272866 (PA)
 Oxford OX1 3TG, UKFax:  +44 1865 272595

 __**
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/**listinfo/r-helphttps://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/**
 posting-guide.html http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 

Thomas E Adams

Development  Operations Hydrologist
National Weather Service
Ohio River Forecast Center
1901 South State Route 134
Wilmington, OH 45177

http://www.erh.noaa.gov/er/ohrfc/

EMAIL:  thomas.ad...@noaa.gov
VOICE:  937-383-0528
FAX:937-383-0033

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] comparison of large data set

2012-12-28 Thread Adams, Jean
Irucka,

You could assign names to the compare.all list ... for example ...
 names(compare.all) - paste0(Obs, 1:54)
Then, when you create the subset list, justbig, it will have the
appropriate names.

If you just want to see the indices of the successful locations, you could
print
 (1:54)[compare.all  0.7]

Jean



On Fri, Dec 28, 2012 at 11:10 AM, Irucka Embry iruc...@mail2world.comwrote:

  Hi Jean, thank-you.

 It was my fault on not ccing R-help on the previous correspondence. With
 regards to the FALSE/TRUE or 0/1, you are right that they are the same
 logical identities, but it better suits our original numerical data to
 display 0s and 1s.

 I want to thank you for correcting the 2 mistakes, the code works now.

 Is it possible to label each of the 54 matrices so that justbig lists
 the 31 approved matrices along with their name [whatever the name is for
 the 31 matrices out of the 54 total matrices]? For this code to be fully
 applied to this project we have to know which site locations are successful
 ( 0.7) and which ones are not ( 0.7).

 Thank-you Jean.

 Irucka




 -Original Message-
 From: Adams, Jean [jvad...@usgs.gov]
 Sent: 12/21/2012 1:25:24 PM
 To: iruc...@mail2world.com
 Cc: r-help@r-project.org
 Subject: Re: [R] comparison of large data set
 
 Irucka,
 
 
 You should cc R-help on all correspondence so that other readers of the
 list can
 follow the conversation.
 
 
 (1) You say that you need 0s and 1s rather than TRUE/FALSE. Since a 0/1
 matrix
 and TRUE/FALSE matrix behave exactly the same way in most applications,
 I'm not
 sure why it would matter which one you had.
 
 
 (2) My mistake. I had not noticed that you were eliminating the case where
 both the observed and the modeled were FALSE.. I have modified my code to
 give

 the same results as your f2 function.
 compare.all - sapply(modeldepth, function(md)
 mean((md==observeddepth)[md |
 observeddepth]))
 
 
 
 (3) My mistake again. There should be only single brackets, not double
 brackets.
 justbig - modeldepth[compare.all  0.7]
 
 
 Jean
 
 
 
 
 
 On Fri, Dec 21, 2012 at 11:27 AM, Irucka Embry iruc...@mail2world.com
 wrote:
 
 Hi Jean Adams, how are you?
 
 I want to thank you for your response to my request for assistance.
 
 I received some assistance yesterday afternoon and I was able to update
 the code
 which is posted here:
 
 http://r.789695.n4.nabble.com/variable-names-in-numeric-list-and-Bayesian-inference-td4653674.html
 .
 I posted the new code with some new questions that I have with regards to
 the
 code that I have written. Can you look over that post and suggest any code
 revisions for those aspects that do not work? Thank-you.
 
 The code that you suggested worked well overall, except for 3 aspects of
 it:
 
 Here I actually needed the binary 0s and 1s rather than a TRUE/FALSE
 logical matrix
 
 # a function to read in the data as a matrix of logicals
 myreadfun - function(file) {
 as.matrix(read.ascii.grid(file)$data)!=0
 }
 
 
 Here I needed to calculate the f2 probability rather than the mean
 
 compare.all - sapply(modeldepths, function(md) mean(md==observeddepth))
 
  str(compare.all)
 num [1:54] 0.936 0.94 0.944 0.944 0.945 ...
 
 Here most of the entries are greater than 0.7, but it should just be 31
 of the
 54 that are greater than 0.7
 
  justbig - modeldepths[[compare.all  0.7]]
 
 Error in modeldepths[[compare.all  0.7]] :
 recursive indexing failed at level 2
 
 Once again, thank-you for your assistance.
 
 Irucka Embry
 
 
 -Original Message-
 From: Adams, Jean [jvad...@usgs.gov]
 Sent: 12/21/2012 10:32:54 AM
 To: iruc...@mail2world.com
 Cc: r-help@r-project.org
 Subject: Re: [R] comparison of large data set
 
 Irucka,
 
 
 I did not test this code out on any data, but I think it will work.
 
 
 Jean
 
 
 
 
 # a function to read in the data as a matrix of logicals
 myreadfun - function(file) {
 as.matrix(read.ascii.grid(file)$data)!=0
 }
 
 
 # names of the 54 modeled depth files
 modfiles - paste0(MaxFloodDepth_, 1:54, .txt)
 
 
 # read in the observed and modeled depth files
 # observeddepth is a matrix
 observeddepth - myreadfun(MaxFloodDepth_Observed.txt)
 
 # modeldepths is a list of matrices
 modeldepths - lapply(filenames, myreadfun)
 
 
 # calculate the proportion of matrix elements that agree with the
 observed file
 # the results is a vector with one number for each modeled depth matrix
 compare.all - sapply(modeldepths, function(md) mean(md==observeddepth))
 
 
 # select just those matrices that had a large proportion of agreements
 # justbig is a list of matrices
 justbig - modeldepths[[compare.all  0.7]]
 
 
 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide
 http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.

 ___
 Get the 

[R] parallel error message extraction (in mclapply)?

2012-12-28 Thread ivo welch
dear R experts---I am looking at a fairly uninformative error in my program:

Error in mclapply(1:nrow(opts), solveme) :
  (converted from warning) all scheduled cores encountered errors in user code

the doc on ?mclapply tells me that

  In addition, each process is running the job inside try(...,
silent=TRUE) so if error occur they will be stored as try-error
objects in the list.

I looked up try, but this manpage creates as many new riddles as it
solves.  so, it isn't obvious to me how I get to the try-error object
list, or how to find out what triggered the abort.  of course, I can
use lapply to debug this, but this could be slow for some programs.
(I know the answer for my problem is that a uniroot deep inside my
code complained that my endpoints were not opposite.)

in general, is there a way to get some more information from mclapply
failures?  could this be added to the docs for mclapply?

/iaw



Ivo Welch (ivo.we...@gmail.com)
http://www.ivo-welch.info/

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


Re: [R] R crashing inconsistently within for loops

2012-12-28 Thread Ben Bolker
Steve Powers powers_s at nd.edu writes:

 
 Hello,
 
 This one has been bugging me for a long time and I have never found a
 solution. I am using R version 2.15.1 but it has come 
 up in older versions of R I have used over the past 2-3 years.
 
 Q: Am I wrong to expect that R should handle hundreds of iterations of the
 base model or statistical functions, embedded within for loops, in one
 script run? I have found that when I write scripts that do this, sometimes
 they have a tendency to crash, seemingly unpredictably.
 
 For example, one problem script of mine employs glm and gls about a hundred
 different times, and output files are being written at the end of each
 iteration. I have used my output files to determine that the crash cause is
 not consistent (R never fails at the same iteration). Note that the data are
 fixed here (no data generation or randomization steps, so that is not the
 issue). But it is clear that scripts with larger numbers of iterations are
 more likely to produce a crash.
 
 And a year or two ago, I had a seemingly stable R script again with for
 looped model fits, but discovered this script was prone to crashing when I
 ran it on a newer PC. Because the new PC also seemed to be blazing through R
 code absurdly fast, I tried adding a short fluff procedure at the end of
 each iteration that required a few seconds of processing time. Low and
 behold, when I added that, the script stopped crashing (and each iteration
 of course took longer). I still don't understand why that fixed things.

  All of the advice given so far is useful (I think), but I thought I would
chime in and answer one of your questions, which is that this is indeed
surprising, especially if you're using only base packages.  I and many
other people routinely run thousands of iterations of these types of
analyses with no problem.  If (as Brian Ripley suggested) crash just
means that some of your scripts stop with errors in some cases, then
that's *not* surprising -- there are lots of ways to get glm() and gls()
to give errors with slightly weird data sets.  However, having R actually
crash (i.e. the whole R session 'terminates abnormally', in the words
of the posting guide) is quite a bit more unusual, and (if you are only
using base R, not any contributed packages that may call compiled code
in bad ways [or calling your own compiled code]) *always* constitutes
a bug.  Nondeterministic behavior in a deterministic function (i.e.
no random number generation) is also unusual/surprising.

 The key here is finding a reproducible example, which can be tough
for these kinds of problems.  But it sounds like you have one, so
if you can trim it down to a manageable size
 (see http://tinyurl.com/reproducible-000 for tips on creating
reproducible examples), and give full information about your system
(as suggested by another poster), this would be of great interest,
especially to the people who hang out on r-de...@r-project.org .
My prior probabilities are fairly strongly on the problem being
something flaky about your system, but helping to identify these
kinds of bugs in R is a community service ...

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] Problem to load a shape file

2012-12-28 Thread dms
Dear all, I've been puzzled why I not able to load a shapefile from a
connection. Does anyone here can give a reasonable answer?
When I try the following script I got this error:
Error in getinfo.shape(fn) : Error opening SHP file


#Reproduction

temp - tempfile()
download.file(http://metodologiapolitica.com/download/Brasil.zip,temp)
shp.br - readShapePoly(unzip(temp, Brasil.shp))
unlink(temp)

Just for acknowledgement, I successful can load the file whenever I open
it from the disk.

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.