Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Martin Maechler
> jeremiah rounds > on Thu, 21 Jul 2016 13:56:17 -0700 writes: > I appreciate the timing, so much so I changed the code to show the issue. > It is a problem of scale. > roll_lm probably has a heavy start-up cost but otherwise completely > out-performs those other vers

Re: [R] How to plot marginal effects (MEM) in R?

2016-07-21 Thread Faradj Koliev
Dear David Winsemius, Thank you! The sample make no sense, I know. The real data is too big. So, I only want to understand how to plot marginal effects, to visualize them in a proper way. Best, > 22 juli 2016 kl. 08:35 skrev David Winsemius : > >> >> On Jul 21, 2016, at 2:22 PM, Faradj

Re: [R] How to plot marginal effects (MEM) in R?

2016-07-21 Thread David Winsemius
> On Jul 21, 2016, at 2:22 PM, Faradj Koliev wrote: > > Dear all, > > I have two logistic regression models: > > > • model <- glm(Y ~ X1+X2+X3+X4, data = data, family = "binomial") > > > > • modelInteraction <- glm(Y ~ X1+X2+X3+X4+X1*X4, data = data, family = > "binomial") > > To ca

Re: [R] installing the lubricate package

2016-07-21 Thread Ismail SEZEN
You don't have to download and install from github. You can install lubridate package easly from cran repository. If you really intend to install from github, i advise you install devtools package first and use install_github function. http://www.inside-r.org/packages/cran/devtools/docs/install_gi

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Mark Leeds
Hi Jeremiah: I think I wasn't that clear. I'm not suggesting the kalman filter to deal with time varying coefficients. As Roy pointed out, one can use the kalman filter to do regular regression where one "sees" a new data point as each time unit passes. It can be assumed that the coefficients do n

[R] adding new values to dataframe based on existing values in two columns

2016-07-21 Thread Alexander.Herr
Hiya, I am trying to assign minimum values to a dataframe based on existing columns. I can do this via loops, but surely R has much more elegant solutions... Here is my code: set.seed(666) xyz<-as.data.frame(cbind(x=rep(rpois(50,10),2)+1, y=rep(rpois(50,10),2)+1,z=runif(100, min=-3, max=40)))

[R] Why the order of parameters in a logistic regression affects results significantly?

2016-07-21 Thread Qinghua He via R-help
Using the same data, if I ran fit2 <-glm(formula=AR~Age+LumA+LumB+HER2+Basal+Normal,family=binomial,data=RacComp1)summary(fit2)exp(coef(fit2))   I obtained: > exp(coef(fit2))(Intercept)         Age        LumA        LumB        HER2     >   Basal      Normal  0.24866935  1.00433781  0.10639937  

[R] installing the lubricate package

2016-07-21 Thread lily li
Hi R users, I'm trying to download lubricate from this website, and then install it on my mac. https://github.com/hadley/lubridate but it says windows version does not apply to mac. How to install the package for mac? Thanks. [[alternative HTML version deleted]]

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Jean-Claude Arbaut
This may be useful: Sven Hammarling and Craig Lucas "Updating the QR factorization and the least squares problem" http://eprints.ma.man.ac.uk/1192/01/covered/MIMS_ep2008_111.pdf http://www.maths.manchester.ac.uk/~clucas/updating/ 2016-07-21 20:02 GMT+02:00 jeremiah rounds : > Hi, > > A not unusua

[R] PDF extraction with tm package

2016-07-21 Thread Steven Kang
Hi R users, I’m having some issues trying to extract texts from PDF file using tm package. Here are the steps that were carried out: 1. Downloaded and installed the following programs: - Xpdf (Copied the ‘bin32’, ‘bin64’, ‘doc’ folders into ‘C:\Program Files\Xpdf’ directory; also added C:\Progr

Re: [R] about interpolating data in r

2016-07-21 Thread Ismail SEZEN
> On 22 Jul 2016, at 01:54, lily li wrote: > > Thanks, I meant if there are missing data at the beginning and end of a > dataframe, how to interpolate according to available data? > > For example, the A column has missing values at the beginning and end, how > to interpolate linearly between 10

Re: [R] about interpolating data in r

2016-07-21 Thread Ismail SEZEN
> On 22 Jul 2016, at 01:34, lily li wrote: > > I have a question about interpolating missing values in a dataframe. First of all, filling missing values action must be taken into account very carefully. It must be known the nature of the data that wanted to be filled and most of the time, to

Re: [R] about interpolating data in r

2016-07-21 Thread lily li
Thanks, I meant if there are missing data at the beginning and end of a dataframe, how to interpolate according to available data? For example, the A column has missing values at the beginning and end, how to interpolate linearly between 10 and 12 for the missing values? df <- data.frame(A=c(NA,

Re: [R] about interpolating data in r

2016-07-21 Thread William Dunlap via R-help
Try approx(), as in: df <- data.frame(A=c(10,11,12),B=c(5,5,4),C=c(3.3,4,3),time=as.Date(c("1990-01-01","1990-02-07","1990-02-14"))) with(df, approx(x=time, y=C, xout=seq(min(time), max(time), by="days"))) Do you notice how one can copy and paste that example out of the mail an into R to see how

Re: [R] how to expand the dataframe

2016-07-21 Thread William Dunlap via R-help
Depending on your situation you may want all=TRUE, all.x=TRUE, or all.y=TRUE. I think the SQL people call these outer joins, left outer joins, and right outer joins. Bill Dunlap TIBCO Software wdunlap tibco.com On Thu, Jul 21, 2016 at 2:35 PM, lily li wrote: > I use this code, and it works. So

Re: [R] plotCI linetypes

2016-07-21 Thread Jim Lemon
Hi Florian, As I suspected, you will have to use something other than the "arrows" function, which most people use to draw "error bars". This is where the "lty" argument gets used for all of the lines. You could substitute a function like this: foobars<-function(x0,y0,x1,y1,length=0.02,col=par("fg

[R] about interpolating data in r

2016-07-21 Thread lily li
I have a question about interpolating missing values in a dataframe. The dataframe is in the following, Column C has no data before 2009-01-05 and after 2009-12-31, how to interpolate data for the blanks? That is to say, interpolate linearly between these two gaps using 5.4 and 6.1? Thanks. df ti

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread jeremiah rounds
I agree that when appropriate Kalman Filter/Smoothing the higher-quality way to go about estimating a time-varying coefficient (given that is what they do), and I have noted that both the R package "dlm" and the function "StructTS" handle these problems quickly. I am working on that in parallel.

Re: [R] how to expand the dataframe

2016-07-21 Thread lily li
I use this code, and it works. So has to set 'all=TRUE'. merge(df, data.frame(time=seq(as.Date("1990-01-01"), to=as.Date("1990-12-31"), by="days")), all=TRUE) On Thu, Jul 21, 2016 at 9:22 AM, Daniel Nordlund wrote: > On 7/20/2016 8:26 PM, lily li wrote: > >> Yes, I tried to create a dataframe

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Roy Mendelssohn - NOAA Federal
I have no idea which method produces the fastest results, but the package KFAS has a function to do recursive regressions using the Kalman filter. One difference is that it is not, as far as a I can telll, a moving window (so past data are being dropped), just a recursively computed regressio

[R] How to plot marginal effects (MEM) in R?

2016-07-21 Thread Faradj Koliev
Dear all, I have two logistic regression models: • model <- glm(Y ~ X1+X2+X3+X4, data = data, family = "binomial") • modelInteraction <- glm(Y ~ X1+X2+X3+X4+X1*X4, data = data, family = "binomial") To calculate the marginal effects (MEM approach) for these models, I used the `mfx` p

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Mark Leeds
Hi Jermiah: another possibly faster way would be to use a kalman filtering framework. I forget the details but duncan and horne have a paper which shows how a regression can be re-computed each time a new data point is added .I forget if they handle taking one off of the back also which is what you

[R] Aggregate data to lower resolution

2016-07-21 Thread Miluji Sb
Dear all, I have the following GDP data by latitude and longitude at 0.5 degree by 0.5 degree. temp <- dput(head(ptsDF,10)) structure(list(longitude = c(-68.25, -67.75, -67.25, -68.25, -67.75, -67.25, -71.25, -70.75, -69.25, -68.75), latitude = c(-54.75, -54.75, -54.75, -54.25, -54.25, -54.25, -5

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread jeremiah rounds
I appreciate the timing, so much so I changed the code to show the issue. It is a problem of scale. roll_lm probably has a heavy start-up cost but otherwise completely out-performs those other versions at scale. I suspect you are timing the nearly constant time start-up cost in small data. I

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Gabor Grothendieck
I would be careful about making assumptions regarding what is faster. Performance tends to be nonintuitive. When I ran rollapply/lm, rollapply/fastLm and roll_lm on the example you provided rollapply/fastLm was three times faster than roll_lm. Of course this could change with data of different di

Re: [R] how to stop interpretation in system()

2016-07-21 Thread Michael Peng
Yes. I forgot to escape the escape. Thank you so much. 2016-07-21 16:07 GMT-04:00 Sarah Goslee : > You could escape the backslash. > > system("cmd 'a\\tb'") > > > On Thu, Jul 21, 2016 at 4:00 PM, Michael Peng > wrote: > > Hi, > > > > I am trying to use system() to run some command in OS. such a

Re: [R] how to stop interpretation in system()

2016-07-21 Thread Sarah Goslee
You could escape the backslash. system("cmd 'a\\tb'") On Thu, Jul 21, 2016 at 4:00 PM, Michael Peng wrote: > Hi, > > I am trying to use system() to run some command in OS. such as > > system("cmd 'a\tb') > > however, it alway runs > cmd 'ab' > instead of > cmd 'a\tb' > > How can I prevent

[R] how to stop interpretation in system()

2016-07-21 Thread Michael Peng
Hi, I am trying to use system() to run some command in OS. such as system("cmd 'a\tb') however, it alway runs cmd 'ab' instead of cmd 'a\tb' How can I prevent system to interpret 'a\tb' to 'ab'? Thanks [[alternative HTML version deleted]] ___

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread jeremiah rounds
Thanks all. roll::roll_lm was essentially what I wanted. I think maybe I would prefer it to have options to return a few more things, but it is the coefficients, and the remaining statistics you might want can be calculated fast enough from there. On Thu, Jul 21, 2016 at 12:36 PM, Achim Zeile

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Achim Zeileis
Jeremiah, for this purpose there are the "roll" and "RcppRoll" packages. Both use Rcpp and the former also provides rolling lm models. The latter has a generic interface that let's you define your own function. One thing to pay attention to, though, is the numerical reliability. Especially o

Re: [R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread Gabor Grothendieck
Just replacing lm with a faster version would speed it up. Try lm.fit or even faster is fastLm in the RcppArmadillo package. On Thu, Jul 21, 2016 at 2:02 PM, jeremiah rounds wrote: > Hi, > > A not unusual task is performing a multiple regression in a rolling window > on a time-series.A stand

[R] C/C++/Fortran Rolling Window Regressions

2016-07-21 Thread jeremiah rounds
Hi, A not unusual task is performing a multiple regression in a rolling window on a time-series.A standard piece of advice for doing in R is something like the code that follows at the end of the email. I am currently using an "embed" variant of that code and that piece of advice is out there

Re: [R] readline issue with 3.3.1

2016-07-21 Thread Martin Maechler
> Ralf Goertz > on Wed, 20 Jul 2016 16:37:53 +0200 writes: > Am Wed, 20 Jul 2016 11:35:31 +0200 > schrieb Ralf Goertz : >> Hi, >> >> after a recent update to version 3.3.1 on Opensuse Leap I have >> problems with command lines longer than the terminal width.

Re: [R] how to expand the dataframe

2016-07-21 Thread Daniel Nordlund
On 7/20/2016 8:26 PM, lily li wrote: Yes, I tried to create a dataframe and merge it with the shortened dataframe. The resulting dataframe goes with the short one and truncates the complete date column, so it does not work. On Wed, Jul 20, 2016 at 6:38 PM, David Winsemius wrote: On Jul 20,

[R] Reading PCorpus created using tm package

2016-07-21 Thread Paul Johnston
Dear All I have created a permanent corpus and I can see the file exist: setwd("E:/textmining/texts") (data_mined_permanent <- PCorpus(DirSource(("E:/textmining/texts")), readerControl = list(languages = "eng"), dbControl = list(

Re: [R] Issue with Transform function in R

2016-07-21 Thread tom
Not sure I can translate the format of your Date column correctly, however the command DF1$Date <- as.Date(DF1$Date, format=’formatstr’) Will convert the dates into a format correctly handled by R. ?strptime Should give you an idea of what formatstr should look like. I.e. if date = 160721 as.

Re: [R] txtProgressBar()

2016-07-21 Thread tom
You may like to look at ?suppressMessages P.S. sorry for posting in HTML, new laptop and it’s next on my list of things to fix. From: Ivan Calandra [[alternative HTML version deleted]] __ R-help@r-project.org mailing list -- To UNSUBSCRIBE a

Re: [R] Issue with Transform function in R

2016-07-21 Thread Ivan Calandra
Oops, missed that one! -- Ivan Calandra, PhD Scientific Mediator University of Reims Champagne-Ardenne GEGENAA - EA 3795 CREA - 2 esplanade Roland Garros 51100 Reims, France +33(0)3 26 77 36 89 ivan.calan...@univ-reims.fr -- https://www.researchgate.net/profile/Ivan_Calandra https://publons.com/a

Re: [R] Help installing Rcmdr on Mac

2016-07-21 Thread Duncan Murdoch
On 20/07/2016 11:47 PM, Judy Munday wrote: I am having trouble installing R Commander on a Mac (OS X 10.7.5). I would really appreciate some help! I receive the following message after following the instructions to install Rcmdr via Package Installer: Warning: dependency 'rglwidget' is not

Re: [R] Help installing Rcmdr on Mac

2016-07-21 Thread Fox, John
Dear Judy, I see that the rglwidget package *is* available on CRAN but also that you're using a very old version of Mac OS X. You may have to upgrade OS X and install the current version of R (which is not available for OS X 10.7) in order to use the Rcmdr package. You may also be able to get

Re: [R] Issue with Transform function in R

2016-07-21 Thread ruipbarradas
Hello, Another thing to consider is to use Variable1 = NA, not '=='. With '==' it will probably return TRUE/FALSE/NA. Hope this helps, Rui Barradas   Citando Ivan Calandra : > This might not be the whole story, but part of the problem is that > you want to select a _*character string*_ great

Re: [R] splitting a vector of strings

2016-07-21 Thread Michael Dewey
Dear Eric I think you are looking for sub or gsub Without an example set of input and output I am not quite sure but you would need to define an expression which matches your separator (;) followed by any characters up to the end of line. If you have trouble with that then someone here will n

Re: [R] Issue with Transform function in R

2016-07-21 Thread Ivan Calandra
This might not be the whole story, but part of the problem is that you want to select a _*character string*_ greater/smaller than another. That doesn't make much sense! I am not sure how to best compare two dates, but if you convert the Date values into numeric, then that would work. The probl

Re: [R] splitting a vector of strings

2016-07-21 Thread Ben Tupper
Hi, I'm not sure about the more generalized solution, but how about this for a start. x <- c("a;b;c", "d;e", "foo;g;h;i") x #[1] "a;b;c" "d;e" "foo;g;h;i" sapply(strsplit(x, ";",fixed = TRUE), '[',1) #[1] "a" "d" "foo" If you want elegance then I suggest you take a look at the s

[R] splitting a vector of strings

2016-07-21 Thread Eric Elguero
Hi everybody, I have a vector of character strings. Each string has the same pattern and I want to split them in pieces and get a vector made of the first pieces of each string. The problem is that strsplit returns a list. All I found is uu<- matrix(unlist(strsplit(x,";")),ncol=3,byrow=T)[,1]

[R] Help installing Rcmdr on Mac

2016-07-21 Thread Judy Munday
I am having trouble installing R Commander on a Mac (OS X 10.7.5). I would really appreciate some help! I receive the following message after following the instructions to install Rcmdr via Package Installer: Warning: dependency 'rglwidget' is not available trying URL 'http://cran.ms.unimel

[R] Issue with Transform function in R

2016-07-21 Thread Bhaskar Mitra
Hello Everyone, I am trying to replace the values in the 2nd column (Variable 1) corresponding to certain dates (Date) with NAs as shown below. Both Date and Variable1 are numeric vectors . I am trying to use the transform function as shown below but it doesn’t seem to work even though if I

[R] R Toolbox (Release 2 of 2016-07-21)

2016-07-21 Thread G . Maubach
Hi All, I have uploaded a new release of the R Toolbox. R Toolbox is a collection of simple but useful functions which I developed for myself to shorten the develoment process. Currently all functions use base R. No other packages are needed. One exception is "t_openxlsx" cause this module dea

Re: [R] txtProgressBar()

2016-07-21 Thread Ivan Calandra
Thank you Greg, This is what I figured out... The problem with txtProgressBar() is that many packages display some information during installation (even with quiet=TRUE), especially the installation of dependencies, so that the progress bar is not very useful. So I have tried with tkProgressBa