Re: [R] How to do double (nested) parSapply in R?

2017-02-02 Thread Henrik Bengtsson
Quick comment: sapply() / parSapply() can behaves "unexpectedly". To
troubleshoot this, use parLapply() instead to see if you at least get the
individual results you think you should get.

Henrik

On Feb 2, 2017 08:03, "Art U"  wrote:

> Hello,
>
> I have a data orig.raw that contains 8 predictors and 1 outcome variable.
> I'm trying to run simulation (bootstrap) and create, let's say, 10
> confidence intervals for coefficients estimated by LASSO. I did it with
> regular function replicate, but now I want to do it by using parallel
> programming. Here is my code:
>
> cl <- makeCluster(detectCores())
> clusterEvalQ(cl,library(glmnet))
> clusterEvalQ(cl,library(MASS))
> clusterExport(cl,c("orig.raw"))
>
> pp=parSapply(cl,1:10,function(i,data=orig.raw,...){
>   library(parallel)
>   cl <- makeCluster(detectCores())
>   clusterEvalQ(cl,library(glmnet))
>   clusterEvalQ(cl,library(MASS))
>   clusterExport(cl,c("orig.raw"))
>
>   repl=parSapply(cl=cl,1:10,function(i,data=orig.raw,...){
> s1=data[sample(nrow(data),size=500,replace=TRUE),]
> cross=cv.glmnet(s1[,1:8],s1[,9])
> penalty <- cross$lambda.min
> fit=glmnet(s1[,1:8],s1[,9],alpha=1,lambda=penalty)
> coe=coef(fit)
> m=as(coe, "matrix")
> return(m)
>   })
>
> stopCluster(cl)
>
>   mr=t(matrix(repl,nrow = 9,ncol=10))
>   means=colMeans(mr)
>   std=apply(mr, 2, sd)
>   lb=means-1.96*std;
>   ub=means+1.96*std;
>   ind=t(as.numeric({beta>lb & beta   return(ind)})
> stopCluster(cl)
>
> And here is the error I'm getting
>
> Error in checkForRemoteErrors(val) : 8 nodes produced errors; first
> error: comparison (6) is possible only for atomic and list types
>
> If I run only function repl - it works and I get the matrix that contains
> coefficients from 10 runs.
>
> Can you please help me to solve the problem?
> Regards,
> Ariel
>
> [[alternative HTML version deleted]]
>
> __
> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/
> posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


[R] How to do double (nested) parSapply in R?

2017-02-02 Thread Art U
Hello,

I have a data orig.raw that contains 8 predictors and 1 outcome variable.
I'm trying to run simulation (bootstrap) and create, let's say, 10
confidence intervals for coefficients estimated by LASSO. I did it with
regular function replicate, but now I want to do it by using parallel
programming. Here is my code:

cl <- makeCluster(detectCores())
clusterEvalQ(cl,library(glmnet))
clusterEvalQ(cl,library(MASS))
clusterExport(cl,c("orig.raw"))

pp=parSapply(cl,1:10,function(i,data=orig.raw,...){
  library(parallel)
  cl <- makeCluster(detectCores())
  clusterEvalQ(cl,library(glmnet))
  clusterEvalQ(cl,library(MASS))
  clusterExport(cl,c("orig.raw"))

  repl=parSapply(cl=cl,1:10,function(i,data=orig.raw,...){
s1=data[sample(nrow(data),size=500,replace=TRUE),]
cross=cv.glmnet(s1[,1:8],s1[,9])
penalty <- cross$lambda.min
fit=glmnet(s1[,1:8],s1[,9],alpha=1,lambda=penalty)
coe=coef(fit)
m=as(coe, "matrix")
return(m)
  })

stopCluster(cl)

  mr=t(matrix(repl,nrow = 9,ncol=10))
  means=colMeans(mr)
  std=apply(mr, 2, sd)
  lb=means-1.96*std;
  ub=means+1.96*std;
  ind=t(as.numeric({beta>lb & beta