[R] Ridge regression and mixed models

2010-10-04 Thread harez...@post.harvard.edu 
Dear R users,
  An equivalence between linear mixed model formulation and penalized 
regression 
models (including the ridge regression and penalized regression splines) has 
proven to be very useful in many aspects. Examples include the use of the lme() 
function in the library(nlme) to fit smooth models including the estimation of 
a 
smoothing parameter using REML. My question concerns the use of the linear 
mixed 
model software to fit a ridge regression with the number of columns in the 
design matrix X (p) exceeding the number of observations (n). Has anybody in 
the 
R community implemented the LME-like approach with estimation of the variance 
components using REML to find the coefficient estimates (BLUEs) and predictors 
(BLUPs) in the ridge regression problem in the p  n  setting?

Sample code below summarizes my problem:

version$version.string
# [1] R version 2.11.1 (2010-05-31)

library(nlme)

# DATA generation:
dim - 200
n - 50
XX - matrix(rnorm(dim*n, 0, 0.1), ncol=dim, nrow=n)
beta - matrix(c(rep(1, 40), rep(2,20), rep(0,140)), ncol=1)
Y - XX %*% beta + rnorm(n)

# MODEL fit:
dummyId - factor(rep(1,n))
Z.block - list(dummyId=pdIdent(~-1+XX))
data.fr - data.frame(Y,XX)
fit - lme(Y~1,
data=data.fr, 
random=Z.block)

# ERROR:
Warning message:
In lme.formula(Y ~ 1, data = data.fr, random = Z.block) :
  Fewer observations than random effects in all level 1 groups
#

Thank you in advance,
Jarek  Harezlak


  
[[alternative HTML version deleted]]

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

Re: [R] Ridge regression and mixed models

2010-10-04 Thread Dimitri Liakhovitski 
Curious - what would be the purpose of this regression?

On Mon, Oct 4, 2010 at 4:39 PM, harez...@post.harvard.edu
jarek...@yahoo.com wrote:
 Dear R users,
  An equivalence between linear mixed model formulation and penalized 
 regression
 models (including the ridge regression and penalized regression splines) has
 proven to be very useful in many aspects. Examples include the use of the 
 lme()
 function in the library(nlme) to fit smooth models including the estimation 
 of a
 smoothing parameter using REML. My question concerns the use of the linear 
 mixed
 model software to fit a ridge regression with the number of columns in the
 design matrix X (p) exceeding the number of observations (n). Has anybody in 
 the
 R community implemented the LME-like approach with estimation of the variance
 components using REML to find the coefficient estimates (BLUEs) and predictors
 (BLUPs) in the ridge regression problem in the p  n  setting?

 Sample code below summarizes my problem:
 
 version$version.string
 # [1] R version 2.11.1 (2010-05-31)

 library(nlme)

 # DATA generation:
 dim - 200
 n - 50
 XX - matrix(rnorm(dim*n, 0, 0.1), ncol=dim, nrow=n)
 beta - matrix(c(rep(1, 40), rep(2,20), rep(0,140)), ncol=1)
 Y - XX %*% beta + rnorm(n)

 # MODEL fit:
 dummyId - factor(rep(1,n))
 Z.block - list(dummyId=pdIdent(~-1+XX))
 data.fr - data.frame(Y,XX)
 fit - lme(Y~1,
        data=data.fr,
        random=Z.block)

 # ERROR:
 Warning message:
 In lme.formula(Y ~ 1, data = data.fr, random = Z.block) :
  Fewer observations than random effects in all level 1 groups
 #

 Thank you in advance,
 Jarek  Harezlak



        [[alternative HTML version deleted]]

 __
 R-help@r-project.org mailing list
 https://stat.ethz.ch/mailman/listinfo/r-help
 PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
 and provide commented, minimal, self-contained, reproducible code.




-- 
Dimitri Liakhovitski
Ninah Consulting
www.ninah.com

__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.