Re: [R] Bayesian regression stepwise function?
On Fri, Oct 23, 2009 at 6:05 PM, Ravi Varadhan rvarad...@jhmi.edu wrote: Frank, I have heard this (i.e. only head-to-head comparisons are valid) and various other folklores about AIC and BIC based model selection before, including one that these information criteria are only applicable for comparing two nested models. Where has it been demonstrated that AIC/BIC cannot be used to find the best subset, i.e. the subset that is closest to the true model (assuming that true model is contained in the set of models considered, and that maximum likelihood estimation is used for estimating parameters in the models)? I would greatly appreciate any reference that shows this. You could look at Claeskens/Hjort: Model Selection and Model Averaging, for instance section 2.3, AIC and the Kullback-Leibler distance. AIC corresponds to searching for the model with smallest KL distance to thruth, while BIC aims at consistent selection of the true model (assuming that beast exists!) Then they go on to show that this two goals cannot be reconciled. Kjetil Best, Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: Frank E Harrell Jr [mailto:f.harr...@vanderbilt.edu] Sent: Friday, October 23, 2009 4:04 PM To: Ravi Varadhan Cc: jlu...@ria.buffalo.edu; 'Allan.Y'; r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? Ravi Varadhan wrote: The stepAIC() function in MASS can be used, with k = log(n), to implement your suggestion of quasi-Bayesian stepwise selection using the BIC criterion. Ravi. Although many statisticians use BIC otherwise, it was only designed to compare two pre-specified models. Frank --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of jlu...@ria.buffalo.edu Sent: Friday, October 23, 2009 2:31 PM To: Allan.Y Cc: r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? The BIC (Raftery) can be used for quasi-Bayesian model selection, but it's not stepwise. Ntzoufras shows how to use WinBUGS to conduct Bayesian model selection, but again it's not stepwise Ntzoufras, I. (2002), 'Gibbs variable selection using BUGS', Journal of Statistical Software 7(7), 1--19. Ntzoufras, I. (2009), Bayesian modeling using WinBUGS, Wiley, Hoboken, NJ. Raftery, A. E. (1995), 'Bayesian model selection in social research', Sociological Methodology 25, 111-163. Allan.Y all...@cmu.edu Sent by: r-help-boun...@r-project.org 10/22/2009 01:09 PM To r-help@r-project.org cc Subject [R] Bayesian regression stepwise function? Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Charles C. Berry wrote: On Thu, 22 Oct 2009, Ben Bolker wrote: Allan.Y wrote: Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Why? That seems so ... un-Bayesian ... If 'fools rush in where angels fear to tread', then Bayesians 'jump' in where frequentists fear to 'step'... Seriously, there are Bayesian regression approaches that priorize the model size (sometimes only implicitly by assigning a prior for the inclusion of each candidate regressors). Then they 'jump' between models of different sizes. On CRAN, Package qtlbim (which is specialized to a particular genetics problem) implements one such, I think. Package bqtl does not implement the jumping approach, but does explore a model space with differing numbers of regressors for the same (qtl) problem. Perhaps the closest to a general purpose 'stepwise flavored' Bayesian regression is implemented in Package BMA, which IIRC borrows step() for some of its work. But CRAN now has more packages than my cortex has neurons, so there are probably more packages that do something like this. Try RSiteSearch(jump regression, restric='functions') and start reading. library(sos) jr - ???'jump regression' summary(jr) # Downloaded 22 links in 14 packages. # packages with 3 help pages matching 'jump regression': # monomvn, CalciOMatic, polydect; # qtl has 1 ... bs - ???'Bayesian step' bsw - ???'Bayesian stepwise' bs. - bs|bsw summary(bs.) Total number of matches: 119 Downloaded 115 links in 63 packages. Packages with at least 2 matches using search pattern 'Bayesian+step | Bayesian+stepwise': Package Count MaxScore TotalScoreDate 1 DPpackage 82 11 2009-03-17 06:38:42 2 BMA 61 6 2009-05-01 12:21:01 3 ensembleBMA 51 5 2009-07-01 07:16:43 4 MMIX 43 8 2009-08-16 05:47:37 5mclust 42 8 2009-07-14 05:39:10 6adlift 41 4 2009-02-01 06:31:16 7 tgp 34 8 2009-07-29 11:14:36 8 geoRglm 34 6 2009-10-19 19:21:57 9 G1DBN 32 5 2008-01-24 16:27:29 10 bayesm 31 3 2008-06-14 15:31:09 11 bqtl 31 3 2009-01-15 07:02:43 12 qtlbim 2 12 13 2009-07-29 10:50:13 13 BAYSTAR 2 11 12 2009-10-19 19:17:18 14 gap 24 5 2009-01-15 07:03:58 15 MSBVAR 24 5 2009-07-29 10:49:07 16bayesSurv 23 5 2008-12-15 10:06:16 17 bnlearn 22 3 2009-09-01 08:25:43 18 spBayes 22 3 2009-03-28 07:33:41 19 dlm 21 2 2009-07-01 07:16:28 20GOFSN 21 2 2009-07-01 07:47:10 21 MCMChybridGP 21 2 2009-10-01 05:27:43 22 mgcv 21 2 2009-08-24 16:03:12 23nlreg 21 2 2007-11-02 13:14:56 24 nlt 21 2 2009-03-02 07:30:34 25 pARtial 21 2 2006-11-01 02:59:23 26plink 21 2 2009-07-01 07:19:44 27 qtl 21 2 2009-09-17 15:46:05 28 timsac 21 2 2009-03-28 07:27:28 # This took more longer to describe it than run. # Hope this helps. # Spencer HTH, Chuck -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p26015081.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Charles C. Berry(858) 534-2098 Dept of Family/Preventive Medicine E mailto:cbe...@tajo.ucsd.eduUC San Diego http://famprevmed.ucsd.edu/faculty/cberry/ La Jolla, San Diego 92093-0901 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- Spencer Graves, PE, PhD President and Chief Operating Officer Structure Inspection and Monitoring, Inc. 751 Emerson Ct. San José, CA 95126 ph: 408-655-4567 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help
Re: [R] Bayesian regression stepwise function?
If 'fools rush in where angels fear to tread', then Bayesians 'jump' in where frequentists fear to 'step'... Very nice, Chuck! Definitely one for my list of fortunes. Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of Charles C. Berry Sent: Friday, October 23, 2009 1:18 AM To: Ben Bolker Cc: r-help@r-project.org Subject: Re: [R] Bayesian regression stepwise function? On Thu, 22 Oct 2009, Ben Bolker wrote: Allan.Y wrote: Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Why? That seems so ... un-Bayesian ... If 'fools rush in where angels fear to tread', then Bayesians 'jump' in where frequentists fear to 'step'... Seriously, there are Bayesian regression approaches that priorize the model size (sometimes only implicitly by assigning a prior for the inclusion of each candidate regressors). Then they 'jump' between models of different sizes. On CRAN, Package qtlbim (which is specialized to a particular genetics problem) implements one such, I think. Package bqtl does not implement the jumping approach, but does explore a model space with differing numbers of regressors for the same (qtl) problem. Perhaps the closest to a general purpose 'stepwise flavored' Bayesian regression is implemented in Package BMA, which IIRC borrows step() for some of its work. But CRAN now has more packages than my cortex has neurons, so there are probably more packages that do something like this. Try RSiteSearch(jump regression, restric='functions') and start reading. HTH, Chuck -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p2601 5081.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Charles C. Berry(858) 534-2098 Dept of Family/Preventive Medicine E mailto:cbe...@tajo.ucsd.edu UC San Diego http://famprevmed.ucsd.edu/faculty/cberry/ La Jolla, San Diego 92093-0901 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
The BIC (Raftery) can be used for quasi-Bayesian model selection, but it's not stepwise. Ntzoufras shows how to use WinBUGS to conduct Bayesian model selection, but again it's not stepwise Ntzoufras, I. (2002), 'Gibbs variable selection using BUGS', Journal of Statistical Software 7(7), 1--19. Ntzoufras, I. (2009), Bayesian modeling using WinBUGS, Wiley, Hoboken, NJ. Raftery, A. E. (1995), 'Bayesian model selection in social research', Sociological Methodology 25, 111-163. Allan.Y all...@cmu.edu Sent by: r-help-boun...@r-project.org 10/22/2009 01:09 PM To r-help@r-project.org cc Subject [R] Bayesian regression stepwise function? Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p26013725.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
The stepAIC() function in MASS can be used, with k = log(n), to implement your suggestion of quasi-Bayesian stepwise selection using the BIC criterion. Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of jlu...@ria.buffalo.edu Sent: Friday, October 23, 2009 2:31 PM To: Allan.Y Cc: r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? The BIC (Raftery) can be used for quasi-Bayesian model selection, but it's not stepwise. Ntzoufras shows how to use WinBUGS to conduct Bayesian model selection, but again it's not stepwise Ntzoufras, I. (2002), 'Gibbs variable selection using BUGS', Journal of Statistical Software 7(7), 1--19. Ntzoufras, I. (2009), Bayesian modeling using WinBUGS, Wiley, Hoboken, NJ. Raftery, A. E. (1995), 'Bayesian model selection in social research', Sociological Methodology 25, 111-163. Allan.Y all...@cmu.edu Sent by: r-help-boun...@r-project.org 10/22/2009 01:09 PM To r-help@r-project.org cc Subject [R] Bayesian regression stepwise function? Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p2601 3725.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. [[alternative HTML version deleted]] __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Ravi Varadhan wrote: The stepAIC() function in MASS can be used, with k = log(n), to implement your suggestion of quasi-Bayesian stepwise selection using the BIC criterion. Ravi. Although many statisticians use BIC otherwise, it was only designed to compare two pre-specified models. Frank --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of jlu...@ria.buffalo.edu Sent: Friday, October 23, 2009 2:31 PM To: Allan.Y Cc: r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? The BIC (Raftery) can be used for quasi-Bayesian model selection, but it's not stepwise. Ntzoufras shows how to use WinBUGS to conduct Bayesian model selection, but again it's not stepwise Ntzoufras, I. (2002), 'Gibbs variable selection using BUGS', Journal of Statistical Software 7(7), 1--19. Ntzoufras, I. (2009), Bayesian modeling using WinBUGS, Wiley, Hoboken, NJ. Raftery, A. E. (1995), 'Bayesian model selection in social research', Sociological Methodology 25, 111-163. Allan.Y all...@cmu.edu Sent by: r-help-boun...@r-project.org 10/22/2009 01:09 PM To r-help@r-project.org cc Subject [R] Bayesian regression stepwise function? Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Frank, I have heard this (i.e. only head-to-head comparisons are valid) and various other folklores about AIC and BIC based model selection before, including one that these information criteria are only applicable for comparing two nested models. Where has it been demonstrated that AIC/BIC cannot be used to find the best subset, i.e. the subset that is closest to the true model (assuming that true model is contained in the set of models considered, and that maximum likelihood estimation is used for estimating parameters in the models)? I would greatly appreciate any reference that shows this. Best, Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: Frank E Harrell Jr [mailto:f.harr...@vanderbilt.edu] Sent: Friday, October 23, 2009 4:04 PM To: Ravi Varadhan Cc: jlu...@ria.buffalo.edu; 'Allan.Y'; r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? Ravi Varadhan wrote: The stepAIC() function in MASS can be used, with k = log(n), to implement your suggestion of quasi-Bayesian stepwise selection using the BIC criterion. Ravi. Although many statisticians use BIC otherwise, it was only designed to compare two pre-specified models. Frank --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of jlu...@ria.buffalo.edu Sent: Friday, October 23, 2009 2:31 PM To: Allan.Y Cc: r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? The BIC (Raftery) can be used for quasi-Bayesian model selection, but it's not stepwise. Ntzoufras shows how to use WinBUGS to conduct Bayesian model selection, but again it's not stepwise Ntzoufras, I. (2002), 'Gibbs variable selection using BUGS', Journal of Statistical Software 7(7), 1--19. Ntzoufras, I. (2009), Bayesian modeling using WinBUGS, Wiley, Hoboken, NJ. Raftery, A. E. (1995), 'Bayesian model selection in social research', Sociological Methodology 25, 111-163. Allan.Y all...@cmu.edu Sent by: r-help-boun...@r-project.org 10/22/2009 01:09 PM To r-help@r-project.org cc Subject [R] Bayesian regression stepwise function? Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Ravi Varadhan wrote: Frank, I have heard this (i.e. only head-to-head comparisons are valid) and various other folklores about AIC and BIC based model selection before, including one that these information criteria are only applicable for comparing two nested models. Where has it been demonstrated that AIC/BIC cannot be used to find the best subset, i.e. the subset that is closest to the true model (assuming that true model is contained in the set of models considered, and that maximum likelihood estimation is used for estimating parameters in the models)? I would greatly appreciate any reference that shows this. Best, Ravi. I don't have a specific reference, but the AIC/BIC approach, when entertaining more than a handful of models, has a very low probability of selecting the 'right' model. AIC/BIC are just restatements of P-values (especially AIC), so they have all the many problems that P-values have. It is for these reasons that Bayesians have much more success with model averaging than with model selection. And if all the models averaged are within one class of models, it is more logical to use penalization, entertaining only one comprehensive model. Penalized regression performs just as well as the much more complex model averaging approach if staying within one model class, and penalized models are easier to explain. Frank --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: Frank E Harrell Jr [mailto:f.harr...@vanderbilt.edu] Sent: Friday, October 23, 2009 4:04 PM To: Ravi Varadhan Cc: jlu...@ria.buffalo.edu; 'Allan.Y'; r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? Ravi Varadhan wrote: The stepAIC() function in MASS can be used, with k = log(n), to implement your suggestion of quasi-Bayesian stepwise selection using the BIC criterion. Ravi. Although many statisticians use BIC otherwise, it was only designed to compare two pre-specified models. Frank --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of jlu...@ria.buffalo.edu Sent: Friday, October 23, 2009 2:31 PM To: Allan.Y Cc: r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? The BIC (Raftery) can be used for quasi-Bayesian model selection, but it's not stepwise. Ntzoufras shows how to use WinBUGS to conduct Bayesian model selection, but again it's not stepwise Ntzoufras, I. (2002), 'Gibbs variable selection using BUGS', Journal of Statistical Software 7(7), 1--19. Ntzoufras, I. (2009), Bayesian modeling using WinBUGS, Wiley, Hoboken, NJ. Raftery, A. E. (1995), 'Bayesian model selection in social research', Sociological Methodology 25, 111-163. Allan.Y all...@cmu.edu Sent by: r-help-boun...@r-project.org 10/22/2009 01:09 PM To r-help@r-project.org cc Subject [R] Bayesian regression stepwise function? Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
I believe such a statement (AIC applicable to nested models only) appears in Brian Ripley. 2004. Selecting amongst large classes of models. Pages 155–170 in N. Adams, M. Crowder, D. J. Hand, and D. Stephens, editors. Methods and Models in Statistics: In Honour of Professor John Nelder, FRS. Imperial College Press, London. Ravi Varadhan wrote: Frank, I have heard this (i.e. only head-to-head comparisons are valid) and various other folklores about AIC and BIC based model selection before, including one that these information criteria are only applicable for comparing two nested models. Where has it been demonstrated that AIC/BIC cannot be used to find the best subset, i.e. the subset that is closest to the true model (assuming that true model is contained in the set of models considered, and that maximum likelihood estimation is used for estimating parameters in the models)? I would greatly appreciate any reference that shows this. Best, Ravi. --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: Frank E Harrell Jr [mailto:f.harr...@vanderbilt.edu] Sent: Friday, October 23, 2009 4:04 PM To: Ravi Varadhan Cc: jlu...@ria.buffalo.edu; 'Allan.Y'; r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? Ravi Varadhan wrote: The stepAIC() function in MASS can be used, with k = log(n), to implement your suggestion of quasi-Bayesian stepwise selection using the BIC criterion. Ravi. Although many statisticians use BIC otherwise, it was only designed to compare two pre-specified models. Frank --- Ravi Varadhan, Ph.D. Assistant Professor, The Center on Aging and Health Division of Geriatric Medicine and Gerontology Johns Hopkins University Ph: (410) 502-2619 Fax: (410) 614-9625 Email: rvarad...@jhmi.edu Webpage: http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h tml -Original Message- From: r-help-boun...@r-project.org [mailto:r-help-boun...@r-project.org] On Behalf Of jlu...@ria.buffalo.edu Sent: Friday, October 23, 2009 2:31 PM To: Allan.Y Cc: r-help@r-project.org; r-help-boun...@r-project.org Subject: Re: [R] Bayesian regression stepwise function? The BIC (Raftery) can be used for quasi-Bayesian model selection, but it's not stepwise. Ntzoufras shows how to use WinBUGS to conduct Bayesian model selection, but again it's not stepwise Ntzoufras, I. (2002), 'Gibbs variable selection using BUGS', Journal of Statistical Software 7(7), 1--19. Ntzoufras, I. (2009), Bayesian modeling using WinBUGS, Wiley, Hoboken, NJ. Raftery, A. E. (1995), 'Bayesian model selection in social research', Sociological Methodology 25, 111-163. Allan.Y all...@cmu.edu Sent by: r-help-boun...@r-project.org 10/22/2009 01:09 PM To r-help@r-project.org cc Subject [R] Bayesian regression stepwise function? Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p26033597.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Ravi Varadhan rvarad...@jhmi.edu wrote I have heard this (i.e. only head-to-head comparisons are valid) and various other folklores about AIC and BIC based model selection before, including one that these information criteria are only applicable for comparing two nested models. Where has it been demonstrated that AIC/BIC cannot be used to find the best subset, i.e. the subset that is closest to the true model (assuming that true model is contained in the set of models considered, and that maximum likelihood estimation is used for estimating parameters in the models)? I would greatly appreciate any reference that shows this. Burnham and Anderson state a different result - not exactly opposite, but different - in that they recommend use of AICC to choose among several competing models. But defining 'best' is tricky. In most situations where there are many variables, each of several models will be almost equally good, and which is 'best' would vary if you took a different sample from the same population. Peter Peter L. Flom, PhD Statistical Consultant Website: www DOT peterflomconsulting DOT com Writing; http://www.associatedcontent.com/user/582880/peter_flom.html Twitter: @peterflom __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
[R] Bayesian regression stepwise function?
Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Thanks -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p26013725.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Allan.Y wrote: Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Why? That seems so ... un-Bayesian ... -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p26015081.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Ben Bolker wrote: Allan.Y wrote: Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Why? That seems so ... un-Bayesian ... Exactly. I hope it doesn't exist. The beauty of Bayes is shrinkage, borrowing of information, and statement of results in an intuitive way. Frank -- Frank E Harrell Jr Professor and Chair School of Medicine Department of Biostatistics Vanderbilt University __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
Frank E Harrell Jr f.harr...@vanderbilt.edu wrote Ben Bolker wrote: Allan.Y wrote: Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Why? That seems so ... un-Bayesian ... Exactly. I hope it doesn't exist. The beauty of Bayes is shrinkage, borrowing of information, and statement of results in an intuitive way. Yeah. Asking for stepwise in Bayesian analysis is like asking for some nuclear waste on your ice cream sundae. Peter Peter L. Flom, PhD Statistical Consultant Website: www DOT peterflomconsulting DOT com Writing; http://www.associatedcontent.com/user/582880/peter_flom.html Twitter: @peterflom __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
On 23/10/2009, at 8:20 AM, Peter Flom wrote:
Frank E Harrell Jr f.harr...@vanderbilt.edu wrote
Ben Bolker wrote:
Allan.Y wrote:
Hi everyone,
I am wondering if there exists a stepwise regression function
for the
Bayesian regression model. I tried googling, but I couldn't find
anything. I know step function exists for regular stepwise
regression,
but nothing for Bayes.
Why? That seems so ... un-Bayesian ...
Exactly. I hope it doesn't exist. The beauty of Bayes is shrinkage,
borrowing of information, and statement of results in an intuitive
way.
Yeah.
Asking for stepwise in Bayesian analysis is like asking for some
nuclear waste on your ice cream sundae.
More like asking for petro-chemical waste in your nuclear waste!
cheers,
Rolf Turner
##
Attention:\ This e-mail message is privileged and confid...{{dropped:9}}
__
R-help@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Re: [R] Bayesian regression stepwise function?
On Thu, 22 Oct 2009, Ben Bolker wrote: Allan.Y wrote: Hi everyone, I am wondering if there exists a stepwise regression function for the Bayesian regression model. I tried googling, but I couldn't find anything. I know step function exists for regular stepwise regression, but nothing for Bayes. Why? That seems so ... un-Bayesian ... If 'fools rush in where angels fear to tread', then Bayesians 'jump' in where frequentists fear to 'step'... Seriously, there are Bayesian regression approaches that priorize the model size (sometimes only implicitly by assigning a prior for the inclusion of each candidate regressors). Then they 'jump' between models of different sizes. On CRAN, Package qtlbim (which is specialized to a particular genetics problem) implements one such, I think. Package bqtl does not implement the jumping approach, but does explore a model space with differing numbers of regressors for the same (qtl) problem. Perhaps the closest to a general purpose 'stepwise flavored' Bayesian regression is implemented in Package BMA, which IIRC borrows step() for some of its work. But CRAN now has more packages than my cortex has neurons, so there are probably more packages that do something like this. Try RSiteSearch(jump regression, restric='functions') and start reading. HTH, Chuck -- View this message in context: http://www.nabble.com/Bayesian-regression-stepwise-function--tp26013725p26015081.html Sent from the R help mailing list archive at Nabble.com. __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code. Charles C. Berry(858) 534-2098 Dept of Family/Preventive Medicine E mailto:cbe...@tajo.ucsd.edu UC San Diego http://famprevmed.ucsd.edu/faculty/cberry/ La Jolla, San Diego 92093-0901 __ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
