Jacob, To address your question, I would compare the uncertainty in models parameters obtained after running a non-parametric bootstrap (NP-BS) with that obtained after a parametric bootstrap (P-BS).
P-BS allows you to understand (under the null hypothesis) the expected posterior distribution of model parameters given the design of the new study and, therefore, you can assess if your new data would contain enough information to speak about certain parameters. For instance, if after running a P-BS, you get the same estimate of a certain parameter in all bootstraps replicates, it means the data available does not speak about that parameter (regardless of how strong the prior is) and, consequently, you can fix it before running the PRIOR and, of course, cannot derive any new conclusion on that parameter based on the new data. If the new data speaks substantially about model parameters (because an informative design and/or large sample size), the P-BS would provide you with the expected value (and uncertainty) of the model parameters (assuming the new data are arising from the same population used to obtain the priors) and will give you an idea about the contribution of the new study design in reducing the uncertainty in model parameter from the priors. However, if the new data are not arising from the same population used to obtain the priors, the uncertainty in model parameters might not be reduced and, therefore, you need to make a decision about the appropriateness of using PRIOR in that situation. You can make that assessment by comparing the point estimates (and uncertainty) of the updated parameters with the results of the P-BS. If the updated parameter estimates falls within the expected updated values of the model parameters, then you can assume the new data are arising from the same population used to obtain the priors, provided you have enough power. However, this exercise tends to be conservative and additional simulation/estimation exercise might be needed. Hope it helps. Enjoy the Holiday Season! Juan J. Perez-Ruixo. -----Original Message----- From: [email protected] [mailto:[email protected]] On Behalf Of Leonid Gibiansky Sent: jueves, 22 de diciembre de 2011 7:08 To: Brogren Jacob Cc: [email protected] Subject: Re: [NMusers] Bootstrap in combination with PRIOR? Jacob, Bootstrap procedure vary only the new data. The information in priors is still the same. Therefore, results will be as dependent on the priors as the parameter estimates of the final model. When the priors are strong (informative) it would require a lot of new data to move the parameters of the final model and the parameter estimates of each of the bootstrap samples. In the extreme case of the very strong priors (e.g., when the parameters are simply fixed) parameters of all bootstrap samples will also be fixed at the same values. Since the results are dependent of the strength (informative content) of priors, I doubt that any conclusions can be made about each particular case based on the other-people examples. This is likely to be decided on the case-by-case basis. Nonmem standard errors are usually in a good agreement with the bootstrap results. Therefore, influence of the priors on the parameter estimates (precision) can be evaluated using the standard errors (much quicker than running bootstrap multiple times with different prior values). Extreme test would be to run the model without priors, and see how this would influence the confidence intervals. Regards, Leonid -------------------------------------- Leonid Gibiansky, Ph.D. President, QuantPharm LLC web: www.quantpharm.com e-mail: LGibiansky at quantpharm.com tel: (301) 767 5566 On 12/22/2011 7:58 AM, Brogren Jacob wrote: > Dear nmusers, > > Merry Holiday! (you may change the variable 'Holiday' to an arbitrary value) > > First, accept my humble excuse - I can't share any code for this example. > > I'm assessing a model where $PRIOR is used to stabilize the fit of an > 'old' model to sparse data using NONMEM. It seems to be the only way > forward. Using the final model the Applicant has then performed a > non-parametric bootstrap (N=1000) to investigate i.a. robustness of the > model parameter estimates. > > From a non-scientific literature search (Google Scholar "NONMEM + PRIOR > + Bootstrap") some examples show up where Bootstrap and PRIOR has been > combined. > > I'm curious about if any nm-user has investigated how much and in what > way the prior distribution of parameters affect the bootstrap estimates > (eg. 2.5^th , 50^th and 97.5^th percentiles)? I would guess that the > stronger the prior the more it would affect the outcome of a bootstrap. > Or is that a misconception? > > Cheers > > Jacob > > LV_Mac3 > > Medical Products Agency > > Jacob Brogren > Assessor > Efficacy and Safety 2 > > > > P.O. Box 26, SE-751 03 Uppsala, Sweden > Visiting address: Dag Hammarskjölds väg 42 > Phone: + 46 (0) 18 17 47 64, switchboard: + 46 (0) 18 17 46 00 > Mobile: + 46 (0) xxx xx xx, Fax: + 46 (0) 18 54 85 66 > www.lakemedelsverket.se <http://www.lakemedelsverket.se> >
