RE: [NMusers] Models that abort before convergence

[Mark Sale - Next Level Solutions]

Leonid et al,    I'm a little confused by this discussion.  To make an analogy, assume that drug company A has a wonderful theory that drug B will treat a disease.  Theory makes sense by your favorite epistemology criteria etc.  But of course, being good scientists, we know that theories must be verified, so we do an experiment, and the data suggest that the theory is wrong.  Most of us would criticize as unscientific someone who who discarded the data (didn't point out flaws in the data, didn't provide opposing data, simply discounted it) in favor of continuing to believe the theory.    Why do we not apply the same standards here?  Theory says that models that do not converge (or fail covariance) are "bad".  Data (that so far as I know no one has found to be flawed, nor provided opposing data) suggests that, by at least one criteria (same parameter estimates, same SD of parameter estimates) there are no important differences.  I don't disagree that failing a covariance step, or failing to converge provide information about a model.  But it doesn't seem to be informative about what we probably really care about -does the line go through the points, how confident are we WRT the precision of the parameters and is the model predictive.   I'm not sure if the small number of published examples (of bootstrap with ~500 samples) are a small number of anecdotes or a small number of trials with N ~ 500, but I've run 5 or so myself and found the same to be consistently the case. That is, a successful covariance step is not informative WRT the parameter values or their precision.  I suspect others have similar experience.  If there are other "studies"/anecdotes with different conclusions, someone should publish them.  Otherwise, it seems like we are obligated to abandon this theory in favor of the data.   Mark Sale MD Next Level Solutions, LLC www.NextLevelSolns.com 919-846-9185 -------- Original Message -------- Subject: RE: [NMusers] Models that abort before convergence From: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> Date: Tue, November 18, 2008 11:13 pm To: [EMAIL PROTECTED], nmusers@globomaxnm.com, [EMAIL PROTECTED] Dennis, I do not support extreme views (from places where people walk upside down :) ) that Nonmem error messages should be ignored: they serve the useful purpose to alert when Nonmem is having some difficulties, and should always be part of the picture. If the data looks good, model is simple, then we need to look for the reason for the poor convergence. Sometimes it helps to use SIGDIG= 5 or 6 to get 3 significant digits precision. But if you are working on the limit of the algorithms (as implemented) abilities: nonlinear model + stiff differential equations + large range of doses and concentrations, etc., then you face the situation when you cannot force convergence even if you try hard. On my recent project, none of the intermediate model converged even though bootstrap provided pretty narrow CI (so it does not look like over-parametrized model), all diagnostic plots were good, and the visual predictive check was reasonable. Then you just blame the algorithm and move on. You loose the ability to justify your covariate selection based on the objective function drop (which is not a good idea any way), and may need to provide a little bit more detailed investigation to convince reviewers (regulatory and/or journal) that the model is adequate for the intended purpose. Thanks Leonid Original Message: ----------------- From: Dennis Fisher [EMAIL PROTECTED] Date: Tue, 18 Nov 2008 11:21:23 -0800 To: nmusers@globomaxnm.com, [EMAIL PROTECTED] Subject: [NMusers] Models that abort before convergence Colleagues, I am curious as to your thoughts about a particular NONMEM issue. I often find myself in a situation where a complex model does not converge to 3 digits ("no of digits: unreportable") yet the objective function is markedly better than a previous model and graphics suggest that the model is quite good (and better than the previous one). Nick Holford has advocated (and I agree) that NONMEM's SE's have minimal utility and the inability to calculate them is not important. However, I have not seen similar discussion about whether one can / should accept a model that did not converge. The particular situation that I dealing with at the moment is that a dataset that I am analyzing yielded a series of results that did not converge as I added parameters (despite an improving fit and a marked decrease in the objective function), then yet a more complicated model yielded 3.0 significant digits. In this case, there is no problem (I can use this final model for bootstrap, VPC, etc.) but what if none of these models had converged. Dennis Dennis Fisher MD P < (The "P Less Than" Company) Phone: 1-866-PLessThan (1-866-753-7784) Fax: 1-415-564-2220 www.PLessThan.com -------------------------------------------------------------------- mail2web.com - Microsoft® Exchange solutions from a leading provider - http://link.mail2web.com/Business/Exchange