Hi Ken Many thanks for the insights on the standard errors & their interpretations. Yes, a careful analysis is required to interpret SE’s of transformed space in untransformed space.
Best regards Santosh On Mon, Jul 29, 2024 at 4:43 PM <kgkowalsk...@gmail.com> wrote: > Hi Santosh, > > > > It’s important to note the distinction between transformations of the > parameters and transformations of the data such as the > log-transform-both-sides approach for a PK model to assume the residual > errors are log-normally distributed. Here we are specifically focusing on > transformations of the parameters not the data. Note that the likelihood > is invariant to transformations of the parameters, so you will get the same > fit and OFV whether you estimate log(CL) or CL as your theta. However, > Wald-based standard errors are very much dependent on the parameter > transformation. > > > > For example, suppose we estimate the typical value of CL as THETA(1). > Assuming the maximum likelihood estimate of THETA(1) is asymptotically > normal then we could construct a confidence interval to reflect the > uncertainty in that parameter estimate as > > > > theta1 +/- Zalpha(SE_theta1) > > > > where SE_theta1 is the Wald-based SE for theta1 and Zalpha is the > two-sided critical value of the standard normal distribution to obtain a > 100x(1 – alpha)% confidence interval. Note that this confidence interval > is symmetric about the estimate theta1. > > > > Now consider a log-transformation of CL such that THETA(1) corresponds to > log(CL). The Wald-based confidence interval for log(CL) would now be > > > > theta1 +/- Zalpha(SE_theta1) > > > > which is symmetric in the log(CL) scale. However, the corresponding > confidence interval for CL requires exponentiating the endpoints of the > log(CL) confidence interval to obtain the confidence interval in the > original CL scale. That is, > > > > exp(theta1 +/- Zalpha(SE_theta1) ) > > > > which will be asymmetric about the estimate of the typical value of CL, > exp(theta1). > > > > When the parameter estimate space is highly asymmetric, transformations > can help with this asymmetry so that the transformed estimates are more > likely to be symmetric and normally distributed. So, to answer your > question, the precision of the estimates may still be valid, but we need to > recognize that the uncertainty in the estimates may be asymmetric in the > untransformed (original) space. > > > > Best, > > > > Ken > > > > *From:* owner-nmus...@globomaxnm.com <owner-nmus...@globomaxnm.com> *On > Behalf Of *Santosh > *Sent:* Monday, July 29, 2024 1:11 PM > *To:* nmusers@globomaxnm.com > *Subject:* Re: [NMusers] Obtaining RSE% > > > > Dear Prof Holford, Ken, Alan, Jeroen & others, > > > > Thanks for the engaging discussions. > > > > In context of monitoring at the iteration level, I vaguely recall that in > NMUSERS or in one of ACOP conferences , there was a presentation & > demonstration with R scripts on looking at the convergence and other > parameters in real time. > > > > The interpretations of SEs is interesting based on linear or non-linear > models, and also based on size of variance of parameters. > > > > On a different note, I am also interested in hearing from you about SEs > when estimated based on transformed distribution space and their values & > interpretations in back-transformed space. Would the notion of precision > still be valid when viewing both transformed and untransformed space? > This is in context of dealing with untransformed space of non-normal or > non-lognormal distributions. > > > > Best regards > > Santosh > > > > > > On Mon, Jul 29, 2024 at 8:52 AM Nick Holford <n.holf...@auckland.ac.nz> > wrote: > > Hi Jeroen, > > A small correction. Please re-read my email to nmusers on 12 Feb 2015 > which I quote here. Sorry I cannot show the original but the 1999 URL is > not available to me anymore. > > ================= start quote =================== > Nick Holford Thu, 12 Feb 2015 11:54:59 -0800 > Hi, > The original quote about electrons comes from a remark I made in 1999 on > nmusers. > http://www.cognigencorp.com/nonmem/nm/99nov121999.html > Lewis Sheiner agreed in the same thread. Thanks to the wonders of living > on a sphere Lewis appears to agree with me the day before I made the > comment :-) > ================= end quote =================== > > I had been meaning to add to Ken's great email which confirms my original > assertion about electrons. > > If Santosh really wanted to calculate SE's after every "iteration" (which > I think was Ken's interpretation of every "estimation") then this can be > done by running a non-parametric bootstrap with the parameter estimates > produced after every iteration. > > I wonder if Santosh would like to spend a few hours doing that and adding > to the nmusers collection about standard errors by reporting the results to > us? > > > Best wishes, > Nick > > > -- > Nick Holford, Professor Emeritus Clinical Pharmacology, MBChB, FRACP > mobile: NZ+64(21) 46 23 53 ; FR+33(6) 62 32 46 72 > email: n.holf...@auckland.ac.nz > web: http://holford.fmhs.auckland.ac.nz/ > > -----Original Message----- > From: owner-nmus...@globomaxnm.com <owner-nmus...@globomaxnm.com> On > Behalf Of Jeroen Elassaiss-Schaap (PD-value B.V.) > Sent: Monday, July 29, 2024 3:37 PM > To: kgkowalsk...@gmail.com; 'Santosh' <santosh2...@gmail.com>; > nmusers@globomaxnm.com > Cc: 'Alan Maloney' <al_in_swe...@hotmail.com>; Pyry Välitalo < > pyry.valit...@gmail.com> > Subject: Re: [NMusers] Obtaining RSE% > > [Some people who received this message don't often get email from > jer...@pd-value.com. Learn why this is important at > https://aka.ms/LearnAboutSenderIdentification ] > > Dear NMusers, > > This is a great reminder for us to consider the reliability of standard > errors in our models, thanks Ken & Alan. The more non-linear the models > become, the less reliable and the more important other perspectives on > parameter values such as sensitivity analysis and prior knowledge. > > The nmusers archive has many great threads on the topic that are available > to review such as > https://www.mail-archive.com/nmusers@globomaxnm.com/msg05423.html and > related https://www.mail-archive.com/nmusers@globomaxnm.com/msg05419.html > . In summary, log-transformation only can get you so far but can perhaps be > seen as a sort of minimal effort. > > To add to the Lewis's quote about SEs - "they are not worth the electrons > used to compute them" (see the links), Pyry had some very interesting > observations he shared with me about the SE of the CV of a log-normal > omega: it inflates with higher values of omega compared to the SE of omega > itself. > > Best regards, > > Jeroen > > http://pd-value.com > jer...@pd-value.com > @PD_value > +31 6 23118438 > -- More value out of your data! > > On 29-07-2024 14:41, kgkowalsk...@gmail.com wrote: > > > > Dear NMusers, > > > > It was recently pointed out to me by a statistical colleague that my > > recent NMusers post about the inverse Hessian (R matrix) evaluated at > > the maximum likelihood estimates is a consistent estimator of the > > covariance matrix (i.e., converges to the true value with large N) is > > only true for linear models. For nonlinear models, the standard > > errors produced by NONMEM and other nonlinear estimation software are > > not only asymptotic but also approximate. Moreover, how well that > > approximation works will also depend on the parameterization. This I > > believe is one of the motivations behind “mu referencing” in NONMEM > > and the use of log transformations of the parameters to help improve > > Wald-based approximations. I thank Alan Maloney for pointing this out > > to me. > > > > Kind regards, > > > > Ken > > > > *From:*kgkowalsk...@gmail.com <kgkowalsk...@gmail.com> > > *Sent:* Saturday, July 27, 2024 12:36 PM > > *To:* 'Santosh' <santosh2...@gmail.com>; nmusers@globomaxnm.com > > *Subject:* RE: [NMusers] Obtaining RSE% > > > > Dear Santosh, > > > > There is a good reason for this. Wald (1943) has shown that the > > inverse of the Hessian (R matrix) evaluated at the maximum likelihood > > estimates is a consistent estimator of the covariance matrix. It is > > based on Wald’s approximation that the likelihood surface locally near > > the maximum likelihood estimates can be approximated by a quadratic > > function in the parameters. This theory does not hold for any set of > > parameter estimates along the algorithm’s search path prior to > > convergence to the maximum likelihood estimates. Moreover, inverting > > the Hessian evaluated at an interim step prior to convergence would > > likely be a poor approximation especially early in the search path > > where the gradients are large (i.e., large changes in OFV for a given > > change in the parameters would probably have substantial curvature and > > not be well approximated by a quadratic model in the parameters). > > > > Thus, the COV step in NONMEM is only applied once convergence is > > obtained during the EST step. > > > > Wald, A. “Tests of statistical hypotheses concerning several > > parameters when the number of observations is large.” /Trans. Amer. > > Math. Soc./ 1943;54:426. > > > > Best, > > > > Ken > > > > Kenneth G. Kowalski > > > > President > > > > Kowalski PMetrics Consulting, LLC > > > > Email: kgkowalsk...@gmail.com <mailto:kgkowalsk...@gmail.com> > > > > Cell: 248-207-5082 > > > > *From:*owner-nmus...@globomaxnm.com > > <mailto:owner-nmus...@globomaxnm.com><owner-nmus...@globomaxnm.com > > <mailto:owner-nmus...@globomaxnm.com>> *On Behalf Of *Santosh > > *Sent:* Friday, July 26, 2024 3:38 AM > > *To:* nmusers@globomaxnm.com <mailto:nmusers@globomaxnm.com> > > *Subject:* [NMusers] Obtaining RSE% > > > > Dear esteemed experts! > > > > When using one or more estimation methods & covariance step in a > > NONMEM control stream, the resulting ext file contains final estimate > > (for all estimation steps) & standard error (only for the last > > estimation step). > > > > Is there a way that standard error is generated for every estimation > step? > > > > TIA > > > > Santosh > > > >
