Hello Jakob, et al.
I would agree that individuals who do not contribute data to the
estimation of a particular element of OMEGA should be excluded from
the ETA-shrinkage calculation or ETA-based diagnostics. I think that
using individual ETA=0 as the filtering criterion may be a reasonable
thing to do when OMEGA is DIAGONAL (e.g. all off-diagonal elements are
zero), but this practice could be misleading when covariance in the
inter-individual random effects exists (e.g. OMEGA BLOCK(n)).
For example, consider a population PK model simultaneously
incorporating parent and metabolite data. Also imagine that the OMEGA
matrix is constructed to allow covariance between ETA[parent CL] and
ETA[metabolite CL]. If the correlation between these ETAs is non-zero,
it is possible that individuals who are entirely missing metabolite
data will still have a non-zero ETA[metabolite CL] estimate. This is
because the expected value for that ETA should be driven by the
covariance structure in OMEGA. Although this ETA estimate is non-zero,
it is shrunken toward the population expected value, and may
contribute to a biased shrinkage calculation and/or diagnostics.
To avoid both this situation and the issue that Douglas raised, it is
preferable to filter ETAs based on design factors rather than
automatically based on individual ETA=0.
Having said all this, I'm not sure how important this particular
source of bias in the ETA-shrinkage calculation is anyway. There are
other potential biases in this calculation, including:
1. Bias in the population estimates of OMEGA variance elements- It's
not uncommon for these terms to be over-estimated, which may lead to
an artificial apparent shrinkage (the calculation for ETA shrinkage
uses estimated variance in the denominator).
2. Bias in the observed sample SD of individual ETAs due to
insufficient sample size- Biased shrinkage estimates may result from
biased sample SD (used in the numerator of the shrinkage calculation),
particularly in small data sets.
I think the take-home message is that ETA-based diagnostics (and
diagnostics of the diagnostics) can be useful, but should be
considered in the context of the design and potential biases.
Best regards,
Marc
Marc R. Gastonguay, Ph.D. < [email protected] >
President & CEO, Metrum Research Group LLC < metrumrg.com >
Scientific Director, Metrum Institute < metruminstitute.org >
2 Tunxis Rd, Suite 112, Tariffville, CT 06081 Direct:
+1.860.670.0744 Main: +1.860.735.7043 Fax: +1.860.760.6014
On Aug 21, 2009, at 9:12 AM, Ribbing, Jakob wrote:
Hi Douglas,
This has been a concern for me as well, although I do not know if
this ever happens(?). For the automatic (generic scripts) exclusion
of etas that I use for eta-diagnostics, I tend to exclude a group
(e.g. each dose or dose-study combination) if all subjects have
eta=0 in that group. This would for example exclude IOV-eta3 from a
study that only hade two occasions, or the placebo group(s) for etas
on drug effect. I feel safe with that exclusion for my diagnostics.
If I had to make the choice between excluding all etas that are
exactly equal to zero or none at all, I would more trust diagnostics
after exclusion.
Jakob
From: Eleveld, DJ [mailto:[email protected]]
Sent: 21 August 2009 13:57
To: Ribbing, Jakob; Pyry Välitalo; [email protected]
Subject: RE: [NMusers] Calculating shrinkage when some etas are zero
Hi Pyry and Jacob,
If you exclude zero etas then what happens to infomative individuals
who just happen to have the population typical values?
This approch would exclude these individuals when trying to indicate
how informative an estimation is about a parameter.
I know this is unlikely, but it is possible.
The etas just tell what value is estimated, its not the whole story
about how infomative an estimation is. I dont think you can do
this without considering how 'certian' you are of each of those eta
values.
Douglas Eleveld
Van: [email protected] namens Ribbing, Jakob
Verzonden: vr 21-8-2009 12:26
Aan: Pyry Välitalo; [email protected]
Onderwerp: RE: [NMusers] Calculating shrinkage when some etas are zero
Hi Pyry,
Yes, when calculating shrinkage or looking at eta-diagnostic plots
it is often better to exclude etas from subjects that has no
information on that parameter at all. For a PK model we would not
include subjects that were only administered placebo (if PK is
exogenous compound). In the same manner placebo subjects are not
informative on the drug-effects parameters of a (PK-)PD model. These
subjects have informative etas for the placebo-part of the PD model,
but not on the drug-effects (etas on Emax, ED50, etc.). For any eta-
diagnostics you can removed these etas based on design (placebo
subject, IV dosing, et c) or the empirical-Bayes estimate of eta
being zero.
Cheers
Jakob
From: [email protected] [mailto:[email protected]
] On Behalf Of Pyry Välitalo
Sent: 21 August 2009 10:45
To: [email protected]
Subject: [NMusers] Calculating shrinkage when some etas are zero
Hi all,
I saw this snippet of information on PsN-general mailing list.
Kajsa Harling wrote in PsN-general:
"I talked to the experts here about shrinkage. Apparently, sometimes
an
individual's eta may be exactly 0 (no effect, placebo, you probably
understand this better than I do). These zeros should not be
included in
the shrinkage calculation, but now they are (erroneously) in PsN."
This led me to wonder about the calculation of shrinkage. I decided
to post here on nmusers, because my question mainly relates to
NONMEM. I could not find previous discussions about this topic
exactly.
As I understand, if a parameter with BSV is not used by some
individuals, the etas for these individuals will be set to zero. An
example would be a dataset with IV and oral dosing data. If oral
absorption rate constant KA with BSV is estimated for this data,
then all eta(KA) values for IV dosing group will be zero.
The shrinkage of etas is calculated as
1-sd(etas)/omega
If the etas that equal exactly zero would have to be removed from
this equation then it would mean that NONMEM estimates the omega
based on only those individuals who need it for the parameter in
question, e.g. the omega(KA) would be estimated only based on the
oral dosing group. Is this a correct interpretation for the
rationale to leave out zero etas?
I guess the inclusion of zero etas into shrinkage calculations
significantly increases the estimate of shrinkage because the zero
etas always reduce the sd(etas). As a practical example, suppose a
dataset of 20 patients with oral and 20 patients with IV
administration. Suppose NONMEM estimates an omega of 0.4 for BSV of
KA. Suppose the sd(etas) for oral group is 0.3 and thus sd(etas) for
all patients is 0.3/sqrt(2) since the etas in IV group for KA are
zero.
Thus, as far as I know, PsN would currently calculate a shrinkage of
1-(0.3/sqrt(2))/0.4=0.47.
Would it be more appropriate to manually calculate a shrinkage of
1-0.3/0.4=0.25 instead?
All comments much appreciated.
Kind regards,
Pyry
Kajsa Harling wrote:
Dear Ethan,
I have also been away for a while, thank you for your patience.
I talked to the experts here about shrinkage. Apparently, sometimes an
individual's eta may be exactly 0 (no effect, placebo, you probably
understand this better than I do). These zeros should not be
included in
the shrinkage calculation, but now they are (erroneously) in PsN.
Does this explain the discrepancy?
Then, the heading shrinkage_wres is incorrect, it should say
shrinkage_iwres (or eps) they say.
Comments are fine as long as they do not have commas in them. But this
is fixed in the latest release.
Best regards,
Kajsa
