Hi Ruben,

A quick response to your comment about simulations and residual error:
    “Should we again sample residual error when we simulate from EBE estimates?
    Or should we estimate individual parameter uncertainty from the OFIM and 
use only that?”

This depends on what you question you want your simulation to answer.
If your question is “what is my best guess about the underlying physical 
processes of a model” then you should simulate without residual error.
On the other hand, if your question is “what kind of observations am I likely 
to see if I performed an experiment” then you should include residual error.

I would think any time you want to simulate randomly selected individuals you 
should use individual parameter uncertainty.

Warm regards,

Douglas Eleveld





From: owner-nmus...@globomaxnm.com [mailto:owner-nmus...@globomaxnm.com] On 
Behalf Of Faelens, Ruben (Belgium)
Sent: maandag 9 april 2018 23:52
To: Tingjie Guo; nmusers@globomaxnm.com
Subject: RE: [NMusers] ETAs & SIGMA in external validation

Hi Tingjie,

A lot of great tips and explanations already. Just wanted to add my two cents.

POSTHOC will estimate the most likely ETA for each individual, taking into 
account the known population parameters THETA and OMEGA. “Most likely” means:

  1.  An individual parameter as close as possible to the typical value (i.e. 
ETA=0).
The likelihood of an ETA is evaluated through the probability density function 
of a normal distribution with mean 0 and variance OMEGA.
  2.  A model prediction as close as possible to the observed values.
The likelihood of a model prediction is evaluated through the probability 
density function of the residual error model.
We find the most likely ETA by using maximum likelihood estimation (I do not 
know the exact algorithm, but I use Nelder-Mead in my own software and that 
produces the same results as nonmem).

You have two questions:

  1.  Can I constrict the ETA search space so only realistic ETA’s are found?
You can, but that would change your original model, and would require 
re-estimating THETA and OMEGA. For some parameters (e.g. disease progression, 
or LOG(BASELINE) ), an absolute inter-individual variability on a parameter may 
make sense.
You may want to re-evaluate (as suggested previously) whether this is valid for 
all parameters. In other words: whether the parameter IIV is truly symmetric 
normal distributed.
In any case, posthoc estimations are linked to the original model. If 
close-to-zero parameter values are unlikely to appear in the training dataset, 
then OMEGA should be small, and therefore negative values of a parameter will 
probably not be estimated anyway (part 1 of our maximum likelihood estimation 
explained above). And if the model does not make any sense with negative 
parameter values, the model predictions will be very far off from the observed 
values as well (part 2 of our maximum likelihood estimation).
I suggest you re-evaluate the ETA distributions of your original model, and 
consider using a lognormal IIV instead.
You could also explore graphically the input data for subjects with negative 
ETA values. Possibly the observed input data can only be explained through 
negative parameter values?

@Jakob: Could you explain how “The solution you initially implemented will bias 
the parameter distribution severely, since only values greater than or equal to 
the typical parameter value is allowed.” ? In case of an IIV of e.g. 20% CV, 
(1+ETA) would require 5 standard deviations on ETA before it becomes negative.


  1.  Which error model should I use? Should I only use the assay error?
Residual error comes from many sources. Assay error is only one of these. 
Others include model misspecification, dosing errors, true dose deviations 
(e.g. use of generics, or inaccuracies in preparing an infusion), bad recording 
of sample times, etc. Unless there is a good reason to assume your new data was 
not subjected to the same errors as the training dataset, you should keep the 
same residual error model.

I myself am still struggling with this question:
“Should we again sample residual error when we simulate from EBE estimates? Or 
should we estimate individual parameter uncertainty from the OFIM and use only 
that?”

Best regards,
Ruben Faelens
Scientist at SGS Exprimo
PhD Student at KULeuven

From: owner-nmus...@globomaxnm.com<mailto:owner-nmus...@globomaxnm.com> 
[mailto:owner-nmus...@globomaxnm.com] On Behalf Of Tingjie Guo
Sent: vrijdag 6 april 2018 18:32
To: nmusers@globomaxnm.com<mailto:nmusers@globomaxnm.com>
Subject: [NMusers] ETAs & SIGMA in external validation

Dear NMusers,

I have two questions regarding the statistical model when performing external 
validation. I have a dataset and would like to validate a published model with 
POSTHOC method i.e. $EST METHOD=0 POSTHOC MAXEVAL=0.

1. The model added etas in proportional way, i.e. Para = THETA * (1+ETA) and 
this made the posthoc estimation fail due to the negative individual parameter 
estimate in some subjects. I constrained it to be positive by adding ABS 
function i.e. Para = THETA * ABS(1+ETA), and the estimation can be successfully 
running. I was wondering if there is better workaround?

2. OMEGA value influences individual ETAs in POSTHOC estimation. Should we 
assign $SIGMA with model value or lab (where external data was determined) 
assay error value? If we use model value, it's understandable that $SIGMA 
contains unexplained variability and thus it is a part of the model. However, I 
may also understand it as that model value contains the unexplained variability 
for original data (in which the model was created) but not for external data. 
I'm a little confused about it. Can someone help me out?

I would appreciate any response! Many thanks in advance!

Your sincerely,
Tingjie Guo

Information in this email and any attachments is confidential and intended 
solely for the use of the individual(s) to whom it is addressed or otherwise 
directed. Please note that any views or opinions presented in this email are 
solely those of the author and do not necessarily represent those of the 
Company. Finally, the recipient should check this email and any attachments for 
the presence of viruses. The Company accepts no liability for any damage caused 
by any virus transmitted by this email. All SGS services are rendered in 
accordance with the applicable SGS conditions of service available on request 
and accessible at http://www.sgs.com/en/Terms-and-Conditions.aspx
________________________________
De inhoud van dit bericht is vertrouwelijk en alleen bestemd voor de 
geadresseerde(n). Anderen dan de geadresseerde(n) mogen geen gebruik maken van 
dit bericht, het niet openbaar maken of op enige wijze verspreiden of 
vermenigvuldigen. Het UMCG kan niet aansprakelijk gesteld worden voor een 
incomplete aankomst of vertraging van dit verzonden bericht.

The contents of this message are confidential and only intended for the eyes of 
the addressee(s). Others than the addressee(s) are not allowed to use this 
message, to make it public or to distribute or multiply this message in any 
way. The UMCG cannot be held responsible for incomplete reception or delay of 
this transferred message.

Reply via email to