On Tuesday, October 26, 2010 09:46:46 am Bernhard Rupp (Hofkristallrat a.D.)
wrote:
> Hi Folks,
>
> Please allow me a few biased reflections/opinions on the numeRology of the
> R-value (not R-factor, because it is neither a factor itself nor does it
> factor in anything but ill-posed reviewer's critique. Historically the term
> originated from small molecule crystallography, but it is only a
> 'Residual-value')
>
> a) The R-value itself - based on the linear residuals and of apparent
> intuitive meaning - is statistically peculiar to say the least. I could not
> find it in any common statistics text. So doing proper statistics with R
> becomes difficult.
As WC Hamilton pointed out originally, two [properly weighted] R factors can
be compared by taking their ratio. Significance levels can then be evaluated
using the standard F distribution. A concise summary is given in chapter 9
of Prince's book, which I highly recommend to all crystallographers.
W C Hamilton "Significance tests on the crystallographic R factor"
Acta Cryst. (1965). 18, 502-510
Edward Prince "Mathematical Techniques in Crystallography and Materials
Science". Springer-Verlag, 1982.
It is true that we normally indulge in the sloppy habit of paying attention
only to the unweighted R factor even though refinement programs report
both the weighted and unweighted versions. (shelx users excepted :-)
But the weighted form is there also if you want to do statistical tests.
You are of course correct that this remains a global test, and as such
is of limited use in evaluating local properties of the model.
cheers,
Ethan
> b) rules of thumb (as much as they conveniently obviate the need for
> detailed explanations, satisfy student's desire for quick answers, and
> allow superficial review of manuscripts) become less valuable if they have a
> case-dependent large variance, topped with an unknown parent distribution.
> Combined with an odd statistic, that has great potential for misguidance and
> unnecessarily lost sleep.
>
> c) Ian has (once again) explained that for example the Rf-R depends on the
> exact knowledge of the restraints and their individual weighting, which we
> generally do not have. Caution is advised.
>
> d) The answer which model is better - which is actually what you want to
> know - becomes a question of model selection or hypothesis testing, which,
> given the obscurity of R cannot be derived with some nice plug-in method. As
> Ian said the models to be compared must also be based on the same and
> identical data.
>
> e) One measure available that is statistically at least defensible is the
> log-likelihood. So what you can do is form a log-likelihood ratio (or Bayes
> factor (there is the darn factor again, itÂ’s a ratio)) and see where this
> falls - and the answers are pretty soft and, probably because of that,
> correspondingly realistic. This also makes - based on statistics alone -
> deciding between different overall parameterizations difficult.
>
> http://en.wikipedia.org/wiki/Bayes_factor
>
> f) so having said that, what really remains is that the model that fits the
> primary evidence (minimally biased electron density) best and is at the same
> time physically meaningful, is the best model, i. e., all plausibly
> accountable electron density (and not more) is modeled. You can convince
> yourself of this by taking the most interesting part of the model out (say a
> ligand or a binding pocket) and look at the R-values or do a model selection
> test - the result will be indecisive. Poof goes the global rule of thumb.
>
> g) in other words: global measures in general are entirely inadequate to
> judge local model quality (noted many times over already by Jones, Kleywegt,
> others, in the dark ages of crystallography when poorly restrained
> crystallographers used to passionately whack each other over the head with
> unfree R-values).
>
> Best, BR
> -----------------------------------------------------------------
> Bernhard Rupp, Hofkristallrat a.D.
> 001 (925) 209-7429
> +43 (676) 571-0536
> [email protected]
> [email protected]
> http://www.ruppweb.org/
> ------------------------------------------------------------------
> Und wieder ein chillout-mix aus der Hofkristall-lounge
> ------------------------------------------------------------------
>
--
Ethan A Merritt
Biomolecular Structure Center, K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742
