Um...
* Given that the weighted Rfactor is weighted by the measurement errors
(1/sig^2)
* and given that the errors in our measurements apparently have no
bearing whatsoever on the errors in our models (for macromolecular
crystals, certainly - the "R-vfactor gap")
is the weighted Rfactor even vaguely relevant for anything at all?
phx.
On 26/10/2010 20:44, Ian Tickle wrote:
Indeed, see: http://scripts.iucr.org/cgi-bin/paper?a07175 .
The Rfree/Rwork ratio that I referred to does strictly use the
weighted ('Hamilton') R-factors, but because only the unweighted
values are given in the PDB we were forced to approximate (against our
better judgment!).
The problem of course is that all refinement software AFAIK writes the
unweighted Rwork& Rfree to the PDB header; there are no slots for the
weighted values, which does indeed make doing serious statistics on
the PDB entries difficult if not impossible!
The unweighted crystallographic R-factor was only ever intended as a
"rule of thumb", i.e. to give a rough idea of the relative quality of
related structures; I hardly think the crystallographers of yesteryear
ever imagined that we would be taking it so seriously now!
In particular IMO it should never be used for something as critical as
validation (either global or local), or for guiding refinement
strategy: use the likelihood instead.
Cheers
-- Ian
PS I've always known it as an 'R-factor', e.g. see paper referenced
above, but then during my crystallographic training I used extensively
software developed by both authors of the paper (i.e. Geoff Ford& the
late John Rollett) in Oxford (which eventually became the 'Crystals'
small-molecule package). Maybe it's a transatlantic thing ...
Cheers
-- Ian
On Tue, Oct 26, 2010 at 7:28 PM, Ethan Merritt<[email protected]> wrote:
On Tuesday, October 26, 2010 09:46:46 am Bernhard Rupp (Hofkristallrat a.D.)
wrote:
Hi Folks,
Please allow me a few biased reflections/opinions on the numeRology of the
R-value (not R-factor, because it is neither a factor itself nor does it
factor in anything but ill-posed reviewer's critique. Historically the term
originated from small molecule crystallography, but it is only a
'Residual-value')
a) The R-value itself - based on the linear residuals and of apparent
intuitive meaning - is statistically peculiar to say the least. I could not
find it in any common statistics text. So doing proper statistics with R
becomes difficult.
As WC Hamilton pointed out originally, two [properly weighted] R factors can
be compared by taking their ratio. Significance levels can then be evaluated
using the standard F distribution. A concise summary is given in chapter 9
of Prince's book, which I highly recommend to all crystallographers.
W C Hamilton "Significance tests on the crystallographic R factor"
Acta Cryst. (1965). 18, 502-510
Edward Prince "Mathematical Techniques in Crystallography and Materials
Science". Springer-Verlag, 1982.
It is true that we normally indulge in the sloppy habit of paying attention
only to the unweighted R factor even though refinement programs report
both the weighted and unweighted versions. (shelx users excepted :-)
But the weighted form is there also if you want to do statistical tests.
You are of course correct that this remains a global test, and as such
is of limited use in evaluating local properties of the model.
cheers,
Ethan
b) rules of thumb (as much as they conveniently obviate the need for
detailed explanations, satisfy student's desire for quick answers, and
allow superficial review of manuscripts) become less valuable if they have a
case-dependent large variance, topped with an unknown parent distribution.
Combined with an odd statistic, that has great potential for misguidance and
unnecessarily lost sleep.
c) Ian has (once again) explained that for example the Rf-R depends on the
exact knowledge of the restraints and their individual weighting, which we
generally do not have. Caution is advised.
d) The answer which model is better - which is actually what you want to
know - becomes a question of model selection or hypothesis testing, which,
given the obscurity of R cannot be derived with some nice plug-in method. As
Ian said the models to be compared must also be based on the same and
identical data.
e) One measure available that is statistically at least defensible is the
log-likelihood. So what you can do is form a log-likelihood ratio (or Bayes
factor (there is the darn factor again, it’s a ratio)) and see where this
falls - and the answers are pretty soft and, probably because of that,
correspondingly realistic. This also makes - based on statistics alone -
deciding between different overall parameterizations difficult.
http://en.wikipedia.org/wiki/Bayes_factor
f) so having said that, what really remains is that the model that fits the
primary evidence (minimally biased electron density) best and is at the same
time physically meaningful, is the best model, i. e., all plausibly
accountable electron density (and not more) is modeled. You can convince
yourself of this by taking the most interesting part of the model out (say a
ligand or a binding pocket) and look at the R-values or do a model selection
test - the result will be indecisive. Poof goes the global rule of thumb.
g) in other words: global measures in general are entirely inadequate to
judge local model quality (noted many times over already by Jones, Kleywegt,
others, in the dark ages of crystallography when poorly restrained
crystallographers used to passionately whack each other over the head with
unfree R-values).
Best, BR
-----------------------------------------------------------------
Bernhard Rupp, Hofkristallrat a.D.
001 (925) 209-7429
+43 (676) 571-0536
[email protected]
[email protected]
http://www.ruppweb.org/
------------------------------------------------------------------
Und wieder ein chillout-mix aus der Hofkristall-lounge
------------------------------------------------------------------
--
Ethan A Merritt
Biomolecular Structure Center, K-428 Health Sciences Bldg
University of Washington, Seattle 98195-7742
