This is a very educational thread but I should remind you that the
assumed distributions are NOT reliable when either a) the data is very
anisotropic, or b) the data is very incomplete or c) there is a
non-crystallographic translation vector in the structure or d) the data
is twinned.
I for one dont really know what to do about this, but remember the Is
are as measured and are in these cases "safer" reflections of the
experiment..
Eleanor
Ian Tickle wrote:
Having read the remainder of the paper more carefully I note that the
authors do go into an extensive discussion about Jeffreys (which they
don't recommend) and Wilson priors, which indeed overcome my objection
to the use of the improper prior. They conclude that the simpler
expression is adequate for their purposes. George Sheldrick's objection
would be valid for their simple prior since the effect on intensities in
a shell where the true average intensity was zero would be to give a
non-zero positive and hence biased average intensity. However I don't
think it's valid to conclude without more careful analysis that their
simple prior is also adequate in the single crystal case, since the
kinds of errors encountered (namely from deconvoluting overlapping
reflections) are quite different.
-- Ian
-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of [EMAIL PROTECTED]
Sent: 08 September 2008 22:20
To: Jacob Keller
Cc: [email protected]
Subject: Re: [ccp4bb] truncate ignorance
I would also recommend reading of the following paper:
D.S. Sivia & W.I.F. David (1994), Acta Cryst. A50, 703-714. A
Bayesian
Approach to Extracting Structure-Factor Amplitudes from Powder
Diffraction Data.
Despite of the title, most of the analysis presented in this paper
applies equally well to single-crystal data (see especially
sections 3
and 5). If you are not interested in the specific powder-diffraction
problems (i.e. overlapping peaks), you can simply skip
sections 4 and 6.
A few interesting points from this paper :
(1) The conversion from I's to F's can be done (in a Bayesian
way) by
applying two simple formula (equations 11 and 12 in the
paper), which,
for all practical purposes, are as valid as the more complicated
French & Wilson procedure (see discussion in section 5).
(2) Re. the use of I's rather than F's : this is discussed on
page 710
(final part of section 5). The authors seem to be more in favor of
using F's.
Marc Schiltz
Quoting Jacob Keller <[EMAIL PROTECTED]>:
Does somebody have a .pdf of that French and Wilson paper?
Thanks in advance,
Jacob
*******************************************
Jacob Pearson Keller
Northwestern University
Medical Scientist Training Program
Dallos Laboratory
F. Searle 1-240
2240 Campus Drive
Evanston IL 60208
lab: 847.491.2438
cel: 773.608.9185
email: [EMAIL PROTECTED]
*******************************************
----- Original Message -----
From: "Ethan Merritt" <[EMAIL PROTECTED]>
To: <[email protected]>
Sent: Monday, September 08, 2008 3:03 PM
Subject: Re: [ccp4bb] truncate ignorance
On Monday 08 September 2008 12:30:29 Phoebe Rice wrote:
Dear Experts,
At the risk of exposing excess ignorance, truncate makes me
very nervous because I don't quite get exactly what it is
doing with my data and what its assumptions are.
From the documentation:
========================================================
... the "truncate" procedure (keyword TRUNCATE YES, the
default) calculates a best estimate of F from I, sd(I), and
the distribution of intensities in resolution shells (see
below). This has the effect of forcing all negative
observations to be positive, and inflating the weakest
reflections (less than about 3 sd), because an observation
significantly smaller than the average intensity is likely
to be underestimated.
=========================================================
But is it really true, with data from nice modern detectors,
that the weaklings are underestimated?
It isn't really an issue of the detector per se, although in
principle you could worry about non-linear response to the
input rate of arriving photons.
In practice the issue, now as it was in 1977 (French&Wilson),
arises from the background estimation, profile fitting, and
rescaling that are applied to the individual pixel contents
before they are bundled up into a nice "Iobs".
I will try to restate the original French & Wilson argument,
avoiding the terminology of maximum likelihood and
Bayesian statistics.
1) We know the true intensity cannot be negative.
2) The existence of Iobs<0 reflections in the data set means
that whatever we are doing is producing some values of
Iobs that are too low.
3) Assuming that all weak-ish reflections are being processed
equivalently, then whatever we doing wrong for reflections with
Iobs near zero on the negative side surely is also going wrong
for their neighbors that happen to be near Iobs=0 on the positive
side.
4) So if we "correct" the values of Iobs that went negative, for
consistency we should also correct the values that are nearly
the same but didn't quite tip over into the negative range.
Do I really want to inflate them?
Yes.
Exactly what assumptions is it making about the expected
distributions?
Primarily that
1) The histogram of true Iobs is smooth
2) No true Iobs are negative
How compatible are those assumptions with serious anisotropy
and the wierd Wilson plots that nucleic acids give?
Not relevant
Note the original 1978 French and Wilson paper says:
"It is nevertheless important to validate this agreement for
each set of data independently, as the presence of atoms in
special positions or the existence of noncrystallographic
elements of symmetry (or pseudosymmetry) may abrogate the
application of these prior beliefs for some crystal
structures."
It is true that such things matter when you get down to the
nitty-gritty details of what to use as the "expected distribution".
But *all* plausible expected distributions will be non-negative
and smooth.
Please help truncate my ignorance ...
Phoebe
==========================================================
Phoebe A. Rice
Assoc. Prof., Dept. of Biochemistry & Molecular Biology
The University of Chicago
phone 773 834 1723
http://bmb.bsd.uchicago.edu/Faculty_and_Research/01_Faculty/01
_Faculty_Alphabetically.php?faculty_id=123
RNA is really nifty
DNA is over fifty
We have put them
both in one book
Please do take a
really good look
http://www.rsc.org/shop/books/2008/9780854042722.asp
--
Ethan A Merritt
Biomolecular Structure Center
University of Washington, Seattle 98195-7742
Disclaimer
This communication is confidential and may contain privileged information intended solely for the named addressee(s). It may not be used or disclosed except for the purpose for which it has been sent. If you are not the intended recipient you must not review, use, disclose, copy, distribute or take any action in reliance upon it. If you have received this communication in error, please notify Astex Therapeutics Ltd by emailing [EMAIL PROTECTED] and destroy all copies of the message and any attached documents.
Astex Therapeutics Ltd monitors, controls and protects all its messaging traffic in compliance with its corporate email policy. The Company accepts no liability or responsibility for any onward transmission or use of emails and attachments having left the Astex Therapeutics domain. Unless expressly stated, opinions in this message are those of the individual sender and not of Astex Therapeutics Ltd. The recipient should check this email and any attachments for the presence of computer viruses. Astex Therapeutics Ltd accepts no liability for damage caused by any virus transmitted by this email. E-mail is susceptible to data corruption, interception, unauthorized amendment, and tampering, Astex Therapeutics Ltd only send and receive e-mails on the basis that the Company is not liable for any such alteration or any consequences thereof.
Astex Therapeutics Ltd., Registered in England at 436 Cambridge Science Park,
Cambridge CB4 0QA under number 3751674
