Douglas, I think you are missing the point that estimation of the parameters of the proper Bayesian statistical model (i.e. the Wilson prior) in order to perform the integration in the manner you are suggesting requires knowledge of the already integrated intensities! I suppose we could iterate, i.e. assume an approximate prior, integrate, calculate a better prior, re-do the integration with the new prior and so on (hoping of course that the whole process converges), but I think most people would regard that as overkill. Also dealing with the issue of averaging estimates of intensities that no longer have a Gaussian error distribution, and also crucially outlier rejection, would require some rethinking of the algorithms. The question is would it make any difference in the end compared with the 'post-correction' we're doing now?
Cheers -- Ian On 20 June 2013 18:14, Douglas Theobald <dtheob...@brandeis.edu> wrote: > I still don't see how you get a negative intensity from that. It seems > you are saying that in many cases of a low intensity reflection, the > integrated spot will be lower than the background. That is not equivalent > to having a negative measurement (as the measurement is actually positive, > and sometimes things are randomly less positive than backgroiund). If you > are using a proper statistical model, after background correction you will > end up with a positive (or 0) value for the integrated intensity. > > > On Jun 20, 2013, at 1:08 PM, Andrew Leslie <and...@mrc-lmb.cam.ac.uk> > wrote: > > > > > The integration programs report a negative intensity simply because that > is the observation. > > > > Because of noise in the Xray background, in a large sample of intensity > estimates for reflections whose true intensity is very very small one will > inevitably get some measurements that are negative. These must not be > rejected because this will lead to bias (because some of these intensities > for symmetry mates will be estimated too large rather than too small). It > is not unusual for the intensity to remain negative even after averaging > symmetry mates. > > > > Andrew > > > > > > On 20 Jun 2013, at 11:49, Douglas Theobald <dtheob...@brandeis.edu> > wrote: > > > >> Seems to me that the negative Is should be dealt with early on, in the > integration step. Why exactly do integration programs report negative Is > to begin with? > >> > >> > >> On Jun 20, 2013, at 12:45 PM, Dom Bellini <dom.bell...@diamond.ac.uk> > wrote: > >> > >>> Wouldnt be possible to take advantage of negative Is to > extrapolate/estimate the decay of scattering background (kind of Wilson > plot of background scattering) to flat out the background and push all the > Is to positive values? > >>> > >>> More of a question rather than a suggestion ... > >>> > >>> D > >>> > >>> > >>> > >>> From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of > Ian Tickle > >>> Sent: 20 June 2013 17:34 > >>> To: ccp4bb > >>> Subject: Re: [ccp4bb] ctruncate bug? > >>> > >>> Yes higher R factors is the usual reason people don't like I-based > refinement! > >>> > >>> Anyway, refining against Is doesn't solve the problem, it only > postpones it: you still need the Fs for maps! (though errors in Fs may be > less critical then). > >>> -- Ian > >>> > >>> On 20 June 2013 17:20, Dale Tronrud <det...@uoxray.uoregon.edu<mailto: > det...@uoxray.uoregon.edu>> wrote: > >>> If you are refining against F's you have to find some way to avoid > >>> calculating the square root of a negative number. That is why people > >>> have historically rejected negative I's and why Truncate and cTruncate > >>> were invented. > >>> > >>> When refining against I, the calculation of (Iobs - Icalc)^2 couldn't > >>> care less if Iobs happens to be negative. > >>> > >>> As for why people still refine against F... When I was distributing > >>> a refinement package it could refine against I but no one wanted to do > >>> that. The "R values" ended up higher, but they were looking at R > >>> values calculated from F's. Of course the F based R values are lower > >>> when you refine against F's, that means nothing. > >>> > >>> If we could get the PDB to report both the F and I based R values > >>> for all models maybe we could get a start toward moving to intensity > >>> refinement. > >>> > >>> Dale Tronrud > >>> > >>> > >>> On 06/20/2013 09:06 AM, Douglas Theobald wrote: > >>> Just trying to understand the basic issues here. How could refining > directly against intensities solve the fundamental problem of negative > intensity values? > >>> > >>> > >>> On Jun 20, 2013, at 11:34 AM, Bernhard Rupp <hofkristall...@gmail.com > <mailto:hofkristall...@gmail.com>> wrote: > >>> As a maybe better alternative, we should (once again) consider to > refine against intensities (and I guess George Sheldrick would agree here). > >>> > >>> I have a simple question - what exactly, short of some sort of > historic inertia (or memory lapse), is the reason NOT to refine against > intensities? > >>> > >>> Best, BR > >>> > >>> > >>> > >>> > >>> -- > >>> > >>> This e-mail and any attachments may contain confidential, copyright > and or privileged material, and are for the use of the intended addressee > only. If you are not the intended addressee or an authorised recipient of > the addressee please notify us of receipt by returning the e-mail and do > not use, copy, retain, distribute or disclose the information in or > attached to the e-mail. > >>> > >>> Any opinions expressed within this e-mail are those of the individual > and not necessarily of Diamond Light Source Ltd. > >>> > >>> Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for any > damage which you may sustain as a result of software viruses which may be > transmitted in or with the message. > >>> > >>> Diamond Light Source Limited (company no. 4375679). Registered in > England and Wales with its registered office at Diamond House, Harwell > Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > > >
