Along similar lines, if you happen to be working in the DIALS ecosystem there is the dials.damage_analysis program, which takes the scaled data (scaled.expt, scaled.refl) and computes a number of statistics including Rd as discussed below, and the cumulative-pairwise R factor Rcp, which measures how well new batches of reflections agree with those measured before - back in the day this was also implemented in the CCP4 program “chef” which may even still be distributed, which takes scaled but *unmerged* data in MTZ format, and so can work from data from XDS and other software as well
The details of this are described in §5.2 of https://journals.iucr.org/d/issues/2019/03/00/ba5301/ (shameless plug, sorry, but it is the easiest public reference for it) The short version is along the lines of - - the measure of agreement of pairwise R factors is, in the absence of damage, directly related to the I/sigma of the data - if the new measurements agree with those measured thus far, the trends are neutral / flat - if the new measurements are different from those who came before, the trends are monotonically upwards The dials.damage_analysis and chef tools also plot a completeness vs. dose (usually frame number as a proxy) which can give an indication of when you could consider cutting the data without harming the completeness of measurements The reason why Rcp was developed from Rd was that the latter indicates the presence of damage, but the former also gives some hints as to the point where this damage becomes evident I hope this is helpful, best wishes Graeme > On 31 Oct 2023, at 09:26, Harry Powell > <[email protected]> wrote: > > Hi > > I’ve never actually used it in anger (one should never be angry when > processing data…), but doesn’t AutoProc, developed by the good folks at > Global Phasing do a lot of these analyses? Clemens, Claus etc may have > something pertinent to say. > > Harry > >> On 30 Oct 2023, at 13:23, Jorge Iulek <[email protected]> wrote: >> >> Dear all, >> >> I have found many fundamental studies on image processing and >> refinement indexes concerning the decision on cutting resolution for a >> dataset, always meant to get better models, the final objective. Paired >> refinement has been a procedure mostly indicated. >> I have been searching studies alike concerning, in these days of >> thousands of collected images and strong x ray beams, the cutting (or >> truncation) of the (sequentially due to rotation method) recorded images in >> a dataset due to radiation damage. Once again, I understand the idea is to >> always produce better models. >> On one hand, the more images one uses, the higher the multiplicity, >> what (higher multiplicity) leads to better averaged intensity (provided >> scaling makes a good job), on the other hand, the more images one uses, >> lower intensity (due to the radiation damage) equivalent reflections come >> into play for scaling, etc. How to balance this? I have seen a case in which >> truncating images with some radiation damage led to worse CC(1/2) and >> <I/sigI> (at the same high resolution shell, multiplicities around 12.3 and >> then 5.7), but this might not be the general finding. In a word, are there >> indicators of the point where to truncate more precisely the images such >> that the dataset will lead to a better model? I understand tracing a sharp >> borderline might not be trivial, but even a blurred borderline might help, >> specially in the moment of image processing. >> I find that in >> https://ccp4i2.gitlab.io/rstdocs/tasks/aimless_pipe/scaling_and_merging.html#estimation-of-resolution >> there is a suggestion to try refinement with both truncating and not >> truncating. >> Sure other factors come into play here, like diffraction anisotropy, >> crystal internal symmetry, etc., but to start one might consider just the >> radiation damage due to exposure to x rays. Yes, further on, it would be >> nice the talk evolves to those cases when we see peaks and valleys along the >> rotation due to crystal anisotropy, whose average height goes on diminishing. >> Comments and indications to papers and material to study are welcome. >> Thanks. >> Yours, >> >> Jorge >> >> ######################################################################## >> >> To unsubscribe from the CCP4BB list, click the following link: >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 >> >> This message was issued to members of http://www.jiscmail.ac.uk/CCP4BB, a >> mailing list hosted by http://www.jiscmail.ac.uk/, terms & conditions are >> available at https://www.jiscmail.ac.uk/policyandsecurity/ > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 > > This message was issued to members of http://www.jiscmail.ac.uk/CCP4BB, a > mailing list hosted by http://www.jiscmail.ac.uk/, terms & conditions are > available at https://www.jiscmail.ac.uk/policyandsecurity/ -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1 This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list hosted by www.jiscmail.ac.uk, terms & conditions are available at https://www.jiscmail.ac.uk/policyandsecurity/
