Dear Colleagues, I would like to add my voice to those advocating making available raw diffraction images and point out another resource for raw data deposition, https://cxidb.org, particularly for the case of emerging methods such as serial crystallography.
Cheers, Filipe On Thu, 19 Mar 2020 at 09:01, Clemens Vonrhein <[email protected]> wrote: > Dear Petr, > > On Thu, Mar 19, 2020 at 06:45:31AM +0000, Petr Kolenko wrote: > > For those of you who are in touch with these data, would deposition > > of unmerged intensities in P1 in the whole detector range instead of > > complete dataset be good enough to „correctly“ re-evaluete the > > structure? Or is there a doubt that even this approach would not > > lead to optimal structure? > > It would be a step in the right direction, yes ... but: > > * It assumes that all spots have been integrated and nothing else > than those spots: there can be cases of missed unit cell > duplications or incorrect enforcement of a unit cell and symmetry > (because those crystals always come in a known cell/SG, right?). > > * We'll still be stuck with a particular intergration program and a > particular set of user decisions how to run it. Yes, all modern > data processing programs and packages are usually doing a great job > if run in (more or less) default mode on (more or less) good > crystals. But not everything is lysozyme/thaumatin/XYZ collected > fine-sliced on a PAD with low-dose, high-multiplicity on a well > calibrated instrument ;-) > > * If integration was done in P1 the cell parameter restraints of the > (potentiually different) true SG were not enforced - which can lead > to incorrect cell parameters (which has knock-on effects in > refinement ... remember the WhatIf check for correct cell > parameter?). > > If integration was done in a non-P1 SG and that choice was > incorrect: how do we know that those cell parameters didn't enforce > an incorrect set of cell parameter restraints, leading to poor > integration? Again: different programs have different ways of > assigning pixels (and their counts) to a reflection intensity > (concepts like shoebox, profile-fitting, pixel-labeling etc all > come to mind). > > * Integration itself can be suboptimal if there are issues with the > crystal (ice rings, split, multiple lattices etc), the instrument > (damaged pixels, beam jitter, incorrect countrate cut-off handling > etc) or the experiment (radiation damage etc). Data processing > defaults might not always work correctly/best for all cases. > > * The raw images allow such corrections (damaged pixels not yet in > pixel mask, incorrect handling of countrate_cutoff) and additional > analysis (e.g. radiation damage analysis a la F(early)-F(late) maps > [1]). > > So the raw integrated (not scaled/corrected) and unmerged intensities > might indeed often be good enough, but if we are going to try and > deposit more than just the merged and scaled intensities/amplitudes > (as we do now) - and this will involve additional work and change from > our side - then why not bite the bullet once and try to go all the way > to raw images: > > * The infrastructure is already in place (proteindiffraction.org, > data.sbgrid.org, zenodo.org, wwPDB DOIs, etc), ready for use right > now. > > * From experience at multiple worshops: everyone is comfortable to > start data processing from raw images using the various software > tools out there that people are familiar with (while jumping in > half way is probably more challenging to non power-users). > > Anyway, just some ideas - from someone downloading nearly all deposited > raw image data on a regular basis for testing and developing data > processing software (and finding a lot of issues in our software this > way ... as well as issues with experiments and sometimes the > instrument). > > Cheers > > Clemens > > [1] as e.g. done in autoPROC+BUSTER > > ######################################################################## > > To unsubscribe from the CCP4BB list, click the following link: > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1 > ######################################################################## To unsubscribe from the CCP4BB list, click the following link: https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
