Dear Petr,
On Thu, Mar 19, 2020 at 06:45:31AM +0000, Petr Kolenko wrote:
> For those of you who are in touch with these data, would deposition
> of unmerged intensities in P1 in the whole detector range instead of
> complete dataset be good enough to „correctly“ re-evaluete the
> structure? Or is there a doubt that even this approach would not
> lead to optimal structure?
It would be a step in the right direction, yes ... but:
* It assumes that all spots have been integrated and nothing else
than those spots: there can be cases of missed unit cell
duplications or incorrect enforcement of a unit cell and symmetry
(because those crystals always come in a known cell/SG, right?).
* We'll still be stuck with a particular intergration program and a
particular set of user decisions how to run it. Yes, all modern
data processing programs and packages are usually doing a great job
if run in (more or less) default mode on (more or less) good
crystals. But not everything is lysozyme/thaumatin/XYZ collected
fine-sliced on a PAD with low-dose, high-multiplicity on a well
calibrated instrument ;-)
* If integration was done in P1 the cell parameter restraints of the
(potentiually different) true SG were not enforced - which can lead
to incorrect cell parameters (which has knock-on effects in
refinement ... remember the WhatIf check for correct cell
parameter?).
If integration was done in a non-P1 SG and that choice was
incorrect: how do we know that those cell parameters didn't enforce
an incorrect set of cell parameter restraints, leading to poor
integration? Again: different programs have different ways of
assigning pixels (and their counts) to a reflection intensity
(concepts like shoebox, profile-fitting, pixel-labeling etc all
come to mind).
* Integration itself can be suboptimal if there are issues with the
crystal (ice rings, split, multiple lattices etc), the instrument
(damaged pixels, beam jitter, incorrect countrate cut-off handling
etc) or the experiment (radiation damage etc). Data processing
defaults might not always work correctly/best for all cases.
* The raw images allow such corrections (damaged pixels not yet in
pixel mask, incorrect handling of countrate_cutoff) and additional
analysis (e.g. radiation damage analysis a la F(early)-F(late) maps
[1]).
So the raw integrated (not scaled/corrected) and unmerged intensities
might indeed often be good enough, but if we are going to try and
deposit more than just the merged and scaled intensities/amplitudes
(as we do now) - and this will involve additional work and change from
our side - then why not bite the bullet once and try to go all the way
to raw images:
* The infrastructure is already in place (proteindiffraction.org,
data.sbgrid.org, zenodo.org, wwPDB DOIs, etc), ready for use right
now.
* From experience at multiple worshops: everyone is comfortable to
start data processing from raw images using the various software
tools out there that people are familiar with (while jumping in
half way is probably more challenging to non power-users).
Anyway, just some ideas - from someone downloading nearly all deposited
raw image data on a regular basis for testing and developing data
processing software (and finding a lot of issues in our software this
way ... as well as issues with experiments and sometimes the
instrument).
Cheers
Clemens
[1] as e.g. done in autoPROC+BUSTER
########################################################################
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB&A=1
