Dear Kay,
I think I should add a few comments and annotations to your reply to
Arpita, as it was addressed not just to her but to the general readership of
the CCP4BB. This will involve introducing an extra level of interleaving,
which can get a bit unsightly but can hardly be avoided.
Please see below.
On Mon, Feb 12, 2024 at 04:55:02PM +0000, Kay Diederichs wrote:
> Dear Arpita,
>
> I'll try to answer below -
>
> On Mon, 12 Feb 2024 13:57:28 +0530, Arpita Goswami <[email protected]>
> wrote:
>
> >Dear all,
> >
> >Greetings to all! Apologies if the below query seems very naive!
> >
> >This is to query on the consensus to use Staraniso for pdb submission. We
> >have solved a structure previously at 2.3 A resolution. The same data
> >(after reindexing the diffraction images in autoPROC) and after
> >reprocessing by ellipsoidal scaling in Staraniso gave structure at ~2.16 A.
> >
> >The previously solved structure did not have significant anisotropy
> >according to Aimless, so anisotropic scaling was not performed that time.
>
> Maybe different resolution cutoffs were used: previously the
> data were cut a 2.3A (according to some rule, e.g. "cut at <I/sigma>=2"),
> now they are cut at ~2.16A (according to a different rule, e.g. "cut at
> <I/sigma>=1.5").
>
> Alternatively, different programs were used for data processing, and for
> the processing by the one used previously, the rule said "you should cut at
> 2.3A", and for the one used now, the same rule said "cut at ~2.16A".
> After all, different programs give different data!
>
> Speaking of rules, there are different "schools of thought" concerning the
> "best" or "correct" rule for finding the resolution cutoff. I don't want to
> repeat
> the arguments here. It is important to know that there exists an experimental
> and practical way (called "paired refinement") to find a meaningful
> resolution
> cutoff, and this requires refinement of a model, and comparison of
> (basically)
> Rfree values. The CCP4 program for this purpose is called PAIREF.
> There is also a web server called PDB-REDO that does paired refinement with
> your data and model.
Arpita may not have come across the arguments that you don't want to
repeat, so she may find it useful to peruse the following item in the CCP4BB
archive:
https://www.mail-archive.com/[email protected]/msg54145.html
that provides a useful introduction to the content of your paragraph above.
In particular, her use of PAIREF as available from CCP4 or PDB-REDO would
give her a resolution cut-off as a *single* number: applying such a cut-off
would then get her back to the situation where either good data will be
discarded in the better-diffracting direction(s), or essentially pure noise
will be kept in the worse-diffracting one(s). So this would be a huge step
backward from the results she has been alluding to, where we happen to know
that the default STARANISO analysis shows that these tetragonal crystals
diffract to 2.526 A along a* and b*, and to 2.039 A along c*. Trusting
PAIREF to give a useful indication of a single-number resolution in this
situation would seem totally unrealistic.
There has been a publication on the simultaneous use of PAIREF and
STARANISO, using verbatim (and without acknowledgement) the procedure that
was suggested in the archive e-mail cited above (paragraph starting with
"Quite the contrary:"), namely using datasets obtained by running STARANISO
with different cut-off values for the local average of I/sig(I) and applying
the PAIREF procedure to assess which one is the most appropriate, and then
taking the associated diffraction limits as being the *three* diffraction
limits of the dataset. This way, one preserves the benefits of the
anisotropy analysis and one simply tweaks the STARANISO cut-off threshold
around its default value (1.2) by specifying another value through a
command-line parameter provided for that purpose since the inception of the
program. The publication in question can be found at
https://journals.iucr.org/j/issues/2023/04/00/ap5049/index.html
The process of exploring different STARANISO thresholds is manually driven,
and none of the available versions of PAIREF offer that feature - so
directing Arpita towards them may not be to her advantage. Before investing
too much hope in this type of investigation, it should be noted that this
article concludes that
"Corrections for diffraction anisotropy using the STARANISO server [with a
default cut-off of 1.2] and [its output] data dramatically improved the
quality of the observed electron density maps. No differences were observed
with the extension of data from [those obtained with a cut-off value of
1.2] to [those obtained with a cut-off value of 0.5]".
> >The overall spherical completeness of Staraniso structure is low (~73%)
> >while Ellipsoidal completeness is ~94%. Parallel isotropic scaling gives
> >structure with 99.6% completeness (but 2.3 A resolution). The statistics (R
> >merge and others) are better for Staraniso structure (also benefited from
> >removing specific frames with high R merge as indicated by Staraniso). Also
>
> I understand that, based on what you wrote, autoPROC was used for processing
> the most recent data. Actually, Staraniso is run from autoPROC; it is part of
> the
> autoPROC expert system (but there is also a web server for it).
> The removal of specific frames is done by a different part of autoPROC,
> not by Staraniso.
>
> >the interatomic distances in regions of interest in the staraniso structure
> >is on par with parallel molecular dynamics simulation data.
>
> This needs a lot more explanation and context information, and discussion.
>
> >
> >The questions are:
> >
> >1. Can the Staraniso structure be submitted to pdb saying reprocessed
> >structure at higher resolution (through Staraniso)?
>
> As recently reported here on CCP4BB (IIRC), there exists a PDB
> policy that only the previous authors of the PDB entry can re-submit.
> If existing data are re-processed and a new PDB entry submitted, then this
> must be accompanied by a paper.
>
> >
> >2. What is the factor more important for a structure: completeness
> >(spherical vs ellipsoidal) or R statistics?
>
> You ask the correct general question: how do I obtain the best structure?
> The answer to your specific question is: it depends. There is not one answer
> that fits all situations.
> Paired refinement should help you to find the best dataset for refinement.
I will not repeat my arguments, made earlier, for saying that this does
not seem to be a helpful suggestion at all.
> >3. Why is the extra resolution not detected during indexing by iMOSFLM or
> >XDS (using default setup)? The indexed outputs of either of them did not
> >give extra resolution (through anisotropic scaling) in Staraniso, although
> >it said some data was missing.
>
> I'm afraid there is some confusion here.
> To try and clarify: indexing has nothing to do with resolution (indexing means
> assigning indices, i.e. hkl triples, to reflections that are found during an
> initial
> spot finding procedure), so it is not clear that "extra resolution not
> detected
> during indexing" is an appropriate description for what you see.
>
> >
> >4. Is there any option for using all reflections detection (like autoPROC)
> >in iMOSFLM or XDS?
>
> Again, there is some confusion here: autoPROC is an expert system that uses
> XDS. More explicitly: autoPROC creates XDS.INP and then runs XDS. Then it
> analyzes the XDS results, and afterwards uses that analysis to create an
> improved XDS.INP, and re-runs XDS.
Thank you for volunteering to explain to Arpita what autoPROC does.
Your summary seems rather minimalistic, however. It is indeed one way of
describing autoPROC and its basic relation with XDS, but it leaves out the
fact that autoPROC adds a VERY large amount of expert knowledge that is
often difficult for an average - or even an expert - user to keep track of
and to act upon in a systematic manner. To mention just a few: automatic
detection of multiple lattices and of ice-rings, with remediation of the
latter; automatic analysis of poor image ranges and radiation damage, and
exclusion of those ranges; built-in handling of multi-sweep datasets, be
they multi-wavelength, or multi-orientation, or both; iterative scaling,
detection of damaged pixels, handling of dynamic/moving shadows, improved
determination of average background, optimisation of automatic beamstop
masking, coherent indexing across multiple sweeps; automatic detection of
beam-centre conventions, creation of deposition-ready mmCIF files, etc). A
lot of this is based on XDS results, but others are based on fairly advanced
additional analysis that could also be done using integrated intensities
from other programs than XDS (in fact the development of autoPROC started 20
years ago, using MOSFLM as an indexing and integration engine).
> There is nothing to prevent you from running XDS manually, to obtain results
> like those that autoPROC obtains (or even better!). If you want to learn
> about these options: some of them are discussed at
> https://wiki.uni-konstanz.de/xds/index.php/Optimisation , or elsewhere in
> XDSwiki.
>
> Similarly, iMOSFLM can be optimized for your data, and this will result in
> better
> data, and higher resolution than is obtained with default parameters. I'd
> google "imosflm tutorial"; there are some interesting hits.
>
> There are workshops, organized by CCP4 and others, where you can learn these
> things.
>
> Hope this helps,
> Kay
Indeed, there is nothing to prevent people from using programs
manually, but automation has happened and it is very hard to sail against
that wind.
In any case, Arpita's original enquiry was about the deposition of
STARANISO-treated data, and hopefully the reply that Clemens posted
yesterday will have given her useful guidance on what has become possible
thanks to joint work with the wwPDB, that has created a home in the PDB
archive for the results of anisotropy analysis in diffraction datasets.
***** For anyone hesitating about depositing STARANISO results as data with
***** associated descriptors: please do not hesitate to get in touch if you
***** encounter obstacles in that deposition - there should not be any.
With best wishes,
Gerard.
> >
> >
> >Thanks in advance,
> >
> >Best Regards,
> >
> >Arpita
> >
> >########################################################################
> >
> >To unsubscribe from the CCP4BB list, click the following link:
> >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1
> >
> >This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing
> >list hosted by www.jiscmail.ac.uk, terms & conditions are available at
> >https://www.jiscmail.ac.uk/policyandsecurity/
> >
>
> ########################################################################
>
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1
>
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at
> https://www.jiscmail.ac.uk/policyandsecurity/
########################################################################
To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB&A=1
This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list
hosted by www.jiscmail.ac.uk, terms & conditions are available at
https://www.jiscmail.ac.uk/policyandsecurity/
