Re: [ccp4bb] Rescale merged data?

[Clemens Vonrhein]

Dear Randy, Harry et al,

sorry for replying to several emails in one message - but as always,
everything is connected ;-)

On Thu, Apr 18, 2024 at 10:02:40AM +, Randy John Read wrote:
> I’d like to add my strong agreement to what Robbie said, but also point
> out a wrinkle. When the PDB runs validation, it just takes the data that
> are in the first reflection loop of the reflections.cif file.

Just a small adjustment of naming convention here (as I understand it): you
are referring to so-called "data blocks" within an mmCIF file (delimited by
a "data_XYZ" token) while a "loop" is just a CIF format construct. So a
single datablock can have multiple loops - as visible in a model mmCIF
file. Right?

> So if you want the validation statistics to match your reported
> refinement statistics, that loop should contain the set of data you gave
> the refinement program,

Couldn't agree more - but there are a few additional creases (see below) to
try and stay in your picture ...

> especially if you’ve done something like apply an elliptical truncation,
> correct for anisotropy, or convert intensities to amplitudes, all of
> which change the data in ways that can’t be reversed later.

... some of those are actually applied by the refinement program itself
(rejection of outlier reflections or scaling of the observed amplitudes
against the model using anisotropic B-factors). That immediately creates a
problem: we require the output reflection data (containing map coefficients
for e.g. 2mFo-DFc and mFo-DFc maps) but can't rely on the observed
intensities/amplitudes in that file to accurately represent the input
data. Therefore, we might also need to collect the observed data as-is from
the input reflection file - and combine those two files into a single
datablock of a reflection mmCIF file intended for deposition.

There are a few points we could make about (a) ellipsoidal truncation and
its relation to isotropic and anisotropic truncation, and (b) correction
for anisotropy, but we placed them onto a separate page at [1] to keep this
email thread reasonably short.

Your remark about the conversion from intensities to amplitudes in the
context of multiple datablocks in mmCIF files is probably out of a concern
that one might end up just with amplitudes for a given PDB deposition -
which indeed would not be ideal. That danger lies mostly in the way we now
seem to split, shuffle, reassemble and push our original reflection data
files around different programs, online services and GUIs. Originally, any
program that we know of doing that I->F conversion would always carry over
the input intensities into the output (together with the newly created
amplitudes and other items like anomalous data etc), so there shouldn't be
a reason for losing the intensities at this point ... ideally.

The beauty of the good ol' MTZ files was that all those items were always
kept together and - once a test-set flag was added - one only ever needed
to refer to that "master" reflection file for any subsequent steps to keep
all those relations and the content intact ... no daisy-chaining of
reflection data input/output channels with the subsequent loss of
information, provenance and meta-data.

> Whenever you’ve done any of this (and many people are using StarAniso
> these days, which does all of those things),

Just as a clarification for the less experienced users: there are lots of
other programs that do exactly the same thing - or at least variants that
follow the same underlying idea. Most (all?) data-processing
programs/packages/pipelines will apply a data truncation step and several
refinement programs will reject observations or apply anisotropic
corrections to the observed data. STARANISO is not really unique in those
underlying concepts as far as we can see.

> please put in a second reflection loop containing the whole set of
> intensities to the highest resolution you used, without any anisotropic
> scaling or elliptical cutoffs. Then anyone wanting to re-refine your
> structure or check your data for artefacts will have more information
> available.

Absolutely: we couldn't agree more and are trying to provide as simple to
use tools as possible to users (not only of our software - see [2], since
October 2020). The hope is that the deposition-preparation step becomes as
painless as possible, even if it can't (yet) be fully automatic because

  (a) the processed diffraction data usually comes from a synchrotron
  system (provided as mmCIF for deposition, but usually the MTZ file is
  picked up for downstream work), and

  (b) the refinement is done in a separate system resulting in a separate,
  related but not directly linked reflection file after model
  refinement.

> Of course, I hope we’re moving to a world in which we all also deposit
> the intensities before merging, which in principle allows even more
> quality control to be done.

For the last several years we are providing that feature as part of our own

Re: [ccp4bb] About Staraniso

[Clemens Vonrhein]

g
procedure. Acta Crystallographica Section D: Biological
Crystallography, 61(7), pp.850-855.

[9] Afonine, P.V., Grosse-Kunstleve, R.W., Chen, V.B., Headd, J.J.,
Moriarty, N.W., Richardson, J.S., Richardson, D.C., Urzhumtsev,
A., Zwart, P.H. and Adams, P.D., 2010. phenix. model_vs_data: A
high-level tool for the calculation of crystallographic model and
data statistics. Journal of applied crystallography, 43(4),
pp.669-676.

[10] Afonine, P.V., Grosse-Kunstleve, R.W., Adams, P.D. and
 Urzhumtsev, A., 2013. Bulk-solvent and overall scaling revisited:
 faster calculations, improved results. Acta Crystallographica
 Section D: Biological Crystallography, 69(4), pp.625-634.

 Afonine, P.V., Grosse-Kunstleve, R.W., Adams, P.D. and
 Urzhumtsev, A., 2023. Bulk-solvent and overall scaling revisited:
 faster calculations, improved results. Corrigendum. Acta
 Crystallographica Section D: Structural Biology, 79(7).

[11] Shakked, Z., 1983. Anisotropic scaling of three-dimensional
 intensity data. Acta Crystallographica Section A: Foundations of
 Crystallography, 39(3), pp.278-279.

[12] Pohl, E., Schneider, T.R., Dauter, Z., Schmidt, A., Fritz,
 H.J. and Sheldrick, G.M., 1999. 1.7 Å structure of the stabilized
 REIv mutant T39K. Application of local NCS restraints. Acta
 Crystallographica Section D: Biological Crystallography, 55(6),
 pp.1158-1167.

[13] Sheldrick, G.M., 2012. Macromolecular applications of
 SHELX. International Tables for Crystallography
 (2012). Vol. F. ch. 18.9, pp. 529-533.

[14] Blanc, E., Roversi, P., Vonrhein, C., Flensburg, C., Lea,
 S.M. and Bricogne, G., 2004. Refinement of severely incomplete
 structures with maximum likelihood in BUSTER–TNT. Acta
 Crystallographica Section D: Biological Crystallography, 60(12),
 pp.2210-2221.

[15] https://phenix-online.org/documentation/faqs/refine.html#general ("Why
 does phenix.refine not use all data in refinement?") and
 https://phenix-online.org/pipermail/phenixbb/2010-December/016283.html

[16] Spek, A.L., 2015. PLATON SQUEEZE: a tool for the calculation of
 the disordered solvent contribution to the calculated structure
 factors. Acta Crystallographica Section C: Structural Chemistry,
 71(1), pp.9-18.

[17] DeLaBarre, B. and Brunger, A.T., 2006. Considerations for the
 refinement of low-resolution crystal structures. Acta
 Crystallographica Section D: Biological Crystallography, 62(8),
 pp.923-932.

[18] Liu, C. and Xiong, Y., 2014. Electron density sharpening as a
 general technique in crystallographic studies. Journal of
 molecular biology, 426(4), pp.980-993.

[19] Terwilliger, T.C., Sobolev, O.V., Afonine, P.V. and Adams, P.D.,
 2018. Automated map sharpening by maximization of detail and
 connectivity. Acta Crystallographica Section D: Structural
 Biology, 74(6), pp.545-559.

[20] Jiang, J.S. and Brünger, A.T., 1994. Protein hydration observed
 by X-ray diffraction: solvation properties of penicillopepsin and
 neuraminidase crystal structures. Journal of molecular biology,
 243(1), pp.100-115.

[21] https://www.ccp4.ac.uk/html/ctruncate.html

[22] https://staraniso.globalphasing.org/

[23] 
https://www.globalphasing.com/autoproc/ReleaseNotes/ReleaseNotes-autoPROC_snapshot_20190301.txt

[24] https://www.globalphasing.com/buster/wiki/index.cgi?DepositionMmCif

[25] 
https://www.globalphasing.com/autoproc/wiki/index.cgi?RunningAutoProcAtSynchrotrons

[26] https://www.globalphasing.com/buster/

[27] https://staraniso.globalphasing.org/table1/
 (e.g. https://staraniso.globalphasing.org/table1/ar/8ar7.html)

[28] https://pdbj.org/mine/experimental_details/8AR7

[29] https://www.wwpdb.org/task/mmcif

[30] https://github.com/pdbxmmcifwg/mmcif-data-proc

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] About Staraniso

[Clemens Vonrhein]

Dear Arpita,

> Apologies if the below query seems very naive!

Your query is not at all naive, it is very probing.  Sorry for the
necessarily long reply to your questions - but there are a number of topics
you raise where we think a large amount of confusion still exists.  Please
note that this reply represents /our/ view of things only of course.

> This is to query on the consensus to use Staraniso for pdb submission. We
> have solved a structure previously at 2.3 A resolution.

So you had a dataset where you decided on a sphere in reciprocal space with
a radius of 2.3A as a cut-off surface - based on some kind of local
analysis that convinced you that all the measured reflections within that
sphere (i.e. 2.3A and lower) are observed and should be kept, while all
measured reflections outside that sphere should be regarded as unobserved
and can be discarded as pure noise.

> The same data (after reindexing the diffraction images in autoPROC) and
> after reprocessing by ellipsoidal scaling in Staraniso gave structure at
> ~2.16 A.

OK, first some clarification:

  * The scaling of the unmerged reflection data in autoPROC (using AIMLESS)
is neither spherical nor ellipsoidal in itself: it uses the data as it
is with the typical scale parameterisation in AIMLESS, i.e. a scale k
and an image B-factor (plus some absorption), all with default
smoothing.

This then leads to two output reflection files:

 aimless_alldata_unmerged.mtz  = Scaled and unmerged reflections without 
cut-off.
 aimless_alldata.mtz   = Scaled and merged reflections without 
cut-off.

  * The latter (scaled and merged reflection data without any cut-off) is
then given to STARANISO to do the following:

(a) Compute various local statistics that are then used to define a
cut-off surface.

(b) Assume that all reflections within that cut-off surface should be
kept (and could have been observed) and all those outside should be
ignored.

==> See how that is extremely similar to the type of analysis you did
with the initial 2.3A data?  One type of analysis (local 1D-shells
of data in d*) lead to an isotropic sphere as a cut-off surface,
while another (local 3D-spheres in reciprocal space) lead to an
anisotropic cut-off surface.

Remember that "anisotropic" just means "not isotropic" - it doesn't
mean "ellipsoidal" (diffraction from a cubic crystal can be
anisotropic since the [100], [110] and [111] directions have quite
different properties, yet attempts to fit an ellipsoid to it will
produce a sphere).  The cut-off surface assigned this way by
STARANISO can have any shape really (including being a sphere)
because the analysis via local spheres doesn't assume/enforce
isotropy - while the analysis via spherical shells does.

So up to that point there is no difference really between the two
approaches: using a criterion to define a cut-off surface and
considering data within the surface as observable and data outside
as unobservable. It is only the assumptions on which the criterion
is based that differ: one assumes the data is isotropic, while the
other doesn't.

==> The notion of "resolution" is a bit complicated in general here: if
your crystal diffracted better in some directions than in others, a
better description is the use of "diffraction limit" in some
directions - e.g.  defined as a the principal axes of an ellipsoid
fitted to the cut-off surface. This is what autoPROC/STARANISO
provides.

   (c) Analyse the anisotropic fall-off in intensity of the data within the
   cut-off surface to derive anisotropic correction factors and apply
   them to the data.

   This is similar to the anisotropic scaling a refinement program
   would perform using the current model as a reference (to
   anisotropically scale the observed data to the model).  Here we
   apply an internal anisotropic scaling, without a reference to any
   model.

> The previously solved structure did not have significant anisotropy
> according to Aimless, so anisotropic scaling was not performed that time.

See above: you most likely did use anisotropic scaling during refinement
(with the model as the reference).

Please note that using AIMLESS alone is not the best way to detect
anisotropy; that is not its main purpose.  As far as we know, it looks only
along the crystal axes for anisotropy (whereas STARANISO looks in all
directions). That means it will not detect anisotropy eigenvectors lying
close to diagonals as can happen in monoclinic (a*-c* plane only since an
anisotropy eigenvector is constrained to be parallel to the b* axis) and
triclinic lattices.  So if AIMLESS says there is significant anisotropy you
can believe it; OTOH if it says no anisotropy was detected you should

Re: [ccp4bb] Fwd: radiation damage and image discard

[Clemens Vonrhein]

Dear Jorge,

as Harry mentioned, autoPROC [1] will automatically exclude image
ranges that are significantly worse than the rest (i.e. add mostly
noise). This could be due to radiation damage, badly centred crystals
or anything else. The so-called "fitness parameter" it uses takes
multiplicity and completeness into account when judging if a set of
images should be excluded [2].

Those automatic decisions seem to work very well in our and our users
hands ... as far as we can tell [3]

Cheers

Clemens

[1] https://www.globalphasing.com/autoproc/
[2] a paper describing this in detail is under review atm
[3] like most software developers we tend to get feedback if
something doesn't work ;-)

On Tue, Oct 31, 2023 at 10:48:54AM -0400, Jorge Iulek wrote:
> Hi,
> 
>   Well, it seems there are already many good indications to study and to 
> work
> on.
>   Thanks, Oliver, Kay, Harry and Graeme!
>   Now, brain and hands on!
> 
> Jorge
> 
>  Forwarded Message 
> ...
> 
> Dear all,
> 
>   I have found many fundamental studies on image processing and refinement
> indexes concerning the decision on cutting resolution for a dataset, always
> meant to get better models, the final objective. Paired refinement has been
> a procedure mostly indicated.
>   I have been searching studies alike concerning, in these days of 
> thousands
> of collected images and strong x ray beams, the cutting (or truncation) of
> the (sequentially due to rotation method) recorded images in a dataset due
> to radiation damage. Once again, I understand the idea is to always produce
> better models.
>   On one hand, the more images one uses, the higher the multiplicity, what
> (higher multiplicity) leads to better averaged intensity (provided scaling
> makes a good job), on the other hand, the more images one uses, lower
> intensity (due to the radiation damage) equivalent reflections come into
> play for scaling, etc. How to balance this? I have seen a case in which
> truncating images with some radiation damage led to worse CC(1/2) and
>  (at the same high resolution shell, multiplicities around 12.3 and
> then 5.7), but this might not be the general finding. In a word, are there
> indicators of the point where to truncate more precisely the images such
> that the dataset will lead to a better model? I understand tracing a sharp
> borderline might not be trivial, but even a blurred borderline might help,
> specially in the moment of image processing.
>   I find that in 
> https://ccp4i2.gitlab.io/rstdocs/tasks/aimless_pipe/scaling_and_merging.html#estimation-of-resolution
> there is a suggestion to try refinement with both truncating and not
> truncating.
>   Sure other factors come into play here, like diffraction anisotropy,
> crystal internal symmetry, etc., but to start one might consider just the
> radiation damage due to exposure to x rays. Yes, further on, it would be
> nice the talk evolves to those cases when we see peaks and valleys along the
> rotation due to crystal anisotropy, whose average height goes on
> diminishing.
>   Comments and indications to papers and material to study are welcome.
> Thanks.
>   Yours,
> 
> Jorge
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] in memoriam of Raimond Ravelli

[Clemens Vonrhein]

Very, very sad news.

He had always so much enthusiasm, good nature and friendliness that it
was pure joy to work with him on using radiation damage as a source of
small differences to use for phasing. He will be sadly missed!

Kind regards

Clemens

> Dear all, 
> 
> It is with profound sadness I announce the passing of Raimond Ravelli, one of 
> most remarkable and beloved members of our scientific community. 
> 
> Raimond first became widely known to the crystallographic community with his 
> brilliant “strategy” program in the 90’s, which he developed as part of his 
> PhD thesis. Later on, at the EMBL Grenoble and the ESRF, Raimond has been 
> instrumental in the development of the ID14-EH4 beamline, having helped 
> numerous colleagues to collect diffraction data and participating in many 
> exciting scientific projects. At that time, Raimond also developed his keen 
> interest in radiation damage, that influenced the ways that we collect and 
> process X-ray diffraction data even today. Raimond came back to his home 
> country, the Netherlands, first as an assistant professor at Leiden 
> University, and then as associate professor at Maastricht University. During 
> that transition he become an expert in cryo-electron microscopy. He made 
> numerous contributions also to his new field, in such a broad spectrum of 
> techniques, that it would be embarrassing to attempt to enumerate them here. 
> I will only mention that one piece of his work even became a museum exhibit 
> at the Leiden museum of natural history. 
> 
> Raimond has been recently appointed Full Professor of Structural Biology, at 
> Maastricht University, last November. His inauguration speech (Oratie) was 
> simply remarkable. I warmly recommend to anyone to watch it at [ 
> https://youtu.be/Gv3pKRp5S2Y?t=667 | https://youtu.be/Gv3pKRp5S2Y?t=667 ] . 
> It radiates his enthusiasm for science and is inspiring for all, in a way 
> that only he himself knew how to do. 
> 
> Raimond was an accomplished rowing athlete, a formidable cyclist, and a 
> hiking enthusiast. 
> 
> For the many of us who have been his friends it has a been a privilege and an 
> inspiration. Raimond, just 55, leaves behind him Maaike, an excellent 
> scientist and colleague herself, and their kids Seppe and Noé. 
> 
> I have no words that could possibly explain how much he will be missed. 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Low resolution and high anisotropy

[Clemens Vonrhein]

Dear Lande,

On Wed, Oct 26, 2022 at 08:09:10AM +0100, Lande Fu wrote:
> I noticed STARANISO processd the data all the way down to 1.7A, with
> very poor stats on high resolution shell.

Were you looking at the second table in the original email? It is not
in a format I recognize as coming from STARANISO ... maybe this is
from the UCLA server or some manual procedure?

Cheers

Clemens



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Checking X-ray sequence (no more protein).

[Clemens Vonrhein]

Maybe a crazy idea, but couldn't one use various model/geometry
validation tools to figure out some of those ambiguities? As a test
one could take a very good 1.7A structure and do some random ASN->ASP,
THR->VAL etc mutations followed by refinement (including
hydrogens). Wouldn't some validation tool pick up unfavourable
conformations, poor rotamers and/or hydrogen clashes and poor
H-networks (compared to the initial, correct sequence)?

Maybe there is some kind of "fingerprint" in validation results for
such incorrect residue assignments that can distinguish correct from
incorrect sequences ... Or put another way, if model validation can
not pick up such sequence errors: should we be worried about the
reliability of our validation criteria?

A large scale re-refinement of deposited structures with (1) the
current/correct sequence and (2) those ASN/ASP, THR/VAL etc ambiguities
artificially introduced, could provide a clever algorithm (AI?) with
the data basis to figure out those "fingerprints". Maybe even for the
ASN/GLN/HIS side-chain orientations when the sequence is actually
correct.

Cheers

Clemens


On Fri, Jul 29, 2022 at 12:08:58PM +, Jon Cooper wrote:
> Thank you so much for your replies. I apologise for being unclear. The 
> protein is purified from a plant that hasn't had its genome sequence 
> determined. We know the enzyme family of the protein and therefore the 
> structure was originally solved by MR. The 'X-ray sequence' we have is just 
> determined from looking at the 1.7 Angstrom density, which is good, over 
> several refinement and rebuilding rounds. The resulting sequence has been run 
> through blast and it is up to 58% identical with other family members. To me 
> this seemed low but that degree of identity is typical of other family 
> members. The postgrad who did the work did obtain some peptide sequences and 
> prior to that about 20% of the sequence was determined by the Edman method 
> with the usual Asp/Asn and Glu/Gln ambiguity. However, there isn't any 
> prospect of us doing further experimental work, sorry, but that's the way it 
> is!!
> 
> Best wishes, Jon.C.
> 
> Sent from ProtonMail mobile
> 
>  Original Message 
> On 29 Jul 2022, 12:23, Jan Dohnalek wrote:
> 
> > If you know at least something about your protein, organism, type of 
> > molecule, ..., you could try mass spectrometry peptide mapping to known 
> > sequences, this may give you some answers for the ambiguities you might be 
> > seeing, if nothing else ..
> >
> > Jan
> >
> > On Fri, Jul 29, 2022 at 12:15 PM Jon Cooper 
> > <488a26d62010-dmarc-requ...@jiscmail.ac.uk> wrote:
> >
> >> Hello, I am looking for suggestions of ways to check a 1.7 Angstrom X-ray 
> >> sequence for a protein where it is impractical to do experimental 
> >> sequencing, protein or DNA. The structure refines to publishable R/R-free 
> >> and the main ambiguities seem to be Thr/Val, Asp/Asn and Glu/Gln where 
> >> alternative H-bonding networks are possible. Running alpha-fold seems an 
> >> interesting option? Any suggestions much appreciated.
> >>
> >> Cheers, Jon.C.
> >>
> >> Sent from ProtonMail mobile
> >>
> >> ---
> >>
> >> To unsubscribe from the CCP4BB list, click the following link:
> >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >
> > --
> >
> > Jan Dohnalek, Ph.D
> > Institute of Biotechnology
> >
> > Academy of Sciences of the Czech Republic
> > Biocev
> > Prumyslova 595
> > 252 50 Vestec near Prague
> > Czech Republic
> > Tel. +420 325 873 758
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Regarding the correct space group identification

[Clemens Vonrhein]

;> so, then run Zanuda to find out the real space group. 
> >> 
> >> 
> >> 
> >> Best, 
> >> 
> >> Herman 
> >> 
> >> 
> >> 
> >> *Von:* CCP4 bulletin board < [ mailto:CCP4BB@JISCMAIL.AC.UK | 
> >> CCP4BB@JISCMAIL.AC.UK ] > *Im Auftrag von *Sayan 
> >> Saha 
> >> *Gesendet:* Donnerstag, 28. Juli 2022 08:15 
> >> *An:* [ mailto:CCP4BB@JISCMAIL.AC.UK | CCP4BB@JISCMAIL.AC.UK ] 
> >> *Betreff:* [ccp4bb] Regarding the correct space group identification 
> >> 
> >> 
> >> 
> >> Dear All, 
> >> 
> >> 
> >> 
> >> We have collected home-source X-ray intensity data for a protein at 2.6 
> >> Angstrom. The data can be processed in either C2 (a=120, b=80, c=85 and 
> >> beta=115) or P222 (P22121, a=80, b=85, c=110). MR solution can be obtained 
> >> in both the space groups. However, the solution can be refined with an 
> >> Rw/Rf of 29/32% only. The protein is bound to a ligand 
> >> (co-crystallization) 
> >> for which a clear density can be observed. 
> >> 
> >> 
> >> 
> >> Any help and suggestion in this regard would be very helpful. 
> >> 
> >> 
> >> 
> >> With best regards, 
> >> 
> >> Sayan Saha. 
> >> 
> >> 
> >> 
> >> 
> >> -- 
> >> 
> >> To unsubscribe from the CCP4BB list, click the following link: 
> >> [ https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 | 
> >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 ] 
> >> 
> > 
> > 
> > 
> >To unsubscribe from the CCP4BB list, click the following link: 
> > [ https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 | 
> > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 ] 
> > 
> >This message was issued to members of [ http://www.jiscmail.ac.uk/CCP4BB | 
> >www.jiscmail.ac.uk/CCP4BB ] , a mailing list hosted by [ 
> >http://www.jiscmail.ac.uk/ | www.jiscmail.ac.uk ] , terms & conditions are 
> >available at [ https://www.jiscmail.ac.uk/policyandsecurity/ | 
> >https://www.jiscmail.ac.uk/policyandsecurity/ ] 
> > 
> 
>  
> 
> To unsubscribe from the CCP4BB list, click the following link: 
> [ https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 | 
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 ] 
> 
> This message was issued to members of [ http://www.jiscmail.ac.uk/CCP4BB | 
> www.jiscmail.ac.uk/CCP4BB ] , a mailing list hosted by [ 
> http://www.jiscmail.ac.uk/ | www.jiscmail.ac.uk ] , terms & conditions are 
> available at [ https://www.jiscmail.ac.uk/policyandsecurity/ | 
> https://www.jiscmail.ac.uk/policyandsecurity/ ] 
> 
> 
> 
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link: 
> [ https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 | 
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1 ] 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Comparing two datasets

[Clemens Vonrhein]

;>> still works and gives lots of statistics 
> >>> 
> >>> Phil
> >>> 
> >>> Sent from my iPhone
> >>> 
> >>>> On 26 Jul 2022, at 09:37, Andrew Leslie - MRC LMB 
> >>>>  wrote:
> >>>> 
> >>>> I think that POINTLESS works with intensities rather than structure 
> >>>> factors (I’m not sure if this can be changed). Also, SCALEIT gives a 
> >>>> much more detailed breakdown (R factors as a function of resolution and 
> >>>> differences in terms of sigmas etc) than POINTLESS WILL.
> >>>> 
> >>>> Cheers,
> >>>> 
> >>>> Andrew
> >>>> 
> >>>>> On 26 Jul 2022, at 09:24, LEGRAND Pierre 
> >>>>>  wrote:
> >>>>> 
> >>>>> Hello Mirek,
> >>>>> 
> >>>>> A very quick approach for that is offered by pointless:
> >>>>> 
> >>>>> pointless HKLREF 1_1_aimless.mtz  HKLIN 2_1_aimless.mtz
> >>>>> or
> >>>>> pointless HKLREF 1_1_aimless.mtz  XDSIN XDS_ASCII.HK
> >>>>> 
> >>>>> You will obtain a table looking like that, taking into account to 
> >>>>> possible reindexing:
> >>>>> 
> >>>>> Alternative indexing scores relative to reference
> >>>>>Alternative reindexingLklhd  CC R(E^2)Number 
> >>>>> Cell_deviation
> >>>>>  1  [h,k,l]  0.9930.9620.118 19150  
> >>>>> 0.08
> >>>>>  2  [-k,h,l] 0.0070.0780.512 19150  
> >>>>> 0.87
> >>>>> 
> >>>>> 
> >>>>> Best wishes,
> >>>>> 
> >>>>> Pierre Legrand
> >>>>> PROXIMA-1 Beamline
> >>>>> Synchrotron SOLEIL
> >>>>> 
> >>>>> De: "Nicolas Foos" 
> >>>>> À: CCP4BB@JISCMAIL.AC.UK
> >>>>> Envoyé: Mardi 26 Juillet 2022 08:36:35
> >>>>> Objet: Re: [ccp4bb] Comparing two datasets
> >>>>> 
> >>>>> Hi Mirek, 
> >>>>> 
> >>>>> I am pretty sure XSCALE will do that for you : 
> >>>>> https://xds.mr.mpg.de/html_doc/xscale_program.html
> >>>>> 
> >>>>> If not, maybe have a look on SHELXC in SIR mode. 
> >>>>> 
> >>>>> Hope this help. 
> >>>>> 
> >>>>> Nicolas
> >>>>> 
> >>>>> On 25/07/2022 21:52, Cygler, Miroslaw wrote:
> >>>>> Hi,
> >>>>> I would like to calculate the R-merge for Fs from two datasets 
> >>>>> processed from two different crystals. Tried to use Blend but got the 
> >>>>> message that Blend requires R. Downloaded R but do not know how to tell 
> >>>>> CCP4 where it is located on my Mac. Is there another program that would 
> >>>>> take two mtg files and merge the Fs?
> >>>>> Any help would be greatly appreciated. 
> >>>>> 
> >>>>> 
> >>>>> Mirek
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> To unsubscribe from the CCP4BB list, click the following link:
> >>>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >>>>> 
> >>>>> --
> >>>>> Nicolas Foos PhD - ARISE fellow
> >>>>> https://orcid.org/-0003-2331-8399
> >>>>> 
> >>>>> EMBL Grenoble, McCarthy Team
> >>>>> 71 av. des Martyrs,
> >>>>> 38000 Grenoble FRANCE
> >>>>> 
> >>>>> +33 4 57 42 84 67
> >>>>> 
> >>>>> To unsubscribe from the CCP4BB list, click the following link:
> >>>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >>>>> 
> >>>>> 
> >>>>> 
> >>>>> To unsubscribe from the CCP4BB list, click the following link:
> >>>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >>>>> 
> >>>> 
> >>>> 
> >>>> To unsubscribe from the CCP4BB list, click the following link:
> >>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >>>> 
> >>> 
> >>> To unsubscribe from the CCP4BB list, click the following link:
> >>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >>> 
> >> 
> >> To unsubscribe from the CCP4BB list, click the following link:
> >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >> 
> > 
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> > 
> > 
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] problems in Rfree-flags when refining with REFMAC and processing with STARANISO

[Clemens Vonrhein]

Dear Vera,

On Wed, Jul 13, 2022 at 10:13:28AM +0200, Vera Pfanzagl wrote:
> You write that I should make sure REFMAC reads in the test-set from
> the staraniso.mtz and does not replace it with a new one.  This
> would only be the case if it generates a new test set when I
> initially import the reflection file, right? How can I check that
> retrospectively with my history-riddled file?

The REFMAC program/binary itself will obviously read an MTZ file
exactly as-is when run via

  refmac5 xyzin your.pdb hklin your.mtz xyzout refmac.pdb hklout refmac.mtz < tmp.1
  mtzdmp output.mtz -n 100 > tmp.2

and compare those two text files (same number of reflections reported,
same completeness for test-set flags etc).

> PS: the links in [2] are not accurate anymore, the interface of
> CCP4i2 export alone already looks completely different and does not
> seem to want external processing files.

Probably something for wwPDB/CCP4/REFMAC to look into and to provide
an update.

Cheers

Clemens



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] problems in Rfree-flags when refining with REFMAC and processing with STARANISO

[Clemens Vonrhein]

e error message I receive:
> 
> 
> Warning:
>  (data block=1): 30301 reflections have status assigned as '?' (changed 
> to 'x') Warning: (data block=1): 30301 reflections have wrong status. 
> Not in list(o,f,x,-,<,h,l) Warning: No wavelength value was found in 
> SF file (data block=1).
> 
> which is essentially the same as above, so I guess it is derived from the 
> same problem.

I would be inclined to go back to your original REFMAC refinement
(before you did all that SFTOOLS, Phenix-again, server, conversion
steps): do you have just an output MTZ file and no reflection mmCIF?
Maybe by re-running the same job you could switch on some option to
trigger writing of the reflection data in mmCIF format? Maybe even
switching off the DFc completion and/or re-scaling?

> I would be very  grateful for any help. 

Not sure it does 

Cheers

Clemens

[1] In it's most simple form, such a combination consists of just
concatenating several mmCIF files together (as long as their data
blocks have different identifiers). However, if additional
decisions have been made between data processing and actual
refinement (like enantiomorph, space groups, alternative indexing
etc), some checks might be necessary.

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] problems in Rfree-flags when refining with REFMAC and processing with STARANISO

[Clemens Vonrhein]

Dear all,

On Mon, Jul 11, 2022 at 03:12:02PM +, Carrapa Nunes Dias, Joao Miguel wrote:
> I had a similar case and I solved it using the PDB tool pdb_extract from:
> https://pdb-extract.wwpdb.org/
> PDB Extract is a pre-deposition service for assembling structure files for 
> wwPDB OneDep deposition .

That is an option, yes - you might need to check what exactly you are
trying to solve here: overcoming some warning message from the PDB
deposition software or ensuring that the correct, complete and well
annotated reflection data is deposited at the PDB (while obviously
satisfying the deposition system requirements at the same time).

If at all possible, one should always use the deposition-ready
PDBx/mmCIF files that are produced by the relevant
software. E.g. autoPROC/STARANISO will produce such a file
(Data_1_autoPROC_STARANISO_all.cif) for the processing
stage. Refinement programs will also create such files
(e.g. BUSTER_refln.cif for BUSTER): the "only" thing a user needs to
do then, is to combine those two reflection files (the one from
processing and the one from refinement) before deposition. In the end
we would all like to have the following data available for a PDB entry
(going backwards):

  * the refinement results in terms of map coefficients: so we can see
what map the model is supposed to represent

  * the data that went into refinement: so we can potentially (re)refine

  * the original, scaled+merged reflection data (with isotropic or
anisotropic cut-off): in case some data selection happened during
refinement

  * the original, scaled+merged reflection data without any cut-off:
to c heck if the applied cut-off could be improved

  * the original, scaled+unmerged reflection data: to maybe improve
upon scaling and outlier rejection

If you were within the Global Phasing software package: we provide a
tool (aB_deposition_combine) to synchronise the refinement mmCIF with
the processing one and create a single reflection mmCIF file
containing all of the above.

(Ideally, the the raw diffaction images could be deposited too: to
enable users to redo the full processing).

> You have a pdb and you have a mtz file, then you should be able to
> convert them to the latest mmcif format.

There are various tools out there for such a conversion - but often
they concentrate on a single data block only. The result could be that
e.g. the MTZ file after refinement doesn't contain the original data
that was used for refinement, but a re-scaled version (maybe even with
some reflections rejected). Even if MTZ column names are identical,
the values might not be ("mtzdmp some.mtz -n 100" et al is your friend
here for checking).

> After that you can deposit your files in:
> https://deposit-1.wwpdb.org/deposition/
> https://validate-rcsb-2.wwpdb.org/
> 
> Since you used anisotropic data and different software, the final
> R/Rfree will likely be different of the estimated during PDB
> validation, and you will have the opportunity to explain it to the
> person from the PDB that will help you during your deposition.

Ideally, the re-computed R/Rfree should actually be identical if using
the deposited Fobs and Fcalc values from the mmCIF file: this is only
a question of recomputing those statistics (see e.g. our "rvalue" tool
in BUSTER).

However, that recomputation usually involves an actual re-refinement
step (even if only a recomputation of bulk solvent contribution and
overall scaling parameters): so the label might be a bit
misleading. There will always be discrepancies due to differences in
parametrisation between different programs. If the same program (and
same version) is used on a deposition with the same set of parameters
(hydrogens, formfactors, TLS etc) you should end up with extremely
similar R/Rfree values.

Using e.g. STARANISO data at that stage of "R/Rfree computation" with
any program should not really create very different R/Rfree values:
those are only ever computed from the observations present, so missing
data doesn't play a role here. There could be differences in the outer
shell value due to different binning methods though ...

> Finally, if you are using PHENIX, BUSTER, REFMAC/CCP4, you can also
> run a last refinement step with the latest version of the software.

That might work: but make sure you are reporting the /actual/
refinement program that did the heavy lifting for you here
... otherwise all depositions will look like being refined with XYZ
just because it produces mmCIF files that the deposition software is
happy about (while program XYZ didn't do anything to actually arrive
at that final model).

Proper annotation of all used software is important: not necessarily
for users of your structure, but for us developers ;-)

> The latest version will ensure that the mmcif format is compatible
> with the PDB. Sometimes older versions of the software will give you
> that type of error.

Indeed: keep your software reasonably up-to-date ;-)

Cheers

Clemens

Re: [ccp4bb] Improved support for extended PDBx/mmCIF structure factor files

[Clemens Vonrhein]

Dear Dom,

On Fri, Jan 14, 2022 at 10:56:32PM +, Dom Bellini wrote:
> Thanks for your inputs. I wonder why the PDB doesn’t allow to submit
> two separate mtz files,

Ideally, these would actually be mmCIF files since MTZ files (1) can not hold
all meta data a PDBx/mmCIF compatible file could potentially carry and (2) it
will require a conversion after upload. Of course, if your processing/refinement
software doesn't produce mmCIF files you are limited to that format and the
online tool Marcin mentioned is probably the best way to go.

> one from data processing and one from refinement with map
> coefficients.

If your processing and refinement package produces mmCIF files directly, it
would be best to take them as-is into deposition.

As an example, the combination autoPROC [1] and BUSTER [2] allows for this [3]:
taking the mmCIF file from data processing (including scaled merged and unmerged
data with enriched meta data and full data quality metrics) plus the mmCIF file
from refinement (including map coefficients in various forms) and combining them
via the aB_deposition_combine command (which takes care of alternative indexing,
test-set flags etc, since it might not be just a case of concatenating ASCII
mmCIF files together). This should work in creating a full reflection mmCIF with
all relevant types of reflection data.

There are probably similar systems in place to achieve something very similar in
other packages.

Cheers

Clemens

[1] https://www.globalphasing.com/autoproc
[2] https://www.globalphasing.com/buster
[3] https://www.globalphasing.com/buster/wiki/index.cgi?DepositionMmCif



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] what would be the best metric to asses the quality of a mtz file?

[Clemens Vonrhein]

On Wed, Nov 03, 2021 at 12:54:00PM +, Ian Tickle wrote:
> Suppose you had to compare two datasets differing only in their
> high-resolution cut-offs.  Now Rwork, Rfree and Rall will inevitably have
> higher values at the high d* end, which means that if you apply a cut-off
> at the high d* end all the overall R values will get smaller, so use of any
>  R value as a data-quality metric will tend to result in selection of the
> lowest-resolution dataset from your candidate set, which may well give a lower
> quality map: not what you want!

Couldn't that be avoided by using the common set of reflection data
present in both the A and B dataset (from David's example - or even
across a whole series A-Z)? When at the same time we keep the same
model parametrisation (i.e. not adding altConfs or waters, keeping
same TLS definitions etc) it might be useful to compare different data
processing procedures via the R{free,all,work} during refinement.

As far as I can see, the only difference would be the actual values of
F and sig(F) (or I and sig(I)) attached to the identical set of Miller
indices ... right?

Cheers

Clemens



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] what would be the best metric to asses the quality of a mtz file?

[Clemens Vonrhein]

On Thu, Oct 28, 2021 at 06:28:05PM +0530, Shipra Bijpuria wrote:
> I would first look at the dataset stats and define a resolution range
> mainly based on I/sigI >1 and cc1/2 >0.5. Based on this, would take the
> good resolution datasets only.

Some probably obvious word of caution here: these (quite sensible)
suggestions will depend hugely on factors like (1) binning, (2)
definition of a particular data quality metric and sometimes (3) the
treatment of Friedel pairs in computing these values. When comparing
seemingly identical metrics (as labelled) from different
programs/pipelines you have to be careful and aware of the various
differences in implementation. If one is sure that these values are
computed in the same way with the same binning, comparisons will be
possible, yes.

Also, a lot (most?) datasets are poorly described with a single
resolution value, even it it is rather convenient for sorting ;-)

Just thought to add this for the record for future generations of
ccp4bb-archive searching crystallographers ;-)

Clemens



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] what would be the best metric to asses the quality of a mtz file?

[Clemens Vonrhein]

Hi,

An alternative is to assess the quality of the phases (which are the
result of the model refined against the data) via radiation-damage
maps (what we call "F(early)-F(late) maps"). Assuming you collected
high enough multiplicity data, autoPROC (which is what you used for
processing, if I understood you right) would create those extra
amplitudes in the output MTZ files if possible. Refinement with BUSTER
would automatically compute and analyse those raditation damage maps
for "interesting" sites, like decarboxylation, Cys-SG/Met-SD damage
etc. You could then assume that the cleanest/strongest indications of
such radiation damage arise when using the best phases (i.e. best
model - as a result of best data to refine against).

For examples see e.g.

  https://www.globalphasing.com/buster/wiki/index.cgi?Covid19ReRefineRadDam
  https://www.globalphasing.com/buster/wiki/index.cgi?Covid19ReRefine5VZR

and also

  https://www.mail-archive.com/ccp4bb@jiscmail.ac.uk/msg49077.html

This would all be scriptable I guess.

Cheers

Clemens

PS: there are only really two files coming out of autoPROC, namely
truncate-unique.mtz (traditional, isotropic analysis) and
staraniso_alldata-unique.mtz (anisotropic analysis via STARANISO).


On Wed, Oct 27, 2021 at 12:58:53PM -0500, Murpholino Peligro wrote:
> So... how can I get a metric for noise in electron density maps?
> First thing that occurred to me
> open in coot and do validate->difference map peaks-> get number of peaks
> (is this scriptable?)
> or
> Second
> phenix.real_space_correlation detail=residue file.pdb file.mtz
> 
> 
> Thanks again
> 
> El mié, 27 de oct. de 2021 a la(s) 10:52, vincent Chaptal (
> vincent.chap...@ibcp.fr) escribió:
> 
> > Hi,
> >
> > It's hard to find a single metric...
> > Ultimately, the quality of electron density maps, lower noise in fo-fc?
> >
> > Best
> > Vincent
> >
> > Le 27/10/2021 à 17:44, Murpholino Peligro a écrit :
> >
> > Let's say I ran autoproc with different combinations of options for a
> > specific dataset, producing dozens of different (but not so different) mtz
> > files...
> > Then I ran phenix.refine with the same options for the same structure but
> > with all my mtz zoo
> > What would be the best metric to say "hey this combo works the best!"?
> > R-free?
> > Thanks
> >
> > M. Peligro
> >
> > --
> >
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >
> >
> > --
> >
> > Vincent Chaptal, PhD
> >
> > Director of GdR APPICOM
> >
> > Drug Resistance and Membrane Proteins Lab
> >
> >
> > MMSB -UMR5086
> >
> > 7 passage du Vercors
> >
> > 69007 LYON
> >
> > FRANCE
> >
> > +33 4 37 65 29 01
> >
> > http://www.appicom.cnrs.fr
> >
> > http://mmsb.cnrs.fr/en/
> >
> >
> >
> > --
> >
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> >
> 
> ########
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Selfrotation problem for the experts

[Clemens Vonrhein]

On top of that very good explanation by Zbyszek, I find it also
interesting that

  * in the tetragonal dataset the Dn symmetry indications are exactly
on the diagonal (45 degree) with the peaks a bit smeared out

  * in the orthorhombic dataset the Dn symmetry 2-folds are not on the
diagonal, but rather at 55 degree - with much clearer peaks.

You didn't give us the cell dimensions, but I wonder what the relation
between those two crystal forms is - hopefully the tetragonal is truly
different and it's not a case of pseudo-tetragonal orthorhombic (with
processing sometimes going for one or the other SG).

Cheers

Clemens

On Wed, Jul 28, 2021 at 04:37:22PM -0500, zbyszek wrote:
> It's a very interesting crystallographic and likely biological problem.
> 
> These patterns--a series of peaks along the axes of the Chi=180 plot, with
> the peaks corresponding to a great circle in polar (3D) space--indicate 2
> back-to-back rings.  On each great circle (each line of peaks on the Chi=180
> plot) there are 16 distinct peaks, so you have likely D16 molecular symmetry
> i.e. you have two 16-mers forming a two-ring structure.
> 
> This indicates that ASU has megadalton complex inside:
> - in P2221 where only one crystallographic 2-fold can be used as axis of D16
> assembly there will be likely 16 molecules in ASU.
> - in tetragonal form, if this P42 21 2 or P4 21 2, two crystallographic
> 2-folds can coincide with 2-folds of D16 assembly so there will be 8
> molecules in ASU. I would expect for such symmetry (two perpendicular
> assemblies, each 32 sub-units, in the unit cell) to result in very high
> solvent content e.g. 75-80%.
> 
> If in a tetragonal space group you get ~70-80% of solvent for 8 molecules in
> ASU, then there is a very high chance that the entire assembly sits on two
> perpendicular 2-fold axes and 8 NCS operators are determined by the
> orientation of these axes. One of the 2-folds will have to coincide with 42
> screw axis or 4-fold axis, while the second 2-fold is the diagonal along ab
> directions.
> 
> In both cases, the NCS operators will be represented by rotations along
> diagonal 2-fold crystallographic axis by angles 0, 22.5, 45, 67.5, 90,
> 112.5, 135, 157.5. These axes will intersect at the origin of
> crystallographic system in both space groups.
> 
> If you don't have such situation, your best bet is cryoEM. It should work
> for such big assembly.
> 
> Zbyszek
> 
> 
> 
> 
> On 2021-07-28 13:48, Jorg Stetefeld wrote:
> > Hi,
> > 
> >  we are seeking advice interpreting the attached selfrotation patterns
> > in terms of defining the NCS symmetry and eventually NCS-masks for
> > proper averaging/DM.
> > 
> >  We are working with a muliti-domain transmembrane receptor composed
> > of several Ig -and FN-domains. We collected data in P4x212 and in
> > P2221 (see attachments).
> > 
> >  Thx in advance
> >  js
> > 
> > __
> > 
> >  [1] [2] ​
> > 
> >  Jörg Stetefeld
> > 
> > https://stetefeldlab.ca/ [2]
> > 
> >  Find a job you like- You never work a day in your life!
> > 
> >  [1] [2]
> > 
> > -
> > 
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> > 
> > Links:
> > --
> > [1] https://stetefeldlab.c
> > [2] https://stetefeldlab.ca/
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Negative electron density

[Clemens Vonrhein]

Just for completeness: BUSTER includes an automatic so-called "void
correction" that tries to detect those incorrectly bulk-solvent filled
cavities and correct for it. Nothing for you to do extra, since that
correction is on by default ... and works as expected in a lot of cases
as far as we can tell. And yes, it is also approaching this in a "all
or nothing" manner, so not very subtle.

Cheers

Clemens

PS: see also

  https://www.globalphasing.com/buster/
  https://www.globalphasing.com/buster/wiki/index.cgi?BusterFAQ#abruptRchange
  
https://www.ccp4.ac.uk/schools/DLS-2019/course_material/day7/Vonrhein_GlobalPhasing_BUSTER_CCP4-DLS2019.pdf


On Sun, Jul 18, 2021 at 10:57:59AM -0700, James Holton wrote:
> There are two ways to test the "too much bulk solvent" hypothesis.
> 
> 1) examine the bulk solvent electron density map.
>   This is (IMHO) a lot harder than it should be, but here are two recipes:
>   refmac5 : on the command line, put "mskout rawmask.map" somewhere among
> the hklin xyzin, etc directives.  This will create a map you can load into
> coot with the "File:Open map..." menu item.
>   phenix: use phenix.fmodel twice.  Once with k_sol=1, and again with
> k_sol=0.  Load the two resulting mtz files into coot with "File:Open MTZ,
> mmCIF, fcf or phs..." and then create a difference map between them using
> "Calculate:Map Tools...:Make a Difference Map...".  This difference map will
> be the bulk solvent component.
> 
> 2) suppress the bulk solvent.
> The easiest way to do this is fill the region with water atoms at low
> occupancy, such as 0.01.  These will contribute essentially no electron
> density of their own, but will also prevent the refinement program from
> putting bulk solvent anywhere near them.  At least with current algorithms,
> atoms at any occupancy exclude 100% of bulk solvent.  Since they will be
> poorly restrained, you might want to fix these waters in place with harmonic
> restraints, or otherwise tell your favorite refinement program to not move
> them.  Your documentation may vary.

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Problem with peakmax

[Clemens Vonrhein]

On Thu, Jun 10, 2021 at 08:19:01AM -0700, James Holton wrote:
> I'm afraid the only bash command I know is: "tcsh"

I'm using the exact opposite direction ;-)



  To everyone using "echo" (interactively and/or in scripts): be careful
  which command is picked up at any given point. There are shell
  built-in versions of "echo" and then there is /bin/echo. Some
  recognize "\n" and some don't. Some support "-e" flag and some
  don't. A better command might be

printf "THREshold RMS 5.0\n" | peakmax MAPIN "myfile.ccp4" XYZOUT 
"myfiles_omit_atom.pdb"

  or (if no cards are to be used)

peakmax MAPIN "myfile.ccp4" XYZOUT "myfiles_omit_atom.pdb" < /dev/null

  or

printf "\n" | peakmax MAPIN "myfile.ccp4" XYZOUT "myfiles_omit_atom.pdb"

  (not all CCP4 programs support the "END" card").

  If you want to write instantly portable scripts, it might be best to
  have "#!/bin/sh": not all systems will have /bin/bash, /bin/csh or
  /bin/tcsh installed. Of course, /bin/sh might point to /bin/bash, but
  it could also point to /bin/dash (another variant) ... or something
  else entirely (MacOS is known to replace "standard" Linux/UNIX
  commands from time to time, often going from GNU versions to BSD based
  ones - or something else again). With /bin/sh you /should/ be within a
  POSIX sh environment: maybe more limited to other shells, but
  standardized and therefore much more portable since every UNIX-like
  system so far will always have /bin/sh available.



Cheers

Clemens

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Question about tncs and SAD phasing

[Clemens Vonrhein]

CP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/



-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Anomalous signal to noise details

[Clemens Vonrhein]

Dear David,

On Fri, Dec 18, 2020 at 11:53:08AM +, David Waterman wrote:
> The paper "Substructure solution with SHELXD
> "
> (Schneider & Sheldrick, 2002) describes how
> 
> data can be truncated at the resolution at which [ΔF to its estimated
> > standard deviation as a function of the resolution] drops to below about 1.3
> 
> 
> Is this referring to the quantity <|ΔF|>/<σ(ΔF)> calculated in resolution
> shells, or the quantity <|ΔF|/σ(ΔF)> ?

I'm nearly 100% sure this refers to the latter - or at least: the
latter is the only one making sense to me. This sounds very much like
the confusion when it comes to

   (1)

==> PDBx/mmCIF: _reflns.pdbx_netI_over_sigmaI   73.6  % of entries
_reflns_shell.pdbx_netI_over_sigmaI_all  0.001% of entries
_reflns_shell.pdbx_netI_over_sigmaI_obs  2.6  % of entries

versus

  / (2)

==> PDBx/mmCIF: _reflns.pdbx_netI_over_av_sigmaI 2.6  % of entries
_reflns_shell.meanI_over_sigI_all0.2  % of entries
_reflns_shell.meanI_over_sigI_obs   53.0  % of entries

As far as I can remember, we always computed and reported (1) and
never (2) - at least when it comes to the scaling/merging programs I'm
familiar with (SCALE, XDS/XSCALE, AIMLESS, d*TREK). What useful
information would (2) or <|ΔF|>/<σ(ΔF)> convey anyway ... ?

If we were to believe these definitions, then we are storing the
"right/useful" value  in the overall statistics, but a very
different value of / in the per-shell statistics. All those
_reflns_shell.meanI_over_sigI_obs are most like mis-labeled (1)
quantities.

> This entry
> 
> on the ccp4wiki gives a cutoff
> 
> where the mean value of |ΔF|/σ(ΔF) falls below about 1.2 (a value of 0.8
> > would indicate pure noise)
> 
> 
> this version sounds to me like <|ΔF|/σ(ΔF)>
> 
> which is the "better" metric, and what do people mean when they say
> DANO/SIGDANO? What is the justification for the 1.3 (or 1.2) value?

I think everyone always refers to <|ΔF|/σ(ΔF)> no matter what it is
called (sometimes programmers shorten the notation to avoid unwieldly
wide columns).

I tend to look for values above 1 (and the higher, the better) - but
maybe even more importantly: check the trend with resolution (higher
at low resolution), maybe in comparison with expectations (type of
scatterer, fluorescence scan, anomalous signal, number of sites,
potential B-factors of scatteres etc).

Cheers

Clemens



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] [EXTERNAL] Re: [ccp4bb] phenix.refine with ligand with ambiguous electron density

[Clemens Vonrhein]

Dear all,

coming back to those map questions again:

On Wed, Nov 25, 2020 at 12:56:16PM +, Mooers, Blaine H.M. (HSC) wrote:
> The question is about a Fo-Fc map and the replies have been focused on such 
> maps.
> However, I question why you are bothering with Fo-Fc maps.
> The ligand was soaked into the crystal. There is probably an isomorphous apo 
> data set available. 
> A [Fo(ligand complex) - Fo(apo)]*exp(alpha_apo,calc) map should be consulted 
> in 
> preference to various Fo-Fc maps corrected for phase bias.
> It would be even better to substitute in the experimental phases for the apo 
> structure if
> they are available.
> 
> Fo-Fo maps may be nosier than Fo-Fc maps. but they are more reliable when you 
> are
> trying to decide if a ligand is present. 

Generally I would agree, but with a slight caveat: the accurate
scaling of those two sets of Fo can become tricky. If the Fo(apo) was
collected in a very different way (different beamline, different
detector, different protocol, different processing package, re-using a
PDB deposition from 10 years ago etc) compared to Fo(soak), things can
become difficult: the required scaling of noisy measurements to extract
the very small ligand signal has to be very accurate.

So when doing high- or low-throughput ligand screening data
collection, maybe we shoulld always collect another APO dataset during
teh same beamline shift? Of course, this then start to play the same
role as the "ground-state" map in PanDDA (if I understood this
correctly) ... but a very isomorphous (in terms of experiment,
hardware, processing) experimental APO dataset seems like a good idea
nevertheless.

> I second Robbie's "bloody obvious" rule. 

Ditto.

I like to start with the null hypothesis that the experiment (soaking
a crystal with a compound solution) has not worked (ligand has not
bound). That seems a better "bias" than the other way round ("I soaked
therefore it is there.").

Our ligand-detection maps approach three questions in turn - each
depending on a positive outcome of the previous:

  (1) Has something bound (=> detection of residual difference density
  under the null hypothesis)?

  (2) Where has it bound (=> detection of what we call "interesting
  regions")?

  (3) How has it bound (=> removal or potentially wrong dummy
  water/atoms to result in shape description of binding excluding
  bulk-solvent)

So instead of zooming in on our favourite binding site (which fits so
nicely that beautiful theory I have) and immediately dialing map
levels down or playing with B-factor sharpening, it forces us to take
several steps back from any potential bias I think. Note that map
levels and B-factor sharpening/blurring are very useful tools that can
become crucial in the correct interpretation of compound density, but
they tend to mainly modify shape and levels while easily pushing the
initial question ("Has this small compound bound?") into the background.

Anyway, just another minor thoughts ...

Cheers

Clemens

 https://www.globalphasing.com/buster/wiki/index.cgi?DifferenceFourierMaps#iso
 https://www.globalphasing.com/buster/wiki/index.cgi?LigandDetectionModes



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] phenix.refine with ligand with ambiguous electron density

[Clemens Vonrhein]

Dear Nika,

as a possible alternative or second opinion, you could have a look at
the ligand-detection modes [1] in BUSTER too - see e.g.

  https://www.globalphasing.com/buster/wiki/index.cgi?LigandDetectionModes

It's very similar to the Phenix Polder maps - so might not tell you
anything different, but looking at results from different
implementations can be useful as a check.

Anyway, one of the main problems handling partially occupied ligands
(and let's ignore the possibility of partial disorder for
"simplicity") is that you might have a mixture of three models within
your binding site: (1) compound, (2) waters (at ordered positions when
the ligand is not bound) and (3) bulk solvent (at disordered positions
when the ligand is not bound).

What can be useful is to include at least the alternative ordered
water model (from a well refined APO model?) and then refine the
(grouped) occupancies:

  OCC(compound) + OCC(waters) = 1.0

This way you give the refinement the option to chose between two
alternative interpretations [2]. To avoid any bias in that refinement
procedure, you could start from two extremes: once with
OCC(compound)=0.9 and once with 0.1. If the OCC(compound) refines to a
small value (say <0.2 or such) in both cases, it is possibly not really
there.

Cheers

Clemens

[1] Vonrhein, C. and Bricogne, G., 2005. Automated structure
refinement for high‐throughput ligand detection with
BUSTER‐TNT. Acta Crystallogr. Sect. A, 61, p.c248.
[2] 
http://www.globalphasing.com/pipermail/buster-discuss/2015-August/000255.html

On Tue, Nov 24, 2020 at 11:28:42AM +, Nika Žibrat wrote:
> Hello,
> 
> 
> I have a question about protein-ligand, of which ligand displays an ambiguous 
> electron density. I am solving a structure of protein with ligand  which was 
> obtained via soaking. Structural characteristics indicate the ligand is 
> present however the electron density is quite vague and too small for the 
> size of the whole ligand. I did a Polder map which showed much larger area of 
> green density. After insertion of my ligand into the green density in Polder 
> I ran phenix.refine and there is a lot of red on the spot where the ligand is 
> which was to be expected. This leaves me wondering how, if even do I 
> incorporate the polder map data into my refine input.
> 
> 
> My question is, how do I continue refining and validating the structure in 
> this case?
> 
> 
> Thank you,
> 
> 
> Nika Žibrat
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Quote source inquiry

[Clemens Vonrhein]

Hi Robbie,

On Wed, Jul 15, 2020 at 07:23:15PM +, Robbie Joosten wrote:
> At the same time if you have a a more relaxed approach to restraints
> than you might find systematic deviations in bond lengths. A test
> for that has been in WHAT_CHECK for decades and it actually works
> surprisingly well to detect cell dimension problems.

Indeed.

> That said, the problem is uncommon now.

Not so sure about that: we all rely on an accurate value of the
energy/wavelength from the instrument/beamline - and if that is off
(for whatever reasons) it will result in incorrect cell dimensions and
a systematic deviation from the various restraints.

This would even affect the best experiment done on the best crystal
... so fairly easy to spot at the refinement stage, especially if such
an energy/wavelength offset is constant over a long period of time on
a given instrument. To spot this at the data collection stage one
would hope that at some point a crystal with very pronounced ice-rings
will be looked at properly (and the fact these are not where we expect
them to should cause some head-scratching).

Cheers

Clemens



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Completeness question

[Clemens Vonrhein]

Dear Robbie,

On Sat, May 30, 2020 at 08:36:06AM +, Robbie Joosten wrote:
> I've been looking at some recent PDB entries that have much lower
> spherical) completeness than reported in the coordinate file. One
> reason for this is that the data were anisotropicly truncated,
> another reason is some mess-up with the deposition of the reflection
> data. There is a lot of discussion about the former practice and I
> don't want to go in to that, but the second one is obviously an
> error. Now how do I distinguish these cases?
> 
> Sometimes, you can look at the reported number of reflections and
> compare that to the deposited reflection file and you will find that
> something has clearly gone wrong. However, the reported number of
> reflections is not entirely reliable because of other issues so I'd
> rather not use it. If you use PDBpeep (e.g. for 6rjy) you can see
> something is wrong, but that is completely visual. Is there a tool
> in CCP4 that reports both spherical and ellipsoidal completeness (on
> merged reflection data)? That would make it easy to distinguish such
> cases.

One needs a description of the anisotropy in a form that allows for a
simple computation ("Is a reflection above a significance limit or
not?") to relate this to the completeness of reflections above such a
significance level. That model could e.g. be the ellipsoid fitted to
the significance (local ) cutoff surface as done by STARANISO
[1]. You can see that information e.g. at the PDBpeep server [2]:

  http://staraniso.globalphasing.org/cgi-bin/PDBpeep.cgi?ID=6rjy

as

  Diffraction limits & principal axes of ellipsoid fitted to diffraction 
cut-off surface:

  1.747 1.   0.   0.   a*
  1.777 0.   1.   0.   b*
  1.465 0.   0.   1.   c*

Of course, if this information is already present in deposited PDB
entries of interest, no additional computation is required. We are
currently working with our colleagues from the PDBx/mmCIF working
group on definitions and items that would extend the current
PDBx/mmCIF dictionary so that the above information could be deposited
to and retrieved from the PDB database.

In any case, this information is then used for computation of the
spherical and ellipsoidal completness e.g. within STARANISO - see link to
the "complete log file" at the bottom of the page:
http://staraniso.globalphasing.org/PDB/6rjy.log. It contains a table
("Statistics for isotropic, anisotropic & ellipsoidal diffraction
cut-offs") where Cmeas_sph and Cmeas_ell should give the above
information (showing clearly that for 6rjy something else went wrong
during deposition).

If there is unmerged data deposited, one could give that ellipsoid
description e.g. to MRFANA [3] via

  mrfana -ell 1.747 1.0 0.0 0.0 1.777 0.0 1.0 0.0 1.465 0.0 0.0 1.0 unmerged.mtz

to get the same information (but maybe use different binning, limit
resolution ranges etc).

Cheers

Clemens, Claus, Ian & Gerard

[1] http://staraniso.globalphasing.org/
[2] http://staraniso.globalphasing.org/cgi-bin/PDBpeep.cgi
[3] https://github.com/githubgphl/MRFANA, a result of the Data Quality
Metrics Workshop organised by Global Phasing at the ESRF in April
2019



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Completeness question

[Clemens Vonrhein]

Dear all,

On Mon, Jun 01, 2020 at 11:48:41PM +0200, vincent Chaptal wrote:
> since I've been contacted off line several times about this, I'm
> posting here the protocol to combine maps with 2 mtz files of
> original and truncated data.

If this is of wider interest (and since some of our tools for doing
this were mentioned), maybe we can also give a bit of additional
background information and provide some outlook.

> I went through this procedure as I wanted to include the maps, and
> this was the tricky part of the procedure in my hands. This way,
> everyone can look at the same maps when critically assessing a
> structure (especially in the context of redo-ing and severe
> anisotropy).

I think it is a very good idea to also include into the deposition the
maps that were used/seen during model building/refinement. After all,
these were the basis for the interpretation that produced the final
model and the conclusions drawn fom it in a publication. Of course,
one can always re-create similar maps through some external
(re-)refinement approach (be that PDB-REDO or manually using your own
favourite program). Those maps might be similar, better, worse or just
different, perhaps providing added value and additional information -
but it can be difficult to get the exact same maps as seen by the
depositor, due e.g. to different program versions, or to not using the
exact commands and/or parameter settings etc.

Then there is the question of the handling of missing data in 2mFo-DFc
maps (ignoring it, filling it with DFc up to the highest spherical
diffraction limit, or filling it only within the ellipsoidal model of
anisotropy given by STARANISO). BUSTER will create a single mmCIF file
containing all those different map coefficients in separate data
loops. Other programs are most likely to be doing the same. The
important point is that current refinement programs should generate a
single mmCIF file containing all reflection data (and meta-data)
available at the point of refinement - something we are all working
hard at within the PDBx/mmCIF working group. So the task (as described
below) is then to combine this set of reflection data (all nicely
self-contained from model refinement) with the full set of reflection
data (and meta data) from data processing. There are various caveats
and traps one has to consider here (different indexing possibilities,
different cell settings and conventions, enantiomorphs, initially
unassigned screw axes, C2 vs. I2, H3 vs. R3, etc).

Ideally we want an application that takes the two (self-contained)
mmCIF files from data-processing and model refinement, and combines
them in a consistent way, robustly taking care of all of the above
potential issues.  Because there often is a large time-gap between
original data-processing and final structure deposition, a lot of
additional checks need to be in place to avoid picking up the wrong
files by accident. This is precisely what we are currently working on
and hope to provide in one of our next releases: a tool working
ideally directly on mmCIF files produced by any data processing
package and any refinement program, that will combine these in a
consistent and validated manner. We hope that will become useful - not
only to users of autoPROC/STARANISO and BUSTER, but also to people who
use multiple different packages in their workflow.

Cheers

Clemens

> In a new directory, put all 3 mtz files (original Intentisties, merged
> staraniso reflections, maps from buster)
> 
> /PATH_TO-DIR/BUSTER/Refine45/autoPROC/.
> 
> aP_deposition_prep -p combine -r maps.mtz -f staraniso_output.mtz -a
> Original_Intensities.mtz >aP_deposition_prep.log
> 
> It creates combine_all.cif
> 
> This file has everything but it has a “refln” naming issue.
> 
> To fix the “refln” issue:
> 
> cat combine_all.cif | sed -e "s/refln.F_sigma /refln.F_meas_sigma /g" -e
> "s%[ ]* #.*from dataset.*%%g" | awk 
> '/^#/{if(s){print;next};c++;a[c]=$0;next}/^data/{print;if(s)next;s=1;for(i=1;i<=c;i++){print
> a[i]}next}{print}' > combine_all_refln_fixed.cif
> 
>  DONE.
> 
> 
> Best
> Vincent
> ps: again, the credit of this script goes to the globalphasing team who
> kindly helped when I asked them for help with it.
> 
> 
> 
> Le 01/06/2020 à 11:47, Clemens Vonrhein a écrit :
> > Dear all,
> > 
> > On Sat, May 30, 2020 at 03:40:53PM +0100, Eleanor Dodson wrote:
> > > My pennysworth. If you find your maps look better after the
> > > anisotroy correction use it, but it may be helpful to those wo want to 
> > > mine
> > > your data if you deposit the whole sphere..
> > Agree (which is what e.g. we provide when using STARANISO via autoPROC
> > [1]).
> > 
> > And in the same vein: those depositing isotropically truncated data
> > should consider also providi

Re: [ccp4bb] Completeness question

[Clemens Vonrhein]

Dear all,

On Sat, May 30, 2020 at 03:40:53PM +0100, Eleanor Dodson wrote:
> My pennysworth. If you find your maps look better after the
> anisotroy correction use it, but it may be helpful to those wo want to mine
> your data if you deposit the whole sphere..

Agree (which is what e.g. we provide when using STARANISO via autoPROC
[1]).

And in the same vein: those depositing isotropically truncated data
should consider also providing data to a higher diffraction limit to
give a potentially more accurate picture (if there is even a slight
indication of anisotropy - which there often is).

I find it very helpful even looking at an idealised (and therefore
simplified) picture of anisotropy as in

  http://staraniso.globalphasing.org/anisotropy_about.html

We can consider

 (1) for refinement:

 (1a) green+red, i.e. spherical (i.e. isotropically) truncated
  data

 (1b) green+blue, i.e. anisotropycally truncated data

 (2) for deposition:

 (2a) green+red => full sphere, but dropping real observations
  (blue)

 (2b) green+blue => all observations, but not providing
  insignificant/weak data (red) in all directions

 (2c) a sphere to the "tip" of blue (i.e. anisotropic diffraction
  limit) => all observations and all insignificant/weak data

Cheers

Clemens

[1] https://www.globalphasing.com/autoproc/ - which gives a mmCIF file
with (2a), (2b) and (2c) ready for deposition.


> eleanor
> 
> On Sat, 30 May 2020 at 09:36, Robbie Joosten 
> wrote:
> 
> > Hi Everyone,
> >
> > I've been looking at some recent PDB entries that have much lower
> > spherical) completeness than reported in the coordinate file. One reason
> > for this is that the data were anisotropicly truncated, another reason is
> > some mess-up with the deposition of the reflection data. There is a lot of
> > discussion about the former practice and I don't want to go in to that, but
> > the second one is obviously an error. Now how do I distinguish these cases?
> >
> > Sometimes, you can look at the reported number of reflections and compare
> > that to the deposited reflection file and you will find that something has
> > clearly gone wrong. However, the reported number of reflections is not
> > entirely reliable because of other issues so I'd rather not use it. If you
> > use PDBpeep (e.g. for 6rjy) you can see something is wrong, but that is
> > completely visual. Is there a tool in CCP4 that reports both spherical and
> > ellipsoidal completeness (on merged reflection data)? That would make it
> > easy to distinguish such cases.
> >
> > Cheers,
> > Robbie
> >
> > 
> >
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> >
> > This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a
> > mailing list hosted by www.jiscmail.ac.uk, terms & conditions are
> > available at https://www.jiscmail.ac.uk/policyandsecurity/
> >
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] neg density/high B on sidechains

[Clemens Vonrhein]

Dear Len,

maybe a slight side-track, but you could consider taking one step back
and applying a slightly different approach to data-processing -
something we call "F(early)-F(late)" maps in autoPROC [1].

Why could that be useful here? Because it might allow you to

 (1) differentiate between radiation damage (atoms present at the
 beginning but "lost" or "somewhere else" at the end - which could
 be modelled through occupancy refinement) and genuine disorder
 (static throughout the experiment - probably best described
 through large B-factros);

 (2) generate and use a dataset of the least damaged subset of
 reflections to refine against;

 Note: this is different from just processing a subset of your
 images, since all data is still scaled together but merged
 separately.

Of course, this all assumes that data were collected on a modern
detector/beamline using a low-dose, high-multiplicity (and
fine-sliced) strategy - but that is nowadays usually the case anyway
due to fast detectors and shutterless collection (and because it
allows for more flexibility and better data in the end).

There are other approaches to this (e.g. zero-dose extrapolation [2]
or RABDAM [3]), but if you have a set of diffraction images sitting on
disk it might be easy to just push them through our software.

If you want to have a look at some recent examples (of what you might
be able to learn about your data, collection strategy or model
refinement) also related to radiation damage:

  https://www.globalphasing.com/autoproc/wiki/index.cgi?Covid19
  https://www.globalphasing.com/buster/wiki/index.cgi?Covid19

Cheers

Clemens

[1] https://www.globalphasing.com/autoproc/
[2] Diederichs et al (2003). Acta D 59,903-909.
[3] Shelley et al (2018). J Appl. Cryst 51, 552-559.

On Tue, Apr 28, 2020 at 03:37:20PM +, Thomas, Leonard M. wrote:
> Hello all,
> 
> This is one of those issues that seems to come up now and then.  I have been 
> working on a structure that obviously has some radiation damage as indicted 
> by negative density and/or high thermal parameters.  Since we know that 
> residue X is in the sequence the sidechain should be there and is just 
> flopping around or has been damaged/removed by radiation exposure.  My 
> questions is what is the current thinking on presenting these residues for 
> deposition.  Remove the side chain atoms, drop the occupancy to zero, just 
> let them  behave as they want ie high B factors some negative density.
> 
> Cheers,
> Len
> 
> Leonard Thomas
> lmtho...@ou.edu<mailto:lmtho...@ou.edu>
> 
> 
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Raw diffraction images for SARS-CoV-2 related structures

[Clemens Vonrhein]

Dear all,

as a follow-up to our initial message (and similar to other activities
from the MX community), we added some content to our BUSTER wiki at

  https://www.globalphasing.com/buster/wiki/index.cgi?Covid19

to show interested users how to (re-)refine existing PDB entries with
BUSTER and how to look at the results. For anyone wanting to
(re-)process deposited raw diffraction data, we've started a page on
the autoPROC wiki at

  https://www.globalphasing.com/autoproc/wiki/index.cgi?RevisitingExistingData

that might be useful.

We will keep adding and updating those pages over the next days,
especially if more raw diffraction data becomes available and/or more
PDB structures are deposited or becoming relevant. We are very
grateful to any researcher making their (raw) data available under
such pressure and stress!

If there are any questions, suggestions, requests or ideas: please let
us know.

Best regards

Clemens & Gerard



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] CCP4BB vs COVID19

[Clemens Vonrhein]

Dear James,

On Fri, Mar 20, 2020 at 03:59:01PM -0700, James Holton wrote:
> ORF8 has only one homolog in the PDB: 5o32 with 25% identity over a stretch
> of 60 residues.  This homologous region contains the S84L site (Val I544 in
> 5o32).  I had a quick look and appears to be a cavity-filling mutation to
> me.  Not very big, but maybe something could fit in there.  To be sure we'd
> need a structure of ORF8.

This is a very good example of why we all are trying to get raw
diffraction data deposited as soon as possible to the publication and
deposition of a structure. It highlights the potential benefit of
revisting some important structures right at the level of the raw
data. The paper says

  Diffraction data were collected at European Synchrotron Radiation
  Facility (ESRF) and Swiss Light Source at Paul Scherrer Institute
  (SLS-PSI). Diffraction data were processed by XDS and
  AIMLESS. Crystals of C3b–miniFH–FI exhibited space group P1 and
  diffracted to a resolution of 3.8 Å. However, the diffraction data
  was highly anisotropic. The data were therefore truncated to 4.2-Å
  resolution and rescaled using the Diffraction Anisotropy Server.

Obviously, that was state-of-the-art in 2015 (data collection) and
2017 (publication) - but maybe new methods could indeed do better on
this already existing structure if the raw data became
available. Maybe nothing will change in terms of structure quality and
(most importantly) structure interpretation ... who knows.

We've contacted the authors - who were very quick and extremely
helpful in providing us with raw images for some other deposited
project from their lab a few weeks ago, so expect and hope for a
similar response when/if the data can be located.

> Good luck to you all, and stay healthy.

Ditto!

Cheers

Clemens

-- 

*----------
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Raw diffraction images for SARS-CoV-2 related structures

[Clemens Vonrhein]

Dear Julien,

On Thu, Mar 19, 2020 at 08:47:20AM +, Julien Cappèle wrote:
> Though I agree with you Clemens that raw images are amazing to work
> with as you can use any software you are confortable with, we cannot
> forget that depositing several TB of data for each lab would be bad
> for ecological reason.

Of course, there are ecological (carbon footprint) considerations -
and there are lots of papers and studies about that. I haven't looked
at any numbers, but maybe some points:

 * A lot of data is already stored (e.g. at synchrotrons) and would
   "only" needed to be made "visible" via a DOI (caveat: I realise
   that there are huge technical issues with that)

 * How does that energy consumption compare with the energy used to
   perform the experiment in the first place?

 * If by having that data available we can improve software and the
   way experiments are done: wouldn't that potentially save energy in
   hte long run (avoiding poor or unnecessary experiments in the first
   place)?

 * We are looking at a move to increase the number of raw image data
   depositions for deposited PDB structures - not at a requirement to
   deposit raw images for every PDB structure or even for every
   dataset ever collected.

   At the moment there are about 4500 image datasets available for
   about 10 PDB X-Ray structures, i.e. ~5%. 

> And because detectors are always improving (thank you all!), size of
> data will increase exponentially.

True ... and some type of experiment can benefit from those larger,
faster and more numerous types of datasets - if done correctly.

> Could it be possible for a new/already existing software to store
> reflections (area, intensity from center to border, position x/y on
> the image, and information of the image) in a lightweight and text
> only file ? Possibly a new format to be used for integration ?

See my other reply: this all assumes that the initial processing step
caught all spots (and nothing else) on the 2D image correctly.

There have been all kind of initiatives about raw data deposition (in
no particular order)

  https://www.iucr.org/resources/data/dddwg
  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5331468/
  https://www.sciencedaily.com/releases/2016/11/161108130045.htm
  https://journals.iucr.org/d/issues/2016/11/00/yt5099/
  https://onlinelibrary.wiley.com/iucr/doi/10.1107/S0909049513020724
  http://scripts.iucr.org/cgi-bin/paper?S0907444908015540
  https://scripts.iucr.org/cgi-bin/paper?dz5309
  https://bl831.als.lbl.gov/~jamesh/lossy_compression/

So we've been there before. Let's see if we can't do at least
something for the clearly important structures and work right now -
and worry about some long-term impact later (having maybe learned
something along the way). Just because we could be doing something now
doesn't mean we will have to keep doing this in a 1-N years time,
right ;-)

Cheers

Clemens



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Raw diffraction images for SARS-CoV-2 related structures

[Clemens Vonrhein]

Dear Petr,

On Thu, Mar 19, 2020 at 06:45:31AM +, Petr Kolenko wrote:
> For those of you who are in touch with these data, would deposition
> of unmerged intensities in P1 in the whole detector range instead of
> complete dataset be good enough to „correctly“ re-evaluete the
> structure? Or is there a doubt that even this approach would not
> lead to optimal structure?

It would be a step in the right direction, yes ... but:

 * It assumes that all spots have been integrated and nothing else
   than those spots: there can be cases of missed unit cell
   duplications or incorrect enforcement of a unit cell and symmetry
   (because those crystals always come in a known cell/SG, right?).

 * We'll still be stuck with a particular intergration program and a
   particular set of user decisions how to run it. Yes, all modern
   data processing programs and packages are usually doing a great job
   if run in (more or less) default mode on (more or less) good
   crystals. But not everything is lysozyme/thaumatin/XYZ collected
   fine-sliced on a PAD with low-dose, high-multiplicity on a well
   calibrated instrument ;-)

 * If integration was done in P1 the cell parameter restraints of the
   (potentiually different) true SG were not enforced - which can lead
   to incorrect cell parameters (which has knock-on effects in
   refinement ... remember the WhatIf check for correct cell
   parameter?).

   If integration was done in a non-P1 SG and that choice was
   incorrect: how do we know that those cell parameters didn't enforce
   an incorrect set of cell parameter restraints, leading to poor
   integration? Again: different programs have different ways of
   assigning pixels (and their counts) to a reflection intensity
   (concepts like shoebox, profile-fitting, pixel-labeling etc all
   come to mind).

 * Integration itself can be suboptimal if there are issues with the
   crystal (ice rings, split, multiple lattices etc), the instrument
   (damaged pixels, beam jitter, incorrect countrate cut-off handling
   etc) or the experiment (radiation damage etc). Data processing
   defaults might not always work correctly/best for all cases.

 * The raw images allow such corrections (damaged pixels not yet in
   pixel mask, incorrect handling of countrate_cutoff) and additional
   analysis (e.g. radiation damage analysis a la F(early)-F(late) maps
   [1]).

So the raw integrated (not scaled/corrected) and unmerged intensities
might indeed often be good enough, but if we are going to try and
deposit more than just the merged and scaled intensities/amplitudes
(as we do now) - and this will involve additional work and change from
our side - then why not bite the bullet once and try to go all the way
to raw images:

  * The infrastructure is already in place (proteindiffraction.org,
data.sbgrid.org, zenodo.org, wwPDB DOIs, etc), ready for use right
now.

  * From experience at multiple worshops: everyone is comfortable to
start data processing from raw images using the various software
tools out there that people are familiar with (while jumping in
half way is probably more challenging to non power-users).

Anyway, just some ideas - from someone downloading nearly all deposited
raw image data on a regular basis for testing and developing data
processing software (and finding a lot of issues in our software this
way ... as well as issues with experiments and sometimes the
instrument).

Cheers

Clemens

[1] as e.g. done in autoPROC+BUSTER



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Raw diffraction images for SARS-CoV-2 related structures

[Clemens Vonrhein]

Hi Andreas,

On Thu, Mar 19, 2020 at 07:06:32AM +0100, Andreas Förster wrote:
> - The link to the raw data can be found inside the mmCIF file under
> _pdbx_related_exp_data_set.data_reference.  Whether it's also shown on one
> of the PDB sites is up to the designers of these sides.  I cannot find a
> link to the experimental data on rcsb.org.

They are available near the bottom of a rcsb.org page ("Experimental
Data & Validation" section) - see e.g. 6VS4 page

  http://www.rcsb.org/structure/6VS4

(the newest deposited PDB structure with deposited data recorded).

Cheers

Clemens

PS: if anyone is in need of a simple script for finding and
downloading such PDB entries, please let me know.



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] problem in structure solution of multidomain protein

[Clemens Vonrhein]

Hi,

MOLREP [1] has a nice feature of searching for structures in electron
density - with the known parts of your model fixed. Basically follow
the recipe in [1] (search for "refmac.mtz" - but you can use any other
set of amplitudes/phases as well). In your case the 4-domain model
would be given to the -mx flag and the 5th domain search model to -m.

This worked for me a few times in the past ...

Cheers

Clemens

[1] http://www.ccp4.ac.uk/html/molrep.html

On Wed, Feb 12, 2020 at 07:12:48AM +, Rajnesh Kumari Yadav wrote:
> Hello everyone,
> 
> I am working on a protein which have 5 domain in it, we have its 4 domain 
> structure and 3.0 Angstrom data of 5 domain protein crystal. While doing the 
> Molecular replacement we got solution with four domain with good electron 
> density and space for last domain without any density. When we tried 
> Molecular replacement with the missing domain (modelled) only we got solution 
> shows density for this missing domain but not able see electron density for 
> rest four domain even though room for 4 domain was visible in the solution. I 
> need help or suggestion for this issue.
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*------
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] 8/ 2263 spots indexed XDS

[Clemens Vonrhein]

Hi Almu,

some other pointers that might be useful in your case:

 * Check the list of known beamline configurations (and particulars)
   at [1] and [2] to see if there is something you need to tell the
   processing software about.

 * If you have multiple lattices, split spots or ice rings (or
   combinations of these), something like "iterative indexing" can
   ensure that subsequent integration starts from a good, initial
   orientation matrix [3,4].

 * Finally a note about damaged/unmasked pixels: these can greatly
   confuse COLSPOT when it comes to classifying so-called "weak spots"
   with odd results [5,6,7].

As Kay said, there are very few datasets where XDS indexing is
genuinely stuck - as long as (a) there are spots with a hint of a
spotlike shape and (b) the processing is told about beamline and
detector specifics.

Cheers

Clemens

[1] https://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings
[2] https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Generate_XDS.INP
[3] 
https://www.globalphasing.com/autoproc/manual/autoPROC7.html#step1_IterativeIndexing
[4] https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Indexing
[5] 
https://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings#hotpixels
[6] 
https://www.globalphasing.com/autoproc/wiki/index.cgi?AnalyseDataForDamagedPixels
[7] 
https://www.globalphasing.com/autoproc/wiki/index.cgi?DataProcessingHdf5Eiger16bit201910

On Sat, Nov 30, 2019 at 11:06:42AM +0100, Almudena Ponce Salvatierra wrote:
> Hi Raphael,
> 
> Thank you very much. I tried yesterday with different origins and I have
> changed the REFINE(IDXREF) keyword several times as to refine BEAM,
> POSITION, DISTANCE... I have to keep trying because fo far I didn't succeed.
> 
> I will get back to you once I find what the problem is.
> 
> Thank you once again!
> 
> Best,
> 
> Almudena
> 
> El vie., 29 nov. 2019 a las 12:56, R. Gasper-Schönenbrücher (<
> raphael.gas...@mpi-dortmund.mpg.de>) escribió:
> 
> > Hi Almu,
> >
> > this is most often the case, when your beam center is not assigned
> > correctly. Try to find out the beam center:
> >
> >
> > https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Finding_out_ORGX_ORGY
> > Another possibilty is that the distance that you get from the
> > header/machine is not very accurate. You can try to slightly adjust your
> > distance and see, whether this improves your indexing (e.g. instead of 50
> > start from 52).
> >
> > In XDS.INP you can enable distance refinement (as well as beam center
> > refinement) throughout all steps. Just remove the "!" in "Refine"
> >
> > Hope this helps,
> >
> > Best wishes,
> >
> > Raphael
> >
> >
> > On 29.11.19 12:48, Almudena Ponce Salvatierra wrote:
> >
> > Dear all,
> >
> > I have some data sets that don't want to be processed :p
> >
> > In one of them, when I look at IDXREF.LP I see virtually none of the found
> > spots were indexed and the reason is that they are "too far from the
> > expected position". The spots are smeary and elongated, so not the
> > prettiest.
> >
> > I have managed to process so far only one data set with decent statistics
> > from another crystal harvested from the same drop, where the diffraction
> > spots look better.
> >
> > I am trying to find in the xds wiki the keyword I should fine tune in
> > order to make those spots indexable.
> >
> > Could you help me please?
> >
> > Thank you very much in advance.
> >
> > Best wishes,
> >
> > Almu
> >
> > --
> >
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> >
> > --
> > Dr. Raphael Gasper-Schönenbrücher
> > Crystallography and Biophysics Facility
> > Max-Planck-Institute of Molecular Physiology
> > Room F0.92
> > Otto-Hahn-Str. 11
> > 44227 Dortmund, Germany
> > Tel +49 (0)231-133 2729
> >
> >
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Any guesses? - What is this feature modelled (wrongly) as 3 waters

[Clemens Vonrhein]

Dear Eleanor,

Considering some (potential) very strong model bias here: is the
difference mFo-DFc map totally flat or just not shown in your picture?
Does the density come back as such if the three waters are removed?

Cheers

Clemens

On Thu, Nov 07, 2019 at 01:12:14PM +, Eleanor Dodson wrote:
> seen in a high resolution map?
> 
> There is at least one other similar feature???
> 
> Eleanor

-- 

*------
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] How to include in refinement high resolution shells with VERY low completeness ?

[Clemens Vonrhein]

Dear Ivan,

On Mon, Aug 05, 2019 at 05:44:56PM -0400, Ivan Shabalin wrote:
> Dear Kay,
> 
> Thanks a lot for your answers!
> 
> To my best understanding, REFMAC does not have an option of for restoring
> reflections only in certain resolution shells.

Remember that you can always do some simple (but powerfull) operations
within SFTOOLS (*). So using something like

  read refmac.mtz
  CALC col resol = 0.5 stol /
  select col FP = absent
  absent col FWT  if col resol < 3.0
  absent col PHWT if col resol < 3.0
  delete col resol
  select all
  write refmac_new.mtz
  
might/should do the trick: setting all filled-in FWT/PHWT values to a
MNF (missing-number-flag) for reflections with resolution higher than
3A. So you should only have filled-in 2mFo-DFc valeus for the low-res
data (3A and lower).

But check those commands above (typed from memory)

Cheers

Clemens

(*) there is very little one can't do with SFTOOLS



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] [EXTERNAL] [ccp4bb] [6HR5] collected on an Eiger so Rmerge not relevant

[Clemens Vonrhein]

Hi Ivan,

On Wed, Jul 31, 2019 at 05:32:24PM -0400, Ivan Shabalin wrote:
> And Rfree of 36% seems really high.

If you look at the maps (e.g. after some re-refinement with your
favourite refinement package) it seems as if there are a few sequence
shifts (around and after A78), some poor density and additional
unmodelled density ... which all add to those high R-values I guess.

But given the poor data quality and no raw images (to check and maybe
improve upon that), I didn't feel the urge to delve into that any
further ;-)

Cheers

Clemens

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] [EXTERNAL] [ccp4bb] [6HR5] collected on an Eiger so Rmerge not relevant

[Clemens Vonrhein]

Hi,

On Wed, Jul 31, 2019 at 07:11:10PM +0100, Weston Lane wrote:
> Thanks for the response. I did look at the multiplicity of the
> datasets in their table and while I suppose 6.9x redundancy is sort
> of high for P2 spacegroup it's actually lower than some of the other
> datasets (presumably non-Eiger) in the table with good overall
> Rmerge (e.g. a C2 dataset with 10x redundancy and an Rmerge of
> 0.064).

Maybe a multiplicity of 6.9 is not especially low for monoclinic: 5LP9
for example (Pilatus data) has 6.3 for 360 degree of data (and Rmerge
of 0.045 to 0.864A when re-processing). Yes, that is more data than
traditionally collected for monoclinic (180 degree), but still far
away from some real high multiplicity approaches.

So this looks much more like a crysal issue to me: maybe there are
some really poor image ranges (that then give an overall Rmerge of
0.256). The overall I/sigI is also rather low (7), and if you look at

  http://staraniso.globalphasing.org/cgi-bin/PDBpeep.cgi?ID=6hr5

you can see that even in the lowest shell the I/sigI is only about
15 - so maybe a seriously underexposed crystal?

Low dose, high-multiplicity is very good - but only if one still takes
advantage of the dose budget a crystal provides. It doesn't help
underexposing a crystal for only 360 degree if it could give still
good data for 720, 1080 or 1440 degree. The great opportunity of low
dose, high multiplicity is that it allows one to see radiation damage
happening and then selecting a subset of images from the start that
are still complete with a nice multiplicity.

At least that is how I approach this with Pilatus/Eiger detectors (or
other photon counting ones) ...

Cheers

Clemens

PS: it would be great to have those images deposited (at
proteindiffraction.org, sbgrid,org or zenodo.org et al) :-)

-- 

*----------
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] High Rfree in last Shell

[Clemens Vonrhein]

Dear all,

on top of other peoples remarks and out of interest:

 (1) What is the supposed difference between Rmerge and Rsym in your
 table and what software/tool computed those? Are you sure Rsym is
 not supposed to be Rpim?

 There is a slight historical difference between the Rmerge/Rsym
 notation, but for all purposes of modern area detectors and
 processing software we tend to call this only Rmerge
 ... usually.

 (2) Are your Rmerge/Rmeas values truly exactly 100% in the outer
 shells? That seems surprising: those values are not
 intrinsically/mathematically limited to a maximum value of 100%
 and any program writing them out as 100% might do this by
 truncating them (so you might want to check).

 (3) What are the resolution limits for those outer shell values given
 in parentheses? Often they are given somewhere in such a table,
 so maybe only missing from your cut-n-paste to the CCP4bb.

Cheers

Clemens

PS: What is wrong by giving R-values as fractions (R/Rfree =
0.240/0.266, Rmeas=0.162) instead of give in them as percentages?
All (?) formulas I know of in books, dictionaries and papers don't
seem to have this factor of 100 in them ... am I missing a
historical/important relevance to this habit?



On Tue, Apr 16, 2019 at 12:57:06PM -0400, Jan van Agthoven wrote:
> Hi everyone,
> I’m trying to publish two structures at 3.1Å resolution with the following 
> refinement statistics:
> 
> Resolution range (Å)   49.2-3.1   
>49.3-3.1
> Rfactor (%)24.0 (32.4)
>   23.4 (32.0)
> Rfree (%)  26.6 (29.2)
>   26.3 (31.6)
> 
> Data collection
> Completeness  100 (100)   
>  100 (100)
> 
> Redundancy6.9 (7.0)   
>6.2 (6.3)
> 
> Molecules in asymmetric unit  1   
>1
> 
> Average I/σ 14.1 (1.7)
> 15.3 (2.0)
> 
> Rmerge (%)  14.9 (100)
>12.7 (100)
> 
> Rmeas (%)16.2 (100)   
> 13.9 (100)
> 
> Rsym (%)   6.2 (68.6) 
>5.5 (57.1)
> Wilson B-factor 65.6  
>   62.7
> 
> I’ve been told that the Rfree factor in the last shell are too high. Does 
> anyone know how I can improve these Rfree factors other then cutting the 
> resolution, which already is rather low?
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] [EXTERNAL] Re: [ccp4bb] High Rfree - ice ring

[Clemens Vonrhein]

from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCP4BB-26A-3D1=DwMFaQ=Dbf9zoswcQ-CRvvI7VX5j3HvibIuT3ZiarcKl5qtMPo=HK-CY_tL8CLLA93vdywyu3qI70R4H8oHzZyRHMQu1AQ=UECB-BTblxRTCrEA2arfPs5VNLr4ntyGxszsw_5Jaxc=QloAn3zc13ZQX64uxy6UZygomHXUigCIkpTw8ZhYpbk=>
> 
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.jiscmail.ac.uk_cgi-2Dbin_webadmin-3FSUBED1-3DCCP4BB-26A-3D1=DwMFaQ=Dbf9zoswcQ-CRvvI7VX5j3HvibIuT3ZiarcKl5qtMPo=HK-CY_tL8CLLA93vdywyu3qI70R4H8oHzZyRHMQu1AQ=UECB-BTblxRTCrEA2arfPs5VNLr4ntyGxszsw_5Jaxc=QloAn3zc13ZQX64uxy6UZygomHXUigCIkpTw8ZhYpbk=>
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> 
> -- 
> This e-mail and any attachments may contain confidential, copyright and or 
> privileged material, and are for the use of the intended addressee only. If 
> you are not the intended addressee or an authorised recipient of the 
> addressee please notify us of receipt by returning the e-mail and do not use, 
> copy, retain, distribute or disclose the information in or attached to the 
> e-mail.
> Any opinions expressed within this e-mail are those of the individual and not 
> necessarily of Diamond Light Source Ltd. 
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any 
> attachments are free from viruses and we cannot accept liability for any 
> damage which you may sustain as a result of software viruses which may be 
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England and 
> Wales with its registered office at Diamond House, Harwell Science and 
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] change in unit cell volume

[Clemens Vonrhein]

On Thu, Mar 07, 2019 at 02:32:53PM -0600, Murpholino Peligro wrote:
> Let's say I have a protein crystal from which I collected 30 datasets. If I
> plot the unit cell volume per dataset the volume rises.

You also have to be sure that

 (1) there is no significant energy drift during those experiments
 (which if not modelled could be compensated by a change in
 refined cell dimensions)

 (2) each dataset has been collected identically (over the same part
 of the crystal) and contains enough observations to give a stable
 refinement of your cell parameters

I would also check the values of crystal-detector distance as a
function of dose - since these should stay stable and not drift as
well. Sometimes instrumentation changes/drifts can be compensated by a
change in refined cell dimensions while the physical cell within the
crystal actually stays static.

Assuming you have high quality data for each of those datasets, a good
test is to refine your model to convergence against each of those and
see if the final (standard) bonds are systematically shorter or longer
than the expected values (e.g. Engh for protein). WhatCheck does
a check for that and correlates it with a nice suggestion about wrong
cell parameters ... which can highlight e.g. errors in energy or
wavelength values as written by beamlines into image headers ;-)

Cheers

Clemens



> My question is: Is there a rule of thumb of some sort* to consider the
> initial/final datasets isomorphous still?
> 
> * Something like if the unit cell volume changes more than 1% then the
> crystal is not isomorphous.
> 
> My second question is: Meents already said that the unit cell volume
> expansion is a consequence of hydrogen gas building up inside the crystal.
> But...what if the unit cell volume decreases? Is there an explanation for
> that?
> 
> 
> Thank you very much.
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*------
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] AW: [EXTERNAL] [ccp4bb] refmac same residue different names

[Clemens Vonrhein]

Dear Herman,

On Thu, Feb 07, 2019 at 10:26:09AM +, Herman Schreuder wrote:
> I did understand your question correctly and (at least for ligands)
> the procedure I and also Diana Tomchick described, worked. However,
> I just did a test with both Refmac and Buster and it seems that
> these programs have now so far been perfected that “errors” like
> this cannot occur anymore.

You might not have seen the reply to Ed's original question on the
BUSTER discussion mailing list [1]. It is not fully automatic and
might look slightly complicated (the intention was to give the full
background information for such a problem) - so doesn't qualify as a
"native" solution that Ed was looking for.

It is of course possible to handle micro-heterogeneity in BUSTER
(and also in REFMAC - after all: most structures marked with
"MICROHETEROGENEITY" in the PDB have been refined with it). If this
becomes a more common and pressing need for our users, development
of a "native" solution would move higher up the priority list list
in the future. It is very good to hear from users (like Ed) when
something doesn't quite work as expected - even if they then decide
to switch to another package/program for a particular problem.

Cheers

Clemens, Peter & Gerard

[1] 
http://www.globalphasing.com/pipermail/buster-discuss/2019-February/thread.html

> So it seems that the poor crystallographers who have crystallized proteins 
> which are heterogeneous at certain positions, will have to switch to Phenix.



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] AUTOPROC error s

[Clemens Vonrhein]

Dear Bri,

Could you let us know (maybe off-list since not truly CCP4 related)
what beamline those data were collected and when - and which autoPROC
version [1] you are using? There are (nearly) as many ways of writing
HDF5 datasets as there are detectors for collecting those datasets
plus multiple variants and versions ... all with at least slightly
different layout and content of meta-data.

We have a list of known (to us) beamlines at [2] that tries to
summarise our current knowledge about the state of synchrotron
beamlines around the world. You might notice that there are gaps and
possibly some bits of information could be out-of-date [3].

Cheers

Clemens

[1] http://www.globalphasing.com/autoproc/
[2] http://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings
[3] We love to hear from users and/or beamline staff about
new/modified instrumentation at proc-deve...@globalphasing.com

On Mon, Feb 11, 2019 at 02:28:27PM +, Bibel, Brianna wrote:
> Hi. I am trying to use autoproc to process HDF5 files using the command
> process -d test5 -noANO
> 
> and am getting the following errors
> 
> HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140736974844864:
>   #000: H5L.c line 805 in H5Lexists(): not a location
> major: Invalid arguments to routine
> minor: Inappropriate type
>   #001: H5Gloc.c line 246 in H5G_loc(): invalid object ID
> major: Invalid arguments to routine
> minor: Bad value
> HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140736974844864:
>   #000: H5L.c line 805 in H5Lexists(): not a location
> major: Invalid arguments to routine
> minor: Inappropriate type
>   #001: H5Gloc.c line 246 in H5G_loc(): invalid object ID
> major: Invalid arguments to routine
> minor: Bad value
> HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140736974844864:
>   #000: H5L.c line 805 in H5Lexists(): not a location
> major: Invalid arguments to routine
> minor: Inappropriate type
>   #001: H5Gloc.c line 246 in H5G_loc(): invalid object ID
> major: Invalid arguments to routine
> minor: Bad value
> HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140736974844864:
>   #000: H5L.c line 805 in H5Lexists(): not a location
> major: Invalid arguments to routine
> minor: Inappropriate type
>   #001: H5Gloc.c line 246 in H5G_loc(): invalid object ID
> major: Invalid arguments to routine
> minor: Bad value
> HDF5-DIAG: Error detected in HDF5 (1.10.4) thread 140736974844864:
>   #000: H5L.c line 805 in H5Lexists(): not a location
> major: Invalid arguments to routine
> minor: Inappropriate type
>   #001: H5Gloc.c line 246 in H5G_loc(): invalid object ID
> major: Invalid arguments to routine
> minor: Bad value
>  ERROR: [find_images-0002] unable to get listing
> ERROR: [x_check-0010] see above
> ERROR: [process-0023] during initial checks – see above
> 
> If anyone is able to help I would really appreciate it. Thank you!
> 
> Bri Bibel
> PhD candidate
> Cold Spring Harbor Laboratory
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Turning off the bulk solvent modelling in Refmac5 to generate Polder maps?

[Clemens Vonrhein]

Dear Samuel,

On Mon, Feb 04, 2019 at 11:39:58AM +, Samuel Davis (PG Research) wrote:
> I'm wondering if anyone knows if it is possible to turn off the bulk
> solvent modelling in Refmac5, for the purpose of generating Polder
> maps? I know that an option for Polder maps is directly implemented
> in Phenix, but we ideally want to use Refmac5, as we have used it
> for the rest of our refinement and want to keep it consistent if
> possible.

And if you want to try the original implementation of the underlying
idea as an alternative, have a look at the ligand detection mode and
maps [1] produced by BUSTER [2]. See also [3] and some early examples
of their usefulness [4-5].

Cheers

Clemens

[1] https://www.globalphasing.com/buster/wiki/index.cgi?LigandDetectionModes
[2] https://www.globalphasing.com/buster/
[3] Vonrhein, C., & Bricogne, G. (2005). "Automated Structure
Refinement for High-throughput Ligand Detection with
BUSTER-TNT". Acta Crysta A61, C248.
[4] Thoma, Ralf, et al. "Insight into steroid scaffold formation from
the structure of human oxidosqualene cyclase." Nature 432.7013
(2004): 118.
[5] Ekroos, Marika, and Tove Sjogren. "Structural basis for ligand
promiscuity in cytochrome P450 3A4." Proceedings of the National
Academy of Sciences 103.37 (2006): 13682-13687.

-- 

*------
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Generating Additional Phasing Statistics

[Clemens Vonrhein]

On Fri, Feb 01, 2019 at 03:00:09PM +, Eleanor Dodson wrote:
> Mlphare still exists! It gives you all that information...

Indeed ... or SHARP ;-)

I must say I stopped looking at R-Cullis/R-Kraut values a long time
ago (maybe I still should), but found phasing power immensely useful -
much more than FOM.

Of course, if you used program A to get those SIRAS phases at the time
you should only report statistics as given by that program. Maybe
Phenix has those stats "somewhere": asking the phenixBB? I would be
surprised if it doesn't compute phasing power, but maybe it doesn't.

If the stats aren't there then this is just what it is. Re-running a
different program B to get statistics is rather confusing: the
resulting phases could be worse (and stats misleadingly poor) ... or
of course could be much better. But then: some better phasing might
give you nice stats for reporting - but it wasn't what actually
happened at the time. It might be ok to convince a referee, but should
probably not be put into a deposition/paper.

> or just Argue that the map is good?

Yes :-)

Clemens

> 
> On Fri, 1 Feb 2019 at 13:54, McCoy, Jason (mccoyj5) 
> wrote:
> 
> > Dear CCP4bb members,
> >
> > I am compiling a rebuttal for a manuscript that utilized experimental
> > SIRAS phasing in order to generate coordinates that I could then use for
> > molecular replacement.  For phasing, I used phenix auto-sol and I was able
> > to successfully generate a suitable model for MR on a native data set.
> > However, a reviewer requested additional phasing statistics, such as the
> > phasing power and Cullis-R. It appears phenix auto-sol does not produce
> > these statistics. I have reported the FOM pre and post density
> > modification. I am looking to generate additional phasing statistics for my
> > reviewer. Do you have any recommendations?
> >
> > Thank you in advance,
> > Jason McCoy
> >
> > --
> >
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> >
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] old data

[Clemens Vonrhein]

Ooops - typo:

  https://www.globalphasing.com/autoproc/wiki/index.cgi?BeamlineSettings

(thanks Harry).

On Fri, Feb 01, 2019 at 11:31:14AM +, Clemens Vonrhein wrote:
> Some resources with a wide range of information regarding specific
> detectors (even going back to 2005-2010) might be helpful, e.g. the
> beamline information page for autoPROC processing at [1] or the
> "generate_XDS.INP" tool described at [2].
> 
> If you have binary data (like MAR CCD) I find the command "strings
> your.file" quite useful to show you some file and/or directory names
> that can give indications.
> 
> Cheers
> 
> Clemens
> 
> [1] 
> https://www.globalphasing.com/autoproc/wiki/admin/index.cgi?BeamlineSettings
> [2] 
> http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Generate_XDS.INP
> 
> On Thu, Jan 31, 2019 at 09:38:45AM +, Dean Derbyshire wrote:
> > maybe a silly question it there a data base or other way to tell what 
> > detector was used to collect historic data. Image header isn't hugely 
> > helpful
> > ESRF is all I know for the source but I'd like to know what the detector 
> > was at the time... -  I'm talking 2005-2010
> > 
> >Dean Derbyshire
> >Principal Scientist Protein Crystallography
> > [X]
> >Box 1086
> >SE-141 22 Huddinge
> >SWEDEN
> >Visit: Lunastigen 7
> >Direct: +46 8 54683219
> >Mobile: +46731251723
> >www.medivir.com<http://www.medivir.com>
> > --
> > This transmission is intended for the person to whom or the entity
> > to which it is addressed and may contain information that is privileged,
> > confidential and exempt from disclosure under applicable law.
> > If you are not the intended recipient, please be notified that any 
> > dissemination,
> > distribution or copying is strictly prohibited.
> > If you have received this transmission in error, please notify us 
> > immediately.
> > Thank you for your cooperation.
> > 
> > ####
> > 
> > To unsubscribe from the CCP4BB list, click the following link:
> > https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> 
> -- 
> 
> *--
> * Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
> * Global Phasing Ltd., Sheraton House, Castle Park 
> * Cambridge CB3 0AX, UK   www.globalphasing.com
> *------
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] old data

[Clemens Vonrhein]

Some resources with a wide range of information regarding specific
detectors (even going back to 2005-2010) might be helpful, e.g. the
beamline information page for autoPROC processing at [1] or the
"generate_XDS.INP" tool described at [2].

If you have binary data (like MAR CCD) I find the command "strings
your.file" quite useful to show you some file and/or directory names
that can give indications.

Cheers

Clemens

[1] https://www.globalphasing.com/autoproc/wiki/admin/index.cgi?BeamlineSettings
[2] http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Generate_XDS.INP

On Thu, Jan 31, 2019 at 09:38:45AM +, Dean Derbyshire wrote:
> maybe a silly question it there a data base or other way to tell what 
> detector was used to collect historic data. Image header isn't hugely helpful
> ESRF is all I know for the source but I'd like to know what the detector was 
> at the time... -  I'm talking 2005-2010
> 
>Dean Derbyshire
>Principal Scientist Protein Crystallography
> [X]
>Box 1086
>SE-141 22 Huddinge
>SWEDEN
>Visit: Lunastigen 7
>Direct: +46 8 54683219
>Mobile: +46731251723
>www.medivir.com<http://www.medivir.com>
> --
> This transmission is intended for the person to whom or the entity
> to which it is addressed and may contain information that is privileged,
> confidential and exempt from disclosure under applicable law.
> If you are not the intended recipient, please be notified that any 
> dissemination,
> distribution or copying is strictly prohibited.
> If you have received this transmission in error, please notify us immediately.
> Thank you for your cooperation.
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Very strange LINK cards appearing in recent structures

[Clemens Vonrhein]

Dear all,

we also have a lot of "fun" with LINK records when taking deposited
PDB structures into BUSTER refinement. Sometimes (but not often) there
are missing LINK records, but the vast number of problems is due to
erroneous LINK records added by annotation software it seems (based on
a cut-off of 3.5A apparently).

Correcting those issues once reported is one option, but maybe
revisiting the idea about auto-generating LINK records during
deposition would be beneficial. At the end of refinement the various
packages will have all required and no wrong/additional LINK records
in the final PDB file - and deposition software should probably take
those as-is without automatically rewriting/adding/removing any (while
still present a note/warning to depositor about potentially missed
LINK records according to annotation software). This would most likely
result in much fewer LINK problems in deposited PDB structures.

As the format guide says:

  The LINK records specify connectivity between residues that is not
  implied by the primary structure. 

and specifically mentions bonds (not hydrogen bonds or salt-bridges):
https://www.wwpdb.org/documentation/file-format-content/format33/sect6.html#LINK

Cheers

Clemens
  

On Sun, Nov 11, 2018 at 07:31:55PM +, Robbie Joosten wrote:
> Erroneous LINK records happen quite a lot and used to be the combination of 
> aggressive annotation software and depositors not paying attention to the 
> comments from the annotators. They make up a large fraction of the bug 
> reports I have sent to the PDB over the years. They are usually fixed very 
> quickly by the annotators, as long as someone takes time to report them.
> 
> This case looks like an error in a refinement program which nevertheless 
> should have been caught by the depositors. What I would like to know is 
> whether the deposited, pre-annotation model had the LINKs or not.
> 
> LINKs are a bloody nightmare when it comes to annotation. At the moment there 
> is no record keeping of targets and chemical modifications in a dictionary on 
> the side of the PDB so there is also no standardisation. IMO mmCIF makes it 
> easier to store the restraints with the coordinates, but there is still no 
> neat mapping by LINK identfiers the way the LINKR format works in Refmac. I 
> think that is a missed opportunity.
> 
> Sorry for the rant, I blame the F1.
> 
> Cheers,
> Robbie
> 
> 
> Op 11 nov. 2018 19:47 schreef Tristan Croll :
> 
> I've seen instances like the following in roughly half a dozen deposited
> structures over the past year or so. Each time I've contacted the
> authors, who've been just as mystified as me by them - and certainly
> didn't add them on purpose. It seems to me that some fairly
> commonly-used package is erroneously turning clashes into LINK cards in
> some circumstances. I just found the following clearly wrong LINKs in
> 6caj (deposited January this year):
> 
> LINK CD2 PHE I 266 CG2 THR I 272 1555   1555
>   1.47
> LINK CE2 PHE I 266 CG2 THR I 272 1555   1555
>   1.47
> 
> ... which looks like the attached image. The same bonds are also
> specified in the mmCIF file, for the record.
> 
> Anyone have any clue?
> 
> Best regards,
> 
> Tristan
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] VERY old mtz file..

[Clemens Vonrhein]

Hi Eleanor,

You could try running the oldest MTZ2VARIOUS binary you can find -
e.g.

  wget ftp://ftp.ccp4.ac.uk/ccp4/4.2/binaries/ccp4-4.2_Linux.tar.gz
  tar -xvf ccp4-4.2_Linux.tar.gz bin/mtz2various

  bin/mtz2various hklin ...

Any older binaries (ftp://ftp.ccp4.ac.uk/ccp4/4.0.1/) will require an
SGI or Dec/Alpha machine ;-)

If that doesn't help I would first check that it is actually a correct
MTZ file: does the ASCII header (trailer) show up with

  strings your.mtz

towards the end?

Cheers

Clemens

On Fri, Nov 09, 2018 at 12:47:09PM +, Eleanor Dodson wrote:
> Anyone any idea what to do about this?? Created in 1992!!
> Seems unreadable..
> 
> No CTYP lines input for file:  1
> Indices output even if all data items flagged "missing"
>  Warning, NOT all LABOUT data lines given
> Warning: Machine stamp corrupted? Assuming native format.
> >>>>>> CCP4 library signal library_file:End of File (Error)
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] ice rings

[Clemens Vonrhein]

>  20  0.0656   3.90  0.588  0.588  0.094  0.718  0.40425888   21
>  33 130.6  3.9 -  3.12   1.24
>  21  0.0690   3.81  0.822  0.822  0.101  0.974  0.51528128   12
>  15 140.8  2.7 -  1.03   1.00
>  22  0.0724   3.72  1.140  1.140  0.109  1.347  0.709295839
>  18 150.5  2.0 -  1.25   1.01
>  23  0.0757   3.63  0.819  0.819  0.118  0.989  0.54330705   18
>  58 160.3  2.2 -  1.81   1.09
>  24  0.0791   3.56  1.643  1.643  0.128  1.926  0.995316267
>  18 160.4  1.5 -  1.03   1.00
>  25  0.0825   3.48  2.505  2.505  0.139  2.946  1.534323275
>  23 170.2  1.0 -  1.33   1.06
>  26  0.0858   3.41  1.601  1.601  0.151  1.925  1.05132349   10
>  52 180.2  1.4 -  2.72   1.17
> 
> -- 
> 
> ***
> 
> Charles Chen
> 
> Research Instructor
> 
> University of Pittsburgh School of Medicine
> 
> Department of Anesthesiology
> 
> **********
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] processing hd5 files from Dectris detector

[Clemens Vonrhein]

Dear Laurent,

On Thu, May 31, 2018 at 02:58:52PM +0200, maveyrau wrote:
> we recently collected many datasets on dectris detectors producing
> hd5 files. I would like to use some auto processing tools to process
> them (xdsapp, xdsme…). As far as I can say, xdsapp or xdsme cannot
> process hd5 natively. I tried to convert them to cbf format
> (eiger2cbf, hdf2mini-cbf,...), but then it seams that the header of
> the caf files are lacking some required informations…

Since you already mention "hdf2mini-cbf" from autoPROC, why not use
autoPROC itself for doing the actual processing:

  process -h5 /where/ever/your_master.h5 ...

See also [1] and [2] for more information.

> Any idea how to convert hd5 files to complete cdf files?

hdf2mini-cbf [3] should create CBF files that are as compatible with
original Pilatus mini-CBF files as possible: at least we tried very
hard to achieve this in it's original design. If this doesn't work in
your particular case we would be very interested to have access to
some example files from the beamline(s) you collected on - maybe there
are changes in the HDF5 file(s) we aren't aware of (yet). Keeping up
with changes to detectors or beamlines is always tricky, unless
developers are being made aware of this ;-)

Cheers

Clemens

[1] www.globalphasing.com/autoproc/wiki/index.cgi?DataProcessingHdf5
[2] http://www.globalphasing.com/autoproc/manual/autoproc_reference_card.pdf
[3] 
www.globalphasing.com/autoproc/wiki/index.cgi?DataProcessingHdf5#hdf5converter

To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Problems with HEC in CCP4

[Clemens Vonrhein]

elopers,
> > > please fix errors with heme c dictionary file HEC.cif.
> > > Edward Berry described this problem in detail 4 years ago:
> > > https://www.mail-archive.com/ccp4bb@jiscmail.ac.uk/msg36393.html
> > > This error already affects deposited entries, e.g.: 4QO5 clearly
> > > contains errors in vynil groups of heme c.
> > >
> > >
> > > --
> > >
> > > Eugene Osipov
> > > Junior Research Scientist
> > > Laboratory of Enzyme Engineering
> > > A.N. Bach Institute of Biochemistry
> > > Russian Academy of Sciences
> > > Leninsky pr. 33, 119071 Moscow, Russia
> > > e-mail: e.m.osi...@gmail.com<mailto:e.m.osi...@gmail.com> 
> > > <mailto:e.m.osi...@gmail.com<mailto:e.m.osi...@gmail.com>>
> 
> 
> 
> --
> Eugene Osipov
> Junior Research Scientist
> Laboratory of Enzyme Engineering
> A.N. Bach Institute of Biochemistry
> Russian Academy of Sciences
> Leninsky pr. 33, 119071 Moscow, Russia
> e-mail: e.m.osi...@gmail.com<mailto:e.m.osi...@gmail.com>

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--

Re: [ccp4bb] Issues with latest XDS (20171218)

[Clemens Vonrhein]

that 
> at high resolution in particular this default will lead to worse data. 
> generate_XDS.INP (in XDSwiki) and (I think) autoPROC and xia2 explicitly 
> include POSITION in the REFINE(IDXREF), i.e. override the default. Where did 
> you get XDS.INP from?
> Some comments:
> a) this is why I recommend, for data sets that may be important, to always do 
> one cycle of optimization, i.e. after CORRECT, "mv GXPARM.XDS XPARM.XDS", 
> change XDS.INP to have the correct high-resolution cutoff (cutting after the 
> last shell which still has a "*" in the CC1/2 column), adjust the 
> FRIEDEL'S_LAW= setting, and run JOB=INTEGRATE CORRECT. In your case this 
> would make the refined distance available to INTEGRATE, and should result in 
> very good data.
> b) at which beamline did you collect the data? Such a high difference between 
> refined distance and distance from header is unusual, and should be reported 
> to (and fixed by) the beamline staff.
> c) for this beamline at least, one should use REFINE(IDXREF)=CELL BEAM 
> ORIENTATION AXIS POSITION . I understand that you tried this and it didn't 
> work? Maybe there is something else wrong then, e.g. another line later in 
> XDS.INP resetting REFINE(IDXREF) to something else.
>
> If you cannot find the reason why including POSITiON into REFINE(IDXREF) does 
> not help, pls send me enough data to reproduce the problem.
>
> best wishes,
>
> Kay
>
> On Tue, 23 Jan 2018 14:31:09 +, "Weiergräber, Oliver H." 
> <o.h.weiergrae...@fz-juelich.de> wrote:
>
> >Dear all,
> >
> >After upgrading XDS from build date 20170601 to 20171218, I am experiencing 
> >severe degradation of apparent data quality reported by CORRECT for certain 
> >data sets. Following first indications of issues with a slightly problematic 
> >candidate, I went back to a previously very well-behaved test case with 
> >diffraction extending beyond 1.1 A. Using the same input with both program 
> >versions, statistics are not too different out to approx. 1.6 A, but become 
> >more and more divergent in outer shells. These are the values for the 
> >highest-resolution shell (1.10--1.16 A):
> >
> >20170601:  I/sigI = 1.80; Rmeas = 59.9 %; CC(1/2) = 82.0 %
> >20171218:  I/sigI = 0.14; Rmeas = 506.4 %; CC(1/2) = 5.8 %
> >
> >The refined cell constants are unreasonably different as well. Obviously, 
> >something is going awfully wrong here, presumably related to errors in 
> >geometry parameters (which are expected to become increasingly detrimental 
> >with decreasing dmin). As it turns out, geometry refinement behaves 
> >differently in the latest version of XDS: most notably, IDXREF no longer 
> >refines the detector distance by default. This is significant in this case, 
> >as in version 20170601 the distance would shift by as much as 1.3 mm, 
> >resulting in successful integration and scaling. Unfortunately, re-adding 
> >POSITION refinement into REFINE(IDXREF) does not help, and even having it in 
> >all refinement scopes (IDXREF, INTEGRATE, CORRECT) only yields a limited 
> >improvement of CORRECT statistics.
> >It is worth noting that examination of a dataset from an unrelated crystal 
> >(dmin = 1.4 A) did not reveal such enormous differences -- in this case, 
> >however, the shift in detector distance applied in the old version of XDS 
> >was quite small (0.2 mm), which, together with the lower resolution, 
> >explains the absence of large discrepancies.
> >
> >To sum up, I suspect that, due to recent changes to the XDS code, the 
> >refinement of geometry parameters is now overly restrained, resulting in 
> >drastic problems in cases where the detector distance is not as precise as 
> >desirable (and even more so at very high resolution). For such datasets, the 
> >new version appears to be essentially unusable (at least within the 
> >parameter space I have tested), and even in modest cases may often be 
> >inferior to the previous one. Is there an option to revert to the former 
> >behaviour?
> >
> >Best regards
> >Oliver
> >
> >
> >  PD Dr. Oliver H. Weiergr�ber
> >  Institute of Complex Systems
> >  ICS-6: Structural Biochemistry
> >  Tel.: +49 2461 61-2028
> >  Fax: +49 2461 61-9540
> >
> >
> >
> >
> >
> >
> >
> >Forschungszentrum Juelich Gmb

Re: [ccp4bb] Total occupancy of two conformations of one nucleotide is over 1.0? Need help!

[Clemens Vonrhein]

Dear Wei,

remember that you might have "something else" in the location occupied
by each alternate conformation when it is not occupied by that
particular conformation. If you only model two alternate conformations
you are saying something like

 that space A is occupied 50% of the time by this A conformation and
 50% it is vacuum

 that space B is occupied 50% of the time by this B conformation and
 50% it is vacuum

It is more likely that you will have space A occupied N% by altConf A
and (100-N)% e.g. by water/solvent. And for B you have (100-N)%
altConf B and N% e.g. water/solvent. So you will need to model
everything marked as N% as altConf A and everything marked (100-N)% as
altConf B - including water/solvent.

Because this is not yet done, the occupancy refinement might try to
explain the non-vacuum density (water/solvent) by increasing the
occupancy of each conformation so they sum up to larger than 1.

Does that make sense?

Cheers

Clemens

PS: in BUSTER (http://www.globalphasing.com/buster/) you can restrain
the summed occupancy to one, which can help clarifying how to
model the water/solvent region underneath each conformation
... depending on data quality/resolution of course. I'm sure other
refinement programs have a similar feature.

PPS: it can be sometimes useful to start refinement once from
A=0.1/B=0.9 and a second run with A=0.9/B=0.1. Refinement should
reach very similar final values in both cases - which is also a
good way of providing confidence in the final results, I think.


On Fri, Jul 21, 2017 at 01:10:15AM +, Wang, Wei wrote:
> Hi Everyone,
> 
> 
> One quick question:
> 
> 
> When I do phenix refinement, I see continuous omitted map at one RNA chain 
> 3'-end (Fig1).
> 
> There is no possibility of next nucleotide because I have built full-length 
> RNA sequence, which is well fitted into density.
> 
> Hence two conformations is most possible for the last nucleotide. And our 
> biochemical data also supported the result of two conformations.
> 
> However, I found the occupancy is always > 1.0 (fig2 and fig3), when I did 
> refinement of occupancy using phenix refinement,.
> 
> I also think about the possibility of small molecules in the crystal buffer. 
> But in most structures of this protein, there is always no small molecule 
> here.
> 
> 
> Any suggestions for occupancy refinement? I will appreciate your great help.
> 
> 
> Best,
> 
> Wei
> 
> 
> 
> [cid:2b30f1a8-2a3a-4483-ac12-310f4bbed70b]
> 
> 
> 
> 
> 
> 



-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--

Re: [ccp4bb] Problem with Mg2+ binding site refinement

[Clemens Vonrhein]

t; >> metal restrains were used. Herewith I have attached the refinement
> >> statistics.
> >>
> >> please help me to overcome this problem.
> >>
> >> Thank you
> >>
> >> -Mubinur
> >>
> >>

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--

Re: [ccp4bb] Optimising data processing of a I432 dataset with 75% solvent content.

[Clemens Vonrhein]

er of observations   34917  1495  6448
> Total number unique 2057   112   362
> Mean((I)/sd(I)) 18.3 130.9   0.1
> Mn(I) half-set correlation CC(1/2) 1.000 1.000 0.533
> Completeness99.9  97.4 100.0
> Multiplicity17.0  13.3  17.8
> 
>   Overall  InnerShell  OuterShell
> Low resolution limit   43.50 43.50  3.84
> High resolution limit   3.50  8.58  3.50
> 
> Rmerge  (within I+/I-) 0.052 0.011 2.422
> Rmerge  (all I+ and I-)0.056 0.012 2.659
> Rmeas (within I+/I-)   0.055 0.011 2.553
> Rmeas (all I+ & I-)0.058 0.013 2.738
> Rpim (within I+/I-)0.017 0.004 0.804
> Rpim (all I+ & I-) 0.014 0.003 0.644
> Rmerge in top intensity bin0.010- -
> Total number of observations   24596  1690  6071
> Total number unique 1462   120   343
> Mean((I)/sd(I)) 25.8 132.0   1.0
> Mn(I) half-set correlation CC(1/2) 1.000 1.000 0.771
> Completeness99.8  97.6 100.0
> Multiplicity16.8  14.1  17.7
> 

-- 

*--
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
* Global Phasing Ltd., Sheraton House, Castle Park 
* Cambridge CB3 0AX, UK   www.globalphasing.com
*--

Re: [ccp4bb] NCS difference

[Clemens Vonrhein]

Dear all,

On Mon, Apr 24, 2017 at 08:15:57AM +, Bert Van-Den-Berg wrote:
> Not quite sure what you mean but I suppose you refined with NCS
> restraints and the red bar means that your chains in those regions
> are not identical. I would turn NCS restraints off during
> refinement, with your resolution there is no real good reason to
> include them.

While that might have been understandable advice in the days when we
used purely superposition/rmsd-based NCS restraints/constraints (and
true differences between NCS-related copies became a nightmare to
define/model), this should no longer be true for the majority of
refinement programs out there.

We now use much "softer" NCS restraints that allow for local
similarity - which at the same time allows for large differences like
domain/loop movements etc. Most programs will at the same time detect
local outliers and prune those. See e.g. [1] for such an
implementation in BUSTER [2] (-autoncs flag) ... I'm sure other
programs use similar approaches.

Anyway, the traditional recommendation to drop NCS restraints at later
stages of refinement was nearly always based on procedural
complexities and should no longer hold true: there is no reason to
drop NCS restraints at any (within reason) resolution I think. Of
course, the real differences need to be taken care of by exclusion
(what we call 'pruning', mostly done automatically anyway).

Cheers

Clemens

[1] Smart, O. et al (2012). Acta Cryst. D68, 368-380.
[2] http://www.globalphasing.com/buster/

Re: [ccp4bb] excluding residues from NCS

[Clemens Vonrhein]

Dear Alice,

in case you are still working on setting up NCS-restraints in
refinement (especially excluding those residues from NCS-restraints
that are truly different): BUSTER [1] can do this automatically
through the use of so-called "pruning" of the local structure
similarity restraints [2]. It also takes care of symmetrical
side-chains with equivalent atoms that have different atom names
[3].

Please let us know off-list if you need further information.

Cheers

Clemens, Andrew & Gerard

[1] http://www.globalphasing.com/buster/wiki/index.cgi?BusterCite
http://www.globalphasing.com/buster/
[2] http://www.globalphasing.com/buster/manual/gelly/manual/gelly4A.html
[3] http://www.globalphasing.com/buster/wiki/index.cgi?AutoBusterExamples


On Tue, Feb 21, 2017 at 10:39:14AM +, Alice Dawson (Staff) wrote:
> Dear all
> 
> I am working on a structure with 10 monomers in the asymmetric unit (2 
> pentamers). I am using the NCSR local option in Refmac to automatically 
> generate NCS restraints. This is working well (much better than the NCS 
> restraints I defined manually). However there is one residue in one subunit 
> that makes a hydrogen bond with an adjacent chain (due to crystal packing) 
> and this pulls the side chain over by abut 0.3A. Sadly this is too small to 
> cause these residues to be excluded automatically from the NCS restraints but 
> large enough to make the difference map nasty. Is there any way to exclude 
> residues from NCS local?
> 
> Thanks (particularly as I have another 7 structures of the same protein to 
> tidy up)
> Alice
> 
> --
> Alice Dawson
> WNH Lab
> Division of Biological Chemistry and Drug Discovery
> School of Life Sciences
> University of Dundee
> 01382 385744

Re: [ccp4bb] sfall bug?

[Clemens Vonrhein]

Hi Jens,

I do get the same results when running

  sfall xyzin a.pdb hklout a.mtz mapout a.map EOF
  MODE SFCAL XYZIN ATMMAP
  RESO 100 2
  SFSG P1
  SYMM P21
  NOSC
  END

ie. enforcing P1 for the structure-factor calculation. The density map
(on MAPOUT) is on top of the atoms as expected. Unfortunately,
the output MTZ file then has a hemisphere of data (P1) ... which is
probably not what you want.

If I remove the SFSG P1 card (ie let SFALL decide on the SG for
SF-calculation itself): the ATMMAP generated for a.pdb is just wrong - it is
missing density for the 

  35.753   7.581 -12.182  1.00  5.91

atom. As Kay says: maybe an issue with atom sorting ... or in the PDB
reading via MMDB?

It would be interesting to see if this issue is only for SFSG P21 or
also for other non-P1 cases, eg. P212121.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Optimal settings for finding NCS from density

[Clemens Vonrhein]

Dear William,

since it is a question to CCP4bb, I might suggest a CCP4-based
solution ...

On Fri, Jul 03, 2015 at 04:24:04PM +, William Chao wrote:
 What would be the best settings to look for NCS using
 phenix.find_ncs_from_density on an elongated (~150A) molecule? No
 coordinate is available unfortunately and there should be 2 copies
 in a P21 space group.

If you have a convincing looking self-rotation function (or a few
peeks at the kappa=180 section) and you expect your molecules to pack
as a C_2 dimer, then you could try the GETAX [1] program in CCP4 (also
in ccp4i). It runs reasonably fast and you should be able to run
multiple trials with different assumptions for dimer shap (around that
2-fold).

This might not be what you are looking for (I don't know what
functionality phenix.find_ncs_from_density is providing), but if you
want further info about GETAX, please let me know.

Cheers

Clemens

[1] http://www.ccp4.ac.uk/html/getax.html
http://journals.iucr.org/d/issues/1999/01/00/se0241/se0241bdy.html

Re: [ccp4bb] Fwd: [ccp4bb] XDS Rmeas in space group determination

[Clemens Vonrhein]

Dear Chen,

On Thu, May 14, 2015 at 10:05:05AM -0400, Chen Zhao wrote:
 Thanks a lot for your reply! I am a little surprised to learn that the
 centrosymmetry is always considered as a point groups symmetry component.
 That might explain why all the anomalous data I have seen have higher Rmeas
 than their native counterpart.

Somehow related: the fact that one can compute R-values ignoring
anomalous or taking it into account can actually be a good
thing. AIMLESS for example computes RmeasOv (ignoring anomalous) and
Rmeas (keeping I+ and I- separate) - if I understand that
correctly. You can use those two values as a function of resolution to
see up to which resolution you have anomalous signal, ie where RmeasOv
is still higher than Rmeas.

It doesn't necessarily give you new information compared to CCanom or
SigAno, but it is nice to have as another opinion I think.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] XDS INP

[Clemens Vonrhein]

Hi Kay,

On Tue, May 12, 2015 at 08:39:24PM +0100, Kay Diederichs wrote:
 According to
 http://homes.mpimf-heidelberg.mpg.de/~kabsch/xds/html_doc/xds_parameters.html#ROTATION_AXIS=
 the definition of a positive value in the ROTATION_AXIS= line is:
 
 When looking along the axis, the crystal would rotate clockwise
 when proceeding to the next data image.

I find the even better description in that part of the XDS
documentation is

  The direction of the axis is chosen to describe a right-handed
  rotation.

which should follow the right-hand rule a la

  http://en.wikipedia.org/wiki/Right-hand_rule

The looking along the axis doesn't clearly define if it is (a)
looking from the crystal towards the base of the rotation axis or (b)
from the rotation-axis base towards the crystal.

 There is of course no right or wrong when it comes to choosing the
 direction of rotation. However, conventionally the sense of rotation
 is positive; only a small minority of beamlines needs a -1 (reverse
 phi).

Yes: one can always define the rotation axis without the need for a
'-1'. But this has an impact on the chosen lab coordinate system and
therefore might require a change of INCIDENT_BEAM_DIRECTION= and/or
DIRECTION_OF_DETECTOR_{X,y}-AXIS= ... and there might be good reasons
for having those defined in a particular way (eg. to avoid a negative
value for DETECTOR_DISTANCE= or to have them aligned with the
fast/slow changing axis of the image array as X and Y).

 The problem is that 
 a) beamlines do not usually document this on their webpages, and
 sometimes change it without notice

Indeed.

Most beamlines are quite good in providing up-to-date XDS.INP
templates that are known to work with data collected on that
beamline. Ideally, the {X,Y}-GEO_CORR files for Pilatus detectors
should also be placed in a public space (and everything else that
might be required). All so that users can process the data again once
they are back in the home lab with more time and less stress - trying
to reproduce what happened by the automatic processing systems
installed on most beamlines (and through that task especially new
users will actually learn what entails good data processing practice).

 Personally, I wish that beamline designers would be aware of the
 potential problem for users; I suspect they often are not. Life
 would be easier if all beamlines would use the same convention, and
 I'm pretty sure that spindle motors can be
 produced/bought/programmed for both directions.

Sometimes there are restrictions upon beamlines regarding the choice
of coordinate system to be used: this often has to be identical for
everything and all beamlines at a given synchrotron.

Reaching perfection in an imperfect world ;-)

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] XDS INP

[Clemens Vonrhein]

  116.4   93.3  90.2  90.0  90.0
   *  32oP  4.5  64.5   93.3  116.4  90.2  90.0  90.0
  37mC249.9 241.6   64.5   93.3  90.0  90.2  74.5
 
  I would hypothesize that the beam position is incorrect. Personally, I'd
  use
  JOB= DEFPIX INTEGRATE CORRECT
  ORGX=994.64 ORGY= 1035
  for a tentative round of integration, maybe together with
  SPACE_GROUP_NUMBER=19
  UNIT_CELL_CONSTANTS= 64.5   93.3  116.4  90.  90.0  90.0
  and then inspect the result. If the statistics look reasonable (ISa  10
  or so), you should optimize it (see XDSwiki). If it looks very bad (ISa 
  5), you can run
  echo CENTRE | pointless XDS_ASCII.HKL
  afterwards, which will tell you whether an offset in one of the indices
  has to be applied. In that case, you should inspect the INDEX ORIGIN
  table of IDXREF.LP again, to see which ORGX ORGY this corresponds to. After
  this, re-integrate, optimize ...
 
  If you are not successful, compress your frames, upload them to some
  Dropbox-like directory, and send me the link. I'll look at your data,
  treating them confidentially of course, and tell you what I find.
 
  best,
  Kay
 
 
 
 
 
 
  Dear Kay,
  
  I've tune these parameter for many times, and I got best results . :
  
  SPOT_RANGE=1 100
  
  INCLUDE_RESOLUTION_RANGE=50 4.2
  
  MINIMUM_NUMBER_OF_PIXELS_IN_A_SPOT=20
  
  but still got the same error message!
  
  The SPOT.XDS file was ploted (see attachment spot_15.png ), it seems
  that the ice ring and beam stop shadow was excluded. But the result is
  still frustrating.
  
   Best wishes!
  
  LU zuokun
 
 
 
 
 -- 
 *Natalie J. Tatum*
 PhD Researcher
 Durham University
 http://about.me/n.j.tatum

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Sharp: Solomon density modification step

[Clemens Vonrhein]

Hi Frank,

On Wed, Jan 28, 2015 at 07:52:39AM +, Frank von Delft wrote:
 Clemens - what do you mean by correcting anisostropy?

Ooops - typo and not quite clear ;-)

I mean diffraction anisotropy and correcting for it. SHARP will do
that by default if run through autoSHARP and if there is good enough
resolution (at least 3.5A). But one can do that in various other ways
I guess (including the Diffraction Anisotropy Server at UCLA or
various CCP4 programs) - just that I'm most familiar with how SHARP
does it.

Anyway, the output data (FP/SIGFP as well as map coefficients in
eden.mtz) will be corrected for that anisotropy. This means that
density-modification will hopefully not get confused by this when
working in real-space.

I've had cases where this made a significant difference in map
quality: convincing me that it is worth while spending time building
and improving those initial phases.

Cheers

Clemens

 
 On 28/01/2015 07:42, Clemens Vonrhein wrote:
 Dear Nicolas,
 
 On Mon, Jan 26, 2015 at 03:21:00PM +, Nicolas Soler wrote:
 A quick question regarding the density modification interface via
 the Sharp interface. Which resolution range / radius of the solvent
 sphere/ ncycles should be used for optimal result?
 I usually don't play around a lot with those parameters - other than
 possibly increasing the number of cycles to something like 100.
 
 The documentation seems to suggest to restrict yourself up to the
 resolution where good phasing information is available (6.5A in my
 case) and I get excellent indicators only if I do that (they become
 horrible if I use the full resolution range).
 Going from low resolution phase information (say below 4-4.5A) to the
 high resolution limit of your data (even if that is 'only' 3A) is
 notoriously difficult when doing 'only' solvent flattening or
 flipping. To get over that bump, NCS-averaging would be a massive
 help. Or if you have some initial model - maybe a partial MR solution
 or such - you can add this to the mix as well.
 
 The other thing about such low-resolution phase information or data is
 that a lot of the statisitical indicators can become rather noisy and
 much less reliable.
 
 Your statistics will automatically be much worse if you include higher
 resolution data, since you will have many more reflections with poor
 initial phase information going into the same, single value. The
 important thing: do the maps look better than if you use only
 low-resolution data all the way through?
 
 How about phase extension ? Which parameters would you then use?
 Maybe somebody else has more experience with fine-tuning those
 parameters. For me the some other important things are happening a
 step before that density-modification:
 
 * do the two hands/enantiomoprhs show a significant difference in
score?
 
 * is the HA model as complete and accurate as possible (using the LLG
maps in SHARP for checking)?
 
 * is there severe anisotropy in your data? This can be a real pain for
the density-modification and you should try to correct for it before
doing any density-modification procedure if possible.
 
 Cheers
 
 Clemens
 

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Sharp: Solomon density modification step

[Clemens Vonrhein]

Dear Nicolas,

On Mon, Jan 26, 2015 at 03:21:00PM +, Nicolas Soler wrote:
 A quick question regarding the density modification interface via
 the Sharp interface. Which resolution range / radius of the solvent
 sphere/ ncycles should be used for optimal result?

I usually don't play around a lot with those parameters - other than
possibly increasing the number of cycles to something like 100.

 The documentation seems to suggest to restrict yourself up to the
 resolution where good phasing information is available (6.5A in my
 case) and I get excellent indicators only if I do that (they become
 horrible if I use the full resolution range).

Going from low resolution phase information (say below 4-4.5A) to the
high resolution limit of your data (even if that is 'only' 3A) is
notoriously difficult when doing 'only' solvent flattening or
flipping. To get over that bump, NCS-averaging would be a massive
help. Or if you have some initial model - maybe a partial MR solution
or such - you can add this to the mix as well.

The other thing about such low-resolution phase information or data is
that a lot of the statisitical indicators can become rather noisy and
much less reliable.

Your statistics will automatically be much worse if you include higher
resolution data, since you will have many more reflections with poor
initial phase information going into the same, single value. The
important thing: do the maps look better than if you use only
low-resolution data all the way through?

 How about phase extension ? Which parameters would you then use?

Maybe somebody else has more experience with fine-tuning those
parameters. For me the some other important things are happening a
step before that density-modification:

* do the two hands/enantiomoprhs show a significant difference in
  score?

* is the HA model as complete and accurate as possible (using the LLG
  maps in SHARP for checking)?

* is there severe anisotropy in your data? This can be a real pain for
  the density-modification and you should try to correct for it before
  doing any density-modification procedure if possible.

Cheers

Clemens

Re: [ccp4bb] R/R free reported values

[Clemens Vonrhein]

Hi Cedric,

the PDB file reports two R/Rrfee values:

  all data  :  0.110 / 0.170
  F4SIG(F) :  0.103 / 0.162

while the mmCIF version has:

 _refine.ls_R_factor_R_work 0.103 
 _refine.ls_R_factor_R_free 0.134 

But that R-free value doesn't seem to be present in the PDB formatted
version. Also: _refine.ls_R_factor_R_work is probably the wrong item
(if according to the PDB formatted version that R-value was calculated
from both working and test-set reflections): shouldn't it be
_refine.ls_R_factor_obs? See:

  
http://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v40.dic/Items/_refine.ls_R_factor_R_work.html

Without knowing if the mmCIF was generated from a PDB
formatted deposition or the other way round, it might be tricky to
know what values are actually correct from just looking at the PDB
archive.

But looking at the paper (Table 2):

 Reflections with F 4 sigma (for Rfree)
  Isotropic: Rcryst/Rfree (%)14.08 / 16.11
  Anisotropic: Rcryst/Rfree (%)  10.29 / 13.36
 All reflections (for Rfree)
  Anisotropic Rcryst/Rfree (%)   11.01 / 14.14

So I would say that R/Rfree = 0.110/0.141 are the 'correct' values (if
Rcryst==R?).

It looks like typical case of slight confusion when going from a paper
to filling out the deposition form correctly.

Cheers

Clemens

On Mon, Oct 27, 2014 at 12:00:23PM -0800, Cedric wrote:
 Hi Phenix and CCP4 community,
 
 (sorry for the cross posting).
 
 I was looking at a PDB file.
 
 The http://www.rcsb.org/ website page gives the following values:
 
 R-Value: 0.103 (work)
 R-Free:0.134
 
 The actual PDB file gives the following:
 
 R :0.110
 FREE R VALUE : 0.170
 
 I was wondering why the difference. The structure is 1.0A resolution 2CWS.
 
 
 Thank you,
 
 
 FREE ONLINE PHOTOSHARING - Share your photos online with your friends and 
 family!
 Visit http://www.inbox.com/photosharing to find out more!

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] experimental phasing + MR

[Clemens Vonrhein]

Dear Gabriel,

since you are already using autoSHARP: why not giving it the MR model
as input too? This way you will combine the model and the experimental
phasing.

This combination can not only be done in autoSHARP (for SAD, MAD,
SIR(AS) and MIR(AS)), but also in SHARP directly for any phasing
scenario you could think of. See eg.

  
https://www.globalphasing.com/sharp/manual/chapter4.html#ExternalPhaseInformation

If you need additional help, please contact us (or the SHARP
discussion list).

Cheers

Clemens

On Mon, Oct 13, 2014 at 12:48:18PM -0500, Gabriel Salzman wrote:
 Hello,
 
 I am wondering if anyone has experience or advice for my current situation:
 
 I have a native dataset of a ~50kD protein complex at 2.4angstroms. One of 
 the members of the complex (~10kD) was able to be found easily by molecular 
 replacement. The space group was determined as P65. I was able to use 
 phenix.mr_rosetta to generate a partial model of the other member of the 
 complex (~half of this 40kD protein). When I do molecular replacement for 
 these two members of the complex in PHASER, the TFZ score is ~20-30. 
 Therefore, I am confident that this partial model is accurate.
 
 I recently soaked some complex crystals in a variety of heavy atoms to obtain 
 experimental phase information. My best signal is from an iodide soak. These 
 crystals diffracted to 3 angstroms and I can see a very weak but significant 
 anomalous signal to 5-4.5 angstroms. I have used autoSHARP to determine that 
 I have one site of ~99% occupancy and 4 other sites of 50% occupancy. I am 
 inclined to think these other sites are not true. My density modified map 
 seems to me to be better if I only include the single high-occupancy site in 
 the phase determination. Even with this weak experimental phase information, 
 my map is not good enough to see the substantial density from the piece of my 
 protein that was not included in the MR model. 
 
 I have also tried phenix.phaser_EP with mediocre results.
 
 I apologize for the long question, and I would greatly appreciate any 
 suggestions for how to combine all of these data into a workable map so I can 
 build out the rest of the structure.
 
 Thanks,
 Gabriel

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] autosharp error message

[Clemens Vonrhein]

Dear Almudena,

On Tue, Jan 14, 2014 at 07:34:21PM +0100, Almudena Ponce Salvatierra wrote:
 Dear all,
 
 I find this error message while running autosharp:
 
 No column FP of type F found in file
/usr/local/sharp/users/sharp/None.sharp/datafiles/xxx.mtz!
 
 Does anybody know how to solve it? I have tried several things but they did
 not work. I am quite new with this software so any suggestions are welcome!

When giving autoSHARP the reflection data in MTZ format, you also need
to tell it the column labels for the required items: F, SIGF, DANO,
SIGDANO and (optional) ISYM. Remember that MTZ files can contain more
than a single set of amplitudes, anomalous differences etc. It looks
as if you left the defaults (FP) while selecting your own MTZ file
that uses different column names? What does

  % mtzdmp /usr/local/sharp/users/sharp/None.sharp/datafiles/xxx.mtz

give you as column names (table towards the end)?

You can make your life easier by using a SCALEPACK formatted file
(*.sca) as input to autoSHARP: since that will only contain a single
dataset you don't need to specify anything else.

Hope that helps

Clemens

PS: you might want to ask on the sharp-discuss mailing list or have a
browse the SHARP/autoSHARP wiki for tutorials etc ... more info at
www.globalphasing.com/sharp/.

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] the f' and f'' of heavy cluster

[Clemens Vonrhein]

Dear Lisa,

[there is a SHARP discussion list at
http://www.globalphasing.com/mailman/listinfo/sharp-discuss]

On Thu, Nov 21, 2013 at 02:29:23PM +0800, LISA wrote:
 Dear All,
 
 I am running autosharp with a single wavelength data soked with Ta6Br12.
 This data collected at the wavelength of 1.254A. I told the autosharp the
 f' -20 and f''10.5. The autosharp result said these values are not correct?

Are these values from a fluorescence scan? or just theoretical values?

 How can we get the f' and f'' of this cluster?

By far the best: look at the processed fluorescence scan for your
crystal. If you didn't do one (unlikely) or you don't have it any more
(unlucky) you can use calculated values (e.g. with the CCP4 crossec
program).

Remember that for clusters there are two steps you have to be
concerned about: (1) HA detection and (2) HA refinement. SHARP itself
implements a 'spherical cluster' approach for the second tasks
(ie. once you have some sites), which can be very powerful if your
clusters are truly disordered. See

  http://www.globalphasing.com/sharp/manual/appendix3.html#generalSPHCLUSTER
  http://www.globalphasing.com/sharp/manual/chapter4.html#CLISTspecify

Often you will need to use a more manual/traditional approach here:

 * finding the HA positions by running e.g. SHELXD by hand instead of
   as part of an automatic pipeline

 * running the HA parameter refinement and phasing program with the
   correct parametrisation for your cluster (see above), followed by
   manual inspection of LLG residual maps to find minor sites and
   decide if your cluster is ordered/disordered.

 * deciding on the correct hand/enantiomorph

 * doing density modification, averaging, model building etc.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] post to ccp4bb

[Clemens Vonrhein]

Hi,

On Tue, Jul 23, 2013 at 10:11:04AM -0500, Engin Ozkan wrote:
 Isn't the reported Mean(I/sigI) in the reported Aimless table for
 the merged/averaged reflections (because that is what we are
 discussing about)?

Yes - according to

1. 
http://ccp4wiki.org/~ccp4wiki/wiki/index.php?title=Symmetry%2C_Scale%2C_Merge#Analysis_by_resolution

   ... the average signal/noise after averaging symmetry-related
   observations  I/σ(I) , labelled Mn(I)/sd(Mn(I)) in the
   Aimless table, ...

2. section 3.2.1 of http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3689523/

   ... from the average signal-to-noise ratio of the merged
   intensities as a function of resolution. ... the average intensity
   over symmetry mates I h is divided by its estimated error σ(I)
   and this ratio is averaged in resolution bins [reported as Mn(I/sd)
   in the program output].

So the average (denoted by  and  in the output) is over all unique
reflections within a resolution bin - after all, those stats are
reported within resolution bins. And each unique reflection is the
result of (weighted) merging of all its measurements.

... as far as I understand this ...

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] is there any version of xia2 or XDSME compatible with new XDS?

[Clemens Vonrhein]

Dear all,

sorry for the only half-related reply, but ...

On Wed, Jul 03, 2013 at 09:20:56AM +0100, Rain Field wrote:
 Is there any similar software available?

... since you asked for similar software: the autoPROC release from
19th June is compatible. See

  http://www.globalphasing.com/autoproc/
  
http://www.globalphasing.com/autoproc/ReleaseNotes/ReleaseNotes_autoproc-1.0.2.html

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Which program sequence to use in transforming from P1 to orthorhombic?

[Clemens Vonrhein]

Hi Phil,

what I tend to do if I want to stay as close as possible to the
original data:

 scala hklin some.mtz hklout tmp.mtz end_ip
 RUN 1 BATCH 1 TO 
 ONLYMERGE
 ANALYSE NONORMAL NOPLOT
 SDCORR NOREFINE FIXSDB NOADJUST BOTH 1.0 0.0 0.0
 INITIAL UNITY
 REJECT 999.9 ALL 999.9
 END
 end_ip

 truncate hklin tmp.mtz hklout other.mtz end_ip
 SCALE 1.0
 ANOM NO
 END
 end_ip

... maybe with NOTRUNCATE as well.

This is a bit older ... so there might be a way of achieving the same
with the more modern aimless and ctruncate?

Cheers

Clemens

On Tue, Feb 12, 2013 at 06:52:19PM +0100, Phil Evans wrote:
 Ah I'm not sure about that. It may be possible to tell ctruncate not to do 
 this. Actually if you started with Fs you don't want to truncate the data. 
 Maybe use old truncate with the notruncate option
 
 Phil
 
 Sent from my iPad
 
 On 12 Feb 2013, at 18:48, Ethan Merritt merr...@u.washington.edu wrote:
 
  On Tuesday, February 12, 2013 12:39:57 am Phil wrote:
  Scale constant in Aimless or Scala should do it. I should probably make 
  that automatic.
  
  scale constant did indeed persuade aimless/scala to run.
  However, what seems to have happened is that aimless/scala expanded the 
  original
  [I, SIGI] into [I+, SIGI+] [I-, SIGI-], but all the [I-, SIGI-]  entries 
  were
  filled in as zero.  When ctruncate runs, it segfaults on a divide by zero 
  error.
  If I filter out the +/- columns and run ctruncate again, all is well.
  So aside from anything else, I think ctruncate needs some sanity checks for
  all-zero columns.
  
 Ethan
  
  
  
  I should probably also add a CIF reader to Pointless. Is there a good 
  (easy) C++ one out there?
  
  Phil 
  
  Sent from my iPad
  
  On 12 Feb 2013, at 08:08, Jens Kaiser kai...@caltech.edu wrote:
  
  Ethan,
  The last time I attempted similar things, I had to run rotaprep to
  convince scala of using most things that did not come directly out of
  mosflm, but that was before the pointless days. 
  As the reflections are already scaled in P1, I would consider it safe
  to rely on the Pointless Rmerge -- but that's just a guess (and you
  can't do much with the data downstream). I would assume sftools might be
  able to merge the reindexed file output by pointless.
   Nevertheless, if I were faced with the same problem nowadays, I would
  convert to a shelx hkl file and use xprep for the merging and statistics
  -- that's painless.
  
  Cheers,
  
  Jens
  
  On Mon, 2013-02-11 at 13:56 -0800, Ethan Merritt wrote:
  Hi all,
  
  I've downloaded a structure factor file from the PDB that presents
  itself as being triclinic.  It contains F, sig(F), and Rfree only.
  The P1-ness of this structure is dubious, however.
  
  Pointless is 99.6% sure it's orthorhombic and puts out an mtz file
  in P212121 containing 
I SIGI BATCH M/ISYM
  
  where the batch numbers are all 1 and ISYM runs from 1 to 8.
  So far so good, but now I'm stuck.  I can't persuade Scala
  or Aimless to merge the symmetry mates and report a merging
  R factor.Is there a trick to this?  Some other program sequence?
  
Ethan
  
  -- 
  Ethan A Merritt
  Biomolecular Structure Center,  K-428 Health Sciences Bldg
  University of Washington, Seattle 98195-7742

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] P321 space group reindex problem

[Clemens Vonrhein]

Hi Fred,

On Wed, May 30, 2012 at 08:55:35AM +0200, Vellieux Frederic wrote:
 For practical purposes, a derivative is considered non isomorphous
 when the differences in unit cell parameters exceed ca. 1% (this is
 because if you take 2 crystals from the same crystallisation drop
 and collect and process diffraction crystals from these 2 crystals,
 you will never get exactly the very same values for the unit cell
 parameters; non-isomorphism effects start at ca. 1% change and
 you'll never get 2 perfectly isomorphous crystals - even if you
 collect diffraction data twice from the same crystals you will not
 get perfect isomorphism).
 
 From the values mentioned, 1% of the cell parameters of the native
 for a and b is 1.81 Angstroem and for c 1.1 Angstroem (the angles do
 not matter for a trigonal space group).
 
 Had you obtained a value for a, b larger than ca. 183 Angstroem, or
 below ca. 109.2 Angstroem (only in the direction indicated by the
 changes mentioned in your mail - I ignored changes in the opposite
 direction) then you would have been able to say that the crystals
 were non-isomorphous to each other. For me they are isomorphous to
 each other and I ignore these small differences in unit cell
 parameters.

I would be careful with the (popular) percentage-rule here: the
absolute value of cell differences is much more important. At least if
we assume that the change in cell parameters roughly corresponds with
a shift in actual atoms.  If you have a 1000A cell then a 1%
difference could mean a shift of 10A ... clearly, a helix moved 10A
away results in something completely different. But with a cell of 20A
you could have a 0.2A shift, which you might hardly notice.

See eg. 5.2 in Garman  Murray (2003):

  http://journals.iucr.org/d/issues/2003/11/00/ba5042/index.html

which shows

  5.2. Non-isomorphism

One of the biggest problems of heavy-atom derivatization is that
incorporation of a heavy atom into the lattice often induces a
change in the unit cell away from the native crystal values,
i.e. the derivatized crystal is non-isomorphous to the native
crystals. The heavy atom may perturb the arrangement of protein
molecules in the crystal or distort the protein molecule, causing
a change in unit-cell lengths. Note, however, that it is also
possible for the protein to move within the original unit cell
(resulting in a different sampling of the molecular
transform). The same unit cell is thus a necessary but not
sufficient condition for isomorphism.

Crick  Magdoff (1956[Crick, F. H. C.  Magdoff,
B. S. (1956). Acta Cryst. 9, 901-908.]) calculated that a 0.5 Å
change in all three unit-cell edges of a 100 Å cubed unit cell
would change the diffraction intensities by an average of 15% in a
3 Å resolution sphere. The predicted intensity changes induced by
non-isomorphism increase at higher resolution. When faced with a
non-isomorphous derivative, it is the absolute change in the cell
which should be considered compared with the working resolution,
rather than the relative change, i.e. a change of 1.0% in a 100 Å
unit cell edge has a similar effect to that of a 0.5% change in a
200 Å unit cell edge, if compared at similar resolutions. As a
general rule of thumb, a change in cell dimensions of dmin/4 is
tolerable, where dmin is the resolution limit (Drenth,
1999[Drenth, J. (1999). Principles of Protein X-ray
Crystallography, 2nd ed. Berlin: Springer-Verlag.]). For instance,
for 2.5 Å data, a 0.6 Å change in the unit cell might be
acceptable, whereas at 3.5 Å this could rise to 0.8 Å.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] problem in scaling the Zn-MAD data

[Clemens Vonrhein]

Hi,

On Wed, Apr 04, 2012 at 02:07:58PM -0700, Deepthi wrote:
 Hello everyone
 I have a problem scaling the MAD data which was collected a week ago.The
 data was collected at 1.5A resolution using three wavelengths for Zn-MAD
 experiments. Scaling the data for MAD experiments, the number of rejections
 and chi2 values were very high even after adjusting the error-scale factor
 and error model. The space group i used was p312 which i obtained by
 running a self-rotation function in MOLREP. When i scale my data using p312
 spacegroup the chi2 and rejections were huge. But he data was scaling well
 in p321 spacegroup. can anyone explain whats going on?

When you say 'Scaling the data for MAD experiments': do you mean
scaling the various scans for your 3-wvl MAD data in a single scaling
job? Unless you already took care of this during data integration,
remember that your separate scans could have been indexed differently
and therefore don't match up. See eg.

  http://www.ccp4.ac.uk/html/reindexing.html

for some lookup-tables in P312 and P321. You can use the CCP4 program
'reindex' on MTZ files if needed.

But I guess most modern data-processing and scaling programs will take
care of that automatically anyway?

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Structure Determination combining X-ray Data and NMR

[Clemens Vonrhein]

On Fri, Jan 06, 2012 at 04:23:03PM +0800, ?? wrote:
 Also, there is one more information I forgot to mention---I also have the
 NMR assignment(HNCACB spectrum) of the protein, is it possible to combine
 the NMR data in my refinement?

Yes - we did something a few years back for the structure of the human
voltage-dependent anion channel (slightly more reflections, but lower
resolution) using a combination of Se-MET phases (SHARP), NMR and
secondary-structure restraints in refinement (BUSTER). See

  Monika Bayrhuber, Thomas Meins, Michael Habeck, Stefan Becker, Karin
  Giller, Saskia Villinger, Clemens Vonrhein, Christian Griesinger,
  Markus Zweckstetter, Kornelius Zeth(2008): Structure of the human
  voltage dependent anion channel. Proc. Nat. Acad. Sci. USA 105:
  15370-15375. 

or

  http://www.pnas.org/content/105/40/15370.full

It contains a fair amount of background info about methods.

Cheers

Clemens

 
 Regards,
 
 On Fri, Jan 6, 2012 at 4:14 PM, ?? shangyuan5...@gmail.com wrote:
 
  Dear All,
 I have a set of 3.2A data containing only 3000 reflections. From the
  SAD phasing and iterative modeling and density modification, I get a
  preliminary structure with bad geometric conformations(~8/160 ramachandran
  outliers in Coot). After Phenix MLHL refinement, the geometry is still bad
  with (10% ramachandran outliers and 25% Rotamer outliers), and the
  B-factors are all too high(all between 80 to 170, average ~120), and
  R-factor/R-free have a value of 0.328/0.326.
The poor geometry of my model and the unusual B-factors indicates there
  are still a lot improvement in my model. The question is, as I only have
  ~3000 reflections, and the atoms in the sequence is around 1000, and each
  atom there are 4 parameters to be refined(X,Y,Z,B-factor,
  assuming occupancy is 1), so how to refine my model to avoid
  over-refinement? Should I trust the electron-density map of the refined mtz
  data, or should I adjust the local geometries using Coot rotamers tools?
  How to set a reasonable B-factor values in the refinement?
 
  Best Regards,
  Yuan
 

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] ncs axis translation

[Clemens Vonrhein]

Hi Tommi,

I'm not sure, but do you already have the mid-point and want to get RT
operators only?

Or do you have the rotational component and the translation is
missing? In that case you could try GETAX (also available from the
CCP4i gui) which takes the rotation as input (plus a rough description
of shape/size) and searches for the translation in some electron
density map - as long as this is Cn/Dn symmetry (closed NCS).

If you only have the mid-point of 2 sites: check self-rotation
function for a 2-fold axis orientation and then try this mid-point
plus other pairs (symmetry-equivalent) ... ?

Cheers

Clemens

On Fri, Nov 25, 2011 at 10:55:49AM +0200, Tommi Kajander wrote:
 Hi, anybody have a script to to find a translation for 2-fold NCS (rotaion) 
 axis based on the center of NCS symmetry..?? (ie to the midpoint of line 
 between to heavy atoms)
 This search option doenst seem to be endoed anywhere (why??) ...stupid me...
 
 Thanks,
 tommi

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Merging with CAD files

[Clemens Vonrhein]

On Sun, Nov 06, 2011 at 05:35:29AM +, Ramanuj Banerjee wrote:
I used CAD for merging datasets during MIR. I faced the same problem. The 
 solution is: the datasets you are trying to merge should have different 
 labels i.e if dataset 1 has labels: F and SigF, dataset 2 should be F_d1 and 
 SigF_d1. Mention the labels during the cad run. 

That might make life easier - but a simple

  cad hklin1 one.mtz hklin2 two.mtz hklout onetwo.mtz e
  LABI FILE 1 E1=F  E2=SIGF
  LABO FILE 1 E1=F1 E2=SIGF1
  LABI FILE 2 E1=F  E2=SIGF
  LABO FILE 2 E1=F2 E2=SIGF2
  e

can do something similar. That is for 'merging' two MTZ files
(ie. gluing together columns), not merging two datasets into one (see
pointless/scala answers).

You can also do

  cad hklin1 one.mtz hklout one_tmp.mtz e
  LABI FILE 1 E1=FE2=SIGF
  LABO FILE 1 E1=Fcmb E2=SIGFcmb

  cad hklin1 two.mtz hklout two_tmp.mtz e
  LABI FILE 1 E1=FE2=SIGF
  LABO FILE 1 E1=Fcmb E2=SIGFcmb
  e

  mtzutils hklin1 one_tmp.mtz hklin2 two_tmp.mtz hklout onetwo.mtz e
  UNIQ
  e

which will fill up missing data in one file with the data present in
another MTZ file.

There's not a lot one can't achieve with those low-level utilities in
CCP4, but be careful of the order of doing things (both in CAD and
MTZUTILS), sorting issues etc. it can be very educational to have a
look at the input files and the output file with

  % mtzdmp some.mtz -n -1  some.mtzdmp

to check it does exactly what you want to do.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Archiving Images for PDB Depositions

[Clemens Vonrhein]

Hi James,

scary ... I was just looking at exactly the same thing (P21 with
beta~90), using the same tool (POINTLESS).

Currently I'm going through the structures for which images can be
found ... I haven't gone far through that list yet (in fact actually
only the first one), but this first case should indeed be in a higher
spacegroup (P 2 21 21).

As you say (and that's what Graeme looks for): finding 'over-merged'
datasets can be a bit more tricky ... once the damage is done. I have
the hunch that it might happen even more often though: we tend to
look for the highest symmetry that still gives a good indexing score,
right?  Otherwise we would all go for P1 ...

Some other interesting groups for under-merging:

 * orthorhombic with a==b or a==c or b==c (maybe tetragonal?)

 * trigonal (P 3 etc) when it should be P 6

 * monoclinic with beta==120

A few cases for each of those too ... all easy to check in
ftp://ftp.wwpdb.org/pub/pdb/derived_data/index/crystal.idx and then
(if structure factors are deposited) running POINTLESS on it (great
program Phil!).

Cheers

Clemens

On Thu, Nov 03, 2011 at 12:00:33PM -0700, James Holton wrote:
 I tried looking for such evil symmetry problem examples some time
 ago, only to find that primitive monoclinic with a 90-degree beta
 angle is much more rare than one might think by looking at the PDB.
 About 1/3 of them are in the wrong space group.
 
 Indeed, there are at least 366 PDB entries that claim P2-ish, but
 POINTLESS thinks the space group of the deposited data is higher
 (PG222, C2, P6, etc.).  Now, POINTLESS can be fooled by twinned
 data, but at least 286 of these entries do not mention twinning.  Of
 these, 40 explicitly list NCS operators (not sure if the others used
 NCS?), and 35 of those were both solved by molecular replacement an
 explicitly say the free-R set was picked at random.  These are:
 
 Now, I'm sure there is an explanation for each and every one of
 these.  But in the hands of a novice, such cases could easily result
 in a completely wrong structure giving a perfectly reasonable Rfree.
 This would happen if you started with, say, a wrong MR solution, but
 picked your random Rfree set in PG2 and then applied NCS.  Then
 each of your free hkls would actually be NCS-restrained to be the
 same as a member of the working set.  However, I'm sure everyone who
 reads the CCP4BB already knew that.  Perhaps because a discerning
 peer-reviewer, PDB annotator or some clever feature in our modern
 bullet-proof crystallographic software caught such a mistake for
 them. (Ahem)
 
 Of course, what Graeme is asking for is the opposite of this: data
 that would appear as nearly PG222, but was actually lower
 symmetry.  Unfortunately, there is no way to identify such cases
 from deposited Fs alone, as they will have been overmerged.  In
 fact, I did once see a talk where someone managed to hammer an NCS
 7-fold into a crystallographic 2-fold by doing some aggressive
 outlier rejection in scaling.  Can't remember if that ever got
 published...
 
 -James Holton
 MAD Scientist
 
 On 11/2/2011 1:33 AM, Graeme Winter wrote:
 Hi Ed,
 
 Ok, I'll bite: I would be very interested to see any data sets which
 initially were thought to be e.g. PG222 and scale OK ish with that but
 turn out in hindsight to be say PG2. Trying to automatically spot this
 or at least warn inside xia2 would be really handy. Any
 pseudosymmetric examples interesting.
 
 Also any which are pseudocentred - index OK in C2 (say) but should
 really be P2 (with the same cell) as the missing reflections are in
 fact present but are just rather weaker due to NCS.
 
 I have one example of each from the JCSG but more would be great,
 especially in cases where the structure was solved  deposited.
 
 There we go.
 
 Now the matter of actually getting these here is slightly harder but
 if anyone has an example I will work something out. Please get in
 touch off-list... I will respond to the BB in a week or so to feed
 back on how responses to this go :o)
 
 Best wishes,
 
 Graeme

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Archiving Images for PDB Depositions

[Clemens Vonrhein]

On Thu, Nov 03, 2011 at 04:13:44PM -0400, Bryan Lepore wrote:
 not sure I follow this thread, but this table might be interesting :
 
 http://journals.iucr.org/d/issues/2010/05/00/dz5193/dz5193sup1.pdf
 
 from:
 
 Detection and correction of underassigned rotational symmetry prior to
 structure deposition
 B. K. Poon, R. W. Grosse-Kunstleve, P. H. Zwart and N. K. Sauter
 Acta Cryst. (2010). D66, 503-513[ doi:10.1107/S0907444910001502 ]

Oh yes, that is relevant and very interesting. As far as I understand
it, the detection of higher symmetry is based on the atomic
coordinates and not structure factors though (please correct me if I'm
wrong here).

At least some of the cases for which the deposited structure factors
strongly suggest a higher symmetry don't seem to be detected using
that papers approach (I can't find them listed in the supplemental).

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] To archive or not to archive, that's the question!

[Clemens Vonrhein]

Dear Vaheh,

On Mon, Oct 31, 2011 at 03:18:07PM +, Oganesyan, Vaheh wrote:
 But to store those difficult datasets to help the future software
 development sounds really farfetched.

As far as I see the general plan, that would be a second stage (deposit
all datasets) - the first one would be the datasets related directly
to a given PDB entry.

 This assumes that in the future crystallographers will never grow
 crystals that will deliver difficult datasets.

Oh sure they will. And lots of those datasets will be available to
developers ... being thrown a difficult problem under pressure is a
very good thing to get ideas, think out of the box etc. However,
developing solid algorithms is better done in a less hectic
environment with a large collection of similar problems (changing only
one parameter at a time) to test a new method.

 If that is the case and in 10-20-30 years next generation will be
 growing much better crystals then they don't need such a software
 development.

They'll grow better crystals for the type of project we're currently
struggling with, sure. But we'll still get poor crystals for projects we
don't even attempt or tackle right now.

Software development is a slow process, often working on a different
timescale than the typical structure solution project (obvious there
are exceptions). So planing ahead for that time will prepare us.

And yes, it will have an impact on the biology then. It's not just the
here and now (and next grant, next high-profile paper) we should be
thinking about.

 Am I missing a point of discussion here?

One small point maybe: there are very few developers out there - but a
very large number of users that benefit from what they have
done. Often the work is not very visible (It's just pressing a button
or two ... so it must be trivial!) - which is a good thing: it has to
be simple, robust, automtic and useable.

I think if a large enough number of developers consider depositing
images a very useful resource for their future development (and
therefore future benefit to a large number of users), it should be
seriously considered, even if some of the advertised benefits have to
be taken on trust.

Past developments in data processing have had a big impact on a lot of
projects - high-profile or just the standard PhD-student nightmare -
with often small return for the developers in terms of publications,
grants or even citations (main paper or supplementary material).

So maybe in the sprit of the festive season it is time to consider
giving a little bit back? What is there to loose? Another 20 minutes
additional deposition work for the user in return for maybe/hopefully
saving a whole project 5 years down the line? Not a bad investment it
seems to me ...

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Archiving Images for PDB Depositions

[Clemens Vonrhein]

Dear Adrian,

On Mon, Oct 31, 2011 at 06:29:50PM +0200, Adrian Goldman wrote:
 I have no problem with this idea as an opt-in. However I loathe being forced 
 to do things - for my own good or anyone else's. But unless I read the tenor 
 of this discussion completely wrongly, opt-in is precisely what is not being 
 proposed.  

I understood it slightly different - see Gerard Bricogne's points in

  https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind1110L=CCP4BBF=S=P=363135

which sounds very much like an opt-in? Such a starting point sounds
very similar to that we had with initial PDB submission (optional for
publication) and then structure factor deposition.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Linux vs MacOS for crystallographic software

[Clemens Vonrhein]

Hi Sebastiano,

On Thu, Sep 29, 2011 at 09:52:44AM +0200, Sebastiano Pasqualato wrote:
  does this mean that SHARP works on a Mac?

yes (since 2004), same for BUSTER (since 2009) and autoPROC.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Help with deleting reflections from an mtz file

[Clemens Vonrhein]

Dear Eleanore,

Ooops - the ccp4bb rejects attachements with *.sh ending? Let's try it
again with *.sh.txt ...

The attached script works for me - fantastic what one can do with
SFTOOLS :-)

Cheers

Clemens

On Wed, Aug 17, 2011 at 04:55:59PM +0100, Eleanor Dodson wrote:
 There are 2 rogue reflections in a data set I have here. How can I
 eliminate them?
 I thought sftools did this but i cant seem to get the syntax right.
 Short of dumping the whole file, using an editor, then
 reconstructing it I am stuck..
 
 Eleanor

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***
#!/bin/sh

# vonrh...@globalphasing.com 20110816

# input MTZ file
mtz=in.mtz

# output MTZ file
out=out.mtz

# space-separated list of h,k,l indeices to reject
hkls=1,0,-25 1,0,-24

##
[ X$mtz = X ]  echo  ERROR: please define input MTZ file  exit 1
[ X$out = X ]  echo  ERROR: please define output MTZ file  exit 1
[ ! -f $mtz ]  echo  ERROR: input file $mtz not found  exit 1
[   -f $out ]  echo  ERROR: output file $mtz exists (please remove and 
re-run)  exit 1

for hkl in $hkls
do
  sftools e
mode batch
read $mtz
`echo $hkl | awk '{i=split($1,a,,)
print select index H =,a[1]
print select index K =,a[2]
print select index L =,a[3]
}'`
select invert
write __$$.mtz1
select all
quit
Y
e
  mv __$$.mtz1 __$$.mtz
  mtz=__$$.mtz
done
mv -v $mtz $out

Re: [ccp4bb] Trouble compiling ccp4 in osx: stalled at checking for working non-fixed mmap

[Clemens Vonrhein]

Dear Miguel,

yes, I hit the same issue - and for me the machine then completely
freezes and needs a reboot.

The patch for configure:

--- configure.orig  2011-05-05 16:07:55.0 +0100
+++ configure   2011-07-21 16:05:57.0 +0100
@@ -3206,6 +3206,9 @@
   if test $system = sunos64; then
 xopts=$xopts --enable-64-bit=yes
   fi
+  case `uname` in
+Darwin*) xopts=$xopts --disable-mmap;;
+  esac
   if test $with_rxdispencer = yes; then
 rxdir=lib/rxdispencer
 echo 

Works for me.

Cheers

Clemens

On Tue, Aug 02, 2011 at 01:59:59PM +0200, Miguel Ortiz Lombard?a wrote:
 Dear all,
 
 I have successfully compiled ccp4 6.2.0 on an OSX (10.6.8) machine but
 I'm failing to do the same in a different one, though with the same
 system setup. In the machine giving me trouble I want to install ccp4 in
 a nfs disk mounted. When I run configure using exactly the same input as
 in the first, successful machine:
 
 ./configure --with-x Darwin64
 
 the configuration script stalls at:
 
 checking for working non-fixed mmap...
 
 I haven't found any obvious reason for that. This problem is independent
 of whether I use gnu or intel compilers.
 
 Any ideas ?
 
 Best regards,
 
 
 -- 
 Miguel
 
 Architecture et Fonction des Macromolécules Biologiques (UMR6098)
 CNRS, Universités d'Aix-Marseille I  II
 Case 932, 163 Avenue de Luminy, 13288 Marseille cedex 9, France
 Tel: +33(0) 491 82 55 93
 Fax: +33(0) 491 26 67 20
 mailto:miguel.ortiz-lombar...@afmb.univ-mrs.fr
 http://www.afmb.univ-mrs.fr/Miguel-Ortiz-Lombardia
 
 -- 
 This message has been scanned for viruses and
 dangerous content by MailScanner, and is
 believed to be clean.

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Newbie Installation Question

[Clemens Vonrhein]

There is another potential problem in the ccp4.setup-sh file: the
lines

  if test LD_LIBRARY_PATH; then

and

  if test DYLD_LIBRARY_PATH; then

(which always evaluate to true) should be

  if test $LD_LIBRARY_PATH; then

and

  if test $DYLD_LIBRARY_PATH; then

Otherwise you end up with LD_LIBRARY_PATH settings that could contain
a dangling ...: - not sure if that could mess up your environment?

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Permissions and ownerships in ccp4 6.2.0

[Clemens Vonrhein]

Dear all,

ideally, permissions should be either

  rw-r--r-- (0644)

or (for files that need to be executed as well as directories)

  rwxr-xr-x (0755)

One quick fix:

  find . -type d -exec chmod -v 0755 {} ;
  find . -type f -exec chmod -v 0755 {} ;

but that last command makes every single file executable, which is
rather ugly (but doing a selective chmod 0755/0644 is a bit tricky
with all those script files - some need to be executed but others
arent). I don't see a need to have read-only files like all the CIF
dictionaries with permission 0755.


The correct permissions can only be set during packaging
unfortunately.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Ghostview in CCP4 configure interface

[Clemens Vonrhein]

Hi Mar,

are you using your own compiled version of CCP4? Or pre-compiled
binaries? Maybe there is some 32/64-bit issue at play here ... I would
expect your own compiled (on that machine) binaries to work
best/correclty. But I'm just guessing ...

Cheers

Clemens



On Mon, Jun 20, 2011 at 05:25:09PM +0200, Mark J van Raaij wrote:
 Dear Clemens, All,
 
 when trying to open the .plt files I get Bad plot84 file format ... anyone 
 has observed this and found a solution?
 xplot84driver is installed (which xplot84driver gives 
 /usr/local/ccp4-6.1.3/bin/xplot84driver) and the files appear ok, because I 
 can convert them correctly to .ps with pltdev and view them correctly.
 I am using MacOSX 10.6.7, CCP4 6.1.3 (didn't try updating yet, because it is 
 a relatively small problem).
 
 Greetings,
 
 Mark J van Raaij
 Laboratorio M-4
 Dpto de Estructura de Macromoleculas
 Centro Nacional de Biotecnologia - CSIC
 c/Darwin 3, Campus Cantoblanco
 E-28049 Madrid, Spain
 tel. (+34) 91 585 4616
 http://www.cnb.csic.es/content/research/macromolecular/mvraaij
 
 
 
 
 On 17 Jun 2011, at 19:50, Clemens Vonrhein wrote:
 
  On Fri, Jun 17, 2011 at 06:25:14PM +0100, Ian Tickle wrote:
  Hi, the '.plt' files are in an internal CCP4 format  cannot be visualised
  directly.  You have to convert them first to PostScript (.ps) format using
  the 'pltdev' program; then you can visualise them with ghostscript (or any
  other PS viewer).
  
  Or you can view them directly with the 'xplot84driver' program which
  comes as part of CCP4?
  
  Cheers
  
  Clemens
  
  -- 
  
  ***
  * Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
  *
  *  Global Phasing Ltd.
  *  Sheraton House, Castle Park 
  *  Cambridge CB3 0AX, UK
  *--
  * BUSTER Development Group  (http://www.globalphasing.com)
  ***

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Ghostview in CCP4 configure interface

[Clemens Vonrhein]

On Fri, Jun 17, 2011 at 06:25:14PM +0100, Ian Tickle wrote:
 Hi, the '.plt' files are in an internal CCP4 format  cannot be visualised
 directly.  You have to convert them first to PostScript (.ps) format using
 the 'pltdev' program; then you can visualise them with ghostscript (or any
 other PS viewer).

Or you can view them directly with the 'xplot84driver' program which
comes as part of CCP4?

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] autoSHARP/SHELXD : Scatter plot program for OS X

[Clemens Vonrhein]

Dear Raspudin,

you should already have that binary in

  /where/ever/sharp/helpers/darwin/plotmtv

Cheers

Clemens

On Tue, Nov 16, 2010 at 08:52:10PM +0100, Raspudin wrote:
 Dear all,
 
 Could anyone please suggest me a program to open scatter plots ( .mtv  file 
 format) from SHELXD/autoSHARP run in OS X Platform.
 
 Thank you,
 Raspudin

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] [RANT] Publication Data Formats

[Clemens Vonrhein]

Hi,

On Wed, Nov 17, 2010 at 01:42:40AM -0800, James Stroud wrote:
 http://onlinelibrary.wiley.com/doi/10.1002/pmic.200700038/suppinfo
 
 You'll see in the available PDF file Tables S1-S3. Were I to look for any 
 significant amount of time, I could find much more egregious examples.
 
 For this particular example, your eyes may deceive you into thinking that the 
 PDF file can be parsed and the data represented in the tables extracted with 
 a script of some sort. But, if you have the patience, go to Table S3 and 
 start selecting text at Accession Number in the heading. You'll find that 
 the selection goes down that column only about half way and then begins 
 selecting at the next column, Swissprot Identifier.

Pick a better PDF viewer: with my version of xpdf (on Ubuntu 10.04) I
can easily select that table over three pages and get a reasonably
good looking ASCII representation of it. Takes about 10 seconds ...

Acrobat reader is not very good for selecting text in PDF files. I
don't know about others, but xpdf is really good at it.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] diverging Rcryst and Rfree

[Clemens Vonrhein]

Dear Ian,

On Tue, Oct 26, 2010 at 05:15:50PM +0100, Ian Tickle wrote:
 Yes! - the critical piece of information that we're missing is the
 proportion of *all* structures that come from SG centres.  Only
 knowing that can we do any serious statistics ...

The point I was trying to make was not to blame SG centres (or
comparing them with other groups) - I was concerned about technology
mainly.

Clearly, there was some problem at the point of deposition. I would
have thought that especially in structures coming from SG centres
there would be technology in place on both sides (at the SG centre and
at the PDB site) to catch unusual values like an overall Rmerge of
0.99?

Cheers

Clemens

PS: according to some simple search out of those 3026 entries 1473 are
from SG centres and 1553 not.

 


 -- Ian
 
 On Tue, Oct 26, 2010 at 5:07 PM, Frank von Delft
 frank.vonde...@sgc.ox.ac.uk wrote:
   b) very large Rmerge values:
 
       Rmerge  Rwork  Rfree  Rfree-Rwork Resolution
      -
       0.9990 0.1815 0.2086    0.0271     1.80  SG center, unpublished
       0.8700 0.1708 0.2270    0.0562     1.96  unpublished
       0.7700 0.1870 0.2297    0.0428     1.56
       0.7600 0.2380 0.2680    0.0300     2.50  SG center, unpublished
       0.7000 0.1700 0.2253    0.0553     1.71
       0.6400 0.2179 0.2715    0.0536     2.75  SG center, unpublished
 
  The most disturbing to me is that of those with very large overall
  Rmerge values, 3 come from structural genomics centers.
 
  Is that less or more disturbing than that the other 50% come from not-SG
  centers?
 
  Of course, the authors themselves may be willing to help correct the obvious
  typos -- which will presumably disappear forever once we can finally upload
  log files upon deposition (coming soon, I'm told).
 
  On an unrelated note, it's reassuring to see sound statistical principles --
  averages, large N, avoidance of small number-anecdotes, and such rot --
  continue not to be abandoned in the politics of science funding, he said
  airily.
 
  phx
 

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Regarding space group P1, P21

[Clemens Vonrhein]

Dear Mohinder,

On Thu, Oct 21, 2010 at 01:05:42PM +0100, Mohinder Pal wrote:
 The question is , is it a good practice to solve this structure in
 P1 and P21 even if the data has higher symmetry?

On a slightly philosophical note regarding the final model (and not
necessarily the 'good practice' leading to it): shouldn't our model
describe the experiment (intensities from a crystal of given symmetry)
and not the other way round (changing the experimental data to make
modeling easier)?

Or maybe I'm too strict here ...

If your crystal has P21212 then I would model it this way: having a
compound on a 2-fold with half occupancy isn't really a problem
nowadays with modern refinement programs.

And yes: it might confuse molecular biologists downloading the PDB
file. And since their needs often dictate how we are supposed to
produce models for our experiments, the time might come where all
structures being refined in P1 with only the A chain deposited ;-)

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Regarding space group P1, P21

[Clemens Vonrhein]

Hi Herman,

On Thu, Oct 21, 2010 at 05:31:51PM +0200, herman.schreu...@sanofi-aventis.com 
wrote:
 If you process your data in a lower symmetry space group, you will have
 more unique reflections, since reflections which are related by the
 higher symmetry will be avaraged during scaling in a higher symmetry
 space group (i.e. a 2fold or 3fold axis), while in lower symmetry space
 groups they will not. So the observation to parameter ratio stays the
 same and is only depending on resolution and solvent content. 

True - if you count Miller indices as observations. But if you think
about information content than probably not (as you discuss below).

 The question one has to ask of course is: are these reflections really
 different, or are they the same only not averaged?

Yes - by merging we're getting better data (better error estimate on
the intensity due to higher multiplicity). So there isn't really
independent information in 50% of the reflections if e.g. going from
P21 to P1 - we've only increased the noise because the multiplicity of
each reflection has been reduced.

 In the latter case, you have more reflections, but not more
 information. As Ed mentions, using tight NCS restraints would in
 this case mimick the crystallographic symmetry.

Apart from the (good) NCS argument, one could go even further:

We could also just collect 36000 degree of data on a 7A Lysozyme
crystal and refine against completely unmerged data. After all, why
should we stop at removing only the some symmetry operators from our
data merging ... lets get rid of all of them including th x,y,z
operator and use unmerged data. Then we could refine Lysozyme with
anisotropic hydrogens and no restraints against 7A data since we have
a huge number of 'observations' ... right?

But seriously: there is a difference in having reflections (H, K, L)
and independent data (I, SIGI). Maybe we should talk more about
(independent observations)/parameters ratio in the same way we
look at depdencies of parameters (e.g. restraints on Bfactors etc).

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Regarding space group P1, P21

[Clemens Vonrhein]

Hi Ed,

On Thu, Oct 21, 2010 at 12:18:31PM -0400, Ed Pozharski wrote:
 Let's say I have a ligand on symmetry axes and so it appears in two
 conformations.  If I reduce symmetry, there are two possible scenarios.
 
 a. In lower symmetry, ligand still appears in two conformations.  Shall
 use higher symmetry.
 b. In lower symmetry, ligand appears to be in single conformation (this
 is what Mohinder says, if I am not mistaken).  In this case, the true
 symmetry is lower, and it is simply overwhelmed by the fact that most of
 the structure (but not all) obeys higher symmetry.

I think I understand what you're getting at: you have a lower symmetry
with a NCS axis that is basically perfectly aligned with the
corresponding crystallographic axis in the higher symmetry
spacegroup. And the only part of the model not obeying this NCS is the
ligand.

But then what about a water on a special position (2-fold with
occ=0.5)? If I remove that 2-fold from my spacegroup symmetry and
refine I get ... a single water with occupancy 1.0 ... or 2 waters
with occupancy 0.5? Hmmm, diffcult to decide on the true spacegroup
here ;-)

So it all depends

 * how clear the difference between high-symmetry/double conformation
   and low-symmetry/single-conformation is

 * how symmetrical the ligand is

 * how the refinement in the lower-symmetry spacegroup is done - since
   there is a real danger (in case it is the high-symmetry spacegroup
   after all) that because of model bias and poorer (independent
   observations)/parameter ratio what seems like a clear single
   conformation is difficult to confirm.

 I recall Bruce Foxman describing a b) case (I am sure there is more than
 one example) for a small molecule crystal, where a single heavy atom had
 higher symmetry than the rest of the molecule.

There is a recent nice example of a very interesting symmetry/disorder
siuation by Yves Muller: 2xgc. It took some time for me to get my head
around what it is in the PDB file and what it means ... but it's very
neat!

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Regarding space group P1, P21

[Clemens Vonrhein]

Hi,

 I think that when you say as if they were independent, you are begging 
 the question. You could say that refining in higher symmetry treats the  
 molecules as if they were the same. Further, it really assumes more to  
 posit that they are the same.

But we're still talking about crystals, right? The whole reason for
trying to crystallise our proteins/DNA/RNA is because we ideally want
a perfect arrangement of molecules. So taking as a starting hypotheses
the conservative approach that if the data really looks like P21 it
probably is P21 seems a good idea to me.

To me it is more a case of 

  refining in lower symmetry treats the molecules as if they were
  different

when initially we don't have an indication for it (from data
processing). Unless we take the fact that P1 will always give us lower
merging R-factors and better indexing scores as indication that
actually all our crystals are always P1 ... which they well might be,
but probably not within our experimental error.

 Really the crux I think is weighing what benefits one gets from
 treating the data in different ways. If one can know somehow that
 the molecules when treated as p1 differ from each other only as a
 function of experimental noise, there would be no reason to treat
 them as p1.

True: but how do you judge that those differences are within or
outside of experimental noise?

 On the other hand, if somehow a few sidechains became systematically
 different between molecules in the p1 cell, it *would* make sense to
 refine in p1, no?

What if by refining in P1 the parametrisation makes those side-chains
different in the first place? A poorly defined Lys side-chain suddenly
becomes two significantly different poorly defined side-chain?

Cheers

Clemens


-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Big difference between anomalous map and density modification map

[Clemens Vonrhein]

Dear Mousheng,

as others, I'm also a bit confused about the steps you did:

On Sun, Oct 17, 2010 at 02:29:12PM -0500, Wu, Mousheng wrote:
 Dear All,
 
 I have a MAD dataset at 4Å and shelxD can find clear-cut 10
 solutions (my protein has 12 methionines).

Good.

 I ran autosharp to refine them followed by density modification.

Yes.

 After density modification, I ran solvent flattening.

Confused ... what program/procedure did you use for what you call
density modification and what for solvent flattening?

 Then I calculated the anomalous map using the phase from sharp and
 the electron density map from solvent flattening.

More confusion: what do you mean with anomalous map? One might call
a map with anomalous differences and some phases such an anomalous
map ... but I guess you rather mean an electron density map with
phases from some procedure based on anomalous differences, right?

So you calculated an electron density map from SHARP: which map
coefficients did you use? It should be:

  fft hklin eden.mtz mapout eden.map e
  LABI F1=FB PHI=PHIB
  e

How did you calculate the electron density map after density
modification? If you did density modification within the
SHARP/autoSHARP system it should be

  fft hklin eden_flat_53.2pc.mtz mapout eden_flat_53.2pc.map e
  LABI F1=FBshasol PHI=PHIBshasol
  e

Note: do _not_ use any figure-of-merit column when claculating maps!
Nearly all programs will give you map coefficients (amplitude and
phase) that are already correctly weighted!

Note-2: SHARP/autoSHARP outputs slightly different columns - depending
if you want a map of the light-atoms-only or onw that shows the
average HA contribution as well. As a rule: use FB*/PHIB* for MAD/SAD
and Fcent*/PHIcent* for SIRAS/MIRAS.

 The anomalous map shows the density around these 10 selenium sites
 are clear and round.

Good.

 However, the density map from solvent flattening showed that only 4
 selenium sites have clear and round density.

Is the map after density modifcation or the one after density
modification plus solvent flattening? It is very unclear ...

 The density around these 4 sites clearly showed beautiful
 helices. surprisingly, other 6 selenium sites have poor density or
 no density at all.  The electron density around them is not very
 good and the predicted helical density is flat. I check the electron
 density before I ran solvent flattening. The result is same.  I am
 quite confused about the big difference from these two maps.

We first need to establish what steps you performed and how to see if
these are the actually correct maps to look at.

 I also try SnB to find the selenium sites. The solutions are same as
 those from ShelxD. But How to explain the poor density around the
 selenium sites in the density modification map?  Is there any
 problem with my selenium sites?

Remember that the purely HA-phased maps (i.e. directly after SHARP)
will be dominated by those heavy atoms ... so no surprise they are
nice and round Se density.

Se-MET (or S-Met for that matter) often has alternate conformations in
the side-chain: maybe your phases are actually better and therefore
model those side-chains now more accurately, i.e. as disordered
side-chains. If your environment around those sites is also disorderd
('flattened' helices) this is all not surprising.

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Big difference between anomalous map and density modification map

[Clemens Vonrhein]

Hi Mousheng,

[if it gets too SHARP-centric we can also move this discussion also to
sharp-discuss or our developers mailing list sharp-develop]

On Mon, Oct 18, 2010 at 08:33:47AM -0500, Wu, Mousheng wrote:
 my solvent content is supposed to be 65% (one molecule per
 ASU).

which seems to match your lowish resolution limit (as a general rule).

 Sharp automatically did density modification using 37.5% solvent
 content.

I see. Did you get a clear and significant difference in score for the
two hands? It not only tells you which is the correct hand, but a
large difference also tells you that one set of phases is
significantly better than the other and that there is real phasing
information in those.

 that is why I run solvent flattening directly from Sharp.

Did you use the tools within our interface? Probably ...

 More confusion: what do you mean with anomalous map? One might call a map 
 with anomalous differences and some phases such an anomalous map ... but I 
 guess you rather mean an electron density map with phases from some 
 procedure based on anomalous differences, right?
 
 you are right. the anomalous map I calculated is anomalous differences map 
 using FFT 

Ok - so you are comparing an anomalous difference map with an electron
density map? I'm not sure why, but:

 * the anomalous difference map (using phases from SHARP or your
   density modified phases) should only show the substructure,
   i.e. only the Se atoms and no light atoms

 * the electron density map after density modification shows
   everything: light and heavy atoms.

Why are you comparing those two maps? To check if your heavy atom
sites are where they should be or if all are correct?

If you trust your density modified phases (and you should, since they
show nice helical density): just calculate such an anomalous
difference map using your density modified phases ... it's a two
mouse-click operation in the SHARP/autoSHARP interface and should show
you peaks and their distance to your initial heavy-atom sites.

Are those 6 'poor' sites in any way different after SHARP refinement?
Much higher B-factor or lower occupancy? Do you have additional peaks
in your log-likelihood residual maps (showing the true position of
those potentially wrong sites or if they are split)? Do you have peaks
in the ANO residual maps exactly at the sites - showing wrong f
values (similar for f' values in ISO residual maps)? I.e.: is your HA
model as good as it could be?

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] mol rep help needed

[Clemens Vonrhein]

Hi David,

On Fri, Oct 01, 2010 at 11:40:13AM -0400, David Roberts wrote:
 My question - finally - how can I run automolrep with one dimer fixed,  
 looking for the location of the other 2 monomers (so basically I want to  
 fix a dimer as part of my solution, and then search for the other 2  
 molecules in the asu).

I found the latest MOLREP binary from

  http://www.ysbl.york.ac.uk/~alexei/molrep.html#installation

to work very well (thanks Alexei!). Something like:

  molrep_linux -f your.mtz \
   -m monomer.pdb \
   -mx fixed_dimer.pdb

It doesn't get simpler than that I guess ;-)

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***

Re: [ccp4bb] Heavy atom sites?

[Clemens Vonrhein]

Dear Pu,

sorry for the long(ish) answer ... handedness/enantiomorph, heavy atom
sites and consistency (while using SHARP/autoSHARP) are just one of my
favourites ;-)

On Fri, Aug 20, 2010 at 01:39:58PM +0100, Pu Gao wrote:
 I rechecked the sharp logs, and found that the original SAD sites
 were wrong, which had to be turned to its inverted ones for
 calculating the final density (while the SIRAS sites are correct,
 the final density was calculated directly from the original sites).

If you run autoSHARP including the density modification step (!) it
will try and detect which is the correct hand/enantiomorph. It is then
important to either take the last SHARP run (inverted hand) or the
last but one (original hand) when continuing to work with the current
heavy atom model in SHARP (e.g. to then do MAD+native or spherical
averaged heavy atom clusters etc). The correct set of heavy atom
parameters should be clearly stated in the log file from autoSHARP.

If you did run e.g. SHELXD by hand and then used those sites in SHARP
you could obviously have the wrong hand/enantiomorph (see George's
post).

I would generally recommend using the heavy atom sites from SHELXD in
an autoSHARP run: unless you do something more complex than SAD/MAD or
SIR(AS)/MIR(AS). It will then skip it's own heavy-atom detection step
(also using SHELXC/D) and use your supplied sites.

Using sites from your own HA-detection attempt is a good idea for
tricky cases (where you might have to run a lot of trials), cases
where the number of sites is very uncertain and you need to run lots
of slightly different trials ... or if you have very low resolution
data/signal (5A and lower). In any case: make your life easy and plug
them into autoSHARP up to the density modifcation step.

Also, if you run several phasing scenaria (SAD, another one SIRAS,
different derivatives etc) there is a very useful feature in SHARP
that allows input of external (prior) phase information - e.g. in form
of Hendrickson-Lattmann coefficients. This way you could use the SHARP
phases from your SIRAS job (e.g. nat+Hg) for a SAD run (Se-MET peak -
no sites known so far) and detect your Se sites in the log-likelihood
gradient maps ... on the same origina (!) as the other sites. Having
all sites on the same origina makes your life MUCH easier if you have
to deal with lots of derivatives, datasets and slightly different cell
dimensions.

You could also use your phases from a poor or partial MR solution in
the same way to detect your heavy atom sites. This feature (using
prior phase information from partial models, MR or other derivatives)
has been in SHARP since the very early days and has been useful to us
in a lot of cases. See e.g. the 2010 paper by Pietro Roversi
(http://scripts.iucr.org/cgi-bin/paper?ba5141) or some earlier
references:

  Kauppi B, Lee K, Carredano E, Parales RE, Gibson DT, Eklund H,
  Ramaswamy S. (1998). Structure of an aromatic-ring-hydroxylating
  dioxygenase-naphthalene 1,2-dioxygenase. Structure 6, 571-86.

and the manual:

  http://www.globalphasing.com/sharp/manual/chapter2.html#external
  
http://www.globalphasing.com/sharp/manual/chapter4.html#ExternalPhaseInformation

Cheers

Clemens

-- 

***
* Clemens Vonrhein, Ph.D. vonrhein AT GlobalPhasing DOT com
*
*  Global Phasing Ltd.
*  Sheraton House, Castle Park 
*  Cambridge CB3 0AX, UK
*--
* BUSTER Development Group  (http://www.globalphasing.com)
***