Re: [ccp4bb] twinning or not?

[Phil Evans]

You can get apparent twinning if neighbouring spots are overlapped in any direction, and that may be worse with a home source with larger beam divergence. In that case a weak reflection may be corrupted by a neighbouring strong reflection, leading to intensity statistics characteristic of twinningPhilSent from my iPadOn 25 Mar 2024, at 19:24, Thomas, Leonard M.  wrote:CAUTION: This email originated from outside of the LMB:
.-owner-ccp...@jiscmail.ac.uk-.
Do not click links or open attachments unless you recognize the sender and know the content is safe.If you think this is a phishing email, please forward it to phish...@mrc-lmb.cam.ac.uk
--





Hello,


I have run into a very odd situation.  Upon collection and data processing on a crystal on my home source all pointers seemed to show twinning.  Was not surprised since previous crystals under different but related conditions have showed significant
 twinning though the cell dimensions were different but not the resulting apparent space group.  


Now the odd thing, I sent the crystal to the synchrotron figuring I might be able to get some data that might help figure out what is going on.  Upon processing of the synchrotron data, SSRL 9-2, magically no twining was detected.  A couple of
 other crystals of the same compound though not from the same exact condition showed the twinning.  I processed the data using HKL, which I primarily on my home source data and XDS with similar results.  


In summary, the exact same crystal shows twinning on Cu home source and no twinning at all at the synchrotron.


Any ideas would be welcomed.


Len






Leonard Thomas, Ph.D.
Biomolecular Structure Core, Director
Oklahoma COBRE in Structural Biology
Price
 Family Foundation Institute of Structural Biology
University
 of Oklahoma
Department
 of Chemistry and Biochemistry
101
 Stephenson Parkway
Norman,
 OK 73019-5251
Office:
 (405)325-1126
lmtho...@ou.edu
http://www.ou.edu/structuralbiology/cobre-core-facilities/mcl









To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1




To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

Re: [ccp4bb] i2 won't accept 90 degree angles in moniclinic

[Phil Evans]

That looks like an unnecessarily severe test buried in the i2 library, which 
maybe should just be a warning. I can probably have a look at it

Phil

Sent from my iPad

> On 21 Feb 2024, at 16:52, Winter, Graeme (DLSLtd,RAL,LSCI) 
> <6a19cead4548-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> 
> CAUTION: This email originated from outside of the LMB:
> .-owner-ccp...@jiscmail.ac.uk-.
> Do not click links or open attachments unless you recognize the sender and 
> know the content is safe.
> If you think this is a phishing email, please forward it to 
> phish...@mrc-lmb.cam.ac.uk
> Tried reporting to i2 dev list but it got bounced - feels like something 
> others may hit so don’t feel _too_ bad sending to the bb with a tiny 
> attachment
> 
> 
> 
> 
> 
> 
> Hi Folks
> 
> Helping someone out with some rather specialist data processing challenges 
> and she hit a problem: I can reproduce this with very boring thaumatin data 
> so I think this is a bug
> 
> (it obliquely relates some to a current CCP4bb thread about MR, in passing)
> 
> Processing a data set in lower than necessary symmetry e.g. tetragonal as 
> monoclinic you _cannot_ import the merged MTZ file into i2 because it is 
> impossible to have 90 degree angles for P21
> 
> Well, it is possible, and also, it is still correct to under merge the data 
> so this is a bug? Is this a reasonable viewpoint to hold?
> 
> Many thanks Graeme
>  
> 
> -- 
> 
> This e-mail and any attachments may contain confidential, copyright and or 
> privileged material, and are for the use of the intended addressee only. If 
> you are not the intended addressee or an authorised recipient of the 
> addressee please notify us of receipt by returning the e-mail and do not use, 
> copy, retain, distribute or disclose the information in or attached to the 
> e-mail.
> Any opinions expressed within this e-mail are those of the individual and not 
> necessarily of Diamond Light Source Ltd.
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any 
> attachments are free from viruses and we cannot accept liability for any 
> damage which you may sustain as a result of software viruses which may be 
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England and 
> Wales with its registered office at Diamond House, Harwell Science and 
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>  
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Multi-lattice crystal data processing strategy for structure solution

[Phil Evans]

Is the space group really P2? P21 is MUCH more common
Phil

> On 14 Nov 2023, at 15:55, Devbrat Kumar  wrote:
> 
> sed data were integrated with the data reduction tool AIMLESS in the CCP4i2 
> suite.



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Command line use of Indexing reference mtz and autosetting of resolution limits to a specific I/sigI minimum.

[Phil Evans]

Thinking further about this, Aimless essentially would need to be run twice to 
do automatic resolution cutoff. I could make it an internal loop to essentially 
it twice, I suppose. But in I2 my preferred method is to use the “information 
content” from Phaser as the metric for resolution

Phil

> On 16 Jun 2023, at 09:08, Nichols, Charlie  wrote:
> 
> 
> CAUTION: This email originated from outside of the LMB.
> Do not click links or open attachments unless you recognize the sender and 
> know the content is safe.
> .-owner-ccp...@jiscmail.ac.uk-.
> 
> Hi,
>  
> I want to reprocess a large batch of mtz files from Dials using Aimless and 
> cut the resolution so the outer-shell I/sigI is >=1.0
> I also want them to be consistently indexed so want to specify ‘Match index 
> to reference’ and provide a reference mtz with IMEAN used as the label for 
> index matching.
>  
>  
> So, Firstly:
> I am running Aimless from command line with scripts. Is there a way to 
> specify auto-setting of data output resolution cutting to a specific 
> outer-shell I/sigI value?
>   • At present I’ve just been cutting the resolution back by 0.15A and 
> grepping the I/sigI values. This mostly moves them to >=1 with a small set of 
> high-anisotropy data where I need to examine the scaling logfile and apply a 
> further cut
>  
>  
> Secondly:
> How do I specify an indexing reference file from the command line – Aimless 
> documentation only seems to cover Scaling to reference not indexing to 
> reference
>  
>  
> Thanks for your help,
> Take care, Charlie.
>  
>  
>  
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] PAIREF - Warning - not enough free reflections in resolution bin

[Phil Evans]

If AIMLESS is throwing out a lot of reflections, something is wrong. You should 
look closely at what is being rejected, in the list of “ROGUES” and the plot of 
position of outliers on the detector face

DIALS (probably dials.scale) has its own rejection scheme, as does XDS

There are two basic reasons for rejecting reflections 
1) they belong to part of the data collection where all data are unreliable, eg 
because of radiation damage
2) individual reflections which are “wrong” for some other reason, ice spots, 
detector defects, multiple lattices etc

There should never be “a lot” of outliers in category 2

Phil

> On 1 Oct 2022, at 13:02, Matt McLeod  wrote:
> 
> Hey everyone,
> 
> Thanks for all the suggestions there are a few different things I can try 
> now.  The data is very aniosotropic (STARANISO might help) in regards to how 
> the crystal diffracts and I think changing the bin size will help 
> specifically with PAIREF (its an warning so it completes the run).   I 
> collected the data using oil and at room temperature using a vector scan so 
> there are also differences in data quality through the collection (not too 
> severe based on data processing), radiation damage, a changing background 
> from oil, etc.  
> 
> However, diagnosing the problem further it seems that merging with AIMLESS 
> throws a lot of my high resolution reflections out...like alot.  This 
> explains why truncating the data doesnt change the maps and explains why my 
> table 1 statistics for high resolution bin are dismal.  I can supply log 
> files when I find them.  Now I have to determine if the outlier rejections 
> are useful or not and why DIALS processing didn't flag these as rejections.
> 
> I have yet to look into AIMLESS rejection outlier protocol but I would guess 
> that the reflections are real at high resolution but there are not that many 
> of them and they are not that redundant and therefore are being tossed.
> 
> Matt
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Regarding the correct space group identification

[Phil Evans]

It might be worth reminding people that 1 degree rotation range/image is 
(almost) never appropriate, and is likely to lead to spot overlap. Typical 
values with modern detectors are 0.05-0.1 deg, and I am surprised to see 1 deg 
images
Phil

Sent from my iPad

> On 29 Jul 2022, at 05:06, Sayan Saha  wrote:
> 
> 
> Dear Sir,
> 
>  image1.osc
>  image2.osc
> The detector-to-crystal distance was 190 mm. The Oscillation range was 1.0 
> degree. Please find attached two diffraction images.
> With best regards,
> Sayan Saha.
> 
>> On Thu, Jul 28, 2022 at 9:41 PM Sayan Saha  wrote:
>> Dear Sir,
>> 
>> The detector-to-crystal distance was 190 mm. The Oscillation range was 1.0 
>> degree. Please find attached two diffraction images.
>> With best regards,
>> Sayan Saha.
>> 
>> 
>>> On Thu, Jul 28, 2022 at 7:31 PM Kay Diederichs 
>>>  wrote:
>>> Dear Sayan,
>>> 
>>> On Thu, 28 Jul 2022 15:12:30 +0530, Sayan Saha  wrote:
>>> 
>>> >Dear Sir,
>>> >
>>> >1. There are no ice-rings. However, diffraction spots seem to be
>>> >overlapping. This can be seen during the data processing, as the space
>>> >group (C2 or P222) varies even in the consecutive frames.
>>> 
>>> spot overlap results in inaccurate intensity values. Inaccurate intensities 
>>> result in high Rwork/Rfree.
>>> 
>>> Why do the spots overlap? High mosaicity? Detector distance too small? 
>>> Oscillation range too high (0.1° is typically adequate)?
>>> 
>>> It would be good to see the data, otherwise we can only speculate.
>>> 
>>> Space group does not change from one frame to the next. If you use XDS, a 
>>> good guide to decide between higher and lower-symmetry space groups is to 
>>> compare their ISa values.
>>> 
>>> best,
>>> Kay
>>> 
>>> >
>>> >2. Crystal packing of C2 and P22121 seem to be similar (please see the
>>> >attached images).
>>> >
>>> >3. Forgot to mention in my previous email that we have already processed
>>> >the data in P1 and MR solution could be found only in P1 (Phaser was used
>>> >with an option in all possible space groups of that point group).
>>> >
>>> >Please let me know if any other information is required.
>>> >
>>> >With best regards,
>>> >Sayan Saha.
>>> >
>>> >
>>> >On Thu, Jul 28, 2022 at 1:26 PM Schreuder, Herman /DE <
>>> >herman.schreu...@sanofi.com> wrote:
>>> >
>>> >> Dear Sayan,
>>> >>
>>> >>
>>> >>
>>> >> If a subunit is correctly oriented, but the translation is incorrect,
>>> >> density for a ligand may still show up in the binding site of the 
>>> >> protein.
>>> >> It might be that one of the 2-fold axes, you think is crystallographic, 
>>> >> is
>>> >> in fact non crystallographic and a few Angstroms away from the
>>> >> crystallographic position.
>>> >>
>>> >>
>>> >>
>>> >> What I would do:
>>> >>
>>> >>1. Check the images: are there ice-rings or other artifacts that could
>>> >>cause scaling problems that would lead to high Rw/Rf values? In that 
>>> >> case,
>>> >>there is not much you can do.
>>> >>2. Compare the C2 and P22121 solutions: do they have the same overall
>>> >>crystal packing (CS+NCS), or are they different? Do they have the same
>>> >>Rw/Rf values? Can we learn anything from the differences in overall 
>>> >> crystal
>>> >>packing?
>>> >>3. Process, run MR and refine in P1. Do you get lower R-factors? If
>>> >>so, then run Zanuda to find out the real space group.
>>> >>
>>> >>
>>> >>
>>> >> Best,
>>> >>
>>> >> Herman
>>> >>
>>> >>
>>> >>
>>> >> *Von:* CCP4 bulletin board  *Im Auftrag von *Sayan
>>> >> Saha
>>> >> *Gesendet:* Donnerstag, 28. Juli 2022 08:15
>>> >> *An:* CCP4BB@JISCMAIL.AC.UK
>>> >> *Betreff:* [ccp4bb] Regarding the correct space group identification
>>> >>
>>> >>
>>> >>
>>> >> Dear All,
>>> >>
>>> >>
>>> >>
>>> >> We have collected home-source X-ray intensity data for a protein at 2.6
>>> >> Angstrom. The data can be processed in either C2 (a=120, b=80, c=85 and
>>> >> beta=115) or P222 (P22121, a=80, b=85, c=110). MR solution can be 
>>> >> obtained
>>> >> in both the space groups. However, the solution can be refined with an
>>> >> Rw/Rf of 29/32% only. The protein is bound to a ligand 
>>> >> (co-crystallization)
>>> >> for which a clear density can be observed.
>>> >>
>>> >>
>>> >>
>>> >> Any help and suggestion in this regard would be very helpful.
>>> >>
>>> >>
>>> >>
>>> >> With best regards,
>>> >>
>>> >> Sayan Saha.
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >> To unsubscribe from the CCP4BB list, click the following link:
>>> >> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>>> >>
>>> >
>>> >
>>> >
>>> >To unsubscribe from the CCP4BB list, click the following link:
>>> >https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>>> >
>>> >This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
>>> >list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
>>> 

Re: [ccp4bb] Comparing two datasets

[Phil Evans]

If you want to merge them then you can use Pointless/Aimless/ctruncate, with 
the warning that the conversion from F to I is not ideal, and ctruncate should 
be set to regenerate F from I just as sqrt ie not apply the truncate procedure 
again. 
Pointless supersedes combat for this conversion 
Again, just for comparison use scaleit

Phil

Sent from my iPhone

> On 26 Jul 2022, at 13:33, Jon Cooper 
> <488a26d62010-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> Hello again, as far as I can tell, the question is about two already 
> merged/unique datasets which Mirek wishes to merge into one. As far as I can 
> tell/remember, Scaleit is for scaling datasets side-by-side to get 
> isomorphous differences, etc, and I don't know of a way that you could get it 
> to merge 2 datasets as Mirek requested. Probably I am wrong but Scaleit 
> R-factors are different from R-merge, although the latter might be a bit 
> dubious in the circumstances. Anyway, I think the Combat route is a viable 
> way of achieving what the questioner wants to achieve ;-0 
> 
> 
> Sent from ProtonMail mobile
> 
> 
> 
>  Original Message 
> On 26 Jul 2022, 12:25, John R Helliwell < jrhelliw...@gmail.com> wrote:
> 
> Dear Colleagues,
> Scaleit is a terrific program. 
> Amongst its strengths already listed I would also mention its breakdown table 
> with F so that in the strongest F reflections bin any systematic errors 
> between the two data sets can show up if the Rfactor is greater than about 
> 1%. 
> Greetings,
> John 
> Emeritus Professor John R Helliwell DSc
> 
> 
> 
> 
>> On 26 Jul 2022, at 11:40, Eleanor Dodson 
>> <176a9d5ebad7-dmarc-requ...@jiscmail.ac.uk> wrote:
>> 
>> 
>> Not only does SCALEIT do the job - it presented useful plots and an 
>> informative log file..
>> Eleanor
>> (You need to run CAD hklin1 Xtal1.mtz hklin2 Xtal2.mtz ...and obviously the 
>> columns from each Xtal will need different labels..)
>> 
>> On Tue, 26 Jul 2022 at 09:45, Phil Evans  wrote:
>>> If you give Pointless Fs, it squares them to Is (not correct if the Fs have 
>>> been derived from the truncate procedure, but not too bad). It will then 
>>> give you a comparison. If you give the two datasets to Pointless, labelling 
>>> them as different datasets, you can use Aimless to compare them
>>> 
>>> But as Andrew said, the appropriate CCP4 program is Scaleit: it’s old but 
>>> still works and gives lots of statistics 
>>> 
>>> Phil
>>> 
>>> Sent from my iPhone
>>> 
>>>> On 26 Jul 2022, at 09:37, Andrew Leslie - MRC LMB 
>>>>  wrote:
>>>> 
>>>> I think that POINTLESS works with intensities rather than structure 
>>>> factors (I’m not sure if this can be changed). Also, SCALEIT gives a much 
>>>> more detailed breakdown (R factors as a function of resolution and 
>>>> differences in terms of sigmas etc) than POINTLESS WILL.
>>>> 
>>>> Cheers,
>>>> 
>>>> Andrew
>>>> 
>>>>> On 26 Jul 2022, at 09:24, LEGRAND Pierre 
>>>>>  wrote:
>>>>> 
>>>>> Hello Mirek,
>>>>> 
>>>>> A very quick approach for that is offered by pointless:
>>>>> 
>>>>> pointless HKLREF 1_1_aimless.mtz  HKLIN 2_1_aimless.mtz
>>>>> or
>>>>> pointless HKLREF 1_1_aimless.mtz  XDSIN XDS_ASCII.HK
>>>>> 
>>>>> You will obtain a table looking like that, taking into account to 
>>>>> possible reindexing:
>>>>> 
>>>>> Alternative indexing scores relative to reference
>>>>>Alternative reindexingLklhd  CC R(E^2)Number 
>>>>> Cell_deviation
>>>>>  1  [h,k,l]  0.9930.9620.118 19150
>>>>>   0.08
>>>>>  2  [-k,h,l] 0.0070.0780.512 19150
>>>>>   0.87
>>>>> 
>>>>> 
>>>>> Best wishes,
>>>>> 
>>>>> Pierre Legrand
>>>>> PROXIMA-1 Beamline
>>>>> Synchrotron SOLEIL
>>>>> 
>>>>> De: "Nicolas Foos" 
>>>>> À: CCP4BB@JISCMAIL.AC.UK
>>>>> Envoyé: Mardi 26 Juillet 2022 08:36:35
>>>>> Objet: Re: [ccp4bb] Comparing two datasets
>>>>> 
>>>>> Hi Mirek, 
>>>>> 
>>>>> I am pretty sure XSCALE will do that for you

Re: [ccp4bb] Comparing two datasets

[Phil Evans]

If you give Pointless Fs, it squares them to Is (not correct if the Fs have 
been derived from the truncate procedure, but not too bad). It will then give 
you a comparison. If you give the two datasets to Pointless, labelling them as 
different datasets, you can use Aimless to compare them

But as Andrew said, the appropriate CCP4 program is Scaleit: it’s old but still 
works and gives lots of statistics 

Phil

Sent from my iPhone

> On 26 Jul 2022, at 09:37, Andrew Leslie - MRC LMB  
> wrote:
> 
> I think that POINTLESS works with intensities rather than structure factors 
> (I’m not sure if this can be changed). Also, SCALEIT gives a much more 
> detailed breakdown (R factors as a function of resolution and differences in 
> terms of sigmas etc) than POINTLESS WILL.
> 
> Cheers,
> 
> Andrew
> 
>> On 26 Jul 2022, at 09:24, LEGRAND Pierre 
>>  wrote:
>> 
>> Hello Mirek,
>> 
>> A very quick approach for that is offered by pointless:
>> 
>> pointless HKLREF 1_1_aimless.mtz  HKLIN 2_1_aimless.mtz
>> or
>> pointless HKLREF 1_1_aimless.mtz  XDSIN XDS_ASCII.HK
>> 
>> You will obtain a table looking like that, taking into account to possible 
>> reindexing:
>> 
>> Alternative indexing scores relative to reference
>>Alternative reindexingLklhd  CC R(E^2)Number 
>> Cell_deviation
>>  1  [h,k,l]  0.9930.9620.118 19150  
>> 0.08
>>  2  [-k,h,l] 0.0070.0780.512 19150  
>> 0.87
>> 
>> 
>> Best wishes,
>> 
>> Pierre Legrand
>> PROXIMA-1 Beamline
>> Synchrotron SOLEIL
>> 
>> De: "Nicolas Foos" 
>> À: CCP4BB@JISCMAIL.AC.UK
>> Envoyé: Mardi 26 Juillet 2022 08:36:35
>> Objet: Re: [ccp4bb] Comparing two datasets
>> 
>> Hi Mirek, 
>> 
>> I am pretty sure XSCALE will do that for you : 
>> https://xds.mr.mpg.de/html_doc/xscale_program.html
>> 
>> If not, maybe have a look on SHELXC in SIR mode. 
>> 
>> Hope this help. 
>> 
>> Nicolas
>> 
>> On 25/07/2022 21:52, Cygler, Miroslaw wrote:
>> Hi,
>> I would like to calculate the R-merge for Fs from two datasets processed 
>> from two different crystals. Tried to use Blend but got the message that 
>> Blend requires R. Downloaded R but do not know how to tell CCP4 where it is 
>> located on my Mac. Is there another program that would take two mtg files 
>> and merge the Fs?
>> Any help would be greatly appreciated. 
>> 
>> 
>> Mirek
>> 
>> 
>> 
>> 
>> 
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>> 
>> -- 
>> Nicolas Foos PhD - ARISE fellow
>> https://orcid.org/-0003-2331-8399
>>
>> EMBL Grenoble, McCarthy Team
>> 71 av. des Martyrs, 
>> 38000 Grenoble FRANCE
>>
>> +33 4 57 42 84 67
>> 
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>> 
>> 
>> 
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Specifics of different SCALEIT refinements

[Phil Evans]

Dear Alisia

I assume you have read the documentation at

https://www.ccp4.ac.uk/html/scaleit.html

in particular the comment "Note that there is no unique solution to the problem 
of scaling together two different data sets. “

The program was written (a long time ago) to scale a heavy-atom derivative 
dataset roughly to a native set, with the expectation that the scale (and 
relative B-factor (isotropic or anisotropic)) would be refined during the 
derivative refinement procedure, where the underlying difference between 
datasets can be modelled explicitly

The functions used are described in the documentation 

The underlying assumption of any data scaling is that the true intensities 
which are compared, from different crystals or from different parts of data 
collected from the same crystal, are truly the same, and the observations 
differ only by a scale which can be modelled simply, and by random errors. This 
clearly is not the case if the crystals are different, for whatever reason, eg 
by the addition of a heavy atom or a ligand or indeed by radiation damage. So 
different scaling targets are likely to give (slightly) different results, 
because of different weighting of observations. In Scaleit, the observed 
difference is modelled by a relative scale and a relative B-factor (ie a scale 
dependent on resolution). 

Note also that ideally the scaling should be done on intensities, rather than 
on F^2 where the F has been derived from I by the French & Wilson “truncate” 
procedure or its equivalent. However in practice I’m not sure it makes much 
difference, given the other assumptions

Does this help?
Phil

> On 12 May 2022, at 22:43, Fadini, Alisia  
> wrote:
> 
> Dear all,
> 
> I am having trouble following from the documentation what specific functions 
> are minimized in SCALEIT for the different REFINE options and how the 
> Fox method is applied. Any insight would be very helpful!
> 
> All the best,
> Alisia
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] sftools

[Phil Evans]

It should be easy to do with gemmi 

And there was a program for editing datasets can’t remember what it was called

Phil

Sent from my iPhone

> On 5 Apr 2022, at 14:47, Eleanor Dodson 
> <176a9d5ebad7-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> 
> 
> 
> Does ANYONE know how to use this useful but ultra-frustrating program??
> 
> I have an mtz file which lacks WAVElength AND Dataset name.
> 
> I try to follow the sftools documentation, and get an output file which -
>  lacks WAVElength AND Dataset name. 
> 
> G 
> 
> 
> sftools < READ "detwin_job1-IMEANa.mtz" mtz
> SET DWAVE 1.2 WAVElength AND Dataset name.
> write "detwin_job1-IMEANab.mtz" mtz
> EXIT
> YES
> eof
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Citing Combat

[Phil Evans]

The program combat is largely obsolete, and in most cases can be replaced by 
pointless

I doubt that there is a specific paper about it, beyond the general CCP4 ones

Phil


> On 11 Jan 2022, at 11:49, Fulvio Saccoccia  wrote:
> 
> Dear ccp4i community,
> 
>   what is the right way to cite Combat,  is there any paper about it?
> 
> Thanks in advance
> 
> 
> Fulvio
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Rmeas in DIALS?

[Phil Evans]

You can get a negative Rmeas in a shell where  is negative 

Sent from my iPhone

> On 1 Jul 2021, at 14:21, Winter, Graeme (DLSLtd,RAL,LSCI) 
>  wrote:
> 
>  Thanks for sending the logs
> 
> Looking at the output the negative R values don’t come from xia2 directly
> 
> For AUTOMATIC/DEFAULT/NATIVE OverallLow High
> High resolution limit   1.012.741.01
> Low resolution limit   36.95   36.971.03
> Completeness   68.672.216.3
> Multiplicity3.4 4.0 1.1
> I/sigma 6.811.0 0.6
> Rmerge(I) 0.127   0.131   1.543
> Rmerge(I+/-)  0.121   0.127   1.173
> Rmeas(I)  0.141   0.147   2.162
> Rmeas(I+/-)   0.145   0.151   1.659
> Rpim(I)   0.059   0.062   1.512
> Rpim(I+/-)0.078   0.079   1.173
> CC half   0.985   0.971   0.149
> Wilson B factor  11.910
> Anomalous completeness 45.757.6 1.8
> Anomalous multiplicity  2.1 2.5 1.0
> Anomalous correlation-0.106  -0.135   0.000
> Anomalous slope   0.921
> dF/F  0.079
> dI/s(dI)  0.699
> Total observations   1432839600 559
> Total unique  418512397 494
> Assuming spacegroup: P 41 21 2
> Other likely alternatives are:
> P 43 21 2
> Unit cell:
>  78.580  78.580  36.950
>  90.000  90.000  90.000
> 
> While a 1990’s reviewer would have a hard time with those merging stats, I 
> have seen worse and they seem internally consistent
> 
> I wonder, is ccp4i2 doing something itself which arrives at those numbers?
> 
> i2 devs - thoughts?
> 
> Thanks Graeme
> 
>>> On 1 Jul 2021, at 05:49, Winter, Graeme (DLSLtd,RAL,LSCI) 
>>>  wrote:
>>> 
>>> Something odd happening there - please could you send the text log off 
>>> list? i.e. xia2.txt
>>> 
>>> Thanks Graeme
>>> 
>>> On 1 Jul 2021, at 00:58, Murpholino Peligro  wrote:
>>> 
>>> Why Rmeas is negative in DIALS output?
>>> 
>>> 
>>> Thanks
>>> 
>>> 
>>> To unsubscribe from the CCP4BB list, click the following link:
>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>>> 
>>> 
>> 
>>  
>> -- 
>> 
>> This e-mail and any attachments may contain confidential, copyright and or 
>> privileged material, and are for the use of the intended addressee only. If 
>> you are not the intended addressee or an authorised recipient of the 
>> addressee please notify us of receipt by returning the e-mail and do not 
>> use, copy, retain, distribute or disclose the information in or attached to 
>> the e-mail.
>> Any opinions expressed within this e-mail are those of the individual and 
>> not necessarily of Diamond Light Source Ltd. 
>> Diamond Light Source Ltd. cannot guarantee that this e-mail or any 
>> attachments are free from viruses and we cannot accept liability for any 
>> damage which you may sustain as a result of software viruses which may be 
>> transmitted in or with the message.
>> Diamond Light Source Limited (company no. 4375679). Registered in England 
>> and Wales with its registered office at Diamond House, Harwell Science and 
>> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>>  
>> 
>> 
>> To unsubscribe from the CCP4BB list, click the following link:
>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>> 
> 
>  
> 
> -- 
> 
> This e-mail and any attachments may contain confidential, copyright and or 
> privileged material, and are for the use of the intended addressee only. If 
> you are not the intended addressee or an authorised recipient of the 
> addressee please notify us of receipt by returning the e-mail and do not use, 
> copy, retain, distribute or disclose the information in or attached to the 
> e-mail.
> Any opinions expressed within this e-mail are those of the individual and not 
> necessarily of Diamond Light Source Ltd. 
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any 
> attachments are free from viruses and we cannot accept liability for any 
> damage which you may sustain as a result of software viruses which may be 
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England and 
> Wales with its registered office at Diamond House, Harwell Science and 
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
>  
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

Re: [ccp4bb] aimless scale without merging data running Fail

[Phil Evans]

Dear Jiang 

If you gave Mosflm the “correct” space group, then you can skip the Pointless 
step, but I don’t think the pipelines in ccp4i and certainly ccp4i2 let you do 
that. However, it shouldn’t hurt.

I don’t think you can skip the merging step in Aimless, and anyway it is that 
step which generates the Table 1 statistics. You don’t have to use the output 
file, of course

I do recommend the newer ccp4i2 interface (not so new now), it produces much 
better reports, and the Table 1 can be exported as a CSV file for direct import 
into a paper

Good luck
Phil

Sent from my iPad

> On 16 Jun 2021, at 23:35, Jiang Xu  wrote:
> 
> 
> Hello Phil, 
> I just read my email and found I may not ask the question clearly. So my 
> question is, should I use the integrated data from Imosflm, or should I use 
> the output from Pointless as the input of Aimless to do scale without merging 
> the data?
> Thank you,
> Best,
> Jiang
> Lin Chen Research Group
> University of Southern California 
> 
>> On Wed, Jun 16, 2021 at 2:51 PM Jiang Xu  wrote:
>> Hello Phil,
>>Thank you for the comments. I noticed that the ccp4 package I used was 
>> 7.0. After upgrading to the latest version, the problem was solved.
>>I also have another question about Aimless. I want to use aimless to 
>> scale without merging the data for generating the "table1" of my data. I 
>> noticed that in your original publication, one way of doing the scale is to 
>> use the "Quick Scale" button in Imosflm, in which the output from Pointless 
>> will be the input of Aimless. However, I also noticed that some posts claim 
>> that the input of Aimless could also be from the integrated data file from 
>> Imosflm. I tried both ways and found no problem. So which way is the correct 
>> way for my purpose?
>>    Thank you,
>> Best,
>> Jiang
>> Lin Chen Research Group
>> University of Southern California   
>> 
>>> On Wed, Jun 16, 2021 at 12:10 PM Phil Evans  wrote:
>>> I think it’s trying to write a file TILEIMAGE.img to a directory where you 
>>> aren’t allowed to write. I can’t remember what is done in ccp4i - I believe 
>>> it should work ok in ccp4i2, which produces better reports, and is 
>>> generally recommended as a replacement for ccp4i
>>> 
>>> As a workaround, you might be able to assign the file to somewhere you are 
>>> allowed to write. Before running ccp4i, type something like (for tosh / 
>>> cash, not sure about bash)
>>> 
>>> setenv TILEIMAGE /some/suitable/directory/TILEIMAGE.img
>>> 
>>> This file is an image of the correction factors for a tiled ccd detector
>>> 
>>> I can look at this next week, to see if I can work out what is happening 
>>> 
>>> Phil
>>> 
>>> Sent from my iPad
>>> 
>>>>> On 16 Jun 2021, at 18:52, Jiang Xu  wrote:
>>>>> 
>>>> 
>>>> Hello guys,
>>>>I am having some problems running Aimless from the CCP4i packages. I 
>>>> want to scale without merging the data. The input mtz file for aimless was 
>>>> either the mtz file directly from Imosflm integration, or from Pointless 
>>>> from a previous run on Imosflm. I ran both pointless and aimless 
>>>> successfully on the Imosflm UI, after integration. However I just couldn't 
>>>> do it from the CCP4i. From the log it seems there's always an error 
>>>> message:  "#CCP4I TERMINATION STATUS 0 "OpenFile: cannot open file 
>>>> TILEIMAGE.img"
>>>>I googled this line and found no hit. So does anyone know what's 
>>>> happening and how to solve this problem?
>>>> Thank you,
>>>> Best,
>>>> Jiang Xu
>>>> Lin Chen Research Group
>>>> University of Southern California
>>>> 
>>>> 
>>>> To unsubscribe from the CCP4BB list, click the following link:
>>>> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
>>>> 
>>>> 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] aimless scale without merging data running Fail

[Phil Evans]

I think it’s trying to write a file TILEIMAGE.img to a directory where you 
aren’t allowed to write. I can’t remember what is done in ccp4i - I believe it 
should work ok in ccp4i2, which produces better reports, and is generally 
recommended as a replacement for ccp4i

As a workaround, you might be able to assign the file to somewhere you are 
allowed to write. Before running ccp4i, type something like (for tosh / cash, 
not sure about bash)

setenv TILEIMAGE /some/suitable/directory/TILEIMAGE.img

This file is an image of the correction factors for a tiled ccd detector

I can look at this next week, to see if I can work out what is happening 

Phil

Sent from my iPad

> On 16 Jun 2021, at 18:52, Jiang Xu  wrote:
> 
> 
> Hello guys,
>I am having some problems running Aimless from the CCP4i packages. I want 
> to scale without merging the data. The input mtz file for aimless was either 
> the mtz file directly from Imosflm integration, or from Pointless from a 
> previous run on Imosflm. I ran both pointless and aimless successfully on the 
> Imosflm UI, after integration. However I just couldn't do it from the CCP4i. 
> From the log it seems there's always an error message:  "#CCP4I TERMINATION 
> STATUS 0 "OpenFile: cannot open file TILEIMAGE.img"
>I googled this line and found no hit. So does anyone know what's happening 
> and how to solve this problem?
> Thank you,
> Best,
> Jiang Xu
> Lin Chen Research Group
> University of Southern California
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] External: Re: [ccp4bb] AlphaFold: more thinking and less pipetting (?)

[Phil Evans]

Alpha-fold looks great and is clearly a long way towards answering the question 
“this is the sequence, what is the structure?”

But I’ve always thought the more interesting question is “this is the 
structure, what does it do?”  Is there any progress on that question?

Phil


> On 11 Dec 2020, at 12:12, Tristan Croll  wrote:
> 
> I'm not Randy, but I do have an answer: like this. This is T1049-D1. 
> AlphaFold prediction in red, experimental structure (6y4f) in green. 
> Agreement is close to perfect, apart from the C-terminal tail which is way 
> off - but clearly flexible and only resolved in this conformation in the 
> crystal due to packing interactions. GDT_TS is 93.1; RMS_CA is 3.68 - but if 
> you exclude those tail residues, it's 0.79. With an alignment cutoff of 1 A, 
> you can align 109 of 134 CAs with an RMSD of 0.46 A.
> From: CCP4 bulletin board  on behalf of Leonid Sazanov 
> 
> Sent: 11 December 2020 10:36
> To: CCP4BB@JISCMAIL.AC.UK 
> Subject: Re: [ccp4bb] External: Re: [ccp4bb] AlphaFold: more thinking and 
> less pipetting (?)
>  
> Dear Randy,
> 
> Can you comment on why for some of AplhaFold2 models with GDT_TS > 90 
> (supposedly as good as experimental model) the RMS_CA (backbone) is > 3.0 
> Angstrom? Such a deviation can hardly be described as good as experimental. 
> Could it be that GDT_TS is kind of designed to evaluate how well the general 
> sub-domain level fold is predicted, rather than overall detail?
> 
> Thanks,
> Leonid
> 
> 
> >
> Several people have mentioned lack of peer review as a reason to doubt the 
> significance of the AlphaFold2 results.  There are different routes to peer 
> review and, while the results have not been published in a peer review 
> journal, I would have to say (as someone who has been an assessor for two 
> CASPs, as well as having editorial responsibilities for a peer-reviewed 
> journal), the peer review at CASP is much more rigorous than the peer review 
> that most papers undergo.  The targets are selected from structures that have 
> recently been solved but not published or disseminated, and even just 
> tweeting a C-alpha trace is probably enough to get a target cancelled.  In 
> some cases (as we’ve heard here) the people determining the structure are 
> overly optimistic about when their structure solution will be finished, so 
> even they may not know the structure at the time it is predicted.  The 
> assessors are blinded to the identities of the predictors, and they carry out 
> months of calculations and inspections of the models, computing ranking 
> scores before they find out who made the predictions.  Most assessors try to 
> bring something new to the assessment, because the criteria should get more 
> stringent as the predictions get better, and they have new ideas of what to 
> look for, but there’s always some overlap with “traditional” measures such as 
> GDT-TS, GDT-HA (more stringent high-accuracy version of GDT) and lDDT.
> 
> 
> 
> Of course we’d all like to know the details of how AlphaFold2 works, and the 
> DeepMind people could have been (and should be) much more forthcoming, but 
> their results are real.  They didn’t have any way of cheating, being 
> selective about what they reported, or gaming the system in any other way 
> that the other groups couldn’t do.  (And yes, when we learned that DeepMind 
> was behind the exceptionally good results two years ago at CASP13, we made 
> the same half-jokes about whether Gmail had been in the database they were 
> mining!)
> 
> 
> 
> Best wishes,
> 
> 
> 
> Randy Read
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 
> This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing 
> list hosted by www.jiscmail.ac.uk, terms & conditions are available at 
> https://www.jiscmail.ac.uk/policyandsecurity/
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] Pointless error in ccp4i2 import merged data...

[Phil Evans]

That sort of thing works for me

1. Can you try running Pointless from the command line on the same file?

2. Failing that, can you send me the input file to Pointless to see why 
Pointless is failing

Phil

> On 17 Nov 2020, at 10:23, Kevin Cowtan 
> <2ba34e97fcaf-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> Hi!
> 
> I have a student trying to run i2, and a task to import merged data from cif 
> which works fine for the other students is failing at the pointless step.
> 
> The logfiles are identical up to pointless (apart from the random free-R flag 
> assignment), but the problem occurs in pointless. The log file output is as 
> follows:
> 
>  Input command lines 
> 
> # Task 4.3.1 pointless running pointless
> # Mini-MTZ input to HKLIN:
> #   Data type   parameter  jobannotation  
>   
> NAME PROJECT 2b463 CRYSTAL xtal DATASET dest
> HKLIN 
> C:\Users\xxx\CCP4I2_PROJECTS\2b463\CCP4_JOBS\job_4\4_2b463_obsout_import_merged.mtz
> HKLOUT 
> C:\Users\xxx\CCP4I2_PROJECTS\2b463\CCP4_JOBS\job_4\job_3\job_1\MTZUNMERGEDOUT.mtz
> choose spacegroup P 61
> 
>  End of input
> 
> 
> **
> **
> * POINTLESS  *
> *   1.12.2   *
> *    *
> *   Determine Laue group from unmerged intensities   *
> * Phil Evans MRC LMB, Cambridge  *
> * Uses cctbx routines by Ralf Grosse-Kunstleve et al.*
> **
> **
> 
> 
>  ERROR 
> 
> $TEXT:Reference: $$ Please cite $$
> P.R.Evans, 'Scaling and assessment  of data quality' Acta Cryst. D62, 72-82  
> (2006).
> http://journals.iucr.org/d/issues/2006/01/00/ba5084/index.html;>
> PDF
> P.R.Evans, 'An introduction to data reduction: space-group determination, 
> scaling and intensity statistics' Acta Cryst. D67, 282-292 (2011)
> http://journals.iucr.org/d/issues/2011/04/00/ba5158/index.html;>
> PDF
> $$
> 
> 
> -- 
> Professor Kevin Cowtan
> Email:  kevin.cow...@york.ac.uk
> Pronouns:   Please use he/him or they/them when referring to me in 
> professional contexts
> Address:York Structural Biology Laboratory, Department of Chemistry
> University of York, York, YO10 5DD, UK
> ORCiD:  -0002-0189-1437
> Web:https://www.york.ac.uk/chemistry/staff/academic/a-c/kcowtan/
> Disclaimer: http://www.york.ac.uk/docs/disclaimer/email.htm
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1
> 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/WA-JISC.exe?SUBED1=CCP4BB=1

This message was issued to members of www.jiscmail.ac.uk/CCP4BB, a mailing list 
hosted by www.jiscmail.ac.uk, terms & conditions are available at 
https://www.jiscmail.ac.uk/policyandsecurity/

Re: [ccp4bb] [EXTERNAL] Re: [ccp4bb] number of frames to get a full dataset?

[Phil Evans]

I changed the annotation from “Redundancy” to “Multiplicity” in Scala, later in 
Aimless, after I was taken to task by Elspeth Garman with the argument as 
stated, that if it’s redundant why did you bother to measure it?

(this one could run and run …)

Phil

> On 30 Jun 2020, at 14:07, Ian Tickle  wrote:
> 
> 
> I agree about RAID but I would go a lot further.  There seems to be some 
> confusion here over the correct meaning of 'redundant' as used in a 
> scientific context.  I don't think looking it up in an English dictionary is 
> very helpful.  So as has been mentioned the non-scientific and rather 
> imprecise meanings are "not or no longer needed or useful; superfluous" or 
> "exceeding what is necessary or natural; superfluous" and "needlessly 
> repetitive; verbose".  In fact both redundant and abundant have the same 
> Latin etymology, and redundant literally means 're' (again) + 'unda' (wave), 
> i.e. 'repeating as a wave'.  The original meaning in English is in fact 
> 'over-abundant' and is still used in poetry with that meaning (e.g. "as 
> redundant as the poppies in the field").  There's of course also the meaning 
> 'dismissal from a job due to a need to reduce the head count' and from there 
> 'out of work', but that's relatively recent having been coined by a UK 
> Government official in the 1900s!
> 
> The correct and totally precise scientific meaning which is appropriate in 
> the context of this discussion is to be found here: 
> https://en.wikipedia.org/wiki/Redundancy_(engineering) .  Note that it 
> applies equally to both hardware and software engineering:
> 
> Redundancy is the duplication of critical components or functions of a system 
> with the intention of increasing reliability of the system, usually in the 
> form of a backup or fail-safe, or to improve actual system performance.
> 
> Nothing there about not or no longer needed or useful, superfluous, 
> needlessly repetitive, verbose!  Note that 'multiplicity' totally fails to 
> carry the connotation of increasing the system reliability by duplication 
> (i.e. there are multiple copies but there's nothing that indicates the 
> justification for them).  Redundancy occurs in TMR (triple modular 
> redundancy) systems used (as I guess Bernhard knows well) in triplicated 
> control systems in commercial aircraft.  I don't know about you but I 
> wouldn't regard the extra two backup systems in TMR as 'not needed or useful' 
> when I'm an airline passenger !
> 
> https://en.wikipedia.org/wiki/Triple_modular_redundancy
> 
> More is always better when it's critical:
> 
> https://www.isa.org/standards-and-publications/isa-publications/intech-magazine/2003/october/more-is-always-better-when-its-critical
> 
> There's also the question of the same word (redundancy, multiplicity or 
> whatever) having different meanings according to context.  That's unavoidable 
> given that the number of concepts that we might want to name far exceeds the 
> number of words available, so we have to rely heavily on context when 
> assigning meaning.  We don't say what the context is so the context must be 
> obvious and unambiguous.  Whether we're talking about RAID or losing one's 
> job it's obvious what the intended meaning is from the context because the 
> contexts are totally separate.  The important thing is that the contexts 
> should be well-separated so that no confusion is possible.  Graeme says he's 
> not confused by the various meanings of 'multiplicity' but 
> non-crystallographer consumers of Table 1 surely might be!  The various 
> contexts in which 'multiplicity' is used are certainly not well-separated and 
> overlap in program outputs and documentation, allowing plenty of scope for 
> confusion.
> 
> In a scientific context 'redundancy' has a unique precise meaning whereas 
> 'multiplicity' has a multiplicity!
> 
> BTW I use CCP4/Aimless and 'redundancy' (as you no doubt will have guessed, 
> because it's the word that unambiguously describes the concept), so 
> apparently I'm with you lot across the pond on this!
> 
> Cheers
> 
> -- Ian
>  
> 
> 
> On Tue, 30 Jun 2020 at 09:01, David Waterman  wrote:
> Reflections are as "redundant" as the disks in a RAID 0 array
> 
> On Tue, 30 Jun 2020, 02:49 James Holton,  wrote:
> What could possibly go wrong?
> 
> -James Holton
> MAD Scientist
> 
> On 6/29/2020 6:17 PM, Edward A. Berry wrote:
> > Now can we get rid of all the superfluous disks in our RAID? Or at 
> > least not replace them when they fail?
> >
> > On 06/29/2020 06:24 PM, Andreas Förster wrote:
> >> I like to think that the reflections I carefully measured at high 
> >> multiplicity are not redundant, which the dictionary on my computer 
> >> defines as "not or no longer needed or useful; superfluous" and the 
> >> American Heritage Dictionary as "exceeding what is necessary or 
> >> natural; superfluous" and "needlessly repetitive; verbose".
> >>
> >> Please don't use the term Needless repetitivity in your Table 1.  It 
> 

[ccp4bb]

[Phil Evans]

Pointless won’t do this with merged files - CAD is the thing

> On 11 May 2019, at 15:24, CCP4BB 
> <193323b1e616-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> Or use Pointless to merge them together...
> 
> Harry
> --
> Dr Harry Powell
> 
> On 11 May 2019, at 06:48, graeme.win...@diamond.ac.uk 
>  wrote:
> 
>> Hi Scott,
>> 
>> You can run CAD two or more times - merge the columns incrementally (i.e. 
>> 1..9 then (1..9)+10..16)
>> 
>> Best wishes Graeme
>> 
>> On 10 May 2019, at 23:57, Scott Horowitz 
>> mailto:horow...@umich.edu>> wrote:
>> 
>> Hi all,
>> 
>> I’m trying to combine 16 datasets together using Cad, and it’s failing with 
>> the following in the log file:
>> 
>> Key_Word Error, For LABIN FILE_NUMBER line:-
>>  File_Number Given =   10 Is invalid only files 1 to  9 Allowed, Ignoring 
>> this line -
>> 
>> It does that for all the datasets starting with number 10, which makes it 
>> seem like it can only combine 9 datasets at a time. I was just wondering if 
>> that’s something others have run into and dealt with before?
>> 
>> The whole of the log file is below, if you want to peruse it.
>> 
>> Thanks,
>> Scott
>> #CCP4I VERSION CCP4Interface 7.0.045
>> #CCP4I SCRIPT LOG cad
>> #CCP4I DATE 10 May 2019  14:54:40
>> #CCP4I USER 'UNKNOWN'
>> #CCP4I PROJECT Serena
>> #CCP4I JOB_ID 7
>> #CCP4I SCRATCH C:/ccp4temp
>> #CCP4I HOSTNAME DU-1KDZDH2
>> #CCP4I PID 17552
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> ###
>> ###
>> ###
>> ### CCP4 7.0.045: CAD  version 7.0.045 : ##
>> ###
>> User: Scott.Horowitz  Run date: 10/ 5/2019 Run time: 14:54:41
>> 
>> 
>> Please reference: Collaborative Computational Project, Number 4. 2011.
>> "Overview of the CCP4 suite and current developments". Acta Cryst. D67, 
>> 235-242.
>> as well as any specific reference in the program write-up.
>> 
>> 
>> Data line--- title [No title given]
>> Data line--- monitor FULL
>> Data line--- labin file 1 E1 = R-free-flags(+) E2 = R-free-flags(-)
>> Data line--- labout file 1 E1 = R-free-flags(+) E2 = R-free-flags(-)
>> Data line--- ctypin file 1 E1 = I E2 = I
>> Data line--- labin file 2 E1 = F E2 = SIGF E3 = DANO E4 = 
>> SIGDANO E5 = F(+) E6 = SIGF(+) E7 = F(-) E8 = SIGF(-) E9 
>> = ISYM E10 = IMEAN E11 = SIGIMEAN E12 = I(+) E13 = SIGI(+)   
>>   E14 = I(-) E15 = SIGI(-)
>> Data line--- labout file 2 E1 = F E2 = SIGF E3 = DANO E4 = 
>> SIGDANO E5 = F(+) E6 = SIGF(+) E7 = F(-) E8 = SIGF(-) E9 
>> = ISYM E10 = IMEAN E11 = SIGIMEAN E12 = I(+) E13 = SIGI(+)   
>>   E14 = I(-) E15 = SIGI(-)
>> Data line--- ctypin file 2 E1 = F E2 = Q E3 = D E4 = Q 
>> E5 = G E6 = L E7 = G E8 = L E9 = Y E10 = J E11 = Q   
>>   E12 = K E13 = M E14 = K E15 = M
>> Data line--- labin file 3 E1 = R-free-flags
>> Data line--- labout file 3 E1 = R-free-flags_4p5
>> Data line--- ctypin file 3 E1 = I
>> Data line--- labin file 4 E1 = F E2 = SIGF E3 = DANO E4 = 
>> SIGDANO E5 = F(+) E6 = SIGF(+) E7 = F(-) E8 = SIGF(-) E9 
>> = ISYM E10 = IMEAN E11 = SIGIMEAN E12 = I(+) E13 = SIGI(+)   
>>   E14 = I(-) E15 = SIGI(-)
>> Data line--- labout file 4 E1 = F_4p5 E2 = SIGF_4p5 E3 = 
>> DANO_4p5 E4 = SIGDANO_4p5 E5 = F(+)_4p5 E6 = SIGF(+)_4p5 E7 
>> = F(-)_4p5 E8 = SIGF(-)_4p5 E9 = ISYM_4p5 E10 = IMEAN_4p5 
>> E11 = SIGIMEAN_4p5 E12 = I(+)_4p5 E13 = SIGI(+)_4p5 E14 = 
>> I(-)_4p5 E15 = SIGI(-)_4p5
>> Data line--- ctypin file 4 E1 = F E2 = Q E3 = D E4 = Q 
>> E5 = G E6 = L E7 = G E8 = L E9 = Y E10 = J E11 = Q   
>>   E12 = K E13 = M E14 = K E15 = M
>> Data line--- labin file 5 E1 = R-free-flags
>> Data line--- labout file 5 E1 = R-free-flags_2p87
>> Data line--- ctypin file 5 E1 = I
>> Data line--- labin file 6 E1 = F E2 = SIGF E3 = DANO E4 = 
>> SIGDANO E5 = F(+) E6 = SIGF(+) E7 = F(-) E8 = SIGF(-) E9 
>> = ISYM E10 = IMEAN E11 = SIGIMEAN E12 = I(+) E13 = SIGI(+)   
>>   E14 = I(-) E15 = SIGI(-)
>> Data line--- labout file 6 E1 = F_2p87 E2 = SIGF_2p87 E3 = 
>> DANO_2p87 E4 = SIGDANO_2p87 E5 = F(+)_2p87 E6 = SIGF(+)_2p87 
>> E7 = F(-)_2p87 E8 = SIGF(-)_2p87 E9 = ISYM_2p87 E10 = IMEAN_2p87 
>> E11 = SIGIMEAN_2p87 E12 = I(+)_2p87 E13 = SIGI(+)_2p87 E14 = 
>> I(-)_2p87 E15 = SIGI(-)_2p87
>> Data line--- ctypin file 6 E1 = F E2 = Q E3 = D E4 = Q 
>> E5 = G E6 = L E7 = G E8 = L E9 = Y E10 = J 

Re: [ccp4bb] Configuration of inositol hexaphosphate

[Phil Evans]

The stable conformation should be 5 equatorial and one axial (2-position I 
think)

> On 20 Mar 2019, at 10:34, Karthik Selvam  wrote:
> 
> Dear CCP4 community,
> 
> We are working on a complex involving inositol hexaphoshpate. The structure 
> of the ligand given in the coot monomer library has alternate phosphate 
> groups attached to the ring carbon atoms at the equatorial configuration and 
> axial configuration. We also examined the structure of the ligand in PDB IDs 
> 2fvv, 2q9p, 1dkp, and 1bq3. They have different configurations. All of which 
> are again different from that given in the coot library. We shall be grateful 
> if the community could enlighten on this issue.
> 
> Regards,
> Karthik &
> Prateek Raj.
> 
> 
> https://drive.google.com/open?id=13D5j2u_0OxP_X25B_kjKShtV7GQQTwBd
> 
> https://drive.google.com/open?id=1xYc-PJ6jBi8UowsYb9MCZ7QEVW6m30KR
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] VERY old mtz file..

[Phil Evans]

The CCP4 library routines are supposed to read MTZ files in any recognised 
format, provided the correct machine stamp is present, so I’m puzzled that it 
doesn’t work in this case. In the days when multiple float formats were around, 
this certainly did work. The files were always written in the native format, 
and converted if required on reading. As far as I know the code is all still in 
the libraries, so it should still work.
Phil

> On 14 Nov 2018, at 08:58, Kay Diederichs  
> wrote:
> 
> It is not necessary to do error-prone conversions manually: the ifort 
> Compiler understands the convert='VAXD' Option in its OPEN statement - see 
> https://software.intel.com/en-us/fortran-compiler-developer-guide-and-reference-open-convert-specifier
> 
> Thus one could just write a tiny read-write loop.
> 
> HTH
> Kay
> 
> On Wed, 14 Nov 2018 00:51:02 +, Zhijie Li  wrote:
> 
>> It's also said here, at the end of file :
>> 
>> https://www.cs.auckland.ac.nz/~patrice/210LN/DR4.pdf
>> 
>> "add 1 to the left, with the binary point"
>> 
>> 0.1.
>> 
>> 
>> 
>> 
>> From: CCP4 bulletin board  on behalf of Zhijie Li 
>> 
>> Sent: Tuesday, November 13, 2018 7:43 PM
>> To: CCP4BB@JISCMAIL.AC.UK
>> Subject: Re: [ccp4bb] VERY old mtz file..
>> 
>> 
>> Hi all,
>> 
>> 
>> I think I know why it is a division of 4 instead of 2 is involved in 
>> conversion from VAX to IEEE now. Short answer: a 2 is in the exponent bits 
>> (bias of 128 instead of 127, visible), another 2 is hidden in the scientific 
>> notation.
>> 
>> 
>> I found this explanation+example on VAX F-float:
>> 
>> http://courseweb.stthomas.edu/tpsturm/private/notes/qm300/FLOATPT.html
>> 
>> 
>> So for IEEE754 float32, if we want to represent a same 12.75 (1100.11) in 
>> the above example, we  would first conceptually write it in scientific 
>> notation as 1.10011 x 1000 in binary. Then the mantissa part is the part 
>> after the dot filled with zero to 23 bits: '1001100', the 
>> exponent part is 3+127=130 (dec)=1010(bin). Then the binary IEEE754 
>> float32 number is 0[1010][1001100]. (You can check it 
>> here: https://www.h-schmidt.net/FloatConverter/IEEE754.html)
>> 
>> 
>> Now compare this with the VAX 12.75 in the linked example, we can find that 
>> besides the bias becoming 128, the conceptual binary scientific notation is 
>> actually  0.110011 x 1, instead of 1.10011 x 1000.  So the exponent 
>> needed is 4 instead of 3. Then the exponent bits are 4+128=132=1100 and 
>> the VAX float32 becomes
>> 
>> 0[1100][1001100] ---if we write in a IEEE-style order. 
>> Note that the mantissa appears to be same as the ieee mantissa, and the 
>> exponent to be applied is 132-128=4. If this number is interpreted as 
>> IEEE754, then it will be 1.10011 x 2exp(132-127)=1.10011 x 10, four 
>> times of what it should be.
>> 
>> 
>> So, for normalised values, rearranging the VAX F-float bytes, reading as 
>> IEEE, then dividing by 4 gives the correct value. (The C[0]-1 treatment in 
>> the ccp4 lib is neat.)
>> 
>> 
>> In this link describing VAX floats, it is unfortunate that it only states 
>> that the bias for F-float is 128, but not that the mantissa starts from 0.01 
>> instead of 0.1. Therefore the confusion.
>> 
>> https://nssdc.gsfc.nasa.gov/nssdc/formats/VAXFloatingPoint.htm
>> 
>> 
>> 
>> Thanks to all responded!
>> 
>> 
>> Zhijie
>> 
>> 
>> 
>> From: Ian Tickle 
>> Sent: Tuesday, November 13, 2018 4:54 PM
>> To: Zhijie Li
>> Cc: CCP4BB@JISCMAIL.AC.UK
>> Subject: Re: [ccp4bb] VERY old mtz file..
>> 
>> 
>> Hi Zhijie
>> 
>> It's definitely a factor 4.  The code is in subroutine QTIEEE in the Fortran 
>> source I mentioned previously at this line:
>> 
>> See line:
>> 
>> A(I)=((A(I)+SIGN(2,A(I)))/4.AND..NOT.MNAN).OR.MDN2
>> 
>> If you prefer it in C code it's in function vaxF2ieeeF in:
>> 
>> ccp4-7.0-src/checkout/libccp4/ccp4/vmslibrary.c
>> 
>> See line:
>> 
>> out.c[0] = buffer[i].c[1] - (uint8)1; /* subtracts 2 from exponent */
>> 
>> i.e. subtract 2 from exponent -> division by 4.
>> 
>> Cheers
>> 
>> -- Ian
>> 
>> 
>> On Tue, 13 Nov 2018 at 19:52, Zhijie Li 
>> mailto:zhijie...@utoronto.ca>> wrote:
>> If somebody is going to send these files by email, please send one to me 
>> too. Thanks in advance. I actually prefer to get a MTZ file because the 
>> miller indices would serve as good clues for understanding the encodings.  
>> Even the first 1024 bytes of an MTZ would do (data array starts at byte 80 
>> in MTZ).
>> 
>> In my life I had only seen ieee754.  According to what I can find, VAX has 
>> an exponent bias of 128 (ieee754 uses 127). Then it seems to me that when 
>> converting from vax to ieee a division of 2 is involved. However all 
>> procedures I have seen use a division of 4, which is quite puzzling to me. A 
>> real data file containing meaningful numbers (eg., HKL indices) would be 
>> very 

Re: [ccp4bb] R-merge is too high !!

[Phil Evans]

It is well known that Rmerge is a terrible criterion for determining a 
resolution cutoff, see eg
P.A.Karplus,K.Diederichs,Science336,1030(2012)

As the intensities get weaker at high resolution, Rmerge tends to infinity, 
with no sensible cutoff. CC(1/2) is generally considered the best criterion for 
this, at present anyway. Rmerge of 1.59 in the outer shell is perfectly 
acceptable, CC(1/2) in the outer shell is 0.645 which is still good (maybe the 
data go a little beyond there?)

On the other hand, a high Rmerge (or low CC(1/2)) at low resolution, or in the 
top intensity bin, is a indicator of something seriously wrong, some horrible 
systematic error or instability. In your innermost resolution bin, the Rmerge 
values are rather high (what are Norm R-fac and Linear R-fac?), and CC(1/2) is 
a little low

You need to look at other things, notably radiation damage

But if the structure tells you what you want know, enjoy it!

Phil

> On 28 Sep 2018, at 09:09, Zhang Foggy  wrote:
> 
> Dear All,
> 
> Sorry for the off-topic. 
> 
> I recently collected a set of data. The diffraction spots are extremely 
> sharp. However, When I used HKL3000 to scale it, I get a final resolution at 
> 3.1A with overall R-merge ~0.54 (R-merge in the highest 3.2A-3.1A shell: 
> 1.59). Then I solve the structure with final R value 0.19 and R free value 
> 0.24 although I know this Rmerge value is totally unacceptable, and the 
> density looks perfect.
> 
> I also tried to collect other four set of data with different crystals. 
> unfortunately, all of them have same problem. 
> 
> I ask one of my friend who is an expert in HKL3000, but he had no idea about 
> it. Does anyone has suggestions?
> 
> Here is the scale information for your review:
> Space group: P43 (I also tried P1, the Rmerge value is still similar)
> 
> Shell Lower Upper Average  Average Norm. Linear Square
>  limitAngstrom   I   error   stat. Chi**2  R-fac  R-fac  Rmeas   Rpim 
>  CC1/2CC*
>   50.00   6.6711.6 0.9 0.3  1.165  0.191  0.284  0.198  0.052 
>  0.975  0.994
>6.67   5.30 4.5 0.5 0.3  0.952  0.317  0.313  0.329  0.086 
>  0.971  0.993
>5.30   4.63 7.3 0.7 0.5  0.961  0.293  0.297  0.304  0.081 
>  0.975  0.994
>4.63   4.21 7.0 0.8 0.6  0.986  0.369  0.358  0.382  0.101 
>  0.969  0.992
>4.21   3.91 5.6 0.8 0.6  1.040  0.522  0.491  0.541  0.142 
>  0.955  0.988
>3.91   3.68 4.6 0.9 0.7  1.064  0.718  0.669  0.746  0.203 
>  0.929  0.981
>3.68   3.49 3.5 0.9 0.8  1.092  1.059  0.986  1.101  0.299 
>  0.882  0.968
>3.49   3.34 2.6 0.9 0.8  1.092  1.382  1.298  1.438  0.395 
>  0.829  0.952
>3.34   3.21 2.1 0.9 0.8  1.084  1.543  1.489  1.614  0.468 
>  0.772  0.933
>3.21   3.10 1.6 0.9 0.8  1.070  1.591  1.669  1.680  0.529 
>  0.645  0.885
>   All reflections  5.0 0.8 0.6  1.048  0.538  0.487  0.559  0.153
> 
> Thank you.  
> 
> Liang
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1
> 



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] screw axes /system. absenses and phaser/MR solutions

[Phil Evans]

Just to be clear, the Pointless default setting (“CELL-BASED”), which has a On 10 Aug 2018, at 08:29, Kay Diederichs  
> wrote:
> 
> Tommi,
> 
> one aspect that may be confusing is that XDS and POINTLESS by default have a 
> different way to describe the setting of orthorhombic space groups with 1 or 
> 2 screw axes
> This seems relevant for this particular project, and is discussed in 
> https://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Pointless
> In short, to make the POINTLESS setting compatible with that of XDS, you want 
> to use
> SETTING SYMMETRY-BASED
> in the POINTLESS input. XDSGUI does that for you.
> 
> HTH,
> Kay
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Creating map volume of custom size

[Phil Evans]

I think the CCP4 program map rot will do this
Phil

> On 13 Jul 2018, at 10:16, melanie.voll...@diamond.ac.uk 
>  wrote:
> 
> Hello everyone,
> 
> 
> I have a question about manipulating map files.
> 
> 
> I want to use my map derived from X-ray diffraction and create a standard 
> volume for it, say 300A3.
> 
> This is to be able to create slices of equal dimensions along all three 
> directions of the electron density stored in the map file. Currently my 
> slices differ in dimensions as the default volume is defined by the unit cell.
> 
> 
> I don't think there is anything in the map tools from the X-ray field I know. 
> I had a play with Chimera and I think there is a tool in there to use some 
> form of reference cell created from coordinates. But I don't want to make use 
> of a coordinates file.
> 
> 
> I just want to inflate my unit cell to a standard volume.
> 
> 
> Any ideas would be very welcome.
> 
> 
> Thanks
> 
> 
> Melanie
> 
> -- 
> This e-mail and any attachments may contain confidential, copyright and or 
> privileged material, and are for the use of the intended addressee only. If 
> you are not the intended addressee or an authorised recipient of the 
> addressee please notify us of receipt by returning the e-mail and do not use, 
> copy, retain, distribute or disclose the information in or attached to the 
> e-mail.
> Any opinions expressed within this e-mail are those of the individual and not 
> necessarily of Diamond Light Source Ltd. 
> Diamond Light Source Ltd. cannot guarantee that this e-mail or any 
> attachments are free from viruses and we cannot accept liability for any 
> damage which you may sustain as a result of software viruses which may be 
> transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England and 
> Wales with its registered office at Diamond House, Harwell Science and 
> Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
> 
> 
> 
> 
> To unsubscribe from the CCP4BB list, click the following link:
> https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1



To unsubscribe from the CCP4BB list, click the following link:
https://www.jiscmail.ac.uk/cgi-bin/webadmin?SUBED1=CCP4BB=1

Re: [ccp4bb] Script / matrix for coordinate transformation from a cubic I cell to its primitive P cell ?

[Phil Evans]

If you can transform the coordinates correctly, then Pointless can transform 
the data to match, using the coordinates as reference

pointless < On 17 May 2018, at 12:39, Eleanor Dodson 
> <176a9d5ebad7-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> Yes - here is a bit of the  output from reindexing a set of H32 data to C2
>   
> It tells you this and issues some warnings!
> 
> 
> 
>   You are changing the symmetry of merged data  are you SURE you know 
> what you are doing
> 
> 
> 
>  $TEXT:Warning: $$ comment $$
>  WARNING: ** Symmetry change of merged data **
>  $$
> 
> 
>   New unit cell determined from reindexing:   163.50  136.44  106.47   90.00  
> 103.48   90.00
> 
> 
> 
> 
>  Data line--- symmetry C2
>  Data line--- reindex HKL -h/3 -2k/3 -2l/3, h, -h/3-2k/3+l/3
>  Data line--- noreduce
>  Data line--- end
> 
>   Reflections will be reindexed, and unit cell recalculated
> 
>  Reindexing transformation:
>(h' k' l') =  ( h  k  l ) ( -0.3  1.0 -0.3 )
>  ( -0.7  0.0 -0.7 )
>  ( -0.7  0.0  0.3 )
> 
> 
> 
>  Real axes transformed by same matrix:
>(a' b' c') =  ( a  b  c ) ( -0.3  1.0 -0.3 )
>  ( -0.7  0.0 -0.7 )
>  ( -0.7  0.0  0.3 )
> 
> 
> 
>  Reciprocal axes transformed by inverse matrix:
>   (a*')(  0.0 -0.5 -1.0 ) ( a*)
>   (b*')  = (  1.0 -0.5  0.0 ) ( b*)
>   (c*')(  0.0 -1.0  1.0 ) ( c*)
> 
> 
>  FRACTIONAL coordinates transformed by same matrix:
>   (x') (  0.0 -0.5 -1.0 ) ( x)
>   (y')   = (  1.0 -0.5  0.0 ) ( y)
>   (z') (  0.0 -1.0  1.0 ) ( z)
> 
> 
> On 16 May 2018 at 22:20, Eleanor Dodson  wrote:
> Of course you need to give pdbset the new cell too...
> E
> 
> On 16 May 2018 at 22:20, Eleanor Dodson  wrote:
> Hmm - I think you need
> pdbset
> symgen x-y/2,y/2,z
> 
> Dont have reindex output at home but doesnt it tell you 
> [h' k' l'] = [h k l ] [ 1 1 0]
>   [ 0 2 0]
>   [ 0 0 1] 
> 
> [a* ' ].  [ 1. -1/2.  0].   [a*]
> [b* '].   =.  [ 0   1/2   0].  [b*]
> [c* '].[ 0. 01]   [c*]
> 
> [a'  b'  c']  = [a b c ] [ 1 1 0]
>[ 0 2 0]
>[ 0 0 1] 
> 
> [x '][ 1. -1/2.  0].   [x]
> [y '].   =.  [ 0   1/2   0].  [y]
> [z '].[ 0. 01]   [z]
> 
> 
> Cheers Eleanor
> 
> 
> 
> 
> On 16 May 2018 at 01:11, James Holton  wrote:
> 
> Wow, really?  I thought all reindex gives us is the axis transformation, not 
> the coordinate transform.
> 
> I just tried going from P622 into C222, starting with 3hjd.  By using 
> othercell, I can get an operator for transforming the data: h,h+2k,l.  
> Applying this in reindex, I get the matrix:
> 
> Real axes transformed by same matrix:
>(a' b' c') =  ( a  b  c ) (  1.0  1.0  0.0 )
>  (  0.0  2.0  0.0 )
>  (  0.0  0.0  1.0 )
> But if I apply:
> 
> pdbset xyzin 3hjd.pdb xyzout test.pdb << EOF
> symgen X,X+Y+Y,Z
> EOF
> I get some rather significant geometric distortions.
> 
> Am I doing something wrong?  Or is this just harder than it seems?  
> I find it is a common problem my users face.  They can get an MR solution if 
> they over-merge their data, but not when it is merged in any other space 
> group.  The transition from the P622 to C222 is the most thorny one.  
> Twinning in C222 can get you to apparent P622, and I wonder if ignorance of 
> how to transform the coordinates is the reason why there are no examples of 
> this in the PDB.
> 
> Thanks in advance for your always-appreciated insight!
> 
> -James
> 
> 
> 
> On 5/15/2018 3:15 AM, Eleanor Dodson wrote:
>> Well - if you use reindex to change the reflections from I213 to P1 the log 
>> file gives the rotation matrix need to convert I213 coordinates using the 
>> same convention.
>> There are various clever inputs to reindex which allow you to do this 
>> 
>> Then you can use 
>> 
>> pdbset   xyzin I213.pdb xyzout P1.pdb giving that matrix.
>> From the doc..
>> 
>> ROTATE [INVERT] [MATRIX|EULER|POLAR] values
>> 
>> Define rotational transformation, either as MATRIX (this keyword may be 
>> omitted) followed by 9 numbers (r11 r12 r13 r21 r22 r23 r31 r32 r33), by 
>> keyword EULER followed by Eulerian angles alpha, beta, gamma (as in ALMN), 
>> or by keyword POLAR followed by polar angles omega, phi, kappa (as in 
>> POLARRFN). This transformation will be applied to all atoms. The SHIFT 
>> command may be used to define a translation in addition. The transformation 
>> defined 

Re: [ccp4bb] improper rotation matrix

[Phil Evans]

The command ROTATE in pdbset should allow to apply any valid rotation matrix, 
ie determinant = +1.0

Anything else will distort the coordinates


> On 12 Mar 2018, at 10:13, Franck Coste  wrote:
> 
> Hi all,
> 
> I'd like to apply an improper rotation matrix to PDB files but it seems it's 
> not allowed in pdbset. Does anyone know a program where I could do this ?
> 
> Thanks in advance.
> 
> Regards,
> 
> Franck.
> -- 
> 

Re: [ccp4bb] Unknown density

[Phil Evans]

Mg2+(H20)6 ?

> On 6 Mar 2018, at 22:19, Rajesh Kumar  wrote:
> 
> Dear All,
> 
> Have you had experience with this kind of density? I am wandering what this 
> could be?
> 
> Thank you very much for the help.
> 
> -Rajesh
> 
> 
> 
> 

Re: [ccp4bb] reindexing mtz

[Phil Evans]

I don’t think Pointless can (or should) read .x files - they have to go into 
scalepack as that sorts out various things in the data. Pointless can read 
unmerged .sca files from scalepack, but they can’t be (properly) scaled in 
Aimless

Phil

> On 10 Jan 2018, at 21:11, CCP4BB 
> <193323b1e616-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> Hi Peter
> 
> No, don't bother scaling in hkl2000.
> 
> Just take the .x files and read them into Pointless directly - it should 
> auto-detect the file type and any output reflection file from it will be in 
> MTZ format. Then Aimless can do the scaling. 
> 
> See the manual, e.g. 
> 
> https://www.mrc-lmb.cam.ac.uk/harry/pre/pointless.html
> 
> for details.
> 
> You should be able to do this in ccp4i2 as well.
> 
> If you use the already merged reflections for reindexing, there's a chance 
> that reflections that are not equivalent will have been merged.
> 
> Harry
> --
> Dr Harry Powell
> Chairman of European Crystallographic Association SIG9 (Crystallographic 
> Computing) 
> 
> On 10 Jan 2018, at 18:58, Peter Hsu  wrote:
> 
>> Just to be absolutely clear about my approach to doing this, I should scale 
>> in HKL2000 as an unmerged set, and then take the unmerged scalepack, convert 
>> to mtz and run through pointless? 
>> 
>> Also, would taking my current mtz (which is a merged dataset) work using the 
>> reindex program in CCP4? Are there any pitfalls I need to be aware of? 
>> 
>> Apologies for the ignorance.
>> 
>> Thanks,
>> Peter

Re: [ccp4bb] double cell dimensions between P2 and C2

[Phil Evans]

You should look critically at the indexing of the images for both cases. Does 
the lattice interpret all spots, or are half of them missing


> On 9 Nov 2017, at 10:02, Markus Heckmann  wrote:
> 
> Dear all,
>> From a small protein, gives crystals P2 with cell
> Cell 53.16   65.73   72.8990  110.94  90
> (has 3 molecules in the asymmetric unit). Tested with pointless. Does
> not give any other possibility.
> 
> Another crystal if the same protein, similar conditions:
> C2
> Cell 109.14  124.37   73.4290  111.75  90. This has 6
> molecules in the a.s.u. Tested with pointless. Does not give any other
> possibility.
> The cell length a, b of C2 is twice that of P2.
> 
> Is it usual to get such crystals from similar conditions or am I
> missing something?
> 
> Many thanks,
> Mark

Re: [ccp4bb] HKL to mtz

[Phil Evans]

Note that Combat and Scala are both obsolete

ccp4i2 has useful tasks for importing unmerged data (Data reduction) or merged 
data (Import merged data) from various formats

Phil


> On 6 Oct 2017, at 07:05, Ruud Hovius  wrote:
> 
> Hello, 
> 
> Using Combat followed by Scala in CCP4 works well with as input the .HKL from 
> XDS.
> or, Pointless-Aimless-Ctruncate with which it is also easy to exclude batches 
> .
> 
> best greetings, Ruud
> 
> On 6/10/17 05:01, ameya benz wrote:
>> Hi,
>> 
>> I want to convert HKL file to mtz. I tried using F2mtz but somehow the 
>> output mtz is not working. What parameters should I set during conversion. 
>> Or can anyone suggest alternative to F2mtz?
>> 
>> regards,
>> Ameya
>> National chemical laboratory, Pune, India
> 
> -- 
> 
> Ruud Hovius
> EPFL SB ISIC LIP
> BCH 4209
> CH-1015 Lausanne
> +41-21-693-9442
> 
> http://lip.epfl.ch

Re: [ccp4bb] Aimless number of uniques

[Phil Evans]

What do you mean by “unique”? In the absence of anomalous signal, I+ and I- 
should be the same, so are not independent, and it doesn’t make sense to count 
them separately. Even if there is some anomalous signal, it is generally small 
on average compared to the whole intensity, so I+ and I- are highly correlated 
- should they be counted separately? What is the count for anyway?

I think it is confusing to count them differently depending on whether there is 
anomalous signal, however small. 

So “Number of unique refections” does depend on how you define it, and I would 
define it as the count of I+ or I-

Phil


> On 2 Aug 2017, at 21:02, Eugene Osipov <e.m.osi...@gmail.com> wrote:
> 
> Dear Phil, 
> it is much more confusing. Can you please explain me why do you think they 
> are non unique in case of anomalous signal?
> 
> 
> 2 авг. 2017 г. 19:01 пользователь "Phil Evans" <p...@mrc-lmb.cam.ac.uk> 
> написал:
> 
> Anomalous ON or OFF affects some analyses, and outlier rejection: it doesn’t 
> change which reflections are used. It always outputs I+ and I-, even with 
> ANOMALOUS OFF
> 
> Phenix (and maybe XDS) count I+ & I- as two separate unique reflections: 
> Aimless counts them as one, which I think is more reasonable. I+ and I- are 
> strongly correlated
> 
> Phil
> 
> > On 2 Aug 2017, at 16:33, Eugene Osipov <e.m.osi...@gmail.com> wrote:
> >
> > Dear community,
> >
> > I am confused by a reported number of unique reflections in AIMLESS version 
> > 0.5.32 : 29/03/17 ( CCP4 7.0.043).
> > I am processing dataset with strong anomalous signal but it seems that 
> > "anomalous" keyword does not affect number of unique reflections.
> > I run:
> > $pointless XDS_ASCII.HKL HKLOUT pointless.mtz
> > .
> > $aimless hklin pointless.mtz hklout aimless.mtz
> >
> > After a program run with either "ANOMALOUS ON" or "ANOMALOUS OFF" I see 
> > 6629 reflections. Meanwile XDS and Phenix report  12 200 unique reflections.
> > Is it bug or I miss something?
> >
> >
> > --
> > Eugene Osipov
> > Junior Research Scientist
> > Laboratory of Enzyme Engineering
> > A.N. Bach Institute of Biochemistry
> > Russian Academy of Sciences
> > Leninsky pr. 33, 119071 Moscow, Russia
> > e-mail: e.m.osi...@gmail.com
> 

Re: [ccp4bb] ccp4 development resources

[Phil Evans]

Dear Lothar

[I’ve copied this to the CCP4 team in case they have suggestions]

I don’t think there is any specific guidance for writing a “CCP4 program” - the 
programs distributed in the suite have been written in a diversity of styles 
and languages over many years - but I can offer a few thoughts

As a former Fortran programmer, I wouldn’t recommend Fortran now. There some 
really useful libraries to handle common crystallographic calculations, written 
in C++, and accessible also from Python, but not easily from Fortran. These 
include the Clipper library in CCP4 (from Kevin Cowtan); and cctbx, which is 
used by Phenix, but is also distributed by CCP4. These libraries also provide 
access to the file formats for reflection lists and coordinates, see below. 
Personally, I write in C++, but the use of Python at the high level along with 
C++ libraries to do the compute-intensive parts is also popular, and this 
approach is used in Phenix and DIALS

Reflection lists should be read and written using the binary MTZ file format, 
using the libraries. The “normal” merged MTZ files (each hkl occurring only 
once) can most easily be accessed using the Clipper or cctbx libraries, though 
the lower level CCP4 libraries can also be used. Unmerged reflection lists 
(multiple observations fro each hkl), in the data reduction steps, are a bit 
more complicated. Traditionally MTZ files may contain multiple types of data 
for each hkl, identified by column lables,e.g. different observed intensities, 
or Fs, phases, figures of merit, etc, but the recent ccp4 GUI ccp4i2 stores 
“miniMTZ” files containing only one type of data, e.g. observed I/sigma(I), 
phase/figure-of-merit, map coefficients etc, and this may be more used in future

Coordinate files also should be read and written using libraries, as this 
allows automatic switching between old-style PDB format and mmCIF. Clipper 
provides easy APIs for simple use, built on top of the more elaborate but 
flexible MMDB2 library

I find the CCP4 build system inconvenient for development work, though is good 
for distribution:  I just use an editor and a Makefile, old school, but other 
people probably have other more modern ways of working

Looking at existing (preferably reasonably recent) programs is useful to see 
various ways of accessing the general framework of reflection lists and 
coordinate lists which are central to most crystallographic calculations

Good luck
Phil


> On 13 Jul 2017, at 15:54, esse...@helix.nih.gov wrote:
> 
> Dear Prof. Evans,
> 
>  I am writing to you to solicit your expertise as a ccp4 developer. The
> principal investigator of our group has interest in developing ccp4-based
> programs and has done so with some success. However, I was wondering if
> there is a comprehensive set of instructions how to write ccp4 programs well
> and which tools to use incl. perhaps IDEs like Eclipse or so.  I know about
> the developer page on ccp4's website but it seems to be heavy on interfaces
> i.e. I have not found a clear comprehensive set of tools for ccp4 developers
> that includes how to write makefiles etc. I think the PI is mostly
> interested in small stand-alone fortran program that do not require much
> graphics. Can you give me a hint ? It is possible that I cannot see the
> forest for all the trees - in which case I apologize.
> 
> Thanks and kind regards,
> 
>  Dr. Lothar Esser
> 
>  National Institutes of Health
>  National Cancer Institute.
> 

Re: [ccp4bb] [ccp4bb] Rmergicide Through Programming

[Phil Evans]

> 

> (2) Rsym has just:
> 
>> R sym value in percent.
> 
> see 
> .
>  To be honest, if that is the official definition it does make me wonder what 
> it is doing in the dictionary at all.
> 
> Regards,
> Peter.
> 


Indeed

Re: [ccp4bb] AW: [ccp4bb] Rmergicide Through Programming

[Phil Evans]

What is the difference between Rmerge and Rsym - I thought they were the same?
Rrim == Rmeas I think

Phil



> On 10 Jul 2017, at 15:18, John Berrisford  wrote:
> 
> Dear Herman
> 
> The new PDB deposition system (OneDep) allows you to enter values for Rmerge, 
> Rsym, Rpim, Rrim and / or CC half. If, during deposition, you do not provide 
> a value for any of these metrics then we will ask you for a value for one of 
> them.
> 
> Also, PDB format is a legacy format for the PDB. In 2014 mmCIF became the 
> archive format for the PDB and some large entries are no longer distributed 
> in PDB format. mmCIF is not limited by the constraints of punch cards.
> 
> Please see https://www.wwpdb.org/documentation/file-formats-and-the-pdb
> 
> Regards
> 
> John
> 
> PDBe
> 
> 
> 
> On 10/07/2017 09:26, herman.schreu...@sanofi.com wrote:
>> Dear All,
>> 
>> For me this whole discussion is an example of a large number of people 
>> barking at the wrong tree. The real issue is not whether data processing 
>> programs print amongst many quality indicators an Rmerge as well, but the 
>> fact that the PDB and many journals still insist on using the Rmerge as 
>> primary quality indicator. As long as this is true, novice scientist might 
>> be led to believe that Rmerge is the most important quality indicator. As 
>> soon as the PDB and the journals request some other indicator, this will be 
>> over. So that is where we should direct our efforts to.
>> 
>> I don't understand at all, why the PDB still insists on an obsolete quality 
>> indicator. However, the PDB format for the coordinates also dates back to 
>> the 1960's to be used with punch cards.
>> 
>> My 2 cents.
>> Herman
>> 
>> 
>> 
>> -Ursprüngliche Nachricht-
>> Von: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] Im Auftrag von 
>> Edward A. Berry
>> Gesendet: Samstag, 8. Juli 2017 22:31
>> An: CCP4BB@JISCMAIL.AC.UK
>> Betreff: Re: [ccp4bb] Rmergicide Through Programming
>> 
>> But R-merge is not really narrower as a fraction of the mean value- it just 
>> gets smaller proportionantly as all the numbers get smaller:
>> RMSD of .0043 for R-meas multiplied by factor of 0.022/.027 gives 0.0035 
>> which is the RMSD for Rmerge. The same was true in the previous example. You 
>> could multiply R-meas by .5 or .2 and get a sharper distribution yet! And 
>> that factor would be constant, where this only applies for super-low 
>> redundancy.
>> 
>> On 07/08/2017 03:23 PM, James Holton wrote:
>>> The expected distribution of Rmeas values is still wider than that of 
>>> Rmerge for data with I/sigma=30 and average multiplicity=2.0. Graph 
>>> attached.
>>> 
>>> I expect that anytime you incorporate more than one source of information 
>>> you run the risk of a noisier statistic because every source of information 
>>> can contain noise.  That is, Rmeas combines information about multiplicity 
>>> with the absolute deviates in the data to form a statistic that is more 
>>> accurate that Rmerge, but also (potentially) less precise.
>>> 
>>> Perhaps that is what we are debating here?  Which is better? accuracy or 
>>> precision?  Personally, I prefer to know both.
>>> 
>>> -James Holton
>>> MAD Scientist
>>> 
>>> On 7/8/2017 11:02 AM, Frank von Delft wrote:
 It is quite easy to end up with low multiplicities in the low resolution 
 shell, especially for low symmetry and fast-decaying crystals.
 
 It is this scenario where Rmerge (lowres) is more misleading than Reas.
 
 phx
 
 
 On 08/07/2017 17:31, James Holton wrote:
> What does Rmeas tell us that Rmerge doesn't?  Given that we know the 
> multiplicity?
> 
> -James Holton
> MAD Scientist
> 
> On 7/8/2017 9:15 AM, Frank von Delft wrote:
>> Anyway, back to reality:  does anybody still use R statistics to 
>> evaluate anything other than /strong/ data?  Certainly I never look at 
>> it except for the low-resolution bin (or strongest reflections). 
>> Specifically, a "2%-dataset" in that bin is probably healthy, while a 
>> "9%-dataset" probably Has Issues.
>> 
>> In which case, back to Jacob's question:  what does Rmerge tell us that 
>> Rmeas doesn't.
>> 
>> phx
>> 
>> 
>> 
>> 
>> On 08/07/2017 17:02, James Holton wrote:
>>> Sorry for the confusion.  I was going for brevity!  And failed.
>>> 
>>> I know that the multiplicity correction is applied on a per-hkl basis 
>>> in the calculation of Rmeas.  However, the average multiplicity over 
>>> the whole calculation is most likely not an integer. Some hkls may be 
>>> observed twice while others only once, or perhaps 3-4 times in the same 
>>> scaling run.
>>> 
>>> Allow me to do the error propagation properly.  Consider the scenario:
>>> 
>>> Your outer resolution bin has a true I/sigma = 1.00 and average 
>>> multiplicity of 2.0. Let's say there are 100 hkl indices in this 

Re: [ccp4bb] REBATCH error mystery

[Phil Evans]

REBATCH is old and I can’t remember how it works :-(

> On 6 Jul 2017, at 15:18, Graeme Winter <graeme.win...@diamond.ac.uk> wrote:
> 
> HI Phil,
> 
> Yes, probably. However it’s still a mystery I would like to understand. Also, 
> I use REBATCH for lots of things not just trimming data sets (i.e. manually 
> updating BATCH numbers) and - this is the real fun one - if I rerun the 
> processing with 900 images REBATCH works fine!
> 
> … following this train of thought …
> 
> Could this be as simple as MAXBAT needing to be reassigned / code recompiled? 
> In this happy world of 100,000 image + data sets guess 5,000 images may not 
> be enough (though the program does not check whether #batches > MAXBAT)
> 
> Anyhow, I suspect that the kernel of your message, that after 2.3 decades 
> there are better tools available, is almost certainly correct. Will look 
> elsewhere.
> 
> Thanks & best wishes Graeme
> 
> 
>> On 6 Jul 2017, at 15:10, Phil Evans <p...@mrc-lmb.cam.ac.uk> wrote:
>> 
>> REBATCH should generally be replaceable by Pointless I think
>> 
>> Latest latest Pointless (prerelease update 043, version 1.11.3) removes 
>> batch headers  for leading or trailing batch numbers which are not present 
>> in the reflection list (for you, Graeme)
>> 
>> 
>> 
>> 
>>> On 6 Jul 2017, at 15:04, Graeme Winter <graeme.win...@diamond.ac.uk> wrote:
>>> 
>>> Afternoon all,
>>> 
>>> Technical harking back to the 90’s FORTRAN CCP4 question if I may
>>> 
>>> I have an MTZ file which has no spots on first couple of images, so the 
>>> batch headers at the start exist but there are no BATCH values which 
>>> correspond to these in the actual reflection data
>>> 
>>> I have been bashing my head against the wall trying to get away from this 
>>> error:
>>> 
>>> 
>>> 
>>> 
>>> 
>>> Output File
>>> 
>>> 
>>> *** Wrong batch! batch number in file is   3, batch number from 
>>> internal lookup1
>>> 
>>> 
>>> REBATCH:   *** Crazy batch number ***
>>> REBATCH:   *** Crazy batch number ***
>>> Times: User:   0.6s System:0.0s Elapsed: 0:01
>>> 
>>> 
>>> 
>>> 
>>> Does anyone out there know how one should “fix” this?
>>> 
>>> MTZDUMP output follows, which all looks sensible to me…
>>> 
>>> I also find the following at the top of the source code ;o)
>>> 
>>> C REBATCH
>>> C Copyright (C) 1994 Phil Evans
>>> 
>>> However I am certain other experts out there will also be able to help
>>> 
>>> I’ve worked my way through the source code but can’t really work out what 
>>> is happening here...
>>> 
>>> Cheers Graeme
>>> 
>>> 
>>>  
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> ###
>>> ###
>>> ###
>>> ### CCP4 7.0.042: MTZDUMP  version 1.1 : ##
>>> ###
>>> User: graeme  Run date:  6/ 7/2017 Run time: 14:59:21
>>> 
>>> 
>>> Please reference: Collaborative Computational Project, Number 4. 2011.
>>> "Overview of the CCP4 suite and current developments". Acta Cryst. D67, 
>>> 235-242.
>>> as well as any specific reference in the program write-up.
>>> 
>>> 
>>> 
>>> OPENED INPUT MTZ FILE
>>> Logical Name: HKLIN   Filename: integrated.mtz
>>> 
>>> * Title:
>>> 
>>> from dials.export_mtz
>>> 
>>> * Base dataset:
>>> 
>>>  0 HKL_base
>>>HKL_base
>>>HKL_base
>>> 
>>> * Number of Datasets = 1
>>> 
>>> * Dataset ID, project/crystal/dataset names, cell dimensions, wavelength:
>>> 
>>>  1 DIALS
>>>XTAL
>>>FROMDIALS
>>>   54.2171   57.9174   66.5815   90.   90.   90.
>>>   0.96770
>>> 
>>> * Number of Columns = 17
>>> 
>>> * Number of Reflections = 2004565
>>> 
>>> * Missing value set to NaN in input mtz file
>>> 
>>> * Number of Batches = 12000
>&g

Re: [ccp4bb] REBATCH error mystery

[Phil Evans]

REBATCH should generally be replaceable by Pointless I think

Latest latest Pointless (prerelease update 043, version 1.11.3) removes batch 
headers  for leading or trailing batch numbers which are not present in the 
reflection list (for you, Graeme)




> On 6 Jul 2017, at 15:04, Graeme Winter <graeme.win...@diamond.ac.uk> wrote:
> 
> Afternoon all,
> 
> Technical harking back to the 90’s FORTRAN CCP4 question if I may
> 
> I have an MTZ file which has no spots on first couple of images, so the batch 
> headers at the start exist but there are no BATCH values which correspond to 
> these in the actual reflection data
> 
> I have been bashing my head against the wall trying to get away from this 
> error:
> 
> 
> 
> 
> 
> Output File
> 
> 
> *** Wrong batch! batch number in file is   3, batch number from internal 
> lookup1
> 
> 
> REBATCH:   *** Crazy batch number ***
> REBATCH:   *** Crazy batch number ***
> Times: User:   0.6s System:0.0s Elapsed: 0:01
> 
> 
> 
> 
> Does anyone out there know how one should “fix” this?
> 
> MTZDUMP output follows, which all looks sensible to me…
> 
> I also find the following at the top of the source code ;o)
> 
> C REBATCH
> C Copyright (C) 1994 Phil Evans
> 
> However I am certain other experts out there will also be able to help
> 
> I’ve worked my way through the source code but can’t really work out what is 
> happening here...
> 
> Cheers Graeme
> 
> 
>  
> 
> 
> 
> 
> 
> 
> 
> ###
> ###
> ###
> ### CCP4 7.0.042: MTZDUMP  version 1.1 : ##
> ###
> User: graeme  Run date:  6/ 7/2017 Run time: 14:59:21
> 
> 
> Please reference: Collaborative Computational Project, Number 4. 2011.
> "Overview of the CCP4 suite and current developments". Acta Cryst. D67, 
> 235-242.
> as well as any specific reference in the program write-up.
> 
> 
> 
> OPENED INPUT MTZ FILE
> Logical Name: HKLIN   Filename: integrated.mtz
> 
> * Title:
> 
> from dials.export_mtz
> 
> * Base dataset:
> 
>0 HKL_base
>  HKL_base
>  HKL_base
> 
> * Number of Datasets = 1
> 
> * Dataset ID, project/crystal/dataset names, cell dimensions, wavelength:
> 
>1 DIALS
>  XTAL
>  FROMDIALS
> 54.2171   57.9174   66.5815   90.   90.   90.
> 0.96770
> 
> * Number of Columns = 17
> 
> * Number of Reflections = 2004565
> 
> * Missing value set to NaN in input mtz file
> 
> * Number of Batches = 12000
> 
> * HISTORY for current MTZ file :
> 
> From DIALS 1.dev.1541-ge526b95f, run on 06/07/2017 at 13:55:27
> 
> * Column Labels :
> 
> H K L M/ISYM BATCH IPR SIGIPR I SIGI BG SIGBG FRACTIONCALC XDET YDET ROT LP 
> DQE
> 
> * Column Types :
> 
> H H H Y B J Q J Q R R R R R R R R
> 
> * Associated datasets :
> 
> 0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 1 1
> 
> * Cell Dimensions : (obsolete - refer to dataset cell dimensions above)
> 
>   54.2171   57.9174   66.5815   90.   90.   90.
> 
> *  Resolution Range :
> 
>0.000520.68412 ( 43.698 -  1.209 A )
> 
> * Sort Order :
> 
>  0 0 0 0 0
> 
> * Space group = 'P222' (number 16)
> 
> 
> 
> 
> Batch number:
>  1Batch 1
> Batch number:
>  2Batch 2
> Batch number:
>  3Batch 3
> Batch number:
>  4Batch 4
> Batch number:
>  5Batch 5
> Batch number:
>  6Batch 6
> Batch number:
>  7Batch 7
> Batch number:
>  8Batch 8
> Batch number:
>  9Batch 9
> 
> ——8<——
> 
> Batch number:
>  11997Batch 11997
> Batch number:
>  11998Batch 11998
> Batch number:
>  11999Batch 11999
> Batch number:
>  12000Batch 12000
> 
> M/ISYM located at column4
> 
> BATCH located at column5
> 
> 
> OVERALL FILE STATISTICS for resolution range   0.001 -   0.684
> ===
> 
> 
> Col SortMinMaxNum  % Mean Mean   Resolution   Type 
> Column
> num order   Missing complete  abs.   LowHigh   
> label
> 
>   1 NONE 0  36  0  100.00 15.0 15.0  43.70   1.21   H  H
>   2 NONE 0  39  0  100.00 16.6 16.6  43.70   1.21   H  K
>   3 NONE 0 

Re: [ccp4bb] <4SSQ/LL> and resolution (in refmac logfile)

[Phil Evans]

4(sin theta/lambda)^2 = 1/d^2

> On 14 Jun 2017, at 09:30, Wei Ding  wrote:
> 
> Dear all,
> what is the relationship between  <4SSQ/LL> and resolution?
> In refmac logfile, there is a table:
>  <4SSQ/LL> NREFa  FOMa  NREFc FOMc NREFall FOMall  SigmaA_Fc1 FSCfree  
> FSCwork CorrFoFcFree CorrFoFcWork$$
>  $$
>   0.0065130   0.894171   0.872301   0.882  0.734  0.8235  0.8078  
> 0.4204  0.5580
>   0.0178288   0.879174   0.856462   0.871  0.724  0.8470  0.7811  
> 0.6633  0.6237
>   0.0292394   0.882179   0.814573   0.860  0.748  0.8308  0.7431  
> 0.7682  0.5245
> 
> but when I open this logfile by logview, this table become a graph, and those 
> values are convert to Resolution[A]:  4.47, 3.16 
> So what is the transform formula between 4SSQ/LL and Resolution?
> Thank you for your time.
> 
> 
> --
> Wei Ding
> P.O.Box 603
> The Institute of Physics,Chinese Academy of Sciences
> Beijing,China
> 100190
> Tel: +86-10-82649083
> E-mail: ding...@iphy.ac.cn
> 
> 
>  
> 
> 
>  

Re: [ccp4bb] Optimising data processing of a I432 dataset with 75% solvent content.

[Phil Evans]

It is puzzling that in the high res shell (to 3.1A) CC(1/2) is 0.533 (OK), but 
I/sigI is 0.1 (v low). Is the SDcorrection (SDFAC, SDADD) sensible (and 
“Analysis of standard deviations vs. intensity”)? In Aimless the determination 
of SDFAC and SDADD sometimes go haywire - with high multiplicity you could try 
“SDCORRECTION SAMPLESD” to see what it does to mean I/sigI

 Is there a lot of radiation damage (you say not, but if so with the high 
multiplicity you could cut the end of the data)? 

At least the data can’t be anisotropic in the cubic space group!

Probably none of this helps in improving maps. It may be worth looking at 
sharpened maps (from refmac or in coot)

Phil


> On 18 May 2017, at 12:47, Michael Jarva 
> <1295eb3572d0-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> Dear all,
> 
> I have a dataset that have two very interesting properties: a) It's in I432, 
> and b) has a whooping 75% solvent content.  
> You might think that the solvent content obviously is a big red flag, and so 
> did I, but I have phased this successfully with just one monomer, and the 
> packing result does makes a lot of sense. The resulting maps contain no extra 
> umodelled blobs, and trying to phase it with an additional molecules does not 
> give a good solution.
> 
> The problem I have is that the diffraction intensity/Rmerge plummets/explodes 
> around the 3.5Å mark (I assume because of the high solvent content) to such 
> an extent that even though I have little radiation damage, 100% completeness 
> in high resolution shells, and very high redundancy, any attempt to merge the 
> dataset at a higher resolution has so far given no improvement to the maps.
> 
> I'm hoping that there might be a few tricks out there I can apply to the spot 
> finding/integration/scaling steps have it merge in a even slightly higher 
> resolution than I currently have been able to do.
> Although I have a feeling that the only thing I can do is to grow another, 
> much bigger, crystal…
> 
> many thanks for any feedback
> /michael
> 
> See below for sample outputs from aimless:
> 
>Overall  InnerShell  OuterShell
> Low resolution limit   43.50 43.50  3.32
> High resolution limit   3.10  8.78  3.10
> 
> Rmerge  (within I+/I-) 0.079 0.01021.891
> Rmerge  (all I+ and I-)0.081 0.01122.502
> Rmeas (within I+/I-)   0.084 0.01123.102
> Rmeas (all I+ & I-)0.084 0.01123.169
> Rpim (within I+/I-)0.027 0.004 7.335
> Rpim (all I+ & I-) 0.020 0.003 5.450
> Rmerge in top intensity bin0.010- - 
> Total number of observations   34917  1495  6448
> Total number unique 2057   112   362
> Mean((I)/sd(I)) 18.3 130.9   0.1
> Mn(I) half-set correlation CC(1/2) 1.000 1.000 0.533
> Completeness99.9  97.4 100.0
> Multiplicity17.0  13.3  17.8
> 
>   Overall  InnerShell  OuterShell
> Low resolution limit   43.50 43.50  3.84
> High resolution limit   3.50  8.58  3.50
> 
> Rmerge  (within I+/I-) 0.052 0.011 2.422
> Rmerge  (all I+ and I-)0.056 0.012 2.659
> Rmeas (within I+/I-)   0.055 0.011 2.553
> Rmeas (all I+ & I-)0.058 0.013 2.738
> Rpim (within I+/I-)0.017 0.004 0.804
> Rpim (all I+ & I-) 0.014 0.003 0.644
> Rmerge in top intensity bin0.010- - 
> Total number of observations   24596  1690  6071
> Total number unique 1462   120   343
> Mean((I)/sd(I)) 25.8 132.0   1.0
> Mn(I) half-set correlation CC(1/2) 1.000 1.000 0.771
> Completeness99.8  97.6 100.0
> Multiplicity16.8  14.1  17.7
> 
> 

Re: [ccp4bb] convert hkl data to mtz

[Phil Evans]

You can do this in the data reduction task in ccp4i2, which runs Pointless, 
Aimless and Ctruncate
Phil

> On 7 May 2017, at 23:10, Careina Edgooms 
> <02531c126adf-dmarc-requ...@jiscmail.ac.uk> wrote:
> 
> Hi
> 
> I have not been in the game of solving structures for years. In the past I 
> used the program combat in ccp4 to convert my hkl output file from SAINT into 
> mtz file. Now I don't see combat anymore as an option in ccp4 programs and 
> none of the programs recognise my SAINT output file. Please help
> 
> Careina

Re: [ccp4bb] Large number of outliers in the dataset

[Phil Evans]

It is not clear to me why you believe that cutting the resolution of the data 
would improve your model (which after all is the aim of refinement). At the 
edge CC(1/2) and I/sigI are perfectly respectable, and there doesn’t seem to be 
anything wrong with the Wilson plot. Th R-factor will of course be higher if 
you include more weak data, but minimising R is _not_ the aim of refinement. 
You should keep all the data

I don’t know what xtriage means by “large number of outliers”: perhaps someone 
else can explain

Phil



> On 29 Mar 2017, at 14:54, Juliana Ferreira de Oliveira 
>  wrote:
> 
> Hello,
> I have one dataset at 2.3 Å (probably it can be better, I/σ = 2.1 and CC1/2 = 
> 0.779, the summary data is below), but when I perform Xtriage analysis it 
> says that “There are a large number of outliers in the data”. The space group 
> is P212121. When I refine the MR solution the Rfree stops around 30% and it 
> doesn´t decrease (in fact if I continue refining it starts to increase).
> The Wilson plot graph is not fitting very well between 2.3 and 2.6 Å:
>  
> 
>  
> So I decided to cut the data at 2.6A and Xtriage analysis doesn’t notify 
> about outliers anymore. I could refine the MR solution very well, the final 
> Rwork is 0.2427 and Rfree = 0.2730 and validation on Phenix results in a good 
> structure.
> I run Zanuda to confirm the space group and it says that the space group 
> assignment seems to be correct.
> Do you think that I can improve my structure and solve it at 2.3 Å or better? 
> Or I can finish it with 2.6 Å? To publish at 2.6 Å I need to justify the 
> resolution cut, right? What should I say?
> Thank you for your help!
> Regards,
> Juliana
>  
> Summary data:
> OverallInnerShell  OuterShell
> Low resolution limit  51.51  51.51
>2.42
> High resolution limit  2.30 7.27  
>   2.30
> Rmerge   0.147   
> 0.054   0.487
> Rmerge in top intensity bin0.080   -  
> -
> Rmeas (within I+/I-)  0.155   0.057   
> 0.516
> Rmeas (all I+ & I-)0.155   0.057  
>  0.516
> Rpim (within I+/I-)0.048   0.017  
>  0.164
> Rpim (all I+ & I-)  0.048   0.017 
>   0.164
> Fractional partial bias-0.006 -0.003  
>0.146
> Total number of observations83988 2907
> 11885
> Total number unique  8145307  
> 1167
> Mean((I)/sd(I))   9.3   23.9  
>2.1
> Mn(I) half-set correlation CC(1/2)0.991   0.998   
> 0.779
> Completeness 99.9 99.5
>  100.0
> Multiplicity10.3 9.5  
>  10.2
>  
> Average unit cell: 37.57 51.51 88.75 90.00 90.00 90.00
> Space group: P212121
> Average mosaicity: 1.90
>  
>  
> Juliana Ferreira de Oliveira
> Brazilian Laboratory of Biosciences - LNBio
> Brazilian Center for Research in Energy and Materials - CNPEM
> Campinas-SP, Brazil

Re: [ccp4bb] How the IMEAN is calculated in scalepack2mtz

[Phil Evans]

Sorry Ian, I was attributing this to Kevin
apologies 
Phil


> On 12 Mar 2017, at 21:56, Ian Tickle  wrote:
> 
> 
> Eleanor,
> 
> Phil is right about the bias of the weighted mean, here's how I do it using a 
> minimum mean-squared error estimator which seems to give more accurate 
> results than either the unweighted or the weighted mean:
> 
> http://staraniso.globalphasing.org/Optimal-F-estimator.pdf
> 
> This is actually about Fmean but the same applies to Imean.
> 
> This actually only makes a significant difference to the result if there's a 
> big difference between sd(I+) and sd(I-), which is obviously not very common.
> 
> Cheers
> 
> -- Ian
> 
> 
> 
> On 12 March 2017 at 17:58, Eleanor Dodson  wrote:
>  You read: 
> h k l  IPLUS   SIGIPLUS INEG   SIGINEG
> Then program calculates this: 
> 
>SIGIMEAN = SIGIPLUS*SIGINEG/(SIGIPLUS+SIGINEG)
> 
>  IMEAN = (IPLUS/SIGIPLUS + INEG/SIGINEG)*SIGIMEAN
> 
> ie: IMEAN = ( IPLUS*SIGINEG   + INEG * SIGIPLUS ) / /(SIGIPLUS+SIGINEG)
> 
> Is that the right thing to do? Not sure! 
> 
> Eleanor Dodson
> 
> 
> 
> On 11 March 2017 at 15:18, Karthikeyan Subramanian  
> wrote:
> Dear CCP4bb,
> 
> How IMEAN and SIGIMEAN is calculated in scalepack2mtz if the input is with 
> anomalous intensity (obtained from HKL2000). Any guide/reference is highly 
> appreciated.
> 
> Thanks in advance,
> 
> With regards
> 
> Karthikeyan S.
> 
> Principal Scientist
> 
> IMTECH, Chandigarh
> 
> 

Re: [ccp4bb] How the IMEAN is calculated in scalepack2mtz

[Phil Evans]

It should be the unweighted mean to avoid bias towards I+ or I-
(or Kevin Cowtan had a more complicated way)
Phil


> On 12 Mar 2017, at 17:58, Eleanor Dodson  wrote:
> 
>  You read: 
> h k l  IPLUS   SIGIPLUS INEG   SIGINEG
> Then program calculates this: 
> 
>SIGIMEAN = SIGIPLUS*SIGINEG/(SIGIPLUS+SIGINEG)
> 
>  IMEAN = (IPLUS/SIGIPLUS + INEG/SIGINEG)*SIGIMEAN
> 
> ie: IMEAN = ( IPLUS*SIGINEG   + INEG * SIGIPLUS ) / /(SIGIPLUS+SIGINEG)
> 
> Is that the right thing to do? Not sure! 
> 
> Eleanor Dodson
> 
> 
> 
> On 11 March 2017 at 15:18, Karthikeyan Subramanian  
> wrote:
> Dear CCP4bb,
> 
> How IMEAN and SIGIMEAN is calculated in scalepack2mtz if the input is with 
> anomalous intensity (obtained from HKL2000). Any guide/reference is highly 
> appreciated.
> 
> Thanks in advance,
> 
> With regards
> 
> Karthikeyan S.
> 
> Principal Scientist
> 
> IMTECH, Chandigarh
> 

Re: [ccp4bb] POINTLESS error: "NormaliseInRuns::apply Unused run 0"

[Phil Evans]

There’s no reason in Pointless why it has to be a multiple of anything. That 
can’t be the problem
Phil

> On 24 Jan 2017, at 02:46, Kay Diederichs  
> wrote:
> 
> Hmm, that's really cryptic. In which respect does it matter that DATA_RANGE 
> is not a multiple of 50? DATA_RANGE takes two integer-valued parameters, 
>  and . As long as 0 < low <= high , and those frames are stored in 
> the .h5 files, you should be fine, at least from the viewpoint of XDS data 
> processing.
> 
> best,
> 
> Kay
> 
> On Mon, 23 Jan 2017 12:20:18 +, Joseph Brock  wrote:
> 
>> Hi Elanor and bb,
>> 
>> I realised this was due to me creating a .HKL file with XDS from Eiger data 
>> with a DATA_RANGE that was not a multiple of 50!
>> 
>> Apologies for any trouble!
>> 
>> All the best,
>> 
>> Joseph.
>> 
>> 
>> Joseph Brock | PhD
>> Division of Physiological Chemistry II
>> Department of Medical Biochemistry and Biophysics
>> Karolinska Institutet
>> Scheeles v�g 2
>> SE-171 77 Stockholm, Sweden
>> 
>> 
>> From: Eleanor Dodson [eleanor.dod...@york.ac.uk]
>> Sent: Saturday, 21 January 2017 1:07 PM
>> To: Joseph Brock
>> Cc: CCP4BB@JISCMAIL.AC.UK
>> Subject: Re: [ccp4bb] POINTLESS error: "NormaliseInRuns::apply Unused run 0"
>> 
>> can you send the whole log file?
>> Somehow the XDS input isnt agreeing with pointless expectations, so I 
>> suspect no reflections have been read
>> Eleanor
>> 
>> On 20 January 2017 at 23:41, Joseph Brock 
>> > wrote:
>> Dear ccp4 experts,
>> 
>> I am trying to use the ccp4i Pointless, Aimless, Ctruncate application to 
>> convert a XDS.HKL file. This usually works fine, however I suddenly have the 
>> task immediately fail with the error message below pasted from the bottom of 
>> the logfile.
>> 
>> If anyone can translate this error message for me I would be very grateful.
>> 
>> I am running up to date ccp4 (7.0.027) on Ubuntu 14.04.
>> 
>> 
>> 
>> ***
>> * Information from CCP4Interface script
>> ***
>> The program run with command: /usr/local/bin/ccp4-7.0/bin/pointless HKLOUT 
>> "/tmp/joe/jsb_32_2_mtz.tmp"
>> has failed with error message
>> NormaliseInRuns::apply Unused run 0
>> ***
>> 
>> 
>> #CCP4I TERMINATION STATUS 0 "NormaliseInRuns::apply Unused run 0"
>> #CCP4I TERMINATION TIME 21 Jan 2017  00:18:31
>> #CCP4I MESSAGE Task failed
>> 
>> 
>> Thanks in advance for the help.
>> 
>> Cheers,
>> Joe.
>> 
>> Joseph Brock | PhD
>> Division of Physiological Chemistry II
>> Department of Medical Biochemistry and Biophysics
>> Karolinska Institutet
>> Scheeles v�g 2
>> SE-171 77 Stockholm, Sweden

Re: [ccp4bb] CCP4BB Digest - 13 Jan 2017 to 14 Jan 2017 (#2017-15)

[Phil Evans]

Remember that 2xADP can disproportionate into ATP + AMP

> On 19 Jan 2017, at 20:54, Panneerselvam, Saravanan 
>  wrote:
> 
> Dear All,
> Sorry for the little bit off topic. Is there a possibility for covalent 
> bond formation between beta phosphate of ADP and acetate molecule both 
> are coordinated by divalent metal ions? I am working on a Kinase 
> structure  which was crystallized with ADP and in presence of 1M sodium 
> acetate. We observed additional density around ADP that fits perfectly 
> like a gamma phosphate , mimicking like ATP bound state, surrounded and 
> coordinated by two metal ions(resolution is 1.4A). There is a change in 
> space group (from I212121 to P212121 ) and further important 
> conformation changes are observed around ATP binding pocket and distant 
> region. This is the only xtal we obtained in this space group, and all 
> other xtals(measured 10 xtals)  from the same plate belong to I212121.
> 
> 
> 
> Thanks your help and time!
> 
> Saravanan
> - Original Message -
> From: "CCP4BB automatic digest system" 
> To: CCP4BB@JISCMAIL.AC.UK
> Sent: Sunday, 15 January, 2017 01:01:37
> Subject: CCP4BB Digest - 13 Jan 2017 to 14 Jan 2017 (#2017-15)
> 
> There are 2 messages totaling 45 lines in this issue.
> 
> Topics of the day:
> 
>   1. Phenix (2)
> 
> --
> 
> Date:Sat, 14 Jan 2017 20:58:09 +
> From:D Bonsor 
> Subject: Phenix
> 
> Is the phenix website down? Anyone know when it will be back up?
> 
> --
> 
> Date:Sat, 14 Jan 2017 14:45:06 -0800
> From:Pavel Afonine 
> Subject: Re: Phenix
> 
> See notice on Phenix mailing list that answers your question.
> 
> On Sat, Jan 14, 2017 at 12:58 PM, D Bonsor 
> wrote:
> 
>> Is the phenix website down? Anyone know when it will be back up?
>> 
> 
> --
> 
> End of CCP4BB Digest - 13 Jan 2017 to 14 Jan 2017 (#2017-15)
> 

Re: [ccp4bb] why pointless does not give statistics (CC1/2; N_CC; CCfit...) in certain resolution bin

[Phil Evans]

For some reason there is no data in that resolution bin

Phil

> On 9 Dec 2016, at 17:30, Xiao Lei  wrote:
> 
> Hi All, 
> 
> I have an x ray diffraction dataset of protein and dna complex processed with 
> pointless and I am trying to get the resolution cut for this data, the result 
> is below, I do not know why in the N=23 (Dmid=3.68) bin, there is no 
> statistic of CC(1/2), N_cc, CCfit, etc?  The program just put a "-" sign into 
> it.  I guess if this means something bad probably I have to cut the 
> resolution to 3.7A.
> 
> $TABLE: Mn(I/sigI) and CC(1/2) [in P1] vs. resolution:
> $GRAPHS:Resolution estimate 3.89A:0.000213587|0.0981986x0|1:2,4,6,7,9:
>  $$
>   N  1/d^2Dmid CC(1/2)   N_CC   CCfit  Mn(I/sigI)  N (I/sigI)/10   $$ 
> $$
>   1  0.0018  23.27   0.939 51   0.990   92.43142   9.243
>   2  0.0051  13.99   0.993210   0.987   72.33607   7.233
>   3  0.0084  10.92   0.984323   0.984   61.77937   6.177
>   4  0.0116   9.27   0.974344   0.981   54.77   1030   5.477
>   5  0.0149   8.19   0.969486   0.976   29.98   1412   2.998
>   6  0.0182   7.42   0.961491   0.971   13.24   1459   1.324
>   7  0.0214   6.83   0.879535   0.9648.75   1558   0.875
>   8  0.0247   6.36   0.885597   0.9566.70   1755   0.670
>   9  0.0280   5.98   0.939693   0.9477.16   2003   0.716
>  10  0.0312   5.66   0.807697   0.9354.97   2042   0.497
>  11  0.0345   5.38   0.815757   0.9215.53   2195   0.553
>  12  0.0378   5.15   0.910878   0.9046.08   2547   0.608
>  13  0.0410   4.94   0.913890   0.8847.49   2582   0.749
>  14  0.0443   4.75   0.906934   0.8617.42   2710   0.742
>  15  0.0476   4.58   0.824976   0.8345.06   2695   0.506
>  16  0.0508   4.44   0.812   1010   0.8025.32   2859   0.532
>  17  0.0541   4.30   0.843   1099   0.7675.25   3126   0.525
>  18  0.0574   4.17   0.862   1030   0.7274.49   2790   0.449
>  19  0.0606   4.06   0.752   1115   0.6833.04   2927   0.304
>  20  0.0639   3.96   0.675439   0.6362.53   1117   0.253
>  21  0.0672   3.86   0.139176   0.5861.64393   0.164
>  22  0.0704   3.77   0.734   1082   0.5342.65   2641   0.265
>  23  0.0737   3.68- -----   
>  24  0.0770   3.60   0.489681   0.4292.20   1633   0.220
>  25  0.0802   3.53   0.358   1430   0.3781.85   3248   0.185
>  26  0.0835   3.46  -0.428 12   0.3301.38 22   0.138
>  27  0.0868   3.39   0.390663   0.2851.73   1404   0.173
>  28  0.0900   3.33   0.128   1443   0.2441.39   2946   0.139
>  29  0.0933   3.27   0.149   1372   0.2071.37   2835   0.137
>  30  0.0966   3.22   0.183   1578   0.1751.40   3202   0.140
> 
> Thanks ahead.

Re: [ccp4bb] Effects of Multiplicity and Fine Phi with Equivalent Count Numbers

[Phil Evans]

It’s best to think of slicing as just sampling the 3D reciprocal space. Then in 
the absence of errors fine slicing will improve signal/noise by reducing the 
excess background under the peaks. However [Mueller M, Wang M, Schulze-Briese 
C. Acta Cryst (2012) D68, 42-56] show that there are diminishing returns from 
slicing the spot width into more than 4 or 5 slices, and with finer slicing 
handling more data may be a nuisance. Shutterless data collection with modern 
pixel array detectors introduces very little noise / image, so this argument 
holds.

With older systems, there is a compromise with

1. shutter jitter, depending on the speed of data collection compared to the 
shutter speed
2. goniometer accuracy, to handle the stop/start synchronised with the shutter
3. read-out noise of the detector
4. reducing the background under the spots

1-3 favour thicker slices, 4 favours thinner

Judging the balance with shuttered collection is complicated, but the advantage 
of shutterless data collection with fast detectors is clear

Phil

> On 1 Dec 2016, at 04:42, Edward A. Berry  wrote:
> 
> On 11/30/2016 10:16 PM, Keller, Jacob wrote:
>>> If you fine slice and everything is then a partial, isn't that *more* 
>>> sensitive to lack of synchronization between the shutter and rotation axis 
>>> than the wide-frame method where there's a larger proportion of fulls that 
>>> don't approach the frame edges (in rotation space) ?  Especially if you're 
>>> 3D profile fitting ?
>> 
>> That is how the argument seems to go in Pflugrath 1999, but I would think 
>> that shutter jitter is a random error, so it would seem better to have 
>> several of these random errors for a given spot than just one. Perhaps 
>> measuring with high multiplicity would have the same averaging effect.
>> 
>>> Is fine slicing more or less beneficial at high resolutions relative to 
>>> lower ones ?
>> 
>> In terms of I/sigI, it seems to be the same proportional improvement across 
>> all resolutions. See Fig 4 of the Pflugrath 1999 paper.
>> 
>> JPK
> 
> I think the problem there is that, if the shutter jitter is random with a 
> constant sigma, it becomes a larger percent of the total exposure for that 
> frame. It would be like taking a 1ml pipetor with an error of 2% of full 
> scale, i.e. 20 ul. Because you want to average this out, you set it to 200 ul 
> and pipet 5 times. The sigma of that measurement would be sqrt(5) * 20 ul, I 
> think, so worse than doing it all in one shot. On the other hand if you take 
> a 200 ul pipet with sigma 2% of full scale or 4 ul, and take 5 times, the 
> error is sqrt(5) * 4 ul which is less than 20 ul.
> Of course this would not apply to reflections that are fully recorded on one 
> frame since they are not reflecting while the shutter is open/closing. Then 
> it would be only variation in background.
> 
>> 
>> Phil Jeffrey
>> 
>> Princeton
>> 
>> --
>> 
>> *From:*CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Keller, 
>> Jacob [kell...@janelia.hhmi.org]
>> *Sent:* Wednesday, November 30, 2016 5:44 PM
>> *To:* CCP4BB@JISCMAIL.AC.UK 
>> *Subject:* Re: [ccp4bb] Effects of Multiplicity and Fine Phi with Equivalent 
>> Count Numbers
>> 
>> If the mosaicity is, say, 0.5 deg, and one is measuring 1 deg frames, about 
>> half the time is spent measuring non-spot background noise under spots in 
>> phi, which is all lumped into the intensity measurement. Fine slicing 
>> reduces this. But I am conjecturing that there is also fine-slicing-mediated 
>> improvement due to averaging out things like shutter jitter, which would 
>> also be averaged out through plain ol’ multiplicity.
>> 
>> I guess a third equal-count dataset would be useful as well: one sweep with 
>> six-fold finer slicing. So it would be:
>> 
>> One sweep, 0.6 deg, 60s
>> 
>> Six sweeps, 0.6 deg, 10s
>> 
>> One sweep, 0.1 deg, 10s
>> 
>> Or something roughly similar. Who will arrange the bets?
>> 
>> JPK
>> 
>> 

Re: [ccp4bb] Issues with Coot & XQuartz

[Phil Evans]

I can run coot locally (though not remotely) on Quartz 2.7.11 (OS X Yosemite 
10.10.5)
Phil

> On 22 Nov 2016, at 12:39, Martin Montgomery <m...@mrc-mbu.cam.ac.uk> wrote:
> 
> Hi,
> 
> We have had these problems with coot and xquartz versions greater than 2.7.8. 
>  This affects coot running locally on OSX and via ssh -XY from the mac to our 
> linux workstations.  You should still be able to download the xquartz-2.7.8 
> package (https://www.xquartz.org/releases/XQuartz-2.7.8.html).  Rebooting 
> after the installation of a later xquartz has not made any difference on our 
> macs.
> 
> Regards
> 
> MGM
> 
> 
> 
> 
>> On 22 Nov 2016, at 12:32, Phil Evans <p...@mrc-lmb.cam.ac.uk> wrote:
>> 
>> Something we [re]discovered at a recent workshop is that after installing 
>> Quartz you need to reboot (I believe to start the X11 launch daemon). Could 
>> this be the problem?
>> 
>> Phil
>> 
>>> On 22 Nov 2016, at 11:28, Laura Croenen <laura.croe...@durham.ac.uk> wrote:
>>> 
>>> Hello all,
>>> 
>>> I recently downloaded the CCP4 package on Mac (OS X Yosemite 10.10.2). With 
>>> this I am using the latest update of XQuartz. Some aspects of the package 
>>> (ccp4i2, QtMG) are working fine, but others, inc. Coot, are not working at 
>>> all. When I try to open Coot, it appears to start opening (the icon appears 
>>> at the bottom of my screen) and then disappears, with the message:
>>> 
>>> "student-10-245-174-102:MacOS lauracroenen$ ./coot
>>> 2016-11-21 17:07:40.946 coot[39004:3369025] script to run 
>>> /Applications/ccp4-7.0/coot.app/../bin/coot
>>> 
>>> (coot-bin:39010): Gtk-WARNING **: cannot open display: 
>>> 
>>> (coot-crash-catcher.scm:39011): Gtk-WARNING **: cannot open display:"
>>> 
>>> Has anyone else had this problem? Am I missing something?
>>> 
>>> Thank you in advance.
>>> 
>>> Laura Croenen
> 

Re: [ccp4bb] Issues with Coot & XQuartz

[Phil Evans]

Something we [re]discovered at a recent workshop is that after installing 
Quartz you need to reboot (I believe to start the X11 launch daemon). Could 
this be the problem?

Phil

> On 22 Nov 2016, at 11:28, Laura Croenen  wrote:
> 
> Hello all,
> 
> I recently downloaded the CCP4 package on Mac (OS X Yosemite 10.10.2). With 
> this I am using the latest update of XQuartz. Some aspects of the package 
> (ccp4i2, QtMG) are working fine, but others, inc. Coot, are not working at 
> all. When I try to open Coot, it appears to start opening (the icon appears 
> at the bottom of my screen) and then disappears, with the message:
> 
> "student-10-245-174-102:MacOS lauracroenen$ ./coot
> 2016-11-21 17:07:40.946 coot[39004:3369025] script to run 
> /Applications/ccp4-7.0/coot.app/../bin/coot
> 
> (coot-bin:39010): Gtk-WARNING **: cannot open display: 
> 
> (coot-crash-catcher.scm:39011): Gtk-WARNING **: cannot open display:"
> 
> Has anyone else had this problem? Am I missing something?
> 
> Thank you in advance.
> 
> Laura Croenen

Re: [ccp4bb] XDS questions

[Phil Evans]

XDS-CORRECT and Aimless use 

1. different scaling models - CORRECT includes a poorly documented correction 
across the detector plane not present in Aimless: this may or may not be a Good 
Thing

2. different outlier rejection algorithms - XDS seems to reject more 
observations

3. different “correction” of the sigma(I) estimates - XDS seems to do a better 
job at this

In practice, the differences are likely to be marginal, and it is hard to 
decide which is better

Phil

> On 21 Nov 2016, at 11:13, Tim Gruene  wrote:
> 
> Dear Nishant,
> 
> XDS_ASCII.HKL contains corrected, scaled, but not merged reflections. 
> You can specifically ask XDS to merge your data, but I would not do so unless 
> really necessary - you loose a lot of information.
> 
> I would like to offer a different opinion to Graeme's:
> You can read XDS_ASCII.HKL into pointless and aimless and provide aimless 
> with 
> the option 'onlymerge'. This way aimless merges the data, but it does not 
> rescale them.
> 
> XDS performs a couple of corrections in the CORRECT step, the output of which 
> is XDS_ASCII.HKL. And while XDS is extremely well documented, I am not sure 
> aimless takes into account how XDS treats the data. I would therefore trust 
> the step of scaling to the same author and continue with XDS_ASCII.HKL.
> 
> Best,
> Tim
> 
> 
> On Monday, November 21, 2016 11:37:15 AM Nishant Varshney wrote:
>> Dear All,
>> 
>> Just to understand more, the XDS_ASCII.HKL file generated after running XDS
>> contains scaled and merged reflections?
>> 
>> Moreover, what happens exactly, if you use XDS_ASCII.HKL file in AIMLESS
>> instead of INTEGRATE.HKL file??
>> 
>> I ran AIMLESS separately, one using already scaled XDS_ASCII.HKL and
>> another using INTEGRATE.HKL and I found that in the run using XDS_ASCII.HKL
>> little lesser total number of observation but marginally better statistics.
>> 
>> Thanks
>> Nishant
>> 
>> On Thu, Nov 17, 2016 at 10:08 PM, Andreas Forster 
>> 
>> wrote:
>>> Dear Wei,
>>> 
>>> if you process your data with XDS, the best is probably to do the scaling
>>> in XDS (CORRECT) and be done with it.  If you want to use Aimless for
>>> merging, you can turn off scaling with the ONLYMERGE keyword or use SCALES
>>> CONSTANT.
>>> 
>>> All best.
>>> 
>>> 
>>> Andreas
>>> 
>>> On Thu, Nov 17, 2016 at 9:40 PM, Wei Wang  wrote:
 Hi,
 
 Is there a way to let xds_par use less than all processors/threads on the
 machine? Sometimes I would like to process something else while XDS is
 running.
 
 Another question is related to the scaling procedure. My understanding is
 that the XDS already does the scaling during correction. So if I follow
 the
 XDS-Aimless route, then probably I should let Aimless do "skip scaling
 and
 only merge"? Please elucidate me on this issue.
 
 Regards,
 
 Wei
>> 
>> --
>> Dr. Nishant Kumar Varshney,
>> IISc-ICTP Fellow
>> XRD2 Beamline, Elettra-Sincrotrone,
>> In Area Science Park,
>> Basovizza, S.S. 14, Km 163,5,
>> 34012 Trieste, Italy
>> +39-040-375 8737 (office ESP4 P1 031)
>> +39-040-375-8435 (XRD2 beamline)
>> +39 3318809798 (Mobile)
> -- 
> --
> Paul Scherrer Institut
> Dr. Tim Gruene
> - persoenlich -
> Principal Investigator
> Biology and Chemistry
> OFLC/102
> CH-5232 Villigen PSI
> 
> Phone: +41 (0)56 310 5297
> 
> GPG Key ID = A46BEE1A
> 

Re: [ccp4bb] [ccp4bb] Fwd: [ccp4bb] just out of totally idle curiosity ...

[Phil Evans]

From letters in yesterday’s Guardian:-

today is 9/11

Make America grate again



> On 10 Nov 2016, at 08:00, Marjolein Thunnissen 
>  wrote:
> 
> Dear Bill,
> 
> I fully agree with you, awareness has to be spread and one should not ignore 
> politics completely, especially when there are so strong anti-intellectual 
> (anti-science) statements out there.
> 
> best regards
> 
> Marjolein
> 
>> On 10 Nov 2016, at 04:17, William G. Scott  wrote:
>> 
>> Dear Edward et al:
>> 
>> I agree we shouldn’t engage in partisan arguments on the CCP4bb.  However, I 
>> think it is a mistake, and perhaps a missed opportunity, to ignore politics 
>> completely.
>> 
>> For example, Newt Gingrich is currently in the running for Sec HHS.  He has 
>> previously written editorials in the NYT and Wall Street Journal advocating 
>> doubling the budget of the NIH.  
>> 
>> I think it is incumbent upon us to make our voices heard if such an 
>> opportunity arises, regardless of what one may happen to think about the 
>> individual’s political orientation, as it could potentially be of enormous 
>> benefit to the scientific community.
>> 
>> Yours faithfully,
>> 
>> William G. Scott
>> Director, Program in Biochemistry and Molecular Biology
>> Professor, Department of Chemistry and Biochemistry
>> and The Center for the Molecular Biology of RNA
>> University of California at Santa Cruz
>> Santa Cruz, California 95064
>> USA
>> 
>> http://scottlab.ucsc.edu
>> 
>>> On Nov 9, 2016, at 9:02 AM, Edward Snell  wrote:
>>> 
>>> As a Brexit and Trumpet affected person having a foot in both countries 
>>> ,this topic is too far off the normal discussion on CCP4 and probably 
>>> better taken up privately.  CCP4 is not a political discussion site. With 
>>> CCP4 the signal is unusually high and the noise low when compared to any 
>>> discussion board. I for one would like to keep it there. Political views 
>>> aside, we’re all trying to achieve the same scientific goals. Let’s 
>>> remember that and keep that the focus.
>>> 
>>> Edward Snell Ph.D.
>>> President and CEO Hauptman-Woodward Medical Research Institute
>>> Assistant Prof. Department of Structural Biology, University at Buffalo
>>> 700 Ellicott Street, Buffalo, NY 14203-1102
>>> Phone: (716) 898 8631 Fax: (716) 898 8660
>>> Skype:  eddie.snell Email: esn...@hwi.buffalo.edu 
>>> 
>>> Heisenberg was probably here!
> 
> 
> 
> 
> 
> 
> 
> 
> Dr. Marjolein Thunnissen
> Head User Office
> 
> MAX IV Laboratory
> Lund University
> P.O. Box 118, SE-221 00 Lund, Sweden
> Visiting address: Fotongatan 2, 225 94 Lund
> Telephone:+46 46 2224668
> Mobile:  +46 766 32 04 17
> www.maxliv.lu.se
> 
> 

Re: [ccp4bb] C2, I2, completeness, and lattice translocation defects

[Phil Evans]

An MTZ file contains the space group name, space group number, and all the
symmetry operators. Space group numbers such as 4005 are an unofficial
undocumented extension to International Tables rules and shouldn't be used
as reliable information. The name or the operators should take precedence

Does Refmac really rely on the space group number, or it in some
script/interface?

Phil

> Ian,
>
> I think I found the issue just by looking through mtzdmp output, but
> there was a clue from the response I got yesterday:
>
> Hi Paul,
>
> I have come across this problem before, suddenly what was complete
> data is only 50% complete. I seem to recall its because I2 is also
> called space group 5 (same as C2), which confuses some programs.
>
> I tracked spacegroup number in the mtz headers. Pointless, aimless,
> ctruncate use 4005 for I2. After Phaser (v 2.5.7), the spacegroup number
> is just 5. That seemed to be the thing that is bothering refmac and
> phenix.refine. For whatever reason I was using the phaser output for the
> initial refinement.
>
> --paul
>
>
> On 11/04/2016 10:58 AM, Ian Tickle wrote:
>>
>> Paul, mtzdump may not give the full header.  The best way to get this
>> is to use a text editor on the MTZ file (yes I know it looks like
>> garbage!), scroll to the end where you will find the header starting
>> at 'VERS MTZ:V1.1'.  Then copy/paste everything from there to the end
>> (don't worry about formatting it).
>>
>> Hopefully this will give a clue.
>>
>> Cheers
>>
>> -- Ian
>>
>>
>> On 4 November 2016 at 14:49, Ian Tickle > > wrote:
>>
>>
>> Hi Paul
>>
>> I just tried Refmac 5.8.0135 (which must be very similar to the
>> version you are using) with an I2 dataset and I don't see this
>> "conversion to C2".  I doubt very much that the refinement
>> programs need to convert to C2: I'm pretty sure they can do the
>> refinement perfectly well in I2.
>>
>> I think it's much more likely that your MTZ header has somehow got
>> corrupted with inconsistent space group info so you need to track
>> back in the history list in the MTZ header and see which program
>> was responsible for the corruption.  Can you post the MTZ header
>> so we can see the history list and the cell/space group info?
>>
>> Cheers
>>
>> -- Ian
>>
>>
>> On 4 November 2016 at 14:39, Paul Paukstelis
>> > wrote:
>>
>> Refmac and phenix.refine versions I used both seem to be
>> problematic. Both are I2 in and C2 out.
>>
>> --p
>>
>> On 11/04/2016 08:25 AM, Ian Tickle wrote:
>>>
>>> Hi Paul
>>>
>>> This sounds like there might be a recently-introduced bug
>>> which should be reported to the author.  I have several
>>> structures in I2 & I haven't noticed anything like this. Can
>>> you tell which program is introducing this error, e.g. by
>>> looking at the mtzdump outputs?
>>>
>>> Cheers
>>>
>>> -- Ian
>>>
>>>
>>> On 4 November 2016 at 12:00, Paul Paukstelis
>>> >
>>> wrote:
>>>
>>> Thanks to all that responded. I sorted this out.
>>>
>>> It all appears to stem from the C2->I2 conversion.
>>> Forcing everything in processing to stick with C2 fixes
>>> all the issues!
>>>
>>>
>>> Thanks again,
>>>
>>> --paul
>>>
>>>
>>>
>>> On 11/03/2016 12:39 PM, Paul Paukstelis wrote:
>>>
>>> CCP4BB,
>>>
>>> I posted some time back about a DNA oligonucleotide
>>> structure we were working on. I had difficulty
>>> phasing it despite strong signal from bromines, but
>>> finally managed to get reasonable enough maps from a
>>> few datasets to build, only to find that despite the
>>> density looking quite good, it simply wouldn't refine
>>> past R/Rfree of around 28/32. With help from ccp4bb
>>> it began to become clear that this might be a
>>> candidate for a lattice with translocation defects.
>>>
>>> I had my student make a variant in which two 3'
>>> nucleotides that weren't involved in base pairing
>>> contacts were removed. This crystallized under the
>>> same conditions in a different space group and was
>>> now diffracting to ~1.0 A (from about 2.2 in the
>>> previous space group). Images overall looked good,
>>> though we collected on some crystals that clearly had
>>> more than one lattice that made indexing more
>>> difficult. The best looking data still had some tails
>>> on spots, but was easily 

Re: [ccp4bb] New version SHELXL 2016/6 and PDB2INS

[Phil Evans]

Hi George

I’m not sure that anyone here is using shelxl but I’ve just updated our general 
64-bit Linux versions anyway. shell starts OK, but with pdb2ins I get a 
complaint. Do you understand this? (I should probably ask our local guys)

Our system is
Scientific Linux release 6.7 (Carbon)

 ./pdb2ins 
./pdb2ins: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by 
/tmp/_MEIQO85tj/libz.so.1)

best wishes
Phil


> On 25 Oct 2016, at 20:31, George Sheldrick  
> wrote:
> 
> SHELXL may be used to refine both small and macromolecular structures against 
> X-ray or neutron diffraction data, including non-merohedral twins. A new 
> version 2016/6 of SHELXL may now be downloaded from the SHELX server. It has 
> been well tested by about 20 volunteers to whom I am very grateful. A new 
> version of Anna Luebben's program PDB2INS is also available there (for 64 bit 
> systems only). PDB2INS makes the preparation of the .ins and .hkl files to 
> run macromolecular SHELXL refinements much easier. For most structures 
> deposited in the PDB since 2008 these two files can be generated 
> automatically and no changes are needed to run SHELXL.
> 
> George
> -- 
> Prof. George M. Sheldrick FRS
> Dept. Structural Chemistry, 
> University of Goettingen,
> Tammannstr. 4,
> D37077 Goettingen, Germany
> Tel. +49-551-39-33021 or -33068
> Fax. +49-551-39-22582
> 
> 

Re: [ccp4bb] Reindexing lattice

[Phil Evans]

reindex k, l, h

you can use Pointless or Reindex

Phil

On 1 Jul 2015, at 16:04, D Bonsor dbon...@ihv.umaryland.edu wrote:

 Hi,
 
 I am trying to reindex a P212121 lattice to change the axis order from 32.62, 
 64.89, 103.16 to 64.89, 103.16, 32.62. Looking at the REINDEXING manual I 
 can't find how to reindex the lattice. Does anyone know how I should do this 
 and an example of the execute script that I should use? 
 
 Thanks 
 
 
 Dan

Re: [ccp4bb] Last 2 updates - aimless and scala

[Phil Evans]

It's hard to know without more information. If there really is no log file, 
then that suggests the job hasn't even started, so maybe something wrong with 
the ccp4i interface

Which task were you using? 

1. Symmetry, Scale, Merge (Aimless), the recommended one, or
2. Find Symmetry, Scale and Merge (Scala), the old one

Are you sure there is no log file? Do the programs start from the command line, 
e.g. type pointless?

For what it's worth, it's all working for me

Phil

On 24 Jun 2015, at 11:27, Joel Tyndall joel.tynd...@otago.ac.nz wrote:

 Hi folkd,
  
 We have just collected some data and was busy processing and ran aimless 
 having updated to the latest versions (upgrades 10 and 11 I think). Both 
 aimless and then scala stops/fails almost immediately without writing any log 
 files etc. I tried running the same unmerged mtz on an older version and it 
 runs fine. Is this a bug?
  
 J
  
 _
 Joel Tyndall, PhD
 
 Associate Professor in Medicinal Chemistry
 National School of Pharmacy
 University of Otago
 PO Box 56 Dunedin 9054
 New Zealand  
 Skype: jtyndall
  
 Ph: +64 3 479 7293
  

Re: [ccp4bb] Refinement at 4A resolution

[Phil Evans]

The Xtriage  Pointless logs don't show definitively that the space group is 
C2221 as they have been run on the merged data. You may need to check them with 
the unmerged data, and perhaps run molecular replacement in a lower symmetry 
such as C2

Phil

On 22 Apr 2015, at 22:28, Appu kumar appu.kum...@gmail.com wrote:

 Dear CCP4 Member,
 I seek your advice on the refinement issues at the low resolution 4A. I am 
 trying to refine a membrane protein structure after getting the phases from 
 MR using the PHASER. The soluble domain structure which comprises of 40% of 
 protein has been used as template (sequence identity 80%) in MR search . The 
 PHASER gave  a good solution having TFZ value of about 14.3. I have then 
 created the polyA model for the transmembrane domain from distant homolog 
 which share 30% sequence identity for TM region and try to find the phases 
 for whole TM domain keeping the soluble domain fixed. I got lucky in getting 
 the phases for the whole protein using the PHASER (TFZ=17.6) but the during 
 the refinement, Rwork and Rfree got stalled at the 41 and 44 respectively 
 after several cycle of the refinement in both refmac and phenix. I checked 
 the spacegroup with pointless and it suggests C2221. I have attached the 
 pointless and phenix.xtriage run file with this mail for your evaluation. 
 Phenix.xtriage suggests no major pathologies with the data except the mild 
 psuedomerohedral twining. There are two molecules of protein in ASU. 
 Evaluation of the density maps, suggest reasonable map for the most of 
 protein part. I am wondering why Rwork and Rfree are not coming down despite 
 of the good MR solution and what i am doing wrong with refinement and if 
 there is some pathologies associated with the data which needs to be answered 
 before heading to refinement.  
 
 Thanks for your help in advance. 
 
 Appu
 
 93_pointless.logxtriage_50.log

Re: [ccp4bb] Sortwater NCS Matrix input

[Phil Evans]

the test limit is perhaps over-tight

data eps/0.0001/

Phil


On 2 Apr 2015, at 00:34, Dale Tronrud de...@daletronrud.com wrote:

   I think you are on the right track - There are not enough decimal
 points in your matrix elements to pass the orthonormal test.  This test
 checks that the length of each row (x^2+y^2+z^2) is equal to one and the
 dot product of each row with every other row is equal to zero.  If the
 values on your NCS statement are in row order I calculate 0.999337 for
 the length of the first row.  If the program is testing if this is equal
 to one to four decimal points you lose.
 
   You have to add more digits, but just adding zeros isn't going to
 accomplish much.  The best solution is to get your ncs program to report
 its matrix with more digits -- three is pitiful.  Failing that you could
 calculate one element of each row from the other two to ensure the
 length is equal to one at a higher level of precision and hope this
 doesn't mess up the dot product test.  You'll end up with one number in
 each row having more than three decimal places.
 
 Dale Tronrud
 
 On 4/1/2015 2:52 PM, Shane Caldwell wrote:
 Hi ccp4bb,
 
 I'm trying to solve a problem I never quite figured out in the past. I'd
 like to use the *sortwater* utility to send my picked waters to various
 protein chains, and to give them the same residue number if they are
 NCS-equivalent, as the manual outlines.
 
 http://www.ccp4.ac.uk/html/sortwater.html
 
 The first part goes off perfectly, partitioning the waters into their
 respective chains. Where I run into problems is bringing in NCS. I can't
 get the program to recognize the transformation matrix. I can calculate
 the matrix using *superpose*, and manually input these (limited
 precision) values into my script, which looks like:
 
 NCS B C MATRIX 0.072 0.997 -0.012 0.991 -0.073 -0.113 -0.113 -0.004
 -0.994 37.502 -35.283 81.276
 
 and it returns
 
 WARNING:   Matrix is not orthonormal 
 
 
 My linear algebra is very limited, and I don't know exactly what this
 means in the context of this program, though I suspect it could be
 either linked to converting to fractional coordinates (I'm in a
 monoclinic system), or a product of the limited precision of the matrix
 values.
 
 Using the identity matrix, like so:
 
 NCS B C MATRIX 1.0 0.0 0.0 0.0 1.0 0.0 0.0
 0.0 1.0 0.000 0.000 0.000
 
 doesn't trigger the warning. These values have more digits, but adding
 extra zeroes to the original matrix as a very crude workaround still
 returns the error.
 
 So, I'm now stuck trying to parse what's going on. I know *sortwater*
 also takes O datablocks as matrix input, and that's something I could
 look into (especially if calculating in a different program might get me
 better precision). Although, I'm not sure the format is a factor given
 the identity matrix is accepted as a keyword input.
 
 Skimming the archives, I get the sense this isn't something that many
 users do any more. I have quite a few structures with hundreds of waters
 each and I'd like to get the waters organized, but doing it by hand will
 take a very long time. Any help getting this program running would be
 greatly appreciated!
 
 
 Shane Caldwell
 McGill University

Re: [ccp4bb] Sortwater NCS Matrix input

[Phil Evans]

I should maybe add some code to orthonormalise the matrix if it's not too far 
off (which would be approximate). I haven't looked at this program for a long 
time (nor indeed used it), so don't hold your breath

Phil

On 1 Apr 2015, at 22:52, Shane Caldwell shane.caldwel...@gmail.com wrote:

 Hi ccp4bb,
 
 I'm trying to solve a problem I never quite figured out in the past. I'd like 
 to use the sortwater utility to send my picked waters to various protein 
 chains, and to give them the same residue number if they are NCS-equivalent, 
 as the manual outlines.
 
 http://www.ccp4.ac.uk/html/sortwater.html
 
 The first part goes off perfectly, partitioning the waters into their 
 respective chains. Where I run into problems is bringing in NCS. I can't get 
 the program to recognize the transformation matrix. I can calculate the 
 matrix using superpose, and manually input these (limited precision) values 
 into my script, which looks like:
 
 NCS B C MATRIX 0.072 0.997 -0.012 0.991 -0.073 -0.113 -0.113 -0.004 -0.994 
 37.502 -35.283 81.276
 
 and it returns 
 
  WARNING:   Matrix is not orthonormal 
 
 
 My linear algebra is very limited, and I don't know exactly what this means 
 in the context of this program, though I suspect it could be either linked to 
 converting to fractional coordinates (I'm in a monoclinic system), or a 
 product of the limited precision of the matrix values. 
 
 Using the identity matrix, like so: 
 
 NCS B C MATRIX 1.0 0.0 0.0 0.0 1.0 0.0 0.0 
 0.0 1.0 0.000 0.000 0.000
 
 doesn't trigger the warning. These values have more digits, but adding extra 
 zeroes to the original matrix as a very crude workaround still returns the 
 error.
 
 So, I'm now stuck trying to parse what's going on. I know sortwater also 
 takes O datablocks as matrix input, and that's something I could look into 
 (especially if calculating in a different program might get me better 
 precision). Although, I'm not sure the format is a factor given the identity 
 matrix is accepted as a keyword input.
 
 Skimming the archives, I get the sense this isn't something that many users 
 do any more. I have quite a few structures with hundreds of waters each and 
 I'd like to get the waters organized, but doing it by hand will take a very 
 long time. Any help getting this program running would be greatly appreciated!
 
 
 Shane Caldwell
 McGill University

Re: [ccp4bb] Absence of contact between layers in a crystal

[Phil Evans]

why are these structures still in the PDB?

On 6 Feb 2015, at 11:08, David Briggs drdavidcbri...@gmail.com wrote:

 Haven'tthat paper and the associated structure been retracted?
 
 http://www.nature.com/news/2009/091222/full/462970a.html
 
 There was a huge scandal when it was discovered that Krishna Murthy had 
 falsified data, including the structure you refer to. 
 
 See http://en.wikipedia.org/wiki/H.M._Krishna_Murthy
 
 A crystallographer with a wikipedia entry for all the wrong reasons...
 
 Dave
 
 
 On Fri Feb 06 2015 at 11:02:12 AM Kerff Fred fke...@ulg.ac.be wrote:
 Hello,
 
 Looking at structure 2HR0 (The structure of complement C3b provides insights 
 into complement activation and regulation. »,Abdul Ajees, A.,  Gunasekaran, 
 K.,  Volanakis, J.E.,  Narayana, S.V.,  Kotwal, G.J.,  Krishna Murthy, H.M.;  
 (2006) Nature 444: 221-225), I noticed the absence of contacts between layers 
 in the crystal. Is it something that has already been observed in other 
 crystals?
 
 Best regards,
 
 Fred
 -
 Frédéric Kerff
 Chercheur qualifié F.R.S.-FNRS
 Cristallographie des protéines
 Centre d'Ingénierie des Protéines
 Université de Liège
 17, Allée du 6 Août - Bat B5a
 4000 Liège (Belgium)
 Tel.: +32 (0)4 3663620
 Fax: +32 (0)4 3663772
 
 
 
  Le 6 févr. 2015 à 10:12, Tim Gruene t...@shelx.uni-ac.gwdg.de a écrit :
 
  -BEGIN PGP SIGNED MESSAGE-
  Hash: SHA1
 
  Dear Smith,
 
  The sca file most likely does not contain flags. pointless can read
  the sca file, standardise it to ccp4 standards and freerflag marks
  random reflections. You should use the maximum of 500 unique
  reflections or 5% of the unique reflections, whichever is larger.
 
  Best,
  Tim
 
  On 02/06/2015 09:49 AM, Smith Lee wrote:
  Dear All, I have a sca file. Will you please tell me by which
  software or how I can know whether the sca file contains R-free
  tags? If not, by which software or how I can add the R-free tags?
  And how much of the reflections I add the R-free tags? I am looking
  forward to getting your reply. Smith
 
 
  - --
  - --
  Dr Tim Gruene
  Institut fuer anorganische Chemie
  Tammannstr. 4
  D-37077 Goettingen
 
  GPG Key ID = A46BEE1A
 
  -BEGIN PGP SIGNATURE-
  Version: GnuPG v1.4.12 (GNU/Linux)
 
  iD8DBQFU1IWVUxlJ7aRr7hoRAmZHAJ4+6wREnwkFN0EhfErAA0tPSopKKwCgiLdi
  j0JFZac4kAh8twpov71MG84=
  =XN57
  -END PGP SIGNATURE-

Re: [ccp4bb] phase shift vs non-isomorphism

[Phil Evans]

On 27 Jan 2015, at 20:25, James Holton jmhol...@lbl.gov wrote:
… snip

 Now you can ask questions about the phase shift.  The problem with phases is 
 that the phase of weak reflections doesn't really matter because they don't 
 contribute to the map.  Also, poorly-measured phases get low FOM weights in 
 map calculations as well.  For this reason, the phase shift in rms degrees is 
 not really all that useful.  Better to consider the correlation coefficient 
 of the two electron density maps.  In the case of 3aw6 vs 3aw7 you get CC= 
 0.7.  Not too bad really.

Gerard Bricogne pointed out a long time ago that the clearest comparison 
between two sets of phases is the complex correlation coefficient between two 
best structure factors ( m F exp(i phi) ) - this is equivalent to map 
correlation, but can be analysed in resolution bins.

I'm not sure whether there are programs which calculate this, though it's 
straightforward, and I've had various jiffy programs in the past (which I've 
probably lost)

Phil

Re: [ccp4bb] CCP4 Scalepack2mtz problem: Anisotropy correction failed

[Phil Evans]

It is hard to tell without having the unmerged data (don't post that to the 
BB), but running the data through Pointless, Aimless ctruncate (though I had to 
fix Aimless) suggests

1. the data are not very good
2. the space group may be C2
3. it may be twinned (or just some spot overlap with the long 288A c axis)

Phil

On 13 Jan 2015, at 00:34, Xiao Lei xiaolei...@gmail.com wrote:

 Hi Eleanor,
 
 Thank you for the advice.  The .sca file is here. The data may not be useful 
 as you said, but the anisotropic analysis for this data is confusing. From 
 the phenix xtriage output, it seems the data has no anisotropic issue (I may 
 interpret wrong the log file, I attach part of the xtriage log file here). 
 But from CCP4, I know the data has anisotropic problem (Negative 
 Eigenvalues). 
 
 Below is part of log file from Phenix.
 
 Anisotropy analyses 
 
 Anisotropy( [MaxAnisoB-MinAnisoB]/[MaxAnisoB] ) :  3.233e-01
   Anisotropic ratio p-value :  0.000e+00
 
  The p-value is a measure of the severity of anisotropy as observed in 
 the PDB.
  The p-value of 0.000e+00 indicates that roughly 100.0 % of datasets 
 available in the PDB have
  an anisotropy equal to or worse than this dataset.
 
 Xiao
 
 On Sun, Jan 11, 2015 at 7:51 AM, Eleanor Dodson eleanor.dod...@york.ac.uk 
 wrote:
 Well - I really need to see the output.sca file with the reflection list to 
 test the problem properly, but this log file does show your reflections at 
 the highest resolution are very weak, and maybe not useful. Try cutting the 
 resolute a bit and see what happens.
 Eleanor
 
 On 11 January 2015 at 00:08, Xiao Lei xiaolei...@gmail.com wrote:
 Hi Eleanor,
 
 Thanks for the input, I attach the scale log file here. 
 
 On Sat, Jan 10, 2015 at 2:16 PM, Eleanor Dodson eleanor.dod...@york.ac.uk 
 wrote:
 Certainly a negative eigen value is bad - can you attach the data? There may 
 be some obvious problem..
 Eleanor
 
 On 10 January 2015 at 04:42, Xiao Lei xiaolei...@gmail.com wrote:
 Dear All,
 
 I tried to convert my x-ray diffraction sca data from HKL200 (.sca file) to 
 mtz in CCP4 using Scalepack2mtz and it failed, I do not know what should I 
 supposed to do next to correct the problem, any suggestions are appreciated. 
 I pasted the message from part of log file below:
 
 
 ANISOTROPY ANALYSIS (using intensities):
 
 
 Eigenvalues: -0.5668 0.1479 0.3584
 
 Eigenvalue ratios: -1.5815 0.4128 1.
 
 ctruncate: Anisotropy correction failed - negative eigenvalue.
 
 Times: User: 0.4s System: 0.0s Elapsed: 0:00
 
 ***
 
 * Information from CCP4Interface script
 
 ***
 
 The program run with command: /Applications/ccp4-6.4.0/bin/ctruncate -hklin 
 /LAB/CCP4/TEMP/RelADD_12212014ALS502_3_1_mtz.tmp -hklout 
 /LAB/CCP4/TEMP/RelADD_12212014ALS502_3_3_mtz.tmp -colin 
 /*/*/\[IMEAN,SIGIMEAN\] -colout P1_12
 
 has failed with error message
 
 ctruncate: Anisotropy correction failed - negative eigenvalue.
 
 ObjectCache: Leaked 0005 refs to
 
 ***
 
 
 
 #CCP4I TERMINATION STATUS 0  ctruncate:  Anisotropy correction failed - 
 negative eigenvalue. ObjectCache: Leaked 0005 refs to 
 
 
 
 #CCP4I TERMINATION TIME 09 Jan 2015  19:55:50
 
 #CCP4I MESSAGE Task failed
 
 
 
 
 
 
 P1_12_P1_rej2.6_output.sca

Re: [ccp4bb] query regarding interchanging cell axis

[Phil Evans]

It looks as if you have a and b about the same length. In that case indexing of 
the patterns may randomly switch h and k. The CCP4 program Pointless (in ccp4i 
Data reduction - Symmetry, Scale, Merge (aimless) or Find or Match Laue 
group). Give your 1st dataset as the reference.

Phil

On 30 Dec 2014, at 06:04, Thyageshwar Chandran biotechnologi...@gmail.com 
wrote:

 Dear all,
 I have a query regarding the interchanging of axis !
 
 My native crystals are orthorhombic (P21212), and i did soaking with 
 different ligands, after solving the structures when i compared the unit 
 cell, the orientation of the molecule was different in three cases (90 degree 
 rotation), hence had to interchange a and b for them (changed it in SF h=k, 
 k=h) , my query is thought the conditions were similar in all the cases, how 
 come only in the said cases the change occurred! i wonder if i am doing the 
 correct thing by interchanging the axisplease advice and let me know if 
 you had come across similar such cases
 
 i had attached the figure for clarity ( i am talking about the last three 
 cases)
 
 many thanks
 
 regards,
 thii
 
 On 28 December 2014 at 12:18, Thii biotechnologi...@gmail.com wrote:
 Dear all,
 I have a query regarding the interchanging of axis !
 
 My native crystals are orthorhombic (P21212), and i did soaking with 
 different ligands, after solving the structures when i compared the unit 
 cell, the orientation of the molecule was different in three cases (90 degree 
 rotation), hence had to interchange a and b for them (changed it in SF h=k, 
 k=h) , my query is thought the conditions were similar in all the cases, how 
 come only in the said cases the change occurred! i wonder if i am doing the 
 correct thing by interchanging the axisplease advice and let me know if 
 you had come across similar such cases
 
 
 many thanks
 
 regards,
 thii
 
 
 
 
 -- 
 When we long for life without difficulties, remind us that oaks grow strong 
 in contrary winds and diamonds are made under pressure.
 combined-all-picture.png

Re: [ccp4bb] query regarding interchanging cell axis

[Phil Evans]

OK they looked similar in your pictures

Maybe you have two different crystal forms

Phil

On 30 Dec 2014, at 16:56, Thii Chand biotechnologi...@gmail.com wrote:

 Dear Dr.Evans, 
 
 Thanks for your reply.
 
 The unit cell parameters are a=140 , b=110, c=44., Hence i am afraid, the 
 interchange couldn't have occurred due to similar a and b. I had performed 
 dehydration using PEG in the said case. I agree that dehydration or soaking 
 with ligand can hardly lead to the interchange of axis.But, what i find 
 difficult to understand is how come only in three out of nice 
 structures,interchange of cell axis occurs, where i had maintained similar 
 experimental conditions for all the nine cases.  Is there any logical 
 explanation for this or am i doing something wrong?
 
 The sp group are P21212 in all the cases.
 
 Thanks in advance 
 
 Regards, 
 T.chandran

Re: [ccp4bb] XDS orientation matrix inconsistent with Mosflm, DENZO

[Phil Evans]

(I think) these matrices are roughly 180 degrees apart, so they may just 
correspond to different signs of the axes
Phil

On 28 Dec 2014, at 21:37, Igor Petrik petr...@illinois.edu wrote:

 I am working on a small project which requires me to obtain the proper 
 orientation of a crystal lattice with respect to the gonistat and source. I 
 have until now successfully used the matrices from Mosflm and DENZO, which 
 are consistent with each other and define the orientation of the reciprocal 
 lattice in the lab space when the spindle is at 0deg.
 
 I am trying to use the orientation computed by XDS, but this seems to not be 
 consistent with the others. Here is an example:
 
 mosflm .mat file:
  -0.00458796 -0.01054146 -0.01052990
   0.00600652  0.01617284 -0.00696834
   0.02352343 -0.00618559 -0.00027442
0.000   0.000   0.000
   -0.1856881  -0.5200068  -0.8337343
0.2431015   0.7978011  -0.5517381
0.9520617  -0.3051332  -0.0217278
  39.6412 48.3160 77.5507 90. 90. 90.
0.000   0.000   0.000
 SYMM P222  
 
 (top part is reciprocal matrix in the format:
  a*x  b*x  c*x
  a*y  b*y  c*y
  a*z  b*z  c*z
 where x is the x-ray beam axis and z is the spindle axis)
 
 DENZO (HKL2000) gives an equivalent matrix.
 
 XDS orientation parameters:
 CORRECT.LP
 ...
  REFINED VALUES OF DIFFRACTION PARAMETERS DERIVED FROM  30955 INDEXED SPOTS
  REFINED PARAMETERS:   DISTANCE BEAM ORIENTATION CELL AXIS
  STANDARD DEVIATION OF SPOTPOSITION (PIXELS) 0.70
  STANDARD DEVIATION OF SPINDLE POSITION (DEGREES)0.08
  SPACE GROUP NUMBER 16
  UNIT CELL PARAMETERS 39.52848.15377.542  90.000  90.000  90.000
  E.S.D. OF CELL PARAMETERS  4.0E-02 3.3E-02 3.8E-02 0.0E+00 0.0E+00 0.0E+00
  REC. CELL PARAMETERS   0.025299  0.020767  0.012896  90.000  90.000  90.000
  COORDINATES OF UNIT CELL A-AXIS   -37.65011.269-4.236
  COORDINATES OF UNIT CELL B-AXIS14.62441.546   -19.461
  COORDINATES OF UNIT CELL C-AXIS-1.764   -32.374   -70.439
  CRYSTAL MOSAICITY (DEGREES) 0.211
  LAB COORDINATES OF ROTATION AXIS  0.62  0.007232 -0.004834
  DIRECT BEAM COORDINATES (REC. ANGSTROEM)   0.003134  0.005401  1.020962
  DETECTOR COORDINATES (PIXELS) OF DIRECT BEAM1230.87   1260.93
  DETECTOR ORIGIN (PIXELS) AT 1226.25   1252.96
  CRYSTAL TO DETECTOR DISTANCE (mm)   259.04
  LAB COORDINATES OF DETECTOR X-AXIS  1.00  0.00  0.00
  LAB COORDINATES OF DETECTOR Y-AXIS  0.00  1.00  0.00
 ...
 
 (XDS defines spindle as X and beam as Z)
 
 Converted to reciprocal lattice orientation matrix in mosflm axis conventions:
 (output from xds2mos; manual calculation is consistent with this output)
  -0.00273114 -0.00809777 -0.01150308
  -0.00724876 -0.01754758  0.00521075
  -0.02353677  0.00634416 -0.00027012
0.000   0.000   0.000
  -0.11022162 -0.39811300 -0.91068616
  -0.29254071 -0.86269689  0.41252918
  -0.94988163  0.31189970 -0.02138535
  39.5280 48.1530 77.5420 90. 90. 90.
0.000   0.000   0.000
 
 
 As you can see they are different. You can note that the component of each 
 vector along the mosflm-Z (spindle) axis is consistent, suggesting that it is 
 only the angle of rotation around the spindle axis that is inconsistent 
 between the two. I know that for mosflm and DENZO the orientation matrix 
 defines the orientation of the reciprocal lattice when the spindle is at 0 
 deg. XDS seems to be using a different reference point. Why is this and what 
 is the proper way to obtain the absolute reciprocal orientation at 0 deg from 
 XDS?
 
 (If anyone wants to test this on their own, I can provide the frames I used 
 to obtain these files.)
 
 Thanks,
 - Igor Petrik
 University of Illinois
 

Re: [ccp4bb] Correct usage of AIMLESS in combination with XDS/XSCALE

[Phil Evans]

You have several options:

1. scale together in XSCALE, use XDSCONV to export and avoid Aimless altogether
2. scale together in XSCALE, import into Pointless, then Aimless ONLYMERGE
3. take the two XDS_ASCII.HKL files into Pointless, then AIMLESS, either (a) 
SCALE CONSTANT to give a single scale for each dataset, or (b) rescale
4. take the two INTEGRATE.HKL files into Pointless and do all the scaling in 
Aimless

I have no idea which of these is best

Phil

On 10 Dec 2014, at 20:38, Renato Weisse rwei...@strubio.uni-kiel.de wrote:

 Hi CCP4BBers,
 
 I would like to ask a question about how to use AIMLESS correctly with
 scaled data from XDS. I collected two data sets of the same crystal, both
 of which scaled individually with CORRECT from XDS. Merging of reflexes
 should be done with AIMLESS. Also to obtain a more comprehensive data
 statistic.
 So I fed both data sets into AIMLESS with the option ONLYMERGE. But I
 think that both data sets were not scaled to each other (which indeed is
 no surprise with key ONLYMERGE).
 So, my question is the following: what is the right order to scale and
 merge multiple datasets with XDS/AIMLESS.
 The output from XSCALE is a file with suffix .ahkl. Is it the same format
 like .hkl and does it fit into AIMLESS?
 
 I really appreciate your valuable sugestions.
 
 Kind regards,
 Renato

Re: [ccp4bb] CCP4: scalepack2mtz problem

[Phil Evans]

Processing (integration and scaling) only depends on the point group, not the 
space group, so it is the same in P222 and P212121. Structure solution is 
different so you may need to try different space groups

Phil

On 9 Dec 2014, at 14:36, Uma Ratu rosiso2...@gmail.com wrote:

 Hi, Eleanor:
 
 Thank you for your suggestions.
 
 In HKL, I integrated the data set with P222, and scaled it with P222 and 
 P212121.
 
 From pointless, it says the data better fit with P212121.
 So I would like to process the data with P212121 as well.
 
  And you have NCS translation at 0,0,0.5 which makes it hard to be sure if 
  the SG is P21212 or P212121 
 That gives the (probably incorrect) suggestion that the data might be twinned
 
 This might be the reason that import failed with P212121.
 
 Regard
 
 Uma
 
 
 On Tue, Dec 9, 2014 at 6:31 AM, Eleanor Dodson eleanor.dod...@york.ac.uk 
 wrote:
 I suspect this is an external problem - there is no failure message in the  
 P212121 case..
 Obviously the input data is different for the 2 sets, do you know why? The 
 integration should be identical.
 
 And you have NCS translation at 0,0,0.5 which makes it hard to be sure if the 
 SG is P21212 or P212121 
 That gives the (probably incorrect) suggestion that the data might be twinned
 
 
 Not sure why there are differences in the log files and the tasks - 
 scale[pack2mtz and ctruncate should run more or less identically for these 
 two cases. I suggest checking your input, and trying again
 Eleanor
 
 
 
 
 
 
 
 On 8 December 2014 at 21:51, Uma Ratu rosiso2...@gmail.com wrote:
 Dear All:
 
 I try to import the .sca file from hkl2000.
 
 The data was scaled as P212121 from HKL2000.
 
 In ccp4, I used scalepack2mtz  from Scalepack(Denzo) into MTZ format, run 
 as Ctruncate.
 
 The import stopped at Twin fractione estimates exluding operators and 
 didn't go any further.
 
 I scaled the data as P222, and imported it using scalepack2mtz.
 It went through without problem.
 
 I have the log files from both imports attached.
 
 Could you advice why the P212121 can't go through?
 
 Thank you
 
 Uma
 
 
 
 

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

[Phil Evans]

Actually Pointless knows that the INTEGRATE file is corrected for an  
unpolarised beam and recorrects for a synchrotron unless the wavelength is one 
of the home source ones. See docs. You can specify explicitly I think
Phil

Sent from my iPhone

 On 17 Nov 2014, at 09:44, Graeme Winter graeme.win...@gmail.com wrote:
 
 Dear Nukri,
 
 The following is my opinion which I think is worth discussion, and are based 
 on my understanding of what XDS does in the CORRECT step.
 
 Firstly, I tend to find the global refinement in the CORRECT step useful for 
 getting a good unit cell  recycling the orientation matrix etc. for 
 reintegration. This is not related to scaling, but is useful, e.g.:
 
 http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Optimisation#Re-INTEGRATEing_with_the_correct_spacegroup.2C_refined_geometry_and_fine-slicing_of_profiles
 
 More relevant to the intensities: in integration the LP correction is 
 calculated assuming an unpolarized beam - if the data are from a synchrotron 
 these need to be corrected again for the correct polarization - something 
 which the correct step does (obviously given this on the command-line). 
 Pointless will also do this but assumes unless given a correct value that the 
 beam is quite polarized. Mostly: care needs to be taken, particularly if 
 using a wavelength which may be confused with a lab source...
 
 I also understand that the XDS CORRECT step applies a DQE correction for 
 Pilatus data, taking into account the geometry of the experiment, the sensor 
 thickness  photon energy. If you have a two theta offset and are using 
 relatively high energy (say 14 keV or so?) then this may have odd effects on 
 your data. At detector two theta = 0 this is less of a problem. This can be a 
 gotcha with processing small molecule data recorded with a little Pilatus.
 
 Best wishes Graeme
 
 
 
 
 
 On Fri Nov 14 2014 at 6:15:31 PM Sanishvili, Ruslan rsanishv...@anl.gov 
 wrote:
 Dear Graeme,
 
 Could you elaborate on There are also some subtleties to making (b) work 
 properly... some more? I have a feeling, from observing the beamline users, 
 that many choose to use this option. It would be very helpful for them to 
 know what are those subtleties and how to best make it work properly.
 Many thanks,
 Nukri
 
 
 Ruslan Sanishvili (Nukri)
 Macromolecular Crystallographer
 GM/CA@APS
 X-ray Science Division, ANL
 9700 S. Cass Ave.
 Lemont, IL 60439
 
 Tel: (630)252-0665
 Fax: (630)252-0667
 rsanishv...@anl.gov
 
 From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Graeme Winter 
 [graeme.win...@gmail.com]
 Sent: Thursday, November 13, 2014 2:15 AM
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to 
 POINTLESS/AIMLESS
 
 Dear Kay
 
 Just to comment on (e) since you say you don't know why anyone would want to 
 do this, yet this is exactly what xia2 -3d does :o)
 
 I use AIMLESS to merge data already scaled by XDS CORRECT or XSCALE as a way 
 to get a report on the merging statistics which includes all of the AIMLESS 
 analysis, and to generate harvesting files for deposition.
 
 Like you, I look forward to studies of (a) - (e)  think of all of these (c) 
 is by far the worst idea, from gut instinct. There are also some subtleties 
 to making (b) work properly...
 
 For anyone who has time on their hands  would like to do this study, be 
 sure to consider a range of crystal symmetries as it is possible that some 
 strategies which are safe in PG 422 (say) are not in PG 2.
 
 Best wishes Graeme
 
 
 
 On Wed Nov 12 2014 at 10:07:10 PM Kay Diederichs 
 kay.diederi...@uni-konstanz.de wrote:
 Hi Wolfram,
 
 it took me a while until I realized that you mean overfitting when you 
 said o-word.
 
 You can abuse XDS in a number of ways, and I would call them overfitting 
 the data although that would be using the word in a somewhat strained way: 
 reducing WFAC1 below 1, decreasing REFLECTIONS/CORRECTION_FACTOR below 50 
 come to mind, but in an extended sense there are other ways: rejecting 
 frames for no other reason than that they have low I/sigma or high Rmeas, 
 ...
 
 People always seem to find ways to beautify their precision indicators, but 
 they are just fooling themselves, because rejecting data just for cosmetic 
 reasons creates bias. In other words, they trade random error against 
 systematic error. Guess what is worse. A deeper reason of the problem is 
 that crystallographers have been fixated on data R-factors for decades, and 
 have become really spoilt by this. Our science has been completely mis-lead 
 when it comes to data statistics, and is recovering only slowly.
 
 Concerning non-cautious use of SCALA/AIMLESS after CORRECT: actually I know 
 of no systematic studies in this respect. But I know one thing: it is 
 better to be critical with respect to recipes, than to follow them blindly. 
 So I suggest the following project: compare SAD structure solution with the 
 following routes
 

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

[Phil Evans]

You can take XDS data into Pointless  Aimless (the CCP4 Data Reduction task) 
either from the unscaled INTEGRATE.HKL or the scaled XDS_ASCII.HKL file (or 
files). In the case of a single XDS_ASCII.HKL you don't need to rescale it in 
Aimless, though you can if you want.

Aimless uses a similar but not identical scaling model to XDS, which may be 
better or worse (and how do you judge?).

Phil

On 11 Nov 2014, at 19:50, wtempel wtem...@gmail.com wrote:

 Hello all,
 in a discussion on this board, Kay Diederichs questioned the effect of 
 scaling data in AIMLESS after prior scaling in XDS (CORRECT). I understand 
 that the available alternatives in this work flow are to specify the AIMLESS 
 ‘onlymerge’ command, or not.
 Are there any arguments for the preference of one alternative over the other?
 Thank you for your insights,
 Wolfram Tempel
 

Re: [ccp4bb] Aimless low resolution shell

[Phil Evans]

This is not a bug. The Aimless program itself will cut the low resolution limit 
to 30Å but then the uniqueify step generates all reflections out to the maximum 
resolution, in order to make FreeR flags for them all. Since the extra 
reflections have no data, they will not contribute to any further steps in 
solution or refinement. Don't worry!

Phil

On 3 Oct 2014, at 01:15, Alisa Glukhova alis...@umich.edu wrote:

 Dear ccp4bb,
 
 I am having an issue with low resolution shell when converting unmerged data 
 from Scalepack using Aimless.
 
 My data has a resolution of 30- 2.6 angstroms as written  in the .sca file. 
 However, when I convert it to mtz using Aimless the resolution changes 
 automatically to 91.46 - 2.6. 
 I was trying to manually change
  the resolution limits, and while the high resolution limit does change, the 
 low resolution remains at 91.46 angstroms. 
 When looking at my mtz file in HKLview I can see that only H,K,L and Rfree 
 are to 91.46 angstroms. All other columns are to the requested 30 angstroms.
 
 I am wondering if it is supposed to do that or there is something I am not 
 doing right?
 
 Thank you for your help!
 
 -- 
 Alisa Glukhova
 postdoctoral fellow
 Tesmer lab
 University of Michigan

Re: [ccp4bb] Space group numbers

[Phil Evans]

If you know the indexing that you want, from a user's point of view it is much 
easier to specify e.g. I want to choose spacegroup P6522 (with or without 
spaces), rather than Oh what's the number, I'll have to go and look it up.

Phil

On 2 Oct 2014, at 21:12, Kay Diederichs kay.diederi...@uni-konstanz.de wrote:

 On Thu, 2 Oct 2014 16:00:30 +0100, Ian Tickle ianj...@gmail.com wrote:
 
 On 2 October 2014 13:51, Kay Diederichs kay.diederi...@uni-konstanz.de
 wrote:
 
 
 I don't see any sticking to initial indexing as worthwhile to worry
 about, since in the first integration, P1 is often used anyway, and it is
 quite normal (and easy) to re-index after the intensities become available,
 during scaling. Re-indexing from P1 to the true spacegroup often changes
 the cell parameters and their order, and this is sufficiently easy and
 well-documented in the output.
 
 
 
 Ian,
 
 I'm not aware that I tried to impose re-indexing on anyone, which your 
 reaction seems to imply. Quite to the contrary: re-indexing needs to be under 
 the control of the user - and the user specifies cell parameters and space 
 group number in XDS.INP. If I understand correctly, your use case is not 
 the typical one encountered by novice crystallographers, and I'm quite sure 
 you know very well how to deal with it. 
 My whole point is about the default SETTING in POINTLESS which may lead to 
 problems for XDS users, for space groups 17 and 18. To fix it, there is no 
 need to re-invent the wheel, write new volumes of ITC, specify all space 
 group operators, or specify space group symbols instead of numbers.
 
 best,
 
 Kay
 

Re: [ccp4bb] Space group numbers

[Phil Evans]

How does XDS decide on eg P 21 21 2 when say c  b  a? The initial indexing 
may decide that the cell fits a primitive orthorhombic system, but I presume 
that it will then have some convention, probably a  b  c, since the 
identification of screws can only be done after integration, and even then may 
be uncertain. If e.g. Pointless later decides that the a axis is a dyad, and 
the b  c axes are 2(1) screws, then it can assign space group P 2 21 21 
without permuting the indices. If XDS is assigning space group P 21 21 2 then 
it must have permuted the axes from the initial indexing. It seems to me more 
straightforward to stick to the initial indexing rather than having to reindex 
after you have decided the true space group: this was Ian Tickle's point and is 
also supposedly the official IUCr-approved convention.

There are of course ambiguous cases e.g. a ~= b, but that is the same as the 
indexing ambiguities in e.g. P3, and that needs a reference dataset to resolve.

There is no problem in solving a structure in e.g. P 2 21 21, and indeed I 
would always run MR in all 8 possible primitive orthorhombic space groups, very 
easy to do in Phaser

Phil

On Tue, 30 Sep 2014 13:29:02 +0100, Phil Evans p...@mrc-lmb.cam.ac.uk wrote:

 Be careful: the International Tables space group number may be ambiguous. For 
 example sg number 18 may refer to P 21 21 2 or its permuted settings P 21 2 
 21 or P 2 21 21, if you follow the proper IUCr convention that primitive 
 orthorhombic space groups have abc

I would like to point out that there is an alternative interpretation of the 
International Tables (Vol A, 4th ed. 1995). In that interpretation (which e.g. 
XDS follows) space group 18 has the 'standard' space group symbol, P21 21 2 
(bold letters in Table 3.2). This is of course not ambiguous at all; the pure 
2-fold then corresponds to the c axis and there is always a permuation of 
axes to achieve this. As a result, the axes are not necessarily ordered such 
that abc . The latter ordering is just a convention which was chosen for 
convenience and the convention refer(s) to the cell obtained by the 
transformations from Table 9.3.1 (citing from table 9.3.2) - in other words, 
the convention is fulfilled _after_ the transformation (which of course is just 
order-permuting while keeping right-handedness) - nothing new here. 

In my understanding, CCP4 developers have (years ago) understood this 
convention as a condition, which lead them to  invent CCP4 space group 
symbols 1017 and 2017 as well as 1018, 2018, 3018. This also seems to be the 
reason for the default being SETTING CELL-BASED in POINTLESS. 

Users of XDS should be aware that by default, POINTLESS therefore permutes the 
axes such that abc . This however may lead to space groups 1017 / 2017 / 
1018/ 2018/ 3018 - indicated in the MTZ file, but not in the POINTLESS log file 
(last I checked).

In consequence, XDS will use the space group 17 or 18 (which is what POINTLESS 
reports), but the user must provide  the correct ordering (which does not 
necessarily mean abc) of cell parameters in XDS.INP. The easiest way, for XDS 
users, would be to run POINTLESS with the SETTING SYMMETRY-BASED option (I 
wish the latter were the default because the default SETTING CELL-BASED has no 
advantages that I can see). Or they use the good old manual way of 
inspecting, by eye, the systematic absences along H00 0K0 00L - this cannot 
fail.

To me, symmetry trumps cell metric so SETTING SYMMETRY-BASED should be the 
default.

I'm harping on this because I have recently seen how a Molecular Replacement 
solution was not obtained in space group 18 because of the misleading (I'd say) 
ordering abc . 

I'm probably also harping on this because it took me so many years to discover 
this failure mode, and I would like to prevent others from falling into this 
trap.

HTH,

Kay



 
 The space group names are unambiguous (though also watch out for R3  R32 
 which are normally indexed as centred hexagonal, but could be indexed in a 
 primitive cell)
 
 Phil 
 
 
 On 30 Sep 2014, at 13:07, Simon Kolstoe simon.kols...@port.ac.uk wrote:
 
 Dear ccp4bb,
 
 Could someone either provide, or point me to, a list of space-groups 
 relevant to protein crystallography just by space group number? I can find 
 lots of tables that list them by crystal system, lattice etc. but no simple 
 list of numbers.
 
 Thanks,
 
 Simon

Re: [ccp4bb] Space group numbers

[Phil Evans]

I'm still a bit confused about why there is a problem: why use SG numbers? P 2 
21 21 (or indeed P22121) is clear and unambiguous. There is no need to use 
the numbers (and certainly not the weird CCP4 numbers like 3018 which I was 
trying to hide in Pointless

Phil

On 2 Oct 2014, at 15:04, Kay Diederichs kay.diederi...@uni-konstanz.de wrote:

 On Thu, 2 Oct 2014 11:58:42 +0100, Ian Tickle ianj...@gmail.com wrote:
 
 Hi, we shouldn't be using numbers at all (CCP4-style or otherwise, since
 no-one else outside the MX community uses these).  We should be using the
 unique full Hermann-Mauguin symbol, since the 'standard setting' space
 group number in IT obviously does not uniquely define the setting, and it's
 the setting that matters.  Note that the standard setting symbol P2221
 means 'either P2122 or P2212 or P2221' according to the a=b=c convention
 (this is universal amongst the crystallography communities), so you still
 
 Once again, citing from ITC Vol A Table 9.3.2 (p. 747 in my 1995 edition) , 
 these conventions refer to the cell obtained by the transformations from 
 Table 9.3.1. They have been chosen for convenience in this table. To me, 
 this indicates that abc _could_ be obtained _if_ one were to transform. But 
 the question is: why would one want to transform? I don't see sticking to 
 the original indexing as a convincing convenience.
 
 have to define the setting if you refer to the standard symbol.  I'm aware
 
 My copy of ITC Vol A says (p 41) about Table 3.2: the 'standard' space group 
 symbols ... are printed in bold face. The Table has P 21 21 2 (18) and P 
 2 2 21 (17) in bold face. There is no ambiguity here.
 
 that some software uses the list of general equivalent positions to define
 the space group but IMO that's overkill.  If I wan't to talk about space
 group F432 you can't expect me to recite the list of 96 g.e.p.s! - the H-M
 symbol is sufficient.  There are of course other cases besides P2221 where
 the setting is ambiguous (e.g. C2/A2/I2 and various cases of origin
 shifting), so using the correct symbol for the setting is critical.
 
 The most important features of any convention are a) that it's documented
 in an 'official' publication (i.e. not informal such as software
 documentation, otherwise how am I supposed to reference it?), and b)
 everyone subscribes to it.  If you think we should be using a different
 convention then I want to see the proper documentation for it, with
 everything spelled out in excruciating detail (so it should be at least as
 thick as ITC!).  It seems to me that ITC fulfils these requirements
 admirably!
 
 Switching the default in POINTLESS from SETTING CELL-BASED to SETTING 
 SYMMETRY-BASED would make me happy, but more importantly, would avoid a lot 
 of problems.
 
 thanks,
 
 Kay
 
 
 Cheers
 
 -- Ian
 
 On 2 October 2014 10:25, Kay Diederichs kay.diederi...@uni-konstanz.de
 wrote:
 
 On Tue, 30 Sep 2014 13:29:02 +0100, Phil Evans p...@mrc-lmb.cam.ac.uk
 wrote:
 
 Be careful: the International Tables space group number may be ambiguous.
 For example sg number 18 may refer to P 21 21 2 or its permuted settings P
 21 2 21 or P 2 21 21, if you follow the proper IUCr convention that
 primitive orthorhombic space groups have abc
 
 I would like to point out that there is an alternative interpretation of
 the International Tables (Vol A, 4th ed. 1995). In that interpretation
 (which e.g. XDS follows) space group 18 has the 'standard' space group
 symbol, P21 21 2 (bold letters in Table 3.2). This is of course not
 ambiguous at all; the pure 2-fold then corresponds to the c axis and
 there is always a permuation of axes to achieve this. As a result, the axes
 are not necessarily ordered such that abc . The latter ordering is just a
 convention which was chosen for convenience and the convention
 refer(s) to the cell obtained by the transformations from Table 9.3.1
 (citing from table 9.3.2) - in other words, the convention is fulfilled
 _after_ the transformation (which of course is just order-permuting while
 keeping right-handedness) - nothing new here.
 
 In my understanding, CCP4 developers have (years ago) understood this
 convention as a condition, which lead them to  invent CCP4 space group
 symbols 1017 and 2017 as well as 1018, 2018, 3018. This also seems to be
 the reason for the default being SETTING CELL-BASED in POINTLESS.
 
 Users of XDS should be aware that by default, POINTLESS therefore permutes
 the axes such that abc . This however may lead to space groups 1017 /
 2017 / 1018/ 2018/ 3018 - indicated in the MTZ file, but not in the
 POINTLESS log file (last I checked).
 
 In consequence, XDS will use the space group 17 or 18 (which is what
 POINTLESS reports), but the user must provide  the correct ordering (which
 does not necessarily mean abc) of cell parameters in XDS.INP. The easiest
 way, for XDS users, would be to run POINTLESS with the SETTING
 SYMMETRY-BASED option (I wish the latter were the default because

Re: [ccp4bb] Space group numbers

[Phil Evans]

Be careful: the International Tables space group number may be ambiguous. For 
example sg number 18 may refer to P 21 21 2 or its permuted settings P 21 2 21 
or P 2 21 21, if you follow the proper IUCr convention that primitive 
orthorhombic space groups have abc

The space group names are unambiguous (though also watch out for R3  R32 which 
are normally indexed as centred hexagonal, but could be indexed in a primitive 
cell)

Phil 


On 30 Sep 2014, at 13:07, Simon Kolstoe simon.kols...@port.ac.uk wrote:

 Dear ccp4bb,
 
 Could someone either provide, or point me to, a list of space-groups relevant 
 to protein crystallography just by space group number? I can find lots of 
 tables that list them by crystal system, lattice etc. but no simple list of 
 numbers.
 
 Thanks,
 
 Simon

Re: [ccp4bb] Problems about i

[Phil Evans]

You may write Polish if you wish - it's case insensitive :-)

Sent from my iPhone

 On 26 Aug 2014, at 21:37, Harry Powell ha...@mrc-lmb.cam.ac.uk wrote:
 
 hi Jacob
 
 You'd have to ask Phil Evans for the definitive answer, but my understanding 
 is that it's a tribute to the developers of an integration and scaling 
 package which isn't distributed by CCP4 (and which doesn't come from the MPI 
 either).
 
 Harry
 --
 Dr Harry Powell, MRC Laboratory of Molecular Biology, Francis Crick Avenue, 
 Cambridge Biomedical Campus, Cambridge, CB2 0QH
 Chairman of European Crystallographic Association SIG9 (Crystallographic 
 Computing)
 
 On 26 Aug 2014, at 19:44, Keller, Jacob kell...@janelia.hhmi.org wrote:
 
 I’ve chuckled at the “polish” nomenclature before, as I assumed this was a 
 reference to certain software developers, but if so, shouldn’t it be 
 “Polish?” Or does it mean polish, in the sense of shoe polish?
  
 JPK
  
 From: CCP4 bulletin board [mailto:CCP4BB@JISCMAIL.AC.UK] On Behalf Of Harry 
 Powell
 Sent: Tuesday, August 26, 2014 2:32 PM
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] Problems about i
  
 hi
  
 (1) provided the predictions match the spots (i.e. indexing etc has been 
 successful) the best things to look at in the Integration task initially are 
 the graphs - if they vary reasonably smoothly, and there are no big jumps 
 then things have probably gone okay. 
  
 (2) Really, the only way to reduce the mosaicity is to grow better crystals. 
 Are you getting much larger values than with hkl2000? How many overlaps are 
 you getting (both as the overall number and as a fraction of the whole?); 
 there may be nothing wrong at all...
  
 (3) For everything apart from finding the heavy atom sub-structure with 
 SHELXC/D/E, you want the MTZ file from ctruncate (so this is the one you 
 want for molecular replacement). 
  
 For SHELXC/D/E, you really want unmerged F^2 values which can be obtained by 
 using the Aimless option output polish unmerged (as a processing option 
 under Integration).
 
 HTH
 
 Harry
 --
 Dr Harry Powell, MRC Laboratory of Molecular Biology, Francis Crick Avenue, 
 Cambridge Biomedical Campus, Cambridge, CB2 0QH
 Chairman of European Crystallographic Association SIG9 (Crystallographic 
 Computing)
 
 On 26 Aug 2014, at 16:51, 陈昂 angsc...@outlook.com wrote:
 
 Dear all：
  
 It is my first time to use IMOSFLM  instead of HKL2000. Here are my 
 problems. Hope you can help me.
  
 1、what parameters can indicate the result quality of my imosflm?  mosaicity 
 ,anything else?
  
 2、what can I do to reduce the mosaicity and decrease bad spots,resolution or 
 the spot finding range?
  
 3、after imosflm, I've got some MTZ maps such as original one and pointless 
 one,aimless one, ctruncate one. Which one should I take to the next 
 step?what steps do I need to take before  molecular replacement and why?
  
  
 THANKS a lot
  
  
 Peter Chen

Re: [ccp4bb] A bug in aimless in last update

[Phil Evans]

There was indeed a bug for which I apologise. It is fixed in version 0.3.11 
which will be filtering through CCP4 updates in due course. In the mean time if 
you get this bug you will have to revert to an earlier version (0.3.6 or 
earlier I think), or you can get it (Linux version) from 

ftp://ftp.mrc-lmb.cam.ac.uk/pub/pre/aimless-0.3.11.linux64

Apologies
Phil


On 12 Aug 2014, at 08:10, Robert Gustafsson robert.gustafs...@dbb.su.se wrote:

 Dear all,
 
 It seems that the last update of the ccp4 suite contains at least one bug in 
 aimless. Failed every time with the same error message (below) and worked 
 again when update was uninstalled. It is the update 6.4.0-020 that was the 
 cause. 
 
 Did not know who to send to but hopefully at least someone here knows or is 
 that person.
 
 Have a nice day!
 
 Sincerely,
 Robert Gustafsson
 
 ***
 The program run with command: 
 
 (Here was a list of files and options such as HKLIN)
 
 has failed with error message
 Assertion failed: (sd  0.0), function Average, file 
 /Users/buildbot/Buildslaves/ccp4-slave/release-6_4_0-mac10_6/build/devtools/checkout/aimless/selectedobservations.cpp,
  line 250.
 ***
 
 ___
 Robert Gustafsson
 PhD Student
 Department of Biochemistry and Biophysics
 Stockholm University
 106 91 Stockholm, Sweden
 
 e-mail: robert.gustafs...@dbb.su.se
 

Re: [ccp4bb] CC-half value ??

[Phil Evans]

I should make the estimation in Aimless more robust, and curve fitting sounds 
like a good idea (but what function?). Outliers are a difficult problem, but 
anyway I think you should look at the curve and not just the number estimated. 
I would look at I/sigI as well, and anisotropy to decide the resolution. 
However, the final cutoff should probably be based on refinement, and also I 
don't think the exact cutoff makes a huge difference (see 
http://www.ncbi.nlm.nih.gov/pubmed/23793146)

Phil

On 15 Aug 2014, at 15:54, Ed Pozharski pozharsk...@gmail.com wrote:

 Same here.  Ultimately, the KD test must be used in the end to finalize the 
 resolution (keeping in mind recently discussed issues of effective resolution 
 given data completeness).  I just want to add that at least some versions of 
 aimless report overestimated resolution based on CC1/2 cutoff when outliers 
 are present (e.g. due to ice rings or salt diffraction). It seems that 
 aimless just picks the highest resolution bin where cc1/2 0.5 even if some 
 lower resolution bins are below 0.5 as well. I have written a script for more 
 robust automated evaluation of these curves.  In a nutshell, it fits CC1/2 
 (d) curve to 1/(1+exp (-x)) and returns the resolution at midpoint.  I'm 
 pretty sure that theoretical CC1/2 (d) dependence is different from this, but 
 it seems good enough for a rough estimate. 
 
 
 Sent on a Sprint Samsung Galaxy S® III
 
 
 
 
 
 
 
 
  Original message 
 From: Roger Rowlett
 Date:08/14/2014 5:44 PM (GMT-05:00)
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] CC-half value ??
 
 Exactly. Aimless will give you suggested resolution cutoffs based on CC 1/2 
 in the log file.
 
 Roger Rowlett
 
 On Aug 14, 2014 5:04 PM, conan仙人指路 conan_...@hotmail.com wrote:
 Hi Faisal,
 
   CC-half standard is valuable in evaluating the cut-off of highest 
 resolution. Sometimes even if I/sigI is close to 1 and completeness is not as 
 high, if CC-half is still significant, it may be worth incorporate the extra 
 high-res shell data and extend the resolution. Again, if only the reliability 
 and unbias are carefully confirmed, and the apparent significant CC-half is 
 not due to an artifact of some other factors like ice ring etc.
 (Ref: Karplus PA and Diederichs K. 2012 Science 336, 1030-1033 
 https://www.pubmed.com/pubmed/22628654)
 
   It has yet to be appreciated by most population of the crystallography 
 society, unlike the I/sigI, completeness, Rsym. In particular, Rsym has 
 gradually less a direct measurement of the data quality and or determinant of 
 resolution cut-off. 
 
 Best,
 Conan
 
 Hongnan Cao, Ph.D.
 Department of Biochemistry
 Rice University
 
 Date: Fri, 15 Aug 2014 01:39:48 +0530
 From: faisaltari...@gmail.com
 Subject: [ccp4bb] CC-half value ??
 To: CCP4BB@JISCMAIL.AC.UK
 
 Dear all
 
 How CC-half value of a data set determines the maximum resolution limit 
 during data processing ?? Although much we know about the Rsym and I/Isig 
 values of the highest resolution shell while processing the data, what are 
 the parameters we need to check related to CC-half values ?? 
 
 -- 
 Regards
 
 Faisal
 School of Life Sciences
 JNU
 

Re: [ccp4bb] m_range_check error in aimless

[Phil Evans]

What version of Aimless are you running? I thought I had fixed that bug

Phil

On 13 Jun 2014, at 11:59, Andreas Förster docandr...@gmail.com wrote:

 Dear all,
 
 I'm trying to scale/merge mtz files in ccp4 
 (Pointless/Aimless/ctruncate/Rfree pipeline) and keep getting an
 
 UNHANDLED EXCEPTION: vector::_M_range_check
 
 at the end of the aimless step, right after the standard deviation v. 
 intensity tables.  The output mtz file is not written and consequently 
 ctruncate craps out because of no input file.
 
 This occurs with the temporary directory on NSF and local, and with one or 
 two mtz files as input.
 
 How can I go beyond this?
 
 
 Andreas
 
 
 -- 
  Andreas Förster
 Crystallization and X-ray Facility Manager
   Centre for Structural Biology
  Imperial College London

Re: [ccp4bb] merging of reflection with identical indices

[Phil Evans]

XDS_ASCII.HKL or scalepack format ('no merge original index') can be read 
directly by Pointless which should (I hope) do a better job than COMBAT.

Aimless ( Pointless) assume two reflections are part of the same one if they 
have the same true hkl (same ISYM) and adjacent batch numbers (and a few 
other conditions, e.g. total fraction). Output from COMBAT may be wrong in this 
respect

Phil

On 10 Jun 2014, at 17:23, wtempel wtem...@gmail.com wrote:

 Hello all,
 suppose I extracted 
 H  K  L  Intensity  sigma[Intensity]
 from a file of unmerged intensities, such XDS_ASCII.HKL or scalepack format 
 ('no merge original index'). Batch or rotation angle information would have 
 been omitted, due to a limitation of the output file's format. 
 Should I not still be able to merge these intensities without scaling, such 
 as in AIMLESS with the 'onlymerge' option?
 I coerced the ascii-formatted reflections into MTZ format using COMBAT, 
 specifying '1' for the mandatory BATCH keyword. Subsequently, POINTLESS 
 output the following lines
 
 ## 
Number of reflections = 62739
Number of observations =210959
Number of parts=252273
 ##
 
 The discrepancy between numbers for observations and parts exactly 
 matches double the number of HKLs with two occurrences in my input file. How 
 could I force treatment of duplicate HKLs as independent observations, given 
 that I have lost the batch information?
 Would it be sufficient to apply 'artificial' batch numbers 1, 2, ... to 
 disambiguate between duplicate HKLs?
 Thanking you in advance for any advice,
 Wolfram Tempel

Re: [ccp4bb] (high) values of R-factors in outermost resolution shell

[Phil Evans]

That looks good to me Mean(I/sigma) = 2.0, CC1/2 0.74

See endless discussions on this BB about the uselessness of Rmerge as a 
resolution criterion

Phil

On 2 Jun 2014, at 09:27, sreetama das somon_...@yahoo.co.in wrote:

 Dear All,
   What are reasonable values of Rmerge in the outermost 
 resolution shell? 
 
 Some of the recent discussions suggest going to those sheels where I/sig(I) 
 ~2 and CC1/2 = 0.5. But I am getting Rmerge  Rmeas  1 in the outermost 
 shell for those values of I/sig(I) and CC1/2, and I don't think that makes 
 any sense. Reducing the resolution cut-off while data reduction  scaling 
 (aimless) reduces the R-values, but I am not sure how much I should reduce 
 the resolution (if at all). 
 
 Following are the results from the aimless log file:
 At a resolution cut-off of 1.62A:
Overall  InnerShell  OuterShell
 Low resolution limit   42.12 42.12  1.65
 High resolution limit   1.62  8.87  1.62
 
 Rmerge  (within I+/I-) 0.071 0.019 1.325
 Rmerge  (all I+ and I-)0.074 0.020 1.381
 Rmeas (within I+/I-)   0.078 0.021 1.446
 Rmeas (all I+  I-)0.077 0.022 1.441
 Rpim (within I+/I-)0.031 0.009 0.575
 Rpim (all I+  I-) 0.022 0.007 0.410
 Rmerge in top intensity bin0.031- - 
 Total number of observations  311022  1839 14876
 Total number unique25311   184  1212
 Mean((I)/sd(I)) 19.2  52.7   2.0
 Mn(I) half-set correlation CC(1/2) 1.000 1.000 0.740
 Completeness   100.0  99.4 100.0
 Multiplicity12.3  10.0  12.3
 
 At 1.6A, the I/sig(I) and CC1/2 in the outermost shell are lower, and the 
 R-merge,meas,pim are higher.
 
 looking forward to your suggestions,
 thanking you,
 sreetama

Re: [ccp4bb] Potentially serious if unusual bug in handling symmetry

[Phil Evans]

Surely Refmac or any other program should honour the symmetry operators in the 
MTZ file - that's why they are there!

Phil

On 23 May 2014, at 12:34, Eleanor Dodson eleanor.dod...@york.ac.uk wrote:

 
  Someone here has seen his Rfactors leap from 18% to 42% and was naturally 
 distressed! 
 We have tracked down the problem to this:
 
 Steps he took were:
 
 1)  Integrate with XDS and feed output through pointless/aimless etc.
 He used a phenix refined mtz to Match Laue and indexing 
 
 The space group there is given by PHENIX 
 as P 2 21 21  but the spacegroup NUMBER is given as 18 (ie P21 21 2 in CCP4 
 symmetry library) .
 
 pointless then output an mtz file with SPACEGROUP number as 18 and SG name as 
 P 2 21 21.
 The symmetry operators are correctly listed for P 2 21 21
  This is the version number: CCP4 6.4: POINTLESS  version 1.9.8 : 
 02/05/14
 
 2) When the refinement was repeated with REFMAC  the spacegroup was taken as 
 P21 21 2 - ie the SG number 18 overrode the listed symmetry operators in the 
 mtz file, and the SG name P 2 21 21 given on the CRYST1 card was ignored.
 
 The refmac version used is  CCP4 6.4: Refmac_5.8.0071 version 5.8.0071 : 
 04/04/14
 
 
 The user thinks that in REFMAC 5.8.0033 this problem did not occur, but I 
 havent checked that..
 
 
 Anyway - it does emphasise the importance of checking symmetry operators for 
 consistency with both the SG name and number (Yes, George, I know you have 
 always said this is the only correct policy!) Maybe an extra subroutine could 
 be added to the symmetry library and called from other programs..
 
 And ideally checking all sources of symmetry information for consistency and 
 stopping if there is a serious disagreement. 
 
 
 In the short term we simply took the processed mtz file and ran
 mtzutils hklin1 corrupted.mtz hklout fixed.mtz
 SYMM 3018   (or symm P2 21 21 ) 
 and the fixed file was OK
 
 Eleanor Dodson
 
 
 

Re: [ccp4bb] Potentially serious if unusual bug in handling symmetry

[Phil Evans]

OK I guess Pointless is accepting the number as given in the reference file, 
rather than changing it.

If Pointless is just determining the space group it does give the correct 
(i.e. weird CCP4 convention) number

* Space group = 'P 2 21 21' (number 3018)

I'm not sure how to test this as I don't know how to make an mtz file with P 2 
21 21 /sgn 18, but I will look into it

Phil

On 23 May 2014, at 12:34, Eleanor Dodson eleanor.dod...@york.ac.uk wrote:

 
  Someone here has seen his Rfactors leap from 18% to 42% and was naturally 
 distressed! 
 We have tracked down the problem to this:
 
 Steps he took were:
 
 1)  Integrate with XDS and feed output through pointless/aimless etc.
 He used a phenix refined mtz to Match Laue and indexing 
 
 The space group there is given by PHENIX 
 as P 2 21 21  but the spacegroup NUMBER is given as 18 (ie P21 21 2 in CCP4 
 symmetry library) .
 
 pointless then output an mtz file with SPACEGROUP number as 18 and SG name as 
 P 2 21 21.
 The symmetry operators are correctly listed for P 2 21 21
  This is the version number: CCP4 6.4: POINTLESS  version 1.9.8 : 
 02/05/14
 
 2) When the refinement was repeated with REFMAC  the spacegroup was taken as 
 P21 21 2 - ie the SG number 18 overrode the listed symmetry operators in the 
 mtz file, and the SG name P 2 21 21 given on the CRYST1 card was ignored.
 
 The refmac version used is  CCP4 6.4: Refmac_5.8.0071 version 5.8.0071 : 
 04/04/14
 
 
 The user thinks that in REFMAC 5.8.0033 this problem did not occur, but I 
 havent checked that..
 
 
 Anyway - it does emphasise the importance of checking symmetry operators for 
 consistency with both the SG name and number (Yes, George, I know you have 
 always said this is the only correct policy!) Maybe an extra subroutine could 
 be added to the symmetry library and called from other programs..
 
 And ideally checking all sources of symmetry information for consistency and 
 stopping if there is a serious disagreement. 
 
 
 In the short term we simply took the processed mtz file and ran
 mtzutils hklin1 corrupted.mtz hklout fixed.mtz
 SYMM 3018   (or symm P2 21 21 ) 
 and the fixed file was OK
 
 Eleanor Dodson
 
 
 

Re: [ccp4bb] C2/I2 space groups

[Phil Evans]

I2 and C2 are different settings of the same group. The official IUCr 
convention is to use the one which gives the beta angle closer to 90 degrees. 
As far as I know all programs should now be able to use the I2 setting, but if 
it worries you, you can reindex to C2

Phil

On 21 May 2014, at 17:54, Roberto Battistutta roberto.battistu...@unipd.it 
wrote:

 Dear All,
 a question about C2 and I2 space groups.
 Processing a dataset, XDS output says C2, with dimensions 122.8, 56,9 81,5 
 and beta 125.1°.
 Aimless reindexes to I2 with 81.6, 56.9, 101.1 and 96.2°.
 Phenix (refine) returns a warning NOTE: non-standard setting used: I 1 2 1.
 In the PDB there are indeed several examples of I2 choices.
 
 The two space groups are equivalent as indicated in the International Tables 
 (all with number 5), but why different programs use different conventions? Is 
 there any rationale? By the way, the volumes of the two unit cells are very 
 similar.
 
 Thanks,
 Roberto.
 
 
 Roberto Battistutta
 Associate Professor
 Department of Chemistry
 University of Padua
 via Marzolo 1, 35131 Padova - ITALY
 tel. +39.049.827.5262
 fax. +39.049.827.5829
 roberto.battistu...@unipd.it
 www.chimica.unipd.it/roberto.battistutta/
 VIMM (Venetian Institute of Molecular Medicine)
 via Orus 2, 35129 Padova - ITALY
 tel. +39.049.7923.236
 fax +39.049.7923.250
 www.vimm.it

Re: [ccp4bb] anomalous signal

[Phil Evans]

Was there a reason that you turned off the scaling in Aimless (onlymerge)? If 
the data have come from Mosflm, this is definitely wrong - the result is that 
(among other things) you have negative CCanom values which is unusual to say 
the least

Just run it with the default options, that's usually the best thing to do to 
start with

Phil

On 26 Apr 2014, at 14:38, Faisal Tarique faisaltari...@gmail.com wrote:

 Dear Eleanor and Tim.
 
 i have reprocessed the data through imosflm and run the aimless through the 
 unmerged output mtz..i am attaching the output log file of the 
 aimless..please tell me how to interpret the anomalous signal from the log 
 file and where the information is written..
 
 Thanks again for your much needed help.
 
 regards
 
 Faisal
 
 
 On Sat, Apr 26, 2014 at 5:22 PM, Tim Gruene t...@shelx.uni-ac.gwdg.de wrote:
 Dear Faisal,
 
 the lack of the CCanom line in the shelxc output suggests that your data
 are already merged, and my guess is you processed your data with HKL2000
 - all other integration programs I am aware of do not merge the data at
 such an early stage giving you access to the CCanom Eleanor mentioned.
 
 There might be a switch in HKL2000 to not merge the data. A CCanom 30%
 is a good indicator of the presence of an anomalous signal.
 
 Best,
 Tim
 
 On 04/26/2014 12:18 PM, Eleanor Dodson wrote:
  Look at the aimless plot of CCanom . That is the best indicator I think and 
  very sensitive when you have such high redundancy
  Eleanor
 
 
  On 25 Apr 2014, at 22:13, Jim Pflugrath wrote:
 
  d/sig should be above 0.80
 
  There seems to be plenty of signal there with all values above 1.02.  We 
  have solved structures with less multiplicity and lower d/sig.
 
  There is a different criteria of signal for when you know the positions 
  of the anomalous substructure atoms and when you need to find the 
  positions of the anomalous substructure atoms.
 
  As for no signal, I think I am on record that there is always an 
  anomalous signal. :)  But can you detect it?
 
  Jim
 
  From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Faisal 
  Tarique [faisaltari...@gmail.com]
  Sent: Friday, April 25, 2014 4:06 PM
  To: CCP4BB@JISCMAIL.AC.UK
  Subject: [ccp4bb] anomalous signal
 
  Dear all
 
  sorry about my previous mail where i forgot to mention that the data was 
  collected on home source at Cuk alpha and at 1.54A.
 
  written below is the log file of an anomalous data processed through 
  SHELXC..my question is ..what is the strength of anomalous signal ?? as it 
  is said For zero signal d'/sig and d/sig should be about 0.80. Then 
  in the present case is there really a signal or can be assumed no 
  signal..we are expecting one Ca atom bound to the protein at its active 
  site..the redundancy of the data is 11.6..with this signal strength can we 
  assume Ca to be present there or whatever little anomalous if present is 
  due to something elseor there is no signal at all ??...
 
  Resl.   Inf - 8.0 - 6.0 - 5.0 - 4.0 - 3.8 - 3.6 - 3.4 - 3.2 - 3.0 - 2.8 - 
  2.60
   N(data) 375   493   580  1319   450   538   679   866  1081  1414  
  1709
   I/sig58.8  38.6  32.6  38.3  27.7  27.2  21.9  18.4  12.6   9.5   
  6.1
   %Complete  94.7  99.0  99.3  99.5 100.0  99.6  99.7  99.8  99.6  99.6  
  90.9
   d/sig   1.65  1.27  1.18  1.25  1.19  1.12  1.11  1.11  0.97  1.02  
  1.05
 
  --
  Regards
 
  Faisal
  School of Life Sciences
  JNU
 
 
 
 --
 Dr Tim Gruene
 Institut fuer anorganische Chemie
 Tammannstr. 4
 D-37077 Goettingen
 
 GPG Key ID = A46BEE1A
 
 
 
 
 -- 
 Regards
 
 Faisal
 School of Life Sciences
 JNU
 
 223_aimless.log

Re: [ccp4bb] CCP4i aimless problem, no HKLIN

[Phil Evans]

Just tested with a clean binary install on a Mac (running 10.8), and it works 
for me

Phil


On 8 Apr 2014, at 15:19, Björn Kauppi bjorn.kau...@karobio.se wrote:

 
 Hi
  
 I run latest CCP4 as of today, 6.4.0 update 12 and CCP4i 2.2.1
 When trying to run pointless/aimless from the interface the script generated 
 show no HKLIN and hence program fails running. If I do “RunView Com-file” 
 and manually add HKLIN at proper place in the script, it runs fine.
 This problem was introduced a few updates ago.
  
 Scala has no problem
  
 Is it just me or something general.
 What do do?
  
 --Björn
  
 
  
 __
 This e-mail may contain confidential information proprietary to
 Karo Bio AB and is meant for the intended addressee(s) only. Any
 unauthorized review, use, disclosure or distribution is prohibited.
 If you have received this message in error, please advise the sender
 and delete the e-mail and any attachments from your files. Thank you!
 __
 

Re: [ccp4bb] difference between polar angle and eulerian angle

[Phil Evans]

The polar angles ϕ, ω define the direction of an axis about which a rotation by 
angle κ occurs, i.e. a single rotation around a defined axis. This is different 
from Eulerian angles which define 3 successive rotations around principal axes

On 27 Mar 2014, at 06:11, Qixu Cai caiq...@gmail.com wrote:

 Dear all,
 
 From the definition of CCP4 
 (http://www.ccp4.ac.uk/html/rotationmatrices.html), the polar angle (ϕ, ω, κ) 
 can be visualised as rotation ϕ about Z, rotation ω about the new Y, rotation 
 κ about the new Z. It seems the same as the ZXZ convention of eulerian angle 
 definition. What's the difference between the CCP4 polar angle definition and 
 eulerian angle ZXZ definition?
 
 And what's the definition of polar angle XYK convention in GLRF program?
 
 Thank you very much!
 
 Best wishes,
 
 -- 
 Qixu Cai
 Email: caiq...@gmail.com
 School of Life Sciences,
 Xiamen University, Fujian, China
 **from thunderbird**

Re: [ccp4bb] pairwise CCano

[Phil Evans]

If you assigns them to different datasets in Pointless, then Aimless will give 
you the cross-dataset correlations. By default it will scale them to together 
first, but you can skip that if you want

It might not scale well to a large number of files (OK up to about 10 I guess)

Phil

On 14 Mar 2014, at 09:33, Tim Gruene t...@shelx.uni-ac.gwdg.de wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Dear all,
 
 I am looking for a tool that prints (and preferably plots, e.g. as
 postscript) the pairwise anomalous CC vs. resolution for several input
 HKL-files.
 xprep does this, but it is interactive and requires a fair bit of
 typing. Since I have a large number of HKL-files from XDS, I would like
 to script that and then flip through the pages of the postscript-plots.
 
 I looked into pointless but could not find even a table.
 
 Since there recently were some publications one the use of many files
 for phasing, I though such a tool should exist!?
 
 Best,
 Tim
 
 - -- 
 - --
 Dr Tim Gruene
 Institut fuer anorganische Chemie
 Tammannstr. 4
 D-37077 Goettingen
 
 GPG Key ID = A46BEE1A
 
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.12 (GNU/Linux)
 Comment: Using GnuPG with Icedove - http://www.enigmail.net/
 
 iD8DBQFTIszcUxlJ7aRr7hoRApYyAKDt36pF11DAbkfcXVq+uLjqvm91HgCfSZ+V
 BVpiCx4q9DZZCqDLenHX374=
 =itCO
 -END PGP SIGNATURE-

Re: [ccp4bb] I/sigmaI or I/sigmaI

[Phil Evans]

I/sigmaI

On 12 Feb 2014, at 11:43, Cai Qixu caiq...@gmail.com wrote:

 Dear all,
 
 Does the I/sigmaI in “Table 1” mean for I/sigmaI or I/sigmaI ?
 
 Thanks for your answer.
 
 Best wishes,
 
 Qixu Cai

Re: [ccp4bb] determining best of alternative indexes with POINTLESS

[Phil Evans]

Indeed that is a bug. I've never tried that combination before. I'll fix it

Phil

On 16 Jan 2014, at 20:32, wtempel wtem...@gmail.com wrote:

 Hello,
 using merged scalepack intensities and a reference MTZ file as inputs, I 
 would like to prepare an MTZ of scalepack intensities reindexed so that the 
 intensities optimally correspond to those in the reference MTZ file.
 
 Invoking POINTLESS with the
 CELL, SCAIN, HKLREF keywords results in 
 
 CCP4MTZfile: open_read - File missing or corrupted:myscalepackoutput.sca
 hkl_merged_list: object not constructed
 
 
 
 
 Does pointless expect non-merged scalepack intensities, even though I assume 
 the Laue group of the reference MTZ file?
 
 
 
 
 I could use no merge original index scalepack reflections were not that 
 reflection file format lacking the unit cell dimensions. For simplicity and 
 reliability, I would like to use an input file that provides both intensities 
 and the corresponding cell dimensions.
 
 Can I achieve my goal by invoking POINTLESS differently. Could I provide 
 merged reflections in MTZ format? Are there better approaches to my problem?
 
 Thank you for your consideration,
 
 
 
 
 Wolfram Tempel
 

Re: [ccp4bb] scala discrepancy between versions

[Phil Evans]

Dear Jens

I'm surprised by this as Scala hasn't changed for quite a long time, as it is 
now now superseded by Aimless. However, I haven't implemented a reference 
dataset in Aimless since in the end I decided that it wasn't very useful, so if 
you find it useful then you will indeed need to use Scala.

I will investigate, but in the meantime you could just copy an old version of 
Scala into the updated ccp4 directory (i.e. into $CBIN == CCP4/bin/)

Phil

On 10 Dec 2013, at 04:05, Jens Kaiser kai...@caltech.edu wrote:

 Dear developers,
  We have been doing the local scaling with scala of multiple
 wavelengths for some time. We scaled one dataset collected remotely from
 the absorption edge, and input this to scala as a reference as one batch
 without any problems. With the latest release of CCP4, though, we get
 the following error (reference dataset is assigned to batch 9):
 
  Missing phi limits for batch   9
 
 
 
  No valid orientation data for batch  9: this is not allowed
 with
 TAILS, SECONDARY, ABSORPTION or BEAMS options
 
 Scala:   Failed in SETSCL 
 Times: User:   0.0s System:0.0s Elapsed: 0:00
 
 Our current workaround is to use the last, not the latest, release of
 CCP4.
 
 Any help is appreciated,
 
 Jens

Re: [ccp4bb] 100% Rmerge in high resolution shell

[Phil Evans]

I've generally found that adding lines to the standard table works, and they 
are not removed by editors


On 19 Nov 2013, at 09:32, Tim Gruene t...@shelx.uni-ac.gwdg.de wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Dear Graeme,
 
 On 11/19/2013 09:02 AM, Graeme Winter wrote:
 [...] For the merged I/sig(I) Rpim is much more instructive. I'd
 love it if people reported merged and unmerged I/sig(I), Rmerge,
 Rmeas, Rpim, CC1/2, ... as each of these tells something
 different.
 Depending on where you publish the editor will ask you to use their
 standard layout for the table which was probably last updated in the
 1990's given the presence of something as sophisticated as an Rfree...
 
 That's my recent experience, which undermined my preference for
 scientifically sound journals over tabloids. Unfortunately, it's the
 latter that funding agency like better ...
 
 Best,
 Tim
 
 
 
 Best wishes,
 
 Graeme
 
 Possibly useful papers:
 
 http://www.nature.com/nsmb/journal/v4/n4/abs/nsb0497-269.html 
 http://scripts.iucr.org/cgi-bin/paper?he0191 
 http://scripts.iucr.org/cgi-bin/paper?he0268
 
 
 
 
 On 19 November 2013 06:43, Shanti Pal Gangwar
 gangwar...@gmail.com wrote:
 
 Dear  All
 
 
 Can anyone explain the meaning and relevance of data when the
 Rmerge is 100% in high resolution shell and I/sig(I) is 3.
 
 
 
 Thanks
 
 
 
 --  regards Shanti Pal Gangwar School of Life
 Sciences Jawaharlal Nehru University New Delhi-110067 India 
 Email:gangwar...@gmail.com
 
 
 
 
 
 - -- 
 - --
 Dr Tim Gruene
 Institut fuer anorganische Chemie
 Tammannstr. 4
 D-37077 Goettingen
 
 GPG Key ID = A46BEE1A
 
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.12 (GNU/Linux)
 Comment: Using GnuPG with Icedove - http://www.enigmail.net/
 
 iD8DBQFSizAgUxlJ7aRr7hoRAtW8AJ9faxDJ6Wz2F5frob8PlOOXne2ZMACfdGxv
 fA0SSd2GsXKQRqZwg6MHjOk=
 =fyIi
 -END PGP SIGNATURE-

Re: [ccp4bb] ccp4 man-pages

[Phil Evans]

AFAICS the .doc files look as if they were auto-generated from the .html files 
in 6.3.0. Certainly the only documentation I have written for Pointless and 
Aimless is in html

Phil

On 15 Nov 2013, at 14:33, Edward A. Berry ber...@upstate.edu wrote:

  (assuming of course the .doc files aren't also missing).
 
 CDOC: Undefined variable.
 [berry@sbserv ~]$ ls $CCP4/doc
 ls: cannot access /sw/lnx/ccp4-6.4.0/doc: No such file or directory
 
 I second the request for continued man pages and .doc files
 
 Ian Tickle wrote:
 I agree completely with Tim: I use 'man' (or 'info' for gfortran  the like) 
 all the time
 - but then I'm a die-hard command-liner/shell-scripter who never uses ccp4i! 
  I haven't
 got around to installing 6.4 yet but if  when I do  I find the man pages 
 missing as Tim
 says I'll probably copy over the ones from 6.3 - even though they're maybe 
 now a bit out
 of date, it's better than no man pages!  ... or maybe better would be 
 something like:
 
 alias manc 'less $CDOC/\!^.doc'
 
 (assuming of course the .doc files aren't also missing).
 
 ... or this looks promising:
 ftp://jaguar.ncsl.nist.gov/current_docs/sctk-1.2/doc/html2man.pl (haven't 
 tried it though).
 
 Cheers
 
 -- Ian
 
 
 On 15 November 2013 09:48, Tim Gruene t...@shelx.uni-ac.gwdg.de
 mailto:t...@shelx.uni-ac.gwdg.de wrote:
 
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
 
Dear all,
 
I am sorry about this 'double' post, but I had no reply on ccp4bb-dev
within about three weeks.
 
I wonder if I was the only one using the ccp4 man-pages. They were
available until ccp4-6.3 but seem to be absent in the latest version.
 
I find them utterly handy, much faster than locating the
html-documentation through a web-browser and they probably do not take
up much space.
 
So I hope to get some votes for redistributing the man-pages through
the ccp4bb.
 
Best,
Tim
 
- --
- --
Dr Tim Gruene
Institut fuer anorganische Chemie
Tammannstr. 4
D-37077 Goettingen
 
GPG Key ID = A46BEE1A
 
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Icedove - http://www.enigmail.net/
 
iD8DBQFShe4AUxlJ7aRr7hoRAh5qAJ4sHMCDcv1QfVUEmBCzLzNTbqkdvQCfRe4Q
RKQR1yeSfLPRFmt/C0RVxWA=
=U9wW
-END PGP SIGNATURE-
 
 

Re: [ccp4bb] observed criterion sigma

[Phil Evans]

Always deprecated, hopefully never common!



On 14 Nov 2013, at 11:27, Mark J van Raaij mjvanra...@cnb.csic.es wrote:

 It used to be common to only include reflections for which I  x sigma(I) in 
 refinement, with x often being 3.
 However, nowadays this is not considered good practise, as reflections with 
 small Is are likely to have I  3 sigma(I), but are also important for 
 refinement.
 
 In small molecule crystallography I think it is still common to have a sigma 
 criterion.
 
 if, in this respect, you have used default settings in data reduction and 
 refinement, you should select none or n.a. in the deposition process.
 
 Mark J van Raaij
 Lab 20B
 Dpto de Estructura de Macromoleculas
 Centro Nacional de Biotecnologia - CSIC
 c/Darwin 3
 E-28049 Madrid, Spain
 tel. (+34) 91 585 4616
 http://www.cnb.csic.es/~mjvanraaij
 
 
 
 
 
 On 14 Nov 2013, at 12:06, Faisal Tarique wrote:
 
 Dear all
 
 I request you to please explain the Observed criterion sigma value
 required during pdb deposition. Does it depends on the methods of data
 integration and scaling ? if yes then what are the values for the data
 processed through scalepack2mtz (HKL2000) and scala (mosflm)..
 
 -- 
 Regards
 
 Faisal
 School of Life Sciences
 JNU

Re: [ccp4bb] Fwd: Citing Aimless

[Phil Evans]

It should be printed at the end of the log file

 P.R.Evans and G.N.Murshudov, 'How good are my data and what is the 
resolution?' Acta Cryst. D69, 1204-1214  (2013).

Phil

On 8 Nov 2013, at 11:35, Zheng Zhou zhengzho...@gmail.com wrote:

 Dear all
 
 I am also looking for aimless reference. Which paper should I cite? Thanks 
 for the powerful tools for the community. 
 
 Best, 
 
 Joe
 
 
 On Sun, Feb 24, 2013 at 9:58 PM, Morten Groftehauge m...@mb.au.dk wrote:
 Hi guys,
 
 In the log from Aimless the only reference mentioned is the 1994 CCP4 paper 
 and then as well as any specific reference in the program write-up. 
 First of all, is this the correct CCP4 reference to use?
 And second of all, since running Aimless through the interface always invokes 
 Pointless and ctruncate, wouldn't I always cite those as well? I might not 
 need Pointless to determine the space group but it doesn't hurt and doesn't 
 Aimless use information from that run?
 
 Cheers,
 Morten
 
 -- 
 Morten K Grøftehauge, PhD 
 Pohl Group
 Durham University
 
 
 
 -- 
 Morten K Grøftehauge, PhD 
 Pohl Group
 Durham University
 

Re: [ccp4bb] Conflicting Qt on OS X 10.6

[Phil Evans]

I had this problem last week but it seems to have been solved by reinstalling 
the ccp4 6.4.0 package today, slightly updated I believe

There was message from Eugene Krissinel which may be relevant to this

Phil

On 28 Oct 2013, at 16:00, Dmitry Rodionov d.rodio...@gmail.com wrote:

 Good day!
 
 I'm having problems with qtrview (on OS X 10.6): it crashes or hangs after 
 spitting out a screenful of messages like
 
 objc[2975]: Class QCocoaMenu is implemented in both 
 /Applications/ccp4-6.4.0/bin/../Frameworks/QtGui.framework/Versions/4/QtGui 
 and /Library/Frameworks/QtGui.framework/Versions/4/QtGui. One of the two will 
 be used. Which one is undefined.
 
 Just like the message says, I have Qt 4 installed in the standard location. 
 It seems like the global Qt it is being used instead of the one supplied with 
 CCP4 6.4 since moving /Library/Frameworks/Qt* elsewhere fixes the problem.
 Oddly enough, qtrview of CCP4 6.3 worked fine under the same conditions.
 
 Is there a way to make qtrview use CCP4's Qt and ignore other versions?
 
 Thanks!
 
 Best regards,
   Dmitry
 

Re: [ccp4bb] Tool to compute F/FP from F(+) / F(-)

[Phil Evans]

That won't work. Aimless expects intensities not Fs. I would think you could do 
it with sftools

Phil

On 25 Oct 2013, at 09:54, Tim Gruene t...@shelx.uni-ac.gwdg.de wrote:

 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Dear Rojan,
 
 I would try aimless with the option 'onlymerge'.
 
 Best,
 Tim
 
 On 10/25/2013 09:33 AM, Rojan Shrestha wrote:
 Hello,
 
 
 
 What is the tool in CCP4 to calculate F from F(+) and F(-) with
 their standard deviations?
 
 
 
 Regards,
 
 
 
 Rojan
 
 
 
 - -- 
 - --
 Dr Tim Gruene
 Institut fuer anorganische Chemie
 Tammannstr. 4
 D-37077 Goettingen
 
 GPG Key ID = A46BEE1A
 
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.15 (GNU/Linux)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
 
 iD8DBQFSajG9UxlJ7aRr7hoRApvtAKDAUqTykSs06NhOAveRMIYQyeou9wCdH4YN
 E3/HNFgQlCCzYHfXcfzpxPs=
 =IOXY
 -END PGP SIGNATURE-

Re: [ccp4bb] how to cut back resolution of a well-refined model

[Phil Evans]

Please explain how you think that cutting back the resolution will improve your 
model
Phil

On 10 Oct 2013, at 21:57, Yafang Chen yafangche...@gmail.com wrote:

 Hi All,
 
 I have a structure at 2.45A which has been well refined. However, since the 
 R-merge at the last shell is above 1 (although I/sigmaI at the last shell is 
 more than 2), we now decide to cut back the resolution to about 2.6A. Is 
 there a way to do this based on the well-refined model instead of doing the 
 MR and refinement all over again? Thank you so much for your help!
 
 Best,
 Yafang
 
 -- 
 Yafang Chen
  
 Graduate Research Assistant
 Mesecar Lab
 Department of Biological Sciences
 Purdue University
 Hockmeyer Hall of Structural Biology
 240 S. Martin Jischke Drive
 West Lafayette, IN 47907

Re: [ccp4bb] Code to handle the syntax of (mm)CIF data correctly.

[Phil Evans]

Do you really want to read the whole of a long reflection loop into memory 
rather than parsing it one line at a time (which should be possible once you 
have worked out what is in the file)? That would end up with storing the 
reflection list twice, the memory copy of the input file and the internal 
representation for the program. I do get complaints from people trying to run 
e.g. Pointless with large datasets on 32-bit machines, crashing because it runs 
out of memory

If you imagine someone corresponding to the XDS INTEGRATE.HKL file with 120 
characters/reflection, then a dataset with 10^7 reflections (not outrageously 
large these days) occupies 1.2e9 bytes, over 1GB, which seems a lot to add 
gratuitously to memory demands even on today's computers 

Of course (in my opinion) a working format (as opposed to an archive format) 
should be binary for size, accuracy (FP dynamic range) and speed. 
A quick comparison (using Pointless)

Read 5.3e6 reflections from a formatted XDS INTEGRATE.HKL file, 608MB, 15 secs
Read equivalent binary MTZ file, 262MB, 2.6 secs

Phil

On 18 Sep 2013, at 15:58, yayahjb yaya...@gmail.com wrote:

 Dear Colleagues,
 
  There are two major issues that tend to trip up CIF programmers:
 
   1.  Dealing with the order independence of CIF.  Unlike PDB format, tags in 
 CIF can validly
 be presented in any order.  This means you cannot simply scan a CIF for a tag 
 you want and
 start processing from that point forward as you do with a PDB file.  In 
 general to read
 a CIF properly, you need to read all of it into memory before you can do 
 anything with it.
 A common mistake is to assume that just because many CIFs have been written 
 with tags in
 a given order, the next CIF you encounter will also have the tags in that 
 order.
 
  2.  Doing the lexical scan (the tokenizing) correctly.  CIF uses a context 
 sensitive grammar,
 so lexers based on simple BNF tend to make mistakes, and most reliable CIF 
 lexers are
 hand-written rather than being generated from a grammar.  The advice to use a 
 pre-written
 and tested lexer is sensible.
 
 The bottom line is that, while it is relatively easy to write a valid CIF, 
 reading CIFs reliably
 can be a very challenging programming task, because you need to write code 
 that will handle
 the very complex general case, rather than just specific examples.  
 Fortunately there are
 software packages to help you do this.
 
  Herbert J. Bernstein
 
 On 9/18/13 10:41 AM, Peter Keller wrote:
 Hi Phil,
 
 I agree that the issue that you raise (about the need to define the data 
 items and categories propery) is an important one that needs proper 
 consideration. However, your mail could be read to suggest that correct 
 parsing of CIF-format data is a secondary issue that doesn't deserve the 
 same attention from developers.
 
 I hope that this isn't quite what you meant  There are already 
 mutually-incompatible CIF dialects out there that have been created by 
 developers coding to their own understanding and interpretations of the 
 CIF/STAR format. I am sure that you would not want to be the creator of yet 
 another one :-) Correct tokenising is a necessary (but not sufficient) 
 condition for preventing the problem getting worse.
 
 In practice, the code and applications that I have seen, and the discussions 
 about this that I have had, all suggest that developers find it more 
 difficult to write code that tokenises CIF/STAR-format data correctly than 
 code that handles other text formats that they have to deal with in this 
 field. My experience suggests that this is an important practical issue with 
 real-world ramifications, and it is worthwhile devoting some effort to it.
 
 Regards,
 Peter.
 
 On Wed, 18 Sep 2013, Phil Evans wrote:
 
 Date: Wed, 18 Sep 2013 13:38:07 +0100
 From: Phil Evans p...@mrc-lmb.cam.ac.uk
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] Code to handle the syntax of (mm)CIF data correctly.
 
 As a novice looking at mmCIF from a developers point of view, for 
 reflection data, the complication is not so much tokenising (parsing), but 
 what items to write or to expect to read. For example as far as I can see 
 an observed intensity may be encoded in a reflection loop (merged or 
 unmerged) as any one of the following, and there seem to be similar choices 
 for other items:-
 
 
 _refln_intensity_meas
 _refln.F_squared_meas
 _refln.pdbx_I_plus, _refln.pdbx_I_minus
 
 _diffrn_refln.counts_net
 _diffrn_refln.intensity_net
 
 If I'm writing a file, which should I use, and if I'm reading one which 
 ones should I expect? And is there a distinction between merged and 
 unmerged data?
 
 confused (easily)
 Phil
 
 
 
 On 17 Sep 2013, at 15:30, Peter Keller pkel...@globalphasing.com wrote:
 
 Dear all,
 
 At Global Phasing, we have seen that there are still issues with the way 
 that different applications deal with mmCIF-format data, and this 
 continues to cause problems for users. I believe that part

Re: [ccp4bb] Code to handle the syntax of (mm)CIF data correctly.

[Phil Evans]

There's definitely a case for working formats, but as we think about improved 
working binary formats following on from MTZ (maybe based on HDF5 containers), 
particularly for unmerged data  (for example in the DIALS project), it makes 
sense to use [mm]/[image]/cif definitions where appropriate, for ease of 
conversion and to reduce confusion. However not all data items that are needed 
(or are desirable) are currently defined, so there will need to be some 
extensions. There is a particular problem with multiple lattices (e.g. 
non-merohedral twins) where a single observation (intensity) may have multiple 
indices hkl attached to it, and this doesn't fit neatly into a rectangular 
array without padding.

Phil

On 19 Sep 2013, at 12:11, Eugene Krissinel eugene.krissi...@stfc.ac.uk wrote:

 Completely agree. Perhaps there could be a distinction between working and 
 deposition formats. They are for different purposes and, therefore, could 
 differ from each other -- Eugene
 
 On 19 Sep 2013, at 10:45, Phil Evans wrote:
 
 Do you really want to read the whole of a long reflection loop into memory 
 rather than parsing it one line at a time (which should be possible once you 
 have worked out what is in the file)? That would end up with storing the 
 reflection list twice, the memory copy of the input file and the internal 
 representation for the program. I do get complaints from people trying to 
 run e.g. Pointless with large datasets on 32-bit machines, crashing because 
 it runs out of memory
 
 If you imagine someone corresponding to the XDS INTEGRATE.HKL file with 120 
 characters/reflection, then a dataset with 10^7 reflections (not 
 outrageously large these days) occupies 1.2e9 bytes, over 1GB, which seems a 
 lot to add gratuitously to memory demands even on today's computers 
 
 Of course (in my opinion) a working format (as opposed to an archive format) 
 should be binary for size, accuracy (FP dynamic range) and speed. 
 A quick comparison (using Pointless)
 
 Read 5.3e6 reflections from a formatted XDS INTEGRATE.HKL file, 608MB, 15 
 secs
 Read equivalent binary MTZ file, 262MB, 2.6 secs
 
 Phil
 
 On 18 Sep 2013, at 15:58, yayahjb yaya...@gmail.com wrote:
 
 Dear Colleagues,
 
 There are two major issues that tend to trip up CIF programmers:
 
 1.  Dealing with the order independence of CIF.  Unlike PDB format, tags in 
 CIF can validly
 be presented in any order.  This means you cannot simply scan a CIF for a 
 tag you want and
 start processing from that point forward as you do with a PDB file.  In 
 general to read
 a CIF properly, you need to read all of it into memory before you can do 
 anything with it.
 A common mistake is to assume that just because many CIFs have been written 
 with tags in
 a given order, the next CIF you encounter will also have the tags in that 
 order.
 
 2.  Doing the lexical scan (the tokenizing) correctly.  CIF uses a context 
 sensitive grammar,
 so lexers based on simple BNF tend to make mistakes, and most reliable CIF 
 lexers are
 hand-written rather than being generated from a grammar.  The advice to use 
 a pre-written
 and tested lexer is sensible.
 
 The bottom line is that, while it is relatively easy to write a valid CIF, 
 reading CIFs reliably
 can be a very challenging programming task, because you need to write code 
 that will handle
 the very complex general case, rather than just specific examples.  
 Fortunately there are
 software packages to help you do this.
 
 Herbert J. Bernstein
 
 On 9/18/13 10:41 AM, Peter Keller wrote:
 Hi Phil,
 
 I agree that the issue that you raise (about the need to define the data 
 items and categories propery) is an important one that needs proper 
 consideration. However, your mail could be read to suggest that correct 
 parsing of CIF-format data is a secondary issue that doesn't deserve the 
 same attention from developers.
 
 I hope that this isn't quite what you meant  There are already 
 mutually-incompatible CIF dialects out there that have been created by 
 developers coding to their own understanding and interpretations of the 
 CIF/STAR format. I am sure that you would not want to be the creator of 
 yet another one :-) Correct tokenising is a necessary (but not sufficient) 
 condition for preventing the problem getting worse.
 
 In practice, the code and applications that I have seen, and the 
 discussions about this that I have had, all suggest that developers find 
 it more difficult to write code that tokenises CIF/STAR-format data 
 correctly than code that handles other text formats that they have to deal 
 with in this field. My experience suggests that this is an important 
 practical issue with real-world ramifications, and it is worthwhile 
 devoting some effort to it.
 
 Regards,
 Peter.
 
 On Wed, 18 Sep 2013, Phil Evans wrote:
 
 Date: Wed, 18 Sep 2013 13:38:07 +0100
 From: Phil Evans p...@mrc-lmb.cam.ac.uk
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] Code to handle

Re: [ccp4bb] from sca to CC1/2

[Phil Evans]

As long as the .sca file is scaled but unmerged, then one way is

pointless -copy scain denzo.sca hklout denzo.mtz

aimless hklin denzo.mtz  EOF
scales constant
onlymerge
EOF

Phil

On 18 Sep 2013, at 11:21, Francesco Angelucci francesco.angelu...@uniroma1.it 
wrote:

 Dear All,
 
 I am wondering if is possible to calculate the CC1/2 factor from the denzo 
 .sca file.
 Thank you in advance
 Francesco Angelucci 

Re: [ccp4bb] Code to handle the syntax of (mm)CIF data correctly.

[Phil Evans]

As a novice looking at mmCIF from a developers point of view, for reflection 
data, the complication is not so much tokenising (parsing), but what items to 
write or to expect to read. For example as far as I can see an observed 
intensity may be encoded in a reflection loop (merged or unmerged) as any one 
of the following, and there seem to be similar choices for other items:-

 
_refln_intensity_meas
_refln.F_squared_meas
_refln.pdbx_I_plus, _refln.pdbx_I_minus

_diffrn_refln.counts_net
_diffrn_refln.intensity_net

If I'm writing a file, which should I use, and if I'm reading one which ones 
should I expect? And is there a distinction between merged and unmerged data?

confused (easily)
Phil



On 17 Sep 2013, at 15:30, Peter Keller pkel...@globalphasing.com wrote:

 Dear all,
 
 At Global Phasing, we have seen that there are still issues with the way that 
 different applications deal with mmCIF-format data, and this continues to 
 cause problems for users. I believe that part of the reason for this is that 
 the underlying syntax (the STAR format) is not universally understood, and 
 that a common and complete understanding of the full STAR syntax amongst 
 programmers who deal with the format will help with some of the existing 
 problems.
 
 I wrote some code for low-level handling of the STAR format a while ago that 
 I have been meaning to release for over a year. Garry Battle's announcement 
 on 23 August about the mmCIF/PDBx workshop at the EBI has prompted me into 
 action: I have written a short article that discusses some examples of the 
 issues that we have encountered, and made my code available for download. The 
 references in the article are given primarily as web links: more conventional 
 citations can usually be found in the pages that I link to. This code has not 
 been used in any released products, but it has had some internal use at 
 Global Phasing. There is an MX bias in the article's discussion, but the 
 issues are not restricted to MX.
 
 As I explain in the article, the handling of the input data is based on an 
 enourmous regular expression that matches STAR data, with only a little logic 
 in the code itself. The regular expression should be usable with a variety of 
 other languages, not only in Java (which I have used in this case). The code, 
 or the regular expression on its own, may be freely used in other projects: 
 see the included licencing for details, but basically you should: (i) give 
 credit for using it, and (ii) if you choose to modify the regular expression, 
 state that you have done so in that credit.
 
 The article, which contains links to a tar file containing the code, and the 
 documentation, is here:
 
   http://www.globalphasing.com/startools/
 
 Hoping that others will find this useful and/or help to resolve or clarify 
 outstanding questions,
 
 Peter.
 
 -- 
 Peter Keller Tel.: +44 (0)1223 353033
 Global Phasing Ltd., Fax.: +44 (0)1223 366889
 Sheraton House,
 Castle Park,
 Cambridge CB3 0AX
 United Kingdom

Re: [ccp4bb] Code to handle the syntax of (mm)CIF data correctly.

[Phil Evans]

I didn't mean that, though reflection loop data shouldn't be problematic I 
think (and may be very long). As you say tokenization is necessary but not 
sufficient

I'm sure you have faced lots of corner cases, and naive approaches (that I 
would have gone for) like divide at spaces, then strip quotes if they occur at 
both ends of the token will probably fall over somewhere

Phil


On 18 Sep 2013, at 15:41, Peter Keller pkel...@globalphasing.com wrote:

 Hi Phil,
 
 I agree that the issue that you raise (about the need to define the data 
 items and categories propery) is an important one that needs proper 
 consideration. However, your mail could be read to suggest that correct 
 parsing of CIF-format data is a secondary issue that doesn't deserve the same 
 attention from developers.
 
 I hope that this isn't quite what you meant  There are already 
 mutually-incompatible CIF dialects out there that have been created by 
 developers coding to their own understanding and interpretations of the 
 CIF/STAR format. I am sure that you would not want to be the creator of yet 
 another one :-) Correct tokenising is a necessary (but not sufficient) 
 condition for preventing the problem getting worse.
 
 In practice, the code and applications that I have seen, and the discussions 
 about this that I have had, all suggest that developers find it more 
 difficult to write code that tokenises CIF/STAR-format data correctly than 
 code that handles other text formats that they have to deal with in this 
 field. My experience suggests that this is an important practical issue with 
 real-world ramifications, and it is worthwhile devoting some effort to it.
 
 Regards,
 Peter.
 
 On Wed, 18 Sep 2013, Phil Evans wrote:
 
 Date: Wed, 18 Sep 2013 13:38:07 +0100
 From: Phil Evans p...@mrc-lmb.cam.ac.uk
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] Code to handle the syntax of (mm)CIF data correctly.
 As a novice looking at mmCIF from a developers point of view, for reflection 
 data, the complication is not so much tokenising (parsing), but what items 
 to write or to expect to read. For example as far as I can see an observed 
 intensity may be encoded in a reflection loop (merged or unmerged) as any 
 one of the following, and there seem to be similar choices for other items:-
 
 
 _refln_intensity_meas
 _refln.F_squared_meas
 _refln.pdbx_I_plus, _refln.pdbx_I_minus
 
 _diffrn_refln.counts_net
 _diffrn_refln.intensity_net
 
 If I'm writing a file, which should I use, and if I'm reading one which ones 
 should I expect? And is there a distinction between merged and unmerged data?
 
 confused (easily)
 Phil
 
 
 
 On 17 Sep 2013, at 15:30, Peter Keller pkel...@globalphasing.com wrote:
 
 Dear all,
 
 At Global Phasing, we have seen that there are still issues with the way 
 that different applications deal with mmCIF-format data, and this continues 
 to cause problems for users. I believe that part of the reason for this is 
 that the underlying syntax (the STAR format) is not universally understood, 
 and that a common and complete understanding of the full STAR syntax 
 amongst programmers who deal with the format will help with some of the 
 existing problems.
 
 I wrote some code for low-level handling of the STAR format a while ago 
 that I have been meaning to release for over a year. Garry Battle's 
 announcement on 23 August about the mmCIF/PDBx workshop at the EBI has 
 prompted me into action: I have written a short article that discusses some 
 examples of the issues that we have encountered, and made my code available 
 for download. The references in the article are given primarily as web 
 links: more conventional citations can usually be found in the pages that I 
 link to. This code has not been used in any released products, but it has 
 had some internal use at Global Phasing. There is an MX bias in the 
 article's discussion, but the issues are not restricted to MX.
 
 As I explain in the article, the handling of the input data is based on an 
 enourmous regular expression that matches STAR data, with only a little 
 logic in the code itself. The regular expression should be usable with a 
 variety of other languages, not only in Java (which I have used in this 
 case). The code, or the regular expression on its own, may be freely used 
 in other projects: see the included licencing for details, but basically 
 you should: (i) give credit for using it, and (ii) if you choose to modify 
 the regular expression, state that you have done so in that credit.
 
 The article, which contains links to a tar file containing the code, and 
 the documentation, is here:
 
  http://www.globalphasing.com/startools/
 
 Hoping that others will find this useful and/or help to resolve or clarify 
 outstanding questions,
 
 Peter.
 
 --
 Peter Keller Tel.: +44 (0)1223 353033
 Global Phasing Ltd., Fax.: +44 (0)1223 366889
 Sheraton House,
 Castle Park

Re: [ccp4bb] Resolution, R factors and data quality

[Phil Evans]

We don't currently have a really good measure of that point where adding the 
extra shell of data adds significant information (whatever that means. 
However, my rough trials (see http://www.ncbi.nlm.nih.gov/pubmed/23793146) 
suggested that the exact cutoff point was not very critical, presumably as the 
information content fades out slowly, so it probably isn't something to 
agonise over too much. K  D's paired refinement may be useful though.

I would again caution against looking too hard at CC* rather than CC1/2: they 
are exactly equivalent, but CC* changes very rapidly at small values, which may 
be misleading. The purpose of CC* is for comparison with CCcryst (i.e. Fo to 
Fc).

I would remind any users of Scala who want to look back at old log files to see 
the statistics for the outer shell at the cutoff they used, that CC1/2 has been 
calculated in Scala for many years under the name CC_IMEAN. It's now called 
CC1/2 in Aimless (and Scala) following Kai's excellent suggestion.

Phil


On 28 Aug 2013, at 08:21, Bernhard Rupp hofkristall...@gmail.com wrote:

 Based on the simulations I've done the data should be cut at CC1/2 = 0. 
 Seriously. Problem is figuring out where it hits zero. 
  
 But the real objective is – where do data stop making an improvement to the 
 model. The categorical statement that all data is good
 is simply not true in practice. It is probably specific to each data set  
 refinement, and as long as we do not always run paired refinement ala KD
 or similar in order to find out where that point is, the yearning for a 
 simple number will not stop (although I believe automation will make the KD 
 approach or similar eventually routine).
  
 As for the resolution of the structure I'd say call that where |Fo-Fc| 
 (error in the map) becomes comparable to Sigma(Fo). This is I/Sigma = 2.5 if 
 Rcryst is 20%.  That is: |Fo-Fc| / Fo = 0.2, which implies |Io-Ic|/Io = 0.4 
 or Io/|Io-Ic| = Io/sigma(Io) = 2.5.
  
 Makes sense to me...
  
 As long as it is understood that this ‘model resolution value’ derived via 
 your argument from I/sigI is not the same as a I/sigI data cutoff (and that 
 Rcryst and Rmerge have nothing in common)….
  
 -James Holton
 MAD Scientist
  
 
 Best, BR
 
  
 
  
 
 
 On Aug 27, 2013, at 5:29 PM, Jim Pflugrath jim.pflugr...@rigaku.com wrote:
 
 I have to ask flamingly: So what about CC1/2 and CC*?  
  
 Did we not replace an arbitrary resolution cut-off based on a value of Rmerge 
 with an arbitrary resolution cut-off based on a value of Rmeas already?  And 
 now we are going to replace that with an arbitrary resolution cut-off based 
 on a value of CC* or is it CC1/2?
  
 I am asked often:  What value of CC1/2 should I cut my resolution at?  What 
 should I tell my students?  I've got a course coming up and I am sure they 
 will ask me again.
  
 Jim
  
 From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Arka 
 Chakraborty [arko.chakrabort...@gmail.com]
 Sent: Tuesday, August 27, 2013 7:45 AM
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] Resolution, R factors and data quality
 
 Hi all,
 does this not again bring up the still prevailing adherence to R factors and 
 not  a shift to correlation coefficients ( CC1/2 and CC*) ? (as Dr. Phil 
 Evans has indicated).?
 The way we look at data quality ( by we I mean the end users ) needs to be 
 altered, I guess.
 
 best,
  
 Arka Chakraborty
  
 On Tue, Aug 27, 2013 at 9:50 AM, Phil Evans p...@mrc-lmb.cam.ac.uk wrote:
 The question you should ask yourself is why would omitting data improve my 
 model?
 
 Phil