Phil Yes our processing pipeline absolutely has to be able to take in data from any internal (in-house X-ray or synchrotron) or external (PDB or collaborator's) source, including those where I, sigI and freeR flag are present. One of the first things I did was to modify truncate so it would pass through the freeR flag column. If the I/sigI are present I always strip out the F/sigF columns. So it seemed logical to run truncate as the very last step, e.g.:
1. sortmtz 2. scala Steps 1 & 2 only for internally collected or unmerged data. 3. refindex External merged data enters pipeline here: auto-re-index to reference. 4. cad Sort; put into standard a.u.; add freeR column from reference if not already present. 5. rescut My own prog for auto-determination of resolution cutoff based on shell <I/sigI> & completeness. 6. truncate Apply resolution cutoff; if Is available convert to Fs. I always run steps 3-6 in that order. I always check that the resolution cutoff is sensible & if Is are available I always run truncate to ensure it's done properly (i.e. correct cell contents are specified). I'm still using truncate because AFAICS ctruncate couldn't handle freeR flags (maybe that's fixed now, maybe not). Also truncate produces a more informative N(Z) plot which shows the expected distribution for a twinned crystal (I believe this feature has now been added to ctruncate). Cheers -- Ian On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans <p...@mrc-lmb.cam.ac.uk> wrote: > The normal use of [c]truncate is to take intensities from Scala, so it > wouldn't expect FreeR flags in the file. > > I suppose this should be added for other uses of the program > > Is this something that is often used? Do people import intensities into CCP4 > to convert them to Fs? > > Phil > > > On 29 Oct 2010, at 13:01, herman.schreu...@sanofi-aventis.com wrote: > >> Dear Peter, >> >> Since I did not hear that your problem is solved here my two cents. I >> did some tests using the ccp4i option "Convert Intensities to SFs" and >> found that here ctruncate completely ignored the FreeRflags. So my >> conclusion is that ctruncate does not need FreeRflags and you can use >> the following procedure: >> >> 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz >> without any special SHELX options. --> mtz 1 >> Careful: a FreeRflag of 1 means an unfree reflection and the free >> reflections have a FreeRflag of zero. >> 2) run ctruncate with the "Convert Intensities to SFs". You will loose >> your FreeRflags in this stage. --> mtz 2 >> 3) add the FreeRflags from mtz 1 to mtz 2 using cad. >> >> If you wish, I can give you a command file which will do this. It is a >> somewhat roundabout procedure and I hope that this bug (or feature) will >> be fixed by the next release of ccp4. >> >> Best, >> Herman >> >> -----Original Message----- >> From: CCP4 bulletin board [mailto:ccp...@jiscmail.ac.uk] On Behalf Of >> George M. Sheldrick >> Sent: Friday, October 29, 2010 12:30 PM >> To: CCP4BB@JISCMAIL.AC.UK >> Subject: Re: [ccp4bb] Bug in c_truncate? >> >> Tim, >> >> Although I always like to advocate XPREP, that would not work because >> the .sca format - most unfortunately - does not know about free R flags. >> >> George >> >> Prof. George M. Sheldrick FRS >> Dept. Structural Chemistry, >> University of Goettingen, >> Tammannstr. 4, >> D37077 Goettingen, Germany >> Tel. +49-551-39-3021 or -3068 >> Fax. +49-551-39-22582 >> >> >> On Fri, 29 Oct 2010, Tim Gruene wrote: >> >>> Hello Peter, >>> >>> the easiest way to overcome the problem might be to use xprep to >>> export to sca-format and use scalepack2mtz for the conversion. That >>> seems to be the least hasslesome way, although I am not totally sure >>> that this procedure preserves the R-free flags set by xprep. >>> >>> Tim >>> >>> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: >>>> >>>> Hello Tim, >>>> >>>> Thank you for the suggestion. I have now tagged the working set as >> "1" and test set as "0". Unfortunately, it still gives the same error >> about all Rfree being the same, and only in c-truncate but not >> old-truncate. Perhaps I should install 6.1.3 and see if the problem >> still persist. >>>> >>>> Best, >>>> Peter >>>> >>>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 >>>>> From: t...@shelx.uni-ac.gwdg.de >>>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>>> To: CCP4BB@JISCMAIL.AC.UK >>>>> >>>>> Hello Peter, >>>>> >>>>> I faintly rememeber a similar kind of problem, and think that if >>>>> you replace "-1" with "0", the problem should go away. It seemed >>>>> that "-1" is not an allowed flag for (some) ccp4 programs. >>>>> >>>>> Please let us know if this resolves the issue. >>>>> >>>>> Tim >>>>> >>>>> On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Dear Crystallographers, >>>>>> >>>>>> Thank you all for the emails. Below are some details of the >> procedures I performed leading up to the problem. >>>>>> >>>>>> The reflection file is my own data, processed in XDS and then >> flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i >> version 6.1.2. I tried looking for known/resolved issues/updates in >> version 6.1.3 but could not find any so I assumed it is the same version >> of f2mtz/ctruncate/uniqueify. >>>>>> >>>>>> >>>>>> I used the GUI version of F2MTZ, with the settings below: >>>>>> >>>>>> - import file in SHELX format >>>>>> >>>>>> - "keep existing FreeR flags" >>>>>> >>>>>> - fortran format (3F4.0,2F8.3,F4.0) >>>>>> >>>>>> - added data label "I other integer" // FreeRflag >>>>>> >>>>>> The hkl file, in SHELX format, output by XPREP look something >> like this: >>>>>> >>>>>> -26 -3 1 777.48 39.19 >>>>>> 26 -3 -1 800.83 36.31 >>>>>> -26 3 -1 782.67 37.97 >>>>>> 27 -3 1 45.722 25.711 -1 >>>>>> -27 3 1 -14.20 31.69 -1 >>>>>> >>>>>> Notice the test set is flagged "-1" and the working set is not >> flagged at all. This actually lead to another error message in f2mtz >> about missing FreeR flags. From my understanding, the SHELX flagging >> convention is "1" for working and "-1" for test. So I manually tagged >> the working set with "1" using vi: >>>>>> >>>>>> -26 -3 1 777.48 39.19 1 >>>>>> 26 -3 -1 800.83 36.31 1 >>>>>> -26 3 -1 782.67 37.97 1 >>>>>> 27 -3 1 45.722 25.711 -1 >>>>>> -27 3 1 -14.20 31.69 -1 >>>>>> >>>>>> This is the file which gives me the error message: "Problem with >> FREE column in input file. All flags apparently identical. Check input >> file.". Apparently, import to mtz works ok when I use old-truncate >> instead of c-truncate. >>>>>> >>>>>> Best, >>>>>> Peter >>>>>> >>>>> -- >>>>> -- >>>>> Tim Gruene >>>>> Institut fuer anorganische Chemie >>>>> Tammannstr. 4 >>>>> D-37077 Goettingen >>>>> >>>>> phone: +49 (0)551 39 22149 >>>>> >>>>> GPG Key ID = A46BEE1A >>>>> >>>> >>> -- >>> -- >>> Tim Gruene >>> Institut fuer anorganische Chemie >>> Tammannstr. 4 >>> D-37077 Goettingen >>> >>> phone: +49 (0)551 39 22149 >>> >>> GPG Key ID = A46BEE1A >>> >>> >
