You are right: I suppose I was thinking of arbitrary real-space transformations (eg an origin shift in P1) and also the reduction to the asymmetric unit., which are straightforward.
I haven't coded the phase changes into Pointless mainly out of laziness & thinking that other things were more important to spend my time on (also I've never wanted to do this myself), and I haven't thought it a particularly useful option. Phil On 1 Nov 2010, at 15:39, Ian Tickle wrote: > Dealing with the phases (and therefore also the Hendrickson-Lattman > coefficients) on re-indexing is trivial: the phases are not changed by > re-indexing because the inverse transformation must simultaneously be > applied to the co-ordinates. This is because in general the > re-indexing transformation is not necessarily a symmetry operator > (think of P1), so you can't rely on being able to compensate for the > effect on the co-ordinates by using a symmetry operator. So the > effect on the phases cancels out ... unless of course your > re-indexing operator inverts the hand, in which case you almost > certainly don't want also to invert the hand of your co-ordinates, so > in that case you must compensate by transforming the structure factors > to their complex conjugates (i.e. multiply phases by -1). I guess > you're thinking of the subsequent necessary transformation of the > indices to the asymmetric unit, where the phases & H-L coeffs do in > general change (because then you are only changing the indices, not > the co-ordinates); however CAD will do that transformation for you. > > Incidentally this is a neat illustration of the difference between a > vector and a complex number. The re-indexing transformation is a > transformation of the reference frame, which as long as it doesn't > invert the hand, leaves the complex structure factors invariant, so > they must be complex scalars (except in centric zones where they can > sometimes be represented by real scalars). The indices (whether > reflection or Miller!) obviously form a 3-D vector with integer > elements (unless of course you're interested in diffuse scattering > when they have to be reals). Either way, this is a vector because in > the general case (there will be exceptions for reflections on symmetry > axes) its elements change on re-indexing (that's what re-indexing > means!). If the structure were in 1-D or 2-D exactly the same would > apply: the 1- or 2-D elements would still in general change on > transforming the reference frame so would be represented by 1- or 2-D > vectors; the structure factors would still be invariant, thus > illustrating the important difference between a real scalar and a 1-D > vector, and between a complex scalar and a 2-D vector. > > Cheers > > --Ian > > On Mon, Nov 1, 2010 at 1:28 PM, Phil Evans <[email protected]> wrote: >> I can see we need to make sure that data can come in at any point, as Is of >> Fs >> >> Pointless can do automatic reindexing to a reference, and will preserve all >> columns from a merged file, but can't cope with phases, as I've not got >> round to working out appropriate phase shifts on reindexing >> >> Phil >> >> >> On 1 Nov 2010, at 13:17, Ian Tickle wrote: >> >>> Phil >>> >>> Yes our processing pipeline absolutely has to be able to take in data >>> from any internal (in-house X-ray or synchrotron) or external (PDB or >>> collaborator's) source, including those where I, sigI and freeR flag >>> are present. One of the first things I did was to modify truncate so >>> it would pass through the freeR flag column. If the I/sigI are >>> present I always strip out the F/sigF columns. So it seemed logical >>> to run truncate as the very last step, e.g.: >>> >>> 1. sortmtz >>> 2. scala Steps 1 & 2 only for internally collected or unmerged data. >>> 3. refindex External merged data enters pipeline here: >>> auto-re-index to reference. >>> 4. cad Sort; put into standard a.u.; add freeR column from >>> reference if not already present. >>> 5. rescut My own prog for auto-determination of resolution cutoff >>> based on shell <I/sigI> & completeness. >>> 6. truncate Apply resolution cutoff; if Is available convert to Fs. >>> >>> I always run steps 3-6 in that order. I always check that the >>> resolution cutoff is sensible & if Is are available I always run >>> truncate to ensure it's done properly (i.e. correct cell contents are >>> specified). I'm still using truncate because AFAICS ctruncate >>> couldn't handle freeR flags (maybe that's fixed now, maybe not). Also >>> truncate produces a more informative N(Z) plot which shows the >>> expected distribution for a twinned crystal (I believe this feature >>> has now been added to ctruncate). >>> >>> Cheers >>> >>> -- Ian >>> >>> On Fri, Oct 29, 2010 at 1:05 PM, Phil Evans <[email protected]> wrote: >>>> The normal use of [c]truncate is to take intensities from Scala, so it >>>> wouldn't expect FreeR flags in the file. >>>> >>>> I suppose this should be added for other uses of the program >>>> >>>> Is this something that is often used? Do people import intensities into >>>> CCP4 to convert them to Fs? >>>> >>>> Phil >>>> >>>> >>>> On 29 Oct 2010, at 13:01, [email protected] wrote: >>>> >>>>> Dear Peter, >>>>> >>>>> Since I did not hear that your problem is solved here my two cents. I >>>>> did some tests using the ccp4i option "Convert Intensities to SFs" and >>>>> found that here ctruncate completely ignored the FreeRflags. So my >>>>> conclusion is that ctruncate does not need FreeRflags and you can use >>>>> the following procedure: >>>>> >>>>> 1) convert your hkl file (including FreeRflags) into an mtz with f2mtz >>>>> without any special SHELX options. --> mtz 1 >>>>> Careful: a FreeRflag of 1 means an unfree reflection and the free >>>>> reflections have a FreeRflag of zero. >>>>> 2) run ctruncate with the "Convert Intensities to SFs". You will loose >>>>> your FreeRflags in this stage. --> mtz 2 >>>>> 3) add the FreeRflags from mtz 1 to mtz 2 using cad. >>>>> >>>>> If you wish, I can give you a command file which will do this. It is a >>>>> somewhat roundabout procedure and I hope that this bug (or feature) will >>>>> be fixed by the next release of ccp4. >>>>> >>>>> Best, >>>>> Herman >>>>> >>>>> -----Original Message----- >>>>> From: CCP4 bulletin board [mailto:[email protected]] On Behalf Of >>>>> George M. Sheldrick >>>>> Sent: Friday, October 29, 2010 12:30 PM >>>>> To: [email protected] >>>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>>> >>>>> Tim, >>>>> >>>>> Although I always like to advocate XPREP, that would not work because >>>>> the .sca format - most unfortunately - does not know about free R flags. >>>>> >>>>> George >>>>> >>>>> Prof. George M. Sheldrick FRS >>>>> Dept. Structural Chemistry, >>>>> University of Goettingen, >>>>> Tammannstr. 4, >>>>> D37077 Goettingen, Germany >>>>> Tel. +49-551-39-3021 or -3068 >>>>> Fax. +49-551-39-22582 >>>>> >>>>> >>>>> On Fri, 29 Oct 2010, Tim Gruene wrote: >>>>> >>>>>> Hello Peter, >>>>>> >>>>>> the easiest way to overcome the problem might be to use xprep to >>>>>> export to sca-format and use scalepack2mtz for the conversion. That >>>>>> seems to be the least hasslesome way, although I am not totally sure >>>>>> that this procedure preserves the R-free flags set by xprep. >>>>>> >>>>>> Tim >>>>>> >>>>>> On Thu, Oct 28, 2010 at 12:48:14PM -0400, Peter Chan wrote: >>>>>>> >>>>>>> Hello Tim, >>>>>>> >>>>>>> Thank you for the suggestion. I have now tagged the working set as >>>>> "1" and test set as "0". Unfortunately, it still gives the same error >>>>> about all Rfree being the same, and only in c-truncate but not >>>>> old-truncate. Perhaps I should install 6.1.3 and see if the problem >>>>> still persist. >>>>>>> >>>>>>> Best, >>>>>>> Peter >>>>>>> >>>>>>>> Date: Thu, 28 Oct 2010 16:29:31 +0200 >>>>>>>> From: [email protected] >>>>>>>> Subject: Re: [ccp4bb] Bug in c_truncate? >>>>>>>> To: [email protected] >>>>>>>> >>>>>>>> Hello Peter, >>>>>>>> >>>>>>>> I faintly rememeber a similar kind of problem, and think that if >>>>>>>> you replace "-1" with "0", the problem should go away. It seemed >>>>>>>> that "-1" is not an allowed flag for (some) ccp4 programs. >>>>>>>> >>>>>>>> Please let us know if this resolves the issue. >>>>>>>> >>>>>>>> Tim >>>>>>>> >>>>>>>> On Thu, Oct 28, 2010 at 10:21:20AM -0400, Peter Chan wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Dear Crystallographers, >>>>>>>>> >>>>>>>>> Thank you all for the emails. Below are some details of the >>>>> procedures I performed leading up to the problem. >>>>>>>>> >>>>>>>>> The reflection file is my own data, processed in XDS and then >>>>> flagging FreeR's in XPREP in thin resolution shells. I am using CCP4i >>>>> version 6.1.2. I tried looking for known/resolved issues/updates in >>>>> version 6.1.3 but could not find any so I assumed it is the same version >>>>> of f2mtz/ctruncate/uniqueify. >>>>>>>>> >>>>>>>>> >>>>>>>>> I used the GUI version of F2MTZ, with the settings below: >>>>>>>>> >>>>>>>>> - import file in SHELX format >>>>>>>>> >>>>>>>>> - "keep existing FreeR flags" >>>>>>>>> >>>>>>>>> - fortran format (3F4.0,2F8.3,F4.0) >>>>>>>>> >>>>>>>>> - added data label "I other integer" // FreeRflag >>>>>>>>> >>>>>>>>> The hkl file, in SHELX format, output by XPREP look something >>>>> like this: >>>>>>>>> >>>>>>>>> -26 -3 1 777.48 39.19 >>>>>>>>> 26 -3 -1 800.83 36.31 >>>>>>>>> -26 3 -1 782.67 37.97 >>>>>>>>> 27 -3 1 45.722 25.711 -1 >>>>>>>>> -27 3 1 -14.20 31.69 -1 >>>>>>>>> >>>>>>>>> Notice the test set is flagged "-1" and the working set is not >>>>> flagged at all. This actually lead to another error message in f2mtz >>>>> about missing FreeR flags. From my understanding, the SHELX flagging >>>>> convention is "1" for working and "-1" for test. So I manually tagged >>>>> the working set with "1" using vi: >>>>>>>>> >>>>>>>>> -26 -3 1 777.48 39.19 1 >>>>>>>>> 26 -3 -1 800.83 36.31 1 >>>>>>>>> -26 3 -1 782.67 37.97 1 >>>>>>>>> 27 -3 1 45.722 25.711 -1 >>>>>>>>> -27 3 1 -14.20 31.69 -1 >>>>>>>>> >>>>>>>>> This is the file which gives me the error message: "Problem with >>>>> FREE column in input file. All flags apparently identical. Check input >>>>> file.". Apparently, import to mtz works ok when I use old-truncate >>>>> instead of c-truncate. >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> Peter >>>>>>>>> >>>>>>>> -- >>>>>>>> -- >>>>>>>> Tim Gruene >>>>>>>> Institut fuer anorganische Chemie >>>>>>>> Tammannstr. 4 >>>>>>>> D-37077 Goettingen >>>>>>>> >>>>>>>> phone: +49 (0)551 39 22149 >>>>>>>> >>>>>>>> GPG Key ID = A46BEE1A >>>>>>>> >>>>>>> >>>>>> -- >>>>>> -- >>>>>> Tim Gruene >>>>>> Institut fuer anorganische Chemie >>>>>> Tammannstr. 4 >>>>>> D-37077 Goettingen >>>>>> >>>>>> phone: +49 (0)551 39 22149 >>>>>> >>>>>> GPG Key ID = A46BEE1A >>>>>> >>>>>> >>>> >> >>
