Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-18 Thread Kay Diederichs
Dear Graeme, some points for further clarification- The CORRECT corrections you mention all depend on the geometric description of the experiment. This geometric description of the experiment is refined by CORRECT, to come up with accurate values for a) application of polarization correction

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-17 Thread Graeme Winter
Dear Nukri, The following is my opinion which I think is worth discussion, and are based on my understanding of what XDS does in the CORRECT step. Firstly, I tend to find the global refinement in the CORRECT step useful for getting a good unit cell recycling the orientation matrix etc. for

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-17 Thread Phil Evans
Actually Pointless knows that the INTEGRATE file is corrected for an unpolarised beam and recorrects for a synchrotron unless the wavelength is one of the home source ones. See docs. You can specify explicitly I think Phil Sent from my iPhone On 17 Nov 2014, at 09:44, Graeme Winter

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-17 Thread Graeme Winter
HI Phil, Yep: I was not clear perhaps but I did say this: Pointless will also do this but assumes unless given a correct value that the beam is quite polarized. Mostly: care needs to be taken, particularly if using a wavelength which may be confused with a lab source... For others: if you do

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-14 Thread Sanishvili, Ruslan
Dear Graeme, Could you elaborate on There are also some subtleties to making (b) work properly... some more? I have a feeling, from observing the beamline users, that many choose to use this option. It would be very helpful for them to know what are those subtleties and how to best make it

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-13 Thread Graeme Winter
Dear Kay Just to comment on (e) since you say you don't know why anyone would want to do this, yet this is exactly what xia2 -3d does :o) I use AIMLESS to merge data already scaled by XDS CORRECT or XSCALE as a way to get a report on the merging statistics which includes all of the AIMLESS

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-13 Thread Kay Diederichs
Dear Graeme, good that you set this straight. I consider getting the statistics output from AIMLESS is a perfectly valid reason for going e), and as long as this is well-tested (which I'd bet in case of xia2) it's ok. There is one issue I can see: 99% (obviously my guess could be wrong; just

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-13 Thread Graeme Winter
Dear Kay I cannot comment on the accuracy or otherwise of your 99%, but every time I talk about xia2 or write down what the options do, I try to make it clear that XDS / XSCALE is used for integration scaling then AIMLESS to merge the data. I have had an interest for a while in scaling the data

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-12 Thread Tim Gruene
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear Wolfram Tempel, there might be some confusion about terms. It is correct that xscale scales several data sets together. However, in crystallography, 'merging' might be the better term for this process. Crystallographic 'Scaling' is far more

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-12 Thread Kay Diederichs
Hi Tim, this is incorrect. XSCALE determines the relative scale and B in a first step (this is what you describe). It then, in a second step, re-determines all scale factors (exactly as CORRECT does for the individual data sets), at the exact same supporting points that CORRECT used. (This

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-12 Thread Kay Diederichs
On Wed, 12 Nov 2014 15:32:04 +, Kay Diederichs kay.diederi...@uni-konstanz.de wrote: ... It then, in a second step, re-determines all scale factors (exactly as CORRECT does for the individual data sets), at the exact same supporting points that CORRECT used. (This avoids over-fitting which

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-12 Thread Tim Gruene
Hi Kay, thank you for the clarification. I had understood that using XSCALE after CORRECT does no harm, but did not understand that the reason lies in the consistent choice of support points rather than not repeating what might already having been done. Regards, Tim On 11/12/2014 04:32 PM, Kay

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-12 Thread wtempel
Hello Kay, you said the o-word, and you are familiar with the inner workings of XDS. Has the data-to-parameter ratio in even complex scaling models become so small that a doubling (worst case) of model parameters would be a serious concern? Could one detect such overfitting by, say, comparing

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-12 Thread Kay Diederichs
Hi Wolfram, it took me a while until I realized that you mean overfitting when you said o-word. You can abuse XDS in a number of ways, and I would call them overfitting the data although that would be using the word in a somewhat strained way: reducing WFAC1 below 1, decreasing

[ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-11 Thread wtempel
Hello all, in a discussion https://www.jiscmail.ac.uk/cgi-bin/webadmin?A2=ind1307L=CCP4BBH=1P=186901 on this board, Kay Diederichs questioned the effect of scaling data in AIMLESS after prior scaling in XDS (CORRECT). I understand that the available alternatives in this work flow are to specify

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-11 Thread Boaz Shaanan
Hi, I actually choose the option 'constant' further down in the aimless gui but I guess the effect is similar to 'onlymege'. Boaz Boaz Shaanan, Ph.D. Dept. of Life Sciences Ben-Gurion University of the Negev Beer-Sheva 84105 Israel E-mail:

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-11 Thread wtempel
Thank you Boaz. So if CORRECT can do a fully corrected scaling, are there no corrections that XSCALE might apply to XDS_ASCII.HKL data that are beyond CORRECT's capabilities? Wolfram On Tue, Nov 11, 2014 at 3:05 PM, Boaz Shaanan bshaa...@bgu.ac.il wrote: Hi, I actually choose the option

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-11 Thread Sudhir Babu Pothineni
http://strucbio.biologie.uni-konstanz.de/xdswiki/index.php/Xscale XSCALE http://www.mpimf-heidelberg.mpg.de/%7Ekabsch/xds/html_doc/xscale_parameters.html is the scaling program of the XDS suite. It scales reflection files (typically called XDS_ASCII.HKL) produced by XDS. Since the CORRECT

Re: [ccp4bb] To scale or not to scale: XDS_ASCII.HKL input to POINTLESS/AIMLESS

2014-11-11 Thread Phil Evans
You can take XDS data into Pointless Aimless (the CCP4 Data Reduction task) either from the unscaled INTEGRATE.HKL or the scaled XDS_ASCII.HKL file (or files). In the case of a single XDS_ASCII.HKL you don't need to rescale it in Aimless, though you can if you want. Aimless uses a similar but