Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-15 Thread Edward A. Berry

Great! and (oops) now I realize what I meant to say was compare cc* with 
cc-work and cc-free.
eab

Nat Echols wrote:

On Wed, Aug 14, 2013 at 10:31 PM, Edward A. Berry ber...@upstate.edu 
mailto:ber...@upstate.edu wrote:

If you refine once in phenix you can use phenix.cc_star to calculate cc* 
and compare with R and R-free; from the
output mtz file and your unmerged .sca file.


FYI, this should also work with structures refined in Refmac, assuming it can 
recalculate the R-factors to within a
reasonable margin of error.

-Nat


Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-15 Thread Frank von Delft
 lies in the fact that you used HKL2000.  Scalepack has a 
long standing feature where it reports Rmerge 
100% as zero.  Quite why they do that is a mystery, but your Rmerge 
in the outermost shell is NOT zero - the Rmerge
for the lower resolution shells will show up as non-zero if Rmerge  
100%.


That feature is overdue for a fix.

Alternatively export your scaled data with NO MERGE ORIGINAL INDEX 
and import into CCP4 via Pointless and have Scala
or Aimless report the correct statistics.  Reprocessing the data 
using XDS or Mosflm will ultimately lead you to
scaling the data with a program that doesn't have that bug. If you 
do this, report Rmeas rather than Rmerge, the

former being a better measure.

Phil Jeffrey
Princeton


 

*From:* CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of 
Yafang Chen [yafangche...@gmail.com]

*Sent:* Wednesday, August 14, 2013 11:32 AM
*To:* CCP4BB@JISCMAIL.AC.UK
*Subject:* Re: [ccp4bb] Rmerge of the last shell is zero

Dear All,

Here are some more details about the question I asked earlier about 
Rmerge is 0 in the last shell. I processed the
data using HKL2000. The space group is I213. Redundancy is 10.2 
(10.3). I/sigma is 34.8 (2.3). Rmerge is 6.5 (0.0).
Since I/sigmaI is more than 2 in the last shell, I preferred not to 
cut back the resolution any more. But I don't know
how to explain Rmerge in the last shell being 0. Besides, I am 
wondering if this data is publishable (with Rmerge

being 0 in the last shell). Thank you so much for your help!

Best,
Yafang


On Wed, Aug 14, 2013 at 10:59 AM, Yafang Chen 
yafangche...@gmail.com mailto:yafangche...@gmail.com wrote:


Dear All,

I recently processed a dataset, in which I/sigmaI of the last 
shell is 2.3, while Rmerge of the last shell is 0.
Does anyone know why the Rmerge is 0? The completeness is 100 
(100). Thank you so much for your help in advance!


Best,
Yafang

--
Yafang Chen
Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907




--
Yafang Chen
Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907








[ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Yafang Chen
Dear All,

I recently processed a dataset, in which I/sigmaI of the last shell is 2.3,
while Rmerge of the last shell is 0. Does anyone know why the Rmerge is 0?
The completeness is 100 (100). Thank you so much for your help in advance!

Best,
Yafang

-- 
Yafang Chen

Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907


Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Tim Gruene
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Dear Yafang,

a mathematically simple answer would be you collected no symmetry
equivalent reflections, although experimentally this is probably
difficult to achieve, unless your space group is P1. If you could tell
the name of the software you used, the respective developers might
know if '0' indicates something specific.

Regards,
Tim

On 08/14/2013 04:59 PM, Yafang Chen wrote:
 Dear All,
 
 I recently processed a dataset, in which I/sigmaI of the last shell
 is 2.3, while Rmerge of the last shell is 0. Does anyone know why
 the Rmerge is 0? The completeness is 100 (100). Thank you so much
 for your help in advance!
 
 Best, Yafang
 

- -- 
- --
Dr Tim Gruene
Institut fuer anorganische Chemie
Tammannstr. 4
D-37077 Goettingen

GPG Key ID = A46BEE1A

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFSC6C5UxlJ7aRr7hoRAvgiAJ9uGurdV1F1NbwMShnGBQforDPnGgCg7IZ7
6lz3MWS/qAsmAsX6Wf8f4Tg=
=l/px
-END PGP SIGNATURE-


Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Phil Evans
multiplicity = 1.0?

On 14 Aug 2013, at 15:59, Yafang Chen yafangche...@gmail.com wrote:

 Dear All,
 
 I recently processed a dataset, in which I/sigmaI of the last shell is 2.3, 
 while Rmerge of the last shell is 0. Does anyone know why the Rmerge is 0? 
 The completeness is 100 (100). Thank you so much for your help in advance!
 
 Best,
 Yafang
 
 -- 
 Yafang Chen
  
 Graduate Research Assistant
 Mesecar Lab
 Department of Biological Sciences
 Purdue University
 Hockmeyer Hall of Structural Biology
 240 S. Martin Jischke Drive
 West Lafayette, IN 47907


Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Yafang Chen
Dear All,

Here are some more details about the question I asked earlier about Rmerge
is 0 in the last shell. I processed the data using HKL2000. The space
group is I213. Redundancy is 10.2 (10.3). I/sigma is 34.8 (2.3). Rmerge is
6.5 (0.0). Since I/sigmaI is more than 2 in the last shell, I preferred not
to cut back the resolution any more. But I don't know how to explain Rmerge
in the last shell being 0. Besides, I am wondering if this data is
publishable (with Rmerge being 0 in the last shell). Thank you so much for
your help!

Best,
Yafang


On Wed, Aug 14, 2013 at 10:59 AM, Yafang Chen yafangche...@gmail.comwrote:

 Dear All,

 I recently processed a dataset, in which I/sigmaI of the last shell is
 2.3, while Rmerge of the last shell is 0. Does anyone know why the Rmerge
 is 0? The completeness is 100 (100). Thank you so much for your help in
 advance!

 Best,
 Yafang

 --
 Yafang Chen

 Graduate Research Assistant
 Mesecar Lab
 Department of Biological Sciences
 Purdue University
 Hockmeyer Hall of Structural Biology
 240 S. Martin Jischke Drive
 West Lafayette, IN 47907




-- 
Yafang Chen

Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907


Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Jeffrey, Philip D.
Hello Yafang,

The answer lies in the fact that you used HKL2000.  Scalepack has a long 
standing feature where it reports Rmerge  100% as zero.  Quite why they do 
that is a mystery, but your Rmerge in the outermost shell is NOT zero - the 
Rmerge for the lower resolution shells will show up as non-zero if Rmerge  
100%.

That feature is overdue for a fix.

Alternatively export your scaled data with NO MERGE ORIGINAL INDEX and import 
into CCP4 via Pointless and have Scala or Aimless report the correct 
statistics.  Reprocessing the data using XDS or Mosflm will ultimately lead you 
to scaling the data with a program that doesn't have that bug.  If you do this, 
report Rmeas rather than Rmerge, the former being a better measure.

Phil Jeffrey
Princeton



From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Yafang Chen 
[yafangche...@gmail.com]
Sent: Wednesday, August 14, 2013 11:32 AM
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] Rmerge of the last shell is zero

Dear All,

Here are some more details about the question I asked earlier about Rmerge is 
0 in the last shell. I processed the data using HKL2000. The space group is 
I213. Redundancy is 10.2 (10.3). I/sigma is 34.8 (2.3). Rmerge is 6.5 (0.0). 
Since I/sigmaI is more than 2 in the last shell, I preferred not to cut back 
the resolution any more. But I don't know how to explain Rmerge in the last 
shell being 0. Besides, I am wondering if this data is publishable (with Rmerge 
being 0 in the last shell). Thank you so much for your help!

Best,
Yafang


On Wed, Aug 14, 2013 at 10:59 AM, Yafang Chen 
yafangche...@gmail.commailto:yafangche...@gmail.com wrote:
Dear All,

I recently processed a dataset, in which I/sigmaI of the last shell is 2.3, 
while Rmerge of the last shell is 0. Does anyone know why the Rmerge is 0? The 
completeness is 100 (100). Thank you so much for your help in advance!

Best,
Yafang

--
Yafang Chen

Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907



--
Yafang Chen

Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907


Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Pietro Roversi
HKL2000 sets to 0 the Rmerge in any shell where it is higher than 100%

OUCH

Sent from my Desktop

Dr. Pietro Roversi
Oxford University Biochemistry Department - Glycobiology Division
South Parks Road
Oxford OX1 3QU England - UK
Tel. 0044 1865 275339

From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Yafang Chen 
[yafangche...@gmail.com]
Sent: 14 August 2013 16:32
To: CCP4BB@JISCMAIL.AC.UK
Subject: Re: [ccp4bb] Rmerge of the last shell is zero

Dear All,

Here are some more details about the question I asked earlier about Rmerge is 
0 in the last shell. I processed the data using HKL2000. The space group is 
I213. Redundancy is 10.2 (10.3). I/sigma is 34.8 (2.3). Rmerge is 6.5 (0.0). 
Since I/sigmaI is more than 2 in the last shell, I preferred not to cut back 
the resolution any more. But I don't know how to explain Rmerge in the last 
shell being 0. Besides, I am wondering if this data is publishable (with Rmerge 
being 0 in the last shell). Thank you so much for your help!

Best,
Yafang


On Wed, Aug 14, 2013 at 10:59 AM, Yafang Chen 
yafangche...@gmail.commailto:yafangche...@gmail.com wrote:
Dear All,

I recently processed a dataset, in which I/sigmaI of the last shell is 2.3, 
while Rmerge of the last shell is 0. Does anyone know why the Rmerge is 0? The 
completeness is 100 (100). Thank you so much for your help in advance!

Best,
Yafang

--
Yafang Chen

Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907



--
Yafang Chen

Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907


Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread James Holton
No matter what the value printed out for Rmerge in the outer shell is, I
recommend using - or n/a in your paper.  This is because
sum(|I-I0|)/sum(I) actually is equal to n/a when sum(I) = 0.

-James Holton
MAD Scientist



On Wed, Aug 14, 2013 at 8:41 AM, Pietro Roversi 
pietro.rove...@bioch.ox.ac.uk wrote:

 HKL2000 sets to 0 the Rmerge in any shell where it is higher than 100%

 OUCH

 Sent from my Desktop

 Dr. Pietro Roversi
 Oxford University Biochemistry Department - Glycobiology Division
 South Parks Road
 Oxford OX1 3QU England - UK
 Tel. 0044 1865 275339
 
 From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Yafang
 Chen [yafangche...@gmail.com]
 Sent: 14 August 2013 16:32
 To: CCP4BB@JISCMAIL.AC.UK
 Subject: Re: [ccp4bb] Rmerge of the last shell is zero

 Dear All,

 Here are some more details about the question I asked earlier about
 Rmerge is 0 in the last shell. I processed the data using HKL2000. The
 space group is I213. Redundancy is 10.2 (10.3). I/sigma is 34.8 (2.3).
 Rmerge is 6.5 (0.0). Since I/sigmaI is more than 2 in the last shell, I
 preferred not to cut back the resolution any more. But I don't know how to
 explain Rmerge in the last shell being 0. Besides, I am wondering if this
 data is publishable (with Rmerge being 0 in the last shell). Thank you so
 much for your help!

 Best,
 Yafang


 On Wed, Aug 14, 2013 at 10:59 AM, Yafang Chen yafangche...@gmail.com
 mailto:yafangche...@gmail.com wrote:
 Dear All,

 I recently processed a dataset, in which I/sigmaI of the last shell is
 2.3, while Rmerge of the last shell is 0. Does anyone know why the Rmerge
 is 0? The completeness is 100 (100). Thank you so much for your help in
 advance!

 Best,
 Yafang

 --
 Yafang Chen

 Graduate Research Assistant
 Mesecar Lab
 Department of Biological Sciences
 Purdue University
 Hockmeyer Hall of Structural Biology
 240 S. Martin Jischke Drive
 West Lafayette, IN 47907



 --
 Yafang Chen

 Graduate Research Assistant
 Mesecar Lab
 Department of Biological Sciences
 Purdue University
 Hockmeyer Hall of Structural Biology
 240 S. Martin Jischke Drive
 West Lafayette, IN 47907



Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Frank von Delft
This HKL2000 (scalepack) feature is actually extremely sensible:  an 
Rmerge that high is mathematically meaningless, it quite literally tells 
you nothing at all about he signal in your data.


So I second James's advice:  just put n/a in your table 1.

If the reviewer complains, point them to Karplus  Diederichs, Science, 
2012, and Evans and Murshudov, ActaD, 2013, and tell them to join us in 
the 21st century.





On 14/08/2013 16:41, Jeffrey, Philip D. wrote:

Hello Yafang,

The answer lies in the fact that you used HKL2000.  Scalepack has a 
long standing feature where it reports Rmerge  100% as zero.  Quite 
why they do that is a mystery, but your Rmerge in the outermost shell 
is NOT zero - the Rmerge for the lower resolution shells will show up 
as non-zero if Rmerge  100%.


That feature is overdue for a fix.

Alternatively export your scaled data with NO MERGE ORIGINAL INDEX and 
import into CCP4 via Pointless and have Scala or Aimless report the 
correct statistics.  Reprocessing the data using XDS or Mosflm will 
ultimately lead you to scaling the data with a program that doesn't 
have that bug.  If you do this, report Rmeas rather than Rmerge, the 
former being a better measure.


Phil Jeffrey
Princeton



*From:* CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of 
Yafang Chen [yafangche...@gmail.com]

*Sent:* Wednesday, August 14, 2013 11:32 AM
*To:* CCP4BB@JISCMAIL.AC.UK
*Subject:* Re: [ccp4bb] Rmerge of the last shell is zero

Dear All,

Here are some more details about the question I asked earlier about 
Rmerge is 0 in the last shell. I processed the data using HKL2000. 
The space group is I213. Redundancy is 10.2 (10.3). I/sigma is 34.8 
(2.3). Rmerge is 6.5 (0.0). Since I/sigmaI is more than 2 in the last 
shell, I preferred not to cut back the resolution any more. But I 
don't know how to explain Rmerge in the last shell being 0. Besides, I 
am wondering if this data is publishable (with Rmerge being 0 in the 
last shell). Thank you so much for your help!


Best,
Yafang


On Wed, Aug 14, 2013 at 10:59 AM, Yafang Chen yafangche...@gmail.com 
mailto:yafangche...@gmail.com wrote:


Dear All,

I recently processed a dataset, in which I/sigmaI of the last
shell is 2.3, while Rmerge of the last shell is 0. Does anyone
know why the Rmerge is 0? The completeness is 100 (100). Thank you
so much for your help in advance!

Best,
Yafang

-- 
Yafang Chen

Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907




--
Yafang Chen
Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907




Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Edward A. Berry
But it is highly unlikely that sum(I) in the denominator is zero if I/sig(I) is 2 as reported (providing the sig(I) is 
valid- what was chi^2 in the last shell and overall?).


I sort of disagree that R-merge values over 1.0 are meaningless, provided not too far over. Granted R-meas is more 
meaningful, but with high redundancy R-merge approximates R-meas, and R-merge is what the PDB is accepting now.  A value 
a little over 1 tells you the standard deviation of the individual mesurements is a little larger than the average 
signal in those measurements. Understanding a little about distribution of intensities and error propogation (standard 
error of the mean, R-pim) the user will understand that quite a few reflections were stronger than this standard 
deviation, and that the error in the averaged intensities is be less than this standard deviation, and not have a 
problem. The problem comes with the 100%-sum mentality which says that if your error is 100% the signal must be zero%. 
That is why I don't like to express R-whatever as a percent. If the signal was really zero, R-merge or R-meas would be 
plus/minus infinity. So much for 100%-sum.


So rerun scalepack with no merge original index, reprocess through CCp4 or just rune Diedrich's rmerge program or 
phenix.merging_statistics to get the value of r-merge (and r-meas and R-pim and cc1/2) from the .sca file. If it is much 
over 3 than I would rexamine the I/sig(I) value (which can be misestimated) and consider discarding the last shell. If 
it is 3 or less, report it. The PDB ADDIT2 application used to not accept values over .99, but you can put that and ask 
the friendly annotator to correct it in the final PDB file. If the annotator objects, THEN point (him) to the 2012 KD 
paper- R-meas in the last shell there was over 4, and they concluded there was useful information.


Of course by then you will have refined your structure, and unbiased R-free in the last shell can be a good indicator. 
If you refine once in phenix you can use phenix.cc_star to calculate cc* and compare with R and R-free; from the output 
mtz file and your unmerged .sca file.

eab

Frank von Delft wrote:

This HKL2000 (scalepack) feature is actually extremely sensible:  an Rmerge 
that high is mathematically meaningless, it
quite literally tells you nothing at all about he signal in your data.

So I second James's advice:  just put n/a in your table 1.

If the reviewer complains, point them to Karplus  Diederichs, Science, 2012, 
and Evans and Murshudov, ActaD, 2013, and
tell them to join us in the 21st century.




On 14/08/2013 16:41, Jeffrey, Philip D. wrote:

Hello Yafang,

The answer lies in the fact that you used HKL2000.  Scalepack has a long standing 
feature where it reports Rmerge 
100% as zero.  Quite why they do that is a mystery, but your Rmerge in the 
outermost shell is NOT zero - the Rmerge
for the lower resolution shells will show up as non-zero if Rmerge  100%.

That feature is overdue for a fix.

Alternatively export your scaled data with NO MERGE ORIGINAL INDEX and import 
into CCP4 via Pointless and have Scala
or Aimless report the correct statistics.  Reprocessing the data using XDS or 
Mosflm will ultimately lead you to
scaling the data with a program that doesn't have that bug.  If you do this, 
report Rmeas rather than Rmerge, the
former being a better measure.

Phil Jeffrey
Princeton



*From:* CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of Yafang Chen 
[yafangche...@gmail.com]
*Sent:* Wednesday, August 14, 2013 11:32 AM
*To:* CCP4BB@JISCMAIL.AC.UK
*Subject:* Re: [ccp4bb] Rmerge of the last shell is zero

Dear All,

Here are some more details about the question I asked earlier about Rmerge is 0 in 
the last shell. I processed the
data using HKL2000. The space group is I213. Redundancy is 10.2 (10.3). I/sigma 
is 34.8 (2.3). Rmerge is 6.5 (0.0).
Since I/sigmaI is more than 2 in the last shell, I preferred not to cut back 
the resolution any more. But I don't know
how to explain Rmerge in the last shell being 0. Besides, I am wondering if 
this data is publishable (with Rmerge
being 0 in the last shell). Thank you so much for your help!

Best,
Yafang


On Wed, Aug 14, 2013 at 10:59 AM, Yafang Chen yafangche...@gmail.com 
mailto:yafangche...@gmail.com wrote:

Dear All,

I recently processed a dataset, in which I/sigmaI of the last shell is 2.3, 
while Rmerge of the last shell is 0.
Does anyone know why the Rmerge is 0? The completeness is 100 (100). Thank 
you so much for your help in advance!

Best,
Yafang

--
Yafang Chen
Graduate Research Assistant
Mesecar Lab
Department of Biological Sciences
Purdue University
Hockmeyer Hall of Structural Biology
240 S. Martin Jischke Drive
West Lafayette, IN 47907




--
Yafang Chen
Graduate Research Assistant
Mesecar Lab
Department

Re: [ccp4bb] Rmerge of the last shell is zero

2013-08-14 Thread Nat Echols
On Wed, Aug 14, 2013 at 10:31 PM, Edward A. Berry ber...@upstate.eduwrote:

 If you refine once in phenix you can use phenix.cc_star to calculate cc*
 and compare with R and R-free; from the output mtz file and your unmerged
 .sca file.


FYI, this should also work with structures refined in Refmac, assuming it
can recalculate the R-factors to within a reasonable margin of error.

-Nat