Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread Roger Rowlett
Without seeing the raw data and watching the integration of frames, I 
would be suspicious of an incorrect space group assignment with an 
Rmerge  0.10. Are image spots well-predicted by the integration program 
(all spots have predictions, all predictions have spots)? Are the spots 
well-resolved (no or few overlaps) at the camera distance used? The lack 
of falloff of I/sig(I) with resolution bin also looks atypical. 
Typically, you would see a larger spread between the overall I/sig(I) 
and the I/sig(I) in the highest resolution bin (assuming you have 
collected data out to the resolution limit of the crystal). And with 
10-fold redundancy, I would expect the overall I/sig(I) to be much 
higher, if your integration program factors redundancy into the I/sig(I) 
calculation.


I see results similar to this doing automated processing of frames in 
real time during collection when the program goes off the rails and 
chooses the wrong space group.


Cheers,

___
Roger S. Rowlett
Gordon  Dorothy Kline Professor
Department of Chemistry
Colgate University
13 Oak Drive
Hamilton, NY 13346

tel: (315)-228-7245
ofc: (315)-228-7395
fax: (315)-228-7935
email: rrowl...@colgate.edu

On 3/29/2013 9:19 AM, hamid khan wrote:


Dear CCP4BB Members,

I am interested in your expert comments/opinions about two values of a 
protein crystal diffraction data. Basically I am new to this field and 
do not have much idea about diffraction data interpretation and 
crystallography software’s use.


1)What could be the possible reasons for a high *Rmerge value, *say 
like *0.185*?


2)Value *6.2* for average *I/sigma(I)* for higher shell means that the 
resolution of the diffraction data is much higher than actually 
measured, what could be the possible reasons for this?


For your ease I would like to past the table here;

Values in parentheses are for the last resolution shell

Space group P2221

Unit-cell parameters (A°)

  a   58.08

  b   101.32

  c   103.47

Molecules in ASU  1

Resolution range38.63 - 2.50  (2.59 - 2.50)

Total number of reflections  228902

Number of unique reflections   21600

Completeness (%) 99.1 (98.0)

*Rmerge 0.185 *   (0.373)

Reduced χ2   0.94 (1.01)

*Average I/σ(I)   9.8  (6.2)*

Thanks for the tips..,


Hamid Khan





Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread Tim Gruene
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Dear Hamid,

the statistics for I/sigI and the R-value per resolution shell would
shed more light than the overall values.

Judging from the Rmerge in the high resolution shell the data may have
been processed by somebody who still thinks Rmerge = 30% is a good
criterium for resolution cut-off.

The high overall Rmerge might indicate a wrong space-group was picked
with too high symmetry.

If you have a copy of the unmerged data, run it through pointless, if
you even have a copy of the frames, reprocess them in P1 and run the
data through pointless!

If these data are from an article you are refereeing please point out
that Rmerge should not be published anymore and be replaced by Rmeas
(alias Rrim)!

Best,
Tim Gruene

On 03/29/2013 02:19 PM, hamid khan wrote:
 Dear CCP4BB Members,
 
 
 
 I am interested in your expert comments/opinions about two values 
 of a protein crystal diffraction data. Basically I am new to this
 field and do not have much idea about diffraction data
 interpretation and crystallography software’s use.
 
 
 
 1) What could be the possible reasons for a high Rmerge value, say
 like 0.185?
 
 
 
 2) Value 6.2 for average I/sigma(I) for higher shell means that
 the resolution of the diffraction data is much higher than actually
 measured, what could be the possible reasons for this?
 
 
 
 For your ease I would like to past the table here;
 
 
 
 Values in parentheses are for the last resolution shell
 
 Space group P2221
 
 Unit-cell parameters (A°)
 
 a58.08
 
 b101.32
 
 c103.47
 
 Molecules in ASU  1
 
 Resolution range 38.63 - 2.50  (2.59 - 2.50)
 
 Total number of reflections 228902
 
 Number of unique reflections 21600
 
 Completeness (%) 99.1 (98.0)
 
 Rmerge0.185
 (0.373)
 
 Reduced χ2 0.94(1.01)
 
 Average I/σ(I) 9.8
 (6.2)
 
 
 
 Thanks for the tips..,
 
 
 Hamid Khan

- -- 
Dr Tim Gruene
Institut fuer anorganische Chemie
Tammannstr. 4
D-37077 Goettingen

GPG Key ID = A46BEE1A
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.12 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iD8DBQFRVZu4UxlJ7aRr7hoRAs0cAJ45ITAQyygvqtC7fYTHTZcLPW7c3ACfUsvs
Ei8BBrcxfxv2OKGqZgtELH8=
=09R3
-END PGP SIGNATURE-


Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread Jim Pflugrath
As mentioned lots of reasons for this.

a. Poor crystal
b. Poor mount of the crystal
c. Poor equipment or non-working equipment
d. Poor maintenance of good equipment
e. Improper cryoprotection
f. Vibration or movement of goniometer, goniometer head, mounting pin, mounting 
loop, magnet, etc
g. Temperature fluctuation of the environment during the data collection
h. Not enough exposure time or poor signal to noise (improper experimental 
design)
i. Improper data processing (too many things to mention here)
j. etc.
k. et al.

Jim


From: CCP4 bulletin board [CCP4BB@JISCMAIL.AC.UK] on behalf of hamid khan 
[hamid...@yahoo.com]
Sent: Friday, March 29, 2013 8:19 AM
To: CCP4BB@JISCMAIL.AC.UK
Subject: [ccp4bb] High Rmerge and I/sigma values?

Dear CCP4BB Members,

I am interested in your expert comments/opinions about two values of a protein 
crystal diffraction data. Basically I am new to this field and do not have much 
idea about diffraction data interpretation and crystallography software’s use.


1)What could be the possible reasons for a high Rmerge value, say like 
0.185?


2)Value 6.2 for average I/sigma(I) for higher shell means that the 
resolution of the diffraction data is much higher than actually measured, what 
could be the possible reasons for this?

For your ease I would like to past the table here;

Values in parentheses are for the last resolution shell
Space group P2221
Unit-cell parameters (A°)
  a58.08
  b101.32
  c103.47
Molecules in ASU  1
Resolution range   38.63 - 2.50  (2.59 - 2.50)
Total number of reflections 228902
Number of unique reflections   21600
Completeness (%) 99.1(98.0)
Rmerge0.185 (0.373)
Reduced χ2   0.94(1.01)
Average I/σ(I)9.8  (6.2)

Thanks for the tips..,

Hamid Khan



Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread James Holton
I must disagree with Tim on the statement Rmerge should not be
published anymore.  That would be a shame.  Perhaps even a crime.

When Uli Arndt introduced Rmerge he was in no way, shape or form
proposing that it be used for resolution cutoffs, or anything else
about the quality of the structure.  He was, however, trying to
define a good statistic to evaluate a diffractometer system, and
Rmerge is still VERY useful for that!

Any halfway decent modern detector/shutter/beam system should be able
to measure reasonably strong spots to within 5% of their true
intensity.  Note that this is the _overall_ Rmerge value.  The Rmerge
divided up in resolution bins is pretty useless for this, especially
the outermost bin, where you are basically dividing by zero.  The only
useful Rmerge bin is actually the lowest-angle one, where the spots
tend to all be strong.  Remember, Rmerge is defined as the _sum_ of
all the variations in spot intensity divided by the _sum_ of all the
intensity.  This should never be much more than 5% for strong spots.
If it is, then something is wrong with either your detector, or your
shutter, or perhaps your assumptions about symmetry.

Yes, I know multiplicity makes Rmerge higher, but in actual fact
multiplicity makes Rmerge more honest.  It is better to say that low
multiplicity makes your Rmerge appear too low.  Basically, if you
actually do have RMS 5% error per spot, and you only measure each hkl
twice, then you expect to see Rmerge=2.8%, even though the actual
error is 5%.  And of course, if you measure 1e6 photons in one spot
you might fool yourself into thinking the error is only 0.1%.  Its
not.  On the other hand, if all your spots are weak, then you do
expect the variation to be dominated by photon-counting error, and you
will get Rmerge values much greater than 5% on a perfectly good
detector.  It is only at high multiplicities with strong spots that
Rmerge truly shows you how bad your equipment is.  This is why its
always good to check Rmerge in your lowest-angle bin.

Yes, I know we probably all take our local well-maintained and
finely-tuned beamline for granted, but that does not mean we should
stop using the only statistic that tells us something might be wrong
with the machine we used to measure our data.  That is definitely
worth the ~20 extra bytes it takes up in your paper.

-James Holton
MAD Scientist

On Fri, Mar 29, 2013 at 6:48 AM, Tim Gruene t...@shelx.uni-ac.gwdg.de wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Dear Hamid,

 the statistics for I/sigI and the R-value per resolution shell would
 shed more light than the overall values.

 Judging from the Rmerge in the high resolution shell the data may have
 been processed by somebody who still thinks Rmerge = 30% is a good
 criterium for resolution cut-off.

 The high overall Rmerge might indicate a wrong space-group was picked
 with too high symmetry.

 If you have a copy of the unmerged data, run it through pointless, if
 you even have a copy of the frames, reprocess them in P1 and run the
 data through pointless!

 If these data are from an article you are refereeing please point out
 that Rmerge should not be published anymore and be replaced by Rmeas
 (alias Rrim)!

 Best,
 Tim Gruene

 On 03/29/2013 02:19 PM, hamid khan wrote:
 Dear CCP4BB Members,



 I am interested in your expert comments/opinions about two values
 of a protein crystal diffraction data. Basically I am new to this
 field and do not have much idea about diffraction data
 interpretation and crystallography software’s use.



 1) What could be the possible reasons for a high Rmerge value, say
 like 0.185?



 2) Value 6.2 for average I/sigma(I) for higher shell means that
 the resolution of the diffraction data is much higher than actually
 measured, what could be the possible reasons for this?



 For your ease I would like to past the table here;



 Values in parentheses are for the last resolution shell

 Space group P2221

 Unit-cell parameters (A°)

 a58.08

 b101.32

 c103.47

 Molecules in ASU  1

 Resolution range 38.63 - 2.50  (2.59 - 2.50)

 Total number of reflections 228902

 Number of unique reflections 21600

 Completeness (%) 99.1 (98.0)

 Rmerge0.185
 (0.373)

 Reduced χ2 0.94(1.01)

 Average I/σ(I) 9.8
 (6.2)



 Thanks for the tips..,


 Hamid Khan

 - --
 Dr Tim Gruene
 Institut fuer anorganische Chemie
 Tammannstr. 4
 D-37077 Goettingen

 GPG Key ID = A46BEE1A
 -BEGIN PGP SIGNATURE-
 Version: GnuPG v1.4.12 (GNU/Linux)
 Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

 iD8DBQFRVZu4UxlJ7aRr7hoRAs0cAJ45ITAQyygvqtC7fYTHTZcLPW7c3ACfUsvs
 Ei8BBrcxfxv2OKGqZgtELH8=
 =09R3
 -END PGP 

Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread Tim Gruene
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi James,

you misquote me: I was saying that Rmeas should be replacing Rmerge,
and I guess everything you say holds for Rmeas, too, but still it is a
better statistical quantity than Rmerge. So please replace Rmerge with
Rmeas.

Best,
Tim

On 03/29/2013 06:08 PM, James Holton wrote:
 I must disagree with Tim on the statement Rmerge should not be 
 published anymore.  That would be a shame.  Perhaps even a crime.
 
 When Uli Arndt introduced Rmerge he was in no way, shape or form 
 proposing that it be used for resolution cutoffs, or anything else 
 about the quality of the structure.  He was, however, trying to 
 define a good statistic to evaluate a diffractometer system, and 
 Rmerge is still VERY useful for that!
 
 Any halfway decent modern detector/shutter/beam system should be
 able to measure reasonably strong spots to within 5% of their
 true intensity.  Note that this is the _overall_ Rmerge value.
 The Rmerge divided up in resolution bins is pretty useless for
 this, especially the outermost bin, where you are basically
 dividing by zero.  The only useful Rmerge bin is actually the
 lowest-angle one, where the spots tend to all be strong.
 Remember, Rmerge is defined as the _sum_ of all the variations in
 spot intensity divided by the _sum_ of all the intensity.  This
 should never be much more than 5% for strong spots. If it is, then
 something is wrong with either your detector, or your shutter, or
 perhaps your assumptions about symmetry.
 
 Yes, I know multiplicity makes Rmerge higher, but in actual fact 
 multiplicity makes Rmerge more honest.  It is better to say that
 low multiplicity makes your Rmerge appear too low.  Basically, if
 you actually do have RMS 5% error per spot, and you only measure
 each hkl twice, then you expect to see Rmerge=2.8%, even though the
 actual error is 5%.  And of course, if you measure 1e6 photons in
 one spot you might fool yourself into thinking the error is only
 0.1%.  Its not.  On the other hand, if all your spots are weak,
 then you do expect the variation to be dominated by photon-counting
 error, and you will get Rmerge values much greater than 5% on a
 perfectly good detector.  It is only at high multiplicities with
 strong spots that Rmerge truly shows you how bad your equipment is.
 This is why its always good to check Rmerge in your lowest-angle
 bin.
 
 Yes, I know we probably all take our local well-maintained and 
 finely-tuned beamline for granted, but that does not mean we
 should stop using the only statistic that tells us something might
 be wrong with the machine we used to measure our data.  That is
 definitely worth the ~20 extra bytes it takes up in your paper.
 
 -James Holton MAD Scientist
 
 On Fri, Mar 29, 2013 at 6:48 AM, Tim Gruene
 t...@shelx.uni-ac.gwdg.de wrote: Dear Hamid,
 
 the statistics for I/sigI and the R-value per resolution shell
 would shed more light than the overall values.
 
 Judging from the Rmerge in the high resolution shell the data may
 have been processed by somebody who still thinks Rmerge = 30% is a
 good criterium for resolution cut-off.
 
 The high overall Rmerge might indicate a wrong space-group was
 picked with too high symmetry.
 
 If you have a copy of the unmerged data, run it through pointless,
 if you even have a copy of the frames, reprocess them in P1 and run
 the data through pointless!
 
 If these data are from an article you are refereeing please point
 out that Rmerge should not be published anymore and be replaced by
 Rmeas (alias Rrim)!
 
 Best, Tim Gruene
 
 On 03/29/2013 02:19 PM, hamid khan wrote:
 Dear CCP4BB Members,
 
 
 
 I am interested in your expert comments/opinions about two
 values of a protein crystal diffraction data. Basically I am
 new to this field and do not have much idea about diffraction
 data interpretation and crystallography software’s use.
 
 
 
 1) What could be the possible reasons for a high Rmerge
 value, say like 0.185?
 
 
 
 2) Value 6.2 for average I/sigma(I) for higher shell means
 that the resolution of the diffraction data is much higher
 than actually measured, what could be the possible reasons
 for this?
 
 
 
 For your ease I would like to past the table here;
 
 
 
 Values in parentheses are for the last resolution shell
 
 Space group P2221
 
 Unit-cell parameters (A°)
 
 a58.08
 
 b101.32
 
 c103.47
 
 Molecules in ASU  1
 
 Resolution range 38.63 - 2.50  (2.59 - 2.50)
 
 Total number of reflections 228902
 
 Number of unique reflections 21600
 
 Completeness (%) 99.1 (98.0)
 
 Rmerge0.185 
 (0.373)
 
 Reduced χ2 0.94(1.01)
 
 Average I/σ(I) 9.8 (6.2)
 
 
 
 Thanks for the tips..,
 
 
 Hamid 

Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread James Holton
Ahh.  But what I'm saying is that Rmeas is not a replacement for
Rmerge because Rmeas is _always_ lower than Rmerge.  It is even less
useful that a low-multiplicity Rmerge for evaluating the
diffractometer.

I fully realize that Rmeas does have the desirable property of being
flatter with respect to multiplicity, but being equally too low
for all multiplicity is not better than being too low for some
multiplicities.  IMHO.  Yes, I know, we all like R statistics that are
lower.  Indeed, by using the mean absolute deviation |I-I|, Uli was
able to come up with a definition of Rmerge that will always be lower
than the RMS error (for infinite multiplicity and RMS 5% error you
actually get Rmerge=3.99%).  No doubt, this must have contributed to
the  acceptance of Rmerge at the time.  But we can't just keep
re-defining our metric of error every ~20 years so that the same
crappy data keeps looking better and better.  That's a slippery slope
I'd rather not be on.  I think it is important to remember what it is
we are trying to measure, and to be honest and consistent about what
the error bars really are.

But that's just my opinion.  I could be wrong.

-James Holton
MAD Scientist

On Fri, Mar 29, 2013 at 10:28 AM, Tim Gruene t...@shelx.uni-ac.gwdg.de wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1

 Hi James,

 you misquote me: I was saying that Rmeas should be replacing Rmerge,
 and I guess everything you say holds for Rmeas, too, but still it is a
 better statistical quantity than Rmerge. So please replace Rmerge with
 Rmeas.

 Best,
 Tim

 On 03/29/2013 06:08 PM, James Holton wrote:
 I must disagree with Tim on the statement Rmerge should not be
 published anymore.  That would be a shame.  Perhaps even a crime.

 When Uli Arndt introduced Rmerge he was in no way, shape or form
 proposing that it be used for resolution cutoffs, or anything else
 about the quality of the structure.  He was, however, trying to
 define a good statistic to evaluate a diffractometer system, and
 Rmerge is still VERY useful for that!

 Any halfway decent modern detector/shutter/beam system should be
 able to measure reasonably strong spots to within 5% of their
 true intensity.  Note that this is the _overall_ Rmerge value.
 The Rmerge divided up in resolution bins is pretty useless for
 this, especially the outermost bin, where you are basically
 dividing by zero.  The only useful Rmerge bin is actually the
 lowest-angle one, where the spots tend to all be strong.
 Remember, Rmerge is defined as the _sum_ of all the variations in
 spot intensity divided by the _sum_ of all the intensity.  This
 should never be much more than 5% for strong spots. If it is, then
 something is wrong with either your detector, or your shutter, or
 perhaps your assumptions about symmetry.

 Yes, I know multiplicity makes Rmerge higher, but in actual fact
 multiplicity makes Rmerge more honest.  It is better to say that
 low multiplicity makes your Rmerge appear too low.  Basically, if
 you actually do have RMS 5% error per spot, and you only measure
 each hkl twice, then you expect to see Rmerge=2.8%, even though the
 actual error is 5%.  And of course, if you measure 1e6 photons in
 one spot you might fool yourself into thinking the error is only
 0.1%.  Its not.  On the other hand, if all your spots are weak,
 then you do expect the variation to be dominated by photon-counting
 error, and you will get Rmerge values much greater than 5% on a
 perfectly good detector.  It is only at high multiplicities with
 strong spots that Rmerge truly shows you how bad your equipment is.
 This is why its always good to check Rmerge in your lowest-angle
 bin.

 Yes, I know we probably all take our local well-maintained and
 finely-tuned beamline for granted, but that does not mean we
 should stop using the only statistic that tells us something might
 be wrong with the machine we used to measure our data.  That is
 definitely worth the ~20 extra bytes it takes up in your paper.

 -James Holton MAD Scientist

 On Fri, Mar 29, 2013 at 6:48 AM, Tim Gruene
 t...@shelx.uni-ac.gwdg.de wrote: Dear Hamid,

 the statistics for I/sigI and the R-value per resolution shell
 would shed more light than the overall values.

 Judging from the Rmerge in the high resolution shell the data may
 have been processed by somebody who still thinks Rmerge = 30% is a
 good criterium for resolution cut-off.

 The high overall Rmerge might indicate a wrong space-group was
 picked with too high symmetry.

 If you have a copy of the unmerged data, run it through pointless,
 if you even have a copy of the frames, reprocess them in P1 and run
 the data through pointless!

 If these data are from an article you are refereeing please point
 out that Rmerge should not be published anymore and be replaced by
 Rmeas (alias Rrim)!

 Best, Tim Gruene

 On 03/29/2013 02:19 PM, hamid khan wrote:
 Dear CCP4BB Members,



 I am interested in your expert comments/opinions about two
 values of a protein 

Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread Felix Frolow
To support  James Holton, MAD Scientist
 From the very far past:
In 60's we have estimated X-ray diffraction data by eyes, using strips of 
intensities made with propagating exposure factor increase of 2 (I guess in F 
it corresponds to sqrt(2)~=1.4).
We have used to estimate our Weissenberg diffraction data 3 times (we used  
X-ray films). After 1 time, logbooks were locked in the deposit safe box by our 
advisers. The second round of estimation we recorded in a different logbook, 
and it was again locked in the safe deposit box. After 3 time we have punched 
(binary code) our data on cards, translated to ASCII, checked, corrected, and 
finally run averaging programs. Our best data were of about 30% Rmerge. 
However, as for the structure refinement (and maybe some flavours of structure 
determination) absolute error in a measurement of a single set of symmetry 
related reflections (precision) is much less important than relative error in 
measuring of other sets (accuracy), we were able to refine anisotropically 
small molecule structures to R=5%, because our data were not precise, but 
accurate. BTW some erroneous 'discoveries'  in metalo-organic complexes with 
non-centrosymmetric space groups were made, based on our inability to measure 
anomalous signal (it is in the single set of symmetry related reflections), 
differences were averaged and absolute structure information was lost 
introducing artificial asymmetry of the coordination sphere. About 20 year 
later in H. D. Flack (1983). On Enantiomorph-Polarity Estimation, Acta Cryst 
A39: 876–881, all become clearly explained.
To conclude :
 I am not impressed by very low Rmerge. Once ALL 
reflections were overexposed on XENTRONIX (which was the area detector with not 
very good dynamic range), as a result of running after  low Rmerge without 
understanding procedures and instruments to measure diffraction,  and Rmerge of 
1% was seen, but data were useless. 
 I am not depressed by diffraction data of very high 
(but correct) Rmerge of 13% with which a structure of important for us protein 
complex was swiftly  solved by SAD (450 residues, 5 Se).  The crystals were 
small; even 10 sec exposure produced relatively weak (but accurate) data. And 
long live a bending magnet that almost never burn down cryo-maintained crystals!
All depends on circumstances 

My 2 cents...


Dr Felix Frolow   
Professor of Structural Biology and Biotechnology, 
Department of Molecular Microbiology and Biotechnology
Tel Aviv University 69978, Israel

Acta Crystallographica F, co-editor

e-mail: mbfro...@post.tau.ac.il
Tel:  ++972-3640-8723
Fax: ++972-3640-9407
Cellular: 0547 459 608

On Mar 29, 2013, at 20:52 , James Holton jmhol...@lbl.gov wrote:

 Ahh.  But what I'm saying is that Rmeas is not a replacement for
 Rmerge because Rmeas is _always_ lower than Rmerge.  It is even less
 useful that a low-multiplicity Rmerge for evaluating the
 diffractometer.
 
 I fully realize that Rmeas does have the desirable property of being
 flatter with respect to multiplicity, but being equally too low
 for all multiplicity is not better than being too low for some
 multiplicities.  IMHO.  Yes, I know, we all like R statistics that are
 lower.  Indeed, by using the mean absolute deviation |I-I|, Uli was
 able to come up with a definition of Rmerge that will always be lower
 than the RMS error (for infinite multiplicity and RMS 5% error you
 actually get Rmerge=3.99%).  No doubt, this must have contributed to
 the  acceptance of Rmerge at the time.  But we can't just keep
 re-defining our metric of error every ~20 years so that the same
 crappy data keeps looking better and better.  That's a slippery slope
 I'd rather not be on.  I think it is important to remember what it is
 we are trying to measure, and to be honest and consistent about what
 the error bars really are.
 
 But that's just my opinion.  I could be wrong.
 
 -James Holton
 MAD Scientist
 
 On Fri, Mar 29, 2013 at 10:28 AM, Tim Gruene t...@shelx.uni-ac.gwdg.de 
 wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA1
 
 Hi James,
 
 you misquote me: I was saying that Rmeas should be replacing Rmerge,
 and I guess everything you say holds for Rmeas, too, but still it is a
 better statistical quantity than Rmerge. So please replace Rmerge with
 Rmeas.
 
 Best,
 Tim
 
 On 03/29/2013 06:08 PM, James Holton wrote:
 I must disagree with Tim on the statement Rmerge should not be
 published anymore.  That would be a shame.  Perhaps even a crime.
 
 When Uli Arndt introduced Rmerge he was in no way, shape or form
 proposing that it be used for resolution cutoffs, or anything else
 about the quality of the structure.  He was, however, trying to
 define a good statistic to evaluate a diffractometer system, and
 Rmerge is still VERY useful for that!
 
 Any halfway decent modern detector/shutter/beam system should be
 able to measure reasonably strong spots to within 5% of their
 true 

Re: [ccp4bb] High Rmerge and I/sigma values....?

2013-03-29 Thread James Holton
Woops!  Sorry.  I was thinking Rpim, which is always lower than Rmerge.  
Rmeas is always higher, and more correctly estimates the 
infinite-multiplicity Rmerge.


Sorry for the confusion, and thanks for the many reminders I just got 
about the definition!


-James Holton
MAD Scientist

On 3/29/2013 10:28 AM, Tim Gruene wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi James,

you misquote me: I was saying that Rmeas should be replacing Rmerge,
and I guess everything you say holds for Rmeas, too, but still it is a
better statistical quantity than Rmerge. So please replace Rmerge with
Rmeas.

Best,
Tim

On 03/29/2013 06:08 PM, James Holton wrote:

I must disagree with Tim on the statement Rmerge should not be
published anymore.  That would be a shame.  Perhaps even a crime.

When Uli Arndt introduced Rmerge he was in no way, shape or form
proposing that it be used for resolution cutoffs, or anything else
about the quality of the structure.  He was, however, trying to
define a good statistic to evaluate a diffractometer system, and
Rmerge is still VERY useful for that!

Any halfway decent modern detector/shutter/beam system should be
able to measure reasonably strong spots to within 5% of their
true intensity.  Note that this is the _overall_ Rmerge value.
The Rmerge divided up in resolution bins is pretty useless for
this, especially the outermost bin, where you are basically
dividing by zero.  The only useful Rmerge bin is actually the
lowest-angle one, where the spots tend to all be strong.
Remember, Rmerge is defined as the _sum_ of all the variations in
spot intensity divided by the _sum_ of all the intensity.  This
should never be much more than 5% for strong spots. If it is, then
something is wrong with either your detector, or your shutter, or
perhaps your assumptions about symmetry.

Yes, I know multiplicity makes Rmerge higher, but in actual fact
multiplicity makes Rmerge more honest.  It is better to say that
low multiplicity makes your Rmerge appear too low.  Basically, if
you actually do have RMS 5% error per spot, and you only measure
each hkl twice, then you expect to see Rmerge=2.8%, even though the
actual error is 5%.  And of course, if you measure 1e6 photons in
one spot you might fool yourself into thinking the error is only
0.1%.  Its not.  On the other hand, if all your spots are weak,
then you do expect the variation to be dominated by photon-counting
error, and you will get Rmerge values much greater than 5% on a
perfectly good detector.  It is only at high multiplicities with
strong spots that Rmerge truly shows you how bad your equipment is.
This is why its always good to check Rmerge in your lowest-angle
bin.

Yes, I know we probably all take our local well-maintained and
finely-tuned beamline for granted, but that does not mean we
should stop using the only statistic that tells us something might
be wrong with the machine we used to measure our data.  That is
definitely worth the ~20 extra bytes it takes up in your paper.

-James Holton MAD Scientist

On Fri, Mar 29, 2013 at 6:48 AM, Tim Gruene
t...@shelx.uni-ac.gwdg.de wrote: Dear Hamid,

the statistics for I/sigI and the R-value per resolution shell
would shed more light than the overall values.

Judging from the Rmerge in the high resolution shell the data may
have been processed by somebody who still thinks Rmerge = 30% is a
good criterium for resolution cut-off.

The high overall Rmerge might indicate a wrong space-group was
picked with too high symmetry.

If you have a copy of the unmerged data, run it through pointless,
if you even have a copy of the frames, reprocess them in P1 and run
the data through pointless!

If these data are from an article you are refereeing please point
out that Rmerge should not be published anymore and be replaced by
Rmeas (alias Rrim)!

Best, Tim Gruene

On 03/29/2013 02:19 PM, hamid khan wrote:

Dear CCP4BB Members,



I am interested in your expert comments/opinions about two
values of a protein crystal diffraction data. Basically I am
new to this field and do not have much idea about diffraction
data interpretation and crystallography software’s use.



1) What could be the possible reasons for a high Rmerge
value, say like 0.185?



2) Value 6.2 for average I/sigma(I) for higher shell means
that the resolution of the diffraction data is much higher
than actually measured, what could be the possible reasons
for this?



For your ease I would like to past the table here;



Values in parentheses are for the last resolution shell

Space group P2221

Unit-cell parameters (A°)

a58.08

b101.32

c103.47

Molecules in ASU  1

Resolution range 38.63 - 2.50  (2.59 - 2.50)

Total number of reflections 228902

Number of unique reflections 21600

Completeness (%) 99.1 (98.0)