Re: [ccp4bb] phenix.refine and refmac

[Pavel Afonine]

Hi Yang Li,

0) Cool down -:)

1) To see if the refinement is converged in phenix.refine is easy: just 
look how R-factors change from macro-cycle to macro-cycle (this is 
reported in PDB file header and in .log file). If they don't change 
much, then it is "converged". If they still going down, then simply 
increase the number of macro-cycles, say from default 3 to 5 or 10 (the 
keyword is "main.number_of_macro_cycles=5").


2) I would also suggest to run your final refinement with 
"optimize_wxc=true". This will optimize the relative Xray/Geometry 
weight. Since it is slow, just run it overnight.


3) Note that phenix.refine does not perform TLS refinement if you run it 
with all default parameters, so if you want to do TLS refinement then 
you need to use appropriate keyword to request it. (I emphasize this 
just because you mentioned that you ran phenix.refine using default 
settings but as at the same time you said that you included TLS).


4) Also make sure that you use "ordered_solvent=true" so the ordered 
solvent gets updated and refined automatically as the refinement progresses.


Cheers,
Pavel.



On 3/4/2008 6:35 PM, yang li wrote:
Thanks for your replies. I did use default settings in phenix 
refine, included TLS in it. This is an about 2.3A data, and the number 
Rfree used to refine is big enough. I also kept the same Rfree in two 
programs. In refmac I also tried diffenrent weighting sets--from 
defaut 0.3 to 0.02, TLS included--maybe there are some other ways to 
define the restrains. The R and Rfree will increase as the weighting 
is set too low, and the gap
didnot improve much. I donnot know in this case if the phenix refine 
has converged, but the not stable Rfree in refmac made me nervous. Simply,
in this case, which one should I choose? Certainly to my private 
opinion I would prefer to Scott's answer, Rfree is the most obvious 
parameter than something esle like geometry, eg. Just looks good. :)



On 3/5/08, *Partha Chakrabarti* <[EMAIL PROTECTED] 
> wrote:


One point which I don't understand is how can someone compare the two
different programs when  they don't use the same numbers for
xray:geometry terms? Taking the default settings for a given
resolution might not be enough.. !




On Tue, Mar 4, 2008 at 7:58 PM, Savvas Savvides
<[EMAIL PROTECTED] > wrote:
> Hi Yang
>  how many reflections do you have in your test-set for calculating
>  R-free? Too few reflections, typically less than 500, may not
>  constitute a statistically robust cross-validation data set,
and thus may
>  lead to fluctuations in R-free plus a tendency for R-free to
increase
>  as a function of refinement cycle. Some of the early
>  publications from Axel Brunger on crystallographic cross-validation
>  address the need for enough reflections (>500) in the test-set. In
>  addition, the presence of even a handful of strong but inaccurately
>  measured/integrated low-resolution reflections in a limited
test-set
>  can aggravate abnormal behavior in R-free.
>
>  Best wishes
>  Savvas
>
>
>  toQuoting yang li <[EMAIL PROTECTED]
>:
>
>  > Dear All,
>  >   I have post a similar question about CNS and refmac
before, now in
>  > another structure I met a similar problem. I have an almost
finished
>  > structure, the Rfree of which
>  > is about 0.28 by refmac. Then I used phenix to refine it,
below is the
>  > result:
>  > REMARK  REFINEMENT SUMMARY: QUICK FACTS
>  > ***
>  > REMARK Start: r_work = 0.1970 r_free = 0.2892 bonds = 0.006
angles = 1.213
>  > REMARK Final: r_work = 0.1917 r_free = 0.2617 bonds = 0.008
angles = 1.374
>  > REMARK
>  >

>  > Since the map from phenix couldnot be opened by coot
directly--or I
>  > donnot know how to--I used refmac to get a mtz map file. But
I found that at
>  > the first several cycle of
>  > refmac the Rfree decreased, then both the R and Rfree
values  continued
>  > increasing and FOM decreasing.
>  >  The best R/Rfree/FOM during the refinement is
>  >
>  >

-
>  > Overall R factor = 0.1932
>  > Free R factor= 0.2513
>  > Overall figure of merit  = 0.8168
>  >

-
>  >  And after 40 cycles the final result is:
>  >

-
>  > Overall R factor = 0.2008
>  > Free R factor= 

Re: [ccp4bb] phenix.refine and refmac

[yang li]

Thanks for your replies. I did use default settings in phenix refine,
included TLS in it. This is an about 2.3A data, and the number Rfree used to
refine is big enough. I also kept the same Rfree in two programs. In refmac
I also tried diffenrent weighting sets--from defaut 0.3 to 0.02, TLS
included--maybe there are some other ways to define the restrains. The R and
Rfree will increase as the weighting is set too low, and the gap
didnot improve much. I donnot know in this case if the phenix refine has
converged, but the not stable Rfree in refmac made me nervous. Simply,
in this case, which one should I choose? Certainly to my private opinion I
would prefer to Scott's answer, Rfree is the most obvious parameter than
something esle like geometry, eg. Just looks good. :)


On 3/5/08, Partha Chakrabarti <[EMAIL PROTECTED]> wrote:
>
> One point which I don't understand is how can someone compare the two
> different programs when  they don't use the same numbers for
> xray:geometry terms? Taking the default settings for a given
> resolution might not be enough.. !
>
>
>
>
> On Tue, Mar 4, 2008 at 7:58 PM, Savvas Savvides
> <[EMAIL PROTECTED]> wrote:
> > Hi Yang
> >  how many reflections do you have in your test-set for calculating
> >  R-free? Too few reflections, typically less than 500, may not
> >  constitute a statistically robust cross-validation data set, and thus
> may
> >  lead to fluctuations in R-free plus a tendency for R-free to increase
> >  as a function of refinement cycle. Some of the early
> >  publications from Axel Brunger on crystallographic cross-validation
> >  address the need for enough reflections (>500) in the test-set. In
> >  addition, the presence of even a handful of strong but inaccurately
> >  measured/integrated low-resolution reflections in a limited test-set
> >  can aggravate abnormal behavior in R-free.
> >
> >  Best wishes
> >  Savvas
> >
> >
> >  toQuoting yang li <[EMAIL PROTECTED]>:
> >
> >  > Dear All,
> >  >   I have post a similar question about CNS and refmac before, now
> in
> >  > another structure I met a similar problem. I have an almost finished
> >  > structure, the Rfree of which
> >  > is about 0.28 by refmac. Then I used phenix to refine it, below is
> the
> >  > result:
> >  > REMARK  REFINEMENT SUMMARY: QUICK FACTS
> >  > ***
> >  > REMARK Start: r_work = 0.1970 r_free = 0.2892 bonds = 0.006 angles =
> 1.213
> >  > REMARK Final: r_work = 0.1917 r_free = 0.2617 bonds = 0.008 angles =
> 1.374
> >  > REMARK
> >  >
> 
> >  > Since the map from phenix couldnot be opened by coot directly--or
> I
> >  > donnot know how to--I used refmac to get a mtz map file. But I found
> that at
> >  > the first several cycle of
> >  > refmac the Rfree decreased, then both the R and Rfree
> values  continued
> >  > increasing and FOM decreasing.
> >  >  The best R/Rfree/FOM during the refinement is
> >  >
> >  >
> -
> >  > Overall R factor = 0.1932
> >  > Free R factor= 0.2513
> >  > Overall figure of merit  = 0.8168
> >  >
> -
> >  >  And after 40 cycles the final result is:
> >  >
> -
> >  > Overall R factor = 0.2008
> >  > Free R factor= 0.2772
> >  > Overall figure of merit  = 0.7902
> >  >
> -
> >  > The values looks like keep going up if increase the cycles. Then
> which
> >  > value should I take as the final result? The phenix or the best
> Refmac
> >  > result or  I have to take  a converged
> >  > value from refmac?
> >  >
> >
>
>
>
>
> --
> MRC National Institute for Medical Research
> Division of Molecular Structure
> The Ridgeway, NW7 1AA, UK
> Email: [EMAIL PROTECTED]
> Phone: + 44 208 816 2515
>

Re: [ccp4bb] phenix.refine and refmac

[Partha Chakrabarti]

One point which I don't understand is how can someone compare the two
different programs when  they don't use the same numbers for
xray:geometry terms? Taking the default settings for a given
resolution might not be enough.. !



On Tue, Mar 4, 2008 at 7:58 PM, Savvas Savvides
<[EMAIL PROTECTED]> wrote:
> Hi Yang
>  how many reflections do you have in your test-set for calculating
>  R-free? Too few reflections, typically less than 500, may not
>  constitute a statistically robust cross-validation data set, and thus may
>  lead to fluctuations in R-free plus a tendency for R-free to increase
>  as a function of refinement cycle. Some of the early
>  publications from Axel Brunger on crystallographic cross-validation
>  address the need for enough reflections (>500) in the test-set. In
>  addition, the presence of even a handful of strong but inaccurately
>  measured/integrated low-resolution reflections in a limited test-set
>  can aggravate abnormal behavior in R-free.
>
>  Best wishes
>  Savvas
>
>
>  toQuoting yang li <[EMAIL PROTECTED]>:
>
>  > Dear All,
>  >   I have post a similar question about CNS and refmac before, now in
>  > another structure I met a similar problem. I have an almost finished
>  > structure, the Rfree of which
>  > is about 0.28 by refmac. Then I used phenix to refine it, below is the
>  > result:
>  > REMARK  REFINEMENT SUMMARY: QUICK FACTS
>  > ***
>  > REMARK Start: r_work = 0.1970 r_free = 0.2892 bonds = 0.006 angles = 1.213
>  > REMARK Final: r_work = 0.1917 r_free = 0.2617 bonds = 0.008 angles = 1.374
>  > REMARK
>  > 
>  > Since the map from phenix couldnot be opened by coot directly--or I
>  > donnot know how to--I used refmac to get a mtz map file. But I found that 
> at
>  > the first several cycle of
>  > refmac the Rfree decreased, then both the R and Rfree values  continued
>  > increasing and FOM decreasing.
>  >  The best R/Rfree/FOM during the refinement is
>  >
>  > 
> -
>  > Overall R factor = 0.1932
>  > Free R factor= 0.2513
>  > Overall figure of merit  = 0.8168
>  > 
> -
>  >  And after 40 cycles the final result is:
>  > 
> -
>  > Overall R factor = 0.2008
>  > Free R factor= 0.2772
>  > Overall figure of merit  = 0.7902
>  > 
> -
>  > The values looks like keep going up if increase the cycles. Then which
>  > value should I take as the final result? The phenix or the best Refmac
>  > result or  I have to take  a converged
>  > value from refmac?
>  >
>



-- 
MRC National Institute for Medical Research
Division of Molecular Structure
The Ridgeway, NW7 1AA, UK
Email: [EMAIL PROTECTED]
Phone: + 44 208 816 2515

Re: [ccp4bb] phenix.refine and refmac

[Savvas Savvides]

Hi Yang
how many reflections do you have in your test-set for calculating
R-free? Too few reflections, typically less than 500, may not
constitute a statistically robust cross-validation data set, and thus may
lead to fluctuations in R-free plus a tendency for R-free to increase  
as a function of refinement cycle. Some of the early

publications from Axel Brunger on crystallographic cross-validation
address the need for enough reflections (>500) in the test-set. In  
addition, the presence of even a handful of strong but inaccurately  
measured/integrated low-resolution reflections in a limited test-set  
can aggravate abnormal behavior in R-free.


Best wishes
Savvas


toQuoting yang li <[EMAIL PROTECTED]>:


Dear All,
  I have post a similar question about CNS and refmac before, now in
another structure I met a similar problem. I have an almost finished
structure, the Rfree of which
is about 0.28 by refmac. Then I used phenix to refine it, below is the
result:
REMARK  REFINEMENT SUMMARY: QUICK FACTS
***
REMARK Start: r_work = 0.1970 r_free = 0.2892 bonds = 0.006 angles = 1.213
REMARK Final: r_work = 0.1917 r_free = 0.2617 bonds = 0.008 angles = 1.374
REMARK

Since the map from phenix couldnot be opened by coot directly--or I
donnot know how to--I used refmac to get a mtz map file. But I found that at
the first several cycle of
refmac the Rfree decreased, then both the R and Rfree values  continued
increasing and FOM decreasing.
 The best R/Rfree/FOM during the refinement is

-
Overall R factor = 0.1932
Free R factor= 0.2513
Overall figure of merit  = 0.8168
-
 And after 40 cycles the final result is:
-
Overall R factor = 0.2008
Free R factor= 0.2772
Overall figure of merit  = 0.7902
-
The values looks like keep going up if increase the cycles. Then which
value should I take as the final result? The phenix or the best Refmac
result or  I have to take  a converged
value from refmac?

Re: [ccp4bb] phenix.refine and refmac

[Nikos Pinotsis]

Dear Yang,

it would be better if you could provide more details about your 
refinement strategies.
As I can see from your results you might have used the default values in 
phenix giving you finally very low rmsd values. Refmac by default is not 
using so tight restraints and even if you used tight restraints in 
refmac the difference between R and Rfree might be much lower than the 
7% in Phenix. I am guessing that the values that you are reporting have 
to do also with the weight values each program uses and how you are 
define them. Also have you used TLS in any of the two programs?


regards
Nikos

-
Dr. Nikos Pinotsis
Section of Structural Biology 
Institute of Cancer Research
Chester Beatty Laboratories 
237 Fulham Road 
London SW3 6JB, UK
Tel: +44 20 7153 5453 / 5447 
-




yang li wrote:

Dear All,
  I have post a similar question about CNS and refmac before, now 
in another structure I met a similar problem. I have an almost 
finished structure, the Rfree of which
is about 0.28 by refmac. Then I used phenix to refine it, below is the 
result:
REMARK  REFINEMENT SUMMARY: QUICK FACTS 
***

REMARK Start: r_work = 0.1970 r_free = 0.2892 bonds = 0.006 angles = 1.213
REMARK Final: r_work = 0.1917 r_free = 0.2617 bonds = 0.008 angles = 1.374
REMARK 

Since the map from phenix couldnot be opened by coot directly--or 
I donnot know how to--I used refmac to get a mtz map file. But I found 
that at the first several cycle of
refmac the Rfree decreased, then both the R and Rfree values  
continued increasing and FOM decreasing.

 The best R/Rfree/FOM during the refinement is

-

Overall R factor = 0.1932
Free R factor= 0.2513
Overall figure of merit  = 0.8168
-
 And after 40 cycles the final result is:
-
Overall R factor = 0.2008
Free R factor= 0.2772
Overall figure of merit  = 0.7902
-
The values looks like keep going up if increase the cycles. Then 
which value should I take as the final result? The phenix or the best 
Refmac result or  I have to take  a converged

value from refmac?


The Institute of Cancer Research: Royal Cancer Hospital, a charitable Company 
Limited by Guarantee, Registered in England under Company No. 534147 with its 
Registered Office at 123 Old Brompton Road, London SW7 3RP.

This e-mail message is confidential and for use by the addressee only.  If the 
message is received by anyone other than the addressee, please return the 
message to the sender by replying to it and then delete the message from your 
computer and network.

Re: [ccp4bb] phenix.refine and refmac

[William Scott]

I'd take the one with the lowest Rfree, but it would be worthwhile to
figure out why the refinement appears unstable.  phenix should work at
least as well as refmac, and as far as I am aware, use the same bond
parameters.



yang li wrote:
>  Yes, I got the mtz file which can be oopened by coot, so can I take
> the
> phenix result?
> I am customed to use refmac before, but now is confused when several
> choices
> come out.
>
>
> On 3/4/08, William Scott <[EMAIL PROTECTED]> wrote:
>>
>> phenix.refine also produces an mtz file by default, and that can be
>> auto-opened with coot, along with the coordinates.
>>
>> On Mar 4, 2008, at 7:27 AM, yang li wrote:
>>
>> >
>> >Since the map from phenix couldnot be opened by coot directly--or I
>> > donnot know how to--I used refmac to get a mtz map file.
>>
>

Re: [ccp4bb] phenix.refine and refmac

[yang li]

 Yes, I got the mtz file which can be oopened by coot, so can I take the
phenix result?
I am customed to use refmac before, but now is confused when several choices
come out.


On 3/4/08, William Scott <[EMAIL PROTECTED]> wrote:
>
> phenix.refine also produces an mtz file by default, and that can be
> auto-opened with coot, along with the coordinates.
>
> On Mar 4, 2008, at 7:27 AM, yang li wrote:
>
> >
> >Since the map from phenix couldnot be opened by coot directly--or I
> > donnot know how to--I used refmac to get a mtz map file.
>

Re: [ccp4bb] phenix.refine and refmac

[William Scott]

phenix.refine also produces an mtz file by default, and that can be  
auto-opened with coot, along with the coordinates.


On Mar 4, 2008, at 7:27 AM, yang li wrote:



   Since the map from phenix couldnot be opened by coot directly--or I
donnot know how to--I used refmac to get a mtz map file.

[ccp4bb] phenix.refine and refmac

[yang li]

Dear All,
  I have post a similar question about CNS and refmac before, now in
another structure I met a similar problem. I have an almost finished
structure, the Rfree of which
is about 0.28 by refmac. Then I used phenix to refine it, below is the
result:
REMARK  REFINEMENT SUMMARY: QUICK FACTS
***
REMARK Start: r_work = 0.1970 r_free = 0.2892 bonds = 0.006 angles = 1.213
REMARK Final: r_work = 0.1917 r_free = 0.2617 bonds = 0.008 angles = 1.374
REMARK

Since the map from phenix couldnot be opened by coot directly--or I
donnot know how to--I used refmac to get a mtz map file. But I found that at
the first several cycle of
refmac the Rfree decreased, then both the R and Rfree values  continued
increasing and FOM decreasing.
 The best R/Rfree/FOM during the refinement is

-
Overall R factor = 0.1932
Free R factor= 0.2513
Overall figure of merit  = 0.8168
-
 And after 40 cycles the final result is:
-
Overall R factor = 0.2008
Free R factor= 0.2772
Overall figure of merit  = 0.7902
-
The values looks like keep going up if increase the cycles. Then which
value should I take as the final result? The phenix or the best Refmac
result or  I have to take  a converged
value from refmac?