Re: How many rounds are acceptable?

[Johannes Dietschreit]

Dear Troels, 

I have solved the problem. There was apparently some confusion with those who 
have performed the measurements. Now, that all inputs are given in the correct 
format and the correct nucleus is specified, relax works like a charm and the 
results are physically reasonable. 

I think your tutorial is very helpful!

Best, 

Johannes

> Am 13.09.2016 um 22:10 schrieb Troels Emtekær Linnet :
> 
> Dear Johannes.
> 
> As far as I know, Edward is currently on maternity leave.
> 
> To solve this problem, it would be easier to have access to some of your data.
> Can you upload to: https://gna.org/bugs/?group=relax 
> 
> 
> Take each of your data files, and delete all data, except 2 spins.
> Also provide your script file, or a description of which button you press in 
> the GUI.
> 
> Please also provide information about your system with:
> relax -i
> 
> Then I will make a tutorial for you. To be added here:
> http://wiki.nmr-relax.com/Category:Tutorials 
> 
> 
> I have just made a similar tutorial here:
> http://wiki.nmr-relax.com/Tutorial_for_model-free_analysis_sam_mahdi 
> 
> 
> If there is a problem in relax, I will write a systemtest which will solve 
> the problem.
> And the problem will never return.
> 
> If this a user error, the tutorial should help to prevent this, and would be 
> the first step before
> adding/modifying the manual.
> 
> Note:
> I can only help with setting up the loading of the data.
> The interpretation of the results is not my strong side.
> 
> Best
> Troels
> 
> 
> 2016-08-24 17:27 GMT+02:00 Johannes Dietschreit  >:
> Hey guys, 
> 
> I just wondered whether you could help me finding the problem in my usage of 
> relax or whether there is a problem with the data. Do you have any 
> suggestions to the questions I have asked in the previous mails?
> 
> Thank you in advance.
> 
> Johannes 
> 
> 
>> Am 10.08.2016 um 16:37 schrieb Johannes Dietschreit > >:
>> 
>> Hi, 
>> 
>> I wanted to ask whether there is an example data set of relaxation data, PDB 
>> and results that I could use to see whether I get the same results and thus 
>> am using the software correctly in order to find out whether it's the 
>> handling of the programm or the supplied data that causes trouble.
>> 
>> Thanks for all your help!
>> 
>> Johannes 
>> 
>> 
>> 
>> 2016-08-09 11:15 GMT+02:00 Johannes Dietschreit > >:
>> Hello Edward,
>> 
>> I understand that the strict convergence criteria are neccessary. When I set 
>> the max_iter parameter to just 30, I get very similar results as to when I 
>> reduce the opt_tol. I guess the model is just not converged. 
>> 
>> You were quite right, the negative te values are very close to zero, their 
>> error is many orders of magnitude larger than their own value. The avergae 
>> hetNOE values is around 0.7. The measurements were taken at 600 and 700 MHz. 
>> I haven't taken them myself, my work concerns the theoretical side of 
>> things. The protein is not a membrane protein, it is in solution and the 
>> exact molecule given in the crystal structure I used as input for relax was 
>> measured (no tag or similar). The local spectrometer does unfortunately not 
>> allow for fancy temperature control methods.
>> 
>> I feel relatively certain that the set up was in general correct. I tried 
>> both your python script and the GUI. The results I reported were taken from 
>> the txt-files in the final folder. As for the results of the previous runs. 
>> I am not sure how to access them or to read them. They are all zipped xml 
>> files where I cannot find a clear results section. Is there a certain 
>> program/command I should use? 
>> 
>> I wanted to use your visualization script xh_vector_dist.py, but I am not 
>> sure whoch file should be the  input for
>> 'select.read(file=pardir+sep+'rates.txt', change_all=True, res_num_col=2)'
>> I visualized the tensor.pdb which is printed in the final directory. It is 
>> huge compared to the protein. It has the shape of a eight of a spheroid. 
>> 
>> Maybe just a simple question regarding the input. The R1 and R2 input files 
>> contained values in Hz (not radian). Is that correct? 
>> 
>> 
>> Regards, 
>> 
>> Johannes
>> 
>> 
>> 
>> 
>> 2016-08-08 16:49 GMT+02:00 Edward d'Auvergne > >:
>> Hi Johannes,
>> 
>> Sorry for the late response.  I've been quite busy in the last two
>> months.  Please see below:
>> 
>> 
>> On 8 August 2016 at 13:25, Johannes Dietschreit > > wrote:
>> > Hi,
>> >
>> > thank you so much for your quick and long response. I did not change the
>> > hard coded value, I 

Re: How many rounds are acceptable?

[Johannes Dietschreit]

Hi,

I wanted to ask whether there is an example data set of relaxation data,
PDB and results that I could use to see whether I get the same results and
thus am using the software correctly in order to find out whether it's the
handling of the programm or the supplied data that causes trouble.

Thanks for all your help!

Johannes



2016-08-09 11:15 GMT+02:00 Johannes Dietschreit :

> Hello Edward,
>
> I understand that the strict convergence criteria are neccessary. When I
> set the max_iter parameter to just 30, I get very similar results as to
> when I reduce the opt_tol. I guess the model is just not converged.
>
> You were quite right, the negative te values are very close to zero, their
> error is many orders of magnitude larger than their own value. The avergae
> hetNOE values is around 0.7. The measurements were taken at 600 and 700
> MHz. I haven't taken them myself, my work concerns the theoretical side of
> things. The protein is not a membrane protein, it is in solution and the
> exact molecule given in the crystal structure I used as input for relax was
> measured (no tag or similar). The local spectrometer does unfortunately not
> allow for fancy temperature control methods.
>
> I feel relatively certain that the set up was in general correct. I tried
> both your python script and the GUI. The results I reported were taken from
> the txt-files in the final folder. As for the results of the previous runs.
> I am not sure how to access them or to read them. They are all zipped xml
> files where I cannot find a clear results section. Is there a certain
> program/command I should use?
>
> I wanted to use your visualization script xh_vector_dist.py, but I am not
> sure whoch file should be the  input for
> 'select.read(file=pardir+sep+'rates.txt', change_all=True, res_num_col=2)'
> I visualized the tensor.pdb which is printed in the final directory. It is
> huge compared to the protein. It has the shape of a eight of a spheroid.
>
> Maybe just a simple question regarding the input. The R1 and R2 input
> files contained values in Hz (not radian). Is that correct?
>
>
> Regards,
>
> Johannes
>
>
>
>
> 2016-08-08 16:49 GMT+02:00 Edward d'Auvergne :
>
>> Hi Johannes,
>>
>> Sorry for the late response.  I've been quite busy in the last two
>> months.  Please see below:
>>
>>
>> On 8 August 2016 at 13:25, Johannes Dietschreit 
>> wrote:
>> > Hi,
>> >
>> > thank you so much for your quick and long response. I did not change the
>> > hard coded value, I decided to leave that one alone, but I followed the
>> > example of the test_suite and set the convergence criterion to 1e-7.
>> This
>> > seems still strict, but now the calculation converges and final results
>> are
>> > written.
>>
>> The "strict" values are quite important for making sure there are no
>> strange results.  You'll see that in:
>>
>> d'Auvergne, E. J. and Gooley, P. R. (2008). Optimisation of NMR
>> dynamic models I. Minimisation algorithms and their performance within
>> the model-free and Brownian rotational diffusion spaces. J. Biomol.
>> NMR, 40(2), 107-119. (http://dx.doi.org/10.1007/s10858-007-9214-2)
>>
>> Reading this paper is essential for understanding these cut-off
>> values, and why these high precision values are important.  However if
>> there is a problem with the input data, then you will see problems.
>> In your example, I am not sure why this is not stopping after 30
>> iterations.  If you have a look at the automated protocol:
>>
>> http://www.nmr-relax.com/api/4.0/auto_analyses.dauvergne_pro
>> tocol-module.html
>> http://www.nmr-relax.com/api/4.0/auto_analyses.dauvergne_pro
>> tocol.dAuvergne_protocol-class.html
>>
>> you will see the max_iter parameter.  Ok, I see that this is set to 30
>> in the GUI, but it is not set in the sample script.  If you require
>> more than 30 iterations of the global protocol (
>> http://www.nmr-relax.com/manual/Model_free_analysis_in_reverse.html ),
>> then this is an indication that the input data is problematic.
>>
>>
>> > My question now regards the results in the folder "final". My analysis
>> was
>> > performed regarding the classical model free ansatz but I allowed for
>> all
>> > possible diffusion models. The protein I am dealing with is rather
>> stiff,
>> > it contains a beta barrel and some flexible loops. However, all residues
>> > have a S^2 value of about 0.03. I expected the residues in the barrel to
>> > have much larger values.
>>
>> These values indicate a severe problem with the input data.  What is
>> your average HetNOE value?  At which field strengths did you measure?
>>
>>
>> > Also the t_e values are somewhat spurious. Most of them are unphisically
>> > small (~e-24 seconds), I had expected values in the range of ten to
>> hundred
>> > pico seconds. And there are a few t_e values with a negative sign. How
>> is
>> > that possible?
>>
>> These should disappear through model selection. 

Re: How many rounds are acceptable?

[Johannes Dietschreit]

Hello Edward,

I understand that the strict convergence criteria are neccessary. When I
set the max_iter parameter to just 30, I get very similar results as to
when I reduce the opt_tol. I guess the model is just not converged.

You were quite right, the negative te values are very close to zero, their
error is many orders of magnitude larger than their own value. The avergae
hetNOE values is around 0.7. The measurements were taken at 600 and 700
MHz. I haven't taken them myself, my work concerns the theoretical side of
things. The protein is not a membrane protein, it is in solution and the
exact molecule given in the crystal structure I used as input for relax was
measured (no tag or similar). The local spectrometer does unfortunately not
allow for fancy temperature control methods.

I feel relatively certain that the set up was in general correct. I tried
both your python script and the GUI. The results I reported were taken from
the txt-files in the final folder. As for the results of the previous runs.
I am not sure how to access them or to read them. They are all zipped xml
files where I cannot find a clear results section. Is there a certain
program/command I should use?

I wanted to use your visualization script xh_vector_dist.py, but I am not
sure whoch file should be the  input for
'select.read(file=pardir+sep+'rates.txt', change_all=True, res_num_col=2)'
I visualized the tensor.pdb which is printed in the final directory. It is
huge compared to the protein. It has the shape of a eight of a spheroid.

Maybe just a simple question regarding the input. The R1 and R2 input files
contained values in Hz (not radian). Is that correct?


Regards,

Johannes




2016-08-08 16:49 GMT+02:00 Edward d'Auvergne :

> Hi Johannes,
>
> Sorry for the late response.  I've been quite busy in the last two
> months.  Please see below:
>
>
> On 8 August 2016 at 13:25, Johannes Dietschreit 
> wrote:
> > Hi,
> >
> > thank you so much for your quick and long response. I did not change the
> > hard coded value, I decided to leave that one alone, but I followed the
> > example of the test_suite and set the convergence criterion to 1e-7. This
> > seems still strict, but now the calculation converges and final results
> are
> > written.
>
> The "strict" values are quite important for making sure there are no
> strange results.  You'll see that in:
>
> d'Auvergne, E. J. and Gooley, P. R. (2008). Optimisation of NMR
> dynamic models I. Minimisation algorithms and their performance within
> the model-free and Brownian rotational diffusion spaces. J. Biomol.
> NMR, 40(2), 107-119. (http://dx.doi.org/10.1007/s10858-007-9214-2)
>
> Reading this paper is essential for understanding these cut-off
> values, and why these high precision values are important.  However if
> there is a problem with the input data, then you will see problems.
> In your example, I am not sure why this is not stopping after 30
> iterations.  If you have a look at the automated protocol:
>
> http://www.nmr-relax.com/api/4.0/auto_analyses.dauvergne_
> protocol-module.html
> http://www.nmr-relax.com/api/4.0/auto_analyses.dauvergne_
> protocol.dAuvergne_protocol-class.html
>
> you will see the max_iter parameter.  Ok, I see that this is set to 30
> in the GUI, but it is not set in the sample script.  If you require
> more than 30 iterations of the global protocol (
> http://www.nmr-relax.com/manual/Model_free_analysis_in_reverse.html ),
> then this is an indication that the input data is problematic.
>
>
> > My question now regards the results in the folder "final". My analysis
> was
> > performed regarding the classical model free ansatz but I allowed for all
> > possible diffusion models. The protein I am dealing with is rather stiff,
> > it contains a beta barrel and some flexible loops. However, all residues
> > have a S^2 value of about 0.03. I expected the residues in the barrel to
> > have much larger values.
>
> These values indicate a severe problem with the input data.  What is
> your average HetNOE value?  At which field strengths did you measure?
>
>
> > Also the t_e values are somewhat spurious. Most of them are unphisically
> > small (~e-24 seconds), I had expected values in the range of ten to
> hundred
> > pico seconds. And there are a few t_e values with a negative sign. How is
> > that possible?
>
> These should disappear through model selection.  If te ~ 0, then the
> simpler model without te will be selected, as these models should have
> converged to the same result.  Unless of course there is a major
> failure of the whole analysis.  For the te values with negative
> values, what are there values?  The lower quality cut-off values will
> allow for very small minus te values.
>
>
> > Is this a common error? Have I just made a mistake regarding the input? I
> > attached my dauvergne_protocol.py file and some input data.
>
> Note that you cannot attach files when posting to a public