Re: missing data in relaxation dispersion

2016-06-06 Thread Petr Padrta
On Fri, Jun 03, 2016 at 04:43:18PM +0200, Edward d'Auvergne wrote:
> On 3 June 2016 at 12:08, Edward d'Auvergne  wrote:
> > Hi Petr,
> >
> > For the new bug report at https://gna.org/bugs/?24675, now looking at
> > the results I can see that this is actually quite a deliberate
> > feature.  For reference, I will attach the plots for residues 133 and
> > 134 for the CR2 model as a PDF file to that bug report
> > (disp_CKIRD_133_N.pdf and disp_CKIRD_134_N.pdf).  For residue 133,
> > there the blue line can be seen to be the dispersion curve predicted
> > for the missing 600 MHz data.  For 134, the green line is the
> > predicted dispersion curve for the 950 MHz data.
> >
> > The R20 values come from the fitting of the single field strength data
> > points, and this value is independent of field strength.  So the
> > predicted curves are perfectly correct - they are exactly where they
> > are supposed to be.  Assuming the model to be correct, the predicted
> > curves are what the real data should look like.  So the curves without
> > measured data are actually quite powerful tools for investigating why
> > the data is missing.  They are also useful for investigating how well
> > the model fits to single field strength data.  It is also quite easy
> > in Grace to double click on the curve and hide it, if desired.
> > Therefore I think I'll leave the plotting code as it is.
> 
> Hi Petr,
> 
> Sorry, the R20 values are field strength dependent, but both are fit
> during the optimisation of the dispersion models with missing data.
> This is problematic as one parameter is undefined - the R20 for the
> missing field - and this interferes with optimisation algorithms.  The
> undefined R20 value can float around to any value, and this breaks the
> rules that most algorithms absolutely rely on.  Hence the solution
> found is unlikely to be the minimum in the optimisation space.  I
> would therefore suggest deselecting these spins for now, as the
> results are nonsensical.  It might take me a while to update each part
> of relax to detect the missing data for one entire field strength from
> the N fields, and drop back to N-1 fields.  This bug might take a
> while to fix!  I might have to spend a few days working on that one
> next week.
> 
> Regards,
> 
> Edward

Hello Edward,

Hey, that would be nice if R20 were really independent of B0 ;-)

Well, the undefined R20 can float to any value but within the bounds (0-200) so
at least some algorithms should be able to tackle that, Nelder-Mead very
probably amongst them. This seems to be corroborated also by real data. I.e.
when analysing our complete data, (so far) I'm getting sensible results,
comparable to calculating separate residues, of course not counting the R20s
for missing B0s, which sometimes float to 0, at other times go to 200. That's
why I suggested in my bugreports and mails that for the case of leaving all
R20s in, it is probably better to force them to 0. Of course it's even better
to remove missing R20s completely but I expected that to be much more effort.

Regards,

Petr

___
relax (http://www.nmr-relax.com)

This is the relax-users mailing list
relax-users@gna.org

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users


Re: missing data in relaxation dispersion

2016-06-03 Thread Edward d'Auvergne
On 3 June 2016 at 12:08, Edward d'Auvergne  wrote:
> Hi Petr,
>
> For the new bug report at https://gna.org/bugs/?24675, now looking at
> the results I can see that this is actually quite a deliberate
> feature.  For reference, I will attach the plots for residues 133 and
> 134 for the CR2 model as a PDF file to that bug report
> (disp_CKIRD_133_N.pdf and disp_CKIRD_134_N.pdf).  For residue 133,
> there the blue line can be seen to be the dispersion curve predicted
> for the missing 600 MHz data.  For 134, the green line is the
> predicted dispersion curve for the 950 MHz data.
>
> The R20 values come from the fitting of the single field strength data
> points, and this value is independent of field strength.  So the
> predicted curves are perfectly correct - they are exactly where they
> are supposed to be.  Assuming the model to be correct, the predicted
> curves are what the real data should look like.  So the curves without
> measured data are actually quite powerful tools for investigating why
> the data is missing.  They are also useful for investigating how well
> the model fits to single field strength data.  It is also quite easy
> in Grace to double click on the curve and hide it, if desired.
> Therefore I think I'll leave the plotting code as it is.

Hi Petr,

Sorry, the R20 values are field strength dependent, but both are fit
during the optimisation of the dispersion models with missing data.
This is problematic as one parameter is undefined - the R20 for the
missing field - and this interferes with optimisation algorithms.  The
undefined R20 value can float around to any value, and this breaks the
rules that most algorithms absolutely rely on.  Hence the solution
found is unlikely to be the minimum in the optimisation space.  I
would therefore suggest deselecting these spins for now, as the
results are nonsensical.  It might take me a while to update each part
of relax to detect the missing data for one entire field strength from
the N fields, and drop back to N-1 fields.  This bug might take a
while to fix!  I might have to spend a few days working on that one
next week.

Regards,

Edward

___
relax (http://www.nmr-relax.com)

This is the relax-users mailing list
relax-users@gna.org

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users


Re: missing data in relaxation dispersion

2016-06-03 Thread Edward d'Auvergne
Hi Petr,

For the new bug report at https://gna.org/bugs/?24675, now looking at
the results I can see that this is actually quite a deliberate
feature.  For reference, I will attach the plots for residues 133 and
134 for the CR2 model as a PDF file to that bug report
(disp_CKIRD_133_N.pdf and disp_CKIRD_134_N.pdf).  For residue 133,
there the blue line can be seen to be the dispersion curve predicted
for the missing 600 MHz data.  For 134, the green line is the
predicted dispersion curve for the 950 MHz data.

The R20 values come from the fitting of the single field strength data
points, and this value is independent of field strength.  So the
predicted curves are perfectly correct - they are exactly where they
are supposed to be.  Assuming the model to be correct, the predicted
curves are what the real data should look like.  So the curves without
measured data are actually quite powerful tools for investigating why
the data is missing.  They are also useful for investigating how well
the model fits to single field strength data.  It is also quite easy
in Grace to double click on the curve and hide it, if desired.
Therefore I think I'll leave the plotting code as it is.

Cheers,

Edward



On 2 May 2016 at 17:37, Edward d'Auvergne  wrote:
> On 29 April 2016 at 11:58, Petr Padrta  wrote:
>> On Wed, Apr 27, 2016 at 11:27:09AM +0200, Edward d'Auvergne wrote:
>>> On 22 April 2016 at 20:43, Petr Padrta  wrote:
>>> > Hi Edward,
>>> >
>>> > OK,  I finally managed to create a bug report on gna with attached 
>>> > example.
>>>
>>> Hi Petr,
>>>
>>> Thank you for that.  For reference, the report is at
>>> https://gna.org/bugs/?24601 .  I have taken your data and script and
>>> created the following relax system test:
>>>
>>> $ relax -s Relax_disp.test_bug_24601_r2eff_missing_data
>>>
>>> I used this to catch the bug you saw and to fix all the issues.  Note
>>> that it took a bit longer than normal as the first error you saw was
>>> only one of 3.  I also had to update the Monte Carlo simulation error
>>> analysis code, and the Grace plotting code.  I have committed the
>>> changes:
>>>
>>> http://article.gmane.org/gmane.science.nmr.relax.scm/25955
>>>
>>> When using this, please carefully check that all is working as
>>> expected.  The system test now passes, however there might be other
>>> strangeness (for example, check your Grace graphs for incorrect data
>>> labels).  For any other issues you see, if you could create a separate
>>> bug report for each, that would be appreciated.  As before, if you
>>> could include truncated data and a script to replicate the issue, that
>>> would be much appreciated as I could then create the system test to
>>> help fix the problem.  The more bugs reported - from critical to
>>> superficial - the smoother we can make the analysis process!  Even
>>> confusing text messages by relax deserve a report.
>>>
>>> For obtaining the code, if you need this soon, I would suggest
>>> directly obtaining it from the source code repository:
>>>
>>> http://www.nmr-relax.com/download.html#Source_code_repository
>>>
>>> The reason is because there are a few newly introduced bugs that I
>>> would like solved before we release relax 4.0.2.  So it might take a
>>> few weeks before the new release is out.
>>>
>>> Regards,
>>>
>>> Edward
>>
>> Hi Edward,
>>
>> Thanks! I fetched last SVN version (r28204) and so far it seems that your new
>> dispersion code works as expected. The (rhetorical) question is what to do 
>> with
>> interpolated dispersion curves from missing B0 fields - they have the right
>> shape but are shifted along R2eff-axis as their r20 is unknown. Hmm, I can
>> always delete them from the grace plots. Or maybe I'll do another bug report
>> after some more testing.
>
> Hi Petr,
>
> A new bug report for such things would be much appreciated.  I can
> then list the bugs one after the other and their details in the
> release notes.  It also makes it incredibly easy for a developer to
> fix the bug if truncated, minimal data sets and a script and/or
> instructions are attached to the bug report.  That way a system test
> can be quickly constructed to reproducibly capture the bug.  In most
> cases where a system test can be set up, the fix then only requires
> 5-10 minutes to resolve.
>
> Cheers,
>
> Edward

___
relax (http://www.nmr-relax.com)

This is the relax-users mailing list
relax-users@gna.org

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users


Re: missing data in relaxation dispersion

2016-05-02 Thread Edward d'Auvergne
On 29 April 2016 at 11:58, Petr Padrta  wrote:
> On Wed, Apr 27, 2016 at 11:27:09AM +0200, Edward d'Auvergne wrote:
>> On 22 April 2016 at 20:43, Petr Padrta  wrote:
>> > Hi Edward,
>> >
>> > OK,  I finally managed to create a bug report on gna with attached example.
>>
>> Hi Petr,
>>
>> Thank you for that.  For reference, the report is at
>> https://gna.org/bugs/?24601 .  I have taken your data and script and
>> created the following relax system test:
>>
>> $ relax -s Relax_disp.test_bug_24601_r2eff_missing_data
>>
>> I used this to catch the bug you saw and to fix all the issues.  Note
>> that it took a bit longer than normal as the first error you saw was
>> only one of 3.  I also had to update the Monte Carlo simulation error
>> analysis code, and the Grace plotting code.  I have committed the
>> changes:
>>
>> http://article.gmane.org/gmane.science.nmr.relax.scm/25955
>>
>> When using this, please carefully check that all is working as
>> expected.  The system test now passes, however there might be other
>> strangeness (for example, check your Grace graphs for incorrect data
>> labels).  For any other issues you see, if you could create a separate
>> bug report for each, that would be appreciated.  As before, if you
>> could include truncated data and a script to replicate the issue, that
>> would be much appreciated as I could then create the system test to
>> help fix the problem.  The more bugs reported - from critical to
>> superficial - the smoother we can make the analysis process!  Even
>> confusing text messages by relax deserve a report.
>>
>> For obtaining the code, if you need this soon, I would suggest
>> directly obtaining it from the source code repository:
>>
>> http://www.nmr-relax.com/download.html#Source_code_repository
>>
>> The reason is because there are a few newly introduced bugs that I
>> would like solved before we release relax 4.0.2.  So it might take a
>> few weeks before the new release is out.
>>
>> Regards,
>>
>> Edward
>
> Hi Edward,
>
> Thanks! I fetched last SVN version (r28204) and so far it seems that your new
> dispersion code works as expected. The (rhetorical) question is what to do 
> with
> interpolated dispersion curves from missing B0 fields - they have the right
> shape but are shifted along R2eff-axis as their r20 is unknown. Hmm, I can
> always delete them from the grace plots. Or maybe I'll do another bug report
> after some more testing.

Hi Petr,

A new bug report for such things would be much appreciated.  I can
then list the bugs one after the other and their details in the
release notes.  It also makes it incredibly easy for a developer to
fix the bug if truncated, minimal data sets and a script and/or
instructions are attached to the bug report.  That way a system test
can be quickly constructed to reproducibly capture the bug.  In most
cases where a system test can be set up, the fix then only requires
5-10 minutes to resolve.

Cheers,

Edward

___
relax (http://www.nmr-relax.com)

This is the relax-users mailing list
relax-users@gna.org

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users


Re: missing data in relaxation dispersion

2016-04-30 Thread Petr Padrta
On Wed, Apr 27, 2016 at 11:27:09AM +0200, Edward d'Auvergne wrote:
> On 22 April 2016 at 20:43, Petr Padrta  wrote:
> > Hi Edward,
> >
> > OK,  I finally managed to create a bug report on gna with attached example.
> 
> Hi Petr,
> 
> Thank you for that.  For reference, the report is at
> https://gna.org/bugs/?24601 .  I have taken your data and script and
> created the following relax system test:
> 
> $ relax -s Relax_disp.test_bug_24601_r2eff_missing_data
> 
> I used this to catch the bug you saw and to fix all the issues.  Note
> that it took a bit longer than normal as the first error you saw was
> only one of 3.  I also had to update the Monte Carlo simulation error
> analysis code, and the Grace plotting code.  I have committed the
> changes:
> 
> http://article.gmane.org/gmane.science.nmr.relax.scm/25955
> 
> When using this, please carefully check that all is working as
> expected.  The system test now passes, however there might be other
> strangeness (for example, check your Grace graphs for incorrect data
> labels).  For any other issues you see, if you could create a separate
> bug report for each, that would be appreciated.  As before, if you
> could include truncated data and a script to replicate the issue, that
> would be much appreciated as I could then create the system test to
> help fix the problem.  The more bugs reported - from critical to
> superficial - the smoother we can make the analysis process!  Even
> confusing text messages by relax deserve a report.
> 
> For obtaining the code, if you need this soon, I would suggest
> directly obtaining it from the source code repository:
> 
> http://www.nmr-relax.com/download.html#Source_code_repository
> 
> The reason is because there are a few newly introduced bugs that I
> would like solved before we release relax 4.0.2.  So it might take a
> few weeks before the new release is out.
> 
> Regards,
> 
> Edward

Hi Edward,

Thanks! I fetched last SVN version (r28204) and so far it seems that your new
dispersion code works as expected. The (rhetorical) question is what to do with
interpolated dispersion curves from missing B0 fields - they have the right
shape but are shifted along R2eff-axis as their r20 is unknown. Hmm, I can
always delete them from the grace plots. Or maybe I'll do another bug report
after some more testing.

--

Petr Padrta

___
relax (http://www.nmr-relax.com)

This is the relax-users mailing list
relax-users@gna.org

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users