Relaxation dispersion clustering calculation time
Dear all, This is my first post here, and I have a question regarding the time it takes for a relaxation dispersion clustering process to finish. I have one clustering calculation that has been running for ~ 20,000 min on a single Xeon 2.66 GHz core. The cluster consists of 13 residues being fit to the ‘CR72 full’ model. I wonder if the long time it is taking is normal? Would it be possible that relax has been stuck in an infinite loop of some sort, without showing up in the log file? Any input would be greatly appreciated. By the way, using a cluster of only 11 residues out of the 13 did finish in ~13,000 min. Chung-ke Chang Biomacromolecular NMR Lab Institute of Biomedical Science Academia Sinica, Taiwan ___ relax (http://www.nmr-relax.com) This is the relax-users mailing list relax-users@gna.org To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-users
Re: Relaxation dispersion clustering calculation time
Hi Chung-ke. Can you put the information about which version of relax you use? You can in terminal do: relax -i and write it here. And then there is the question if you used data from one field or two spectrometer fields. Fitting to one field, can give problems. This is described here: Faithful estimation of dynamics parameters from CPMG relaxation dispersion measurements. Kovrigin, Evgenii L; Kempf, James G; Grey, Michael J; Loria, J Patrick Journal of magnetic resonance, 2006, Vol 180, p 93-104. http://www.ncbi.nlm.nih.gov/pubmed/16458551 DOI: 10.1016/j.jmr.2006.01.010 Figure 9 and 10 shows these rotten bananas. Clustering data, in some way overcome this problem. Since you now starts to add more data, compared to number of fitting parameters. The problem though, is that if you start from single fitted data, and go to Clustering of data, that an average of the global parameter will be taken for the single fitted data. In previous version of relax (a version or two ago), we changed from taking the average to take the median of the parameters. This was to prevent taking the average of an outliers, if one of the single fitted spins have been fitted crazy. You don't want to start with a global kex at 1. I have discussed the CR72 Full model with my supervisor. He have actually never seen it in use in any paper. Always the assumption R20A=R20B is used. If you only have one field, I would not even try this model. If you still would like to try it, please consider using the B14 full model as well, to compare. http://wiki.nmr-relax.com/B14_full Abstract: Faithful estimation of dynamics parameters from CPMG relaxation dispersion measurements. This work examines the robustness of fitting of parameters describing conformational exchange (k(ex), p(a/b), and Deltaomega) processes from CPMG relaxation dispersion data. We have analyzed the equations describing conformational exchange processes for the intrinsic inter-dependence of their parameters that leads to the existence of multiple equivalent solutions, which equally satisfy the experimental data. We have used Monte-Carlo simulations and fitting to the synthetic data sets as well as the direct 3-D mapping of the parameter space of k(ex), p(a/b), and Deltaomega to quantitatively assess the degree of the parameter inter-dependence. The demonstrated high correlation between parameters can preclude accurate dynamics parameter estimation from NMR spin-relaxation data obtained at a single static magnetic field. The strong parameter inter-dependence can readily be overcome through acquisition of spin-relaxation data at more than one static magnetic field thereby allowing accurate assessment of conformational exchange properties. Troels Emtekær Linnet PhD student Copenhagen University SBiNLab, 3-0-41 2014-09-09 9:48 GMT+02:00 Edward d'Auvergne edw...@nmr-relax.com: Hi Chung-ke, Welcome to the relax mailing lists! Thanks to the hard work of one of the relax developers - Troels Linnet - this long calculation time should now be much, much shorter. Have a look at the following release announcement: http://wiki.nmr-relax.com/Relax_3.3.0 For the 'CR72 full' model (http://wiki.nmr-relax.com/CR72_full), the clustering example here gives a ~22x speed up so your calculation time would then drop from ~20,000 min to ~1000 min. If you would like to receive announcements about new relax versions, please subscribe to the relax-announce mailing list (https://mail.gna.org/listinfo/relax-announce/). This list only receives ~10 emails per year. See http://news.gmane.org/gmane.science.nmr.relax.announce. I have a few questions about how you performed the analysis. Did you use a non-clustered set of results to seed the clustered analysis? In the dispersion auto-analysis protocol exposed via the GUI, the results from the non-clustered analysis will be taken as the starting point for optimisation of the clustered analysis, as described in Morin et al., 2014 (http://dx.doi.org/10.1093/bioinformatics/btu166). If you wish, and are capable with scripting, you can also create your own analysis protocol via a relax script and not use the auto-analysis. The relax software is very flexible and you can create quite complex analysis protocols - the auto-analyses are just large relax scripts. Also, did you look at the results from the non-clustered analysis to see if the kinetics of all 13 residues are similar? Or if the dispersion curves look reasonable? Some data might be of low quality and causing difficulties with the optimisation. You should also note that most dispersion data is not good enough to differentiate R20A from R20B. Do the final results (non-clustered and clustered) look reasonable for these two parameters? It could be that differentiating R20A from R20B in your system is difficult and causing optimisation to take much longer than normal. Do you see the same optimisation times with the clustered CR72 model where
Re: Relaxation dispersion clustering calculation time
Dear Troels and Edward, Thank you for the pointers. I was not aware that a new version was out last week, so I’ve asked the IT people to install it on our cluster. Below is the output from ‘relax -i’: [chungke@nmrc10 onc_dAUGA_MES_310K]$ relax -i relax repository checkout r24533 svn://svn.gna.org/svn/relax/trunk Molecular dynamics by NMR data analysis Copyright (C) 2001-2006 Edward d'Auvergne Copyright (C) 2006-2014 the relax development team This is free software which you are welcome to modify and redistribute under the conditions of the GNU General Public License (GPL). This program, including all modules, is licensed under the GPL and comes with absolutely no warranty. For details type 'GPL' within the relax prompt. Assistance in using the relax prompt and scripting interface can be accessed by typing 'help' within the prompt. Processor fabric: Uni-processor. Hardware information: Machine: x86_64 Processor: x86_64 Processor name: Intel(R) Xeon(R) CPU E5430 @ 2.66GHz Endianness: little Total RAM size: 7983 Mb Total swap size: 8189 Mb Operating system information: System: Linux Release: 2.6.18-164.el5 Version: #1 SMP Thu Sep 3 03:28:30 EDT 2009 Distribution:redhat 5.3 Final Full platform string:Linux-2.6.18-164.el5-x86_64-with-redhat-5.3-Final Python information: Architecture:64bit ELF Python version: 2.5.1 Python build:r251:54863, Jul 23 2008 17:35:20 Python compiler: GCC Intel(R) C++ gcc 4.1 mode Libc version:glibc 2.3 Python executable: /program/nmr/bin/python Python module path: ['/program/nmr/relax', '/program/nmr/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg', '/program/nmr/lib/python25.zip', '/program/nmr/lib/python2.5', '/program/nmr/lib/python2.5/plat-linux2', '/program/nmr/lib/python2.5/lib-tk', '/program/nmr/lib/python2.5/lib-dynload', '/program/nmr/lib/python2.5/site-packages', '/program/nmr/lib/python2.5/site-packages/Scientific/linux2'] Python packages and modules (most are optional): Name InstalledVersion Path minfx True 1.0.8 /program/nmr/lib/python2.5/site-packages/minfx bmrblibTrue 1.0.3 /program/nmr/lib/python2.5/site-packages/bmrblib numpy True 1.6.2 /program/nmr/lib/python2.5/site-packages/numpy scipy False wxPython False matplotlib True 0.98.3 /program/nmr/lib/python2.5/site-packages/matplotlib mpi4py True 1.3.1 /program/nmr/lib/python2.5/mpi4py epydoc False optparse True 1.5.3 /program/nmr/lib/python2.5/optparse.pyc readline True /program/nmr/lib/python2.5/lib-dynload/readline.so profileTrue /program/nmr/lib/python2.5/profile.pyc bz2True /program/nmr/lib/python2.5/lib-dynload/bz2.so gzip True /program/nmr/lib/python2.5/gzip.pyc io False xmlTrue 0.8.4 (internal) /program/nmr/lib/python2.5/xml/__init__.pyc xml.dom.minidomTrue /program/nmr/lib/python2.5/xml/dom/minidom.pyc relax information: Version: repository checkout r24533 svn://svn.gna.org/svn/relax/trunk Processor fabric:Uni-processor. relax C modules: ModuleCompiledFile type Path target_functions.relax_fitTrueELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), not stripped /program/nmr/relax/target_functions/relax_fit.so As for the data itself, I am using data obtained on two fields and use both from the start. Upon closer look at the R20 parameters, I think both of you are right: the R20a and R20b numbers are really funky. I shall
Re: Relaxation dispersion clustering calculation time
By the way.
You can start a new analysis from the old results.
I would do this.
1) Upgrade relax
2) Make backup with previous data
3) Start relax in GUI
4) Start a Relaxation dispersion analysis
5) Create the pipe
6) Then go to user functions - results - read
7) Point to the file results.bz2 from a previous run.
This should bring up the setting in the GUI.
Then inspect the clustering is as you wish:
Menu View - Relax prompt
And write:
cdp.clustering
{'test': [':2@N'], 'free spins': [':3@N', ':4@N']}
Here all spins in 'test', will be fitted together.
All in 'free spins' are fitted individual.
If you then take the models:
CR72, CR72 Full, B14 Full,
then relax will first minimise for CR72.
When it proceeds to CR72 Full, and B14 Full, the nesting mechanism
will take similar parameters from
CR72, and use that as the start for minimisation.
Best
Troels
2014-09-09 10:25 GMT+02:00 Troels Emtekær Linnet tlin...@nmr-relax.com:
Hi Chung-ke.
Can you put the information about which version of relax you use?
You can in terminal do:
relax -i
and write it here.
And then there is the question if you used data from one field or two
spectrometer fields.
Fitting to one field, can give problems.
This is described here:
Faithful estimation of dynamics parameters from CPMG relaxation
dispersion measurements.
Kovrigin, Evgenii L; Kempf, James G; Grey, Michael J; Loria, J Patrick
Journal of magnetic resonance, 2006, Vol 180, p 93-104.
http://www.ncbi.nlm.nih.gov/pubmed/16458551
DOI: 10.1016/j.jmr.2006.01.010
Figure 9 and 10 shows these rotten bananas.
Clustering data, in some way overcome this problem.
Since you now starts to add more data, compared to number of fitting
parameters.
The problem though, is that if you start from single fitted data,
and go to Clustering of data, that
an average of the global parameter will be taken for the single fitted data.
In previous version of relax (a version or two ago), we changed from
taking the average to take the median of the parameters.
This was to prevent taking the average of an outliers, if one of the
single fitted spins have been fitted crazy.
You don't want to start with a global kex at 1.
I have discussed the CR72 Full model with my supervisor.
He have actually never seen it in use in any paper.
Always the assumption R20A=R20B is used.
If you only have one field, I would not even try this model.
If you still would like to try it, please consider using the B14 full
model as well, to compare.
http://wiki.nmr-relax.com/B14_full
Abstract: Faithful estimation of dynamics parameters from CPMG
relaxation dispersion measurements.
This work examines the robustness of fitting of parameters describing
conformational exchange (k(ex), p(a/b), and Deltaomega) processes from
CPMG relaxation dispersion data. We have analyzed the equations
describing conformational exchange processes for the intrinsic
inter-dependence of their parameters that leads to the existence of
multiple equivalent solutions, which equally satisfy the experimental
data. We have used Monte-Carlo simulations and fitting to the
synthetic data sets as well as the direct 3-D mapping of the parameter
space of k(ex), p(a/b), and Deltaomega to quantitatively assess the
degree of the parameter inter-dependence. The demonstrated high
correlation between parameters can preclude accurate dynamics
parameter estimation from NMR spin-relaxation data obtained at a
single static magnetic field. The strong parameter inter-dependence
can readily be overcome through acquisition of spin-relaxation data at
more than one static magnetic field thereby allowing accurate
assessment of conformational exchange properties.
Troels Emtekær Linnet
PhD student
Copenhagen University
SBiNLab, 3-0-41
2014-09-09 9:48 GMT+02:00 Edward d'Auvergne edw...@nmr-relax.com:
Hi Chung-ke,
Welcome to the relax mailing lists! Thanks to the hard work of one of
the relax developers - Troels Linnet - this long calculation time
should now be much, much shorter. Have a look at the following
release announcement:
http://wiki.nmr-relax.com/Relax_3.3.0
For the 'CR72 full' model (http://wiki.nmr-relax.com/CR72_full), the
clustering example here gives a ~22x speed up so your calculation time
would then drop from ~20,000 min to ~1000 min. If you would like to
receive announcements about new relax versions, please subscribe to
the relax-announce mailing list
(https://mail.gna.org/listinfo/relax-announce/). This list only
receives ~10 emails per year. See
http://news.gmane.org/gmane.science.nmr.relax.announce.
I have a few questions about how you performed the analysis. Did you
use a non-clustered set of results to seed the clustered analysis? In
the dispersion auto-analysis protocol exposed via the GUI, the results
from the non-clustered analysis will be taken as the starting point
for optimisation of the clustered analysis, as described in Morin et
al., 2014
Re: Relaxation dispersion clustering calculation time
Hi Chung-ke, The only way to find out about new relax releases is the relax-announce mailing list (http://news.gmane.org/gmane.science.nmr.relax.announce). Some relax users were signed up for the freecode announcements (http://freecode.com/projects/nmr-relax), but freecode has unfortunately shut down (http://freecode.com/about). For the version you are currently using, note that this is the repository version of relax installed by the superuser. You should make sure you use the normal releases, as the repository version can sometimes be in a broken or buggy state as development occurs. You can also have a copy in your home directory by typing: $ svn co http://svn.gna.org/svn/relax/trunk ./relax-trunk $ cd relax-trunk $ scons If you already have a repository version on your system, these commands should just work. But you should only use the repository version if you would like a bug fix and cannot wait until the next relax release. Regards, Edward On 9 September 2014 10:37, Chung-ke Chang chun...@ibms.sinica.edu.tw wrote: Dear Troels and Edward, Thank you for the pointers. I was not aware that a new version was out last week, so I’ve asked the IT people to install it on our cluster. Below is the output from ‘relax -i’: [chungke@nmrc10 onc_dAUGA_MES_310K]$ relax -i relax repository checkout r24533 svn://svn.gna.org/svn/relax/trunk Molecular dynamics by NMR data analysis Copyright (C) 2001-2006 Edward d'Auvergne Copyright (C) 2006-2014 the relax development team This is free software which you are welcome to modify and redistribute under the conditions of the GNU General Public License (GPL). This program, including all modules, is licensed under the GPL and comes with absolutely no warranty. For details type 'GPL' within the relax prompt. Assistance in using the relax prompt and scripting interface can be accessed by typing 'help' within the prompt. Processor fabric: Uni-processor. Hardware information: Machine: x86_64 Processor: x86_64 Processor name: Intel(R) Xeon(R) CPU E5430 @ 2.66GHz Endianness: little Total RAM size: 7983 Mb Total swap size: 8189 Mb Operating system information: System: Linux Release: 2.6.18-164.el5 Version: #1 SMP Thu Sep 3 03:28:30 EDT 2009 Distribution:redhat 5.3 Final Full platform string: Linux-2.6.18-164.el5-x86_64-with-redhat-5.3-Final Python information: Architecture:64bit ELF Python version: 2.5.1 Python build:r251:54863, Jul 23 2008 17:35:20 Python compiler: GCC Intel(R) C++ gcc 4.1 mode Libc version:glibc 2.3 Python executable: /program/nmr/bin/python Python module path: ['/program/nmr/relax', '/program/nmr/lib/python2.5/site-packages/setuptools-0.6c9-py2.5.egg', '/program/nmr/lib/python25.zip', '/program/nmr/lib/python2.5', '/program/nmr/lib/python2.5/plat-linux2', '/program/nmr/lib/python2.5/lib-tk', '/program/nmr/lib/python2.5/lib-dynload', '/program/nmr/lib/python2.5/site-packages', '/program/nmr/lib/python2.5/site-packages/Scientific/linux2'] Python packages and modules (most are optional): Name InstalledVersion Path minfx True 1.0.8 /program/nmr/lib/python2.5/site-packages/minfx bmrblibTrue 1.0.3 /program/nmr/lib/python2.5/site-packages/bmrblib numpy True 1.6.2 /program/nmr/lib/python2.5/site-packages/numpy scipy False wxPython False matplotlib True 0.98.3 /program/nmr/lib/python2.5/site-packages/matplotlib mpi4py True 1.3.1 /program/nmr/lib/python2.5/mpi4py epydoc False optparse True 1.5.3 /program/nmr/lib/python2.5/optparse.pyc readline True /program/nmr/lib/python2.5/lib-dynload/readline.so profileTrue /program/nmr/lib/python2.5/profile.pyc bz2True /program/nmr/lib/python2.5/lib-dynload/bz2.so gzip True /program/nmr/lib/python2.5/gzip.pyc io False xmlTrue 0.8.4 (internal) /program/nmr/lib/python2.5/xml/__init__.pyc xml.dom.minidomTrue /program/nmr/lib/python2.5/xml/dom/minidom.pyc relax information: Version: repository checkout r24533 svn://svn.gna.org/svn/relax/trunk Processor fabric:Uni-processor. relax C modules: ModuleCompiledFile type Path target_functions.relax_fitTrueELF 64-bit LSB shared object, AMD x86-64, version 1 (SYSV), not stripped
