Re: Using multi-processor for model_free

[Mahdi, Sam]

Also, Congrats to Edward!

On Thu, Sep 29, 2016 at 4:11 PM, Mahdi, Sam 
wrote:

> Hi Troels,
>
> I will upload a bug report with the pdb file, the script I use, and the
> data I'm using.
>
> Sincerely,
> Sam
>
> On Thu, Sep 29, 2016 at 2:25 PM, Troels Emtekær Linnet <
> tlin...@nmr-relax.com> wrote:
>
>> Hi Sam.
>>
>> Hm...
>>
>> I had a look in: pipe_control/minimise.py
>>
>> The trouble start with line
>> "elif values[i] in [None, {}, []]:"
>>
>> Where the index "i" is running out of bounds.
>>
>> The index is drawn from range(n).
>> n = len(names)
>> names = api.get_param_names(model_info)
>> values = api.get_param_values(model_info)
>>
>> So "something" is not aligned well in the data structures.
>> It seems that the index of parameter names exceeds parameter values.
>>
>>
>> One guess is, that the selection of
>> * diff_model
>> * mf_models
>> * local_tm_models
>>
>> is not correctly set. But I reach my limit of being able to help you.
>> Edward is the expert here, but he is on paternity leave.
>>
>>
>> Another possibility is that some of the spins are in "select" mode, where
>> they maybe should be in "deselect" mode.
>>
>> Maybe the spins do not carry any data from before, and somehow relax
>> expect this.
>>
>> It's very tricky to figure out!
>>
>> A bug report, some minimum data, and a script which make the bug occur
>> can solve this.
>>
>> Then I can write a systemtest, if it relax which is failing.
>>
>> Best
>> Troels
>>
>>
>>
>>
>> 2016-09-29 0:23 GMT+02:00 Mahdi, Sam :
>>
>>> Hi Troels,
>>> Update on both proteins: So for protein 1, I can upload all the spins (H
>>> and N), but then I recieve an error. This is the error I recieved for
>>> protein 2 as well. These are both dimer pdb files. Meaning they have 2 sets
>>> (set A) and set (B) (e.g. http://www.rcsb.org/pdb/explor
>>> e/explore.do?structureId=1DJ8 this pdb protein has 4 sets, A,B,C, and D
>>> ours only have A and B). For both these proteins I recieve this error
>>>  File "/home/crowlab/relax-4.0.2/multi/processor.py", line 494, in run
>>> self.callback.init_master(self)
>>>   File "/home/crowlab/relax-4.0.2/multi/__init__.py", line 318, in
>>> default_init_master
>>> self.master.run()
>>>   File "/home/crowlab/relax-4.0.2/relax.py", line 199, in run
>>> self.interpreter.run(self.script_file)
>>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 279, in
>>> run
>>> return run_script(intro=self.__intro_string, local=locals(),
>>> script_file=script_file, show_script=self.__show_script,
>>> raise_relax_error=self.__raise_relax_error)
>>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 585, in
>>> run_script
>>> return console.interact(intro, local, script_file,
>>> show_script=show_script, raise_relax_error=raise_relax_error)
>>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 484, in
>>> interact_script
>>> exec_script(script_file, local)
>>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 363, in
>>> exec_script
>>> runpy.run_module(module, globals)
>>>   File "/usr/lib64/python2.7/runpy.py", line 180, in run_module
>>> fname, loader, pkg_name)
>>>   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
>>> exec code in run_globals
>>>   File "/home/crowlab/relax-4.0.2/RGS4_modelfree_sample_script.py",
>>> line 31, in 
>>> dAuvergne_protocol(pipe_name=name,pipe_bundle=pipe_bundle,di
>>> ff_model=DIFF_MODEL,mf_models=MF_MODELS,local_tm_models=LOCA
>>> L_TM_MODELS,grid_inc=GRID_INC,min_algor=MIN_ALGOR,mc_sim_num
>>> =MC_NUM,conv_loop=CONV_LOOP)
>>>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
>>> line 246, in __init__
>>> self.execute()
>>>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
>>> line 600, in execute
>>> self.multi_model(local_tm=True)
>>>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
>>> line 888, in multi_model
>>> self.interpreter.minimise.grid_search(inc=self.grid_inc)
>>>   File "/home/crowlab/relax-4.0.2/prompt/uf_objects.py", line 225, in
>>> __call__
>>> self._backend(*new_args, **uf_kargs)
>>>   File "/home/crowlab/relax-4.0.2/pipe_control/minimise.py", line 172,
>>> in grid_search
>>> model_lower, model_upper, model_inc = grid_setup(lower, upper, inc,
>>> verbosity=verbosity, skip_preset=skip_preset)
>>>   File "/home/crowlab/relax-4.0.2/pipe_control/minimise.py", line 341,
>>> in grid_setup
>>> elif values[i] in [None, {}, []]:
>>> IndexError: index out of bounds
>>>
>>> Which from my understanding basically means, the co-ordinates of the
>>> spins are out of the acceptable range for relax. I've checked all the
>>> co-ordinates for both, nothing is extreme or outlandish (all within a range
>>> of -20 to 20).
>>> Is relax unable to process pdb files that are dimers (with 2 sets A and
>>> B).? Furthermore, is it unable to process trimers and tetramers?
>>>
>>> 

Re: Using multi-processor for model_free

[Mahdi, Sam]

Hi Troels,

I will upload a bug report with the pdb file, the script I use, and the
data I'm using.

Sincerely,
Sam

On Thu, Sep 29, 2016 at 2:25 PM, Troels Emtekær Linnet <
tlin...@nmr-relax.com> wrote:

> Hi Sam.
>
> Hm...
>
> I had a look in: pipe_control/minimise.py
>
> The trouble start with line
> "elif values[i] in [None, {}, []]:"
>
> Where the index "i" is running out of bounds.
>
> The index is drawn from range(n).
> n = len(names)
> names = api.get_param_names(model_info)
> values = api.get_param_values(model_info)
>
> So "something" is not aligned well in the data structures.
> It seems that the index of parameter names exceeds parameter values.
>
>
> One guess is, that the selection of
> * diff_model
> * mf_models
> * local_tm_models
>
> is not correctly set. But I reach my limit of being able to help you.
> Edward is the expert here, but he is on paternity leave.
>
>
> Another possibility is that some of the spins are in "select" mode, where
> they maybe should be in "deselect" mode.
>
> Maybe the spins do not carry any data from before, and somehow relax
> expect this.
>
> It's very tricky to figure out!
>
> A bug report, some minimum data, and a script which make the bug occur can
> solve this.
>
> Then I can write a systemtest, if it relax which is failing.
>
> Best
> Troels
>
>
>
>
> 2016-09-29 0:23 GMT+02:00 Mahdi, Sam :
>
>> Hi Troels,
>> Update on both proteins: So for protein 1, I can upload all the spins (H
>> and N), but then I recieve an error. This is the error I recieved for
>> protein 2 as well. These are both dimer pdb files. Meaning they have 2 sets
>> (set A) and set (B) (e.g. http://www.rcsb.org/pdb/explor
>> e/explore.do?structureId=1DJ8 this pdb protein has 4 sets, A,B,C, and D
>> ours only have A and B). For both these proteins I recieve this error
>>  File "/home/crowlab/relax-4.0.2/multi/processor.py", line 494, in run
>> self.callback.init_master(self)
>>   File "/home/crowlab/relax-4.0.2/multi/__init__.py", line 318, in
>> default_init_master
>> self.master.run()
>>   File "/home/crowlab/relax-4.0.2/relax.py", line 199, in run
>> self.interpreter.run(self.script_file)
>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 279, in
>> run
>> return run_script(intro=self.__intro_string, local=locals(),
>> script_file=script_file, show_script=self.__show_script,
>> raise_relax_error=self.__raise_relax_error)
>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 585, in
>> run_script
>> return console.interact(intro, local, script_file,
>> show_script=show_script, raise_relax_error=raise_relax_error)
>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 484, in
>> interact_script
>> exec_script(script_file, local)
>>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 363, in
>> exec_script
>> runpy.run_module(module, globals)
>>   File "/usr/lib64/python2.7/runpy.py", line 180, in run_module
>> fname, loader, pkg_name)
>>   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
>> exec code in run_globals
>>   File "/home/crowlab/relax-4.0.2/RGS4_modelfree_sample_script.py", line
>> 31, in 
>> dAuvergne_protocol(pipe_name=name,pipe_bundle=pipe_bundle,di
>> ff_model=DIFF_MODEL,mf_models=MF_MODELS,local_tm_models=LOCA
>> L_TM_MODELS,grid_inc=GRID_INC,min_algor=MIN_ALGOR,mc_sim_
>> num=MC_NUM,conv_loop=CONV_LOOP)
>>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
>> line 246, in __init__
>> self.execute()
>>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
>> line 600, in execute
>> self.multi_model(local_tm=True)
>>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
>> line 888, in multi_model
>> self.interpreter.minimise.grid_search(inc=self.grid_inc)
>>   File "/home/crowlab/relax-4.0.2/prompt/uf_objects.py", line 225, in
>> __call__
>> self._backend(*new_args, **uf_kargs)
>>   File "/home/crowlab/relax-4.0.2/pipe_control/minimise.py", line 172,
>> in grid_search
>> model_lower, model_upper, model_inc = grid_setup(lower, upper, inc,
>> verbosity=verbosity, skip_preset=skip_preset)
>>   File "/home/crowlab/relax-4.0.2/pipe_control/minimise.py", line 341,
>> in grid_setup
>> elif values[i] in [None, {}, []]:
>> IndexError: index out of bounds
>>
>> Which from my understanding basically means, the co-ordinates of the
>> spins are out of the acceptable range for relax. I've checked all the
>> co-ordinates for both, nothing is extreme or outlandish (all within a range
>> of -20 to 20).
>> Is relax unable to process pdb files that are dimers (with 2 sets A and
>> B).? Furthermore, is it unable to process trimers and tetramers?
>>
>> Sincerely,
>> Sam
>>
>> On Wed, Sep 28, 2016 at 1:44 PM, Mahdi, Sam 
>> wrote:
>>
>>> Hey Troels,
>>>
>>> I ran the relax -x and recieve this error at the GUI tests
>>> =
>>> = GUI tests =
>>> 

Re: Using multi-processor for model_free

[Troels Emtekær Linnet]

Hi Sam.

Hm...

I had a look in: pipe_control/minimise.py

The trouble start with line
"elif values[i] in [None, {}, []]:"

Where the index "i" is running out of bounds.

The index is drawn from range(n).
n = len(names)
names = api.get_param_names(model_info)
values = api.get_param_values(model_info)

So "something" is not aligned well in the data structures.
It seems that the index of parameter names exceeds parameter values.


One guess is, that the selection of
* diff_model
* mf_models
* local_tm_models

is not correctly set. But I reach my limit of being able to help you.
Edward is the expert here, but he is on paternity leave.


Another possibility is that some of the spins are in "select" mode, where
they maybe should be in "deselect" mode.

Maybe the spins do not carry any data from before, and somehow relax expect
this.

It's very tricky to figure out!

A bug report, some minimum data, and a script which make the bug occur can
solve this.

Then I can write a systemtest, if it relax which is failing.

Best
Troels




2016-09-29 0:23 GMT+02:00 Mahdi, Sam :

> Hi Troels,
> Update on both proteins: So for protein 1, I can upload all the spins (H
> and N), but then I recieve an error. This is the error I recieved for
> protein 2 as well. These are both dimer pdb files. Meaning they have 2 sets
> (set A) and set (B) (e.g. http://www.rcsb.org/pdb/explor
> e/explore.do?structureId=1DJ8 this pdb protein has 4 sets, A,B,C, and D
> ours only have A and B). For both these proteins I recieve this error
>  File "/home/crowlab/relax-4.0.2/multi/processor.py", line 494, in run
> self.callback.init_master(self)
>   File "/home/crowlab/relax-4.0.2/multi/__init__.py", line 318, in
> default_init_master
> self.master.run()
>   File "/home/crowlab/relax-4.0.2/relax.py", line 199, in run
> self.interpreter.run(self.script_file)
>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 279, in run
> return run_script(intro=self.__intro_string, local=locals(),
> script_file=script_file, show_script=self.__show_script,
> raise_relax_error=self.__raise_relax_error)
>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 585, in
> run_script
> return console.interact(intro, local, script_file,
> show_script=show_script, raise_relax_error=raise_relax_error)
>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 484, in
> interact_script
> exec_script(script_file, local)
>   File "/home/crowlab/relax-4.0.2/prompt/interpreter.py", line 363, in
> exec_script
> runpy.run_module(module, globals)
>   File "/usr/lib64/python2.7/runpy.py", line 180, in run_module
> fname, loader, pkg_name)
>   File "/usr/lib64/python2.7/runpy.py", line 72, in _run_code
> exec code in run_globals
>   File "/home/crowlab/relax-4.0.2/RGS4_modelfree_sample_script.py", line
> 31, in 
> dAuvergne_protocol(pipe_name=name,pipe_bundle=pipe_bundle,di
> ff_model=DIFF_MODEL,mf_models=MF_MODELS,local_tm_models=
> LOCAL_TM_MODELS,grid_inc=GRID_INC,min_algor=MIN_ALGOR,mc_
> sim_num=MC_NUM,conv_loop=CONV_LOOP)
>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
> line 246, in __init__
> self.execute()
>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
> line 600, in execute
> self.multi_model(local_tm=True)
>   File "/home/crowlab/relax-4.0.2/auto_analyses/dauvergne_protocol.py",
> line 888, in multi_model
> self.interpreter.minimise.grid_search(inc=self.grid_inc)
>   File "/home/crowlab/relax-4.0.2/prompt/uf_objects.py", line 225, in
> __call__
> self._backend(*new_args, **uf_kargs)
>   File "/home/crowlab/relax-4.0.2/pipe_control/minimise.py", line 172, in
> grid_search
> model_lower, model_upper, model_inc = grid_setup(lower, upper, inc,
> verbosity=verbosity, skip_preset=skip_preset)
>   File "/home/crowlab/relax-4.0.2/pipe_control/minimise.py", line 341, in
> grid_setup
> elif values[i] in [None, {}, []]:
> IndexError: index out of bounds
>
> Which from my understanding basically means, the co-ordinates of the spins
> are out of the acceptable range for relax. I've checked all the
> co-ordinates for both, nothing is extreme or outlandish (all within a range
> of -20 to 20).
> Is relax unable to process pdb files that are dimers (with 2 sets A and
> B).? Furthermore, is it unable to process trimers and tetramers?
>
> Sincerely,
> Sam
>
> On Wed, Sep 28, 2016 at 1:44 PM, Mahdi, Sam 
> wrote:
>
>> Hey Troels,
>>
>> I ran the relax -x and recieve this error at the GUI tests
>> =
>> = GUI tests =
>> =
>>
>> **
>> Gtk:ERROR:gtkfilesystemmodel.c:746:gtk_file_system_model_sort: assertion
>> failed: (r == n_visible_rows)
>> Abort (core dumped)
>> crowlab: [~/relax-4.0.2]>
>>
>>
>> On Wed, Sep 28, 2016 at 1:30 PM, Mahdi, Sam 
>> wrote:
>>
>>> Hi Troels,
>>>
>>> An update on protein number 1: I have successfully 

Re: Using multi-processor for model_free

[Troels Emtekær Linnet]

Hi Sam.

Problems with the GUI do not necessary mean a problem.

It could be a problem of wxPython.
Which version do you have? (relax -i )

To find the specific tests that fails, can you do:
> relax --gui-tests --time -t gui_fail.txt

Best
Troels


2016-09-28 22:44 GMT+02:00 Mahdi, Sam :

> Hey Troels,
>
> I ran the relax -x and recieve this error at the GUI tests
> =
> = GUI tests =
> =
>
> **
> Gtk:ERROR:gtkfilesystemmodel.c:746:gtk_file_system_model_sort: assertion
> failed: (r == n_visible_rows)
> Abort (core dumped)
> crowlab: [~/relax-4.0.2]>
>
>
> On Wed, Sep 28, 2016 at 1:30 PM, Mahdi, Sam 
> wrote:
>
>> Hi Troels,
>>
>> An update on protein number 1: I have successfully resolved the problem.
>> Initially the pdb file had HN instead of just H for the backbone hydrogens.
>> So it couldn't read it. I changed all the HN to H. Then I recieved the
>> error
>> RelaxError: Multiple alternate location indicators are present in the PDB
>> file, but the desired coordinate set has not been specified
>> By removing the extra N, all the text for the 3D location (the
>> co-ordinates) for the HN were shifted a space (no longer aligned). Once I
>> aligned them all, relax was able to read all the spins. So its working now.
>> I'm currently running the test suite as well.
>>
>> Sincerely,
>> Sam
>>
>> On Wed, Sep 28, 2016 at 11:45 AM, Troels Emtekær Linnet <
>> tlin...@nmr-relax.com> wrote:
>>
>>> To test the speed difference between script and GUI,
>>> you could try to run the full test-suite through the terminal or
>>> inside the GUI.
>>>
>>> That should give you a clue about time difference.
>>>
>>>
>>> 2016-09-28 20:32 GMT+02:00 Troels Emtekær Linnet 
>>> :
>>>
 If you get different results, for the same setup, this is not good.
 Not at all !

 Have you run the full relax test suite after installation?

 http://wiki.nmr-relax.com/Installation_test

 run it with:
 relax -x

 This takes about 1 Hour to run, and should not be used with multiple
 processors.

 Relax will test itself with thousands of unit tests and system tests,
 and confer that all
 results are the same.

 If the system tests do not pass on each system, something fishy is
 going on.

 This is the best line of defence against "systems" acting weird due to
 software/packages etc. etc.

 Best
 Troels



 2016-09-28 9:44 GMT+02:00 Mahdi, Sam :

> Hi Troels,
>
> I wanted to give a bit of feedback on the results I've obtained
> throughout the few weeks I've been using model free on relax. First off,
> thank you guys (both you and Edward) immensly for your patience and help 
> as
> I attempted to understand and work relax. Secondly, I have noticed a
> difference between using the gui and the terminal (using scripts to run
> relax). I've currently finished about 3 runs using the gui, and 3 runs
> using the terminal (all the same data sets, same pdb files, same settings,
> etc.). The gui takes about a week to finish, where the terminal takes
> approximately 24 hours. I've tried this on 2 proteins, both had the same
> results. The terminal is by far, much faster than the gui. Finally, I've
> run 1 protein on 2 different computers (one using the multi-processor
> platform, and on another computer, single-processor). The data sets were
> all the same, the same pdb file, etc. , but the results I obtained from 
> the
> computers were slightly different. For the most part, most of the
> difference in the data was similar, slightly different, but within the
> error. But there were about 7 or 8 data points that appeared in one run on
> one computer, and were absent in another run on another computer. This
> happened in both the S^2 I analyzed and the Rex.
> I.e. On the fedora 20 (single processor), say I had S^2 values for
> amino acid 24,25 and 26 in the sequence, but not for 28,29, and 30. On the
> fedora 24 (multi-processor), I might be missing a value for amino acid 24,
> but I would have S^2 values for 28,29 and 30. Note the data sets are all
> the same, the pdb files the same, settings the same, I used the same 
> script
> for both. The only difference between these runs is they were run on
> different computer and one was single processor well another was multi.
> I don't know why I obtained different data from 2 different runs, when
> the input was all the same, just on different computers.
> However the S^2 values do make sense. The Rex values were incredibly
> small (1x10^-20), but there are some similarities (in terms of big Rex
> values) between the Rex I obtained from relax, and CPMG data analyzed by
> glove. So I have been able to obtain some