Re: [OMPI devel] Q: project based MCA param files

[Jeff Squyres (jsquyres)]

On May 7, 2013, at 2:44 PM, Thomas Naughton  wrote:

> Ok, looks like this may just do the trick.  We briefly discussed this today
> and probably can change our use case to make use of this mechanism instead
> and avoid any further enhancments.
> 
> Question: If you do a setenv for this MCA param, does that extend the
>   default search path?  Or does it replace/override the default?

It replaces/overrides.

You don't have to do this via setenv-- you should be able to do it 
programatically, too.

> Thanks Jeff for forwarding info to devel list to get broader feedback, and
> to Ralph for providing the suggestion.
> 
> --tjn
> 
> _
>  Thomas Naughton  naught...@ornl.gov
>  Research Associate   (865) 576-4184
> 
> 
> On Tue, 7 May 2013, Ralph Castain wrote:
> 
>> I believe we already have a way of defining where to get the default mca 
>> params:
>> 
>>   ret = mca_base_var_register ("opal", "mca", "base", "param_files", "Path 
>> for MCA "
>>"configuration files containing variable 
>> values",
>>MCA_BASE_VAR_TYPE_STRING, NULL, 0, 0, 
>> OPAL_INFO_LVL_2,
>>MCA_BASE_VAR_SCOPE_READONLY, 
>> _base_var_files);
>> 
>> 
>> So wouldn't it be as easy as defining an envar? It's what we did when using 
>> the OMPI code with ORCM a couple of years ago, and we used it again for a 
>> recent project in Greenplum where the default mca param was specified in a 
>> different location than usual.
>> 
>> 
>> On May 7, 2013, at 6:28 AM, Jeff Squyres (jsquyres)  
>> wrote:
>> 
>>> Given Ralph's questions about 
>>> rhttps://svn.open-mpi.org/trac/ompi/changeset/28456, ORNL's second question 
>>> to me/Nathan about MCA params is probably worth forwarding to the list -- 
>>> see below.
>>> 
>>> Thoughts on this proposal?
>>> 
>>> 
>>> Begin forwarded message:
>>> 
 From: "Boehm, Swen" 
 Subject: Re: Q: project based MCA param files
 Date: May 3, 2013 5:03:43 PM EDT
 To: "Jeff Squyres (jsquyres)" 
 Cc: Nathan Hjelm , "Vallee, Geoffroy R." 
 , "Naughton III, Thomas J." 
 
 Hi Jeff,
 
 Here is a short description of the enhancement we would like to contribute.
 Let us know what you think.
 
 The purpose of the suggested improvements is to enable "projects" to read
 MCA parameters from project specific locations. This enables the usage
 of OPAL and the MCA Frameworks outside the OpenMPI project without
 interfering with OpenMPI specific parameters and removes the need to
 patch OPAL (e.g., to pick up params from different locations).
 
 The possible scenarios would be the following:
 
 a) adding the option to pick up a project specific mca-param.conf file
 Example:
  $HOME/.mca/${project}-mca-param.conf
  and /etc/mca/${project}-mca-param.conf)
 
 b) add the option to pick up the mca-param.conf file from a project 
 specific
 directory
 Example:
  $HOME/.${project}/mca-param.conf
  and /etc/${project}/mca-param.conf
  and/or  /etc/${project}/${project}-mca-param.conf)
 
 c) prefixing the mca param with the project name in the existing 
 mca-param.conf
 file and therefore following the new MCA variable system naming scheme.
 Example:
  mca_${project}_${framework}_${component}_${var_name}
 
 The implementation has to be compatible with the current system, that is,
 it should work as it does today without any added burden to the user. The
 suggested approach is to provide an addition to the MCA API (something like
 mca_base_add_config_file_path ()) to add lookup paths to the MCA system.
 This way additional files can be picked up for the MCA param parsing if
 needed.
 
 To wrap it up:
 1) Is the motivation clear?
 2) Is it possible to implement the desired capability within a
  reasonable time and without changing the current behavior?
 3) Does it line up with the planning / future capabilities?
 4) Which of the above options (A, B, C) would you prefer?
 
 --
 Swen Boehm  | Email: bo...@ornl.gov
 Oak Ridge National Laboratory  | Phone: +1 865-576-6125
 
 
 On Apr 26, 2013, at 7:50 PM, Thomas Naughton  wrote:
 
> Hi,
> 
> Ok, sounds good. We'll check on this next week and get back to you.
> 
> Thanks,
> --tjn
> 
> _
> Thomas Naughton  naught...@ornl.gov
> Research Associate 

Re: [OMPI devel] Q: project based MCA param files

[Thomas Naughton]

Hi,

Ok, looks like this may just do the trick.  We briefly discussed this today
and probably can change our use case to make use of this mechanism instead
and avoid any further enhancments.

 Question: If you do a setenv for this MCA param, does that extend the
   default search path?  Or does it replace/override the default?

Thanks Jeff for forwarding info to devel list to get broader feedback, and
to Ralph for providing the suggestion.

--tjn

 _
  Thomas Naughton  naught...@ornl.gov
  Research Associate   (865) 576-4184


On Tue, 7 May 2013, Ralph Castain wrote:


I believe we already have a way of defining where to get the default mca params:

   ret = mca_base_var_register ("opal", "mca", "base", "param_files", "Path for MCA 
"
"configuration files containing variable 
values",
MCA_BASE_VAR_TYPE_STRING, NULL, 0, 0, 
OPAL_INFO_LVL_2,
MCA_BASE_VAR_SCOPE_READONLY, 
_base_var_files);


So wouldn't it be as easy as defining an envar? It's what we did when using the 
OMPI code with ORCM a couple of years ago, and we used it again for a recent 
project in Greenplum where the default mca param was specified in a different 
location than usual.


On May 7, 2013, at 6:28 AM, Jeff Squyres (jsquyres)  wrote:


Given Ralph's questions about 
rhttps://svn.open-mpi.org/trac/ompi/changeset/28456, ORNL's second question to 
me/Nathan about MCA params is probably worth forwarding to the list -- see 
below.

Thoughts on this proposal?


Begin forwarded message:


From: "Boehm, Swen" 
Subject: Re: Q: project based MCA param files
Date: May 3, 2013 5:03:43 PM EDT
To: "Jeff Squyres (jsquyres)" 
Cc: Nathan Hjelm , "Vallee, Geoffroy R." , "Naughton 
III, Thomas J." 

Hi Jeff,

Here is a short description of the enhancement we would like to contribute.
Let us know what you think.

The purpose of the suggested improvements is to enable "projects" to read
MCA parameters from project specific locations. This enables the usage
of OPAL and the MCA Frameworks outside the OpenMPI project without
interfering with OpenMPI specific parameters and removes the need to
patch OPAL (e.g., to pick up params from different locations).

The possible scenarios would be the following:

a) adding the option to pick up a project specific mca-param.conf file
 Example:
  $HOME/.mca/${project}-mca-param.conf
  and /etc/mca/${project}-mca-param.conf)

b) add the option to pick up the mca-param.conf file from a project specific
 directory
 Example:
  $HOME/.${project}/mca-param.conf
  and /etc/${project}/mca-param.conf
  and/or  /etc/${project}/${project}-mca-param.conf)

c) prefixing the mca param with the project name in the existing mca-param.conf
 file and therefore following the new MCA variable system naming scheme.
 Example:
  mca_${project}_${framework}_${component}_${var_name}

The implementation has to be compatible with the current system, that is,
it should work as it does today without any added burden to the user. The
suggested approach is to provide an addition to the MCA API (something like
mca_base_add_config_file_path ()) to add lookup paths to the MCA system.
This way additional files can be picked up for the MCA param parsing if
needed.

To wrap it up:
1) Is the motivation clear?
2) Is it possible to implement the desired capability within a
  reasonable time and without changing the current behavior?
3) Does it line up with the planning / future capabilities?
4) Which of the above options (A, B, C) would you prefer?

--
Swen Boehm  | Email: bo...@ornl.gov
Oak Ridge National Laboratory  | Phone: +1 865-576-6125


On Apr 26, 2013, at 7:50 PM, Thomas Naughton  wrote:


Hi,

Ok, sounds good. We'll check on this next week and get back to you.

Thanks,
--tjn

_
Thomas Naughton  naught...@ornl.gov
Research Associate   (865) 576-4184


On Fri, 26 Apr 2013, Jeff Squyres (jsquyres) wrote:


Email would probably be easiest -- I will need to page in/refresh this area of 
the code, anyway, so if you guys do the initial homework and submit some ideas, 
that would probably be easiest (For me).  :-D


On Apr 26, 2013, at 6:33 PM, Thomas Naughton 
wrote:


Hi Jeff,

We don't have one yet but we can code something up and submit a patch.

If useful we could quickly sync up beforehand to ensure we are on the same 
page.   Phone or email, whatever would be easiest.

What do you think?
--tjn

Re: [OMPI devel] Q: project based MCA param files

[Ralph Castain]

I believe we already have a way of defining where to get the default mca params:

ret = mca_base_var_register ("opal", "mca", "base", "param_files", "Path 
for MCA "
 "configuration files containing variable 
values",
 MCA_BASE_VAR_TYPE_STRING, NULL, 0, 0, 
OPAL_INFO_LVL_2,
 MCA_BASE_VAR_SCOPE_READONLY, 
_base_var_files);


So wouldn't it be as easy as defining an envar? It's what we did when using the 
OMPI code with ORCM a couple of years ago, and we used it again for a recent 
project in Greenplum where the default mca param was specified in a different 
location than usual.


On May 7, 2013, at 6:28 AM, Jeff Squyres (jsquyres)  wrote:

> Given Ralph's questions about 
> rhttps://svn.open-mpi.org/trac/ompi/changeset/28456, ORNL's second question 
> to me/Nathan about MCA params is probably worth forwarding to the list -- see 
> below.
> 
> Thoughts on this proposal?
> 
> 
> Begin forwarded message:
> 
>> From: "Boehm, Swen" 
>> Subject: Re: Q: project based MCA param files
>> Date: May 3, 2013 5:03:43 PM EDT
>> To: "Jeff Squyres (jsquyres)" 
>> Cc: Nathan Hjelm , "Vallee, Geoffroy R." 
>> , "Naughton III, Thomas J." 
>> 
>> Hi Jeff,
>> 
>> Here is a short description of the enhancement we would like to contribute.
>> Let us know what you think.
>> 
>> The purpose of the suggested improvements is to enable "projects" to read
>> MCA parameters from project specific locations. This enables the usage
>> of OPAL and the MCA Frameworks outside the OpenMPI project without
>> interfering with OpenMPI specific parameters and removes the need to 
>> patch OPAL (e.g., to pick up params from different locations).
>> 
>> The possible scenarios would be the following:
>> 
>> a) adding the option to pick up a project specific mca-param.conf file
>>  Example:
>>   $HOME/.mca/${project}-mca-param.conf
>>   and /etc/mca/${project}-mca-param.conf)
>> 
>> b) add the option to pick up the mca-param.conf file from a project specific
>>  directory
>>  Example:
>>   $HOME/.${project}/mca-param.conf
>>   and /etc/${project}/mca-param.conf
>>   and/or  /etc/${project}/${project}-mca-param.conf)
>> 
>> c) prefixing the mca param with the project name in the existing 
>> mca-param.conf
>>  file and therefore following the new MCA variable system naming scheme.
>>  Example:
>>   mca_${project}_${framework}_${component}_${var_name}
>> 
>> The implementation has to be compatible with the current system, that is, 
>> it should work as it does today without any added burden to the user. The
>> suggested approach is to provide an addition to the MCA API (something like
>> mca_base_add_config_file_path ()) to add lookup paths to the MCA system.
>> This way additional files can be picked up for the MCA param parsing if
>> needed.
>> 
>> To wrap it up:
>> 1) Is the motivation clear?
>> 2) Is it possible to implement the desired capability within a
>>   reasonable time and without changing the current behavior?
>> 3) Does it line up with the planning / future capabilities?
>> 4) Which of the above options (A, B, C) would you prefer?
>> 
>> -- 
>> Swen Boehm  | Email: bo...@ornl.gov
>> Oak Ridge National Laboratory  | Phone: +1 865-576-6125
>> 
>> 
>> On Apr 26, 2013, at 7:50 PM, Thomas Naughton  wrote:
>> 
>>> Hi,
>>> 
>>> Ok, sounds good. We'll check on this next week and get back to you.
>>> 
>>> Thanks,
>>> --tjn
>>> 
>>> _
>>> Thomas Naughton  naught...@ornl.gov
>>> Research Associate   (865) 576-4184
>>> 
>>> 
>>> On Fri, 26 Apr 2013, Jeff Squyres (jsquyres) wrote:
>>> 
 Email would probably be easiest -- I will need to page in/refresh this 
 area of the code, anyway, so if you guys do the initial homework and 
 submit some ideas, that would probably be easiest (For me).  :-D
 
 
 On Apr 26, 2013, at 6:33 PM, Thomas Naughton 
 wrote:
 
> Hi Jeff,
> 
> We don't have one yet but we can code something up and submit a patch.
> 
> If useful we could quickly sync up beforehand to ensure we are on the 
> same page.   Phone or email, whatever would be easiest.
> 
> What do you think?
> --tjn
> 
> _
> Thomas Naughton  naught...@ornl.gov
> Research Associate   (865) 576-4184
> 
> 
> On Fri, 26 Apr 2013, Jeff Squyres (jsquyres) wrote:
> 
>> I'm not aware of any plans to do this.
>> 
>> Do you guys have a patch to submit, perchance?  :-D
>>