Re: [OMPI users] How to keep multiple installations at same time

[Gus Correa]

Thanks for not saying this turn that Environment Modules
is "outdated and not maintained".
That might mislead the OMPI list audience,
which has a big intersection with Environment Modules users.

On May 16, 2014, at 4:07 PM, Maxime Boissonneault 
 wrote:

> Instead of using the outdated and not maintained Module environment,
> why not use Lmod : https://www.tacc.utexas.edu/tacc-projects/lmod
>
> It is a drop-in replacement for Module environment that supports all
> of their features and much, much more, such as :
> - module hierarchies
> - module properties and color highlighting (we use it to higlight
> bioinformatic modules or tools for example)
> - module caching (very useful for a parallel filesystem with
> tons of modules)
> - path priorities (useful to make sure personal modules
> take precendence over system modules)
> - export module tree to json
>
> It works like a charm, understand both TCL and Lua modules
> and is  actively developped and debugged. There are litteraly new
> features every month or so. If it does not do what you want, odds are
> that the developper will add it shortly (I've had it happen).
>
> Maxime
>

Some of the features introduced by LMod seem to be in the
works in the Environment modules as well (cache, hierarchies),
judging from recent mailing list postings.

I am not sure all of them are essential, though.
For instance, I am skeptical that most users,
which are scientifically trained in one way or another,
would not understand the stack nature of the the enviroment modules
and make mistakes like the one you mentioned:

> module load gcc openmpi_gcc
> module unload gcc
> module load intel

Unless they were never told about it.

Hierarchies may simplify the naming convention, but may also work
as a straitjacket, reflecting the sys admin hierarchical choices,
but potentially limiting the user choices.
In one national lab that I have access to,
navigating their module hierarchy, and specially getting around the
official one to do what you need/want, is a pain.

Anyway, this is the OMPI list, not a place for advocacy of either
package, so I am going to stop here.

I just wanted to set the record straight that:

- the Enviroment Modules package is not dead,
- it has a large user base, and
- it is sooo good that among other things it opened the road for
the imitators!  :)

Thank you,
Gus Correa

On 08/05/2014 03:51 PM, Maxime Boissonneault wrote:



The Environment Modules package user base is not negligible,
including many universities, research centers, national labs,
ans private companies, in the US and around the world.
How does the user base of LMod compare?


The user base certainly is much larger for Environment Modules than LMod.
But, as a user of both Lmod and Environment Modules, I can tell you the
following :

Regardless of any virtues that LMod may have,
currently I don't see any reason to switch to LMod,
install everything over again

Nothing needs reinstalling. Lmod understands Tcl modules and can work
fine with your old module tree.

, troubleshoot it,
learn Lua, migrate my modules from Tcl,

Again, migration to Lua is not required. Tcl modules gets converted on
the fly.

educate my users and convince them to use a new
package to achieve the same exact thing that they currently have,

Very little education has to be done. The commands are the same :
module avail
module load/add
module unload/remove
module use
...

and in the end gain little if any
relevant/useful/new functionality.

If you do not want to make any changes, in the way you organize modules,
then don't. You will also get no benefit from changing to Lmod in that
situation.

If you do want to use new features, then there are plenty. Most notably is
- the possibility to organize modules in hierarchy   (which you do not
HAVE to do, but in my opinion, is much more intuitive).
- the possibility to cache the module structure (and avoid reading it
from a parallel filesystem every time a user type a module command).
- the possibility to color-code modules so that users can find what they
want easier out of hundreds of modules

IF you do use hierarchy, you get the added benefit of avoiding user
mistakes such as

"
module load gcc openmpi_gcc
module unload gcc
module load intel

... why is my MPI not working!
"

IF you do use hierarchy, you get the added benefit of not having silly
module names such as
fftw/3.3_gcc4.8_openmpi1.6.3
fftw/3.3_gcc4.6_openmpi1.8.1
...

Again, you do NOT have to, but the benefits much outweight the changes
that need to be made to get them.

My 2 cents,

Maxime Boissonneault



My two cents of opinion
Gus Correa


On 08/05/2014 12:54 PM, Ralph Castain wrote:

Check the repo - hasn't been touched in a very long time

On Aug 5, 2014, at 9:42 AM, Fabricio Cannini  wrote:


On 05-08-2014 13:10, Ralph Castain wrote:

Since modules isn't a supported s/w package any more, you might
consider using LMOD instead:

Re: [OMPI users] How to keep multiple installations at same time

[Maxime Boissonneault]

The Environment Modules package user base is not negligible,
including many universities, research centers, national labs,
ans private companies, in the US and around the world.
How does the user base of LMod compare?


The user base certainly is much larger for Environment Modules than LMod.
But, as a user of both Lmod and Environment Modules, I can tell you the 
following :

Regardless of any virtues that LMod may have,
currently I don't see any reason to switch to LMod,
install everything over again
Nothing needs reinstalling. Lmod understands Tcl modules and can work 
fine with your old module tree.

, troubleshoot it,
learn Lua, migrate my modules from Tcl,
Again, migration to Lua is not required. Tcl modules gets converted on 
the fly.

educate my users and convince them to use a new
package to achieve the same exact thing that they currently have,

Very little education has to be done. The commands are the same :
module avail
module load/add
module unload/remove
module use
...

and in the end gain little if any
relevant/useful/new functionality.
If you do not want to make any changes, in the way you organize modules, 
then don't. You will also get no benefit from changing to Lmod in that 
situation.


If you do want to use new features, then there are plenty. Most notably is
- the possibility to organize modules in hierarchy   (which you do not 
HAVE to do, but in my opinion, is much more intuitive).
- the possibility to cache the module structure (and avoid reading it 
from a parallel filesystem every time a user type a module command).
- the possibility to color-code modules so that users can find what they 
want easier out of hundreds of modules


IF you do use hierarchy, you get the added benefit of avoiding user 
mistakes such as


"
module load gcc openmpi_gcc
module unload gcc
module load intel

... why is my MPI not working!
"

IF you do use hierarchy, you get the added benefit of not having silly 
module names such as

fftw/3.3_gcc4.8_openmpi1.6.3
fftw/3.3_gcc4.6_openmpi1.8.1
...

Again, you do NOT have to, but the benefits much outweight the changes 
that need to be made to get them.


My 2 cents,

Maxime Boissonneault



My two cents of opinion
Gus Correa


On 08/05/2014 12:54 PM, Ralph Castain wrote:

Check the repo - hasn't been touched in a very long time

On Aug 5, 2014, at 9:42 AM, Fabricio Cannini  wrote:


On 05-08-2014 13:10, Ralph Castain wrote:
Since modules isn't a supported s/w package any more, you might 
consider using LMOD instead:


https://www.tacc.utexas.edu/tacc-projects/lmod


Modules isn't supported anymore? :O

Could you please send a link about it ?
___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2014/08/24918.php


___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2014/08/24919.php




___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2014/08/24924.php



--
-
Maxime Boissonneault
Analyste de calcul - Calcul Québec, Université Laval
Ph. D. en physique

Re: [OMPI users] How to keep multiple installations at same time

[Ralph Castain]

Nobody was trying to make a statement here, Gus - I was only quoting something 
based on the last known repo of which I was aware. If the modules folks moved 
to Github, then they might want to make that a little more obvious. :-/

Lmod vs modules - I couldn't care less. My understanding is that TACC wrote 
their own because of the perception that modules was no longer active, but I 
can't say for sure :-)


On Aug 5, 2014, at 11:50 AM, Gus Correa  wrote:

> Hi Ralph and list
> 
> I am no developer, but my impression is that,
> paraphrasing Mark Twain,
> the reports about the death of the
> Environment Modules (http://modules.sourceforge.net/)
> package have been exaggerated.
> That presumed death/obsolescence has been
> repeated a few times on the OMPI list,
> which has a big intersection with the Environment Modules
> actual and potential users,
> a statement which IMHO is inaccurate, to say the least.
> 
> The Environment Modules repository has 2014 entries:
> 
> http://sourceforge.net/p/modules/git/ci/master/tree/
> 
> The Environment Modules mailing list is active as well:
> 
> https://lists.sourceforge.net/lists/listinfo/modules-interest
> http://sourceforge.net/p/modules/mailman/modules-interest/
> 
> The Environment Modules package user base is not negligible,
> including many universities, research centers, national labs,
> ans private companies, in the US and around the world.
> How does the user base of LMod compare?
> 
> Regardless of any virtues that LMod may have,
> currently I don't see any reason to switch to LMod,
> install everything over again, troubleshoot it,
> learn Lua, migrate my modules from Tcl,
> educate my users and convince them to use a new
> package to achieve the same exact thing that they currently have,
> and in the end gain little if any
> relevant/useful/new functionality.
> 
> My two cents of opinion
> Gus Correa
> 
> 
> On 08/05/2014 12:54 PM, Ralph Castain wrote:
>> Check the repo - hasn't been touched in a very long time
>> 
>> On Aug 5, 2014, at 9:42 AM, Fabricio Cannini  wrote:
>> 
>>> On 05-08-2014 13:10, Ralph Castain wrote:
 Since modules isn't a supported s/w package any more, you might consider 
 using LMOD instead:
 
 https://www.tacc.utexas.edu/tacc-projects/lmod
>>> 
>>> Modules isn't supported anymore? :O
>>> 
>>> Could you please send a link about it ?
>>> ___
>>> users mailing list
>>> us...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> Link to this post: 
>>> http://www.open-mpi.org/community/lists/users/2014/08/24918.php
>> 
>> ___
>> users mailing list
>> us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post: 
>> http://www.open-mpi.org/community/lists/users/2014/08/24919.php
>> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2014/08/24924.php

Re: [OMPI users] How to keep multiple installations at same time

[Gus Correa]

Hi Ralph and list

I am no developer, but my impression is that,
paraphrasing Mark Twain,
the reports about the death of the
Environment Modules (http://modules.sourceforge.net/)
package have been exaggerated.
That presumed death/obsolescence has been
repeated a few times on the OMPI list,
which has a big intersection with the Environment Modules
actual and potential users,
a statement which IMHO is inaccurate, to say the least.

The Environment Modules repository has 2014 entries:

http://sourceforge.net/p/modules/git/ci/master/tree/

The Environment Modules mailing list is active as well:

https://lists.sourceforge.net/lists/listinfo/modules-interest
http://sourceforge.net/p/modules/mailman/modules-interest/

The Environment Modules package user base is not negligible,
including many universities, research centers, national labs,
ans private companies, in the US and around the world.
How does the user base of LMod compare?

Regardless of any virtues that LMod may have,
currently I don't see any reason to switch to LMod,
install everything over again, troubleshoot it,
learn Lua, migrate my modules from Tcl,
educate my users and convince them to use a new
package to achieve the same exact thing that they currently have,
and in the end gain little if any
relevant/useful/new functionality.

My two cents of opinion
Gus Correa


On 08/05/2014 12:54 PM, Ralph Castain wrote:

Check the repo - hasn't been touched in a very long time

On Aug 5, 2014, at 9:42 AM, Fabricio Cannini  wrote:


On 05-08-2014 13:10, Ralph Castain wrote:

Since modules isn't a supported s/w package any more, you might consider using 
LMOD instead:

https://www.tacc.utexas.edu/tacc-projects/lmod


Modules isn't supported anymore? :O

Could you please send a link about it ?
___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2014/08/24918.php


___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2014/08/24919.php

Re: [OMPI users] How to keep multiple installations at same time

[David Turner]

I checked with my colleague, who is one of the
module developers.  His response:

> That's a surprise to me?!
> I will admit that I'm a little slow on releases,
> but it's still quite active.

On 8/5/14 11:39 AM, Fabricio Cannini wrote:

On 05-08-2014 13:54, Ralph Castain wrote:

Check the repo - hasn't been touched in a very long time


Yes, the cvs repo hasn't been touched in a long long time, but they have
apparently migrated to git.

cvs:
http://modules.cvs.sourceforge.net/viewvc/modules/

git:
http://sourceforge.net/p/modules/git/ci/master/tree/


There is still activity on git, patches for newest tcl version.
It may not be bursting, but I wouldn't call it "dead". Yet. ;)

[ ]'s
___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2014/08/24921.php



--
Best regards,

David Turner
User Services Groupemail: dptur...@lbl.gov
NERSC Division phone: (510) 486-4027
Lawrence Berkeley Labfax: (510) 486-4316

Re: [OMPI users] How to keep multiple installations at same time

[Bennet Fauber]

There is also the Lmod project, based at TACC, and run by Robert McLay.

https://www.tacc.utexas.edu/tacc-projects/lmod

That's under current, active development, and if you'd be creating a
brand new modules infrastructure, bears a close look.

-- bennet


On Tue, Aug 5, 2014 at 2:39 PM, Fabricio Cannini  wrote:
> On 05-08-2014 13:54, Ralph Castain wrote:
>>
>> Check the repo - hasn't been touched in a very long time
>
>
> Yes, the cvs repo hasn't been touched in a long long time, but they have
> apparently migrated to git.
>
> cvs:
> http://modules.cvs.sourceforge.net/viewvc/modules/
>
> git:
> http://sourceforge.net/p/modules/git/ci/master/tree/
>
>
> There is still activity on git, patches for newest tcl version.
> It may not be bursting, but I wouldn't call it "dead". Yet. ;)
>
> [ ]'s
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2014/08/24921.php

Re: [OMPI users] How to keep multiple installations at same time

[Ralph Castain]

Check the repo - hasn't been touched in a very long time

On Aug 5, 2014, at 9:42 AM, Fabricio Cannini  wrote:

> On 05-08-2014 13:10, Ralph Castain wrote:
>> Since modules isn't a supported s/w package any more, you might consider 
>> using LMOD instead:
>> 
>> https://www.tacc.utexas.edu/tacc-projects/lmod
> 
> Modules isn't supported anymore? :O
> 
> Could you please send a link about it ?
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2014/08/24918.php

Re: [OMPI users] How to keep multiple installations at same time

[Fabricio Cannini]

On 05-08-2014 13:10, Ralph Castain wrote:

Since modules isn't a supported s/w package any more, you might consider using 
LMOD instead:

https://www.tacc.utexas.edu/tacc-projects/lmod


Modules isn't supported anymore? :O

Could you please send a link about it ?

Re: [OMPI users] How to keep multiple installations at same time

[Ralph Castain]

Since modules isn't a supported s/w package any more, you might consider using 
LMOD instead:

https://www.tacc.utexas.edu/tacc-projects/lmod


On Aug 5, 2014, at 9:04 AM, Gus Correa  wrote:

> Hi Ahsan
> 
> Besides Andrew's excellent suggestion for the runtime environment.
> 
> For the installation, Open MPI configuration scripts
> and Makefiles support VPATH.
> Hence, you can create two separate directories to
> *build* it with Gnu and Intel compilers.
> Then you launch configure, make, make install from
> each of these dirctories, using the appropriate compilers,
> and pointing to two distinct *installation directories*
> (with configure -prefix).
> 
> 
> My two cents,
> Gus Correa
> 
> On 08/04/2014 11:54 PM, Andrew Caird wrote:
>> Hi Ahsan,
>> 
>> We, and I think many people, use the Environment Modules software,
> http://modules.sourceforge.net , to do this.
>> 
>> I hope that helps.
>> 
>> --andy
>> 
>>> On Aug 4, 2014, at 11:47 PM, Syed Ahsan Ali  wrote:
>>> 
>>> 
>>> I want to compile openmpi with both intel and gnu compilers. How can I 
>>> install both at the same time and then specify which one to use during job 
>>> submission.
>>> 
>>> 
>>> Regards
>>> Ahsan
>>> ___
>>> users mailing list
>>> us...@open-mpi.org
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> Link to this post: 
>>> http://www.open-mpi.org/community/lists/users/2014/08/24905.php
>> ___
>> users mailing list
>> us...@open-mpi.org
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post: 
>> http://www.open-mpi.org/community/lists/users/2014/08/24906.php
>> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2014/08/24916.php

Re: [OMPI users] How to keep multiple installations at same time

[Gus Correa]

Hi Ahsan

Besides Andrew's excellent suggestion for the runtime environment.

For the installation, Open MPI configuration scripts
and Makefiles support VPATH.
Hence, you can create two separate directories to
*build* it with Gnu and Intel compilers.
Then you launch configure, make, make install from
each of these dirctories, using the appropriate compilers,
and pointing to two distinct *installation directories*
(with configure -prefix).


My two cents,
Gus Correa

On 08/04/2014 11:54 PM, Andrew Caird wrote:

Hi Ahsan,

We, and I think many people, use the Environment Modules software,

http://modules.sourceforge.net , to do this.


I hope that helps.

--andy


On Aug 4, 2014, at 11:47 PM, Syed Ahsan Ali  wrote:


I want to compile openmpi with both intel and gnu compilers. How can I install 
both at the same time and then specify which one to use during job submission.


Regards
Ahsan
___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2014/08/24905.php

___
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2014/08/24906.php

Re: [OMPI users] How to keep multiple installations at same time

[Andrew Caird]

Hi Ahsan,

We, and I think many people, use the Environment Modules software, 
http://modules.sourceforge.net , to do this. 

I hope that helps. 

--andy

> On Aug 4, 2014, at 11:47 PM, Syed Ahsan Ali  wrote:
> 
> 
> I want to compile openmpi with both intel and gnu compilers. How can I 
> install both at the same time and then specify which one to use during job 
> submission. 
>  
>  
> Regards
> Ahsan
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2014/08/24905.php

[OMPI users] How to keep multiple installations at same time

[Syed Ahsan Ali]

I want to compile openmpi with both intel and gnu compilers. How can
I install both at the same time and then specify which one to use during
job submission.


Regards
Ahsan