HI,

 sorry for the late reply - I've been traveling with limited email
access. I think you can leave this issue be. I think I was hoping for a
way to just launch mpirun and have it create the allocation by itself.
It's not super important right now, more something I was wondering
about. 

 Thanks again for looking into this!

 /Marc

On 28.11.2014 17:10, Ralph Castain wrote: 

> Hey Marc - just wanted to check to see if you felt this would indeed solve 
> the problem for you. I'd rather not invest the time if this isn't going to 
> meet the need, and I honestly don't know of a better solution. 
> 
> On Nov 20, 2014, at 2:13 PM, Ralph Castain <r...@open-mpi.org> wrote: 
> 
> Here's what I can provide: 
> 
> * lsrun -n N bash This causes openlava to create an allocation and start you 
> off in a bash shell (or pick your shell) 
> 
> * mpirun ..... Will read the allocation and use openlava to start the 
> daemons, and then the application, on the allocated nodes 
> 
> You can execute as many mpirun's as you like, then release the allocation (I 
> believe by simply exiting the shell) when done. 
> 
> Does that match your expectations? 
> Ralph 
> 
> On Nov 20, 2014, at 2:03 PM, Marc Höppner <marc.hoepp...@bils.se> wrote: 
> 
> Hi,
> 
> yes, lsrun exists under openlava. 
> 
> Using mpirun is fine, but openlava currently requires that to be launched 
> through a bash script (openmpi-mpirun). Would be neater if one could do away 
> with that. 
> 
> Agan, thanks for looking into this!
> 
> /Marc
> 
> Hold on - was discussing this with a (possibly former) OpenLava developer who 
> made some suggestions that would make this work. It all hinges on one thing. 
> 
> Can you please check and see if you have "lsrun" on your system? If you do, 
> then I can offer a tight integration in that we would use OpenLava to 
> actually launch the OMPI daemons. Still not sure I could support you directly 
> launching MPI apps without using mpirun, if that's what you are after. 
> 
> On Nov 18, 2014, at 7:58 AM, Marc Höppner <marc.hoepp...@bils.se> wrote: 
> 
> Hi Ralph,
> 
> I really appreciate you guys looking into this! At least now I know that 
> there isn't a better way to run mpi jobs. Probably worth looking into LSF 
> again..
> 
> Cheers,
> 
> Marc 
> I took a brief gander at the OpenLava source code, and a couple of things 
> jump out. First, OpenLava is a batch scheduler and only supports batch 
> execution - there is no interactive command for "run this job". So you would 
> have to "bsub" mpirun regardless. 
> 
> Once you submit the job, mpirun can certainly read the local allocation via 
> the environment. However, we cannot use the OpenLava internal functions to 
> launch the daemons or processes as the code is GPL2, and thus has a viral 
> incompatible license. Ordinarily, we get around that by just executing the 
> interactive job execution command, but OpenLava doesn't have one. 
> 
> So we'd have no other choice but to use ssh to launch the daemons on the 
> remote nodes. This is exactly what the provided openmpi wrapper script that 
> comes with OpenLava already does. 
> 
> Bottom line: I don't see a way to do any deeper integration minus the 
> interactive execution command. If OpenLava had a way of getting an allocation 
> and then interactively running jobs, we could support what you requested. 
> This doesn't seem to be what they are intending, unless I'm missing something 
> (the documentation is rather incomplete). 
> 
> Ralph 
> 
> On Tue, Nov 18, 2014 at 6:20 AM, Marc Höppner <marc.hoepp...@bils.se> wrote:
> 
> Hi, 
> 
> sure, no problem. And about the C Api, I really don't know more than what I 
> was told in the google group post I referred to (i.e. the API is essentially 
> identical to LSF 4-6, which should be on the web). 
> 
> The output of env can be found here: 
> https://dl.dropboxusercontent.com/u/1918141/env.txt [6] 
> 
> /M 
> 
> Marc P. Hoeppner, PhD 
> Team Leader 
> BILS Genome Annotation Platform 
> Department for Medical Biochemistry and Microbiology 
> Uppsala University, Sweden 
> marc.hoepp...@bils.se 
> 
> On 18 Nov 2014, at 15:14, Ralph Castain <r...@open-mpi.org> wrote: 
> 
> If you could just run a single copy of "env" and send the output along, that 
> would help a lot. I'm not interested in the usual path etc, but would like to 
> see the envars that OpenLava is setting. 
> 
> Thanks 
> Ralph 
> 
> On Tue, Nov 18, 2014 at 2:19 AM, Gilles Gouaillardet 
> <gilles.gouaillar...@iferc.org> wrote:
> 
> Marc,
> 
> the reply you pointed is a bit confusing to me :
> 
> "There is a native C API which can submit/start/stop/kill/re queue jobs"
> this is not what i am looking for :-(
> 
> "you need to make an appropriate call to openlava to start a remote process"
> this is what i am interested in :-)
> could you be more specific (e.g. point me to the functions, since the 
> OpenLava doc is pretty minimal ...)
> 
> the goal here is to spawn the orted daemons as part of the parallel job,
> so these daemons are accounted within the parallel job.
> /* if we use an API that simply spawns orted, but the orted is not related 
> whatsoever to the parallel job,
> then we can simply use ssh */
> 
> Cheers,
> 
> Gilles
> 
> On 2014/11/18 18:24, Marc Höppner wrote:
> 
> Hi Gilles, 
> 
> thanks for the prompt reply. Yes, as far as I know there is a C API to 
> interact with jobs etc. Some mentioning here: 
> https://groups.google.com/forum/#!topic/openlava-users/w74cRUe9Y9E [1] 
> <https://groups.google.com/forum/#!topic/openlava-users/w74cRUe9Y9E> [1]
> 
> /Marc Marc P. Hoeppner, PhD Team Leader BILS Genome Annotation Platform 
> Department for Medical Biochemistry and Microbiology Uppsala University, 
> Sweden marc.hoepp...@bils.se 
> 
> On 18 Nov 2014, at 08:40, Gilles Gouaillardet <gilles.gouaillar...@iferc.org> 
> wrote:
> 
> Hi Marc,
> 
> OpenLava is based on a pretty old version of LSF (4.x if i remember
> correctly)
> and i do not think LSF had support for parallel jobs tight integration
> at that time.
> 
> my understanding is that basically, there is two kind of direct
> integration :
> - mpirun launch: mpirun spawns orted via the API provided by the batch
> manager
> - direct launch: the mpi tasks are launched directly from the
> script/command line and no mpirun/orted is involved
> at that time, it works with SLURM and possibly other PMI capable batch
> manager
> 
> i think OpenLava simply gets a list of hosts from the environment, build
> a machinefile, pass it to mpirun that spawns orted with ssh, so this is
> really loose integration.
> 
> OpenMPI is based on plugins, so as long as the queing system provides an
> API to start/stop/kill tasks, mpirun launch should not
> be a huge effort.
> 
> Are you aware of such an API provided by OpenLava ?
> 
> Cheers,
> 
> Gilles
> 
> On 2014/11/18 16:31, Marc Höppner wrote:
> 
> Hi list,
> 
> I have recently started to wonder how hard it would be to add support for 
> queuing systems to the tight integration function of OpenMPI (unfortunately, 
> I am not a developer myself). Specifically, we are working with OpenLava 
> (www.openlava.org [2]), which is based on an early version of Lava/LSF and 
> open source. It's proven quite useful in environments where some level of LSF 
> compatibility is needed, but without actually paying for a (rather pricey) 
> LSF license. 
> 
> Given that openLava shares quite a bit of DNA with LSF, I was wondering how 
> hard it would be to add OL tight integration support to OpenMPI. Currently, 
> OL enables OpenMPI jobs through a wrapper script, but that's obviously not 
> ideal and doesn't work for some programs that have MPI support built-in (and 
> thus expect to be able to just execute mpirun). 
> 
> Any thoughts on this would be greatly appreciated!
> 
> Regards,
> 
> Marc
> 
> Marc P. Hoeppner, PhD
> Team Leader
> BILS Genome Annotation Platform
> Department for Medical Biochemistry and Microbiology
> Uppsala University, Sweden
> marc.hoepp...@bils.se
> 
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2014/11/16312.php [4]
> 
> _______________________________________________
> devel mailing list
> de...@open-mpi.org
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
> Link to this post: 
> http://www.open-mpi.org/community/lists/devel/2014/11/16313.php [5]

_______________________________________________ devel mailing list
de...@open-mpi.org Subscription:
http://www.open-mpi.org/mailman/listinfo.cgi/devel [3] 

Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16314.php [7]

 _______________________________________________
 devel mailing list
de...@open-mpi.org
 Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
 Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16315.php [11]
_______________________________________________
 devel mailing list
de...@open-mpi.org
 Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16316.php [12] 
 _______________________________________________
 devel mailing list
de...@open-mpi.org
 Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
 Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16317.php [13] 

_______________________________________________
devel mailing list
de...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16318.php [8]

_______________________________________________
devel mailing list
de...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16319.php [14] 

_______________________________________________
devel mailing list
de...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16326.php [9]

_______________________________________________
devel mailing list
de...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16327.php [15] 

_______________________________________________
devel mailing list
de...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel [3]
Link to this post:
http://www.open-mpi.org/community/lists/devel/2014/11/16373.php [10]



Links:
------
[1] https://groups.google.com/forum/#%21topic/openlava-users/w74cRUe9Y9E
[2] http://www.openlava.org/
[3] http://www.open-mpi.org/mailman/listinfo.cgi/devel
[4] http://www.open-mpi.org/community/lists/devel/2014/11/16312.php
[5] http://www.open-mpi.org/community/lists/devel/2014/11/16313.php
[6] https://dl.dropboxusercontent.com/u/1918141/env.txt
[7] http://www.open-mpi.org/community/lists/devel/2014/11/16314.php
[8] http://www.open-mpi.org/community/lists/devel/2014/11/16318.php
[9] http://www.open-mpi.org/community/lists/devel/2014/11/16326.php
[10] http://www.open-mpi.org/community/lists/devel/2014/11/16373.php
[11] http://www.open-mpi.org/community/lists/devel/2014/11/16315.php
[12] http://www.open-mpi.org/community/lists/devel/2014/11/16316.php
[13] http://www.open-mpi.org/community/lists/devel/2014/11/16317.php
[14] http://www.open-mpi.org/community/lists/devel/2014/11/16319.php
[15] http://www.open-mpi.org/community/lists/devel/2014/11/16327.php

Reply via email to