I’ll add: if submitting a job to (for example) a SLURM queuing system, we need to create the correct SLURM submission script and submit it by executing the correct command line operation (sbatch).
From: Marlon Pierce <[email protected]<mailto:[email protected]>> Reply-To: Airavata Dev <[email protected]<mailto:[email protected]>> Date: Wednesday, October 28, 2015 at 12:08 PM To: Airavata Dev <[email protected]<mailto:[email protected]>>, Suresh Marru <[email protected]<mailto:[email protected]>>, Pankaj Saha <[email protected]<mailto:[email protected]>> Subject: Re: launching a job through Airavata to Mesos cluster Hi Pankaj, can you say more about what you mean by “launch a dockerized job”? Marlon From: Pankaj Saha <[email protected]<mailto:[email protected]>> Reply-To: Airavata Dev <[email protected]<mailto:[email protected]>> Date: Wednesday, October 28, 2015 at 11:56 AM To: Suresh Marru <[email protected]<mailto:[email protected]>> Cc: Airavata Dev <[email protected]<mailto:[email protected]>> Subject: Re: launching a job through Airavata to Mesos cluster Hi Suresh, My initial understanding is, I have to launch a dockerized job through Airavata which will be run in the Mesos cluster. I was looking for the code which submits jobs and wanted to make changes such a way that it can submit docker containers to Mesos/Marathon cluster. I can use 0.15 branch and I have no idea about data transfer protocol and job submission protocols that Shameera has mentioned. I may want to submit jobs by submitting a JSON through command line or any other way that you guys feel is more appropriate. I can talk to Prof. Madhu and let you know more on the requirement. Thanks Pankaj On Wed, Oct 28, 2015 at 11:23 AM, Suresh Marru <[email protected]<mailto:[email protected]>> wrote: Pankaj can you clarify the following: Do you want an Airavata instance to run some dockerized applications scheduled by Mesos? Or do you just need a client which will connect to Airavata hosted and managed by Mesos/Marathon? Suresh On Oct 28, 2015, at 10:50 AM, Shameera Rathnayaka <[email protected]<mailto:[email protected]>> wrote: Hi Pankaj, Wich version of Airavata you are working on? what is the data transfer protocol? What is the job submission protocol? Short answer: if you are using Airavata 0.15 then you need to write new Provider implementation to submit the request to Mesos/Marathon cluster. But if you are using Airavata 16.0 which is current master, then you need to write JobSubmissionTask implementation. Either case you can go through the existing implementations, for Provider implementation see SSHProvider and JobSubmissionTask implementation see SSHJobSubmissionTask. If I get the answers to my questions then i can provide exactly what you need to do. BTW we have cleaned our internal architecture in Airavata 16.0, as a developer you would find it easy to work with Airavata 16.0 that Airavata 15.0. But notice master is not yet stable as Airavata 15.0. Regards, Shameera. On Tue, Oct 27, 2015 at 1:53 PM Pankaj Saha <[email protected]<mailto:[email protected]>> wrote: Hello Shameera, I am working on jet-stream project, where I have to find out a way to submit a job in mesos/marathon cluster through Airavata client. I don't have much idea from where to start looking into. Can you please give some clue so that I can start working and making changes to java code for the same. Thanks Pankaj -- Shameera Rathnayaka
