Hi Supun, I need a small clarification here. I am using local docker for PGA and servers but I am using the sam credential service that you guys are using. So do you think making local changes in the pga_config will do the work?
Thanks Pankaj On Sat, Nov 14, 2015 at 3:01 PM, Supun Nakandala <[email protected]> wrote: > Hi Pankaj, > > You can set to admin role to Internal/everyone in pga_config.php (in > <PGA_ROOT>/app/config/pga_config.php) and you will admin permissions. > > On Sat, Nov 14, 2015 at 2:06 PM, Pankaj Saha <[email protected]> > wrote: > >> Hi Suresh and Marlon >> I am trying to launch a hello world sample application from my local >> docker Airavata PGA. Can I get a demo admin role for few days so that I can >> create an application and test some sample experiments? It looks like >> without admin role I can not create applications. >> >> Thanks >> Pankaj >> >> On Wed, Oct 28, 2015 at 1:55 PM, Pierce, Marlon <[email protected]> wrote: >> >>> But you could answer my implied question below with zero knowledge of >>> Airavata. >>> >>> From: Pankaj Saha <[email protected]> >>> Reply-To: Airavata Dev <[email protected]> >>> Date: Wednesday, October 28, 2015 at 1:48 PM >>> >>> To: Marlon Pierce <[email protected]> >>> Cc: Suresh Marru <[email protected]>, Airavata Dev < >>> [email protected]> >>> Subject: Re: launching a job through Airavata to Mesos cluster >>> >>> As Suresh said I think this is the time when I have to understand >>> under the hood details of Airavata Architecture, which I could not gain >>> much during GSoC. >>> >>> On Wed, Oct 28, 2015 at 1:00 PM, Pierce, Marlon <[email protected]> wrote: >>> >>>> The “hello world” for us is usually to just run some linux command on >>>> the remote queuing system. So that is something you could easily wrap via >>>> docker and submit via Mesos (I hope it is easy…). Related to this is >>>> determining that the job has gone through various states (queued, >>>> executing, completed) and communicating that back to Airavata. >>>> >>>> >>>> Marlon >>>> >>>> >>>> From: Pankaj Saha <[email protected]> >>>> Date: Wednesday, October 28, 2015 at 12:54 PM >>>> To: Marlon Pierce <[email protected]> >>>> Cc: Suresh Marru <[email protected]>, Airavata Dev < >>>> [email protected]> >>>> >>>> Subject: Re: launching a job through Airavata to Mesos cluster >>>> >>>> Hi Marlon, >>>> "we more typically use Airavata to run scientific applications on >>>> clusters" - Our plan is to run dockerized scientific application >>>> through Airavata. >>>> Definitely our target is not to run a Tomcat webserver, I just said >>>> that as an example. >>>> >>>> Give me some time, after I discuss with Prof. Madhu and Renan I will >>>> try to reply your queries in details. >>>> >>>> Thanks >>>> Pankaj >>>> >>>> On Wed, Oct 28, 2015 at 12:32 PM, Pierce, Marlon <[email protected]> >>>> wrote: >>>> >>>>> Hi Pankaj— >>>>> >>>>> I may not have the full background on what you want to do, but we more >>>>> typically use Airavata to run scientific applications on clusters. This >>>>> goes through a scheduling and queuing system (Torque/MOAB or SLURM). If >>>>> you want to do this, you would first need to say how you would submit a >>>>> dockerized application using Mesos as your queueing system. You could do >>>>> this for a Tomcat server, but that is a little different from the jobs we >>>>> normally run. >>>>> >>>>> We don’t use Airavata to bootstrap itself, although that may be an >>>>> interesting way to think of things. There it is interesting to use Mesos >>>>> + >>>>> Marathon to run Airavata services (which may themselves by dockerized) but >>>>> that would be done today outside Airavata. >>>>> >>>>> Marlon >>>>> >>>>> >>>>> From: Suresh Marru <[email protected]> >>>>> Date: Wednesday, October 28, 2015 at 12:25 PM >>>>> To: Airavata Dev <[email protected]> >>>>> Cc: Marlon Pierce <[email protected]> >>>>> >>>>> Subject: Re: launching a job through Airavata to Mesos cluster >>>>> >>>>> Hi Pankaj, >>>>> >>>>> You cannot do this out of the box now, but with some guidance you can >>>>> easily write a task to run Dockerized application on a Mesos cluster, >>>>> please use 0.16 for this. As Shameera explained this is a improvised >>>>> architecture and any contributions you will make will carry forward. Note: >>>>> This will require you to understand under the hood details of Airavata >>>>> Architecture but I am sure you will motivate others to work with you to >>>>> speed it up. >>>>> >>>>> Suresh >>>>> >>>>> On Oct 28, 2015, at 12:22 PM, Pankaj Saha <[email protected]> >>>>> wrote: >>>>> >>>>> “launch a dockerized job” means submitting a docker container for any >>>>> specific job (say a java webserver docker container) through Airavata. >>>>> What I remember Airavata php gateway can not submit any docker >>>>> container(correct me if I am wrong). What I want to figure out initially >>>>> is >>>>> whether its feasible to submit a docker container with help of Airavata >>>>> into a Mesos cluster. >>>>> >>>>> Thanks >>>>> Pankaj >>>>> >>>>> On Wed, Oct 28, 2015 at 12:11 PM, Pierce, Marlon <[email protected]> >>>>> wrote: >>>>> >>>>>> I’ll add: if submitting a job to (for example) a SLURM queuing >>>>>> system, we need to create the correct SLURM submission script and submit >>>>>> it >>>>>> by executing the correct command line operation (sbatch). >>>>>> >>>>>> From: Marlon Pierce <[email protected]> >>>>>> Reply-To: Airavata Dev <[email protected]> >>>>>> Date: Wednesday, October 28, 2015 at 12:08 PM >>>>>> To: Airavata Dev <[email protected]>, Suresh Marru < >>>>>> [email protected]>, Pankaj Saha <[email protected]> >>>>>> >>>>>> Subject: Re: launching a job through Airavata to Mesos cluster >>>>>> >>>>>> Hi Pankaj, can you say more about what you mean by “launch a >>>>>> dockerized job”? >>>>>> >>>>>> Marlon >>>>>> >>>>>> >>>>>> From: Pankaj Saha <[email protected]> >>>>>> Reply-To: Airavata Dev <[email protected]> >>>>>> Date: Wednesday, October 28, 2015 at 11:56 AM >>>>>> To: Suresh Marru <[email protected]> >>>>>> Cc: Airavata Dev <[email protected]> >>>>>> Subject: Re: launching a job through Airavata to Mesos cluster >>>>>> >>>>>> Hi Suresh, >>>>>> >>>>>> My initial understanding is, I have to launch a dockerized job >>>>>> through Airavata which will be run in the Mesos cluster. I was looking >>>>>> for >>>>>> the code which submits jobs and wanted to make changes such a way that it >>>>>> can submit docker containers to Mesos/Marathon cluster. >>>>>> >>>>>> I can use 0.15 branch and I have no idea about data transfer protocol >>>>>> and job submission protocols that Shameera has mentioned. I may want to >>>>>> submit jobs by submitting a JSON through command line or any other way >>>>>> that >>>>>> you guys feel is more appropriate. >>>>>> >>>>>> I can talk to Prof. Madhu and let you know more on the requirement. >>>>>> >>>>>> Thanks >>>>>> Pankaj >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Oct 28, 2015 at 11:23 AM, Suresh Marru <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Pankaj can you clarify the following: >>>>>>> >>>>>>> Do you want an Airavata instance to run some dockerized applications >>>>>>> scheduled by Mesos? Or do you just need a client which will connect to >>>>>>> Airavata hosted and managed by Mesos/Marathon? >>>>>>> >>>>>>> Suresh >>>>>>> >>>>>>> On Oct 28, 2015, at 10:50 AM, Shameera Rathnayaka < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>> Hi Pankaj, >>>>>>> >>>>>>> Wich version of Airavata you are working on? what is the data >>>>>>> transfer protocol? What is the job submission protocol? >>>>>>> >>>>>>> Short answer: if you are using Airavata 0.15 then you need to write >>>>>>> new Provider implementation to submit the request to Mesos/Marathon >>>>>>> cluster. But if you are using Airavata 16.0 which is current master, >>>>>>> then >>>>>>> you need to write JobSubmissionTask implementation. Either case you can >>>>>>> go >>>>>>> through the existing implementations, for Provider implementation see >>>>>>> SSHProvider and JobSubmissionTask implementation see >>>>>>> SSHJobSubmissionTask. >>>>>>> >>>>>>> If I get the answers to my questions then i can provide exactly what >>>>>>> you need to do. BTW we have cleaned our internal architecture in >>>>>>> Airavata >>>>>>> 16.0, as a developer you would find it easy to work with Airavata 16.0 >>>>>>> that >>>>>>> Airavata 15.0. But notice master is not yet stable as Airavata 15.0. >>>>>>> >>>>>>> Regards, >>>>>>> Shameera. >>>>>>> >>>>>>> >>>>>>> On Tue, Oct 27, 2015 at 1:53 PM Pankaj Saha <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> Hello Shameera, >>>>>>>> I am working on jet-stream project, where I have to find out a way >>>>>>>> to submit a job in mesos/marathon cluster through Airavata client. I >>>>>>>> don't >>>>>>>> have much idea from where to start looking into. Can you please give >>>>>>>> some >>>>>>>> clue so that I can start working and making changes to java code for >>>>>>>> the >>>>>>>> same. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Pankaj >>>>>>>> >>>>>>>> -- >>>>>>> Shameera Rathnayaka >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>> >>> >> > > > -- > Thank you > Supun Nakandala > Dept. Computer Science and Engineering > University of Moratuwa >
