Hi Udayanga, In addition to enhancements you mention, note that we will have support for Big Red II in Airavata one way or another. We will keep you posted.
Suresh On Aug 13, 2013, at 1:26 PM, Udayanga Wickramasinghe <[email protected]> wrote: > Hi Suresh, > Thanks a lot for your thoughts on this. I see control flow strategy would > work perfect for my case , however sighting your problems at GRAM support on > Bigred i may need to implement an 'synchronous' call at the Bigred side (ie:- > a polling mechanism) . Unfortunately we don't have a Quarry cluster as well > and we are at the moment migrating to Bigred2. I think the future Airavata > support you mentioned would be the perfect case for a scenario like ours > (because we have hundreds of scripts that perform molecular dynamics that we > execute remotely using ssh ) and it would be a great addition to the Airavata > tool set. > > Regards > Udayanga > > > On Mon, Aug 12, 2013 at 9:45 PM, Suresh Marru <[email protected]> wrote: > Hi Udayanga, > > Airavata workflows supports both data flow and control flow executions. That > means if you connect output of first step to input of second step, then > execution of second step will wait until the output from step 1 is available. > In control flow, you can connect the right hand top corner of the step 1 to > left hand bottom corner of step 2. This will indicate the execution of step 2 > will wait until step 1 is executed. This will be independent of data > dependencies. > > Your workflow is not working since you are using SSH provider which as you > described the load leveler script is executing as a synchronous command. > There are plans in upcoming 0.9 release to support asynchronous local and ssh > provider, they should help you. Or if you have a Grid middleware like GRAM, > then they by nature support asynchronous batch submissions. Unfortunately the > load leveler version on Big Red is only partial implementation of the load > leveler specification and only web services version of the GRAM (GRAM 4) used > to work with it. The latest pre-ws gram (GRAM 5), does not support load > leveler version on Big Red. Do you have accounts on Quarry cluster? It has a > GRAM5 installation which should work with Airavata. > > In the near future, we plan to develop native local resource manager > integration so Airavata can directly interact with batch systems through ssh > and gsi-ssh protocols. > > Suresh > > On Aug 12, 2013, at 8:56 PM, Udayanga Wickramasinghe > <[email protected]> wrote: > > > Hi , > > I have a workflow (which is a simple one at the moment) that will execute > > on a remote environment in several stages. Actually I have registered a > > command line application on Airavata that will use SSH provider to run them > > in a pipeline. For a single step this works fine but i am having trouble > > connecting them to execute one after the other. > > > > My requirement is like this , first step of the workflow will take several > > inputs and generate an output file (using a remote load leveler script > > OR/AND MPI application ) and we need to block/wait until a result is > > generated and we take the this result plus some more inputs and execute the > > second. Likewise this process will continue for several stages on a remote > > grid environment (ie:- Bigred) and final result will be copied back to our > > staged servers. How can we achieve a blocking wait in a workflow with > > Airavata specially with load leveler/mpi application job submission being > > asynchronous ? Is there any special constuct that support this out of the > > box or do we have to write extensions ? Even if we can create multiple > > command line application/s easily with Airavata i don't see a way to link > > them in a pipeline manner. I would very much appreciate your thoughts on > > this and possible way to approach this. > > > > Thanks > > Udayanga > > > > > > -- > > http://www.udayangawiki.blogspot.com > > > > > -- > http://www.udayangawiki.blogspot.com
