Hi Suresh, That would be great because we will migrate all our tools and data to BigRed II eventually (possibly before the end of this year).
Thanks , Udayanga On Tue, Aug 13, 2013 at 1:34 PM, Suresh Marru <[email protected]> wrote: > Hi Udayanga, > > In addition to enhancements you mention, note that we will have support > for Big Red II in Airavata one way or another. We will keep you posted. > > Suresh > > On Aug 13, 2013, at 1:26 PM, Udayanga Wickramasinghe < > [email protected]> wrote: > > > Hi Suresh, > > Thanks a lot for your thoughts on this. I see control flow strategy > would work perfect for my case , however sighting your problems at GRAM > support on Bigred i may need to implement an 'synchronous' call at the > Bigred side (ie:- a polling mechanism) . Unfortunately we don't have a > Quarry cluster as well and we are at the moment migrating to Bigred2. I > think the future Airavata support you mentioned would be the perfect case > for a scenario like ours (because we have hundreds of scripts that perform > molecular dynamics that we execute remotely using ssh ) and it would be a > great addition to the Airavata tool set. > > > > Regards > > Udayanga > > > > > > On Mon, Aug 12, 2013 at 9:45 PM, Suresh Marru <[email protected]> wrote: > > Hi Udayanga, > > > > Airavata workflows supports both data flow and control flow executions. > That means if you connect output of first step to input of second step, > then execution of second step will wait until the output from step 1 is > available. In control flow, you can connect the right hand top corner of > the step 1 to left hand bottom corner of step 2. This will indicate the > execution of step 2 will wait until step 1 is executed. This will be > independent of data dependencies. > > > > Your workflow is not working since you are using SSH provider which as > you described the load leveler script is executing as a synchronous > command. There are plans in upcoming 0.9 release to support asynchronous > local and ssh provider, they should help you. Or if you have a Grid > middleware like GRAM, then they by nature support asynchronous batch > submissions. Unfortunately the load leveler version on Big Red is only > partial implementation of the load leveler specification and only web > services version of the GRAM (GRAM 4) used to work with it. The latest > pre-ws gram (GRAM 5), does not support load leveler version on Big Red. Do > you have accounts on Quarry cluster? It has a GRAM5 installation which > should work with Airavata. > > > > In the near future, we plan to develop native local resource manager > integration so Airavata can directly interact with batch systems through > ssh and gsi-ssh protocols. > > > > Suresh > > > > On Aug 12, 2013, at 8:56 PM, Udayanga Wickramasinghe < > [email protected]> wrote: > > > > > Hi , > > > I have a workflow (which is a simple one at the moment) that will > execute on a remote environment in several stages. Actually I have > registered a command line application on Airavata that will use SSH > provider to run them in a pipeline. For a single step this works fine but i > am having trouble connecting them to execute one after the other. > > > > > > My requirement is like this , first step of the workflow will take > several inputs and generate an output file (using a remote load leveler > script OR/AND MPI application ) and we need to block/wait until a result is > generated and we take the this result plus some more inputs and execute the > second. Likewise this process will continue for several stages on a remote > grid environment (ie:- Bigred) and final result will be copied back to our > staged servers. How can we achieve a blocking wait in a workflow with > Airavata specially with load leveler/mpi application job submission being > asynchronous ? Is there any special constuct that support this out of the > box or do we have to write extensions ? Even if we can create multiple > command line application/s easily with Airavata i don't see a way to link > them in a pipeline manner. I would very much appreciate your thoughts on > this and possible way to approach this. > > > > > > Thanks > > > Udayanga > > > > > > > > > -- > > > http://www.udayangawiki.blogspot.com > > > > > > > > > > -- > > http://www.udayangawiki.blogspot.com > > -- http://www.udayangawiki.blogspot.com
