Hi Sanjaya, This looks like a clean approach for me.
Regards Lahiru On Tue, Feb 19, 2013 at 1:52 PM, Sanjaya Medonsa <[email protected]>wrote: > Thanks Lahiru! I have gone through the test classes and the classes in > package org.apache.airavata.gfac. It was really helpful to understand the > new architecture. I have listed down my approach based on new architecture > to use Apache OODT products as an input to Airavata. > 1. Introduce new Data Type to represent Apache OODT Product as an > DataType. Basically new DataType is added into the GFacParameterTypes.xsd. > 2. With new Architecture In Handlers and Out Handlers replaces the > Pre/Post execution chains in old architecture. For the moment I am focusing > on using Apache OODT product ID or file path as an input and stage the file > (product) into host where actual execution happens. File staging requires > to retrieve product from a File Manager server to the host where execution > occurs. File staging can be implemented as an* In Handler* and needs to be > configured as a new item in the list of configured In Handlers. > 3. Handler should first verify the input parameter types listed in > Service Description of the Application context of the > *JobExecutionContext*. > If input type matches the new parameter type, in handler stage file into > host machine using Apache OODT file manager component. Corresponding input > value can be retrieved from *In MessageContex*t. If a parameter type in > MessageContext matches the new input type, then corresponding value is the > id or file path to product managed by Apache OODT File Manager server. > > Best Regards, > Sanjaya > > On Tue, Feb 19, 2013 at 1:29 AM, Lahiru Gunathilake <[email protected] > >wrote: > > > Hi Sanjaya, > > > > If you want to understand the new architecture by looking in to the code, > > please just refer the package org.apache.airavata.gfac, do not refer any > of > > the classes in org.apache.airavata.core.gfac. > > > > Best place to start with is referring is from test classes > > (LocalProviderTest,GramProviderTest), from tehre you can start looking in > > to GFacAPI class and see how to execution is flawing. > > > > if you have further questions please post on the list and more than happy > > to help. I will be doing some documentation about the architecture, once > I > > am done, will post in to the list. And we will be having an architecture > > review this week, so please watch the mailing list, if possible please > try > > to join us. > > > > Regards > > Lahiru > > > > On Mon, Feb 18, 2013 at 1:25 PM, Sanjaya Medonsa <[email protected] > > >wrote: > > > > > Thanks Suresh and Chris! It seems I am moving on the correct path. I > have > > > followed the email thread on improved GFac architecture. Though I am > not > > > entirely clear on the improved GFac architecture, proposed integration > > with > > > OODT is primarily based on the GFac extension, PreExecustionChain, > which > > > has not been modified with the Architecture improvements (As per one of > > the > > > replies from Lahiru, output extension is supported with new > > Architecture. I > > > assume input extension is also supported). > > > > > > I have looked into provenance manager and related implementation. > Still I > > > am unclear how Airavata support provenance aware work flow processing. > > > > > > Best Regards, > > > Sanjaya > > > > > > On Mon, Feb 18, 2013 at 6:35 AM, Suresh Marru <[email protected]> > wrote: > > > > > > > Hi Sanjaya, > > > > > > > > This sounds very exciting. Both Airavata and OODT projects have good > > > > synergies and have been long looking for volunteers who can bridge > them > > > > both. Please do not hesitate to ask any questions to either or both > the > > > dev > > > > lists. The more engaged you are, you will find use cases and feedback > > > which > > > > should help your MSc project. > > > > > > > > Your plan sounds good. If you are following dev list, you may have > > > > noticed, the GFac architecture has been improved to properly support > > this > > > > kind of handler architecture. > > > > > > > > You may also want to look at Airavata Registry API which has > > organically > > > > emerged with the need, but can greatly benefit for OODT's provenance > > > > capabilities. > > > > > > > > Suresh > > > > > > > > On Feb 17, 2013, at 1:25 PM, "Mattmann, Chris A (388J)" < > > > > [email protected]> wrote: > > > > > > > > > +1, sounds like a great idea Sanjaya! > > > > > > > > > > I'm copying dev@oodt so they can be in the conversation too. > > > > > > > > > > Cheers, > > > > > Chris > > > > > > > > > > On 2/17/13 10:22 AM, "Sanjaya Medonsa" <[email protected]> > wrote: > > > > > > > > > >> Hi Dev Team, > > > > >> As I have posted previously, I am working on a Apache Airavata + > > > Apache > > > > >> OODT integration as my MSc project. Following is one of the > possible > > > > >> integration to leverage Apache OODT file management capability > into > > > > Apache > > > > >> Airavata. Please review the proposal and let me know your > feedback. > > > > >> > > > > >> Proposal To Use Apache OODT products as input to Apache Airavata > > > > Workflows > > > > >> and staging product files into node where execution happens > > > > >> > > > > > > > > > > ========================================================================== > > > > >> ============================== > > > > >> 1. Introduce "Apache OODT Product" as a new GFacParameterType. New > > > > "Apache > > > > >> OODT Product" input type can sepcify "Product ID" or "File Path to > > > > >> Product" > > > > >> as an input to Apache Airavata workflows. > > > > >> 2. Introduce new PreExecuteChain to retrieve Apache OODT Products > > from > > > > >> File > > > > >> Manager Server managed using Apache OODT. > > > > >> 1. Using Apache OODT File Manager componenet transfer Product from > > > > Server > > > > >> to input directory of the application as configured using > XBaya-GUI > > > > under > > > > >> advanced configuration. (Here the assumption is that Products are > > > > >> accesible > > > > >> through Apache OODT File Manager server) > > > > >> 2. Finally reset the input value to local file path. I think we > can > > > > remove > > > > >> the OODT Product parameter from invocation context and add new > file > > > > >> parameter with value set to 'local path of the transferred > > product'. I > > > > am > > > > >> not quite sure what are the implications of changing input > parameter > > > > type > > > > >> during the execution. > > > > >> > > > > >> Similar approach has been implemented for GridFTP and HTTP. > > > > >> > > > > >> Best Regards, > > > > >> Sanjaya > > > > > > > > > > > > > > > > > > > > > > > > -- > > System Analyst Programmer > > PTI Lab > > Indiana University > > > -- System Analyst Programmer PTI Lab Indiana University
