Thanks Suresh! I'll do further investigations and let the team know my findings.
Best Regards, Sanjaya On Tue, Oct 15, 2013 at 6:52 AM, Suresh Marru <sma...@apache.org> wrote: > Hi Sanjaya, > > XBaya and Workflow Interpreter went through some changes which probably > dropped the condition which makes use of the enableProvenanceSmartRun flag. > AFAIK, the feature did not get deprecated it should be one of those cases > which was not actively tested during releases and fell through the cracks. > Lets see if any one gets motivated to debug and fix this issue. And as > always, we welcome your patches. Sorry about pointing to a unverified > feature. > > Suresh > > On Oct 14, 2013, at 9:01 PM, Sanjaya Medonsa <sanjaya...@gmail.com> wrote: > > > Hi Suresh, > > I have gone through the research paper in item 6 and try to match it to > the current Airavata implementation. I can see that Airavata collects > provenance data and stored them in a registry. I couldn't able to find out > a place where these data has been actually used to improve the workflow > processing. Xbaya can be configured with option enableProvenanceSmartRun. > It seems this particular configuration is no longer used. Since provenance > aware data is already available in registry, is it the responsibility of > the application to use these data to improve workflow processing based on > provenance aware data ? > > > > Thanks > > Sanjaya > > > > > > On Sat, Aug 24, 2013 at 7:47 AM, Chris Mattmann <mattm...@apache.org> > wrote: > > Hey Sanjaya, > > > > You may want to check out the Traceable Product product type policy > > [1] as well as OODT-139 [2] which includes the PCS services that serve > > up the information and provenance. > > > > Cheers, > > Chris > > > > [1] > > > http://svn.apache.org/repos/asf/oodt/trunk/filemgr/src/main/resources/examp > > les/trace/ > > [2] http://issues.apache.org/jira/browse/OODT-139 > > > > > > -----Original Message----- > > From: Sanjaya Medonsa <sanjaya...@gmail.com> > > Reply-To: <dev@oodt.apache.org> > > Date: Friday, August 23, 2013 6:30 PM > > To: Airavata Dev <d...@airavata.apache.org> > > Cc: "dev@oodt.apache.org" <dev@oodt.apache.org> > > Subject: Re: Povenance Aware Workflow Processing > > > > >Thanks Suresh! I am interested in Provenance aware workflow processing > > >where OODT integration could help greatly with rich metadata available. > > >I'll go through the item 6 and let you know my understanding. > > > > > >Sanjaya > > > > > > > > >On Thu, Aug 8, 2013 at 7:55 PM, Suresh Marru <sma...@apache.org> wrote: > > > > > >> Hi Sanjaya, Lewis, > > >> > > >> Appreciate your interest on this topic, this is a very good question > and > > >> sorry for the delay in responding, these days I feel I am always in > > >>catch > > >> up mode. > > >> > > >> Let me separate this discussion into four parts: Provenance Gathering, > > >> Provenance Analyzing, Provenance Reasoning, Provenance Aware > executions. > > >> > > >> Gathering: Airavata is instrumented with rich provenance centric > > >> information model which captures each step and publishes to an event > > >>bus. > > >> This is encoded in a "workflow tracking" schema [1]. So the current > > >>trunk > > >> of Airavata publishes provenance rich information but not all is > > >>guaranteed > > >> to be captured in the registry. > > >> > > >> Analyzing: Significant work on provenance aspect related to Airavata > is > > >>a > > >> by product of doctoral dissertation of Yogesh Simmhan, currently a > > >>research > > >> asst. professor at USC. His thesis [2] is a good source of reference > on > > >> this topic. Both the research and tools are in active development by > > >>Prof. > > >> Beth Plale's group at IU [3]. Theoretically speaking since both Karma > > >>uses > > >> the same workflow tracking schema and event bus, the tools should be > > >>easily > > >> integratable but did not have a motivating use case to do so. > > >> > > >> Reasoning: Phala is an extension to Airavata XBaya (its legacy LEAD > > >> version) and Karma framework which provides context to the gathered > > >> provenance. Couple of papers on this topic [4], [5]. > > >> > > >> Executions: This is an active topic within Airavata. Current release > of > > >> Airavata supports Provenance aware workflow executions and is > discussed > > >>in > > >> [6]. I think the OODT and Airavata integration will greatly benefit > this > > >> aspect. Airavata Registry metadata cataloging is very primitive. And > > >>Rich > > >> data management capabilities of OODT should nicely enhance airavata > > >> provenance aware executions. > > >> > > >> Please let me know which of the above aspects you are interested in > and > > >>we > > >> can discuss further. > > >> > > >> Suresh > > >> > > >> [1] - > > >> > > >> > https://svn.apache.org/repos/asf/airavata/trunk/modules/commons/workflow- > > >>tracking/src/main/resources/schemas/workflow_tracking_types.xsd > > >> [2] - http://gradworks.umi.com/3297094.pdf > > >> [3] - http://d2i.indiana.edu/provenance_karma > > >> [4] - > > >> > http://link.springer.com/content/pdf/10.1007%2F978-3-540-85502-6_18.pdf > > >> [5] - > > >> > > >> > http://dl.acm.org/citation.cfm?id=1590963.1591575&coll=DL&dl=GUIDE&CFID=2 > > >>38325269&CFTOKEN=73364358 > > >> [6] - http://dl.acm.org/citation.cfm?id=2359844 > > >> > > >> > > >> On Jul 22, 2013, at 10:57 AM, Lewis John Mcgibbney < > > >> lewis.mcgibb...@gmail.com> wrote: > > >> > > >> > Hi Sanjaya, > > >> > I am seeing some traffic floating around RE: Airavata OODT > integration > > >> but I am not clued up. > > >> > Can you point me to the code? The issues? The mailing list threads? > > >> > I am also extremely interested in provenance tracking of (record???) > > >> data which passes through OODT. I would like to expose this as some > > >>stage > > >> to people who access the data. > > >> > Thank you so much. > > >> > > > >> > On Monday, July 22, 2013, Sanjaya Medonsa <sanjaya...@gmail.com> > > >>wrote: > > >> > > Hi, > > >> > > I am trying to find out ways to improve Airavata provenance aware > > >> workflow > > >> > > processing based on rich metadata available in Apache OODT. As > part > > >>of > > >> this > > >> > > task, I am investigating how Apache Airavata is currently > performing > > >> > > provenance aware workflow processing. I have looked into the > > >>provenance > > >> > > manager implementation, where I could see lot of data related to > > >> workflow > > >> > > execution is saved in registry. But I could see very little of use > > >>of > > >> those > > >> > > data. Could some one please explain the details of provenance > aware > > >> > > processing in Airavata and any future plans on this area? > > >> > > > > >> > > Best Regards, > > >> > > Sanjaya > > >> > > > > >> > > > >> > -- > > >> > Lewis > > >> > > > >> > > >> > > > > > > > >
