Outputs are files, those need to moved to gateway (portal) or user machine. We currently use GridFTP[1]/SCP/GSISCP libraries for file movement . We explored https://www.globus.org/ service also some time back but don’t have a direct integration yet. Globus service can allow Airavata users to move files to and from their workstation.
Thanks Raminder 1. http://toolkit.globus.org/toolkit/docs/latest-stable/gridftp/ On May 31, 2014, at 11:39 AM, Gagan Juneja <[email protected]> wrote: > Hi Team, > > Where do we keep the output of a job? Is it somewhere in our database, given > back to user or configurable behavior? > > > Regards, > Gagan > > > On Tue, May 27, 2014 at 9:49 PM, Gagan Juneja <[email protected]> > wrote: > Thanks Suresh!! for sharing details about EC2 integration stuff, I am reading > these resources. I would love to contribute in designing and implementing new > gen XBaya but before that I want to have complete understanding of all the > modules in Airavata so that I can contribute in much effective way. > > > > Regards, > Gagan > > > On Mon, May 26, 2014 at 11:21 PM, Suresh Marru <[email protected]> wrote: > Hi Gagan, > > It was nice talking to you at ApacheCon. We will appreciate your > contributions. There is a ongoing Google Summer of code project by Nipun > Udara to revisit EC2 integration. We are moving towards integrating JClouds > API. Some examples of Amazon use of Airavata can be found at [1] [2] [3]. > > If you are looking for something concrete to contribute, one important one > will be to determine the next generation XBaya. Are you interested? If so, > please sign up to Airavata Architecture mailing list > (http://airavata.apache.org/community/mailing-lists.html) and I will start a > discussion there. > > Suresh > [1] - http://www.biomedcentral.com/content/pdf/2047-2501-1-6.pdf > [2] - http://www.cs.ucsb.edu/~rich/publications/book2010.pdf#page=324 > [3] - http://grids.ucs.indiana.edu/ptliupages/publications/biovlab-mmia.v9.pdf > > > On May 26, 2014, at 1:26 PM, Marlon Pierce <[email protected]> wrote: > > > Hi Gagan-- > > > > I think you are looking for GFAC, which provides the connection between > > Airavata and the backend resource. There is a description of GFAC here > > [1]. In short, GFAC has two cooperating plugins: a provider is a client > > to a remote resource (Grid, cloud, etc) and a handler supports the > > provider to do specialized tasks for a specific scenario. > > > > We have a simple tutorial document at [2] on writing handlers (this > > should move to the wiki soon), and you can find several example > > providers in the GFAC module of Airavata. > > > > Marlon > > > > [1] > > https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=40511561 > > > > [2] > > https://docs.google.com/document/d/1RdR2vUIGzV_Nd0wBBCwmzE-Lpc6i6z_jtOgD1L4wUQ4/edit > > > > > > On 5/26/14 1:15 PM, Gagan Juneja wrote: > >> Hi Marlon, > >> > >> This make sense. I will take some time to understand PHP code but meanwhile > >> what I understood is We have two apis. > >> > >> Client api helps us in creating Experiment and other stuff and put them > >> into some database. > >> Orchestrator api helps in submitting these experiments as jobs for > >> execution. > >> > >> Obviously we are going to perform some business logic or some computational > >> stuff on input data that we set as part of experiment. But where are we > >> writing that code? > >> > >> > >> As far as Cloud is concern do let me know when you think of running this > >> over Amazon Cloud. I am working with Amazon cloud for last 6 months and > >> have good experience on automating stuff as well like one click EC2 cluster > >> spawning etc. > >> > >> > >> Regards, > >> Gagan > >> > >> > >> On Mon, May 26, 2014 at 9:27 PM, Marlon Pierce <[email protected]> wrote: > >> > >>> Hi Gagan-- > >>> > >>> Thanks for the questions and please keep them coming. > >>> > >>> Airavata 0.11 from late last year is our latest release, and the trunk > >>> is now very different. Airavata 0.11 is the last version with support > >>> for XBaya and an older version of the client API (developed mostly by > >>> Saminda) that has full support for workflows. > >>> > >>> We made decisions after Airavata 0.11 to make some major changes. > >>> > >>> * Concentrate on Science Gateway use cases at the expense of workflows > >>> in the near term. Gateways typically run single jobs through a web > >>> browser. Scaling and multi-tenancy are the challenges. Hopefully "near > >>> term" is drawing to a close. > >>> > >>> * Introduce a new component called the Orchestrator that manages job > >>> submissions and provide a simpler mechanism for single job submissions > >>> (see previous bullet). Previously, a user would need to define an > >>> entire workflow for a single job, which was pretty complicated for most > >>> of our driving use cases. > >>> > >>> * Define our API in Thrift. This has several advantages in addition to > >>> the multi-language support. Thrift can support richer data models, API > >>> methods, and exceptions than REST, which was a good fit for our project. > >>> > >>> * Make significant changes to the Registry to support the new API. > >>> > >>> * Put workflow support on lower priority until the Thrift API, Registry, > >>> and Orchestrator were all stable. > >>> > >>> These have resulted in several changes that are finally getting wrapped > >>> up and will be released in Airavata 0.12 (tentatively in mid-June). > >>> After this, we will get back to more frequent releases and also > >>> resurrect the workflow work. Besides workflows, revising the way > >>> Airavata manages its application and resource descriptions (called the > >>> Application Catalog) will be a big focus. > >>> > >>> Airavata typically gets used to submit jobs to Grids like the NSF's > >>> XSEDE, which use Globus GRAM, UNICORE, or GSI-SSH clients. Other > >>> resources (clouds, the Open Science Grid, non-US Grids) are interesting > >>> to us but we just don't have the resources to look at these much. These > >>> are interesting places for contributions. > >>> > >>> At this moment, the most actively developed Thrift clients are > >>> temporarily outside Airavata's Git repo, although these should be merged > >>> back soon: > >>> > >>> * https://github.com/SciGaP/Airavata-PHP-Client-Samples > >>> > >>> * https://github.com/SciGaP/PHP-Reference-Gateway/ > >>> > >>> Marlon > >>> > >>> > >>> On 5/26/14 11:30 AM, Gagan Juneja wrote: > >>>> Thanks Saminda and Marlon for your prompt response on my query. > >>>> > >>>> I understand from Airavata is "Airavata is a platform which helps user to > >>>> execute his workflow over any distributed environment it could be Hadoop > >>> or > >>>> anything else of same sort." Is this understanding correct. > >>>> > >>>> I am having following question. > >>>> 1. How this project is being used by the users or client any brief idea. > >>>> Are we exposing cloud, grids as a service or user needs to implement on > >>>> their on. > >>>> 2. I have looked at class CreateLaunchExperimentUS3.java. Here most of > >>> the > >>>> work is like creating an experiment and launching it. I am not able to > >>>> find any thing related to workflow or job that we want to run this input > >>>> data that we are setting in Experiment object. Where can we create Jobs, > >>>> tasks or workflows? > >>>> 3. Why do Xbaya is deprecated and replaced with thrift API? (Anything > >>>> special apart from multi language support.) > >>>> > >>>> > >>>> I have lot more question will keep bothering you :). > >>>> > >>>> Regards, > >>>> Gagan > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> > >>>> On Sun, May 25, 2014 at 11:31 PM, Marlon Pierce <[email protected]> wrote: > >>>> > >>>>> The QuickStart info is out of date, and we will be updating it soon. > >>>>> After compiling and starting the server, you can try out PHP sample > >>>>> scripts in > >>>>> > >>>>> > >>> ./airavata-api/airavata-client-sdks/airavata-php-sdk/src/main/resources/samples/ > >>>>> > >>>>> Marlon > >>>>> > >>>>> On 5/24/14 4:13 AM, Saminda Wijeratne wrote: > >>>>>> XBaya is deprecated in the upcoming release. Please find all the > >>> release > >>>>>> artifacts under modules/distribution/release/target/release-artifacts. > >>>>>> > >>>>>> > >>>>>> On Fri, May 23, 2014 at 11:14 PM, Gagan Juneja < > >>>>> [email protected]>wrote: > >>>>>>> Hi Team, > >>>>>>> I am new to Airavata project. I was following Quick Start document. I > >>>>>>> build Airavata project using default profile. I did not find any > >>>>> artifact > >>>>>>> in modules/distribution target directory and even many of the modules > >>>>> did > >>>>>>> not build such as xbaya. > >>>>>>> > >>>>>>> Please guide me how to start with this. > >>>>>>> > >>>>>>> Thanks & Regards, > >>>>>>> Gagan > >>>>>>> > >>> > > > > >
