Provide cloud bursting like capabilities to Airavata computational workflows
integrating with Apache Whirr
----------------------------------------------------------------------------------------------------------
Key: AIRAVATA-357
URL: https://issues.apache.org/jira/browse/AIRAVATA-357
Project: Airavata
Issue Type: New Feature
Components: GFac, Workflow Interpreter, XBaya
Reporter: Suresh Marru
Fix For: WISHLIST
Apache Airavata provides capabilities to construct execute and monitor
computational workflows with built in providers to execute applications on
compute intensive resources. The users of Airavata constantly have the need to
execute workflows which have tasks with a hybrid combination of computational
grids and computational clouds. More over, some applications can be executed on
multiple types of resources. The selection will depend on the distribution of
data to be processed, the tolerable latency on shared batch queued grid
clusters, and on the nature and size of the problem to be solved and data
analysis tasks. Especially for applications which reduce the data as they
proceed in the graph, they better fit for local map-reduce executions residing
on hadoop based file systems.
This idea has two implementation paths:
Firstly extend airavata workflow providers to integrate with cloud based run
times. Integrating with higher level API's like Apache Whirr provides a great
value addition to this project to encompass multiple cloud services.
Secondly, extending Airavata enactment called Workflow Interpreter so a
component in the workflow is capable of initially running on local clusters
deploying hadoop file systems and if the execution is saturating the local
resources, the executions can scale out to commercial on-demand clouds or
leverage high speed low latency interconnects on computational grids. This
scaling technique is often referred to as Cloud Bursting. This project will not
only enable this capability to Airavata workflow.
User community & Impact of the software: Airavata is primarily targeted to
build science gateways using computational resources from various disciplines.
The initial targeted set of gateways include projects supporting research and
education in chemistry, life sciences, biophysics, environmental sciences,
geosciences astronomy and nuclear physics. The goal of airavata is to enhance
productivity of these gateways to utilize cyberinfrastructure of resources
(e.g., local lab resources, the Extreme Science and Engineering Discovery
Environment (XSEDE), the Open Science Grid (OSG), University Clusters, Academic
and Commercial Computational Clouds like FutureGrid & Amazon EC2). By using
open community based software components and services like Airavata, gateways
will be able to focus on providing additional scientific capabilities and to
expanding the number of supported users. The capabilities of these gateways
will offer clear benefits to society.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
