Repository: airavata Updated Branches: refs/heads/0.14_release b6d9a5e4d -> d314cdb4f
update README, INSTALL - AIRAVATA-1370 Project: http://git-wip-us.apache.org/repos/asf/airavata/repo Commit: http://git-wip-us.apache.org/repos/asf/airavata/commit/d314cdb4 Tree: http://git-wip-us.apache.org/repos/asf/airavata/tree/d314cdb4 Diff: http://git-wip-us.apache.org/repos/asf/airavata/diff/d314cdb4 Branch: refs/heads/0.14_release Commit: d314cdb4fefdc32229e9469afb5e44491e6248ca Parents: b6d9a5e Author: Chathuri Wimalasena <[email protected]> Authored: Fri Dec 12 11:04:32 2014 -0500 Committer: Chathuri Wimalasena <[email protected]> Committed: Fri Dec 12 11:04:32 2014 -0500 ---------------------------------------------------------------------- INSTALL | 12 +--- README | 59 ++++++++++------ RELEASE_NOTES | 74 ++++++++++++++++++++ .../airavata-cpp-sdk/src/main/resources/INSTALL | 3 +- .../client/java/src/main/resources/INSTALL | 31 -------- .../client/java/src/main/resources/README | 66 ++++------------- .../server/src/main/resources/INSTALL | 47 +++---------- .../server/src/main/resources/README | 73 +++++++++++-------- 8 files changed, 184 insertions(+), 181 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/INSTALL ---------------------------------------------------------------------- diff --git a/INSTALL b/INSTALL index 24dddc8..ede4598 100644 --- a/INSTALL +++ b/INSTALL @@ -1,13 +1,9 @@ For detailed installation and further instructions please refer http://airavata.apache.org/: Documentation section in left hand panel. The website lists step by step instructions -Installing Apache Airavata 0.12 +Installing Apache Airavata 0.14 ------------------------------------------------ -IMPORTANT Note: The installation instructions are out of date. 0.12 release is a developer oriented release. Please -contact the developer mailing list for help. A subsequent release 0.13 will soon supercede. - - Prerequisites ------------- Java 1.5 or later @@ -15,7 +11,7 @@ Maven (tested on v 3.0.2) Build Apache Airavata from Source --------------------------------- -* Unzip/untar the source file or check out from svn. +* Unzip/untar the source file or clone from git. * cd to project folder and type $ mvn clean install Note: in order to skip tests use the command @@ -34,6 +30,4 @@ Running Tests Tutorials ---------- The airavata website has instructions for basic tutorials: -* For basic understanding of how Airavata works - http://airavata.apache.org/documentation/tutorials/airavata-in-5-minutes.html -* Describing and executing applications using Airavata - http://airavata.apache.org/documentation/tutorials/airavata-in-10-minutes.html -* Advanced tutorial to provide understanding of how to run sample workflows distributed with Airavata - http://airavata.apache.org/documentation/tutorials/advanced-workflow-samples.html +* Describing and executing applications using Airavata - follow "XBAYA Quick-Start Tutorial" tutorial at https://cwiki.apache.org/confluence/display/AIRAVATA/XBAYA+Quick-Start+Tutorial \ No newline at end of file http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/README ---------------------------------------------------------------------- diff --git a/README b/README index f8b0989..c2223ff 100644 --- a/README +++ b/README @@ -44,37 +44,49 @@ Description of Airavata Directory Structure Airavata Source Distribution Directory Structure ================================================ - AIRAVATA_TRUNK + AIRAVATA_MASTER + âââ airavata-api âââ modules â âââ airavata-client + â âââ app-catalog â âââ commons â â âââ gfac-schema â â âââ utils â â âââ workflow-execution-context â â âââ workflow-tracking + â âââ credential-store-service â âââ distribution - â â âââ airavata-client - â â âââ airavata-server - â â âââ xbaya-gui + â â âââ api-server + â â âââ client + â â âââ gfac-server + â â âââ orchestrator-server + â â âââ server â â âââ release - â âââ gfac-core + â â âââ xbaya-gui + â âââ gfac + â â âââ airavata-gfac-service + â â âââ gfac-bes + â â âââ gfac-core + â â âââ gfac-ec2 + â â âââ gfac-gram + â â âââ gfac-gsissh + â â âââ gfac-hadoop + â â âââ gfac-local + â â âââ gfac-monitor + â â âââ gfac-ssh + â â âââ gfac-thrift-descriptions â âââ integration-tests + â âââ messaging + â âââ orchestrator â âââ registry â â âââ airavata-jpa-registry - â â âââ airavata-registry-service - â â âââ airavata-registry-test - â â âââ registry-api - â âââ rest - â â âââ client - â â âââ mappings - â â âââ service - â â âââ webapp + â â âââ registry-cpi â âââ security â âââ credential-store â âââ server â âââ test-suite - â âââ workflow-interpreter â âââ workflow-model + â â âââ workflow-engine â â âââ workflow-model-component-node â â âââ workflow-model-core â âââ ws-messenger @@ -87,6 +99,8 @@ Airavata Source Distribution Directory Structure â âââ xbaya-gui âââ samples âââ tools + â âââ gsissh + â âââ gsissh-cli-tools â âââ phoebus-integration â âââ registry-migrate âââ INSTALL @@ -104,10 +118,14 @@ Server Distributions The Airavata Server binary distribution allows you to run a standalone Airavata Server which includes all the airavata services shipped with a default derby database as the backend registry. For stable purposes, a mysql configuration is recommended. -* Airavata Server Web Application - This is similar package as the Airavata Server but is distributed as the server Web Application archive. - This war is compatible with Apache Tomcat application server. The war bundles all airavata services - enabled by defualt to startup a derby database as the backend registry. For stable purposes, a mysql configuration is recommended. +* Airavata API Server + This is the server that contains Airavata API Server. + +* Airavata Orchestrator Server + This is the stand-alone orchestrator server + +* Airavata GFac Server + This is the standalone GFac Server Client Distributions -------------------- @@ -123,6 +141,5 @@ Client Distributions =========================== * If you built Airavata from source, and if you see "BUILD SUCCESS", then the test cases should have passes. * The test cases are beyond unit level, they startup embedded services and run through basic workflow use cases. -* To walk through Airavata features, follow "Airavata in Five Minutes" tutorial at http://airavata.apache.org/documentation/tutorials/airavata-in-5-minutes.html. -* For intermediate level Airavata features, follow "Airavata in Ten Minutes" tutorial at http://airavata.apache.org/documentation/tutorials/airavata-in-5-minutes.html. -* For advanced use cases, please contact mailing lists - http://airavata.apache.org/community/mailing-lists.html +* To walk through Airavata features, follow "XBAYA Quick-Start Tutorial" tutorial at https://cwiki.apache.org/confluence/display/AIRAVATA/XBAYA+Quick-Start+Tutorial. +* For advanced use cases, please contact mailing lists - http://airavata.apache.org/community/mailing-lists.html http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/RELEASE_NOTES ---------------------------------------------------------------------- diff --git a/RELEASE_NOTES b/RELEASE_NOTES index 0fcaab2..df59a9d 100644 --- a/RELEASE_NOTES +++ b/RELEASE_NOTES @@ -2,6 +2,80 @@ Apache Airavata Release Notes See http://issues.apache.org/jira/browse/AIRAVATA-* (where * is the number of the issue below) +Apache Airavata Version 0.14 Released on 2014-12-20 +=================================================== +Below you will find the detail list of JIRA tasks/issues resolutions incorporated in 0.14. + +Release Notes - Airavata - Version 0.14 + +** Tasks + * AIRAVATA-1063 HostAddress in ComputationalResourceScheduling is required. + * AIRAVATA-1471 Modify Airavata Distribution to Enable Separate Distributions for Orchestrator, Gfac and Xbaya. + * AIRAVATA-1467 Zookeeper Implementation to Advertise Available/up and running Airavata Services/Components + * AIRAVATA-1463 Remove airavata-client-api dependency from other module + * AIRAVATA-1511 Replace legacy application descriptors used in gfac with app-catalog + +** Sub-tasks + * AIRAVATA-1009 Craft the Workflow Interpreter CPI + * AIRAVATA-1446 Provide a way to enable and disable Job count validation + * AIRAVATA-1444 Implement RabbitMQ publisher + * AIRAVATA-1438 Add throtle-resources flag to airavata API + * AIRAVATA-1472 Cleanup Airavata property file for old service properties + * AIRAVATA-1428 Add a inmemory job count map to Orchestrator + * AIRAVATA-1427 Submit real time job count details to zookeeper + * AIRAVATA-1389 Updated Xbaya to enable execution of a single application through thrift api + * AIRAVATA-1510 Remove connection to Workflow service from Xbaya + * AIRAVATA-468 Support option with parameters for GramProvider + +** Story + * AIRAVATA-965 Update Airavata JGLOBUS and Unicore jar + +** New Feature + AIRAVATA-1494 Add ResourceJobManger add, update, get, delete methods to airavata-api + +** Improvement + * AIRAVATA-1036 ActualParameter and DataObjectType have same objective + * AIRAVATA-1030 Multiple Security context to run a grid job + * AIRAVATA-899 API need enhancement so user can set a user identifier into Context header + * AIRAVATA-1487 Add API methods to get job submission information + * AIRAVATA-1479 Change return type of jobSubmission API methods to string + * AIRAVATA-1474 Remove Registry.wsdl stub autogeneration + * AIRAVATA-1451 RabbitMQ messaging is not supported for SGE resources + * AIRAVATA-1449 Get partial results incase outputs are not produced. + * AIRAVATA-1464 XBaya doesn't support multi-app workflows with new Thrift API + * AIRAVATA-1462 Update log messages to track life cycle of experiments , jobs and tasks. + * AIRAVATA-1405 API function to get data transfer details. + * AIRAVATA-1404 Airavata API function to get Job details + * AIRAVATA-1410 Input data transfer details are not updated + * AIRAVATA-1419 Allow multiple client to move the data using same server + * AIRAVATA-1333 Can we have a way of searching for all the projects, experiments in the gateway? + * AIRAVATA-495 Implement logger in WorkflowInterpreter + * AIRAVATA-473 Support For-each for Local providers + +** Bug + * AIRAVATA-1370 Update Client samples READMEs and INSTALL files + * AIRAVATA-1418 App catalog database connection timeout issue + * AIRAVATA-1413 Output path is not getting set to output context incase we scan output directory + * AIRAVATA-1091 PersistenceException while executing BES or Gram Provider + * AIRAVATA-986 No free ports available to run GRAM provider and GridFTP transfers + * AIRAVATA-1495 Experiment Status is not updated to Complete + * AIRAVATA-1488 AdvancedSCPInputHandler is not working if we need to transfer data to BigRed + * AIRAVATA-1477 Host Aliases and IpAddresses not getting saved in Compute Resource Object + * AIRAVATA-1435 Experiment Status is updated to completed on job complete + * AIRAVATA-1434 Rewrite CreateLaunchExperiment class in order to work with registerApplication method + * AIRAVATA-1433 NullPointerException to create a SSH connection + * AIRAVATA-1430 GSISSHAbstractCluster not throwing error if there is issue submitting job + * AIRAVATA-1423 Orchestrator context does't set initial GFac instant list correctly in one of it's constructor. + * AIRAVATA-1409 Error details are saved twice which adding two records in the database + * AIRAVATA-1402 Command for SLURM job cancel need correction + * AIRAVATA-1542 Error message when configuring XBaya input parameter configuration + * AIRAVATA-1521 SSHInputHandler cluster object is null + * AIRAVATA-1520 SCP to a newer version of Centos VM does not work + * AIRAVATA-911 Unable to set a different gateway name other than "default" + * AIRAVATA-767 Provide GridFTP URL incase of URL and URLArray type + * AIRAVATA-723 Error message not sent to Xbaya as fault + + Apache Airavata Version 0.12 Released on 2014-06-27 =================================================== This release is a first iteration of using Apache thrift as a framework to implement Airavata API. http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/airavata-api/airavata-client-sdks/airavata-cpp-sdk/src/main/resources/INSTALL ---------------------------------------------------------------------- diff --git a/airavata-api/airavata-client-sdks/airavata-cpp-sdk/src/main/resources/INSTALL b/airavata-api/airavata-client-sdks/airavata-cpp-sdk/src/main/resources/INSTALL index c815ce0..50c7822 100644 --- a/airavata-api/airavata-client-sdks/airavata-cpp-sdk/src/main/resources/INSTALL +++ b/airavata-api/airavata-client-sdks/airavata-cpp-sdk/src/main/resources/INSTALL @@ -1,4 +1,4 @@ -Installing Apache Airavata Client 0.11 +Installing Apache Airavata Client 0.14 -------------------------------------- Prerequisites @@ -28,4 +28,3 @@ Tutorials ---------- The airavata website has instructions for basic tutorials: * For basic understanding on how to use Airavata API please look at the samples shipped with the distribution -* Follow Airavata Wiki for more examples on how to use the Airavata API - https://cwiki.apache.org/confluence/display/AIRAVATA/Gateway+Developer+Guide http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/modules/distribution/client/java/src/main/resources/INSTALL ---------------------------------------------------------------------- diff --git a/modules/distribution/client/java/src/main/resources/INSTALL b/modules/distribution/client/java/src/main/resources/INSTALL deleted file mode 100644 index c815ce0..0000000 --- a/modules/distribution/client/java/src/main/resources/INSTALL +++ /dev/null @@ -1,31 +0,0 @@ -Installing Apache Airavata Client 0.11 --------------------------------------- - -Prerequisites -------------- -Java 1.5 or later -Maven (tested on v 3.0.2) - -Build Apache Airavata from Source ---------------------------------- -* Unzip/untar the source file or check out from svn. -* cd to project folder and type - $ mvn clean install - Note: in order to skip tests use the command - $ mvn clean install -Dmaven.test.skip=true -* The compressed binary distribution is created at <PROJECT DIR>/modules/distribution/airavata-client/target - -Installing the Airavata Client Libraries ----------------------------------------- -* Add all the libraries (jar files) in the <AIRAVATA_CLIENT_HOME>/lib directory to the classpath -* Add the <AIRAVATA_CLIENT_HOME>/conf directory to the classpath - -Running Tests -------------- -Once the binary is unzipped, instructions to run the tests should be follow from README - -Tutorials ----------- -The airavata website has instructions for basic tutorials: -* For basic understanding on how to use Airavata API please look at the samples shipped with the distribution -* Follow Airavata Wiki for more examples on how to use the Airavata API - https://cwiki.apache.org/confluence/display/AIRAVATA/Gateway+Developer+Guide http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/modules/distribution/client/java/src/main/resources/README ---------------------------------------------------------------------- diff --git a/modules/distribution/client/java/src/main/resources/README b/modules/distribution/client/java/src/main/resources/README index 7853c5f..4ed133e 100644 --- a/modules/distribution/client/java/src/main/resources/README +++ b/modules/distribution/client/java/src/main/resources/README @@ -21,48 +21,7 @@ Airavata Client Distribution ============================ This distribution is a set of libraries and configurations files that allow a 3rd party -application to programatically access Airavata functionality through Airavata API. - - AIRAVATA_CLIENT - âââ conf - â âââ airavata-client.properties - â âââ log4j.properties - âââ lib - âââ samples - âââ create-application - âââ workflow-run - âââ INSTALL - âââ LICENSE - âââ NOTICE - âââ README - - -How to test and run samples -=========================== -* If you built Airavata from source, and if you see "BUILD SUCCESS", then the test cases should have passes. -* The samples directory contains samples on 2 base use cases on Airavata Client API. See the README files contained - within the samples for more details. -* For more examples on how to use the Airavata API go to https://cwiki.apache.org/confluence/display/AIRAVATA/Gateway+Developer+Guide -* For advanced use cases, please contact mailing lists - http://airavata.apache.org/community/mailing-lists.html - -Description of Directory Structure -================================== - - samples - This contains 2 basecase usecases on how to use Airavata client. Both the samples are having its README file - So users have to refer each README file before running each sample. - - - lib - This contains all the libraries required to run the client side code. Add these libraries to the classpath of java applications which uses Airavata - Client API. - - - conf - Contains all the configuration files for airavata client API to work. Refer the comments in the configuration file for more details. - - - README - This document. - - - INSTALL - This document will contain information on how to use Airavata Client API libraries. +application to programmatically access Airavata functionality through Airavata API. Other Available Distributions @@ -71,13 +30,17 @@ Other Available Distributions Server Distributions -------------------- * Airavata Server - The Airavata Server binary distribution allows you to run a standalone Airavata Server which includes all the airavata services - shipped with a default derby database as the backend registry. For stable purposes, a mysql configuration is recommended. + The Airavata Server binary distribution allows you to run a standalone Airavata Server which includes all the airavata services + shipped with a default derby database as the backend registry. For stable purposes, a mysql configuration is recommended. + +* Airavata API Server + This is the server that contains Airavata API Server. + +* Airavata Orchestrator Server + This is the stand-alone orchestrator server -* Airavata Server Web Application - This is similar package as the Airavata Server but is distributed as the server Web Application archive. - This war is compatible with Apache Tomcat application server. The war bundles all airavata services - enabled by defualt to startup a derby database as the backend registry. For stable purposes, a mysql configuration is recommended. +* Airavata GFac Server + This is the standalone GFac Server Client Distributions -------------------- @@ -85,7 +48,6 @@ Client Distributions The Airavata XBaya distribution is a client GUI application with features to register applications as web services, construct workflows, execute and monitor workflows and browse the generated results from the airavata registry. -* Airavata XBaya JNLP - The Airavata XBaya JNLP distribution is the simular GUI distribution but prepackeged to be ready to be deployed to - a web server as a web start application. The GUI provides features to register applications as web services, construct workflows, - execute and monitor workflows and browse the generated results from the airavata registry. +* Airavata Client + The Airavata Client distribution is a set of libraries and configurations files that allow a 3rd party application to programatically + access Airavata functionality through Airavata API. \ No newline at end of file http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/modules/distribution/server/src/main/resources/INSTALL ---------------------------------------------------------------------- diff --git a/modules/distribution/server/src/main/resources/INSTALL b/modules/distribution/server/src/main/resources/INSTALL index 0324e61..53d0550 100644 --- a/modules/distribution/server/src/main/resources/INSTALL +++ b/modules/distribution/server/src/main/resources/INSTALL @@ -1,4 +1,4 @@ -Installing Apache Airavata 0.11 +Installing Apache Airavata 0.14 ------------------------------- Prerequisites @@ -8,48 +8,23 @@ Maven (tested on v 3.0.2) Build Apache Airavata from Source --------------------------------- -* Unzip/untar the source file or check out from svn. +* Unzip/untar the source file or clone from git. * cd to project folder and type $ mvn clean install Note: in order to skip tests use the command $ mvn clean install -Dmaven.test.skip=true -* The compressed binary distribution is created at <PROJECT DIR>/modules/distribution/airavata-server/target/apache-airavata-server-<airavata-version>-bin.zip - -Installing the Airavata Server ------------------------------- -No installation is necessary. Just extract the compressed distribution. -Note: For customizing the default configurations of the Airavata Server please - refer to Airavata web-site (http://airavata.apache.org/) and/or Airavata - mailing lists (http://airavata.apache.org/community/mailing-lists.html) - -Starting Apache Airavata Server -------------------------------- -* Navigate to <AIRAVATA_HOME>/bin -* type for following command to start the Airavata Server - MAC/Unix systems - $ sh airavata-server.sh - Windows - > airavata-server.bat - Note: Pass "-h" as parameters to see more options when starting the server - -Starting Apache Derby Server -------------------------------- -Users have the option to star the derby server separately -* Navigate to <AIRAVATA_HOME>/bin -* type for following command to start the Airavata Server - MAC/Unix systems - $ sh derby.sh - Windows - <Not supported in this version> - Note: Pass "-h" as parameters to see more options when starting the server +* Alternatively, all compressed binary distributions can be found at <PROJECT DIR>/modules/distribution/release/target/release-artifacts Running Tests ------------- -Once the binary is unzipped, instructions to run the tests should be followed from README +* Unit tests & integrations tests will run while Apache Airavata is built from source (without "-Dmaven.test.skip=true"). +* To run the test samples + - You can find the binary distributions at <PROJECT DIR>/modules/distribution/release/target/release-artifacts or from + the Apache Airavata download site. + - Extract the binary distributions and once the binary is unzipped, instructions to run the tests should be followed + from README files found within. -Tutorials +Tutorials ---------- The airavata website has instructions for basic tutorials: -* For basic understanding of how Airavata works - http://airavata.apache.org/documentation/tutorials/airavata-in-5-minutes.html -* Describing and executing applications using Airavata - http://airavata.apache.org/documentation/tutorials/airavata-in-10-minutes.html -* Advanced tutorial to provide understanding of how to run sample workflows distributed with Airavata - http://airavata.apache.org/documentation/tutorials/advanced-workflow-samples.html +* Describing and executing applications using Airavata - follow "XBAYA Quick-Start Tutorial" tutorial at https://cwiki.apache.org/confluence/display/AIRAVATA/XBAYA+Quick-Start+Tutorial \ No newline at end of file http://git-wip-us.apache.org/repos/asf/airavata/blob/d314cdb4/modules/distribution/server/src/main/resources/README ---------------------------------------------------------------------- diff --git a/modules/distribution/server/src/main/resources/README b/modules/distribution/server/src/main/resources/README index 1539b8c..fb064a1 100644 --- a/modules/distribution/server/src/main/resources/README +++ b/modules/distribution/server/src/main/resources/README @@ -18,48 +18,61 @@ airavata services shipped with a default derby database as the backend registry. Release Notes ============= -0.11 is the tenth release of Airavata (skipped 0.1-INCUBATNG). This release focuses bug fixes and GSISSH library for beta testing. For detailed tasks list, please see RELEASE_NOTES. +0.14 is the thirteenth release of Airavata (skipped 0.1-INCUBATNG). For detailed tasks list, please see RELEASE_NOTES. Building from source ==================== For brief installation instructions, see INSTALL For detailed installation and further instructions refer http://airavata.apache.org/ - Documentation section in left hand panel. Step by step with proper documentation are provided. -Known Issues in This Release -============================ -This is the base release and is focused on a good foundation and less on features. This -version is not recommended for production usage. Airavata Binary Distribution Directory Structure ================================================ AIRAVATA_HOME - âââ bin - â âââ database_scripts <dir> - â âââ airavata-server.bat - â âââ airavata-server.properties - â âââ airavata-server.sh - â âââ authenticators.xml - â âââ axis2.xml - â âââ derby.sh - â âââ host.xml - â âââ log4j.properties - â âââ logo.txt - â âââ setenv.bat - â âââ setenv.sh - â âââ startNetworkServer - âââ lib <dir> - âââ repository - â âââ modules - â âââ services - âââ samples - â âââ workflows <dir> - â âââ echo_out.sh - â âââ echo.sh âââ INSTALL - âââ LICENSE - âââ NOTICE - âââ README + âââ LICENSE + âââ NOTICE + âââ README + âââ RELEASE_NOTES + âââ bin + â  âââ PBSTemplate.xslt + â  âââ SGETemplate.xslt + â  âââ SLURMTemplate.xslt + â  âââ airavata-server.bat + â  âââ airavata-server.properties + â  âââ airavata-server.sh + â  âââ api-server.sh + â  âââ database_scripts + â  â  âââ appcatalog-derby.sql + â  â  âââ appcatalog-mysql.sql + â  â  âââ cleanDBScript.sql + â  â  âââ msgBox-derby.sql + â  â  âââ msgBox-mysql.sql + â  â  âââ msgBroker-derby.sql + â  â  âââ msgBroker-mysql.sql + â  â  âââ registry-derby.sql + â  â  âââ registry-mysql.sql + â  âââ derby.sh + â  âââ gfac-config.xml + â  âââ gfac-server.sh + â  âââ gsissh.properties + â  âââ log4j.properties + â  âââ logo.txt + â  âââ orchestrator-server.sh + â  âââ setenv.bat + â  âââ setenv.sh + â  âââ startNetworkServer + â  âââ workflow-server.sh + â  âââ zoo.cfg + âââ lib + âââ samples + âââ registerSample.sh + âââ scripts + âââ add.sh + âââ echo.sh + âââ multiply.sh + âââ subtract.sh How to test and run samples
