lahirujayathilake commented on code in PR #507: URL: https://github.com/apache/airavata/pull/507#discussion_r2109335097
########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop Review Comment: I believe this step is no longer needed as the `master` branch has everything needed to run in the ide ########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop -### Setting up the development environment +# Build the project (this may take a few minutes) +mvn clean install -DskipTests +``` + +### 2๏ธโฃ Open in IntelliJ IDEA -* Clone Airavata repository to a local directory +1. **Launch IntelliJ IDEA** +2. **Open Project** โ Navigate to your cloned `airavata` directory +3. **Navigate to:** `modules` โ `ide-integration` module +4. **Wait for indexing** to complete - ``` - git clone https://github.com/apache/airavata - ``` +## ๐ณ Backend Services Setup -* Checkout develop branch +### 3๏ธโฃ Configure Host Resolution - ``` - git checkout develop - ``` -* Build the develop branch using Maven +Add the following entry to your system's hosts file: - ``` - mvn clean install -DskipTests - ``` -* Open the project using InteliJ IDEA +**Linux/macOS:** `/etc/hosts` +**Windows:** `C:\Windows\System32\drivers\etc\hosts` -* Browse to modules -> ide-integration module +```bash +127.0.0.1 airavata.host +``` -### Starting backend components (Database, Keycloak, Kafka, RabbitMQ, SSHD Server) +### 4๏ธโฃ Start Backend Services -* Add a host entry to /etc/hosts file in local machine +Navigate to the containers directory and start all required services: - ``` - 127.0.0.1 airavata.host - ``` +```bash +cd modules/ide-integration/src/main/containers +docker-compose up -d +``` -* Go to src/main/containers directory and run +**Services Started:** +- ๐๏ธ **MySQL Database** +- ๐ **Keycloak** (Authentication) +- ๐จ **Apache Kafka** (Messaging) +- ๐ฐ **RabbitMQ** (Message Queue) +- ๐ **SSHD Server** (Secure connections) - ``` - docker-compose up - ``` +### 5๏ธโฃ Initialize Database + +Apply database migrations: + +```bash +cd modules/ide-integration/src/main/containers +cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 +``` -* Apply any database migrations. Go to src/main/containers directory and run +โณ **Wait for all services to be ready** (usually 2-3 minutes) Review Comment: better to remove the time estimation, as startup time may vary across user environments ########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop -### Setting up the development environment +# Build the project (this may take a few minutes) +mvn clean install -DskipTests +``` + +### 2๏ธโฃ Open in IntelliJ IDEA -* Clone Airavata repository to a local directory +1. **Launch IntelliJ IDEA** +2. **Open Project** โ Navigate to your cloned `airavata` directory +3. **Navigate to:** `modules` โ `ide-integration` module +4. **Wait for indexing** to complete Review Comment: minor comment, I think it's ok to remove this step 4 ########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop -### Setting up the development environment +# Build the project (this may take a few minutes) +mvn clean install -DskipTests +``` + +### 2๏ธโฃ Open in IntelliJ IDEA -* Clone Airavata repository to a local directory +1. **Launch IntelliJ IDEA** +2. **Open Project** โ Navigate to your cloned `airavata` directory +3. **Navigate to:** `modules` โ `ide-integration` module +4. **Wait for indexing** to complete - ``` - git clone https://github.com/apache/airavata - ``` +## ๐ณ Backend Services Setup -* Checkout develop branch +### 3๏ธโฃ Configure Host Resolution - ``` - git checkout develop - ``` -* Build the develop branch using Maven +Add the following entry to your system's hosts file: - ``` - mvn clean install -DskipTests - ``` -* Open the project using InteliJ IDEA +**Linux/macOS:** `/etc/hosts` +**Windows:** `C:\Windows\System32\drivers\etc\hosts` -* Browse to modules -> ide-integration module +```bash +127.0.0.1 airavata.host +``` -### Starting backend components (Database, Keycloak, Kafka, RabbitMQ, SSHD Server) +### 4๏ธโฃ Start Backend Services -* Add a host entry to /etc/hosts file in local machine +Navigate to the containers directory and start all required services: - ``` - 127.0.0.1 airavata.host - ``` +```bash +cd modules/ide-integration/src/main/containers +docker-compose up -d +``` -* Go to src/main/containers directory and run +**Services Started:** +- ๐๏ธ **MySQL Database** +- ๐ **Keycloak** (Authentication) +- ๐จ **Apache Kafka** (Messaging) +- ๐ฐ **RabbitMQ** (Message Queue) +- ๐ **SSHD Server** (Secure connections) - ``` - docker-compose up - ``` +### 5๏ธโฃ Initialize Database + +Apply database migrations: + +```bash +cd modules/ide-integration/src/main/containers +cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 +``` -* Apply any database migrations. Go to src/main/containers directory and run +โณ **Wait for all services to be ready** (usually 2-3 minutes) - ``` - cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 - ``` +## ๐ฅ๏ธ Starting Airavata Components -* Wait until all the services come up. This will initialize all utilities required to start Airavata server +### 6๏ธโฃ Start API Server -### Starting API Server +1. **Navigate to:** `org.apache.airavata.ide.integration.APIServerStarter` +2. **Right-click** in the editor +3. **Select:** `Run 'APIServerStarter.main()'` -#### Note: For JDK 11+ -you have to add ``--add-opens java.base/java.lang=ALL-UNNAMED`` as a JVM argument +> ๐ก **JDK 17+ Note:** Add this JVM argument in your run configuration: +> ``` +> --add-opens java.base/java.lang=ALL-UNNAMED +> ``` -* Go to org.apache.airavata.ide.integration.APIServerStarter class and right click on the editor and click Run option. This will start Airavata server +### 7๏ธโฃ Start Job Execution Engine -### Starting Job Execution Engine +1. **Navigate to:** `org.apache.airavata.ide.integration.JobEngineStarter` +2. **Right-click** and select **Run** -* Go to org.apache.airavata.ide.integration.JobEngineStarter class and right click on the editor and click Run option. -This will start all components of Job Execution Engine including Helix Controller, Helix Participant, Pre Workflow Manager and -Post Workflow Manager +**Components Started:** +- ๐ Helix Controller +- ๐ฅ Helix Participant +- โ๏ธ Pre Workflow Manager +- ๐ Post Workflow Manager -### Starting Job Monitoring components +### 8๏ธโฃ Start Job Monitoring -* This will start the Email Based Job Monitoring agent. Before starting this, you have to create a new gmail account by going to -https://accounts.google.com/signup +**Setup Email Monitor (One-time setup):** -* Once the account is created, turn on 2-Step Verification and create an App Password (Use the type "Other" from the App type selection and give the name as "Airavata") -https://myaccount.google.com/security +1. **Create Gmail Account:** [accounts.google.com/signup](https://accounts.google.com/signup) +2. **Enable 2-Step Verification:** [myaccount.google.com/security](https://myaccount.google.com/security) +3. **Generate App Password:** Review Comment: mention that the user has to navigate to `https://myaccount.google.com/apppasswords` to create an app pw ########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop -### Setting up the development environment +# Build the project (this may take a few minutes) +mvn clean install -DskipTests +``` + +### 2๏ธโฃ Open in IntelliJ IDEA -* Clone Airavata repository to a local directory +1. **Launch IntelliJ IDEA** +2. **Open Project** โ Navigate to your cloned `airavata` directory +3. **Navigate to:** `modules` โ `ide-integration` module +4. **Wait for indexing** to complete - ``` - git clone https://github.com/apache/airavata - ``` +## ๐ณ Backend Services Setup -* Checkout develop branch +### 3๏ธโฃ Configure Host Resolution - ``` - git checkout develop - ``` -* Build the develop branch using Maven +Add the following entry to your system's hosts file: - ``` - mvn clean install -DskipTests - ``` -* Open the project using InteliJ IDEA +**Linux/macOS:** `/etc/hosts` +**Windows:** `C:\Windows\System32\drivers\etc\hosts` -* Browse to modules -> ide-integration module +```bash +127.0.0.1 airavata.host +``` -### Starting backend components (Database, Keycloak, Kafka, RabbitMQ, SSHD Server) +### 4๏ธโฃ Start Backend Services -* Add a host entry to /etc/hosts file in local machine +Navigate to the containers directory and start all required services: - ``` - 127.0.0.1 airavata.host - ``` +```bash +cd modules/ide-integration/src/main/containers +docker-compose up -d +``` -* Go to src/main/containers directory and run +**Services Started:** +- ๐๏ธ **MySQL Database** +- ๐ **Keycloak** (Authentication) +- ๐จ **Apache Kafka** (Messaging) +- ๐ฐ **RabbitMQ** (Message Queue) +- ๐ **SSHD Server** (Secure connections) - ``` - docker-compose up - ``` +### 5๏ธโฃ Initialize Database + +Apply database migrations: + +```bash +cd modules/ide-integration/src/main/containers +cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 +``` -* Apply any database migrations. Go to src/main/containers directory and run +โณ **Wait for all services to be ready** (usually 2-3 minutes) - ``` - cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 - ``` +## ๐ฅ๏ธ Starting Airavata Components -* Wait until all the services come up. This will initialize all utilities required to start Airavata server +### 6๏ธโฃ Start API Server -### Starting API Server +1. **Navigate to:** `org.apache.airavata.ide.integration.APIServerStarter` +2. **Right-click** in the editor +3. **Select:** `Run 'APIServerStarter.main()'` -#### Note: For JDK 11+ -you have to add ``--add-opens java.base/java.lang=ALL-UNNAMED`` as a JVM argument +> ๐ก **JDK 17+ Note:** Add this JVM argument in your run configuration: +> ``` +> --add-opens java.base/java.lang=ALL-UNNAMED +> ``` -* Go to org.apache.airavata.ide.integration.APIServerStarter class and right click on the editor and click Run option. This will start Airavata server +### 7๏ธโฃ Start Job Execution Engine -### Starting Job Execution Engine +1. **Navigate to:** `org.apache.airavata.ide.integration.JobEngineStarter` +2. **Right-click** and select **Run** -* Go to org.apache.airavata.ide.integration.JobEngineStarter class and right click on the editor and click Run option. -This will start all components of Job Execution Engine including Helix Controller, Helix Participant, Pre Workflow Manager and -Post Workflow Manager +**Components Started:** +- ๐ Helix Controller +- ๐ฅ Helix Participant +- โ๏ธ Pre Workflow Manager +- ๐ Post Workflow Manager -### Starting Job Monitoring components +### 8๏ธโฃ Start Job Monitoring -* This will start the Email Based Job Monitoring agent. Before starting this, you have to create a new gmail account by going to -https://accounts.google.com/signup +**Setup Email Monitor (One-time setup):** -* Once the account is created, turn on 2-Step Verification and create an App Password (Use the type "Other" from the App type selection and give the name as "Airavata") -https://myaccount.google.com/security +1. **Create Gmail Account:** [accounts.google.com/signup](https://accounts.google.com/signup) +2. **Enable 2-Step Verification:** [myaccount.google.com/security](https://myaccount.google.com/security) +3. **Generate App Password:** + - Type: "Other" + - Name: "Airavata" -* Update the email address and App Password in src/main/resources/airavata-server.properties file +4. **Update Configuration:** + Edit `src/main/resources/airavata-server.properties`: + ```properties + email.based.monitor.address=your-em...@gmail.com + email.based.monitor.password=your-app-password + ``` - email.based.monitor.address=CHANGEME - email.based.monitor.password=CHANGEME - -* Go to org.apache.airavata.ide.integration.JobMonitorStarter class and right click on the editor and click Run option. +5. **Start Monitor:** + - Navigate to: `org.apache.airavata.ide.integration.JobMonitorStarter` + - Right-click and **Run** -### Starting User Portal (Django Portal) +## ๐ User Portal Setup (Django) -* You can create and launch experiments and manage credentials using this portal Review Comment: I believe we should keep this line so the user knows what this Django portal is supposed to do ########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop -### Setting up the development environment +# Build the project (this may take a few minutes) +mvn clean install -DskipTests +``` + +### 2๏ธโฃ Open in IntelliJ IDEA -* Clone Airavata repository to a local directory +1. **Launch IntelliJ IDEA** +2. **Open Project** โ Navigate to your cloned `airavata` directory +3. **Navigate to:** `modules` โ `ide-integration` module +4. **Wait for indexing** to complete - ``` - git clone https://github.com/apache/airavata - ``` +## ๐ณ Backend Services Setup -* Checkout develop branch +### 3๏ธโฃ Configure Host Resolution - ``` - git checkout develop - ``` -* Build the develop branch using Maven +Add the following entry to your system's hosts file: - ``` - mvn clean install -DskipTests - ``` -* Open the project using InteliJ IDEA +**Linux/macOS:** `/etc/hosts` +**Windows:** `C:\Windows\System32\drivers\etc\hosts` -* Browse to modules -> ide-integration module +```bash +127.0.0.1 airavata.host +``` -### Starting backend components (Database, Keycloak, Kafka, RabbitMQ, SSHD Server) +### 4๏ธโฃ Start Backend Services -* Add a host entry to /etc/hosts file in local machine +Navigate to the containers directory and start all required services: - ``` - 127.0.0.1 airavata.host - ``` +```bash +cd modules/ide-integration/src/main/containers +docker-compose up -d +``` -* Go to src/main/containers directory and run +**Services Started:** +- ๐๏ธ **MySQL Database** +- ๐ **Keycloak** (Authentication) +- ๐จ **Apache Kafka** (Messaging) +- ๐ฐ **RabbitMQ** (Message Queue) +- ๐ **SSHD Server** (Secure connections) - ``` - docker-compose up - ``` +### 5๏ธโฃ Initialize Database + +Apply database migrations: + +```bash +cd modules/ide-integration/src/main/containers +cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 +``` -* Apply any database migrations. Go to src/main/containers directory and run +โณ **Wait for all services to be ready** (usually 2-3 minutes) - ``` - cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 - ``` +## ๐ฅ๏ธ Starting Airavata Components -* Wait until all the services come up. This will initialize all utilities required to start Airavata server +### 6๏ธโฃ Start API Server -### Starting API Server +1. **Navigate to:** `org.apache.airavata.ide.integration.APIServerStarter` +2. **Right-click** in the editor +3. **Select:** `Run 'APIServerStarter.main()'` -#### Note: For JDK 11+ -you have to add ``--add-opens java.base/java.lang=ALL-UNNAMED`` as a JVM argument +> ๐ก **JDK 17+ Note:** Add this JVM argument in your run configuration: +> ``` +> --add-opens java.base/java.lang=ALL-UNNAMED +> ``` -* Go to org.apache.airavata.ide.integration.APIServerStarter class and right click on the editor and click Run option. This will start Airavata server +### 7๏ธโฃ Start Job Execution Engine -### Starting Job Execution Engine +1. **Navigate to:** `org.apache.airavata.ide.integration.JobEngineStarter` +2. **Right-click** and select **Run** -* Go to org.apache.airavata.ide.integration.JobEngineStarter class and right click on the editor and click Run option. -This will start all components of Job Execution Engine including Helix Controller, Helix Participant, Pre Workflow Manager and -Post Workflow Manager +**Components Started:** +- ๐ Helix Controller +- ๐ฅ Helix Participant +- โ๏ธ Pre Workflow Manager +- ๐ Post Workflow Manager -### Starting Job Monitoring components +### 8๏ธโฃ Start Job Monitoring -* This will start the Email Based Job Monitoring agent. Before starting this, you have to create a new gmail account by going to -https://accounts.google.com/signup +**Setup Email Monitor (One-time setup):** -* Once the account is created, turn on 2-Step Verification and create an App Password (Use the type "Other" from the App type selection and give the name as "Airavata") -https://myaccount.google.com/security +1. **Create Gmail Account:** [accounts.google.com/signup](https://accounts.google.com/signup) +2. **Enable 2-Step Verification:** [myaccount.google.com/security](https://myaccount.google.com/security) +3. **Generate App Password:** + - Type: "Other" + - Name: "Airavata" -* Update the email address and App Password in src/main/resources/airavata-server.properties file +4. **Update Configuration:** + Edit `src/main/resources/airavata-server.properties`: + ```properties + email.based.monitor.address=your-em...@gmail.com + email.based.monitor.password=your-app-password + ``` - email.based.monitor.address=CHANGEME - email.based.monitor.password=CHANGEME - -* Go to org.apache.airavata.ide.integration.JobMonitorStarter class and right click on the editor and click Run option. +5. **Start Monitor:** + - Navigate to: `org.apache.airavata.ide.integration.JobMonitorStarter` + - Right-click and **Run** -### Starting User Portal (Django Portal) +## ๐ User Portal Setup (Django) -* You can create and launch experiments and manage credentials using this portal +### 9๏ธโฃ Django Portal Installation -* This is a separate project so you need to clone this in to a new directory outside the Airavata code base - - ``` - git clone https://github.com/apache/airavata-django-portal - ``` - -* Go to airavata-django-portal directory and run +```bash +# Clone portal repository (outside airavata directory) +cd .. +git clone https://github.com/apache/airavata-django-portal.git +cd airavata-django-portal - ``` - python3 -m venv venv - source venv/bin/activate - pip install -r requirements.txt - ``` -* Create a local settings file. Copy - `django_airavata/settings_local.py.ide` to - `django_airavata/settings_local.py` +# Create virtual environment +python3 -m venv venv +source venv/bin/activate # On Windows: venv\Scripts\activate -* Run Django migrations +# Install dependencies +pip install -r requirements.txt +``` - ``` - python3 manage.py migrate - ``` +### ๐ Configure Django Portal -* Build the JavaScript sources. There are a few JavaScript packages in the source tree, colocated with the Django apps in which they are used. The `build_js.sh` script will build them all. +```bash +# Create local settings +cp django_airavata/settings_local.py.ide django_airavata/settings_local.py - ``` - ./build_js.sh - ``` +# Run database migrations +python3 manage.py migrate -* Load the default Wagtail CMS pages. +# Build JavaScript components +./build_js.sh - ``` - python3 manage.py load_default_gateway - ``` +# Load default CMS pages +python3 manage.py load_default_gateway -* Run the server +# Start development server +python3 manage.py runserver +``` - ``` - python3 manage.py runserver - ``` - -* Point your browser to http://localhost:8000/auth/login. Use user name : default-admin and password : 123456 +### ๐ Access User Portal -### Optional: Starting Super Admin Portal (PGA) +- **URL:** [http://localhost:8000/auth/login](http://localhost:8000/auth/login) +- **Username:** `default-admin` +- **Password:** `123456` -* This portal is required when you are going to register new compute resources or storage resources into the gateway Review Comment: better to keep this too ########## README.md: ########## @@ -1,108 +1,186 @@ # Apache Airavata -[](https://travis-ci.org/apache/airavata) -[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) +  -## About + + [](https://travis-ci.org/apache/airavata) + [](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) + [](https://opensource.org/licenses/Apache-2.0) + [](https://github.com/apache/airavata/graphs/contributors) +</div> -Apache Airavata is a software framework for executing and managing computational -jobs on distributed computing resources including local clusters, -supercomputers, national grids, academic and commercial clouds. Airavata builds -on general concepts of service oriented computing, distributed messaging, and -workflow composition and orchestration. Airavata bundles a server package with -an API, client software development Kits and a general purpose reference UI -implementation - -[Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal). +## ๐ About -Learn more about Airavata at -[https://airavata.apache.org](https://airavata.apache.org). +Apache Airavata is a modern, scalable software framework designed for executing and managing computational jobs across distributed computing resources. Whether you're working with local clusters, supercomputers, national grids, or cloud platforms, Airavata provides a unified interface for computational workflow management. -## Building Apache Airavata +**Key Features:** +- ๐ง Service-oriented architecture with distributed messaging +- ๐ Workflow composition and orchestration +- โ๏ธ Multi-cloud and hybrid cloud support +- ๐ฅ๏ธ Comprehensive API and SDK ecosystem +- ๐ Reference UI implementation via [Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal) -### Prerequisites +> Learn more at [airavata.apache.org](https://airavata.apache.org) -- Sources compilation requires Java SDK 11. -- The project is built with Apache Maven 3+. -- Set or export JAVA_HOME to point to JDK. For example in Ubuntu: -`export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-11` -- Git +## ๐ Prerequisites -### Build the distribution +Before building Apache Airavata, ensure you have: - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install +| Requirement | Version | Notes | +|-------------|---------|-------| +| **Java SDK** | 17+ | Set `JAVA_HOME` environment variable | +| **Apache Maven** | 3.8+ | Build automation tool | +| **Git** | Latest | Version control | -To build without running tests, use `mvn clean install -Dmaven.test.skip=true`. -The compressed binary distribution is created at -PROJECT_DIR/modules/distribution/target. +### ๐ง Environment Setup -### Build and run docker distribution (Experimental and not recommended for production deployments) - -* This requires docker and docker-compose installed in your system +**Ubuntu/Debian:** +```bash +export JAVA_HOME=/usr/lib/jvm/openjdk-17-jdk +``` -* Build the source and docker images for each microservice +**macOS:** +```bash +export JAVA_HOME=$(/usr/libexec/java_home -v 17) ``` - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install - mvn docker:build -pl modules/distribution + +**Windows:** +```powershell +set JAVA_HOME=C:\Program Files\OpenJDK\openjdk-17 ``` -* Start supporting services and Airavata miroservices (API Server, Helix Components and the Job Monitors) + +## ๐๏ธ Building from Source + +### Quick Start + +```bash +# Clone the repository +git clone https://github.com/apache/airavata.git +cd airavata + +# Build with tests +mvn clean install + +# Build without tests (faster) +mvn clean install -DskipTests + +# Find your distribution +ls -la modules/distribution/target/ ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml up + +### ๐ณ Docker Development (Experimental) + +> โ ๏ธ **Note:** Docker deployment is experimental and not recommended for production use. + +**Prerequisites:** +- Docker Engine 20.10+ +- Docker Compose 2.0+ + +**Build and Deploy:** + +```bash +# 1. Build source and Docker images +git clone https://github.com/apache/airavata.git +cd airavata +mvn clean install +mvn docker:build -pl modules/distribution + +# 2. Start all services Review Comment: mention the services that will be started `Start supporting services and Airavata miroservices (API Server, Helix Components and the Job Monitors)` ########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop -### Setting up the development environment +# Build the project (this may take a few minutes) +mvn clean install -DskipTests +``` + +### 2๏ธโฃ Open in IntelliJ IDEA -* Clone Airavata repository to a local directory +1. **Launch IntelliJ IDEA** +2. **Open Project** โ Navigate to your cloned `airavata` directory +3. **Navigate to:** `modules` โ `ide-integration` module +4. **Wait for indexing** to complete - ``` - git clone https://github.com/apache/airavata - ``` +## ๐ณ Backend Services Setup -* Checkout develop branch +### 3๏ธโฃ Configure Host Resolution - ``` - git checkout develop - ``` -* Build the develop branch using Maven +Add the following entry to your system's hosts file: - ``` - mvn clean install -DskipTests - ``` -* Open the project using InteliJ IDEA +**Linux/macOS:** `/etc/hosts` +**Windows:** `C:\Windows\System32\drivers\etc\hosts` -* Browse to modules -> ide-integration module +```bash +127.0.0.1 airavata.host +``` -### Starting backend components (Database, Keycloak, Kafka, RabbitMQ, SSHD Server) +### 4๏ธโฃ Start Backend Services -* Add a host entry to /etc/hosts file in local machine +Navigate to the containers directory and start all required services: - ``` - 127.0.0.1 airavata.host - ``` +```bash +cd modules/ide-integration/src/main/containers +docker-compose up -d +``` -* Go to src/main/containers directory and run +**Services Started:** +- ๐๏ธ **MySQL Database** +- ๐ **Keycloak** (Authentication) +- ๐จ **Apache Kafka** (Messaging) +- ๐ฐ **RabbitMQ** (Message Queue) +- ๐ **SSHD Server** (Secure connections) - ``` - docker-compose up - ``` +### 5๏ธโฃ Initialize Database + +Apply database migrations: + +```bash +cd modules/ide-integration/src/main/containers +cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 +``` -* Apply any database migrations. Go to src/main/containers directory and run +โณ **Wait for all services to be ready** (usually 2-3 minutes) - ``` - cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 - ``` +## ๐ฅ๏ธ Starting Airavata Components -* Wait until all the services come up. This will initialize all utilities required to start Airavata server +### 6๏ธโฃ Start API Server -### Starting API Server +1. **Navigate to:** `org.apache.airavata.ide.integration.APIServerStarter` +2. **Right-click** in the editor +3. **Select:** `Run 'APIServerStarter.main()'` -#### Note: For JDK 11+ -you have to add ``--add-opens java.base/java.lang=ALL-UNNAMED`` as a JVM argument +> ๐ก **JDK 17+ Note:** Add this JVM argument in your run configuration: +> ``` +> --add-opens java.base/java.lang=ALL-UNNAMED +> ``` -* Go to org.apache.airavata.ide.integration.APIServerStarter class and right click on the editor and click Run option. This will start Airavata server +### 7๏ธโฃ Start Job Execution Engine -### Starting Job Execution Engine +1. **Navigate to:** `org.apache.airavata.ide.integration.JobEngineStarter` +2. **Right-click** and select **Run** -* Go to org.apache.airavata.ide.integration.JobEngineStarter class and right click on the editor and click Run option. -This will start all components of Job Execution Engine including Helix Controller, Helix Participant, Pre Workflow Manager and -Post Workflow Manager +**Components Started:** +- ๐ Helix Controller +- ๐ฅ Helix Participant +- โ๏ธ Pre Workflow Manager +- ๐ Post Workflow Manager -### Starting Job Monitoring components +### 8๏ธโฃ Start Job Monitoring -* This will start the Email Based Job Monitoring agent. Before starting this, you have to create a new gmail account by going to -https://accounts.google.com/signup +**Setup Email Monitor (One-time setup):** -* Once the account is created, turn on 2-Step Verification and create an App Password (Use the type "Other" from the App type selection and give the name as "Airavata") -https://myaccount.google.com/security +1. **Create Gmail Account:** [accounts.google.com/signup](https://accounts.google.com/signup) +2. **Enable 2-Step Verification:** [myaccount.google.com/security](https://myaccount.google.com/security) +3. **Generate App Password:** + - Type: "Other" + - Name: "Airavata" -* Update the email address and App Password in src/main/resources/airavata-server.properties file +4. **Update Configuration:** + Edit `src/main/resources/airavata-server.properties`: + ```properties + email.based.monitor.address=your-em...@gmail.com + email.based.monitor.password=your-app-password + ``` - email.based.monitor.address=CHANGEME - email.based.monitor.password=CHANGEME - -* Go to org.apache.airavata.ide.integration.JobMonitorStarter class and right click on the editor and click Run option. +5. **Start Monitor:** + - Navigate to: `org.apache.airavata.ide.integration.JobMonitorStarter` + - Right-click and **Run** -### Starting User Portal (Django Portal) +## ๐ User Portal Setup (Django) -* You can create and launch experiments and manage credentials using this portal +### 9๏ธโฃ Django Portal Installation -* This is a separate project so you need to clone this in to a new directory outside the Airavata code base - - ``` - git clone https://github.com/apache/airavata-django-portal - ``` - -* Go to airavata-django-portal directory and run +```bash +# Clone portal repository (outside airavata directory) +cd .. +git clone https://github.com/apache/airavata-django-portal.git +cd airavata-django-portal - ``` - python3 -m venv venv - source venv/bin/activate - pip install -r requirements.txt - ``` -* Create a local settings file. Copy - `django_airavata/settings_local.py.ide` to - `django_airavata/settings_local.py` +# Create virtual environment +python3 -m venv venv +source venv/bin/activate # On Windows: venv\Scripts\activate -* Run Django migrations +# Install dependencies +pip install -r requirements.txt +``` - ``` - python3 manage.py migrate - ``` +### ๐ Configure Django Portal -* Build the JavaScript sources. There are a few JavaScript packages in the source tree, colocated with the Django apps in which they are used. The `build_js.sh` script will build them all. +```bash +# Create local settings +cp django_airavata/settings_local.py.ide django_airavata/settings_local.py - ``` - ./build_js.sh - ``` +# Run database migrations +python3 manage.py migrate -* Load the default Wagtail CMS pages. +# Build JavaScript components +./build_js.sh - ``` - python3 manage.py load_default_gateway - ``` +# Load default CMS pages +python3 manage.py load_default_gateway -* Run the server +# Start development server +python3 manage.py runserver +``` - ``` - python3 manage.py runserver - ``` - -* Point your browser to http://localhost:8000/auth/login. Use user name : default-admin and password : 123456 +### ๐ Access User Portal -### Optional: Starting Super Admin Portal (PGA) +- **URL:** [http://localhost:8000/auth/login](http://localhost:8000/auth/login) +- **Username:** `default-admin` +- **Password:** `123456` -* This portal is required when you are going to register new compute resources or storage resources into the gateway +## ๐ ๏ธ Admin Portal Setup (Optional) -* Go to src/main/containers/pga directory and run +For registering compute resources and storage resources: - ``` - docker-compose up -d - ``` +### 1๏ธโฃ Start PGA Portal -* Run following command to get the ip address of host machine +```bash +cd modules/ide-integration/src/main/containers/pga +docker-compose up -d +``` - For Mac OSX +### 2๏ธโฃ Configure Host Resolution - ``` - docker-compose exec pga getent hosts docker.for.mac.host.internal | awk '{ print $1 }' - ``` - - For Windows - - ``` - docker-compose exec pga getent hosts host.docker.internal - ``` +**Get host machine IP:** -* Update the host entries of pga container with above ip address +**macOS:** +```bash +docker-compose exec pga getent hosts docker.for.mac.host.internal | awk '{ print $1 }' +``` - ``` - docker-compose exec pga /bin/sh -c "echo '<host-machine ip> airavata.host' >> /etc/hosts" - ``` +**Windows:** +```bash +docker-compose exec pga getent hosts host.docker.internal +``` -* Now PGA should be accessible through http://airavata.host:8008 +**Update container hosts:** Review Comment: mention to update with the above ip ########## README.md: ########## @@ -1,108 +1,186 @@ # Apache Airavata -[](https://travis-ci.org/apache/airavata) -[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) +  -## About + + [](https://travis-ci.org/apache/airavata) + [](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) + [](https://opensource.org/licenses/Apache-2.0) + [](https://github.com/apache/airavata/graphs/contributors) +</div> -Apache Airavata is a software framework for executing and managing computational -jobs on distributed computing resources including local clusters, -supercomputers, national grids, academic and commercial clouds. Airavata builds -on general concepts of service oriented computing, distributed messaging, and -workflow composition and orchestration. Airavata bundles a server package with -an API, client software development Kits and a general purpose reference UI -implementation - -[Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal). +## ๐ About -Learn more about Airavata at -[https://airavata.apache.org](https://airavata.apache.org). +Apache Airavata is a modern, scalable software framework designed for executing and managing computational jobs across distributed computing resources. Whether you're working with local clusters, supercomputers, national grids, or cloud platforms, Airavata provides a unified interface for computational workflow management. -## Building Apache Airavata +**Key Features:** +- ๐ง Service-oriented architecture with distributed messaging +- ๐ Workflow composition and orchestration +- โ๏ธ Multi-cloud and hybrid cloud support +- ๐ฅ๏ธ Comprehensive API and SDK ecosystem +- ๐ Reference UI implementation via [Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal) -### Prerequisites +> Learn more at [airavata.apache.org](https://airavata.apache.org) -- Sources compilation requires Java SDK 11. -- The project is built with Apache Maven 3+. -- Set or export JAVA_HOME to point to JDK. For example in Ubuntu: -`export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-11` -- Git +## ๐ Prerequisites -### Build the distribution +Before building Apache Airavata, ensure you have: - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install +| Requirement | Version | Notes | +|-------------|---------|-------| +| **Java SDK** | 17+ | Set `JAVA_HOME` environment variable | +| **Apache Maven** | 3.8+ | Build automation tool | +| **Git** | Latest | Version control | -To build without running tests, use `mvn clean install -Dmaven.test.skip=true`. -The compressed binary distribution is created at -PROJECT_DIR/modules/distribution/target. +### ๐ง Environment Setup -### Build and run docker distribution (Experimental and not recommended for production deployments) - -* This requires docker and docker-compose installed in your system +**Ubuntu/Debian:** +```bash +export JAVA_HOME=/usr/lib/jvm/openjdk-17-jdk +``` -* Build the source and docker images for each microservice +**macOS:** +```bash +export JAVA_HOME=$(/usr/libexec/java_home -v 17) ``` - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install - mvn docker:build -pl modules/distribution + +**Windows:** +```powershell +set JAVA_HOME=C:\Program Files\OpenJDK\openjdk-17 ``` -* Start supporting services and Airavata miroservices (API Server, Helix Components and the Job Monitors) + +## ๐๏ธ Building from Source + +### Quick Start + +```bash +# Clone the repository +git clone https://github.com/apache/airavata.git +cd airavata + +# Build with tests +mvn clean install + +# Build without tests (faster) +mvn clean install -DskipTests + +# Find your distribution +ls -la modules/distribution/target/ ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml up + +### ๐ณ Docker Development (Experimental) + +> โ ๏ธ **Note:** Docker deployment is experimental and not recommended for production use. + +**Prerequisites:** +- Docker Engine 20.10+ +- Docker Compose 2.0+ + +**Build and Deploy:** + +```bash +# 1. Build source and Docker images +git clone https://github.com/apache/airavata.git +cd airavata +mvn clean install +mvn docker:build -pl modules/distribution + +# 2. Start all services +docker-compose \ + -f modules/ide-integration/src/main/containers/docker-compose.yml \ + -f modules/distribution/src/main/docker/docker-compose.yml \ + up -d + +# 3. Verify services are running +docker-compose ps ``` -* Django portal and PGA Portal can be pointed to airavata.host:8930 (API) , airavata.host:8962 (Profile Service), airavata.host:8443 (Keycloak). -Make sure that you add a host entry that maps airavata.host -> 127.0.0.1 +**Service Endpoints:** +- **API Server:** `airavata.host:8930` +- **Profile Service:** `airavata.host:8962` +- **Keycloak:** `airavata.host:8443` -* To stop all the services +**Host Configuration:** +Add to your `/etc/hosts` file: ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml down +127.0.0.1 airavata.host +``` + +**Stop Services:** +```bash +docker-compose \ + -f modules/ide-integration/src/main/containers/docker-compose.yml \ + -f modules/distribution/src/main/docker/docker-compose.yml \ + down ``` -* If you do any code change and need to reflect them in the deployment, stop the docker deployment, rebuild docker images and start the docker deployment - - -## Getting Started +## ๐ Getting Started + +The fastest way to set up a development environment is through our IDE integration: + +1. **Follow our [IDE Integration Guide](./modules/ide-integration/README.md)** +2. **IntelliJ IDEA setup instructions included** +3. **Automated development environment configuration** + +## ๐ Documentation & Resources -The easiest way to get started with running Airavata locally and setting up a -development environment is to follow the instructions in the -[ide-integration README](./modules/ide-integration/README.md). Those -instructions will guide you on setting up a development environment with -IntelliJ IDEA. +| Resource | Description | Link | +|----------|-------------|------| +| ๐ **Documentation** | Complete user and developer guides | [docs.airavata.org](https://docs.airavata.org/en/master/) | +| ๐ง **Developer Wiki** | Technical specifications and architecture | [Apache Confluence](https://cwiki.apache.org/confluence/display/AIRAVATA) | +| ๐ **Issue Tracker** | Bug reports and feature requests | [JIRA](https://issues.apache.org/jira/projects/AIRAVATA) | +| ๐ฌ **Mailing Lists** | Community discussions | [airavata.apache.org/mailing-list](https://airavata.apache.org/mailing-list.html) | -## Contact +## ๐ค Contributing -For additional information about Apache Airavata, please contact the user or dev -mailing lists: https://airavata.apache.org/mailing-list.html +We welcome contributions from the community! Here's how you can help: -## Contributing +1. **๐ด Fork the repository** +2. **๐ฟ Create a feature branch** +3. **โจ Make your changes** +4. **๐งช Add tests if applicable** +5. **๐ Submit a pull request** -Want to help contribute to the development of Apache Airavata? Check out our -[contributing documentation](http://airavata.apache.org/get-involved.html). +**Learn More:** +- [Contributing Guidelines](http://airavata.apache.org/get-involved.html) +- [Code of Conduct](https://www.apache.org/foundation/policies/conduct.html) +- [Developer Resources](https://cwiki.apache.org/confluence/display/AIRAVATA) -## Links +## ๐ฌ Community & Support -- [Documentation](https://docs.airavata.org/en/master/) -- Developer [wiki](https://cwiki.apache.org/confluence/display/AIRAVATA) -- [Issue Tracker](https://issues.apache.org/jira/projects/AIRAVATA) +**Get Help:** +- ๐ง **User Mailing List:** [us...@airavata.apache.org](mailto:us...@airavata.apache.org) +- ๐จโ๐ป **Developer Mailing List:** [d...@airavata.apache.org](mailto:d...@airavata.apache.org) +- ๐ **All Mailing Lists:** [airavata.apache.org/mailing-list](https://airavata.apache.org/mailing-list.html) -## License +**Stay Connected:** +- ๐ Star this repository +- ๐ Watch for updates +- ๐ฆ Follow [@ApacheAiravata](https://twitter.com/ApacheAiravata) Review Comment: I prefer removing this ########## modules/ide-integration/README.md: ########## @@ -1,228 +1,292 @@ -## Introduction +# Apache Airavata - IDE Integration Setup -Using this module, you can setup a full Airavata installation inside Intelij IDEA for development purposes +<div align="center"> + <h3>๐ Complete Development Environment Setup for IntelliJ IDEA</h3> + <p>Set up a full Airavata installation inside IntelliJ IDEA for seamless development</p> +</div> -## Prerequisites +--- -* Docker installed with 'docker-compose' utility - https://docs.docker.com/compose/ +## ๐ Prerequisites -* InteliJ IDEA with Java 8 installed - https://www.jetbrains.com/idea/download/#section=mac +Before starting, ensure you have the following installed on your system: -* Maven +| Tool | Version | Purpose | Installation Link | +|------|---------|---------|-------------------| +| ๐ณ **Docker & Docker Compose** | Latest | Container orchestration | [Get Docker](https://docs.docker.com/compose/) | +| ๐ก **IntelliJ IDEA** | Latest | IDE with Java 17+ | [Download IDEA](https://www.jetbrains.com/idea/download/) | +| โ **Java JDK** | 17+ | Runtime environment | [OpenJDK 17](https://openjdk.org/projects/jdk/17/) | +| ๐ง **Apache Maven** | 3.8+ | Build tool | [Install Maven](https://maven.apache.org/install.html) | +| ๐ **Git** | Latest | Version control | [Install Git](https://git-scm.com/downloads) | +| ๐ **Python** | 3.8+ | Django portal | [Python.org](https://www.python.org/downloads/) | +| ๐ฆ **Node.js & npm** | Latest LTS | Frontend build tools | [Node.js](https://nodejs.org/) | -* Git +## ๐๏ธ Development Environment Setup -* python3 +### 1๏ธโฃ Clone and Prepare Repository -* npm (install or update to latest version) - https://www.npmjs.com/get-npm +```bash +# Clone the main repository +git clone https://github.com/apache/airavata.git +cd airavata -## Steps +# Switch to development branch +git checkout develop -### Setting up the development environment +# Build the project (this may take a few minutes) +mvn clean install -DskipTests +``` + +### 2๏ธโฃ Open in IntelliJ IDEA -* Clone Airavata repository to a local directory +1. **Launch IntelliJ IDEA** +2. **Open Project** โ Navigate to your cloned `airavata` directory +3. **Navigate to:** `modules` โ `ide-integration` module +4. **Wait for indexing** to complete - ``` - git clone https://github.com/apache/airavata - ``` +## ๐ณ Backend Services Setup -* Checkout develop branch +### 3๏ธโฃ Configure Host Resolution - ``` - git checkout develop - ``` -* Build the develop branch using Maven +Add the following entry to your system's hosts file: - ``` - mvn clean install -DskipTests - ``` -* Open the project using InteliJ IDEA +**Linux/macOS:** `/etc/hosts` +**Windows:** `C:\Windows\System32\drivers\etc\hosts` -* Browse to modules -> ide-integration module +```bash +127.0.0.1 airavata.host +``` -### Starting backend components (Database, Keycloak, Kafka, RabbitMQ, SSHD Server) +### 4๏ธโฃ Start Backend Services -* Add a host entry to /etc/hosts file in local machine +Navigate to the containers directory and start all required services: - ``` - 127.0.0.1 airavata.host - ``` +```bash +cd modules/ide-integration/src/main/containers +docker-compose up -d +``` -* Go to src/main/containers directory and run +**Services Started:** +- ๐๏ธ **MySQL Database** +- ๐ **Keycloak** (Authentication) +- ๐จ **Apache Kafka** (Messaging) +- ๐ฐ **RabbitMQ** (Message Queue) +- ๐ **SSHD Server** (Secure connections) - ``` - docker-compose up - ``` +### 5๏ธโฃ Initialize Database + +Apply database migrations: + +```bash +cd modules/ide-integration/src/main/containers +cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 +``` -* Apply any database migrations. Go to src/main/containers directory and run +โณ **Wait for all services to be ready** (usually 2-3 minutes) - ``` - cat ./database_scripts/init/*-migrations.sql | docker exec -i containers-db-1 mysql -p123456 - ``` +## ๐ฅ๏ธ Starting Airavata Components -* Wait until all the services come up. This will initialize all utilities required to start Airavata server +### 6๏ธโฃ Start API Server -### Starting API Server +1. **Navigate to:** `org.apache.airavata.ide.integration.APIServerStarter` +2. **Right-click** in the editor +3. **Select:** `Run 'APIServerStarter.main()'` -#### Note: For JDK 11+ -you have to add ``--add-opens java.base/java.lang=ALL-UNNAMED`` as a JVM argument +> ๐ก **JDK 17+ Note:** Add this JVM argument in your run configuration: +> ``` +> --add-opens java.base/java.lang=ALL-UNNAMED +> ``` -* Go to org.apache.airavata.ide.integration.APIServerStarter class and right click on the editor and click Run option. This will start Airavata server +### 7๏ธโฃ Start Job Execution Engine -### Starting Job Execution Engine +1. **Navigate to:** `org.apache.airavata.ide.integration.JobEngineStarter` +2. **Right-click** and select **Run** -* Go to org.apache.airavata.ide.integration.JobEngineStarter class and right click on the editor and click Run option. -This will start all components of Job Execution Engine including Helix Controller, Helix Participant, Pre Workflow Manager and -Post Workflow Manager +**Components Started:** +- ๐ Helix Controller +- ๐ฅ Helix Participant +- โ๏ธ Pre Workflow Manager +- ๐ Post Workflow Manager -### Starting Job Monitoring components +### 8๏ธโฃ Start Job Monitoring -* This will start the Email Based Job Monitoring agent. Before starting this, you have to create a new gmail account by going to -https://accounts.google.com/signup +**Setup Email Monitor (One-time setup):** -* Once the account is created, turn on 2-Step Verification and create an App Password (Use the type "Other" from the App type selection and give the name as "Airavata") -https://myaccount.google.com/security +1. **Create Gmail Account:** [accounts.google.com/signup](https://accounts.google.com/signup) +2. **Enable 2-Step Verification:** [myaccount.google.com/security](https://myaccount.google.com/security) +3. **Generate App Password:** Review Comment: It seems the instructions are outdated. Last I checked, it just asked for an 'app name' and then showed the pw ########## README.md: ########## @@ -1,108 +1,186 @@ # Apache Airavata -[](https://travis-ci.org/apache/airavata) -[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) +  -## About + + [](https://travis-ci.org/apache/airavata) + [](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) + [](https://opensource.org/licenses/Apache-2.0) + [](https://github.com/apache/airavata/graphs/contributors) +</div> -Apache Airavata is a software framework for executing and managing computational -jobs on distributed computing resources including local clusters, -supercomputers, national grids, academic and commercial clouds. Airavata builds -on general concepts of service oriented computing, distributed messaging, and -workflow composition and orchestration. Airavata bundles a server package with -an API, client software development Kits and a general purpose reference UI -implementation - -[Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal). +## ๐ About -Learn more about Airavata at -[https://airavata.apache.org](https://airavata.apache.org). +Apache Airavata is a modern, scalable software framework designed for executing and managing computational jobs across distributed computing resources. Whether you're working with local clusters, supercomputers, national grids, or cloud platforms, Airavata provides a unified interface for computational workflow management. -## Building Apache Airavata +**Key Features:** +- ๐ง Service-oriented architecture with distributed messaging +- ๐ Workflow composition and orchestration +- โ๏ธ Multi-cloud and hybrid cloud support +- ๐ฅ๏ธ Comprehensive API and SDK ecosystem +- ๐ Reference UI implementation via [Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal) -### Prerequisites +> Learn more at [airavata.apache.org](https://airavata.apache.org) -- Sources compilation requires Java SDK 11. -- The project is built with Apache Maven 3+. -- Set or export JAVA_HOME to point to JDK. For example in Ubuntu: -`export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-11` -- Git +## ๐ Prerequisites -### Build the distribution +Before building Apache Airavata, ensure you have: - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install +| Requirement | Version | Notes | +|-------------|---------|-------| +| **Java SDK** | 17+ | Set `JAVA_HOME` environment variable | +| **Apache Maven** | 3.8+ | Build automation tool | +| **Git** | Latest | Version control | -To build without running tests, use `mvn clean install -Dmaven.test.skip=true`. -The compressed binary distribution is created at -PROJECT_DIR/modules/distribution/target. +### ๐ง Environment Setup -### Build and run docker distribution (Experimental and not recommended for production deployments) - -* This requires docker and docker-compose installed in your system +**Ubuntu/Debian:** +```bash +export JAVA_HOME=/usr/lib/jvm/openjdk-17-jdk +``` -* Build the source and docker images for each microservice +**macOS:** +```bash +export JAVA_HOME=$(/usr/libexec/java_home -v 17) ``` - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install - mvn docker:build -pl modules/distribution + +**Windows:** +```powershell +set JAVA_HOME=C:\Program Files\OpenJDK\openjdk-17 ``` -* Start supporting services and Airavata miroservices (API Server, Helix Components and the Job Monitors) + +## ๐๏ธ Building from Source + +### Quick Start + +```bash +# Clone the repository +git clone https://github.com/apache/airavata.git Review Comment: Shall we go with the SSH-based version? (including line 83) `git clone g...@github.com:apache/airavata.git` ########## README.md: ########## @@ -1,108 +1,186 @@ # Apache Airavata -[](https://travis-ci.org/apache/airavata) -[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) +  -## About + + [](https://travis-ci.org/apache/airavata) + [](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) + [](https://opensource.org/licenses/Apache-2.0) + [](https://github.com/apache/airavata/graphs/contributors) +</div> -Apache Airavata is a software framework for executing and managing computational -jobs on distributed computing resources including local clusters, -supercomputers, national grids, academic and commercial clouds. Airavata builds -on general concepts of service oriented computing, distributed messaging, and -workflow composition and orchestration. Airavata bundles a server package with -an API, client software development Kits and a general purpose reference UI -implementation - -[Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal). Review Comment: I prefer having this about section without trimming it down ########## README.md: ########## @@ -1,108 +1,186 @@ # Apache Airavata -[](https://travis-ci.org/apache/airavata) -[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) +  -## About + + [](https://travis-ci.org/apache/airavata) + [](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) + [](https://opensource.org/licenses/Apache-2.0) + [](https://github.com/apache/airavata/graphs/contributors) +</div> -Apache Airavata is a software framework for executing and managing computational -jobs on distributed computing resources including local clusters, -supercomputers, national grids, academic and commercial clouds. Airavata builds -on general concepts of service oriented computing, distributed messaging, and -workflow composition and orchestration. Airavata bundles a server package with -an API, client software development Kits and a general purpose reference UI -implementation - -[Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal). +## ๐ About -Learn more about Airavata at -[https://airavata.apache.org](https://airavata.apache.org). +Apache Airavata is a modern, scalable software framework designed for executing and managing computational jobs across distributed computing resources. Whether you're working with local clusters, supercomputers, national grids, or cloud platforms, Airavata provides a unified interface for computational workflow management. -## Building Apache Airavata +**Key Features:** +- ๐ง Service-oriented architecture with distributed messaging +- ๐ Workflow composition and orchestration +- โ๏ธ Multi-cloud and hybrid cloud support +- ๐ฅ๏ธ Comprehensive API and SDK ecosystem +- ๐ Reference UI implementation via [Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal) -### Prerequisites +> Learn more at [airavata.apache.org](https://airavata.apache.org) -- Sources compilation requires Java SDK 11. -- The project is built with Apache Maven 3+. -- Set or export JAVA_HOME to point to JDK. For example in Ubuntu: -`export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-11` -- Git +## ๐ Prerequisites -### Build the distribution +Before building Apache Airavata, ensure you have: - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install +| Requirement | Version | Notes | +|-------------|---------|-------| +| **Java SDK** | 17+ | Set `JAVA_HOME` environment variable | +| **Apache Maven** | 3.8+ | Build automation tool | +| **Git** | Latest | Version control | -To build without running tests, use `mvn clean install -Dmaven.test.skip=true`. -The compressed binary distribution is created at -PROJECT_DIR/modules/distribution/target. +### ๐ง Environment Setup -### Build and run docker distribution (Experimental and not recommended for production deployments) - -* This requires docker and docker-compose installed in your system +**Ubuntu/Debian:** +```bash +export JAVA_HOME=/usr/lib/jvm/openjdk-17-jdk +``` -* Build the source and docker images for each microservice +**macOS:** +```bash +export JAVA_HOME=$(/usr/libexec/java_home -v 17) ``` - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install - mvn docker:build -pl modules/distribution + +**Windows:** +```powershell +set JAVA_HOME=C:\Program Files\OpenJDK\openjdk-17 ``` -* Start supporting services and Airavata miroservices (API Server, Helix Components and the Job Monitors) + +## ๐๏ธ Building from Source + +### Quick Start + +```bash +# Clone the repository +git clone https://github.com/apache/airavata.git +cd airavata + +# Build with tests +mvn clean install + +# Build without tests (faster) +mvn clean install -DskipTests + +# Find your distribution +ls -la modules/distribution/target/ ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml up + +### ๐ณ Docker Development (Experimental) + +> โ ๏ธ **Note:** Docker deployment is experimental and not recommended for production use. + +**Prerequisites:** +- Docker Engine 20.10+ +- Docker Compose 2.0+ + +**Build and Deploy:** + +```bash +# 1. Build source and Docker images +git clone https://github.com/apache/airavata.git +cd airavata +mvn clean install +mvn docker:build -pl modules/distribution + +# 2. Start all services +docker-compose \ + -f modules/ide-integration/src/main/containers/docker-compose.yml \ + -f modules/distribution/src/main/docker/docker-compose.yml \ + up -d + +# 3. Verify services are running +docker-compose ps ``` -* Django portal and PGA Portal can be pointed to airavata.host:8930 (API) , airavata.host:8962 (Profile Service), airavata.host:8443 (Keycloak). -Make sure that you add a host entry that maps airavata.host -> 127.0.0.1 +**Service Endpoints:** +- **API Server:** `airavata.host:8930` +- **Profile Service:** `airavata.host:8962` +- **Keycloak:** `airavata.host:8443` -* To stop all the services +**Host Configuration:** +Add to your `/etc/hosts` file: ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml down +127.0.0.1 airavata.host +``` + +**Stop Services:** +```bash +docker-compose \ + -f modules/ide-integration/src/main/containers/docker-compose.yml \ + -f modules/distribution/src/main/docker/docker-compose.yml \ + down ``` -* If you do any code change and need to reflect them in the deployment, stop the docker deployment, rebuild docker images and start the docker deployment - - -## Getting Started +## ๐ Getting Started + +The fastest way to set up a development environment is through our IDE integration: + +1. **Follow our [IDE Integration Guide](./modules/ide-integration/README.md)** +2. **IntelliJ IDEA setup instructions included** +3. **Automated development environment configuration** + +## ๐ Documentation & Resources -The easiest way to get started with running Airavata locally and setting up a -development environment is to follow the instructions in the -[ide-integration README](./modules/ide-integration/README.md). Those -instructions will guide you on setting up a development environment with -IntelliJ IDEA. +| Resource | Description | Link | +|----------|-------------|------| +| ๐ **Documentation** | Complete user and developer guides | [docs.airavata.org](https://docs.airavata.org/en/master/) | +| ๐ง **Developer Wiki** | Technical specifications and architecture | [Apache Confluence](https://cwiki.apache.org/confluence/display/AIRAVATA) | +| ๐ **Issue Tracker** | Bug reports and feature requests | [JIRA](https://issues.apache.org/jira/projects/AIRAVATA) | +| ๐ฌ **Mailing Lists** | Community discussions | [airavata.apache.org/mailing-list](https://airavata.apache.org/mailing-list.html) | Review Comment: This is mentioned in two places. Better to remove this one ########## README.md: ########## @@ -1,108 +1,186 @@ # Apache Airavata -[](https://travis-ci.org/apache/airavata) -[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) +  -## About + + [](https://travis-ci.org/apache/airavata) + [](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) + [](https://opensource.org/licenses/Apache-2.0) + [](https://github.com/apache/airavata/graphs/contributors) +</div> -Apache Airavata is a software framework for executing and managing computational -jobs on distributed computing resources including local clusters, -supercomputers, national grids, academic and commercial clouds. Airavata builds -on general concepts of service oriented computing, distributed messaging, and -workflow composition and orchestration. Airavata bundles a server package with -an API, client software development Kits and a general purpose reference UI -implementation - -[Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal). +## ๐ About -Learn more about Airavata at -[https://airavata.apache.org](https://airavata.apache.org). +Apache Airavata is a modern, scalable software framework designed for executing and managing computational jobs across distributed computing resources. Whether you're working with local clusters, supercomputers, national grids, or cloud platforms, Airavata provides a unified interface for computational workflow management. -## Building Apache Airavata +**Key Features:** +- ๐ง Service-oriented architecture with distributed messaging +- ๐ Workflow composition and orchestration +- โ๏ธ Multi-cloud and hybrid cloud support +- ๐ฅ๏ธ Comprehensive API and SDK ecosystem +- ๐ Reference UI implementation via [Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal) -### Prerequisites +> Learn more at [airavata.apache.org](https://airavata.apache.org) -- Sources compilation requires Java SDK 11. -- The project is built with Apache Maven 3+. -- Set or export JAVA_HOME to point to JDK. For example in Ubuntu: -`export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-11` -- Git +## ๐ Prerequisites -### Build the distribution +Before building Apache Airavata, ensure you have: - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install +| Requirement | Version | Notes | +|-------------|---------|-------| +| **Java SDK** | 17+ | Set `JAVA_HOME` environment variable | +| **Apache Maven** | 3.8+ | Build automation tool | +| **Git** | Latest | Version control | -To build without running tests, use `mvn clean install -Dmaven.test.skip=true`. -The compressed binary distribution is created at -PROJECT_DIR/modules/distribution/target. +### ๐ง Environment Setup -### Build and run docker distribution (Experimental and not recommended for production deployments) - -* This requires docker and docker-compose installed in your system +**Ubuntu/Debian:** +```bash +export JAVA_HOME=/usr/lib/jvm/openjdk-17-jdk +``` -* Build the source and docker images for each microservice +**macOS:** +```bash +export JAVA_HOME=$(/usr/libexec/java_home -v 17) ``` - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install - mvn docker:build -pl modules/distribution + +**Windows:** +```powershell +set JAVA_HOME=C:\Program Files\OpenJDK\openjdk-17 ``` -* Start supporting services and Airavata miroservices (API Server, Helix Components and the Job Monitors) + +## ๐๏ธ Building from Source + +### Quick Start + +```bash +# Clone the repository +git clone https://github.com/apache/airavata.git +cd airavata + +# Build with tests +mvn clean install + +# Build without tests (faster) +mvn clean install -DskipTests + +# Find your distribution +ls -la modules/distribution/target/ ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml up + +### ๐ณ Docker Development (Experimental) + +> โ ๏ธ **Note:** Docker deployment is experimental and not recommended for production use. + +**Prerequisites:** +- Docker Engine 20.10+ +- Docker Compose 2.0+ + +**Build and Deploy:** + +```bash +# 1. Build source and Docker images +git clone https://github.com/apache/airavata.git +cd airavata +mvn clean install +mvn docker:build -pl modules/distribution + +# 2. Start all services +docker-compose \ + -f modules/ide-integration/src/main/containers/docker-compose.yml \ + -f modules/distribution/src/main/docker/docker-compose.yml \ + up -d + +# 3. Verify services are running +docker-compose ps ``` -* Django portal and PGA Portal can be pointed to airavata.host:8930 (API) , airavata.host:8962 (Profile Service), airavata.host:8443 (Keycloak). -Make sure that you add a host entry that maps airavata.host -> 127.0.0.1 +**Service Endpoints:** +- **API Server:** `airavata.host:8930` +- **Profile Service:** `airavata.host:8962` +- **Keycloak:** `airavata.host:8443` -* To stop all the services +**Host Configuration:** +Add to your `/etc/hosts` file: ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml down +127.0.0.1 airavata.host +``` + +**Stop Services:** +```bash +docker-compose \ + -f modules/ide-integration/src/main/containers/docker-compose.yml \ + -f modules/distribution/src/main/docker/docker-compose.yml \ + down ``` -* If you do any code change and need to reflect them in the deployment, stop the docker deployment, rebuild docker images and start the docker deployment - - -## Getting Started +## ๐ Getting Started + +The fastest way to set up a development environment is through our IDE integration: + +1. **Follow our [IDE Integration Guide](./modules/ide-integration/README.md)** +2. **IntelliJ IDEA setup instructions included** +3. **Automated development environment configuration** Review Comment: for this section, instead of having points, I think it is better to keep the previous paragraph version ########## README.md: ########## @@ -1,108 +1,186 @@ # Apache Airavata -[](https://travis-ci.org/apache/airavata) -[](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) +  -## About + + [](https://travis-ci.org/apache/airavata) + [](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22org.apache.airavata%22) + [](https://opensource.org/licenses/Apache-2.0) + [](https://github.com/apache/airavata/graphs/contributors) +</div> -Apache Airavata is a software framework for executing and managing computational -jobs on distributed computing resources including local clusters, -supercomputers, national grids, academic and commercial clouds. Airavata builds -on general concepts of service oriented computing, distributed messaging, and -workflow composition and orchestration. Airavata bundles a server package with -an API, client software development Kits and a general purpose reference UI -implementation - -[Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal). +## ๐ About -Learn more about Airavata at -[https://airavata.apache.org](https://airavata.apache.org). +Apache Airavata is a modern, scalable software framework designed for executing and managing computational jobs across distributed computing resources. Whether you're working with local clusters, supercomputers, national grids, or cloud platforms, Airavata provides a unified interface for computational workflow management. -## Building Apache Airavata +**Key Features:** +- ๐ง Service-oriented architecture with distributed messaging +- ๐ Workflow composition and orchestration +- โ๏ธ Multi-cloud and hybrid cloud support +- ๐ฅ๏ธ Comprehensive API and SDK ecosystem +- ๐ Reference UI implementation via [Apache Airavata Django Portal](https://github.com/apache/airavata-django-portal) -### Prerequisites +> Learn more at [airavata.apache.org](https://airavata.apache.org) -- Sources compilation requires Java SDK 11. -- The project is built with Apache Maven 3+. -- Set or export JAVA_HOME to point to JDK. For example in Ubuntu: -`export JAVA_HOME=/usr/lib/jvm/adoptopenjdk-11` -- Git +## ๐ Prerequisites -### Build the distribution +Before building Apache Airavata, ensure you have: - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install +| Requirement | Version | Notes | +|-------------|---------|-------| +| **Java SDK** | 17+ | Set `JAVA_HOME` environment variable | +| **Apache Maven** | 3.8+ | Build automation tool | +| **Git** | Latest | Version control | -To build without running tests, use `mvn clean install -Dmaven.test.skip=true`. -The compressed binary distribution is created at -PROJECT_DIR/modules/distribution/target. +### ๐ง Environment Setup -### Build and run docker distribution (Experimental and not recommended for production deployments) - -* This requires docker and docker-compose installed in your system +**Ubuntu/Debian:** +```bash +export JAVA_HOME=/usr/lib/jvm/openjdk-17-jdk +``` -* Build the source and docker images for each microservice +**macOS:** +```bash +export JAVA_HOME=$(/usr/libexec/java_home -v 17) ``` - git clone https://github.com/apache/airavata.git - cd airavata - mvn clean install - mvn docker:build -pl modules/distribution + +**Windows:** +```powershell +set JAVA_HOME=C:\Program Files\OpenJDK\openjdk-17 ``` -* Start supporting services and Airavata miroservices (API Server, Helix Components and the Job Monitors) + +## ๐๏ธ Building from Source + +### Quick Start + +```bash +# Clone the repository +git clone https://github.com/apache/airavata.git +cd airavata + +# Build with tests +mvn clean install + +# Build without tests (faster) +mvn clean install -DskipTests + +# Find your distribution +ls -la modules/distribution/target/ ``` - docker-compose -f modules/ide-integration/src/main/containers/docker-compose.yml -f modules/distribution/src/main/docker/docker-compose.yml up + +### ๐ณ Docker Development (Experimental) + +> โ ๏ธ **Note:** Docker deployment is experimental and not recommended for production use. + +**Prerequisites:** +- Docker Engine 20.10+ +- Docker Compose 2.0+ + +**Build and Deploy:** + +```bash +# 1. Build source and Docker images +git clone https://github.com/apache/airavata.git +cd airavata +mvn clean install +mvn docker:build -pl modules/distribution + +# 2. Start all services +docker-compose \ + -f modules/ide-integration/src/main/containers/docker-compose.yml \ + -f modules/distribution/src/main/docker/docker-compose.yml \ + up -d + +# 3. Verify services are running +docker-compose ps ``` -* Django portal and PGA Portal can be pointed to airavata.host:8930 (API) , airavata.host:8962 (Profile Service), airavata.host:8443 (Keycloak). -Make sure that you add a host entry that maps airavata.host -> 127.0.0.1 +**Service Endpoints:** +- **API Server:** `airavata.host:8930` Review Comment: 8930 is not the API server port, right? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@airavata.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
