Hi Airavata team, I’m Ramya Vedi, proposing to containerize the Agent service for GSoC 2025. I’ve attached my full proposal, including a workflow diagram, and summarized key sections below. I’d appreciate your review and guidance on two points: the correct `*airavata.storageResourceId*` value, and whether the `*protoSourceRoot*` in `agent-service/pom.xml` should be explicitly defined (details in Section 5).
**Keywords**: Containerization, Docker, Docker Compose, DevOps, Microservices, Java, Maven, gRPC, MariaDB, Cloud, Scientific Workflows, Open Source, Deployment, Kubernetes, CI/CD, Documentation *Contributor Information* - Name: Ramya Vedi - GitHub: [http://github.com/RamyaVedi](http://github.com/RamyaVedi) - LinkedIn: [ https://www.linkedin.com/in/ramya-vedi/](https://www.linkedin.com/in/ramya-vedi/) - Email: ramyasasidara...@gmail.com - Location: Ipswich, UK - Time Zone: UTC/GMT+1 hour - Current Work Experience: DevOps Engineer at Ariyath Technologies - Education: Bachelor of Engineering in Computer Science, Anna University (2007–2011) - Motivation: I’m excited to contribute to Apache Airavata and grow my open-source experience through GSoC 2025, applying my DevOps skills to enhance scientific workflows for the community. *Personal Background - Work Experience * I’m a DevOps Engineer at Ariyath Technologies, where I bring hands-on expertise to infrastructure and deployment challenges. I hold a Bachelor of Engineering in Computer Science from Anna University (2007–2011). My experience includes: - Leading the deployment of over 15 containerized microservices using Docker and Kubernetes, enhancing scalability for enterprise applications at Ariyath Technologies. - Implemented CI/CD pipelines with Jenkins and GitHub Actions, automating builds and deployments for 10+ projects, reducing release cycles by 30%. - Configured and optimized MariaDB and cloud-based databases, ensuring high availability for production workloads—skills I applied to integrate MariaDB with Apache Airavata’s Agent service. - Began my open-source journey by contributing a pull request to `first-contributions` on GitHub, igniting my passion for collaborative development, which led me to GSoC 2025 via YouTube’s open-source insights and Apache’s community focus. -* Successfully containerized Airavata’s Agent service (ports 18880/9090) with MariaDB (3306), achieving a running proof-of-concept (PoC) with `docker ps` validation, driving my proposal to modernize Airavata’s deployment ecosystem.* *Skill Set* I bring a combination of technical expertise, DevOps experience, and a collaborative mindset to this project, ensuring I can deliver a robust containerized deployment for Apache Airavata. My key skills include: - Java: Proficient in Java, with hands-on experience building Java-based applications, as demonstrated by compiling the Airavata Agent service (`agent-service-0.21-SNAPSHOT.jar`). - Maven: Experienced in using Maven for dependency management and builds, resolving issues like `airavata.storageResourceId` and `proto source root` errors in Airavata. - Docker: Skilled in containerizing applications with Docker, creating optimized Dockerfiles (e.g., using `openjdk:21-jdk-slim`) and managing multi-container setups with Docker Compose, as shown by my Agent service deployment. - DevOps: Experienced in CI/CD pipeline automation (e.g., Jenkins, GitHub Actions) and infrastructure optimization at Ariyath Technologies, where I deployed 15+ microservices and reduced release cycles by 30%. Applied these skills to troubleshoot Airavata’s deployment issues (e.g., `KeyError: 'ContainerConfig'`). - Microservices: Familiar with microservices architecture and gRPC, used in Airavata for inter-service communication (e.g., Agent service on port 9090). - Collaboration and Open-Source Contribution: New to open source but eager to learn, with an initial contribution to `first-contributions` on GitHub. Actively engaged with the Airavata dev mailing list. - Documentation and Communication**: Able to document technical processes clearly, as shown by my setup steps and troubleshooting notes for the Agent service. *Project Progress * *I successfully containerized the Agent service: * - Cloned the Airavata repository, built it with Maven, and joined the dev mailing list. - Encountered a `proto source root` error during the build, as the `protobuf-maven-plugin` couldn’t locate the `.proto` files. Resolved by adding `<protoSourceRoot>${project.basedir}/../proto</protoSourceRoot>` to `agent-service/pom.xml`, pointing to the `proto` directory. Should this path be explicitly defined, or is a root-level build (`mvn clean install` from the project root) the recommended approach? - Created a `Dockerfile` using `openjdk:21-jdk-slim`, copying the JAR and configuration files. - Set up `docker-compose.yml` to run the Agent service with MariaDB, configuring datasource properties and ports (18880 for HTTP, 9090 for gRPC). - Resolved a Spring error (`Could not resolve placeholder 'airavata.storageResourceId'`) by adding it to `application.properties` and as an environment variable (`default-storage-id`). - Addressed a Compose compatibility issue (`KeyError: 'ContainerConfig'`) and a health check failure by simplifying the configuration and clearing stale state. - Confirmed with `docker ps`: CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 3312ca8e2fc0* target_agent-service* "java -Dspring.confi…" 21 seconds ago Up 15 seconds 0.0.0.0:9090->9090/tcp, 0.0.0.0:18880->18880/tcp target_agent-service_1 96718fa79e21 *mariadb:10.11* "docker-entrypoint.s…" About a minute ago Up About a minute (healthy) 0.0.0.0:3307->3306/tcp target_mariadb_1 - Tested with `curl http://localhost:18880`, *receiving a 404 response*, which verifies the HTTP server is running (as expected, since the endpoint isn’t implemented). I plan to test gRPC endpoints next using `grpcurl`. *Proposal Summary* Apache Airavata facilitates scientific workflows but faces deployment challenges due to complex dependencies. This project aims to containerize Airavata’s core services using Docker and Docker Compose, starting with the Agent service, to enable portable, consistent, and scalable deployments for researchers. As a DevOps Engineer at Ariyath Technologies with expertise in Docker, Java, and Maven, I’ve already containerized the Agent service (ports 18880/9090) with MariaDB (3306), resolving issues like the `proto source root` error and `airavata.storageResourceId` configuration. The deployment was validated using `docker ps` and `curl http://localhost:18880` (returning a 404, confirming the HTTP server is active). Over 8 weeks, I’ll finalize the Agent, containerize the Registry and PGA services, and deliver optimized Dockerfiles, a multi-service `docker-compose.yml`, and a comprehensive deployment guide. This will simplify Airavata’s setup for scientific workflows, with future steps including Kubernetes orchestration. See Section 9 for the workflow diagram. *Project Goals* - Analyze Airavata Services and Dependencies: Deeply understand the architecture, dependencies, and build process of Airavata services (e.g., Agent, Registry, PGA) by studying the codebase and documentation, ensuring each service’s JAR file is correctly built using Maven. I’ve already built the Agent service JAR (`agent-service-0.21-SNAPSHOT.jar`) and resolved dependency issues like `airavata.storageResourceId`. - Develop Dockerfiles for Each Service: Create optimized Dockerfiles for each Airavata service, starting with the Agent service, using lightweight base images (e.g., `openjdk:21-jdk-slim`). The Dockerfile will package the JAR, configuration files, and runtime dependencies. I’ve completed this for the Agent service, successfully building the `target_agent-service` image. - Deploy Services with Docker Compose: Implement a multi-container deployment using Docker Compose, integrating services with dependencies like MariaDB, and ensuring reliable startup with health checks. I’ve deployed the Agent service (ports 18880/9090) alongside MariaDB (port 3306), validated by `docker ps` in Section 5. - Test and Validate Deployments: Verify functionality of deployed services through HTTP (`curl`) and gRPC (`grpcurl`) tests, ensuring stability and documenting configurations. I’ve tested the Agent service with `curl http://localhost:18880` (received a 404, confirming the HTTP server is active) and plan to test gRPC endpoints next. - Iterate and Expand to Other Services: Extend containerization to additional Airavata services (e.g., Registry, PGA) over the 8-week GSoC period, refining the setup based on community feedback and creating a comprehensive deployment guide for developers as a key deliverable. *Planned GSoC Work Hours - Commitments* I commit to dedicate 175+ hours to this project. I do not plan on any personal vacations. If anything comes up, I will inform the mentor and communicate promptly. - **Weekdays**: 6 Hours (UTC) - Monday and Friday: 19:30 to 00:30 UTC [Fixed] - **Weekends**: 5 Hours (UTC) - Saturday and Sunday: 17:30 to 22:30 UTC [Flexible] - **Estimated Hours Per Week**: 40 hours (Weekdays: 6 hours × 5 days = 30 hours; Weekends: 5 hours × 2 days = 10 hours) Based on circumstances, I am willing to adjust and re-plan the timing. I will write one blog article each week to explain my progress and stay in touch via the dev mailing list and any preferred channels like Slack or Zoom. Thanks for your feedback! Ramya Vedi
docker-compose.yml
Description: Binary data
Dockerfile
Description: Binary data
