Hi Pankaj, I still don’t see your application in the GSOC site. Please make sure you have done this correctly.
Thanks, Marlon From: Pankaj Saha <psa...@binghamton.edu<mailto:psa...@binghamton.edu>> Date: Monday, March 21, 2016 at 5:17 PM To: marpierc <marpi...@iu.edu<mailto:marpi...@iu.edu>> Cc: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" <dev@airavata.apache.org<mailto:dev@airavata.apache.org>> Subject: Re: [GSoC Proposal] - Integrating Resource Information from Apache Mesos with Apache Airavata’s Job Management Modules Hi Marlon, Here is the link that I have created. https://docs.google.com/document/d/1qtFvg4-usT4D_1TDNBsQDFQGZIkH99ideYQ1T3HU9nY/edit?usp=sharing The draft is created under the GSoC proposal site under Apache foundation with the same title. Thanks Pankaj On Mon, Mar 21, 2016 at 4:47 PM, Pierce, Marlon <marpi...@iu.edu<mailto:marpi...@iu.edu>> wrote: Hi Pankaj, I have some comments, but it would be easier if you created a proposal draft in the GSOC site. The google doc option for your draft is better than pointing to the Airavata wiki. Please make sure you give comment and suggestion permissions. Marlon From: Pankaj Saha <psa...@binghamton.edu<mailto:psa...@binghamton.edu>> Reply-To: "dev@airavata.apache.org<mailto:dev@airavata.apache.org>" <dev@airavata.apache.org<mailto:dev@airavata.apache.org>> Date: Monday, March 21, 2016 at 11:16 AM To: dev <dev@airavata.apache.org<mailto:dev@airavata.apache.org>> Subject: [GSoC Proposal] - Integrating Resource Information from Apache Mesos with Apache Airavata’s Job Management Modules Hi Dev Team, Please review the following GSoC proposal that I plan to submit: Title: Integrating Resource Information from Apache Mesos with Apache Airavata’s Job Management Modules Abstract: Apache Airavata provides gateway computing capability across clustered environments for scientific users. It abstracts away the complexities of submitting jobs to HPC platforms and provides users with an intuitive and elegant web-based interface to submit jobs. Apache Mesos is a distributed kernel that manages distributed computing resources as a single computer. As Airavata is being extended to use Big Data and Cloud tools to launch jobs in cloud environments, it needs to retrieve the resource and job execution information from the Big Data framework back to the Apache portal accessible to the end user. In this project we will develop code and scripts to be integrated with the Airavata that will use the HTTP API of Mesos to continuously fetch the complete resource and scheduling information. This information can then be used by Airavata to dynamically monitor and improve its job submission strategy in cloud environments such as Jetstream. Introduction: Apache Mesos provides HTTP API endpoints for scheduler, executor, internal and admin related queries. To fetch information regarding a clustered environment that is managed by the Mesos master, the API can be accessed via curl requests over HTTP. The response to such requests will be received as well formed json document. We will parse the json response and present the information in the format desired. The retrieved information will include resource usage, resource available for further jobs, job status, time elapsed since the job started, etc. Airavata, in turn, will use this information to determine the resource usage, performance of the jobs on a job submission, rapid diagnosis on the health of the submitted jobs. We will use the observer pattern to continuously pull information from Cloud and big Data Resource Managers, such as Apache Mesos, to Airavata. Any comment and suggestions would be very helpful. Thanks Pankaj
