[
https://issues.apache.org/jira/browse/AIRAVATA-2689?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dimuthu Upeksha resolved AIRAVATA-2689.
---------------------------------------
Resolution: Fixed
Fixed as a part of new helix implementation. Job monitors were taken out from
the core execution logic.
https://github.com/apache/airavata/tree/develop/modules/job-monitor/email-monitor
> Distributed email clients to improve email monitoring
> ------------------------------------------------------
>
> Key: AIRAVATA-2689
> URL: https://issues.apache.org/jira/browse/AIRAVATA-2689
> Project: Airavata
> Issue Type: New Feature
> Reporter: Dimuthu Upeksha
> Assignee: Dimuthu Upeksha
> Priority: Major
> Labels: HackIllinois2018
>
> Once Airavata submits a job to a compute resource, scheduler in compute
> resource sends emails about the status of the job. Content in the email is
> different to each application type so we have written a set of parsers [2]
> which can extract correct information form email messages. Airavata has an
> email monitoring system which reads those emails, parse them and perform
> necessary actions depending on the content of the emails. However this email
> monitoring system is tightly coupled into the task execution engine so we
> can't easily replicate it to have high availability.
> Idea is to come up with a standalone email monitoring client that reads
> emails from a given email account, parse them and convert it into a standard
> message format. Once the message is parsed into the known message format, put
> it in to a queue ( rabbitmq, kafka) in order to be consumed by task execution
> engine. There are few non functional requirements
> # To improve the availability, we need to have more than one monitoring
> client to be running at a given time. However we need to make sure only
> exactly one client consumes a given email. So we need the coordination among
> email clients
> # In future, this will be deployed as a micro service, so final packaging
> should be compatible with docker
> Current email monitor implementation is this [1]. Set of parsers available
> depending on the application [2]
> [1]
> [https://github.com/apache/airavata/blob/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/EmailBasedMonitor.java]
> [2]
> https://github.com/apache/airavata/tree/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/parser
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
