Suresh Marru commented on AIRAVATA-2689:

[~apoorvp2] has worked on this issues as part of GSOC last year and has 
architecture diagram and discussion at - 
 I am still looking for the code related to this contribution. 

> Distributed email clients to improve email monitoring 
> ------------------------------------------------------
>                 Key: AIRAVATA-2689
>                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2689
>             Project: Airavata
>          Issue Type: New Feature
>            Reporter: Dimuthu Upeksha
>            Assignee: Dimuthu Upeksha
>            Priority: Major
>              Labels: HackIllinois2018
> Once Airavata submits a job to a compute resource, scheduler in compute 
> resource sends emails about the status of the job. Content in the email is 
> different to each application type so we have written a set of parsers [2] 
> which can extract correct information form email messages. Airavata has an 
> email monitoring system which reads those emails, parse them and perform 
> necessary actions depending on the content of the emails. However this email 
> monitoring system is tightly coupled into the task execution engine so we 
> can't easily replicate it to have high availability.
> Idea is to come up with a standalone email monitoring client that reads 
> emails from a given email account, parse them and convert it into a standard 
> message format. Once the message is parsed into the known message format, put 
> it in to a queue ( rabbitmq, kafka) in order to be consumed by task execution 
> engine. There are few non functional requirements
>  # To improve the availability, we need to have more than one monitoring 
> client to be running at a given time. However we need to make sure only 
> exactly one client consumes a given email. So we need the coordination among 
> email clients
>  #  In future, this will be deployed as a micro service, so final packaging 
> should be compatible with docker
> Current email monitor implementation is this [1]. Set of parsers available 
> depending on the application [2]
> [1] 
> [https://github.com/apache/airavata/blob/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/EmailBasedMonitor.java]
> [2] 
> https://github.com/apache/airavata/tree/master/modules/gfac/gfac-impl/src/main/java/org/apache/airavata/gfac/monitor/email/parser

This message was sent by Atlassian JIRA

Reply via email to