Dimuthu Upeksha created AIRAVATA-2689:

             Summary: Distributed email clients to improve email monitoring 
                 Key: AIRAVATA-2689
                 URL: https://issues.apache.org/jira/browse/AIRAVATA-2689
             Project: Airavata
          Issue Type: New Feature
            Reporter: Dimuthu Upeksha
            Assignee: Dimuthu Upeksha

Once Airavata submits a job to a compute resource, scheduler in compute 
resource sends emails about the status of the job. Content in the email is 
different to each application type so we have written a set of parsers [2] 
which can extract correct information form email messages. Airavata has an 
email monitoring system which reads those emails, parse them and perform 
necessary actions depending on the content of the emails. However this email 
monitoring system is tightly coupled into the task execution engine so we can't 
easily replicate it to have high availability.

Idea is to come up with a standalone email monitoring client that reads emails 
from a given email account, parse them and convert it into a standard message 
format. Once the message is parsed into the known message format, put it in to 
a queue ( rabbitmq, kafka) in order to be consumed by task execution engine. 
There are few non functional requirements
 # To improve the availability, we need to have more than one monitoring client 
to be running at a given time. However we need to make sure only exactly one 
client consumes a given email. So we need the coordination among email clients
 #  In future, this will be deployed as a micro service, so final packaging 
should be compatible with docker

Current email monitor implementation is this [1]. Set of parsers available 
depending on the application [2]



This message was sent by Atlassian JIRA

Reply via email to