Nice summary, Siddharth. From: Siddharth Jain <[email protected]<mailto:[email protected]>> Reply-To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Date: Friday, March 18, 2016 at 9:06 PM To: "[email protected]<mailto:[email protected]>" <[email protected]<mailto:[email protected]>> Subject: Summary: Job Monitor Discussion
Hello all, A discussion was held on 3/17/2016 regarding the implementation of a separate demon service which reads job status from email and sends it out to a message bus. For more background information about this topic, please refer: https://issues.apache.org/jira/browse/AIRAVATA-1912. Following were the contributors to this discussion: · Marlon Pierce · Shameera Rathnayaka · Siddharth Jain · Suresh Marru Summary of discussion, Approaches for receiving messages from job executor/HPC 1.) Develop a custom IMAP/SMTP server, which can receive the messages directly sent by the job executor and place the email on message bus immediately. 2.) Setup a simple IMAP/SMTP server on our network, fetch the messages from it and place the message on message bus. 3.) Use the third-party provider for email-service, connect to the email service provider periodically, fetch the messages and put it on the message bus. Approach 3 was finalized, because: · Maintenance of custom server or a separate server might be a challenge in future · Third-party email service provider take care of the up-time and any issues with regular email service Approaches for putting messages on message bus 1.) Fetch the unread messages from the mail-box and broadcast them as it is to all the consumer queues bound to exchange. With minimum-invasive technique have the messages handled with the existing code on the consumer side. 2.) Fetch the unread messages, classify them and accordingly route the messages to appropriate consumer queue. For classifying and determining which message is sent to which consumer queue, existing logic from GFAC module can be reused. 3.) Fetch the unread messages, send all of them to a common endpoint, which in turn will classify them and accordingly route the messages to appropriate consumer queues. Initially approach 1 will be implemented, but the goal is to implement approach 3 ultimately. In addition to the above, in general the design has to be such that it can accommodate consuming messages from different e-mail service providers and any other protocol (for example UDP) Please feel free to add anything that has been missed out. Best regards, Siddharth Jain
