Good work man!! :) On Mar 19, 2016 12:37 AM, "Siddharth Jain" <[email protected]> wrote:
> Hello all, > > I have completed creating a stand-alone implementation/POC of job monitor in > the monitoring module of the codebase. > > > > I did fork from a dev branch and code, but I was unable to commit. I tried > creating patch, but the patch did not aggregate all the changes, so I went > ahead and just created a copy of the module (I think it should be as good > as a patch) for now and attached it to the JIRA issues: > > https://issues.apache.org/jira/browse/AIRAVATA-1912 > > https://issues.apache.org/jira/browse/AIRAVATA-1914 > > https://issues.apache.org/jira/browse/AIRAVATA-1915 > > > > This code addresses the sub-task: > > · AIRAVATA-1914 > > · AIRAVATA-1915 > > I have also flagged the above two sub-tasks as patch. > > > > Briefly this is what stand-alone implementation does right now: > > · Creates a fanout exchange with durable queues (will persist after > server restarts) and auto-recovery option enabled; declares and binds all > the consumer queues > > · Fetch e-mails from a dummy email address > > · Converts the *non-serializable* javax Message objects into custom > *serializable* MessageExtract objects, which consists of only the > necessary information, viz. “from”, “to”, “subject” and “content” fields, > which are being used in the existing codebase > > · Serialize the messages > > · Send the message to the exchange, which in turn will deliver the > message to all the queues bound to it > > · Two consumers threads fetch the messages from their respective > queues > > · Deserialize the message and convert it into javax Message object, > so that it can be used with the existing code > > · Print the received message content > > > > I have tried to keep all the components object-oriented. Following is the > explanation of the components, > > Core Components: > > · RabbitMQEmailPublisher: RabbitMQ publisher for e-mail messages > > · EmailReceiver: Represents a consumer, which spawns a thread and > keeps on receiving messages and process them. Multiple EmailReceiver can be > instantiated and they will automatically run in different threads. > > · GmailSMTPMailBox: represents gmail mail box > > · EmailConsumer: This has the implementation of call back for > received messages. > > · MessageExtract: Represents the extract from e-mail, which is > serializable > > Simulation Components: > > · FetchPublish: Fetches emails with GmailSMTPMailBox and publishes > to message bus with RabbitMQEmailPublisher > > · Simulator (also the main class to run): > > o Instantiates two consumers (EmailReceiver) in separate threads > > o Fetches email and publishes it with FetchPublish > > o Shuts down the consumers after a minute > > > > Following is the TO-DO list: > > · Use the actual email server configs > > o Currently I am using my own dummy account for testing, did not want > to accidentally mess up the production mail-boxes > > · Use the codebase broker config > > o Currently I am using a local broker url and config > > · Fetch the actual email text/body > > o Currently I am using a hard-coded string for email text, as somehow > when I extract the content, I always get multi-part content. The code you > guys already have in the repository seems to be written with an assumption > that the received message content will be text only, I tried creating an > email like that, but it always has multi-part component. So once I have > integrated the code with the existing monitor, I can get back to fetching > the actual content > > · Open the inbox in read_write mode > > o Currently I am opening the inbox in read mode, so that the messages > are not marked as read and so I don’t have to repeatedly create new dummy > messages > > · Probably implement a RabbitMQ Factory singleton > > · Disable message(from message queue) auto-ack and only ack after > processing has been completed, this might help making the workflow more > robust > > · Use the codebase properties file for configuration of broker and > email service, right now creating properties on run time > > · Create test-cases > > I looked into integrating this into the live work-flow, but following > things kind of need time: > > · The message processing work-flow will have to be inverted, it’s > kind of different than we assumed. Currently the messages are fetched > explicitly and then processing is done. The way I have implemented the > consumers is, the queue messages will be delivered asynchronously, so we > don’t have to poll for it. So all the processing part will have to be wired > with call-back function. > > · Setup PGA and its dependencies on local for testing > > · Understand in which part of the life-cycle the consumer(or > equivalent of that) threads are to be invocated and what specific shutdown > interrupts are they to follow and respond to > > · Understand properties files conventions > > Hence the aforementioned list with the TO-DO list will be part of > milestone 1. > > > > Best regards, > > Siddharth Jain >
