Hi Tom, I uploaded my final proposal to the GSoC Dashboard a few minutes ago.
I have followed these two architecture files until now to understand how internal communications of OODT actually works. - File Manager: http://oodt.apache.org/site_docs/cas-filemgr/development/developer.html - Workflow Manager: https://oodt.apache.org/site_docs/cas-workflow/development/developer.html However, I could not find an architecture diagram for Resource Manager yet. I would be grateful if you could share the architecture document of Resource Manager if any. Finally, I would be much grateful if you could read my proposal completely and understand my capability of completing this project successfully. :-) Thanks and best regards, Eranga Heshan *Undergraduate* Computer Science & Engineering University of Moratuwa Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> Email: [email protected] <https://www.facebook.com/erangaheshan> <https://twitter.com/erangaheshan> <https://www.linkedin.com/in/erangaheshan> On Sat, Apr 1, 2017 at 1:14 PM, Eranga Heshan <[email protected]> wrote: > Hi Tom, > > I shared my draft proposal with you in Apache Software Foundation through > the GSoC dashboard. You can view it from there also. I really appreciate > your comments and feedback on it. > > I will submit my final proposal after resolving all your suggestions. > > Thanks & Regards, > > > > Eranga Heshan > *Undergraduate* > Computer Science & Engineering > University of Moratuwa > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> > Email: [email protected] > <https://www.facebook.com/erangaheshan> > <https://twitter.com/erangaheshan> > <https://www.linkedin.com/in/erangaheshan> > > On Sat, Apr 1, 2017 at 3:41 AM, Eranga Heshan <[email protected]> > wrote: > >> Hi Tom, >> >> Thank you for your quick reply. >> >> I re-created the design in my draft proposal since I had misunderstood >> some concepts about OODT. I really appreciate if you could have a look into >> my >> draft proposal >> <https://docs.google.com/document/d/1BP55CNevap-88Vd7gPzOd2trqvAOHb2e3_jfoazA9Q8> >> again and advice me further. >> >> Please let me bother you again for some questions. :-) >> >> 1. What is meant by "ingest hook" in your previous mail? >> 2. I am not much familiar with Workflow Engine and Resource Manager >> at this point. Hence, I have added only the proposed Kafka messaging >> system >> for the File Manager in my design diagram. Until I catch up with those, >> would that be enough? >> >> Thanks & Regards, >> >> >> >> >> Eranga Heshan >> *Undergraduate* >> Computer Science & Engineering >> University of Moratuwa >> Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> >> Email: [email protected] >> <https://www.facebook.com/erangaheshan> >> <https://twitter.com/erangaheshan> >> <https://www.linkedin.com/in/erangaheshan> >> >> On Fri, Mar 31, 2017 at 8:01 AM, Tom Barber <[email protected]> >> wrote: >> >>> Hi Eranga >>> >>> Sorry for the delay on getting back to you. >>> >>> OODT 2.0 is basically whats on the development branch of the repository: >>> https://github.com/apache/oodt >>> >>> As for communications basically you have a File Manager Client and File >>> Manager Server, the server currently speaks XMLRPC or Avro. In terms of >>> communication the most obvious path would be: https://github.com/apache/ >>> oodt/blob/master/filemgr/src/main/java/org/apache/oodt/cas/f >>> ilemgr/system/XmlRpcFileManager.java XML encoded products come into the >>> file manager, then depending on what your output source would be the >>> metadata is written to either Lucene, Solr or something custom: eg: >>> https://github.com/apache/oodt/blob/master/filemgr/src/m >>> ain/java/org/apache/oodt/cas/filemgr/catalog/solr/SolrCatalo >>> gFactory.java >>> Personally I'd be interested in seeing those calls get managed by Kafka, >>> so that for example multiple backends could register with an ingest hook, >>> or you could have an audit listener trigger when kafka messages are passed >>> for an audit log etc. >>> >>> I think that diagram is pretty good, basically, comms are mostly passed >>> between components via RPC calls, but it would certainly be more effective >>> for both inter component calls and internal bus calls, for example on the >>> workflow engine where objects are passed around on a trigger type >>> mechanism, for these to be replaced by Kafka messaging. >>> >>> Tom >>> >>> >>> On Thu, Mar 30, 2017 at 6:21 PM, Eranga Heshan <[email protected]> >>> wrote: >>> >>>> Hi Tom, >>>> >>>> I am currently looking into http://oodt.apache.org/site_do >>>> cs/cas-filemgr/development/developer.html to get an abstract idea of >>>> how OODT File Manager handles its internal communications. I hope that >>>> would help me to identify which components needed to be implemented in >>>> Kafka. If my approach is wrong please help me to make it correct. >>>> >>>> Further, I really appreciate if you could have a look into my proposal >>>> and give your opinions on how to make it strong. >>>> >>>> Thanks & Regards, >>>> >>>> >>> >> >
