Hi Tom, I shared my draft proposal with you in Apache Software Foundation through the GSoC dashboard. You can view it from there also. I really appreciate your comments and feedback on it.
I will submit my final proposal after resolving all your suggestions. Thanks & Regards, Eranga Heshan *Undergraduate* Computer Science & Engineering University of Moratuwa Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> Email: [email protected] <https://www.facebook.com/erangaheshan> <https://twitter.com/erangaheshan> <https://www.linkedin.com/in/erangaheshan> On Sat, Apr 1, 2017 at 3:41 AM, Eranga Heshan <[email protected]> wrote: > Hi Tom, > > Thank you for your quick reply. > > I re-created the design in my draft proposal since I had misunderstood > some concepts about OODT. I really appreciate if you could have a look into my > draft proposal > <https://docs.google.com/document/d/1BP55CNevap-88Vd7gPzOd2trqvAOHb2e3_jfoazA9Q8> > again and advice me further. > > Please let me bother you again for some questions. :-) > > 1. What is meant by "ingest hook" in your previous mail? > 2. I am not much familiar with Workflow Engine and Resource Manager at > this point. Hence, I have added only the proposed Kafka messaging system > for the File Manager in my design diagram. Until I catch up with those, > would that be enough? > > Thanks & Regards, > > > > > Eranga Heshan > *Undergraduate* > Computer Science & Engineering > University of Moratuwa > Mobile: +94 71 138 2686 <%2B94%2071%20552%202087> > Email: [email protected] > <https://www.facebook.com/erangaheshan> > <https://twitter.com/erangaheshan> > <https://www.linkedin.com/in/erangaheshan> > > On Fri, Mar 31, 2017 at 8:01 AM, Tom Barber <[email protected]> > wrote: > >> Hi Eranga >> >> Sorry for the delay on getting back to you. >> >> OODT 2.0 is basically whats on the development branch of the repository: >> https://github.com/apache/oodt >> >> As for communications basically you have a File Manager Client and File >> Manager Server, the server currently speaks XMLRPC or Avro. In terms of >> communication the most obvious path would be: https://github.com/apache/ >> oodt/blob/master/filemgr/src/main/java/org/apache/oodt/cas/f >> ilemgr/system/XmlRpcFileManager.java XML encoded products come into the >> file manager, then depending on what your output source would be the >> metadata is written to either Lucene, Solr or something custom: eg: >> https://github.com/apache/oodt/blob/master/filemgr/src/m >> ain/java/org/apache/oodt/cas/filemgr/catalog/solr/SolrCatalogFactory.java >> Personally I'd be interested in seeing those calls get managed by Kafka, >> so that for example multiple backends could register with an ingest hook, >> or you could have an audit listener trigger when kafka messages are passed >> for an audit log etc. >> >> I think that diagram is pretty good, basically, comms are mostly passed >> between components via RPC calls, but it would certainly be more effective >> for both inter component calls and internal bus calls, for example on the >> workflow engine where objects are passed around on a trigger type >> mechanism, for these to be replaced by Kafka messaging. >> >> Tom >> >> >> On Thu, Mar 30, 2017 at 6:21 PM, Eranga Heshan <[email protected]> >> wrote: >> >>> Hi Tom, >>> >>> I am currently looking into http://oodt.apache.org/site_do >>> cs/cas-filemgr/development/developer.html to get an abstract idea of >>> how OODT File Manager handles its internal communications. I hope that >>> would help me to identify which components needed to be implemented in >>> Kafka. If my approach is wrong please help me to make it correct. >>> >>> Further, I really appreciate if you could have a look into my proposal >>> and give your opinions on how to make it strong. >>> >>> Thanks & Regards, >>> >>> >> >
