This sounds fabulous. I will be keen to help. ------------------------ Chris Mattmann [email protected]
-----Original Message----- From: Lewis John Mcgibbney <[email protected]> Reply-To: <[email protected]> Date: Wednesday, February 25, 2015 at 1:54 PM To: Aditya Dhulipala <[email protected]> Cc: "[email protected]" <[email protected]> Subject: Re: GSoC 2015 >I think that you should aim to implement it on all components and we >should >be looking to merge to code into OODT (branch) incrementally. >It is OK that you may not get every component ported to Avro RPC, what is >impoirtant is that there is an optimistic but realistic GSoC put forward. >That is what we are looking for. >Thank you >LEwis > > >On Wed, Feb 25, 2015 at 1:48 PM, Aditya Dhulipala <[email protected]> >wrote: > >> Hi Lewis, >> >> Thanks for your reply! >> >> Your responses have helped immensely when I'm stuck on something! >> >> In the proposal that I was preparing I had listed out all the components >> that would require schema definitions and then when I checked the OODT >> patch 658, I realized that a lot of this was done for the Gora project. >>But >> your email has clarified that I can use that as a starting point for the >> Avro project. This is extremely useful >> >> And thanks for the rest of the info as well (about ensuring backwards >> compaitibility, testing, regression testing).. Now I have a much better >> idea of formulating a proposal (and the project to-dos also). >> >> I'll will have it ready ASAP. I will post it to the group by end of >>today >> so that I can get more feedback on it >> >> I think I should at least be able to define Avro RPC implementations for >> one of the components of OODT in the GSoC duration, right? >> Define the schema >> Implement the services >> Write unit tests >> Regression test against XML-RPC >> >> Hopefully I should implement it for more than one component, but I'm >>still >> no able to estimate the workload. I'll continue reading up on this >> >> I'll continue to work on the proposal and keep you updated. >> >> Thanks for all the help!.. I think if I start early, then I can spend >>the >> summer coding from the begining.. >> >> Thanks! >> >> -- >> Aditya >> >> >> adi >> >> On Wed, Feb 25, 2015 at 9:34 AM, Lewis John Mcgibbney < >> [email protected]> wrote: >> >>> Hi Adi, >>> >>> On Wed, Feb 25, 2015 at 12:34 AM, Aditya Dhulipala <[email protected]> >>> wrote: >>> >>>> Hi Lewis, >>>> >>>> I was going through the path you posted earlier. OODT- 658 >>>> https://issues.apache.org/jira/browse/OODT-658 >>>> >>>> I think this is a substantial part of the project we're currently >>>> talking about (XML-RPC overhaul). >>>> >>> >>> Substantial may be a wee bit optimistic ;) But yes a significant >>>portion >>> of thinking in to the OODT data structures logic has been done. We DO >>>need >>> to implement Metadata in exactly the right way without loosing existing >>> functionality so please begin to think about that. >>> >>> >>>https://github.com/apache/oodt/blob/trunk/metadata/src/main/java/org/apa >>>che/oodt/cas/metadata/Metadata.java >>> >>> >>>> My understanding is that this patch was implemented to make Apache >>>>Gora >>>> communicate with OODT, so that's why you've implemented the schema >>>> definitions for all the data structures used by OODT. >>>> >>> >>> Correct >>> >>> >>>> Gora generates some statically typed code from this schema >>>> >>> >>> Using the GoraCompiler >>> http://gora.apache.org/current/compiler.html, >>> invoked via CompilerCLI >>> >>> >>> >>>> and the next step is to implement OODT logic to store the data in Gora >>>> (as opposed to MySQL or Solr) >>>> >>> >>> YES. This will tidy A LOT of the current configuration up. Will also >>>have >>> a unified and well documented manner for configuring the mappings and >>> datastore specific configuration. All of the Gora datastores are >>>documented >>> here >>> http://gora.apache.org/current/index.html >>> I've been hacking away on documentation for Gora for about a year so it >>> is now relatively OK. I hope you find it useful. >>> >>> >>>> >>>> So from the viewpoint of the project we're talking about i.e. >>>>Replacing >>>> XML-RPC with Avro, >>>> the schema definition part is pretty much done (or almost done? Need >>>>to >>>> define it within OODT as well?). >>>> >>> >>> Note, that NONE of the Avro RPC logic is implemented. So it is nowhere >>> nearly done ;) The core project definition is still to be addressed >>>and I >>> am nearly 100% sure that we will have some trciky issues to address >>> regarding 1) maintaining as close to backwards compatability as >>>possible 2) >>> documenting the entire Avro RPC communications within OODT, 3) Hooking >>>up >>> all services, 4) Testing the new implementation, 5) regression testing >>>it >>> against the existing XML-RPC layer, 6) setting a roadmap fro >>>deprecation >>> and eventual removal of the XML-RPC material >>> >>> >>>> The next step would be to define RPC logic for the client server >>>> communication within OODT itself i.e. within filemgr, workflowmgr etc. >>>> >>> >>> Correct, this should make up the majority of your proposal OK. >>> >>> >>>> >>>> Am I correct in understanding this? >>>> >>>> >>>> Yes and thank you for joining the dots, it is nice to see a student >>> interpreting and investigating the problem this much prior to the >>>project >>> starting. I am really looking forward to this now. >>> Thanks >>> LEwis >>> >> >> > > >-- >*Lewis*
