Hi Aditya, On Tue, Feb 10, 2015 at 11:05 PM, Aditya Dhulipala <[email protected]> wrote:
> Hi Lewis, > > Thanks for the info! > No problems, thanks for looking in to this one. If this project goes ahead I can guarantee you it will be a MAJOR contribution to OODT. > > I've been looking at the code you've posted above and trying to get a sense > of what it does. > I'm looking through the imports and I found some schema definitions in > XmlStructFactory & XmlRpcStructFactory make this suitable for transfer > over xml-rpc. > Correct > > My guess is that this project would be to write .avpr files (Avro files) > that define these file manager structs This would be my advice. We first define the Schema and Protocol implementations, then we move towards functional implementations. > & then replace all the xml-rpc > (XmlStructFactory & XmlRpcStructFactory) with Avro specific implementations > like this > https://github.com/phunt/avro-rpc-quickstart > Yes, although Patrick's code is a 'little' out of date (running on 1.7.4 (from memory), whereas Avro is running on 1.7.7.) Patrick has captured very conveniently some excellent integration and definitions for us. > > I understand what Avro is and how its better than xmlrpc for performance > reasons. I'm interested in reading more about this topic. And in particular > how it affects various components of OODT. > Well, this is exactly what needs to be within a proposal for a GSoC project so I am VERy keen to help you out here. > > Can you point me in the right direction? > > So, some more higher level Javadoc https://cwiki.apache.org/confluence/display/OODT/Apache+OODT+APIs#ApacheOODTAPIs-XML-RPC You will also see that the data structrues in the OODT services packages e.g. File Manager, Workflow Manager and Resource Manager are always located in a /structs/ package. These are the fundamental data structures we need to expose when thinking about the Avro Protocol implementations. The basic user documentation for File Manager can be found http://oodt.apache.org/components/maven/filemgr/user/basic.html The advanced (certainly advanced to a level we currently require) documentation for Mile Manager can e found http://oodt.apache.org/components/maven/filemgr/development/developer.html I am sure you will find the latter useful. Please write back here once you've engaged with this, we will take it from there. It may also be a good idea to be jotting down how you see a potential workflow coming together e.g. what tasks do you think will need implemented in Avro RPC. This will make it much easier when we actually come to running and planning against a schedule of work. Thanks again Lewis
