Hi Etienne, Thanks for your query. The default protocol used by the RemoteDataTransfer is XML-RPC. It's a call-back mechanism that I baked up myself with a configurable chunk size parameter.
My suggestion is rather than use that DataTransfer, I would leverage: (1) a distributed filesystem like HadoopFS, Tachyon (from BDAS), GlusterFS, or even NFS that logically mounts a set of local shared nothing disks as a virtual global mount (2) use the LocalDataTransfer in the File Manager, with #1 in place The main reason for this is that it allows the people who work on distributed and reliable filesystems that solve those problems in a great way, it allows us in the OODT community to take advantage of that work without having to write all the complex functionality ourselves. That way we just have light-weight insulated pieces/components in OODT that rely on the workhorses that do things right. HTH! Cheers, Chris ------------------------ Chris Mattmann [email protected] -----Original Message----- From: Etienne Koen <[email protected]> Date: Wednesday, September 10, 2014 1:57 AM To: Shakeh Khudikyan <[email protected]> Cc: Chris Mattmann <[email protected]> Subject: filemgr ingestion >Hi Shakeh, >What is the default protocol that filemgr uses for RemoteDataTransfer >ingestion? > >Also, is the a way to specify the protocol? > >Cheers >Etienne >
