Hi, On 27/07/17 13:54, Traiano Welcome wrote: > Hi Stephen > > > On Thu, Jul 27, 2017 at 12:19 PM, Stephen Gran <[email protected]> > wrote: > Both spark and flink integrate natively with mesos, so no need for an > intermediate yarn layer. For batch work, we're looking at the aurora > project for job scheduling. > > > > I haven't looked at Aurora before - would you consider it a drop in > replacement for hadoop for distributed batch workloads?
It's definitely not a drop in replacement - they have very different APIs and capabilities. What aurora gives us is a DSL to build the DAG of an execution, and with a little work, some primitives to run those executions. So, the functionality ends up being similar for 'just batch', but the language, the bindings, etc are all very different. Cheers, -- Stephen Gran Senior Technical Architect picture the possibilities | piksel.com
