Hi Roman, At this point the integration is pluggable in memory file system, GGFS. It works just like HDFS (same API), but in reality serves as a caching layer on top of HDFS. GGFS caches the hottest file blocks and then synchronizes them with underlying HDFS either synchronously or asynchronously, depending on configuration.
Since, GGFS implements standard Hadoop File System API, it automatically integrates with other Hadoop ecosystem pieces via File System API as well. Going forward, we are planning to add same native API integration for MapReduce component as well. Hope this answers your question. -Dmitriy On Mon, Mar 24, 2014 at 11:11 PM, Roman Shaposhnik <[email protected]> wrote: > Hi Dmitriy! > > Welcome to the Bigtop community! > > On Mon, Mar 24, 2014 at 10:43 PM, Konstantin Boudnik <[email protected]> > wrote: > >> One of the main pieces of our platform is our In-Memory Apache Hadoop > >> Accelerator which aims to accelerate HDFS and Map/Reduce by bringing > both, > >> data and computations into memory. We do it with our GGFS - Hadoop > >> compliant in-memory file system. For I/O intensive jobs GridGain GGFS > >> offers performance close to 100x faster than standard HDFS. More > >> information can be found here: > >> http://www.gridgain.org/features/hadoop-acceleration/ > >> > >> We would like to have an opportunity to integrate our Apache Hadoop > >> Accelerator with Apache Bigtop. Please let us know if this is possible > and > >> what steps are required of us. > > I've been actually fascinated by the in-memory analytics platforms lately. > Things like Apache Spark seem to be a really good addition to the > Hadoop ecosystem. > > Now, I understand that you've got a piece of technology that can > essentially > serve as a replacement for HDFS, but could you please elaborate on > what other integration points do you have between GridGain and the rest > of Hadoop ecosystem? > > That, I think, would be a much wider discussion. > > Thanks, > Roman. >
