Hey, Milind, Thanks for your interests. - We are working on moving the design to Hadoop 2.x. It will be available in the near future.
- This is an independent project. Xiaoyi On Mon, Nov 11, 2013 at 2:43 PM, Milind Bhandarkar <[email protected]> wrote: > Hi Xiaoyi, > > This is great ! > > For clusters that have moved to Hadoop 2.x versions, do you have any work > in progress to rebase Hadoop-RDMA to 2.x ? > > Also, I am wondering if the development that you are doing will be > available in Mellanox UDA ? Or is it an independent project ? > > - milind > > > --- > Milind Bhandarkar > Chief Scientist > Pivotal > +1-650-523-3858 (W) > +1-408-666-8483 (C) > > > On Sat, Nov 9, 2013 at 5:44 AM, Xiaoyi [email protected] < > [email protected]> wrote: > >> The Hadoop-RDMA team is pleased to announce the release of Hadoop-RDMA >> 0.9.8 >> (technology preview) package with the following features and bug-fixes: >> >> * Hadoop-RDMA 0.9.8 Features >> >> - Based on Apache Hadoop 1.2.1 >> - High performance design with native InfiniBand and RoCE support >> at the verbs-level for HDFS, MapReduce, and RPC components >> - Compliant with Apache Hadoop 1.2.1 APIs and applications >> - Easily configurable for native InfiniBand, RoCE and the traditional >> sockets-based support (Ethernet and InfiniBand with IPoIB) >> - HDFS over native InfiniBand and RoCE >> - RDMA-based write >> - RDMA-based replication >> - Tuned packet size >> - MapReduce over native InfiniBand and RoCE >> - RDMA-based shuffle >> - Pre-fetching and caching of map outputs >> - In-memory merge >> - Overlapping of merge and reduce >> - RPC over native InfiniBand and RoCE >> - JVM-bypassed buffer management >> - On-demand InfiniBand connection setup >> - RDMA or send/recv based adaptive communication >> - Tested with >> - Mellanox InfiniBand adapters (DDR, QDR, and FDR) >> - RoCE >> - Various multi-core platforms >> - Different file systems with disks and SSDs >> >> * Bug Fixes (since 0.9.1) >> >> - MapReduce over InfiniBand >> - Reducing the waiting time for the pre-fetch cache read >> - Enabling Reduce tasks to randomize the shuffle pattern >> - HDFS over InfiniBand >> - Fix an issue in processing multiple blocks concurrently in a >> single task >> >> For downloading Hadoop-RDMA 0.9.8 package and the associated user guide, >> please visit the following URL: >> >> http://hadoop-rdma.cse.ohio-state.edu >> >> Sample performance numbers for benchmarks using Hadoop-RDMA 0.9.8 >> version can be viewed by visiting the `Performance' tab of the above >> website. >> >> All questions, feedbacks and bug reports are welcome. Please post it >> to the hadoop-rdma-discuss mailing list (hadoop-rdma-discuss at >> cse.ohio-state.edu). >> >> Thanks, >> >> The Hadoop-RDMA Team >>
