+1, I've been involved starting from design and development of ErasureCoding. I think phase 1 of this development is ready to be merged to trunk. It had come a long way to the current state with significant effort of many Contributors and Reviewers for both design and code.
Thanks Everyone for the efforts. Regards, Vinay On Wed, Sep 23, 2015 at 10:53 PM, Jing Zhao <ji...@apache.org> wrote: > +1 > > I've been involved in both development and review on the branch, and I > believe it's now ready to get merged into trunk. Many thanks to all the > contributors and reviewers! > > Thanks, > -Jing > > On Tue, Sep 22, 2015 at 6:17 PM, Zheng, Kai <kai.zh...@intel.com> wrote: > > > Non-binding +1 > > > > According to our extensive performance tests, striping + ISA-L coder > based > > erasure coding not only can save storage, but also can increase the > > throughput of a client or a cluster. It will be a great addition to HDFS > > and its users. Based on the latest branch codes, we also observed it's > very > > reliable in the concurrent tests. We'll provide the perf test report > after > > it's sorted out and hope it helps. > > Thanks! > > > > Regards, > > Kai > > > > -----Original Message----- > > From: Gangumalla, Uma [mailto:uma.ganguma...@intel.com] > > Sent: Wednesday, September 23, 2015 8:50 AM > > To: hdfs-dev@hadoop.apache.org; common-...@hadoop.apache.org > > Subject: Re: [VOTE] Merge HDFS-7285 (erasure coding) branch to trunk > > > > +1 > > > > Great addition to HDFS. Thanks all contributors for the nice work. > > > > Regards, > > Uma > > > > On 9/22/15, 3:40 PM, "Zhe Zhang" <zhezh...@cloudera.com> wrote: > > > > >Hi, > > > > > >I'd like to propose a vote to merge the HDFS-7285 feature branch back > > >to trunk. Since November 2014 we have been designing and developing > > >this feature under the umbrella JIRAs HDFS-7285 and HADOOP-11264, and > > >have committed approximately 210 patches. > > > > > >The HDFS-7285 feature branch was created to support the first phase of > > >HDFS erasure coding (HDFS-EC). The objective of HDFS-EC is to > > >significantly reduce storage space usage in HDFS clusters. Instead of > > >always creating 3 replicas of each block with 200% storage space > > >overhead, HDFS-EC provides data durability through parity data blocks. > > >With most EC configurations, the storage overhead is no more than 50%. > > >Based on profiling results of production clusters, we decided to > > >support EC with the striped block layout in the first phase, so that > > >small files can be better handled. This means dividing each logical > > >HDFS file block into smaller units (striping cells) and spreading them > > >on a set of DataNodes in round-robin fashion. Parity cells are > > >generated for each stripe of original data cells. We have made changes > > >to NameNode, client, and DataNode to generalize the block concept and > > >handle the mapping between a logical file block and its internal > > >storage blocks. For further details please see the design doc on > > >HDFS-7285. > > >HADOOP-11264 focuses on providing flexible and high-performance codec > > >calculation support. > > > > > >The nightly Jenkins job of the branch has reported several successful > > >runs, and doesn't show new flaky tests compared with trunk. We have > > >posted several versions of the test plan including both unit testing > > >and cluster testing, and have executed most tests in the plan. The most > > >basic functionalities have been extensively tested and verified in > > >several real clusters with different hardware configurations; results > > >have been very stable. We have created follow-on tasks for more > > >advanced error handling and optimization under the umbrella HDFS-8031. > > >We also plan to implement or harden the integration of EC with existing > > >features such as WebHDFS, snapshot, append, truncate, hflush, hsync, > > >and so forth. > > > > > >Development of this feature has been a collaboration across many > > >companies and institutions. I'd like to thank J. Andreina, Takanobu > > >Asanuma, Vinayakumar B, Li Bo, Takuya Fukudome, Uma Maheswara Rao G, > > >Rui Li, Yi Liu, Colin McCabe, Xinwei Qin, Rakesh R, Gao Rui, Kai > > >Sasaki, Walter Su, Tsz Wo Nicholas Sze, Andrew Wang, Yong Zhang, Jing > > >Zhao, Hui Zheng and Kai Zheng for their code contributions and reviews. > > >Andrew and Kai Zheng also made fundamental contributions to the initial > > >design. Rui Li, Gao Rui, Kai Sasaki, Kai Zheng and many other > > >contributors have made great efforts in system testing. Many thanks go > > >to Weihua Jiang for proposing the JIRA, and ATM, Todd Lipcon, Silvius > > >Rus, Suresh, as well as many others for providing helpful feedbacks. > > > > > >Following the community convention, this vote will last for 7 days > > >(ending September 29th). Votes from Hadoop committers are binding but > > >non-binding votes are very welcome as well. And here's my non-binding > +1. > > > > > >Thanks, > > >--- > > >Zhe Zhang > > > > >
