Thanks Dhruba! Can I try using it? Is it open for use?
-Ajit. On Tue, Sep 20, 2011 at 2:48 PM, Dhruba Borthakur <dhr...@gmail.com> wrote: > Hi andy, > > we do run a version of HDFS RAID that is backported from Apache trunk to a > 0.20 based release. Our code is in > https://github.com/facebook/hadoop-20-warehouse/tree/master/src/contrib/raid > But I do not have an elegant way to contribute this code to Apache > 0.20.2xx.x. > > thanks, > dhruba > > > On Sat, Sep 17, 2011 at 9:16 AM, Andrew Purtell <apurt...@apache.org>wrote: > >> Hi Dhruba, >> >> Would you consider a contribution of this to branch-0.20-security aka >> 0.20.2xx.x? >> >> If I am mistaken and you do not have a 0.22-ish HDFS RAID backported to an >> 0.20-ish platform, please disregard. >> >> Best regards, >> >> - Andy >> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >> (via Tom White) >> >> ------------------------------ >> *From:* Dhruba Borthakur <dhr...@gmail.com> >> *To:* hdfs-user@hadoop.apache.org; Andrew Purtell <apurt...@apache.org> >> *Sent:* Thursday, September 15, 2011 10:14 AM >> >> *Subject:* Re: Need help regarding HDFS-RAID >> >> That's right Andy. 0.22+. We are running a HDFS-RAID code base that is >> pretty close to what is available in Apache hdfs trunk. >> >> -dhruba >> >> On Thu, Sep 15, 2011 at 10:08 AM, Andrew Purtell <apurt...@apache.org>wrote: >> >> But that is the HDFS RAID effectively in 0.22+, not 0.21, right Dhruba? >> >> Best regards, >> >> - Andy >> >> Problems worthy of attack prove their worth by hitting back. - Piet Hein >> (via Tom White) >> >> ------------------------------ >> *From:* Dhruba Borthakur <dhr...@gmail.com> >> *To:* hdfs-user@hadoop.apache.org >> *Sent:* Thursday, September 15, 2011 10:06 AM >> *Subject:* Re: Need help regarding HDFS-RAID >> >> We use HDFS RAID in a big way. Data older than 12 days are RAIDED using >> XOR encoding (effective replication of 2.5). Data older than a few months >> are raided using ReedSolomon (effective observed replication factor of 1.5). >> This is running on our 60 PB size cluster for about an year now. >> >> thanks >> dhruba >> >> On Thu, Sep 15, 2011 at 5:31 AM, Ajit Ratnaparkhi < >> ajit.ratnapar...@gmail.com> wrote: >> >> Hi, >> >> We were planning to use it for past data archival(instead of moving it to >> archival store). >> Archiving it in HDFS gives advantage of making it easily available for >> processing whenever required. >> >> Is there any archival solution in hadoop ecosystem? >> >> thanks, >> Ajit. >> >> >> On Thu, Sep 15, 2011 at 5:05 PM, Harsh J <ha...@cloudera.com> wrote: >> >> Hey Ajit, >> >> HDFS-RAID was never part of the 0.20 release. It made its debut in the >> 0.21 release [1]. I know that Facebook uses it (and also did develop >> it), but unsure of users beyond Facebook. >> >> While 0.21 overall is not entirely deemed as production-usable yet >> (and is in fact, possibly abandoned for efforts on 0.22+), you can >> give that release a whirl on a test cluster and see for yourself if >> your need beats the stability. >> >> Just curious though - why are you looking to use this specifically? >> >> [1] - >> http://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.21/mapreduce/src/contrib/raid/ >> >> On Thu, Sep 15, 2011 at 4:37 PM, Ajit Ratnaparkhi >> <ajit.ratnapar...@gmail.com> wrote: >> > Hi, >> > We want to use HDFS-RAID in our production cluster. >> > (http://wiki.apache.org/hadoop/HDFS-RAID) >> > I am not able to find source/binaries/configs for this in official >> hadoop >> > distribution from apache hadoop. (checked in 0.20.1 and 0.20.2). >> > Can somebody please tell me where can I find that? and installation >> > procedure? >> > Also, is HDFS-RAID implementation stable enough to use in production? >> > thanks, >> > Ajit. >> > >> >> >> >> -- >> Harsh J >> >> >> >> >> >> -- >> Connect to me at http://www.facebook.com/dhruba >> >> >> >> >> >> -- >> Connect to me at http://www.facebook.com/dhruba >> >> >> > > > -- > Connect to me at http://www.facebook.com/dhruba >
