Is this so we can avoid having the DFSClient fall into an infinite loop in the case where there are no more replicas, the reason for backing out hdfs-127 from 0.20 branch?
If we have to, we can call a new API which tells us when we dip below wanted replicas and close the file (if we can per Hairong's comment) and reopen a new. Any danger of running out of replicas in between calls to the new method? (But I like Ryan's comment up in the issue best, that DFSClient just handles it for us) St.Ack On Tue, Jan 12, 2010 at 3:11 PM, Dhruba Borthakur <dhr...@gmail.com> wrote: > Here is a JIRA that I opened after some offline discussion with Joydeep. > > http://issues.apache.org/jira/browse/HDFS-826 > > This will allow a client (e.g. Region Server) to periodically query the > dfs > file handle to retrieve the number of currently active replicas. if the > number of replicas falls below 3, then the region server could close the > Hfile and open a new hfile. what do people think about that approach? > > thanks, > dhruba > > > On Tue, Jan 12, 2010 at 12:28 PM, stack <st...@duboce.net> wrote: > > > (Andrew, see hdfs-127 on its tail for update) > > > > Zlatin: > > > > IMO, its better keeping hdfs-127. You actually can't run an hbase > cluster > > of any size without it. The rate at which files go bad -- dfsclient > turns > > into a zombie after accumulating 3 errors after any 3 transient failures > -- > > does the cluster a mortal blow. The scenario where all replicas go away > is > > relatively rare and probably indicative of a more serious hdfs issue > > though, > > according to the issue, when it happens, its ugly with dfsclient going > into > > an infinite loop. > > > > St.Ack > > > > > > On Tue, Jan 12, 2010 at 12:05 PM, Andrew Purtell <apurt...@apache.org > > >wrote: > > > > > Hi, > > > > > > > Out of curiosity, since HDFS-127 was recently deemed harmful and > > > > reverted, will 0.20.3 still ship with patched version of Hadoop? > > > > > > Do you have a reference for that? > > > > > > > Do you still recommend that patch be applied when running Hbase? > > > > > > We needed that patch so long running HBase processes could ride over > > > intermittent DFS problems. Otherwise, the stream would become > > > unusable as soon as the errors accumulate, like the issue discusses. > > > > > > It may still be an important patch for HBase even though there is > > > some problem with it otherwise. > > > > > > - Andy > > > > > > > > > > > > ----- Original Message ---- > > > > From: "zlatin.balev...@barclayscapital.com" < > > > zlatin.balev...@barclayscapital.com> > > > > To: hbase-dev@hadoop.apache.org > > > > Sent: Tue, January 12, 2010 11:34:49 AM > > > > Subject: RE: Vote on hbase-0.20.3RC1 > > > > > > > > Out of curiosity, since HDFS-127 was recently deemed harmful and > > > > reverted, will 0.20.3 still ship with patched version of Hadoop? > > > > > > > > Do you still recommend that patch be applied when running Hbase? > > > > > > > > Zlatin > > > > _______________________________________________ > > > > > > > > This e-mail may contain information that is confidential, privileged > or > > > > otherwise protected from disclosure. If you are not an intended > > recipient > > > of > > > > this e-mail, do not duplicate or redistribute it by any means. Please > > > delete it > > > > and any attachments and notify the sender that you have received it > in > > > error. > > > > Unless specifically indicated, this e-mail is not an offer to buy or > > sell > > > or a > > > > solicitation to buy or sell any securities, investment products or > > other > > > > financial product or service, an official confirmation of any > > > transaction, or an > > > > official statement of Barclays. Any views or opinions presented are > > > solely those > > > > of the author and do not necessarily represent those of Barclays. > This > > > e-mail is > > > > subject to terms available at the following link: > > > > www.barcap.com/emaildisclaimer. By messaging with Barclays you > consent > > > to the > > > > foregoing. Barclays Capital is the investment banking division of > > > Barclays Bank > > > > PLC, a company registered in England (number 1026167) with its > > registered > > > office > > > > at 1 Churchill Place, London, E14 5HP. This email may relate to or > be > > > sent from > > > > other members of the Barclays Group. > > > > _______________________________________________ > > > > > > > > > > > > > > > > > > > > > > > > -- > Connect to me at http://www.facebook.com/dhruba >
