I don't get it, why would HBase need to reopen the hfile? Has it gone bad? If it has, why would reopening help?
Or are you talking about resetting the state in the DFSClient by forcing the end-client to know more about the namenode/client internals? This doesn't sound right... the DFSClient abstraction should leak this badly... On Tue, Jan 12, 2010 at 3:11 PM, Dhruba Borthakur <dhr...@gmail.com> wrote: > Here is a JIRA that I opened after some offline discussion with Joydeep. > > http://issues.apache.org/jira/browse/HDFS-826 > > This will allow a client (e.g. Region Server) to periodically query the dfs > file handle to retrieve the number of currently active replicas. if the > number of replicas falls below 3, then the region server could close the > Hfile and open a new hfile. what do people think about that approach? > > thanks, > dhruba > > > On Tue, Jan 12, 2010 at 12:28 PM, stack <st...@duboce.net> wrote: > >> (Andrew, see hdfs-127 on its tail for update) >> >> Zlatin: >> >> IMO, its better keeping hdfs-127. You actually can't run an hbase cluster >> of any size without it. The rate at which files go bad -- dfsclient turns >> into a zombie after accumulating 3 errors after any 3 transient failures -- >> does the cluster a mortal blow. The scenario where all replicas go away is >> relatively rare and probably indicative of a more serious hdfs issue >> though, >> according to the issue, when it happens, its ugly with dfsclient going into >> an infinite loop. >> >> St.Ack >> >> >> On Tue, Jan 12, 2010 at 12:05 PM, Andrew Purtell <apurt...@apache.org >> >wrote: >> >> > Hi, >> > >> > > Out of curiosity, since HDFS-127 was recently deemed harmful and >> > > reverted, will 0.20.3 still ship with patched version of Hadoop? >> > >> > Do you have a reference for that? >> > >> > > Do you still recommend that patch be applied when running Hbase? >> > >> > We needed that patch so long running HBase processes could ride over >> > intermittent DFS problems. Otherwise, the stream would become >> > unusable as soon as the errors accumulate, like the issue discusses. >> > >> > It may still be an important patch for HBase even though there is >> > some problem with it otherwise. >> > >> > - Andy >> > >> > >> > >> > ----- Original Message ---- >> > > From: "zlatin.balev...@barclayscapital.com" < >> > zlatin.balev...@barclayscapital.com> >> > > To: hbase-dev@hadoop.apache.org >> > > Sent: Tue, January 12, 2010 11:34:49 AM >> > > Subject: RE: Vote on hbase-0.20.3RC1 >> > > >> > > Out of curiosity, since HDFS-127 was recently deemed harmful and >> > > reverted, will 0.20.3 still ship with patched version of Hadoop? >> > > >> > > Do you still recommend that patch be applied when running Hbase? >> > > >> > > Zlatin >> > > _______________________________________________ >> > > >> > > This e-mail may contain information that is confidential, privileged or >> > > otherwise protected from disclosure. If you are not an intended >> recipient >> > of >> > > this e-mail, do not duplicate or redistribute it by any means. Please >> > delete it >> > > and any attachments and notify the sender that you have received it in >> > error. >> > > Unless specifically indicated, this e-mail is not an offer to buy or >> sell >> > or a >> > > solicitation to buy or sell any securities, investment products or >> other >> > > financial product or service, an official confirmation of any >> > transaction, or an >> > > official statement of Barclays. Any views or opinions presented are >> > solely those >> > > of the author and do not necessarily represent those of Barclays. This >> > e-mail is >> > > subject to terms available at the following link: >> > > www.barcap.com/emaildisclaimer. By messaging with Barclays you consent >> > to the >> > > foregoing. Barclays Capital is the investment banking division of >> > Barclays Bank >> > > PLC, a company registered in England (number 1026167) with its >> registered >> > office >> > > at 1 Churchill Place, London, E14 5HP. This email may relate to or be >> > sent from >> > > other members of the Barclays Group. >> > > _______________________________________________ >> > >> > >> > >> > >> > >> > >> > > > > -- > Connect to me at http://www.facebook.com/dhruba >
