No exceptions before that, sending few lines before them in the log file. Also there was a similar retry error in master log also just before this. Attached both master and rs log
Thanks, Murali Krishna ________________________________ From: Jean-Daniel Cryans <[email protected]> To: [email protected] Sent: Thu, 3 March, 2011 8:33:48 PM Subject: Re: Log rolling failed with ioe Anything that comes before that? More exceptions? Can you see what's at the beginning of them? I'm more interested in the symptoms rather than the side effects. Thx, J-D On Thu, Mar 3, 2011 at 5:49 AM, Murali Krishna. P <[email protected]> wrote: > Hi, > I get following error frequently (twice in a day atleast) and the region server > closes and exits after this. I am on 0.20.6 and we are stuck with this for some > time. Will be upgrading to 0.90.0 later, but right now, would be great if we >can > have a workaround. Let me know if you need more info. > > > WARNING: DFSClient_1483413998 could not complete file > /user/x/hbase/.logs/host.com,60020,1299145240135/hlog.dat.1299152151 > 119. Retrying... > Mar 3, 2011 3:53:25 AM org.apache.hadoop.hbase.regionserver.LogRoller run > SEVERE: Log rolling failed with ioe: > java.io.IOException: DFSClient_1483413998 could not complete file > /user/x/hbase/.logs/host,60020,1299145240135/hlog.da > t.1299152151119. Giving up. > at >org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.closeInternal(DFSClient.java:3331) >) > > at > org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.close(DFSClient.java:3240) > at >org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:61) >) > > at > org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:86) > at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:959) > at > org.apache.hadoop.hbase.regionserver.HLog.cleanupCurrentWriter(HLog.java:491) > at org.apache.hadoop.hbase.regionserver.HLog.rollWriter(HLog.java:342) > at org.apache.hadoop.hbase.regionserver.LogRoller.run(LogRoller.java:80) > Mar 3, 2011 3:53:25 AM org.apache.hadoop.hbase.regionserver.HLog optionalSync > > Thanks, > Murali Krishna >
