[
https://issues.apache.org/jira/browse/HBASE-1894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12763285#action_12763285
]
elsif commented on HBASE-1894:
-------------------------------
I only see a single assignment for the region name in the master log:
hbase-root-master-hfs-030012.log:2009-10-07 11:37:07,784 INFO
org.apache.hadoop.hbase.master.RegionManager: Assigning region
fc_test...,1254561810331 to hfs-030015,60020,1254759830300
> Region server references deleted compaction log
> -----------------------------------------------
>
> Key: HBASE-1894
> URL: https://issues.apache.org/jira/browse/HBASE-1894
> Project: Hadoop HBase
> Issue Type: Bug
> Components: regionserver
> Affects Versions: 0.20.0
> Environment: HBase Version 0.20.0, r805538
> HBase Compiled Wed Aug 19 14:38:36 PDT 2009, root
> Hadoop Version 0.20.0-plus4681, r767961
> Hadoop Compiled Mon Jun 8 18:18:06 UTC 2009, stack
> Reporter: elsif
>
> The region server appears to reference blocks from deleted log files after
> compaction. Stopping the affected region server causes the region to be
> reassigned and appears to function properly again.
> We have seen two instances this week where this caused the region server to
> get stuck with the following error:
> hbase-root-regionserver-hfs-030015.log:2009-10-07 12:24:41,853 INFO
> org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_6092375271544310747_250240 from any node: java.io.IOException: No live
> nodes contain current block
> .
> .
> .
> /opt/hbase/logs/hbase-root-regionserver-hfs-030015.log:java.io.IOException:
> Cannot open filename /hbase/fc_test/1209236691/json/9131415140626575165
> /opt/hbase/logs/hbase-root-regionserver-hfs-030015.log:2009-10-07
> 13:09:47,343 INFO org.apache.hadoop.hdfs.DFSClient: Could not obtain block
> blk_6092375271544310747_250240 from any node: java.io.IOException: No live
> nodes contain current block
> From the hdfs log we can see that the block in question was part of a
> compaction log:
> hadoop-root-namenode-hfs-030010.log:2009-10-07 08:48:31,271 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.allocateBlock:
> /hbase/fc_test/compaction.dir/1209236691/2099727166320402414.
> blk_6092375271544310747_250240
> hadoop-root-namenode-hfs-030010.log:2009-10-07 08:48:31,337 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock:
> blockMap updated: 10.4.30.39:50010 is added to blk_6092375271544310747_250240
> size 360813
> hadoop-root-namenode-hfs-030010.log:2009-10-07 08:48:31,338 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock:
> blockMap updated: 10.4.30.23:50010 is added to blk_6092375271544310747_250240
> size 360813
> hadoop-root-namenode-hfs-030010.log:2009-10-07 08:48:31,339 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK* NameSystem.addStoredBlock:
> blockMap updated: 10.4.30.16:50010 is added to blk_6092375271544310747_250240
> size 360813
> From the same log we see the block deleted:
> hadoop-root-namenode-hfs-030010.log:2009-10-07 12:13:20,738 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK* ask 10.4.30.39:50010 to delete
> blk_-8366798629992987785_250239 blk_-278212026264018289_224708
> blk_4131756856877489141_230749 blk_6092375271544310747_250240
> blk_-1303356519220271320_144130
> hadoop-root-namenode-hfs-030010.log:2009-10-07 12:13:26,739 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK* ask 10.4.30.23:50010 to delete
> blk_3405064566022477311_135928 blk_-7806500863137897827_139699
> blk_6092375271544310747_250240
> hadoop-root-namenode-hfs-030010.log:2009-10-07 12:13:29,739 INFO
> org.apache.hadoop.hdfs.StateChange: BLOCK* ask 10.4.30.16:50010 to delete
> blk_-3938002954120963098_139149 blk_2284074818158875873_144134
> blk_519099260905493191_222718 blk_7684159545746656462_144137
> blk_7857071994919747100_144127 blk_6311161030310597085_135919
> blk_-8366798629992987785_250239 blk_5638304438869005376_142347
> blk_-1019479064035180992_143997 blk_4131756856877489141_230749
> blk_6092375271544310747_250240 blk_-1520990752128999182_192595
> blk_1714900485284856973_230761 blk_-1303356519220271320_144130
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
