[
https://issues.apache.org/jira/browse/HBASE-8912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13834617#comment-13834617
]
chenjiajun commented on HBASE-8912:
-----------------------------------
2013-11-27 18:26:18,102 FATAL org.apache.hadoop.hbase.master.HMaster: Master
server abort: loaded coprocessors are: []
2013-11-27 18:26:18,102 FATAL org.apache.hadoop.hbase.master.HMaster:
Unexpected state :
H,http://istock.jrj.com.cn/article,002024,6567377.html,1385541132079.18c9cb11b3e673dec07038f166fb3ef7.
state=PENDING_O
PEN, ts=1385547978102, server=d199.uuc.com,60020,1385047501649 .. Cannot
transit it to OFFLINE.
java.lang.IllegalStateException: Unexpected state :
H,http://istock.jrj.com.cn/article,002024,6567377.html,1385541132079.18c9cb11b3e673dec07038f166fb3ef7.
state=PENDING_OPEN, ts=1385547978102, server=d199.uuc.com,60020,1385047501649
.. Cannot transit it to OFFLINE.
at
org.apache.hadoop.hbase.master.AssignmentManager.setOfflineInZooKeeper(AssignmentManager.java:1890)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1690)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1426)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1398)
at
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1393)
at
org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:105)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2013-11-27 18:26:18,104 INFO org.apache.hadoop.hbase.master.HMaster: Aborting
2013-11-27 18:26:18,104 INFO org.apache.hadoop.ipc.HBaseServer: Stopping server
on 60000
2013-11-27 18:26:18,104 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server
handler 1 on 60000: exiting
......
RS 's log:
2013-11-27 18:24:33,375 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://master.uc.uuc.com:9000/hbase/H/18c9cb11b3e673dec07038f166fb3ef7/.tmp/832ec249071c45b3934a186046ca429d:
CompoundBloomFilterWriter
2013-11-27 18:24:33,385 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO
General Bloom and NO DeleteFamily was added to HFile
(hdfs://master.uc.uuc.com:9000/hbase/H/18c9cb11b3e673dec07038f166fb3ef7/.tmp/832ec249071c45b3934a186046ca429d)
2013-11-27 18:24:33,388 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at
hdfs://master.uc.uuc.com:9000/hbase/H/18c9cb11b3e673dec07038f166fb3ef7/.tmp/832ec249071c45b3934a186046ca429d
to
hdfs://master.uc.uuc.com:9000/hbase/H/18c9cb11b3e673dec07038f166fb3ef7/page/832ec249071c45b3934a186046ca429d
2013-11-27 18:24:33,421 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed compaction of 3 file(s) in page of
H,http://istock.jrj.com.cn/article,002024,6567377.html,1385541132079.18c9cb11b3e673dec07038f166fb3ef7.
into 832ec249071c45b3934a186046ca429d, size=85.7k; total size for store is
533.2m
2013-11-27 18:24:33,422 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: completed
compaction:
regionName=H,http://istock.jrj.com.cn/article,002024,6567377.html,1385541132079.18c9cb11b3e673dec07038f166fb3ef7.,
storeName=page, fileCount=3, fileSize=91.8k, priority=3,
time=1230511587222436; duration=0sec
2013-11-27 18:24:37,189 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer:
regionserver60020.periodicFlusher requesting flush for region
DocSortByHour,\x00\x00\x011\xAE\xCB\xD8
\x00\x00,B\x00\x00\x05\x84\x00\x01\xE6\xAF\x00\x02\x09\x0D\x00\x00\x00\x00\x06\xB5c\xB3,1314093149893.c5e2ce4602fb62714c72cd8da3b50bd5.
after a delay of 5137
2013-11-27 18:24:43,658 INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem
doesn't support getDefaultReplication
2013-11-27 18:24:43,658 INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem
doesn't support getDefaultBlockSize
2013-11-27 18:24:43,681 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/.tmp/dd7c3908111d4fcc8945b2b6d5291364:
CompoundBloomFilterWriter
2013-11-27 18:24:43,691 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO
General Bloom and NO DeleteFamily was added to HFile
(hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/.tmp/dd7c3908111d4fcc8945b2b6d5291364)
2013-11-27 18:24:43,691 INFO org.apache.hadoop.hbase.regionserver.Store:
Flushed , sequenceid=26736748922, memsize=832.0, into tmp file
hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/.tmp/dd7c3908111d4fcc8945b2b6d5291364
2013-11-27 18:24:43,699 INFO org.apache.hadoop.hbase.regionserver.Store: Added
hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/data/dd7c3908111d4fcc8945b2b6d5291364,
entries=4, sequenceid=26736748922, filesize=1000.0
2013-11-27 18:24:43,700 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Finished memstore flush of ~832.0/832, currentsize=0.0/0 for region
DocSortByHour,\x00\x00\x011\xAE\xCB\xD8
\x00\x00,B\x00\x00\x05\x84\x00\x01\xE6\xAF\x00\x02\x09\x0D\x00\x00\x00\x00\x06\xB5c\xB3,1314093149893.c5e2ce4602fb62714c72cd8da3b50bd5.
in 44ms, sequenceid=26736748922, compaction requested=true
2013-11-27 18:24:43,701 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Starting compaction on data in region DocSortByHour,\x00\x00\x011\xAE\xCB\xD8
\x00\x00,B\x00\x00\x05\x84\x00\x01\xE6\xAF\x00\x02\x09\x0D\x00\x00\x00\x00\x06\xB5c\xB3,1314093149893.c5e2ce4602fb62714c72cd8da3b50bd5.
2013-11-27 18:24:43,701 INFO org.apache.hadoop.hbase.regionserver.Store:
Starting compaction of 3 file(s) in data of
DocSortByHour,\x00\x00\x011\xAE\xCB\xD8
\x00\x00,B\x00\x00\x05\x84\x00\x01\xE6\xAF\x00\x02\x09\x0D\x00\x00\x00\x00\x06\xB5c\xB3,1314093149893.c5e2ce4602fb62714c72cd8da3b50bd5.
into
tmpdir=hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/.tmp,
seqid=26736748922, totalSize=47.9m
2013-11-27 18:24:43,755 INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem
doesn't support getDefaultReplication
2013-11-27 18:24:43,756 INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem
doesn't support getDefaultBlockSize
2013-11-27 18:24:43,757 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/.tmp/c2047a8a560645af936bcbcfe2625e5e:
CompoundBloomFilterWriter
2013-11-27 18:24:47,354 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO
General Bloom and DeleteFamily was added to HFile
(hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/.tmp/c2047a8a560645af936bcbcfe2625e5e)
2013-11-27 18:24:47,359 INFO
org.apache.hadoop.hbase.regionserver.StoreFile$Reader: Loaded Delete Family
Bloom (CompoundBloomFilter) metadata for c2047a8a560645af936bcbcfe2625e5e
2013-11-27 18:24:47,359 INFO org.apache.hadoop.hbase.regionserver.Store:
Renaming compacted file at
hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/.tmp/c2047a8a560645af936bcbcfe2625e5e
to
hdfs://master.uc.uuc.com:9000/hbase/DocSortByHour/c5e2ce4602fb62714c72cd8da3b50bd5/data/c2047a8a560645af936bcbcfe2625e5e
2013-11-27 18:24:47,364 INFO
org.apache.hadoop.hbase.regionserver.StoreFile$Reader: Loaded Delete Family
Bloom (CompoundBloomFilter) metadata for c2047a8a560645af936bcbcfe2625e5e
2013-11-27 18:24:47,404 INFO org.apache.hadoop.hbase.regionserver.Store:
Completed compaction of 3 file(s) in data of
DocSortByHour,\x00\x00\x011\xAE\xCB\xD8
\x00\x00,B\x00\x00\x05\x84\x00\x01\xE6\xAF\x00\x02\x09\x0D\x00\x00\x00\x00\x06\xB5c\xB3,1314093149893.c5e2ce4602fb62714c72cd8da3b50bd5.
into c2047a8a560645af936bcbcfe2625e5e, size=47.9m; total size for store is
223.8m
2013-11-27 18:24:47,404 INFO
org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest: completed
compaction: regionName=DocSortByHour,\x00\x00\x011\xAE\xCB\xD8
\x00\x00,B\x00\x00\x05\x84\x00\x01\xE6\xAF\x00\x02\x09\x0D\x00\x00\x00\x00\x06\xB5c\xB3,1314093149893.c5e2ce4602fb62714c72cd8da3b50bd5.,
storeName=data, fileCount=3, fileSize=47.9m, priority=3,
time=1230521954697044; duration=3sec
2013-11-27 18:27:07,191 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer:
regionserver60020.periodicFlusher requesting flush for region
h,!\xACU\xD2,1375542268855.ae4d6fae898f79e9fa019b78c975dd95. after a delay of
10647
2013-11-27 18:27:17,190 INFO
org.apache.hadoop.hbase.regionserver.HRegionServer:
regionserver60020.periodicFlusher requesting flush for region
h,!\xACU\xD2,1375542268855.ae4d6fae898f79e9fa019b78c975dd95. after a delay of
19328
2013-11-27 18:27:17,841 INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem
doesn't support getDefaultReplication
2013-11-27 18:27:17,841 INFO org.apache.hadoop.hbase.util.FSUtils: FileSystem
doesn't support getDefaultBlockSize
2013-11-27 18:27:17,851 INFO org.apache.hadoop.hbase.regionserver.StoreFile:
Delete Family Bloom filter type for
hdfs://master.uc.uuc.com:9000/hbase/h/ae4d6fae898f79e9fa019b78c975dd95/.tmp/135b6223aa9a41c1bf3511c7a389baea:
CompoundBloomFilterWriter
2013-11-27 18:27:17,863 INFO org.apache.hadoop.hbase.regionserver.StoreFile: NO
General Bloom and NO DeleteFamily was added to HFile
(hdfs://master.uc.uuc.com:9000/hbase/h/ae4d6fae898f79e9fa019b78c975dd95/.tmp/135b6223aa9a41c1bf3511c7a389baea)
------
and the Master worked well If I used version 0.94.3 ship with RS used version
0.94.13 .
> [0.94] AssignmentManager throws IllegalStateException from PENDING_OPEN to
> OFFLINE
> ----------------------------------------------------------------------------------
>
> Key: HBASE-8912
> URL: https://issues.apache.org/jira/browse/HBASE-8912
> Project: HBase
> Issue Type: Bug
> Reporter: Enis Soztutar
> Fix For: 0.94.15
>
> Attachments: HBase-0.94 #1036 test - testRetrying [Jenkins].html
>
>
> AM throws this exception which subsequently causes the master to abort:
> {code}
> java.lang.IllegalStateException: Unexpected state :
> testRetrying,jjj,1372891751115.9b828792311001062a5ff4b1038fe33b.
> state=PENDING_OPEN, ts=1372891751912,
> server=hemera.apache.org,39064,1372891746132 .. Cannot transit it to OFFLINE.
> at
> org.apache.hadoop.hbase.master.AssignmentManager.setOfflineInZooKeeper(AssignmentManager.java:1879)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1688)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1424)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1399)
> at
> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1394)
> at
> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:105)
> at
> org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:175)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
> at java.lang.Thread.run(Thread.java:662)
> {code}
> This exception trace is from the failing test TestMetaReaderEditor which is
> failing pretty frequently, but looking at the test code, I think this is not
> a test-only issue, but affects the main code path.
> https://builds.apache.org/job/HBase-0.94/1036/testReport/junit/org.apache.hadoop.hbase.catalog/TestMetaReaderEditor/testRetrying/
--
This message was sent by Atlassian JIRA
(v6.1#6144)
