hellon， hbase groups， hadppy to write the message for you .
Today,My Hbase cluster the logs error as same as your
https://issues.apache.org/jira/browse/HBASE-12063 privode about :
wal.ProtobufLogWriter: Got IOException while writing trailer
more logs message you can see mine attachment name by logs.txt. I'm really
worried 。Hope receice you answer .
Here is my logs message:
2014-10-31 22:33:52,197 DEBUG [regionserver60020-WAL.AsyncNotifier] wal.FSHLog:
regionserver60020-WAL.AsyncNotifier interrupted while waiting for notification
from AsyncSyncer thread
2014-10-31 22:33:52,197 INFO [regionserver60020-WAL.AsyncNotifier] wal.FSHLog:
regionserver60020-WAL.AsyncNotifier exiting
2014-10-31 22:33:52,197 DEBUG [regionserver60020-WAL.AsyncSyncer0] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer0 interrupted while waiting for notification
from AsyncWriter thread
By guilin.zhang
2,wgdata_jingdong.product_info:1169421832:2014-10-23,1414615885356.374c13df97145a9e89eb21f0bf2f76e3.
2014-10-31 22:33:50,789 DEBUG [RS_OPEN_REGION-datanode33:60020-0]
zookeeper.ZKAssign: regionserver:60020-0x5493865d4f4016a,
quorum=datanode33.shadoop.co
m:2181,datanode32.shadoop.com:2181,namenode31.shadoop.com:2181,namenode30.shadoop.com:2181,datanode34.shadoop.com:2181,
baseZNode=/hbase Transitioning 3
74c13df97145a9e89eb21f0bf2f76e3 from RS_ZK_REGION_OPENING to RS_ZK_REGION_OPENED
2014-10-31 22:33:50,792 DEBUG [RS_OPEN_REGION-datanode33:60020-0]
zookeeper.ZKAssign: regionserver:60020-0x5493865d4f4016a,
quorum=datanode33.shadoop.co
m:2181,datanode32.shadoop.com:2181,namenode31.shadoop.com:2181,namenode30.shadoop.com:2181,datanode34.shadoop.com:2181,
baseZNode=/hbase Transitioned no
de 374c13df97145a9e89eb21f0bf2f76e3 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_OPENED
2014-10-31 22:33:50,792 DEBUG [RS_OPEN_REGION-datanode33:60020-0]
handler.OpenRegionHandler: Transitioned 374c13df97145a9e89eb21f0bf2f76e3 to
OPENED in
zk on datanode33.shadoop.com,60020,1414766007046
2014-10-31 22:33:50,792 DEBUG [RS_OPEN_REGION-datanode33:60020-0]
handler.OpenRegionHandler: Opened
test2,wgdata_jingdong.product_info:1169421832:2014-1
0-23,1414615885356.374c13df97145a9e89eb21f0bf2f76e3. on
datanode33.shadoop.com,60020,1414766007046
2014-10-31 22:33:50,809 INFO [regionserver60020-EventThread]
replication.ReplicationTrackerZKImpl:
/hbase/rs/datanode41.shadoop.com,60020,1414766007342
znode expired, triggering replicatorRemoved event
2014-10-31 22:33:50,825 DEBUG [regionserver60020-EventThread]
regionserver.SplitLogWorker: tasks arrived or departed
2014-10-31 22:33:50,882 DEBUG [regionserver60020-EventThread]
regionserver.SplitLogWorker: tasks arrived or departed
2014-10-31 22:33:51,084 INFO [regionserver60020-EventThread]
replication.ReplicationTrackerZKImpl:
/hbase/rs/datanode45.shadoop.com,60020,1414766009327
znode expired, triggering replicatorRemoved event
2014-10-31 22:33:51,102 DEBUG [regionserver60020-EventThread]
regionserver.SplitLogWorker: tasks arrived or departed
2014-10-31 22:33:51,158 DEBUG [regionserver60020-EventThread]
regionserver.SplitLogWorker: tasks arrived or departed
2014-10-31 22:33:51,170 INFO [PriorityRpcServer.handler=3,queue=0,port=60020]
regionserver.HRegionServer: Open SYSTEM.CATALOG,,1414743520507.8c82c108a0
2f9cf385788430199b4e07.
2014-10-31 22:33:51,180 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
zookeeper.ZKAssign: regionserver:60020-0x5493865d4f4016a,
quorum=datanode33.shadoop.co
m:2181,datanode32.shadoop.com:2181,namenode31.shadoop.com:2181,namenode30.shadoop.com:2181,datanode34.shadoop.com:2181,
baseZNode=/hbase Transitioning 8
c82c108a02f9cf385788430199b4e07 from M_ZK_REGION_OFFLINE to RS_ZK_REGION_OPENING
2014-10-31 22:33:51,186 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
zookeeper.ZKAssign: regionserver:60020-0x5493865d4f4016a,
quorum=datanode33.shadoop.co
m:2181,datanode32.shadoop.com:2181,namenode31.shadoop.com:2181,namenode30.shadoop.com:2181,datanode34.shadoop.com:2181,
baseZNode=/hbase Transitioned no
de 8c82c108a02f9cf385788430199b4e07 from M_ZK_REGION_OFFLINE to
RS_ZK_REGION_OPENING
2014-10-31 22:33:51,187 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegion: Opening region: {ENCODED =>
8c82c108a02f9cf385788430199b4e07, NA
ME => 'SYSTEM.CATALOG,,1414743520507.8c82c108a02f9cf385788430199b4e07.',
STARTKEY => '', ENDKEY => ''}
2014-10-31 22:33:51,197 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: Loading coprocessor class
org.apache.phoenix.coprocessor.
MetaDataRegionObserver with path null and priority 2
2014-10-31 22:33:51,198 ERROR [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: The coprocessor
org.apache.phoenix.coprocessor.MetaDataRe
gionObserver threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.MetaDataRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,201 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: ABORTING region server
datanode33.shadoop.com,60020,141476
6007046: The coprocessor org.apache.phoenix.coprocessor.MetaDataRegionObserver
threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.MetaDataRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,202 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2014-10-31 22:33:51,216 INFO [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: STOPPED: The coprocessor
org.apache.phoenix.coprocessor.Me
taDataRegionObserver threw an unexpected exception
2014-10-31 22:33:51,216 INFO [regionserver60020] ipc.RpcServer: Stopping
server on 60020
2014-10-31 22:33:51,217 INFO [RpcServer.listener,port=60020] ipc.RpcServer:
RpcServer.listener,port=60020: stopping
2014-10-31 22:33:51,217 INFO [RpcServer.responder] ipc.RpcServer:
RpcServer.responder: stopped
2014-10-31 22:33:51,221 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: Loading coprocessor class
org.apache.phoenix.coprocessor.
MetaDataEndpointImpl with path null and priority 1
2014-10-31 22:33:51,217 INFO [regionserver60020] regionserver.SplitLogWorker:
Sending interrupt to stop the worker thread
2014-10-31 22:33:51,418 INFO [regionserver60020] regionserver.HRegionServer:
Stopping infoServer
2014-10-31 22:33:51,396 INFO [RpcServer.responder] ipc.RpcServer:
RpcServer.responder: stopping
2014-10-31 22:33:51,418 ERROR [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: The coprocessor
org.apache.phoenix.coprocessor.MetaDataEn
dpointImpl threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.MetaDataEndpointImpl
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,631 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: ABORTING region server
datanode33.shadoop.com,60020,141476
6007046: The coprocessor org.apache.phoenix.coprocessor.MetaDataEndpointImpl
threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.MetaDataEndpointImpl
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,418 INFO
[SplitLogWorker-datanode33.shadoop.com,60020,1414766007046]
regionserver.SplitLogWorker: SplitLogWorker interrupted while
waiting for task, exiting: java.lang.InterruptedException
2014-10-31 22:33:51,784 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2014-10-31 22:33:51,784 INFO
[SplitLogWorker-datanode33.shadoop.com,60020,1414766007046]
regionserver.SplitLogWorker: SplitLogWorker datanode33.shadoop
.com,60020,1414766007046 exiting
2014-10-31 22:33:51,785 INFO [regionserver60020] mortbay.log: Stopped
[email protected]:60030
2014-10-31 22:33:51,789 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: Loading coprocessor class
org.apache.phoenix.coprocessor.
ServerCachingEndpointImpl with path null and priority 1
2014-10-31 22:33:51,791 ERROR [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: The coprocessor
org.apache.phoenix.coprocessor.ServerCach
ingEndpointImpl threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.ServerCachingEndpointImpl
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,791 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: ABORTING region server
datanode33.shadoop.com,60020,141476
6007046: The coprocessor
org.apache.phoenix.coprocessor.ServerCachingEndpointImpl threw an unexpected
exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.ServerCachingEndpointImpl
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,792 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2014-10-31 22:33:51,796 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: Loading coprocessor class
org.apache.phoenix.coprocessor.
GroupedAggregateRegionObserver with path null and priority 1
2014-10-31 22:33:51,797 ERROR [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: The coprocessor
org.apache.phoenix.coprocessor.GroupedAgg
regateRegionObserver threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,797 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: ABORTING region server
datanode33.shadoop.com,60020,141476
6007046: The coprocessor
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver threw an
unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.GroupedAggregateRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,798 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2014-10-31 22:33:51,802 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: Loading coprocessor class
org.apache.phoenix.coprocessor.
UngroupedAggregateRegionObserver with path null and priority 1
2014-10-31 22:33:51,803 ERROR [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: The coprocessor
org.apache.phoenix.coprocessor.UngroupedA
ggregateRegionObserver threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,804 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: ABORTING region server
datanode33.shadoop.com,60020,141476
6007046: The coprocessor
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver threw an
unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,804 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2014-10-31 22:33:51,809 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: Loading coprocessor class
org.apache.phoenix.coprocessor.
ScanRegionObserver with path null and priority 1
2014-10-31 22:33:51,809 ERROR [RS_OPEN_REGION-datanode33:60020-2]
coprocessor.CoprocessorHost: The coprocessor
org.apache.phoenix.coprocessor.ScanRegion
Observer threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.ScanRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,810 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: ABORTING region server
datanode33.shadoop.com,60020,141476
6007046: The coprocessor org.apache.phoenix.coprocessor.ScanRegionObserver
threw an unexpected exception
java.io.IOException: No jar path specified for
org.apache.phoenix.coprocessor.ScanRegionObserver
at
org.apache.hadoop.hbase.coprocessor.CoprocessorHost.load(CoprocessorHost.java:200)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.loadTableCoprocessors(RegionCoprocessorHost.java:207)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.<init>(RegionCoprocessorHost.java:163)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:622)
at org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:529)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4208)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4519)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4492)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4448)
at
org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4399)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:465)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:139)
at
org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2014-10-31 22:33:51,811 FATAL [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegionServer: RegionServer abort: loaded coprocessors are: []
2014-10-31 22:33:51,812 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
regionserver.MetricsRegionSourceImpl: Creating new MetricsRegionSourceImpl for
table S
YSTEM.CATALOG 8c82c108a02f9cf385788430199b4e07
2014-10-31 22:33:51,813 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegion: Instantiated
SYSTEM.CATALOG,,1414743520507.8c82c108a02f9cf385788
430199b4e07.
2014-10-31 22:33:51,818 INFO [StoreOpener-8c82c108a02f9cf385788430199b4e07-1]
compactions.CompactionConfiguration: size [134217728, 9223372036854775807
); files [3, 10); ratio 1.200000; off-peak ratio 5.000000; throttle point
2684354560; delete expired; major period 604800000, major jitter 0.500000
2014-10-31 22:33:51,823 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegion: Found 0 recovered edits file(s) under
hdfs://myhadoop/hbase/data
/default/SYSTEM.CATALOG/8c82c108a02f9cf385788430199b4e07
2014-10-31 22:33:51,825 INFO [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegion: Onlined 8c82c108a02f9cf385788430199b4e07; next
sequenceid=1
2014-10-31 22:33:51,825 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
zookeeper.ZKAssign: regionserver:60020-0x5493865d4f4016a,
quorum=datanode33.shadoop.co
m:2181,datanode32.shadoop.com:2181,namenode31.shadoop.com:2181,namenode30.shadoop.com:2181,datanode34.shadoop.com:2181,
baseZNode=/hbase Attempting to r
etransition opening state of node 8c82c108a02f9cf385788430199b4e07
2014-10-31 22:33:51,827 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegion: Closing
SYSTEM.CATALOG,,1414743520507.8c82c108a02f9cf38578843019
9b4e07.: disabling compactions & flushes
2014-10-31 22:33:51,827 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegion: Updates disabled for region
SYSTEM.CATALOG,,1414743520507.8c82c1
08a02f9cf385788430199b4e07.
2014-10-31 22:33:51,828 INFO
[StoreCloserThread-SYSTEM.CATALOG,,1414743520507.8c82c108a02f9cf385788430199b4e07.-1]
regionserver.HStore: Closed 0
2014-10-31 22:33:51,830 INFO [RS_OPEN_REGION-datanode33:60020-2]
regionserver.HRegion: Closed
SYSTEM.CATALOG,,1414743520507.8c82c108a02f9cf385788430199
b4e07.
2014-10-31 22:33:51,830 INFO [RS_OPEN_REGION-datanode33:60020-2]
handler.OpenRegionHandler: Opening of region {ENCODED =>
8c82c108a02f9cf385788430199b4
e07, NAME => 'SYSTEM.CATALOG,,1414743520507.8c82c108a02f9cf385788430199b4e07.',
STARTKEY => '', ENDKEY => ''} failed, transitioning from OPENING to FAIL
ED_OPEN in ZK, expecting version 1
2014-10-31 22:33:51,830 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
zookeeper.ZKAssign: regionserver:60020-0x5493865d4f4016a,
quorum=datanode33.shadoop.co
m:2181,datanode32.shadoop.com:2181,namenode31.shadoop.com:2181,namenode30.shadoop.com:2181,datanode34.shadoop.com:2181,
baseZNode=/hbase Transitioning 8
c82c108a02f9cf385788430199b4e07 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_FAILED_OPEN
2014-10-31 22:33:51,841 DEBUG [RS_OPEN_REGION-datanode33:60020-2]
zookeeper.ZKAssign: regionserver:60020-0x5493865d4f4016a,
quorum=datanode33.shadoop.co
m:2181,datanode32.shadoop.com:2181,namenode31.shadoop.com:2181,namenode30.shadoop.com:2181,datanode34.shadoop.com:2181,
baseZNode=/hbase Transitioned no
de 8c82c108a02f9cf385788430199b4e07 from RS_ZK_REGION_OPENING to
RS_ZK_REGION_FAILED_OPEN
2014-10-31 22:33:51,887 INFO [regionserver60020]
snapshot.RegionServerSnapshotManager: Stopping RegionServerSnapshotManager
abruptly.
2014-10-31 22:33:51,887 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher:
MemStoreFlusher.0 exiting
2014-10-31 22:33:51,887 INFO [regionserver60020.compactionChecker]
regionserver.HRegionServer$CompactionChecker:
regionserver60020.compactionChecker ex
iting
2014-10-31 22:33:51,887 INFO [regionserver60020.nonceCleaner]
regionserver.ServerNonceManager$1: regionserver60020.nonceCleaner exiting
2014-10-31 22:33:51,887 INFO [MemStoreFlusher.1] regionserver.MemStoreFlusher:
MemStoreFlusher.1 exiting
2014-10-31 22:33:51,887 INFO [regionserver60020.logRoller]
regionserver.LogRoller: LogRoller exiting.
2014-10-31 22:33:51,889 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
handler.CloseRegionHandler: Processing close of
test2,wgdata_jingdong.product_info:11
69421832:2014-10-23,1414615885356.374c13df97145a9e89eb21f0bf2f76e3.
2014-10-31 22:33:51,890 DEBUG [RS_CLOSE_REGION-datanode33:60020-1]
handler.CloseRegionHandler: Processing close of
test2,wgdata_amazon.sort_product_list
:B00OCKSE66:2014-10-16,1414618497171.ddf80e37847ce808a5b19e1a8339288b.
2014-10-31 22:33:51,891 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
handler.CloseRegionHandler: Processing close of
test2,wgdata_jingdong.sort_product_li
st:1296217225:2014-10-18,1414614046578.bb2c7118c10ac3ec23476a9fc2e9742b.
2014-10-31 22:33:51,891 INFO [regionserver60020] regionserver.HRegionServer:
aborting server datanode33.shadoop.com,60020,1414766007046
2014-10-31 22:33:51,891 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
regionserver.HRegion: Closing
test2,wgdata_jingdong.product_info:1169421832:2014-10-2
3,1414615885356.374c13df97145a9e89eb21f0bf2f76e3.: disabling compactions &
flushes
2014-10-31 22:33:51,891 DEBUG [RS_CLOSE_REGION-datanode33:60020-1]
regionserver.HRegion: Closing
test2,wgdata_amazon.sort_product_list:B00OCKSE66:2014-1
0-16,1414618497171.ddf80e37847ce808a5b19e1a8339288b.: disabling compactions &
flushes
2014-10-31 22:33:51,891 DEBUG [regionserver60020] catalog.CatalogTracker:
Stopping catalog tracker org.apache.hadoop.hbase.catalog.CatalogTracker@6d8b74
5f
2014-10-31 22:33:51,892 DEBUG [RS_CLOSE_REGION-datanode33:60020-1]
regionserver.HRegion: Updates disabled for region
test2,wgdata_amazon.sort_product_li
st:B00OCKSE66:2014-10-16,1414618497171.ddf80e37847ce808a5b19e1a8339288b.
2014-10-31 22:33:51,892 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
regionserver.HRegion: Closing
test2,wgdata_jingdong.sort_product_list:1296217225:2014
-10-18,1414614046578.bb2c7118c10ac3ec23476a9fc2e9742b.: disabling compactions &
flushes
2014-10-31 22:33:51,892 INFO [regionserver60020]
client.HConnectionManager$HConnectionImplementation: Closing zookeeper
sessionid=0x24938657b810191
2014-10-31 22:33:51,891 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
regionserver.HRegion: Updates disabled for region
test2,wgdata_jingdong.product_info:
1169421832:2014-10-23,1414615885356.374c13df97145a9e89eb21f0bf2f76e3.
2014-10-31 22:33:51,892 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
regionserver.HRegion: Updates disabled for region
test2,wgdata_jingdong.sort_product_
list:1296217225:2014-10-18,1414614046578.bb2c7118c10ac3ec23476a9fc2e9742b.
2014-10-31 22:33:51,894 INFO [regionserver60020] zookeeper.ZooKeeper: Session:
0x24938657b810191 closed
2014-10-31 22:33:51,992 INFO [regionserver60020-EventThread]
zookeeper.ClientCnxn: EventThread shut down
2014-10-31 22:33:51,996 INFO [regionserver60020] regionserver.HRegionServer:
Waiting on 5 regions to close
2014-10-31 22:33:51,996 DEBUG [regionserver60020] regionserver.HRegionServer:
{374c13df97145a9e89eb21f0bf2f76e3=test2,wgdata_jingdong.product_info:11694
21832:2014-10-23,1414615885356.374c13df97145a9e89eb21f0bf2f76e3.,
ddf80e37847ce808a5b19e1a8339288b=test2,wgdata_amazon.sort_product_list:B00OCKSE66:2014
-10-16,1414618497171.ddf80e37847ce808a5b19e1a8339288b.,
bb2c7118c10ac3ec23476a9fc2e9742b=test2,wgdata_jingdong.sort_product_list:1296217225:2014-10-18,1
414614046578.bb2c7118c10ac3ec23476a9fc2e9742b.,
6d2021fc1cdfbd1a9081b27d0eb66ccb=test2,wgdata_tmall.product_info:15646793963:2014-10-26,1414590389813.6d
2021fc1cdfbd1a9081b27d0eb66ccb.,
cc8dea54f876071db04340ade5b5a2ca=test2,wgdata_tmall.product_info:40248082087:2014-10-23,1414588652332.cc8dea54f876071db
04340ade5b5a2ca.}
2014-10-31 22:33:52,060 INFO
[StoreCloserThread-test2,wgdata_jingdong.sort_product_list:1296217225:2014-10-18,1414614046578.bb2c7118c10ac3ec23476a9fc2e
9742b.-1] regionserver.HStore: Closed in
2014-10-31 22:33:52,060 INFO
[StoreCloserThread-test2,wgdata_jingdong.product_info:1169421832:2014-10-23,1414615885356.374c13df97145a9e89eb21f0bf2f76e3
.-1] regionserver.HStore: Closed in
2014-10-31 22:33:52,060 INFO
[StoreCloserThread-test2,wgdata_amazon.sort_product_list:B00OCKSE66:2014-10-16,1414618497171.ddf80e37847ce808a5b19e1a83392
88b.-1] regionserver.HStore: Closed in
2014-10-31 22:33:52,061 INFO [RS_CLOSE_REGION-datanode33:60020-2]
regionserver.HRegion: Closed
test2,wgdata_jingdong.sort_product_list:1296217225:2014-
10-18,1414614046578.bb2c7118c10ac3ec23476a9fc2e9742b.
2014-10-31 22:33:52,062 INFO [RS_CLOSE_REGION-datanode33:60020-0]
regionserver.HRegion: Closed
test2,wgdata_jingdong.product_info:1169421832:2014-10-23
,1414615885356.374c13df97145a9e89eb21f0bf2f76e3.
2014-10-31 22:33:52,063 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
handler.CloseRegionHandler: Closed
test2,wgdata_jingdong.sort_product_list:1296217225
:2014-10-18,1414614046578.bb2c7118c10ac3ec23476a9fc2e9742b.
2014-10-31 22:33:52,063 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
handler.CloseRegionHandler: Closed
test2,wgdata_jingdong.product_info:1169421832:2014
-10-23,1414615885356.374c13df97145a9e89eb21f0bf2f76e3.
2014-10-31 22:33:52,063 INFO [RS_CLOSE_REGION-datanode33:60020-1]
regionserver.HRegion: Closed
test2,wgdata_amazon.sort_product_list:B00OCKSE66:2014-10
-16,1414618497171.ddf80e37847ce808a5b19e1a8339288b.
2014-10-31 22:33:52,063 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
handler.CloseRegionHandler: Processing close of
test2,wgdata_tmall.product_info:40248
082087:2014-10-23,1414588652332.cc8dea54f876071db04340ade5b5a2ca.
2014-10-31 22:33:52,063 DEBUG [RS_CLOSE_REGION-datanode33:60020-1]
handler.CloseRegionHandler: Closed
test2,wgdata_amazon.sort_product_list:B00OCKSE66:2
014-10-16,1414618497171.ddf80e37847ce808a5b19e1a8339288b.
2014-10-31 22:33:52,063 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
handler.CloseRegionHandler: Processing close of
test2,wgdata_tmall.product_info:15646
793963:2014-10-26,1414590389813.6d2021fc1cdfbd1a9081b27d0eb66ccb.
2014-10-31 22:33:52,063 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
regionserver.HRegion: Closing
test2,wgdata_tmall.product_info:40248082087:2014-10-23,
1414588652332.cc8dea54f876071db04340ade5b5a2ca.: disabling compactions & flushes
2014-10-31 22:33:52,063 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
regionserver.HRegion: Updates disabled for region
test2,wgdata_tmall.product_info:402
48082087:2014-10-23,1414588652332.cc8dea54f876071db04340ade5b5a2ca.
2014-10-31 22:33:52,064 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
regionserver.HRegion: Closing
test2,wgdata_tmall.product_info:15646793963:2014-10-26,
1414590389813.6d2021fc1cdfbd1a9081b27d0eb66ccb.: disabling compactions & flushes
2014-10-31 22:33:52,064 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
regionserver.HRegion: Updates disabled for region
test2,wgdata_tmall.product_info:156
46793963:2014-10-26,1414590389813.6d2021fc1cdfbd1a9081b27d0eb66ccb.
2014-10-31 22:33:52,066 INFO
[StoreCloserThread-test2,wgdata_tmall.product_info:40248082087:2014-10-23,1414588652332.cc8dea54f876071db04340ade5b5a2ca.-
1] regionserver.HStore: Closed in
2014-10-31 22:33:52,066 INFO
[StoreCloserThread-test2,wgdata_tmall.product_info:15646793963:2014-10-26,1414590389813.6d2021fc1cdfbd1a9081b27d0eb66ccb.-
1] regionserver.HStore: Closed in
2014-10-31 22:33:52,066 INFO [RS_CLOSE_REGION-datanode33:60020-0]
regionserver.HRegion: Closed
test2,wgdata_tmall.product_info:40248082087:2014-10-23,1
414588652332.cc8dea54f876071db04340ade5b5a2ca.
2014-10-31 22:33:52,066 INFO [RS_CLOSE_REGION-datanode33:60020-2]
regionserver.HRegion: Closed
test2,wgdata_tmall.product_info:15646793963:2014-10-26,1
414590389813.6d2021fc1cdfbd1a9081b27d0eb66ccb.
2014-10-31 22:33:52,067 DEBUG [RS_CLOSE_REGION-datanode33:60020-0]
handler.CloseRegionHandler: Closed
test2,wgdata_tmall.product_info:40248082087:2014-1
0-23,1414588652332.cc8dea54f876071db04340ade5b5a2ca.
2014-10-31 22:33:52,067 DEBUG [RS_CLOSE_REGION-datanode33:60020-2]
handler.CloseRegionHandler: Closed
test2,wgdata_tmall.product_info:15646793963:2014-1
0-26,1414590389813.6d2021fc1cdfbd1a9081b27d0eb66ccb.
2014-10-31 22:33:52,196 INFO [regionserver60020] regionserver.HRegionServer:
stopping server datanode33.shadoop.com,60020,1414766007046; all regions cl
osed.
2014-10-31 22:33:52,197 DEBUG [regionserver60020-WAL.AsyncNotifier] wal.FSHLog:
regionserver60020-WAL.AsyncNotifier interrupted while waiting for notif
ication from AsyncSyncer thread
2014-10-31 22:33:52,197 INFO [regionserver60020-WAL.AsyncNotifier] wal.FSHLog:
regionserver60020-WAL.AsyncNotifier exiting
2014-10-31 22:33:52,197 DEBUG [regionserver60020-WAL.AsyncSyncer0] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer0 interrupted while waiting for notifica
tion from AsyncWriter thread
2014-10-31 22:33:52,198 INFO [regionserver60020-WAL.AsyncSyncer0] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer0 exiting
2014-10-31 22:33:52,198 DEBUG [regionserver60020-WAL.AsyncSyncer1] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer1 interrupted while waiting for notifica
tion from AsyncWriter thread
2014-10-31 22:33:52,198 INFO [regionserver60020-WAL.AsyncSyncer1] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer1 exiting
2014-10-31 22:33:52,198 DEBUG [regionserver60020-WAL.AsyncSyncer2] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer2 interrupted while waiting for notifica
tion from AsyncWriter thread
2014-10-31 22:33:52,198 INFO [regionserver60020-WAL.AsyncSyncer2] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer2 exiting
2014-10-31 22:33:52,199 DEBUG [regionserver60020-WAL.AsyncSyncer3] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer3 interrupted while waiting for notifica
tion from AsyncWriter thread
2014-10-31 22:33:52,199 INFO [regionserver60020-WAL.AsyncSyncer3] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer3 exiting
2014-10-31 22:33:52,199 DEBUG [regionserver60020-WAL.AsyncSyncer4] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer4 interrupted while waiting for notifica
tion from AsyncWriter thread
2014-10-31 22:33:52,199 INFO [regionserver60020-WAL.AsyncSyncer4] wal.FSHLog:
regionserver60020-WAL.AsyncSyncer4 exiting
2014-10-31 22:33:52,199 DEBUG [regionserver60020-WAL.AsyncWriter] wal.FSHLog:
regionserver60020-WAL.AsyncWriter interrupted while waiting for newer writ
es added to local buffer
2014-10-31 22:33:52,199 INFO [regionserver60020-WAL.AsyncWriter] wal.FSHLog:
regionserver60020-WAL.AsyncWriter exiting
2014-10-31 22:33:52,200 DEBUG [regionserver60020] wal.FSHLog: Closing WAL
writer in hdfs://myhadoop/hbase/WALs/datanode33.shadoop.com,60020,141476600704
6
2014-10-31 22:33:52,313 INFO [regionserver60020] regionserver.Leases:
regionserver60020 closing leases
2014-10-31 22:33:52,313 INFO [regionserver60020] regionserver.Leases:
regionserver60020 closed leases
2014-10-31 22:33:52,315 DEBUG [regionserver60020-EventThread]
regionserver.SplitLogWorker: tasks arrived or departed
2014-10-31 22:33:52,449 INFO [regionserver60020.periodicFlusher]
regionserver.HRegionServer$PeriodicMemstoreFlusher:
regionserver60020.periodicFlusher
exiting
2014-10-31 22:33:52,449 INFO [regionserver60020]
regionserver.CompactSplitThread: Waiting for Split Thread to finish...
2014-10-31 22:33:52,449 INFO [regionserver60020]
regionserver.CompactSplitThread: Waiting for Merge Thread to finish...
2014-10-31 22:33:52,449 INFO [regionserver60020]
regionserver.CompactSplitThread: Waiting for Large Compaction Thread to
finish...
2014-10-31 22:33:52,449 INFO [regionserver60020]
regionserver.CompactSplitThread: Waiting for Small Compaction Thread to
finish...
2014-10-31 22:33:52,450 INFO [regionserver60020.leaseChecker]
regionserver.Leases: regionserver60020.leaseChecker closing leases
2014-10-31 22:33:52,452 INFO [regionserver60020.leaseChecker]
regionserver.Leases: regionserver60020.leaseChecker closed leases
2014-10-31 22:33:52,455 INFO [regionserver60020]
client.HConnectionManager$HConnectionImplementation: Closing zookeeper
sessionid=0x4493865871c019c
2014-10-31 22:33:52,456 INFO [regionserver60020] zookeeper.ZooKeeper: Session:
0x4493865871c019c closed
2014-10-31 22:33:52,456 INFO [regionserver60020-EventThread]
zookeeper.ClientCnxn: EventThread shut down
2014-10-31 22:33:52,461 INFO [regionserver60020] zookeeper.ZooKeeper: Session:
0x5493865d4f4016a closed
2014-10-31 22:33:52,461 INFO [regionserver60020] regionserver.HRegionServer:
stopping server datanode33.shadoop.com,60020,1414766007046; zookeeper conn
ection closed.
2014-10-31 22:33:52,461 INFO [regionserver60020-EventThread]
zookeeper.ClientCnxn: EventThread shut down
2014-10-31 22:33:52,461 INFO [regionserver60020] regionserver.HRegionServer:
regionserver60020 exiting
2014-10-31 22:33:52,462 ERROR [main] regionserver.HRegionServerCommandLine:
Region server exiting
java.lang.RuntimeException: HRegionServer Aborted
at
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:66)
at
org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:85)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at
org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2421)
2014-10-31 22:33:52,464 INFO [Thread-9] regionserver.ShutdownHook: Shutdown
hook starting; hbase.shutdown.hook=true; fsShutdownHook=org.apache.hadoop.f
s.FileSystem$Cache$ClientFinalizer@6c565c85
2014-10-31 22:33:52,465 INFO [Thread-9] regionserver.ShutdownHook: Starting fs
shutdown hook thread.
2014-10-31 22:33:52,466 INFO [Thread-9] regionserver.ShutdownHook: Shutdown
hook finished.
