Davey Yan created HBASE-7139:
--------------------------------
Summary: RegionSplitter's rollingSplit cannot split empty table or
region
Key: HBASE-7139
URL: https://issues.apache.org/jira/browse/HBASE-7139
Project: HBase
Issue Type: Bug
Components: util
Affects Versions: 0.94.1
Environment: Ubuntu Server 10.04, Hadoop 1.0.3
Reporter: Davey Yan
Execute rollingSplit on empty table or region will cause original region in
SPLITTING state, and will be not out of transition list unless unassigning it
(with forse) in shell.
Steps to reproduce:
{code}
shell> create 'test2', 'i'
{code}
{noformat}
$ bin/hbase org.apache.hadoop.hbase.util.RegionSplitter -r -o 2 test2
HexStringSplit
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.3-1240972, built on 02/06/2012 10:48 GMT
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client environment:host.name=dev-vm0
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:java.version=1.6.0_29
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client environment:java.vendor=Sun
Microsystems Inc.
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:java.home=/usr/lib/jvm/jdk1.6.0_29/jre
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:java.class.path=/opt/hbase/bin/../conf:/usr/lib/jvm/default-java/lib/tools.jar:/opt/hbase/bin/..:/opt/hbase/bin/../hbase-0.94.1.jar:/opt/hbase/bin/../hbase-0.94.1-tests.jar:/opt/hbase/bin/../lib/activation-1.1.jar:/opt/hbase/bin/../lib/asm-3.1.jar:/opt/hbase/bin/../lib/avro-1.5.3.jar:/opt/hbase/bin/../lib/avro-ipc-1.5.3.jar:/opt/hbase/bin/../lib/commons-beanutils-1.7.0.jar:/opt/hbase/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/hbase/bin/../lib/commons-cli-1.2.jar:/opt/hbase/bin/../lib/commons-codec-1.4.jar:/opt/hbase/bin/../lib/commons-collections-3.2.1.jar:/opt/hbase/bin/../lib/commons-configuration-1.6.jar:/opt/hbase/bin/../lib/commons-digester-1.8.jar:/opt/hbase/bin/../lib/commons-el-1.0.jar:/opt/hbase/bin/../lib/commons-httpclient-3.1.jar:/opt/hbase/bin/../lib/commons-io-2.1.jar:/opt/hbase/bin/../lib/commons-lang-2.5.jar:/opt/hbase/bin/../lib/commons-logging-1.1.1.jar:/opt/hbase/bin/../lib/commons-math-2.1.jar:/opt/hbase/bin/../lib/commons-net-1.4.1.jar:/opt/hbase/bin/../lib/core-3.1.1.jar:/opt/hbase/bin/../lib/guava-11.0.2.jar:/opt/hbase/bin/../lib/hadoop-core-1.0.3.jar:/opt/hbase/bin/../lib/high-scale-lib-1.1.1.jar:/opt/hbase/bin/../lib/httpclient-4.1.2.jar:/opt/hbase/bin/../lib/httpcore-4.1.3.jar:/opt/hbase/bin/../lib/jackson-core-asl-1.8.8.jar:/opt/hbase/bin/../lib/jackson-jaxrs-1.8.8.jar:/opt/hbase/bin/../lib/jackson-mapper-asl-1.8.8.jar:/opt/hbase/bin/../lib/jackson-xc-1.8.8.jar:/opt/hbase/bin/../lib/jamon-runtime-2.3.1.jar:/opt/hbase/bin/../lib/jasper-compiler-5.5.23.jar:/opt/hbase/bin/../lib/jasper-runtime-5.5.23.jar:/opt/hbase/bin/../lib/jaxb-api-2.1.jar:/opt/hbase/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/hbase/bin/../lib/jersey-core-1.8.jar:/opt/hbase/bin/../lib/jersey-json-1.8.jar:/opt/hbase/bin/../lib/jersey-server-1.8.jar:/opt/hbase/bin/../lib/jettison-1.1.jar:/opt/hbase/bin/../lib/jetty-6.1.26.jar:/opt/hbase/bin/../lib/jetty-util-6.1.26.jar:/opt/hbase/bin/../lib/jruby-complete-1.6.5.jar:/opt/hbase/bin/../lib/jsp-2.1-6.1.14.jar:/opt/hbase/bin/../lib/jsp-api-2.1-6.1.14.jar:/opt/hbase/bin/../lib/jsr305-1.3.9.jar:/opt/hbase/bin/../lib/junit-4.10-HBASE-1.jar:/opt/hbase/bin/../lib/libthrift-0.8.0.jar:/opt/hbase/bin/../lib/log4j-1.2.16.jar:/opt/hbase/bin/../lib/metrics-core-2.1.2.jar:/opt/hbase/bin/../lib/netty-3.2.4.Final.jar:/opt/hbase/bin/../lib/protobuf-java-2.4.0a.jar:/opt/hbase/bin/../lib/servlet-api-2.5-6.1.14.jar:/opt/hbase/bin/../lib/slf4j-api-1.4.3.jar:/opt/hbase/bin/../lib/slf4j-log4j12-1.4.3.jar:/opt/hbase/bin/../lib/snappy-java-1.0.3.2.jar:/opt/hbase/bin/../lib/stax-api-1.0.1.jar:/opt/hbase/bin/../lib/velocity-1.7.jar:/opt/hbase/bin/../lib/xmlenc-0.52.jar:/opt/hbase/bin/../lib/zookeeper-3.4.3.jar:/opt/hbase/bin/../libextra/mybk-commons-cc.jar:/opt/hbase/bin/../libextra/hbase.jar:/opt/hbase/bin/../libextra/sfdcloud-hbase.jar:
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:java.library.path=/opt/hbase/bin/../lib/native/Linux-i386-32
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/tmp
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client environment:os.name=Linux
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client environment:os.arch=i386
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:os.version=2.6.32-33-generic
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client environment:user.name=pcer
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:user.home=/home/pcer
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Client
environment:user.dir=/opt/hbase-0.94.1
12/11/09 16:24:57 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=dev-vm0:2181 sessionTimeout=60000 watcher=hconnection
12/11/09 16:24:57 INFO zookeeper.ClientCnxn: Opening socket connection to
server /172.16.20.140:2181
12/11/09 16:24:57 INFO zookeeper.RecoverableZooKeeper: The identifier of this
process is 16033@dev-vm0
12/11/09 16:24:57 WARN client.ZooKeeperSaslClient: SecurityException:
java.lang.SecurityException: Unable to locate a login configuration occurred
when trying to find JAAS configuration.
12/11/09 16:24:57 INFO client.ZooKeeperSaslClient: Client will not
SASL-authenticate because the default JAAS configuration section 'Client' could
not be found. If you are not using SASL, you may ignore this. On the other
hand, if you expected SASL to work, please fix your JAAS configuration.
12/11/09 16:24:57 INFO zookeeper.ClientCnxn: Socket connection established to
dev-vm0/172.16.20.140:2181, initiating session
12/11/09 16:24:57 INFO zookeeper.ClientCnxn: Session establishment complete on
server dev-vm0/172.16.20.140:2181, sessionid = 0x13ad3e9ba7001c3, negotiated
timeout = 40000
12/11/09 16:24:58 DEBUG client.HConnectionManager$HConnectionImplementation:
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a;
serverName=dev-vm0,46871,1352175042380
12/11/09 16:24:58 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for .META.,,1.1028785192 is dev-vm0:46871
12/11/09 16:24:58 DEBUG client.MetaScanner: Scanning .META. starting at
row=test2,,00000000000000 for max=10 rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a
12/11/09 16:24:58 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5. is
dev-vm0:46871
12/11/09 16:24:59 DEBUG util.RegionSplitter: No _balancedSplit file.
Calculating splits...
12/11/09 16:24:59 DEBUG client.MetaScanner: Scanning .META. starting at
row=test2,,00000000000000 for max=2147483647 rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a
12/11/09 16:24:59 DEBUG util.RegionSplitter: Table test2 has 1 regions that
will be split.
12/11/09 16:24:59 DEBUG util.RegionSplitter: Will Split [00000000 , ffffffff)
at 7fffffff
12/11/09 16:24:59 DEBUG util.RegionSplitter: Bucketing regions by
regionserver...
12/11/09 16:24:59 DEBUG util.RegionSplitter: Done with bucketing. Split time!
12/11/09 16:24:59 DEBUG util.RegionSplitter: 1 RS have regions to splt.
12/11/09 16:24:59 DEBUG client.MetaScanner: Scanning .META. starting at
row=test2,,00000000000000 for max=2147483647 rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a
12/11/09 16:24:59 DEBUG util.RegionSplitter: Finding a region on dev-vm0:46871
12/11/09 16:24:59 DEBUG util.RegionSplitter: Splitting at 7fffffff
12/11/09 16:24:59 INFO zookeeper.ZooKeeper: Initiating client connection,
connectString=dev-vm0:2181 sessionTimeout=60000
watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a
12/11/09 16:24:59 INFO zookeeper.ClientCnxn: Opening socket connection to
server /172.16.20.140:2181
12/11/09 16:24:59 WARN client.ZooKeeperSaslClient: SecurityException:
java.lang.SecurityException: Unable to locate a login configuration occurred
when trying to find JAAS configuration.
12/11/09 16:24:59 INFO client.ZooKeeperSaslClient: Client will not
SASL-authenticate because the default JAAS configuration section 'Client' could
not be found. If you are not using SASL, you may ignore this. On the other
hand, if you expected SASL to work, please fix your JAAS configuration.
12/11/09 16:24:59 INFO zookeeper.ClientCnxn: Socket connection established to
dev-vm0/172.16.20.140:2181, initiating session
12/11/09 16:24:59 INFO zookeeper.RecoverableZooKeeper: The identifier of this
process is 16033@dev-vm0
12/11/09 16:24:59 DEBUG catalog.CatalogTracker: Starting catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@691dee
12/11/09 16:24:59 INFO zookeeper.ClientCnxn: Session establishment complete on
server dev-vm0/172.16.20.140:2181, sessionid = 0x13ad3e9ba7001c4, negotiated
timeout = 40000
12/11/09 16:24:59 DEBUG client.ClientScanner: Creating scanner over .META.
starting at key 'test2,,'
12/11/09 16:24:59 DEBUG client.ClientScanner: Advancing internal scanner to
startKey at 'test2,,'
12/11/09 16:24:59 DEBUG client.ClientScanner: Creating scanner over .META.
starting at key 'test2,,'
12/11/09 16:24:59 DEBUG client.ClientScanner: Advancing internal scanner to
startKey at 'test2,,'
12/11/09 16:24:59 DEBUG client.ClientScanner: Finished with scanning at {NAME
=> '.META.,,1', STARTKEY => '', ENDKEY => '', ENCODED => 1028785192,}
12/11/09 16:24:59 DEBUG catalog.CatalogTracker: Stopping catalog tracker
org.apache.hadoop.hbase.catalog.CatalogTracker@691dee
12/11/09 16:24:59 INFO zookeeper.ClientCnxn: EventThread shut down
12/11/09 16:24:59 INFO zookeeper.ZooKeeper: Session: 0x13ad3e9ba7001c4 closed
12/11/09 16:24:59 DEBUG client.HConnectionManager$HConnectionImplementation:
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a;
serverName=dev-vm0,46871,1352175042380
12/11/09 16:24:59 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for .META.,,1.1028785192 is dev-vm0:46871
12/11/09 16:24:59 DEBUG client.MetaScanner: Scanning .META. starting at
row=test2,,00000000000000 for max=10 rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a
12/11/09 16:24:59 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5. is
dev-vm0:46871
12/11/09 16:24:59 DEBUG util.RegionSplitter: Split Scan: 0 finished / 1 split
wait / 0 reference wait
12/11/09 16:25:29 DEBUG client.HConnectionManager$HConnectionImplementation:
Looked up root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a;
serverName=dev-vm0,46871,1352175042380
12/11/09 16:25:29 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for .META.,,1.1028785192 is dev-vm0:46871
12/11/09 16:25:29 DEBUG client.MetaScanner: Scanning .META. starting at
row=test2,,00000000000000 for max=10 rows using
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@ba5c7a
12/11/09 16:25:29 DEBUG client.HConnectionManager$HConnectionImplementation:
Cached location for test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5. is
dev-vm0:46871
12/11/09 16:25:29 DEBUG util.RegionSplitter: Split Scan: 0 finished / 1 split
wait / 0 reference wait
{noformat}
Log of server site:
{noformat}
2012-11-09 16:24:59,597 DEBUG
org.apache.hadoop.hbase.regionserver.CompactSplitThread: Split requested for
test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5..
compaction_queue=(0:0), split_queue=1
2012-11-09 16:24:59,597 INFO
org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of region
test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5.
2012-11-09 16:24:59,597 DEBUG
org.apache.hadoop.hbase.regionserver.SplitTransaction:
regionserver:46871-0x13ad3e9ba700001 Creating ephemeral node for
6c53236aff39bca3e77e79fcf2419ba5 in SPLITTING state
2012-11-09 16:24:59,601 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:46871-0x13ad3e9ba700001 Attempting to transition node
6c53236aff39bca3e77e79fcf2419ba5 from RS_ZK_REGION_SPLITTING to
RS_ZK_REGION_SPLITTING
2012-11-09 16:24:59,613 INFO org.apache.zookeeper.server.PrepRequestProcessor:
Processed session termination for sessionid: 0x13ad3e9ba7001c4
2012-11-09 16:24:59,615 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:46871-0x13ad3e9ba700001 Successfully transitioned node
6c53236aff39bca3e77e79fcf2419ba5 from RS_ZK_REGION_SPLITTING to
RS_ZK_REGION_SPLITTING
2012-11-09 16:24:59,618 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Closing test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5.: disabling
compactions & flushes
2012-11-09 16:24:59,618 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
Updates disabled for region
test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5.
2012-11-09 16:24:59,624 WARN org.apache.zookeeper.server.NIOServerCnxn: caught
end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid
0x13ad3e9ba7001c4, likely client has closed socket
at
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
at
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:224)
at java.lang.Thread.run(Thread.java:662)
2012-11-09 16:24:59,625 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed
socket connection for client /172.16.20.140:38233 which had sessionid
0x13ad3e9ba7001c4
2012-11-09 16:24:59,626 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
Handling transition=RS_ZK_REGION_SPLITTING, server=dev-vm0,46871,1352175042380,
region=6c53236aff39bca3e77e79fcf2419ba5
2012-11-09 16:24:59,627 DEBUG org.apache.hadoop.hbase.regionserver.Store:
closed i
2012-11-09 16:24:59,627 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Closed test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5.
2012-11-09 16:24:59,627 INFO org.apache.hadoop.hbase.regionserver.SplitRequest:
Running rollback/cleanup of failed split of
test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5.; null
java.lang.IllegalArgumentException
at
java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:589)
at
java.util.concurrent.ThreadPoolExecutor.<init>(ThreadPoolExecutor.java:514)
at java.util.concurrent.Executors.newFixedThreadPool(Executors.java:84)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.splitStoreFiles(SplitTransaction.java:592)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.createDaughters(SplitTransaction.java:290)
at
org.apache.hadoop.hbase.regionserver.SplitTransaction.execute(SplitTransaction.java:452)
at
org.apache.hadoop.hbase.regionserver.SplitRequest.run(SplitRequest.java:67)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
2012-11-09 16:24:59,655 INFO org.apache.hadoop.hbase.regionserver.Store: time
to purge deletes set to 0ms in store null
2012-11-09 16:24:59,661 INFO
org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old failed
split transaction detritus:
hdfs://dev-vm0:9000/hbase/test2/6c53236aff39bca3e77e79fcf2419ba5/.splits
2012-11-09 16:24:59,662 INFO org.apache.hadoop.hbase.regionserver.HRegion:
Onlined test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5.; next
sequenceid=1
2012-11-09 16:24:59,663 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:46871-0x13ad3e9ba700001 Deleting existing unassigned node for
6c53236aff39bca3e77e79fcf2419ba5 that is in expected state
RS_ZK_REGION_SPLITTING
2012-11-09 16:24:59,667 DEBUG org.apache.hadoop.hbase.master.AssignmentManager:
The znode of region test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5. has
been deleted.
2012-11-09 16:24:59,668 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:46871-0x13ad3e9ba700001 Successfully deleted unassigned node for
region 6c53236aff39bca3e77e79fcf2419ba5 in expected state RS_ZK_REGION_SPLITTING
2012-11-09 16:24:59,668 DEBUG org.apache.hadoop.hbase.zookeeper.ZKAssign:
regionserver:46871-0x13ad3e9ba700001 Deleting existing unassigned node for
6c53236aff39bca3e77e79fcf2419ba5 that is in expected state
RS_ZK_REGION_SPLITTING
2012-11-09 16:24:59,669 INFO org.apache.hadoop.hbase.regionserver.SplitRequest:
Successful rollback of failed split of
test2,,1352449471288.6c53236aff39bca3e77e79fcf2419ba5.
... org.apache.hadoop.hbase.master.AssignmentManager: Regions in transition
timed out
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
