I'm looking into an issue with truncate too. This was a regression I introduced in HBASE-12219.
On Monday, November 3, 2014, Andrew Purtell (JIRA) <[email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > > [ > https://issues.apache.org/jira/browse/HBASE-12142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14195604#comment-14195604 > ] > > Andrew Purtell edited comment on HBASE-12142 at 11/4/14 3:01 AM: > ----------------------------------------------------------------- > > Truncation is not working correctly, at least when testing 0.98.8-SNAPSHOT > on a single master single slave cluster. > > To reproduce: > # ./bin/hbase ltt -init_only > # ./bin/hbase shell > > truncate 'cluster_test' > > TruncateTableHandler runs and claims to create table info, but it might > not be moving the file into the right place: > {noformat} > 2014-11-04 02:30:23,653 DEBUG [FifoRpcScheduler.handler1-thread-18] > handler.TruncateTableHandler: Table 'cluster_test' archived! > 2014-11-04 02:30:23,679 DEBUG [FifoRpcScheduler.handler1-thread-18] > util.FSTableDescriptors: Wrote descriptor into: > hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp/.tabledesc/.tableinfo.0000000001 > 2014-11-04 02:30:23,680 INFO [FifoRpcScheduler.handler1-thread-18] > handler.TruncateTableHandler: Truncate will not preserve the regions > 2014-11-04 02:30:23,681 INFO [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: creating HRegion cluster_test HTD == 'cluster_test', > {TABLE_ATTRIBUTES => {DURABILITY => 'USE_DEFAULT'}, {NAME => 'test_cf', > DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => > '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS > => '0', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => > 'false', BLOCKCACHE => 'true'} RootDir = > hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp Table name > == cluster_test > 2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Instantiated > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Closing > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.: disabling > compactions & flushes > 2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Updates disabled for region > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,703 INFO [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Closed > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,707 INFO [FifoRpcScheduler.handler1-thread-18] > catalog.MetaEditor: Added 1 > 2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] > master.AssignmentManager: Assigning 1 region(s) to > ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305 > 2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] > zookeeper.ZKAssign: master:8100-0x149780b006a010a, > quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase > Async create of unassigned node c23e76525f0f025d51f74006db7903e2 with > OFFLINE state > {noformat} > > But the region will fail to open > {noformat} > 2014-11-04 02:30:23,738 DEBUG [main-EventThread] > zookeeper.ZooKeeperWatcher: master:8100-0x149780b006a010a, > quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase > Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, > path=/hbase/region-in-transition/c23e76525f0f025d51f74006db7903e2 > 2014-11-04 02:30:23,738 DEBUG [AM.ZK.Worker-pool2-t14] > master.AssignmentManager: Handling RS_ZK_REGION_FAILED_OPEN, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305, > region=c23e76525f0f025d51f74006db7903e2, > current_state={c23e76525f0f025d51f74006db7903e2 state=OPENING, > ts=1415068223734, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305} > 2014-11-04 02:30:23,738 INFO [AM.ZK.Worker-pool2-t14] > master.RegionStates: Transition {c23e76525f0f025d51f74006db7903e2 > state=OPENING, ts=1415068223734, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305} to > {c23e76525f0f025d51f74006db7903e2 state=CLOSED, ts=1415068223738, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305} > 2014-11-04 02:30:23,738 WARN [AM.ZK.Worker-pool2-t14] > master.AssignmentManager: Can't move c23e76525f0f025d51f74006db7903e2, > there is no destination server available. > 2014-11-04 02:30:23,739 DEBUG [AM.ZK.Worker-pool2-t14] > handler.ClosedRegionHandler: Handling CLOSED event for > c23e76525f0f025d51f74006db7903e2 > {noformat} > > Because this is happening on the RegionServer side: > {noformat} > 2014-11-04 02:30:23,704 INFO > [PriorityRpcServer.handler=16,queue=0,port=8120] > regionserver.HRegionServer: Open > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,717 DEBUG > [PriorityRpcServer.handler=16,queue=0,port=8120] util.FSTableDescriptors: > Exception during readTableDecriptor. Current table name = cluster_test > org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor > file under > hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/data/default/cluster_test > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:510) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:487) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:173) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:3826) > at > org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:20164) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:110) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:90) > at java.lang.Thread.run(Thread.java:745) > 2014-11-04 02:30:23,719 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105, quorum= > ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase > Transitioning c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE to > RS_ZK_REGION_OPENING > 2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105, quorum= > ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase > Transitioned node c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE > to RS_ZK_REGION_OPENING > 2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > regionserver.HRegion: Opening region: {ENCODED => > c23e76525f0f025d51f74006db7903e2, NAME => > 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.', STARTKEY > => '', ENDKEY => ''} > 2014-11-04 02:30:23,723 ERROR [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > handler.OpenRegionHandler: Failed open of > region=cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2., > starting to roll back the global memstore size. > java.lang.IllegalStateException: Could not instantiate a region instance. > at > org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4266) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4574) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4547) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4503) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4454) > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:482) > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:145) > at > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4263) > ... 10 more > Caused by: java.lang.IllegalArgumentException: Need table descriptor > at > org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:556) > at > org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:532) > ... 15 more > 2014-11-04 02:30:23,724 INFO [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > handler.OpenRegionHandler: Opening of region {ENCODED => > c23e76525f0f025d51f74006db7903e2, NAME => > 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.', STARTKEY > => '', ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in > ZK, expecting version 1 > {noformat} > > > > was (Author: apurtell): > Truncation is not working correctly. > > To reproduce: > # ./bin/hbase ltt -init_only > # ./bin/hbase shell > > truncate 'cluster_test' > > TruncateTableHandler runs and claims to create table info, but it might > not be moving the file into the right place: > {noformat} > 2014-11-04 02:30:23,653 DEBUG [FifoRpcScheduler.handler1-thread-18] > handler.TruncateTableHandler: Table 'cluster_test' archived! > 2014-11-04 02:30:23,679 DEBUG [FifoRpcScheduler.handler1-thread-18] > util.FSTableDescriptors: Wrote descriptor into: > hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp/.tabledesc/.tableinfo.0000000001 > 2014-11-04 02:30:23,680 INFO [FifoRpcScheduler.handler1-thread-18] > handler.TruncateTableHandler: Truncate will not preserve the regions > 2014-11-04 02:30:23,681 INFO [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: creating HRegion cluster_test HTD == 'cluster_test', > {TABLE_ATTRIBUTES => {DURABILITY => 'USE_DEFAULT'}, {NAME => 'test_cf', > DATA_BLOCK_ENCODING => 'NONE', BLOOMFILTER => 'ROW', REPLICATION_SCOPE => > '0', COMPRESSION => 'NONE', VERSIONS => '1', TTL => 'FOREVER', MIN_VERSIONS > => '0', KEEP_DELETED_CELLS => 'false', BLOCKSIZE => '65536', IN_MEMORY => > 'false', BLOCKCACHE => 'true'} RootDir = > hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/.tmp Table name > == cluster_test > 2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Instantiated > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Closing > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.: disabling > compactions & flushes > 2014-11-04 02:30:23,703 DEBUG [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Updates disabled for region > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,703 INFO [RegionOpenAndInitThread-cluster_test-1] > regionserver.HRegion: Closed > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,707 INFO [FifoRpcScheduler.handler1-thread-18] > catalog.MetaEditor: Added 1 > 2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] > master.AssignmentManager: Assigning 1 region(s) to > ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305 > 2014-11-04 02:30:23,708 DEBUG [FifoRpcScheduler.handler1-thread-18] > zookeeper.ZKAssign: master:8100-0x149780b006a010a, > quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase > Async create of unassigned node c23e76525f0f025d51f74006db7903e2 with > OFFLINE state > {noformat} > > But the region will fail to open > {noformat} > 2014-11-04 02:30:23,738 DEBUG [main-EventThread] > zookeeper.ZooKeeperWatcher: master:8100-0x149780b006a010a, > quorum=ip-10-33-129-9.us-west-2.compute.internal:2181, baseZNode=/hbase > Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, > path=/hbase/region-in-transition/c23e76525f0f025d51f74006db7903e2 > 2014-11-04 02:30:23,738 DEBUG [AM.ZK.Worker-pool2-t14] > master.AssignmentManager: Handling RS_ZK_REGION_FAILED_OPEN, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305, > region=c23e76525f0f025d51f74006db7903e2, > current_state={c23e76525f0f025d51f74006db7903e2 state=OPENING, > ts=1415068223734, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305} > 2014-11-04 02:30:23,738 INFO [AM.ZK.Worker-pool2-t14] > master.RegionStates: Transition {c23e76525f0f025d51f74006db7903e2 > state=OPENING, ts=1415068223734, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305} to > {c23e76525f0f025d51f74006db7903e2 state=CLOSED, ts=1415068223738, > server=ip-10-204-3-67.us-west-2.compute.internal,8120,1415068107305} > 2014-11-04 02:30:23,738 WARN [AM.ZK.Worker-pool2-t14] > master.AssignmentManager: Can't move c23e76525f0f025d51f74006db7903e2, > there is no destination server available. > 2014-11-04 02:30:23,739 DEBUG [AM.ZK.Worker-pool2-t14] > handler.ClosedRegionHandler: Handling CLOSED event for > c23e76525f0f025d51f74006db7903e2 > {noformat} > > Because this is happening on the RegionServer side: > {noformat} > 2014-11-04 02:30:23,704 INFO > [PriorityRpcServer.handler=16,queue=0,port=8120] > regionserver.HRegionServer: Open > cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2. > 2014-11-04 02:30:23,717 DEBUG > [PriorityRpcServer.handler=16,queue=0,port=8120] util.FSTableDescriptors: > Exception during readTableDecriptor. Current table name = cluster_test > org.apache.hadoop.hbase.TableInfoMissingException: No table descriptor > file under > hdfs://ip-10-33-129-9.us-west-2.compute.internal:8020/hbase/data/default/cluster_test > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:510) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.getTableDescriptorFromFs(FSTableDescriptors.java:487) > at > org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:173) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:3826) > at > org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$2.callBlockingMethod(AdminProtos.java:20164) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2027) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108) > at > org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:110) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:90) > at java.lang.Thread.run(Thread.java:745) > 2014-11-04 02:30:23,719 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105, quorum= > ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase > Transitioning c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE to > RS_ZK_REGION_OPENING > 2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > zookeeper.ZKAssign: regionserver:8120-0x149780b006a0105, quorum= > ec2-54-71-67-141.us-west-2.compute.amazonaws.com:2181, baseZNode=/hbase > Transitioned node c23e76525f0f025d51f74006db7903e2 from M_ZK_REGION_OFFLINE > to RS_ZK_REGION_OPENING > 2014-11-04 02:30:23,723 DEBUG [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > regionserver.HRegion: Opening region: {ENCODED => > c23e76525f0f025d51f74006db7903e2, NAME => > 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.', STARTKEY > => '', ENDKEY => ''} > 2014-11-04 02:30:23,723 ERROR [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > handler.OpenRegionHandler: Failed open of > region=cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2., > starting to roll back the global memstore size. > java.lang.IllegalStateException: Could not instantiate a region instance. > at > org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4266) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4574) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4547) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4503) > at > org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:4454) > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:482) > at > org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:145) > at > org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.hbase.regionserver.HRegion.newHRegion(HRegion.java:4263) > ... 10 more > Caused by: java.lang.IllegalArgumentException: Need table descriptor > at > org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:556) > at > org.apache.hadoop.hbase.regionserver.HRegion.<init>(HRegion.java:532) > ... 15 more > 2014-11-04 02:30:23,724 INFO [RS_OPEN_REGION-ip-10-204-3-67:8120-2] > handler.OpenRegionHandler: Opening of region {ENCODED => > c23e76525f0f025d51f74006db7903e2, NAME => > 'cluster_test,,1415068223680.c23e76525f0f025d51f74006db7903e2.', STARTKEY > => '', ENDKEY => ''} failed, transitioning from OPENING to FAILED_OPEN in > ZK, expecting version 1 > {noformat} > > > > Truncate command does not preserve ACLs table > > --------------------------------------------- > > > > Key: HBASE-12142 > > URL: https://issues.apache.org/jira/browse/HBASE-12142 > > Project: HBase > > Issue Type: Bug > > Affects Versions: 0.98.6 > > Reporter: Vandana Ayyalasomayajula > > Assignee: Vandana Ayyalasomayajula > > Priority: Minor > > Labels: security > > Fix For: 2.0.0, 0.98.8, 0.99.2 > > > > Attachments: HBASE-12142_0.patch, HBASE-12142_1.patch, > HBASE-12142_2.patch, HBASE-12142_98.patch, HBASE-12142_98_2.patch, > HBASE-12142_branch_1.patch, HBASE-12142_branch_1.patch, > HBASE-12142_master_addendum.patch > > > > > > The current truncate command does not preserve acls on a table. > > > > -- > This message was sent by Atlassian JIRA > (v6.3.4#6332) > -- Sent from iPad
