[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588916#comment-14588916
]
Hudson commented on HBASE-5356:
---
FAILURE: Integrated in HBase-0.94 #1471 (See
[https://builds.apache.org/job/HBase-0.94/1471/])
HBASE-12921 Port HBASE-5356 'region_mover.rb can hang if table region it
belongs to is deleted' to 0.94. (Liu Shaohui) (larsh: rev
4a1d46487cd83bb6f62b6367f688fbb44bbc6f82)
* bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14588920#comment-14588920
]
Hudson commented on HBASE-5356:
---
FAILURE: Integrated in HBase-0.94-JDK7 #239 (See
[https://builds.apache.org/job/HBase-0.94-JDK7/239/])
HBASE-12921 Port HBASE-5356 'region_mover.rb can hang if table region it
belongs to is deleted' to 0.94. (Liu Shaohui) (larsh: rev
4a1d46487cd83bb6f62b6367f688fbb44bbc6f82)
* bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14589018#comment-14589018
]
Hudson commented on HBASE-5356:
---
SUCCESS: Integrated in HBase-0.94-security #584 (See
[https://builds.apache.org/job/HBase-0.94-security/584/])
HBASE-12921 Port HBASE-5356 'region_mover.rb can hang if table region it
belongs to is deleted' to 0.94. (Liu Shaohui) (larsh: rev
4a1d46487cd83bb6f62b6367f688fbb44bbc6f82)
* bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894780#comment-13894780
]
Jimmy Xiang commented on HBASE-5356:
Integrated into trunk, 0.98, and 0.96. Thanks.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894968#comment-13894968
]
Hudson commented on HBASE-5356:
---
SUCCESS: Integrated in HBase-0.98 #139 (See
[https://builds.apache.org/job/HBase-0.98/139/])
HBASE-5356 region_mover.rb can hang if table region it belongs to is deleted
(jxiang: rev 1565743)
* /hbase/branches/0.98/bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894773#comment-13894773
]
Jimmy Xiang commented on HBASE-5356:
Good question. I put a break point before moving a region, then I tried to move
a region after the table was either disabled or deleted. The script works well
for these scenarios now.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895122#comment-13895122
]
Hudson commented on HBASE-5356:
---
SUCCESS: Integrated in hbase-0.96 #285 (See
[https://builds.apache.org/job/hbase-0.96/285/])
HBASE-5356 region_mover.rb can hang if table region it belongs to is deleted
(jxiang: rev 1565744)
* /hbase/branches/0.96/bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895175#comment-13895175
]
Hudson commented on HBASE-5356:
---
SUCCESS: Integrated in HBase-TRUNK #4899 (See
[https://builds.apache.org/job/HBase-TRUNK/4899/])
HBASE-5356 region_mover.rb can hang if table region it belongs to is deleted
(jxiang: rev 1565742)
* /hbase/trunk/bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895195#comment-13895195
]
Hudson commented on HBASE-5356:
---
SUCCESS: Integrated in HBase-0.98-on-Hadoop-1.1 #129 (See
[https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/129/])
HBASE-5356 region_mover.rb can hang if table region it belongs to is deleted
(jxiang: rev 1565743)
* /hbase/branches/0.98/bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895312#comment-13895312
]
Hudson commented on HBASE-5356:
---
FAILURE: Integrated in HBase-TRUNK-on-Hadoop-1.1 #83 (See
[https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-1.1/83/])
HBASE-5356 region_mover.rb can hang if table region it belongs to is deleted
(jxiang: rev 1565742)
* /hbase/trunk/bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13895324#comment-13895324
]
Hudson commented on HBASE-5356:
---
SUCCESS: Integrated in hbase-0.96-hadoop2 #197 (See
[https://builds.apache.org/job/hbase-0.96-hadoop2/197/])
HBASE-5356 region_mover.rb can hang if table region it belongs to is deleted
(jxiang: rev 1565744)
* /hbase/branches/0.96/bin/region_mover.rb
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Fix For: 0.98.0, 0.96.2, 0.99.0
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894064#comment-13894064
]
Jimmy Xiang commented on HBASE-5356:
Posted a patch that handled cases when a table is disabled/deleted in the
middle.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894179#comment-13894179
]
Hadoop QA commented on HBASE-5356:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12627521/hbase-5356.patch
against trunk revision .
ATTACHMENT ID: 12627521
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop
1.0 profile.
{color:green}+1 hadoop1.1{color}. The patch compiles against the hadoop
1.1 profile.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/8621//console
This message is automatically generated.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13894235#comment-13894235
]
stack commented on HBASE-5356:
--
lgtm Jimmy. How'd you test it?
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.92.0, 0.94.0
Reporter: Jonathan Hsieh
Assignee: Jimmy Xiang
Priority: Minor
Attachments: hbase-5356.patch
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:103)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204052#comment-13204052
]
Jonathan Hsieh commented on HBASE-5356:
---
Looks like we just need to properly catch TableNotFoundExceptions and continue.
Also we should probably abort if it gets another kind of exception that it
cannot handle.
Separate issue should probably update the graceful shutdown script so that it
fails fast on unexpected failures as well.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.94.0, 0.92.0
Reporter: Jonathan Hsieh
Priority: Minor
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204116#comment-13204116
]
Jonathan Hsieh commented on HBASE-5356:
---
Related issue -- if you create a new table and region mover had an old list,
new regions get assigned to the region we are trying to gracefully remove
regions from.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.94.0, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Minor
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at org.jruby.runtime.BlockBody.call(BlockBody.java:73)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:268)
at org.jruby.RubyProc.call(RubyProc.java:228)
at org.jruby.RubyProc$i$0$0$call.call(RubyProc$i$0$0$call.gen:65535)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:209)
at
org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:205)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
[jira] [Commented] (HBASE-5356) region_mover.rb can hang if table region it belongs to is deleted.
[
https://issues.apache.org/jira/browse/HBASE-5356?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13204137#comment-13204137
]
Jonathan Hsieh commented on HBASE-5356:
---
rephrase of previous comment: The region_mover script initially caches a list
of regions to move. If the table is deleted after the cached list is gathered
but before all regions are moved, the script can get stuck attempting to move a
deleted region.
In a related but likely separate issue -- if a new presplit table is created,
as the region_mover is emptying an RS, the emptying RS is a candidate for new
regions and will get some of the new regions. Ideally this is fenced off so
that it does not get regions assigned to it, but this would require some ZK.
This seems less important because the majority of regions will be moved off the
RS and the few new regions can rely on the automatically fail over to other
RS's.
region_mover.rb can hang if table region it belongs to is deleted.
--
Key: HBASE-5356
URL: https://issues.apache.org/jira/browse/HBASE-5356
Project: HBase
Issue Type: Bug
Affects Versions: 0.90.3, 0.94.0, 0.92.0
Reporter: Jonathan Hsieh
Assignee: Jonathan Hsieh
Priority: Minor
I was testing the region_mover.rb script on a loaded hbase and noticed that
it can hang (thus hanging graceful shutdown) if a region that it is
attempting to move gets deleted (by a table delete operation).
Here's the start of the relevent stack dump
{code}
12/02/08 13:27:13 WARN client.HConnectionManager$HConnectionImplementation:
Encountered problems when prefetch META table:
org.apache.hadoop.hbase.TableNotFoundException: Cannot find row in .META. for
table: TestLoadAndVerify_1328735001040, row=TestLoadAnd\
Verify_1328735001040,yC^P\xD7\x945\xD4,99
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:136)
at
org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:95)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:64\
9)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:703\
)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:594)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.relocateRegion(HConnectionManager.java:565)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionLocation(HConnectionManager.java:416)
at
org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
at
org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:63)
at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getRegionServerWithRetries(HConnectionManager.\
java:1018)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.nextScanner(HTable.java:1104)
at
org.apache.hadoop.hbase.client.HTable$ClientScanner.initialize(HTable.java:1027)
at org.apache.hadoop.hbase.client.HTable.getScanner(HTable.java:535)
at sun.reflect.GeneratedMethodAccessor24.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:525)
at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:380)
at
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:58)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:137)
at
usr.lib.hbase.bin.region_mover.method__7$RUBY$isSuccessfulScan(/usr/lib/hbase/bin/region_mover.rb:133)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
usr$lib$hbase$bin$region_mover#method__7$RUBY$isSuccessfulScan.call(usr$lib$hbase$bin$region_mover#method__7$RUBY$isSucces\
sfulScan:65535)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:171)
at
usr.lib.hbase.bin.region_mover.block_4$RUBY$__for__(/usr/lib/hbase/bin/region_mover.rb:326)
at
usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__.call(usr$lib$hbase$bin$region_mover#block_4$RUBY$__for__:65535)
at org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:133)
at
