[
https://issues.apache.org/jira/browse/HBASE-12377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189381#comment-14189381
]
Enis Soztutar commented on HBASE-12377:
---------------------------------------
This looks good. While we are at it, can we do fix the below code for doing a
custom master rpc as well. Smt like this:
{code}
- tableExists = false;
- GetTableDescriptorsResponse htds;
- MasterKeepAliveConnection master =
connection.getKeepAliveMasterService();
- try {
- GetTableDescriptorsRequest req =
- RequestConverter.buildGetTableDescriptorsRequest(tableName);
- htds = master.getTableDescriptors(null, req);
- } catch (ServiceException se) {
- throw ProtobufUtil.getRemoteException(se);
- } finally {
- master.close();
- }
- tableExists = !htds.getTableSchemaList().isEmpty();
- if (!tableExists) {
+ HTableDescriptor htd =
+ executeCallable(new
MasterCallable<HTableDescriptor>(getConnection()) {
+ @Override
+ public HTableDescriptor call() throws ServiceException {
+ GetTableDescriptorsRequest req =
+
RequestConverter.buildGetTableDescriptorsRequest(tableName);
+ GetTableDescriptorsResponse htds =
master.getTableDescriptors(null, req);
+ if (!htds.getTableSchemaList().isEmpty()) {
+ return
HTableDescriptor.convert(htds.getTableSchemaList().get(0));
+ }
+ return null;
+ }
+ });
+ if (htd == null) {
{code}
> HBaseAdmin#deleteTable fails when META region is moved around the same
> timeframe
> --------------------------------------------------------------------------------
>
> Key: HBASE-12377
> URL: https://issues.apache.org/jira/browse/HBASE-12377
> Project: HBase
> Issue Type: Bug
> Components: Client
> Affects Versions: 0.98.4
> Reporter: Stephen Yuan Jiang
> Assignee: Stephen Yuan Jiang
> Fix For: 2.0.0, 0.98.8, 0.99.2
>
> Attachments: HBASE-12377.v1-2.0.patch
>
>
> This is the same issue that HBASE-10809 tried to address. The fix of
> HBASE-10809 refetch the latest meta location in retry-loop. However, there
> are 2 problems: (1). inside the retry loop, there is another try-catch block
> that would throw the exception before retry can kick in; (2). It looks like
> that HBaseAdmin::getFirstMetaServerForTable() always tries to get meta data
> from meta cache, which means if the meta cache is stale and out of date,
> retries would not solve the problem by fetching from the stale meta cache.
> Here is the call stack of the issue:
> {noformat}
> 2014-10-27
> 10:11:58,495|beaver.machine|INFO|18218|140065036261120|MainThread|org.apache.hadoop.hbase.NotServingRegionException:
> org.apache.hadoop.hbase.NotServingRegionException: Region hbase:meta,,1 is
> not online on ip-172-31-0-48.ec2.internal,60020,1414403435009
> 2014-10-27
> 10:11:58,496|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegionByEncodedName(HRegionServer.java:2774)
> 2014-10-27
> 10:11:58,496|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getRegion(HRegionServer.java:4257)
> 2014-10-27
> 10:11:58,497|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3156)
> 2014-10-27
> 10:11:58,497|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29994)
> 2014-10-27
> 10:11:58,498|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2078)
> 2014-10-27
> 10:11:58,498|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:108)
> 2014-10-27
> 10:11:58,499|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:114)
> 2014-10-27
> 10:11:58,499|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:94)
> 2014-10-27
> 10:11:58,499|beaver.machine|INFO|18218|140065036261120|MainThread|at
> java.lang.Thread.run(Thread.java:745)
> 2014-10-27 10:11:58,500|beaver.machine|INFO|18218|140065036261120|MainThread|
> 2014-10-27
> 10:11:58,500|beaver.machine|INFO|18218|140065036261120|MainThread|at
> sun.reflect.GeneratedConstructorAccessor12.newInstance(Unknown Source)
> 2014-10-27
> 10:11:58,500|beaver.machine|INFO|18218|140065036261120|MainThread|at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 2014-10-27
> 10:11:58,501|beaver.machine|INFO|18218|140065036261120|MainThread|at
> java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> 2014-10-27
> 10:11:58,501|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
> 2014-10-27
> 10:11:58,502|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
> 2014-10-27
> 10:11:58,502|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:306)
> 2014-10-27
> 10:11:58,502|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.client.HBaseAdmin.deleteTable(HBaseAdmin.java:699)
> 2014-10-27
> 10:11:58,503|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.client.HBaseAdmin.deleteTable(HBaseAdmin.java:654)
> 2014-10-27
> 10:11:58,503|beaver.machine|INFO|18218|140065036261120|MainThread|at
> org.apache.hadoop.hbase.IntegrationTestManyRegions.tearDown(IntegrationTestManyRegions.java:99)
> {noformat}
> The META region was Online in RS1 when the delete table starts, it was moved
> to RS2 during the delete table operation. And the problem appears.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
