[
https://issues.apache.org/jira/browse/HBASE-24178?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
huangtianhua updated HBASE-24178:
---------------------------------
Description:
I build and test Hbase on aarch64, and tests
org.apache.hadoop.hbase.TestZooKeeper.testMasterZKSessionRecoveryFailure and
testMasterSessionExpired failed by chance. I found ARM CI in jenkins failed by
chance too
https://builds.apache.org/view/H-L/view/HBase/job/HBase-Nightly-ARM/73/testReport/junit/org.apache.hadoop.hbase/TestZooKeeper/testMasterZKSessionRecoveryFailure/,
see:
Error Message
Unable to instantiate exception received from
server:org.apache.hadoop.hbase.master.HMaster$MasterStoppedException.<init>(java.lang.String)
Stacktrace
org.apache.hadoop.hbase.DoNotRetryIOException: Unable to instantiate exception
received from
server:org.apache.hadoop.hbase.master.HMaster$MasterStoppedException.<init>(java.lang.String)
at
org.apache.hadoop.hbase.TestZooKeeper.testSanity(TestZooKeeper.java:153)
at
org.apache.hadoop.hbase.TestZooKeeper.testMasterZKSessionRecoveryFailure(TestZooKeeper.java:140)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException:
org.apache.hadoop.hbase.master.HMaster$MasterStoppedException
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2932)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2135)
at
org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:716)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:400)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
It looks like the master is still stopped when we want to create table after
stop it. I made thread to sleep 1 second in testSanity function before calling
admin.createTable(desc), then I run the tests for more than 10 times and the
tests are success:
**************************************************************************
private void testSanity(final String testName) throws Exception {
String tableName = testName + "_" + System.currentTimeMillis();
TableDescriptor desc =
TableDescriptorBuilder.newBuilder(TableName.valueOf(tableName))
.setColumnFamily(ColumnFamilyDescriptorBuilder.of("fam")).build();
{color:#DE350B}Thread.sleep(1000);{color}
LOG.info("Creating table " + tableName);
Admin admin = TEST_UTIL.getAdmin();
try {
admin.createTable(desc);
} finally {
admin.close();
}
**************************************************************************
was:
I build and test Hbase on aarch64, and tests
org.apache.hadoop.hbase.TestZooKeeper.testMasterZKSessionRecoveryFailure and
testMasterSessionExpired failed by chance. I found ARM CI in jenkins failed by
chance too, see:
Error Message
Unable to instantiate exception received from
server:org.apache.hadoop.hbase.master.HMaster$MasterStoppedException.<init>(java.lang.String)
Stacktrace
org.apache.hadoop.hbase.DoNotRetryIOException: Unable to instantiate exception
received from
server:org.apache.hadoop.hbase.master.HMaster$MasterStoppedException.<init>(java.lang.String)
at
org.apache.hadoop.hbase.TestZooKeeper.testSanity(TestZooKeeper.java:153)
at
org.apache.hadoop.hbase.TestZooKeeper.testMasterZKSessionRecoveryFailure(TestZooKeeper.java:140)
Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException:
org.apache.hadoop.hbase.master.HMaster$MasterStoppedException
at
org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2932)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2135)
at
org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:716)
at
org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:400)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
It looks like the master is still stopped when we want to create table after
stop it. I made thread to sleep 1 second in testSanity function before calling
admin.createTable(desc), then I run the tests for more than 10 times and the
tests are success:
private void testSanity(final String testName) throws Exception {
String tableName = testName + "_" + System.currentTimeMillis();
TableDescriptor desc =
TableDescriptorBuilder.newBuilder(TableName.valueOf(tableName))
.setColumnFamily(ColumnFamilyDescriptorBuilder.of("fam")).build();
{color:#DE350B}Thread.sleep(1000);{color}
LOG.info("Creating table " + tableName);
Admin admin = TEST_UTIL.getAdmin();
try {
admin.createTable(desc);
} finally {
admin.close();
}
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
at
org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> some tests of org.apache.hadoop.hbase.TestZooKeeper.* failed on aarch64
> -----------------------------------------------------------------------
>
> Key: HBASE-24178
> URL: https://issues.apache.org/jira/browse/HBASE-24178
> Project: HBase
> Issue Type: Bug
> Reporter: huangtianhua
> Priority: Major
>
> I build and test Hbase on aarch64, and tests
> org.apache.hadoop.hbase.TestZooKeeper.testMasterZKSessionRecoveryFailure and
> testMasterSessionExpired failed by chance. I found ARM CI in jenkins failed
> by chance too
> https://builds.apache.org/view/H-L/view/HBase/job/HBase-Nightly-ARM/73/testReport/junit/org.apache.hadoop.hbase/TestZooKeeper/testMasterZKSessionRecoveryFailure/,
> see:
> Error Message
> Unable to instantiate exception received from
> server:org.apache.hadoop.hbase.master.HMaster$MasterStoppedException.<init>(java.lang.String)
> Stacktrace
> org.apache.hadoop.hbase.DoNotRetryIOException: Unable to instantiate
> exception received from
> server:org.apache.hadoop.hbase.master.HMaster$MasterStoppedException.<init>(java.lang.String)
> at
> org.apache.hadoop.hbase.TestZooKeeper.testSanity(TestZooKeeper.java:153)
> at
> org.apache.hadoop.hbase.TestZooKeeper.testMasterZKSessionRecoveryFailure(TestZooKeeper.java:140)
> Caused by: org.apache.hadoop.hbase.ipc.RemoteWithExtrasException:
> org.apache.hadoop.hbase.master.HMaster$MasterStoppedException
> at
> org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2932)
> at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:2135)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:716)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:400)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:133)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> It looks like the master is still stopped when we want to create table after
> stop it. I made thread to sleep 1 second in testSanity function before
> calling admin.createTable(desc), then I run the tests for more than 10 times
> and the tests are success:
> **************************************************************************
> private void testSanity(final String testName) throws Exception {
> String tableName = testName + "_" + System.currentTimeMillis();
> TableDescriptor desc =
> TableDescriptorBuilder.newBuilder(TableName.valueOf(tableName))
> .setColumnFamily(ColumnFamilyDescriptorBuilder.of("fam")).build();
> {color:#DE350B}Thread.sleep(1000);{color}
> LOG.info("Creating table " + tableName);
> Admin admin = TEST_UTIL.getAdmin();
> try {
> admin.createTable(desc);
> } finally {
> admin.close();
> }
> **************************************************************************
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
