[
https://issues.apache.org/jira/browse/HBASE-9085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
gautam updated HBASE-9085:
--------------------------
Description:
I was running the following test over a Distributed Cluster:
bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver
IntegrationTestDataIngestSlowDeterministic
The IntegrationTestingUtility.restoreCluster() is called in the teardown phase
of the test.
For a distributed cluster, it ends up calling
DistributedHBaseCluster.restoreClusterStatus, which does the task
of restoring the cluster back to original state.
The restore steps done here, does not solve one specific case:
When the initial HBase Master is currently down, and the current HBase Master
is different from the initial one.
You get into this flow:
//check whether current master has changed
if (!ServerName.isSameHostnameAndPort(initial.getMaster(),
current.getMaster())) {
.............
}
In the above code path, the current backup masters are stopped, and the current
active master is also stopped.
At this point, for the aforementioned usecase, none of the Hbase Masters would
be available, hence the subsequent
attempts to do any operation over the cluster would fail, resulting in Test
Failure.
was:
Let me split this requirement into 2 parts:
i) ChaosMonkey
I was trying to add more tests around new actions and policies by leveraging
the existing classes nested inside ChaosMonkey.
But it turned out that some of the classes cannot be used outside, unless we
make those visible to the world.
Here is an example:
I cannot extend ChaosMonkey.Action, as the init(ActionContext context) method
has package-wide visibility.
There are other places as well which makes it impossible for anyone to extend
on top of this hierarchy.
ii) LoadTestTool
I wanted to extend this tool to define failure/pass criteria based on % of
read/write failed, rather than comparing against absolute 0.
For that this beautiful class should mark some of its properties usable by its
child, by marking those protected.
I wanted to get unblocked here first.
Once this gets fixed, I think I can take up a JIRA item to refactor these
tools, if required.
> Integration Tests fails because of bug in teardown phase where the cluster
> state is not being restored properly.
> ----------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-9085
> URL: https://issues.apache.org/jira/browse/HBASE-9085
> Project: HBase
> Issue Type: Test
> Components: test
> Affects Versions: 0.95.2
> Reporter: gautam
> Assignee: gautam
> Fix For: 0.98.0, 0.95.2, 0.94.10
>
>
> I was running the following test over a Distributed Cluster:
> bin/hbase org.apache.hadoop.hbase.IntegrationTestsDriver
> IntegrationTestDataIngestSlowDeterministic
> The IntegrationTestingUtility.restoreCluster() is called in the teardown
> phase of the test.
> For a distributed cluster, it ends up calling
> DistributedHBaseCluster.restoreClusterStatus, which does the task
> of restoring the cluster back to original state.
> The restore steps done here, does not solve one specific case:
> When the initial HBase Master is currently down, and the current HBase Master
> is different from the initial one.
> You get into this flow:
> //check whether current master has changed
> if (!ServerName.isSameHostnameAndPort(initial.getMaster(),
> current.getMaster())) {
> .............
> }
> In the above code path, the current backup masters are stopped, and the
> current active master is also stopped.
> At this point, for the aforementioned usecase, none of the Hbase Masters
> would be available, hence the subsequent
> attempts to do any operation over the cluster would fail, resulting in Test
> Failure.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
