[
https://issues.apache.org/jira/browse/PHOENIX-6288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17262139#comment-17262139
]
Istvan Toth edited comment on PHOENIX-6288 at 1/10/21, 9:57 AM:
----------------------------------------------------------------
In theory I am not against splitting some tests into a new module.
However, that would only help us if those were running on different VMs /
executors.
Our current test setup is not prepared for that, and hogging even more ASF
Hadoop executors may not go down well.
(We are already using 7 for a commit into both branches)
As long as we stick to single VM/executor, increasing forkCount via numForkedIT
is better than splitting the tests into modules and running the with --threads.
Say, we split the tests into two new modules, one with a run time of 1, and one
with a runtime of 2, setting forkCount=8 for each amd running it with
--threads=2.
In this case, for the first half of the execution we'll have 16 JVMs running
(and using resources), while in the second half we have 8 JVMs running.
Keeping the tests in a single module with forkCount=16 will generate the same
maximum load on the executor, but the tests will finish in 75% of the time
(assuming infinite resources on the VM).
Thinking about it, we can split test execution with failsafe groups /
@Categories the same way as with modules.
Say, if we ran on two VMs, we could use one to run the @NeedsOwnCluster tests,
and the other for the rest.
was (Author: stoty):
In theory I am not against splitting some tests into a new module.
However, that would only help us if those were running on different VMs /
executors.
Our current test setup is not prepared for that, and hogging even more ASF
Hadoop executors may not go down well.
(We are already using 7 for a commit into both branches)
As long as we stick to single VM/executor, increasing forkCount via numForkedIT
is better than splitting the tests into modules and running the with --threads.
Say, we split the tests into two new modules, one with a run time of 1, and one
with a runtime of 2, setting forkCount=8 for each amd running it with
--threads=2.
In this case, for the first half of the execution we'll have 16 JVMs running
(and using resources), while in the second half we have 8 JVMs running.
Keeping the tests in a single module with forkCount=16 will generate the same
maximum load on the executor, but the tests will finish in 75% of the time
(assuming infinite resources on the VM).
Thinking about it, we can split test execution with failsafe groups /
@Categories as well as with modules.
Say, if we ran on two VMs, we could use one to run the @NeedsOwnCluster tests,
and the other for the rest.
> Minicluster startup problems on Jenkins
> ---------------------------------------
>
> Key: PHOENIX-6288
> URL: https://issues.apache.org/jira/browse/PHOENIX-6288
> Project: Phoenix
> Issue Type: Bug
> Reporter: Istvan Toth
> Assignee: Istvan Toth
> Priority: Critical
> Fix For: 5.1.0, 4.16.0
>
>
> We are sporadically getting Test failures on Jenkins that are caused by the
> miniCluster startup timeouts.
> {noformat}
> Caused by: java.lang.RuntimeException: Master not active after 30000ms at
> org.apache.hadoop.hbase.util.JVMClusterUtil.waitForEvent(JVMClusterUtil.java:232)
>
> at
> org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:188)
> at
> org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:430)
> at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:259)
> ... 43 more
> {noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
