sodonnel commented on a change in pull request #2554:
URL: https://github.com/apache/ozone/pull/2554#discussion_r695566169
##########
File path:
hadoop-ozone/integration-test/src/test/java/org/apache/hadoop/ozone/scm/node/TestDecommissionAndMaintenance.java
##########
@@ -98,8 +104,123 @@
private ContainerOperationClient scmClient;
- @Before
- public void setUp() throws Exception {
+ private static MiniClusterProvider clusterProvider;
+
+ /**
+ * Class to create mini-clusters in the background.
+ */
+ public static class MiniClusterProvider {
+
+ private int preCreatedLimit = 1;
+
+ private final OzoneConfiguration conf;
+ private final MiniOzoneCluster.Builder builder;
+ private boolean shouldRun = true;
+ private boolean shouldReap = true;
+ private Thread createThread;
+ private Thread reapThread;
+
+ private final BlockingQueue<MiniOzoneCluster> clusters
+ = new ArrayBlockingQueue<>(preCreatedLimit);
+ private final BlockingQueue<MiniOzoneCluster> expiredClusters
+ = new ArrayBlockingQueue<>(1024);
+
+
+ public MiniClusterProvider(OzoneConfiguration conf,
+ MiniOzoneCluster.Builder builder) {
+ this.conf = conf;
+ this.builder = builder;
+ createThread = createClusters();
+ reapThread = reapClusters();
+ }
+
+ public MiniOzoneCluster provide() throws InterruptedException {
+ return clusters.poll(100, SECONDS);
+ }
+
+ public void destroy(MiniOzoneCluster c) throws InterruptedException {
+ expiredClusters.put(c);
+ }
+
+ public void shutdown() throws InterruptedException {
+ shouldRun = false;
+ createThread.interrupt();
+ createThread.join();
+ destroyRemainingClusters();
+ shouldReap = false;
+ reapThread.join();
+ }
+
+ private Thread reapClusters() {
+ Thread t = new Thread(() -> {
+ while(shouldReap || !expiredClusters.isEmpty()) {
+ try {
+ MiniOzoneCluster c = expiredClusters.take();
+ c.shutdown();
+ } catch (InterruptedException e) {
+ break;
+ }
+ }
+ });
+ t.start();
+ return t;
+ }
+
+ private Thread createClusters() {
+ Thread t = new Thread(() -> {
+ while (shouldRun && !Thread.interrupted()) {
+ MiniOzoneCluster cluster = null;
+ try {
+ builder.setClusterId(UUID.randomUUID().toString());
+
+ OzoneConfiguration newConf = new OzoneConfiguration(conf);
+ List<Integer> portList = getFreePortList(4);
+ newConf.set(OMConfigKeys.OZONE_OM_ADDRESS_KEY,
+ "127.0.0.1:" + portList.get(0));
+ newConf.set(OMConfigKeys.OZONE_OM_HTTP_ADDRESS_KEY,
+ "127.0.0.1:" + portList.get(1));
+ newConf.set(OMConfigKeys.OZONE_OM_HTTPS_ADDRESS_KEY,
+ "127.0.0.1:" + portList.get(2));
+ newConf.setInt(OMConfigKeys.OZONE_OM_RATIS_PORT_KEY,
+ portList.get(3));
+ builder.setConf(newConf);
+
+ cluster = builder.build();
+ cluster.waitForClusterToBeReady();
+ clusters.put(cluster);
Review comment:
I think solving this is tricky. The tests don't really know how many
clusters they will need. We can assume its one cluster per test, so if the
class has 5 tests, we configure 5. Later, someone will add another test and
forget to update the setting and the last test will need to wait on a cluster
or get an error.
Alternatively we could use reflection and check for methods starting "test"
but that seems like more hassle than its worth.
I guess if the provider throws an exception if you ask for more clusters
than it was configured to provide, it would catch the case of not configuring
enough clusters and hence would enforce setting the correct limit.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
