Ethanlm commented on a change in pull request #3217: STORM-3590: Adds test to
validate that GRAS's node sort is stable and…
URL: https://github.com/apache/storm/pull/3217#discussion_r384816921
##########
File path:
storm-server/src/test/java/org/apache/storm/scheduler/resource/strategies/scheduling/TestGenericResourceAwareStrategy.java
##########
@@ -300,7 +302,63 @@ public void testGrasRequiringEviction() {
assertTopologiesFullyScheduled(cluster, noGpu);
assertTopologiesFullyScheduled(cluster, gpu2);
}
-
+
+ /*
+ * test GRAS should not evict other topologies for the sake of satisfying
generic resource requirements.
+ * GRAS should also not resort nodes and 'pack' a node instead of
spreading it out causing
+ * fragmentation.
+ */
+ @Test
+ public void testGrasStarvation() {
+ double cpuPercent = 200;
+ double memoryOnHeap = 100;
+ double memoryOffHeap = 100;
+
+ TopologyBuilder builder = new TopologyBuilder();
+
+ // non-gpu topology
+ builder.setSpout("spout", new TestSpout(), 2);
+ StormTopology stormTopologyNoGpu = builder.createTopology();
+ String noGpu = "hasNoGpu";
+ List<TopologyDetails> topologyDetails = new ArrayList();
+ Config nonGpuconf = createClusterConfig(cpuPercent, memoryOnHeap,
memoryOffHeap, null);
+ nonGpuconf.put(Config.TOPOLOGY_SCHEDULER_STRATEGY,
DefaultResourceAwareStrategy.class.getName());
+
+ cpuPercent = 100;
+
+ // gpu topology (requires 4 gpu's in total)
+ builder = new TopologyBuilder();
+ builder.setSpout("spout", new TestSpout(), 4).addResource("gpu.count",
1.0);
+ StormTopology stormTopologyWithGpu = builder.createTopology();
+
+
+ Config conf = createGrasClusterConfig(cpuPercent, memoryOnHeap,
memoryOffHeap, null, Collections.emptyMap());
+ conf.put(Config.TOPOLOGY_SCHEDULER_STRATEGY,
GenericResourceAwareStrategy.class.getName());
+
+ String gpu = "hasGpu";
+ Topologies topologies = new
Topologies(createTestStormTopology(stormTopologyWithGpu, 10, gpu, conf));
+
+ Map<String, Double> genericResourcesMap = new HashMap<>();
+ genericResourcesMap.put("gpu.count", 2.0);
+ Map<String, SupervisorDetails> supMap = genSupervisors(4, 4, 200,
2000, genericResourcesMap);
+ Cluster cluster = new Cluster(new INimbusTest(), new
ResourceMetrics(new StormMetricsRegistry()), supMap, new HashMap<>(),
topologies, conf);
+
+ // should schedule gpu and noGpu successfully
+ scheduler = new ResourceAwareScheduler();
+
nonGpuconf.put(DaemonConfig.RESOURCE_AWARE_SCHEDULER_MAX_TOPOLOGY_SCHEDULING_ATTEMPTS,
1); // allows no evictions
+ scheduler.prepare(nonGpuconf);
+ // schedule gpu topology
+ scheduler.schedule(topologies, cluster);
+
+
cluster.addTopologyToClusterToBeScheduled(createTestStormTopology(stormTopologyNoGpu,
10, noGpu, nonGpuconf));
Review comment:
It doesn't look like a good idea to add a new function in the regular code
solely for unit test. We don't need to add this function to achieve what we
want. We can follow this
https://github.com/apache/storm/blob/master/storm-server/src/test/java/org/apache/storm/scheduler/resource/TestResourceAwareScheduler.java#L504-L511
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
