Ethanlm commented on a change in pull request #3217: STORM-3590: Adds test to
validate that GRAS's node sort is stable and…
URL: https://github.com/apache/storm/pull/3217#discussion_r402631134
##########
File path:
storm-server/src/test/java/org/apache/storm/scheduler/resource/strategies/scheduling/TestGenericResourceAwareStrategy.java
##########
@@ -300,7 +302,64 @@ public void testGrasRequiringEviction() {
assertTopologiesFullyScheduled(cluster, noGpu);
assertTopologiesFullyScheduled(cluster, gpu2);
}
-
+
+ /*
+ * test GRAS should not evict other topologies for the sake of satisfying
generic resource requirements.
+ * GRAS should also not resort nodes and 'pack' a node instead of
spreading it out causing
+ * fragmentation.
+ */
+ @Test
+ public void testGrasStarvation() {
+ double cpuPercent = 200;
+ double memoryOnHeap = 100;
+ double memoryOffHeap = 100;
+
+ TopologyBuilder builder = new TopologyBuilder();
+
+ // non-gpu topology
+ builder.setSpout("spout", new TestSpout(), 2);
+ StormTopology stormTopologyNoGpu = builder.createTopology();
+ String noGpu = "hasNoGpu";
+ List<TopologyDetails> topologyDetails = new ArrayList();
+ Config nonGpuconf = createClusterConfig(cpuPercent, memoryOnHeap,
memoryOffHeap, null);
+ nonGpuconf.put(Config.TOPOLOGY_SCHEDULER_STRATEGY,
DefaultResourceAwareStrategy.class.getName());
+
+ cpuPercent = 100;
+
+ // gpu topology (requires 4 gpu's in total)
+ builder = new TopologyBuilder();
+ builder.setSpout("spout", new TestSpout(), 4).addResource("gpu.count",
1.0);
+ StormTopology stormTopologyWithGpu = builder.createTopology();
+
+
+ Config conf = createGrasClusterConfig(cpuPercent, memoryOnHeap,
memoryOffHeap, null, Collections.emptyMap());
+ conf.put(Config.TOPOLOGY_SCHEDULER_STRATEGY,
GenericResourceAwareStrategy.class.getName());
+
+ String gpu = "hasGpu";
+ Topologies topologies = new
Topologies(createTestStormTopology(stormTopologyWithGpu, 9, gpu, conf));
+
+ Map<String, Double> genericResourcesMap = new HashMap<>();
+ genericResourcesMap.put("gpu.count", 2.0);
+ Map<String, SupervisorDetails> supMap = genSupervisors(4, 4, 200,
2000, genericResourcesMap);
+ Cluster cluster = new Cluster(new INimbusTest(), new
ResourceMetrics(new StormMetricsRegistry()), supMap, new HashMap<>(),
topologies, conf);
+
+ // should schedule gpu and noGpu successfully
+ scheduler = new ResourceAwareScheduler();
+
nonGpuconf.put(DaemonConfig.RESOURCE_AWARE_SCHEDULER_MAX_TOPOLOGY_SCHEDULING_ATTEMPTS,
1); // allows no evictions
Review comment:
This is not able to indicate if eviction happens or not.
Waiting on https://github.com/apache/storm/pull/3213 so we can have a right
way to detect eviction.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
