[
https://issues.apache.org/jira/browse/STORM-898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15063312#comment-15063312
]
ASF GitHub Bot commented on STORM-898:
--------------------------------------
Github user d2r commented on a diff in the pull request:
https://github.com/apache/storm/pull/921#discussion_r47988385
--- Diff: storm-core/test/jvm/backtype/storm/TestConfigValidate.java ---
@@ -628,9 +628,89 @@ public void TestImpersonationAclUserEntryValidator()
throws InvocationTargetExce
} catch (IllegalArgumentException Ex) {
}
}
+ }
+
+ @Test
+ public void TestResourceAwareSchedulerUserPool() {
+ TestConfig config = new TestConfig();
+ Collection<Object> passCases = new LinkedList<Object>();
+ Collection<Object> failCases = new LinkedList<Object>();
+
+ Map<String, Map<String, Integer>> passCase1 = new HashMap<String,
Map<String, Integer>>();
+ passCase1.put("jerry", new HashMap<String, Integer>());
+ passCase1.put("bobby", new HashMap<String, Integer>());
+ passCase1.put("derek", new HashMap<String, Integer>());
+
+ passCase1.get("jerry").put("cpu", 10000);
+ passCase1.get("jerry").put("memory", 20148);
+ passCase1.get("bobby").put("cpu", 20000);
+ passCase1.get("bobby").put("memory", 40148);
+ passCase1.get("derek").put("cpu", 30000);
+ passCase1.get("derek").put("memory", 60148);
+
+ passCases.add(passCase1);
+
+ for (Object value : passCases) {
+ config.put(TestConfig.TEST_MAP_CONFIG_7, value);
+ ConfigValidation.validateFields(config, TestConfig.class);
+ }
+
+ Map<String, Map<String, Integer>> failCase1 = new HashMap<String,
Map<String, Integer>>();
+ failCase1.put("jerry", new HashMap<String, Integer>());
+ failCase1.put("bobby", new HashMap<String, Integer>());
+ failCase1.put("derek", new HashMap<String, Integer>());
+
+ failCase1.get("jerry").put("cpu", 10000);
+ failCase1.get("jerry").put("memory", 20148);
+ failCase1.get("bobby").put("cpu", 20000);
+ failCase1.get("bobby").put("memory", 40148);
+ failCase1.get("derek").put("cpu", 30000);
+
+ Map<String, Map<String, Integer>> failCase2 = new HashMap<String,
Map<String, Integer>>();
+ failCase2.put("jerry", new HashMap<String, Integer>());
+ failCase2.put("bobby", new HashMap<String, Integer>());
+ failCase2.put("derek", new HashMap<String, Integer>());
+ failCase2.get("bobby").put("cpu", 20000);
+ failCase2.get("bobby").put("memory", 40148);
+ failCase2.get("derek").put("cpu", 30000);
+ failCase2.get("derek").put("memory", 60148);
+
+ failCases.add(failCase1);
+ failCases.add(failCase2);
+
+ for (Object value : failCases) {
+ try {
+ config.put(TestConfig.TEST_MAP_CONFIG_7, value);
+ ConfigValidation.validateFields(config, TestConfig.class);
+ Assert.fail("Expected Exception not Thrown for value: " +
value);
+ } catch (IllegalArgumentException Ex) {
+ }
+ }
+ }
+
+ @Test
+ public void TestImplementsClassValidator() {
+ TestConfig config = new TestConfig();
+ Collection<Object> passCases = new LinkedList<Object>();
+ Collection<Object> failCases = new LinkedList<Object>();
+
passCases.add("backtype.storm.networktopography.DefaultRackDNSToSwitchMapping");
+ for (Object value : passCases) {
+ config.put(TestConfig.TEST_MAP_CONFIG_8, value);
+ ConfigValidation.validateFields(config, TestConfig.class);
+ }
+ failCases.add("backtype.storm.nimbus.NimbusInfo");
+ failCases.add(null);
--- End diff --
Comment: fail because they are not `DefaultRackDNSToSwitchMapping`
> Add priorities and per user resource guarantees to Resource Aware Scheduler
> ---------------------------------------------------------------------------
>
> Key: STORM-898
> URL: https://issues.apache.org/jira/browse/STORM-898
> Project: Apache Storm
> Issue Type: New Feature
> Components: storm-core
> Reporter: Robert Joseph Evans
> Assignee: Boyang Jerry Peng
> Attachments: Resource Aware Scheduler for Storm.pdf
>
>
> In a multi-tenant environment we would like to be able to give individual
> users a guarantee of how much CPU/Memory/Network they will be able to use in
> a cluster. We would also like to know which topologies a user feels are the
> most important to keep running if there are not enough resources to run all
> of their topologies.
> Each user should be able to specify if their topology is production, staging,
> or development. Within each of those categories a user should be able to give
> a topology a priority, 0 to 10 with 10 being the highest priority (or
> something like this).
> If there are not enough resources on a cluster to run a topology assume this
> topology is running using resources and find the user that is most over their
> guaranteed resources. Shoot the lowest priority topology for that user, and
> repeat until, this topology is able to run, or this topology would be the one
> shot. Ideally we don't actually shoot anything until we know that we would
> have made enough room.
> If the cluster is over-subscribed and everyone is under their guarantee, and
> this topology would not put the user over their guarantee. Shoot the lowest
> priority topology in this workers resource pool until there is enough room to
> run the topology or this topology is the one that would be shot. We might
> also want to think about what to do if we are going to shoot a production
> topology in an oversubscribed case, and perhaps we can shoot a non-production
> topology instead even if the other user is not over their guarantee.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
