Github user revans2 commented on a diff in the pull request:
https://github.com/apache/storm/pull/2385#discussion_r147236417
--- Diff:
storm-server/src/main/java/org/apache/storm/scheduler/Cluster.java ---
@@ -503,62 +553,70 @@ public boolean wouldFit(
WorkerSlot ws,
ExecutorDetails exec,
TopologyDetails td,
- double maxHeap,
- double memoryAvailable,
- double cpuAvailable) {
- //NOTE this is called lots and lots by schedulers, so anything we
can do to make it faster is going to help a lot.
- //CPU is simplest because it does not have odd interactions.
- double cpuNeeded = td.getTotalCpuReqTask(exec);
- if (cpuNeeded > cpuAvailable) {
- if (LOG.isTraceEnabled()) {
- LOG.trace("Could not schedule {}:{} on {} not enough CPU
{} > {}",
- td.getName(),
- exec,
- ws,
- cpuNeeded,
- cpuAvailable);
+ Map<String, Double> resourcesAvailable,
+ double maxHeap) {
+
+ Map<String, Double> requestedResources =
td.getTotalResources(exec);
+
+ LOG.info(td.getName());
+ LOG.info("requested");
+ LOG.info(requestedResources.toString());
+ LOG.info("available");
+ LOG.info(resourcesAvailable.toString());
+ LOG.info(ws.toString());
+ for (Entry resourceNeededEntry : requestedResources.entrySet()) {
+ String resourceName = resourceNeededEntry.getKey().toString();
+ if (resourceName ==
Constants.COMMON_OFFHEAP_MEMORY_RESOURCE_NAME || resourceName ==
Constants.COMMON_ONHEAP_MEMORY_RESOURCE_NAME) {
+ continue;
}
- //Not enough CPU no need to try any more
- return false;
- }
-
- //Lets see if we can make the Memory one fast too, at least in the
failure case.
- //The totalMemReq is not really that accurate because it does not
include shared memory, but if it does not fit we know
- // Even with shared it will not work
- double minMemNeeded = td.getTotalMemReqTask(exec);
- if (minMemNeeded > memoryAvailable) {
- if (LOG.isTraceEnabled()) {
- LOG.trace("Could not schedule {}:{} on {} not enough Mem
{} > {}", td.getName(), exec, ws, minMemNeeded, memoryAvailable);
+ Double resourceNeeded =
ObjectReader.getDouble(resourceNeededEntry.getValue());
+ Double resourceAvailable = ObjectReader.getDouble(
+ resourcesAvailable.getOrDefault(resourceName, null),
0.0);
+ if (resourceNeeded > resourceAvailable) {
+ if (true) {
--- End diff --
I think this is a debug logging change. Do you want to change it back?
---
