wanglijie95 commented on code in PR #22424:
URL: https://github.com/apache/flink/pull/22424#discussion_r1174525306
##########
flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/slotpool/DefaultAllocatedSlotPool.java:
##########
@@ -218,11 +205,61 @@ public Collection<FreeSlotInfo> getFreeSlotsInformation()
{
return freeSlotInfos;
}
+ public double getTaskExecutorUtilization(ResourceID resourceId) {
+ Set<AllocationID> slots = slotsPerTaskExecutor.get(resourceId);
+ Preconditions.checkNotNull(slots, "There is no slots on %s",
resourceId);
+
+ return (double) (slots.size() -
freeSlots.getSlotsOfTaskExecutor(resourceId).size())
+ / slots.size();
+ }
+
@Override
public Collection<? extends SlotInfo> getAllSlotsInformation() {
return registeredSlots.values();
}
+ private static final class FreeSlots {
+ /** Map containing all free slots and since when they are free. */
+ private final Map<AllocationID, Long> freeSlotsSince = new HashMap<>();
+
+ /** Index containing a mapping between TaskExecutors and their free
slots. */
+ private final Map<ResourceID, Set<AllocationID>>
freeSlotsPerTaskExecutor = new HashMap<>();
Review Comment:
It seems that we only need the size of `Set<AllocationID>` here, how about
changing it to `Map<ResourceID, Long>` ?
##########
flink-runtime/src/main/java/org/apache/flink/runtime/jobmaster/slotpool/SlotSelectionStrategy.java:
##########
@@ -55,15 +57,15 @@ final class SlotInfoAndResources {
@Nonnull private final ResourceProfile remainingResources;
- private final double taskExecutorUtilization;
+ @Nonnull private final Function<ResourceID, Double>
taskExecutorUtilizationLookup;
public SlotInfoAndResources(
Review Comment:
I think we can directly remove the `SlotInfoAndResources` and replace it
with `SlotInfoWithUtilization`. This two class has the same functions. (this
change is better to be a separate commit.)
##########
flink-runtime/src/test/java/org/apache/flink/runtime/clusterframework/types/SlotSelectionStrategyTestBase.java:
##########
@@ -59,26 +58,25 @@ public abstract class SlotSelectionStrategyTestBase extends
TestLogger {
protected final SlotInfoWithUtilization slotInfo1 =
SlotInfoWithUtilization.from(
- new SimpleSlotContext(aid1, tml1, 1, taskManagerGateway,
resourceProfile), 0);
+ new SimpleSlotContext(aid1, tml1, 1, taskManagerGateway,
resourceProfile),
+ ignored -> 0.0d);
protected final SlotInfoWithUtilization slotInfo2 =
SlotInfoWithUtilization.from(
new SimpleSlotContext(aid2, tml2, 2, taskManagerGateway,
biggerResourceProfile),
- 0);
+ ignored -> 0.0d);
protected final SlotInfoWithUtilization slotInfo3 =
SlotInfoWithUtilization.from(
- new SimpleSlotContext(aid3, tml3, 3, taskManagerGateway,
resourceProfile), 0);
+ new SimpleSlotContext(aid3, tml3, 3, taskManagerGateway,
resourceProfile),
+ ignored -> 0.0d);
protected final SlotInfoWithUtilization slotInfo4 =
SlotInfoWithUtilization.from(
- new SimpleSlotContext(aid4, tml4, 4, taskManagerGateway,
resourceProfile), 0);
+ new SimpleSlotContext(aid4, tml4, 4, taskManagerGateway,
resourceProfile),
+ ignored -> 0.0d);
protected final Set<SlotSelectionStrategy.SlotInfoAndResources> candidates
=
Collections.unmodifiableSet(createCandidates());
- protected final SlotSelectionStrategy selectionStrategy;
-
- public SlotSelectionStrategyTestBase(SlotSelectionStrategy
slotSelectionStrategy) {
- this.selectionStrategy = slotSelectionStrategy;
- }
Review Comment:
Is the change(remove the constructor) here is necessary?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
