azagrebin commented on a change in pull request #10146: [FLINK-14188][runtime]
TaskExecutor derive and register with default slot resource profile
URL: https://github.com/apache/flink/pull/10146#discussion_r352143376
##########
File path:
flink-runtime/src/main/java/org/apache/flink/runtime/clusterframework/TaskExecutorResourceUtils.java
##########
@@ -593,9 +600,20 @@ private static void sanityCheckShuffleMemory(final
Configuration config, final M
}
}
+ @Nullable
+ private static CPUResource getCpuCores(final Configuration config) {
+ double cpuCores =
config.getDouble(TaskManagerOptions.CPU_CORES);
+ if (cpuCores >= 0.0) {
+ return new CPUResource(cpuCores);
+ } else {
+ return null;
Review comment:
Making CPU spec optional (related to CPU config default value), the current
implementation in PR will basically silently accept all concrete cpu requests
in standalone mode without actually respecting them. It can result in a slow
performance without an obvious reason. Not sure what is the plan for CPU
requirements, I expect, we will introduce concrete specs only for Blink now. As
it is a new feature, it is probably better that the scheduling fails and makes
user explicitly think about the provided resources. If CPU requirement is not
important for the resource request, it can be just set to zero and will match
any slot.
Also, we can fallback to the number of slots here as well if cpus are not
explicitly set.
Another thing to improve standalone mode experience, we could modify the
deployment scripts to detect the number of cores during the startup giving a
warning that it is not provided and will be auto-detected.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
