Radeity commented on code in PR #13896:
URL:
https://github.com/apache/dolphinscheduler/pull/13896#discussion_r1161304585
##########
dolphinscheduler-worker/src/main/java/org/apache/dolphinscheduler/server/worker/task/WorkerHeartBeatTask.java:
##########
@@ -90,16 +88,16 @@ public void writeHeartBeat(WorkerHeartBeat workerHeartBeat)
{
workerRegistryPath, workerHeartBeatJson);
}
- public int getServerStatus(double loadAverage,
- double maxCpuloadAvg,
+ public int getServerStatus(double cpuUsagePercentage,
+ double maxCpuUsePercentage,
double availablePhysicalMemorySize,
double reservedMemory,
int workerExecThreadCount,
int workerWaitingTaskCount) {
- if (loadAverage > maxCpuloadAvg || availablePhysicalMemorySize <
reservedMemory) {
+ if (cpuUsagePercentage > maxCpuUsePercentage ||
availablePhysicalMemorySize < reservedMemory) {
Review Comment:
Here, it still compares the physical memory with reserved memory, their unit
are both GB.
##########
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/OSUtils.java:
##########
@@ -466,23 +433,25 @@ public static int getProcessID() {
/**
* Check memory and cpu usage is overload the given thredshod.
*
- * @param maxCpuLoadAvg maxCpuLoadAvg
- * @param reservedMemory reservedMemory
+ * @param maxCpuLoadAvgThreshold maxCpuLoadAvg
+ * @param reservedMemoryThreshold reservedMemory
* @return True, if the cpu or memory exceed the given thredshod.
*/
- public static Boolean isOverload(double maxCpuLoadAvg, double
reservedMemory) {
+ public static Boolean isOverload(double maxCpuLoadAvgThreshold, double
reservedMemoryThreshold) {
// system load average
- double loadAverage = loadAverage();
+ double freeCPUPercentage = 1 - cpuUsagePercentage();
// system available physical memory
- double availablePhysicalMemorySize = availablePhysicalMemorySize();
- if (loadAverage > maxCpuLoadAvg) {
- log.warn("Current cpu load average {} is too high,
max.cpuLoad.avg={}", loadAverage, maxCpuLoadAvg);
+ double freeMemoryPercentage = 1 - memoryUsagePercentage();
+ if (freeCPUPercentage > maxCpuLoadAvgThreshold) {
+ log.warn("Current cpu load average {} is too high,
max.cpuLoad.avg={}", freeCPUPercentage,
+ maxCpuLoadAvgThreshold);
return true;
}
- if (availablePhysicalMemorySize < reservedMemory) {
+ if (freeMemoryPercentage < reservedMemoryThreshold) {
log.warn(
- "Current available memory {}G is too low,
reserved.memory={}G", maxCpuLoadAvg, reservedMemory);
+ "Current available memory percentage{} is too low,
reserved.memory={}", freeMemoryPercentage,
+ reservedMemoryThreshold);
Review Comment:
The unit of `reservedMemory ` is still GB, if you want to change the memory
and cpu representation to percentage, you may have to also modify the check in
`getServerStatus `.
##########
dolphinscheduler-worker/src/main/resources/application.yaml:
##########
@@ -48,9 +48,9 @@ worker:
tenant-auto-create: true
#Scenes to be used for distributed users.For example,users created by
FreeIpa are stored in LDAP.This parameter only applies to Linux, When this
parameter is true, worker.tenant.auto.create has no effect and will not
automatically create tenants.
tenant-distributed-user: false
- # worker max cpuload avg, only higher than the system cpu load average,
worker server can be dispatched tasks. default value -1: the number of cpu
cores * 2
- max-cpu-load-avg: -1
- # worker reserved memory, only lower than system available memory, worker
server can be dispatched tasks. default value 0.3, the unit is G
+ # worker max cpuload avg, only higher than the system cpu load average,
worker server can be dispatched tasks. default value 1: will use 100% cpu.
+ max-cpu-load-avg: 1
+ # worker reserved memory, only lower than system available memory, worker
server can be dispatched tasks. default value 0.3, only the available memory is
higher than 30%, worker server can receive task.
reserved-memory: 0.3
Review Comment:
Maybe you can modify the doc of `reserved-memory`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
