niyanchun commented on issue #16738: URL: https://github.com/apache/dolphinscheduler/issues/16738#issuecomment-2440705704
> Which version you are using? @niyanchun @SbloodyS 3.2.2. We also find cpu overload in worker logs: ``` dolphinscheduler-worker.2024-10-28_12.0.log:[WI-0][TI-0] - [INFO] 2024-10-28 12:00:40.308 +0800 o.a.d.m.m.BaseServerLoadProtection:[43] - OverLoad: the system cpu usage: 0.9396984924623116 is over then the maxSystemCpuUsagePercentageThresholds 0.9 dolphinscheduler-worker.2024-10-28_12.0.log:[WI-0][TI-0] - [INFO] 2024-10-28 12:00:41.320 +0800 o.a.d.m.m.BaseServerLoadProtection:[43] - OverLoad: the system cpu usage: 0.9603960396039605 is over then the maxSystemCpuUsagePercentageThresholds 0.9 dolphinscheduler-worker.2024-10-28_12.0.log:[WI-0][TI-0] - [INFO] 2024-10-28 12:00:42.320 +0800 o.a.d.m.m.BaseServerLoadProtection:[43] - OverLoad: the system cpu usage: 0.9849246231155779 is over then the maxSystemCpuUsagePercentageThresholds 0.9 dolphinscheduler-worker.2024-10-28_12.0.log:[WI-0][TI-0] - [INFO] 2024-10-28 12:00:43.328 +0800 o.a.d.m.m.BaseServerLoadProtection:[43] - OverLoad: the system cpu usage: 0.9900744416873449 is over then the maxSystemCpuUsagePercentageThresholds 0.9 ``` Up to now, the failed likely caused by worker resource lack, but in fact the server load is low ( avg cpu is 20%）, so we decide to modify some worker configs, and observe for some while: - set `worker.server-load-protection.enabled` as false (default true). - set `worker.exec-threads` as 500 (now 400). - set `worker.task-execute-threads-full-policy` as CONTINUE(default reject), we want the task queued. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
