reele opened a new issue #5929:
URL: https://github.com/apache/dolphinscheduler/issues/5929
When the DS runs in a low configuration (4 processors) environment and the
CPU is overloaded, the OSUtils and TaskPriorityQueueConsumer modules will crazy
output logs to dolphinscheduler-master.log, causing the CPU to be overloaded
all the time, such as:
<pre><code>
[ERROR] 2021-07-31 21:43:34.624
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84563, bodyLen=2041] due to no suitable worker, current task needs
worker group default to execute
[ERROR] 2021-07-31 21:43:34.634
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84564, bodyLen=2089] due to no suitable worker, current task needs
worker group default to execute
[ERROR] 2021-07-31 21:43:34.644
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84565, bodyLen=2083] due to no suitable worker, current task needs
worker group default to execute
[ERROR] 2021-07-31 21:43:34.654
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84566, bodyLen=2041] due to no suitable worker, current task needs
worker group default to execute
[ERROR] 2021-07-31 21:43:34.664
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84567, bodyLen=2089] due to no suitable worker, current task needs
worker group default to execute
[ERROR] 2021-07-31 21:43:34.676
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84568, bodyLen=2083] due to no suitable worker, current task needs
worker group default to execute
[WARN] 2021-07-31 21:43:34.680
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[ERROR] 2021-07-31 21:43:34.687
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84569, bodyLen=2041] due to no suitable worker, current task needs
worker group default to execute
[WARN] 2021-07-31 21:43:34.690
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[ERROR] 2021-07-31 21:43:34.701
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84570, bodyLen=2089] due to no suitable worker, current task needs
worker group default to execute
[ERROR] 2021-07-31 21:43:34.718
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84571, bodyLen=2083] due to no suitable worker, current task needs
worker group default to execute
[WARN] 2021-07-31 21:43:34.722
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[WARN] 2021-07-31 21:43:34.733
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[ERROR] 2021-07-31 21:43:34.734
org.apache.dolphinscheduler.server.master.consumer.TaskPriorityQueueConsumer:[154]
- dispatch error: fail to execute : Command [type=TASK_EXECUTE_REQUEST,
opaque=84572, bodyLen=2041] due to no suitable worker, current task needs
worker group default to execute
[WARN] 2021-07-31 21:43:34.742
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[WARN] 2021-07-31 21:43:34.744
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[WARN] 2021-07-31 21:43:34.756
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[WARN] 2021-07-31 21:43:34.792
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.37G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
[WARN] 2021-07-31 21:43:34.872
org.apache.dolphinscheduler.common.utils.OSUtils:[442] - current cpu load
average 10.87 is too high or available memory 0.38G is too low, under
max.cpuload.avg=8.0 and reserved.memory=0.3G
</code></pre>
**Which version of DolphinScheduler:**
-[1.3.6]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
