wuyi created SPARK-36543:
----------------------------
Summary: Decommission logs too frequent when waiting migration to
finish
Key: SPARK-36543
URL: https://issues.apache.org/jira/browse/SPARK-36543
Project: Spark
Issue Type: Sub-task
Components: Spark Core
Affects Versions: 3.1.0, 3.2.0, 3.3.0
Reporter: wuyi
{code:java}
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:31 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:32 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:33 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:34 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:35 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:36 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:37 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:38 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:39 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:40 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:41 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:42 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:43 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: Checking to see if we can
shutdown.
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: No running tasks, checking
migrations
21/08/18 08:14:44 INFO CoarseGrainedExecutorBackend: All blocks not yet
migrated.
...{code}
It takes some time to migrate data (shuffle or rdd). Logging per second is too
frequent.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
