[
https://issues.apache.org/jira/browse/SPARK-39324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-39324:
----------------------------------
Parent: SPARK-41515
Issue Type: Sub-task (was: Improvement)
> Log ExecutorDecommission as INFO level in TaskSchedulerImpl
> -----------------------------------------------------------
>
> Key: SPARK-39324
> URL: https://issues.apache.org/jira/browse/SPARK-39324
> Project: Spark
> Issue Type: Sub-task
> Components: Spark Core
> Affects Versions: 3.4.0
> Reporter: Dongjoon Hyun
> Assignee: Dongjoon Hyun
> Priority: Major
> Fix For: 3.4.0
>
>
> Like the other module, `TaskSchedulerImpl` should log the decommission as
> `INFO` level.
> {code}
> 22/05/28 01:25:28 INFO KubernetesClusterSchedulerBackend: Decommission
> executors: 8
> 22/05/28 01:25:28 INFO KubernetesClusterSchedulerBackend: Notify executor 8
> to decommissioning.
> 22/05/28 01:25:28 INFO BlockManagerMasterEndpoint: Mark BlockManagers
> (BlockManagerId(8, 100.103.40.13, 43353, None)) as being decommissioning.
> 22/05/28 01:25:29 ERROR TaskSchedulerImpl: Lost executor 8 on 100.103.40.13:
> Executor decommission.
> 22/05/28 01:25:29 INFO ExecutorMonitor: Executor 8 is removed. Remove reason
> statistics: ...
> 22/05/28 01:25:29 INFO DAGScheduler: Executor lost: 8 (epoch 7)
> 22/05/28 01:25:29 INFO BlockManagerMasterEndpoint: Trying to remove executor
> 8 from BlockManagerMaster.
> 22/05/28 01:25:29 INFO BlockManagerMasterEndpoint: Removing block manager
> BlockManagerId(8, 100.103.40.13, 43353, None)
> 22/05/28 01:25:29 INFO BlockManagerMaster: Removed 8 successfully in
> removeExecutor
> 22/05/28 01:25:29 INFO DAGScheduler: Shuffle files lost for executor: 8
> (epoch 7)
> 22/05/28 01:25:34 INFO BlockManagerMaster: Removal of executor 8 requested
> 22/05/28 01:25:34 INFO BlockManagerMasterEndpoint: Trying to remove executor
> 8 from BlockManagerMaster.
> 22/05/28 01:25:34 INFO
> KubernetesClusterSchedulerBackend$KubernetesDriverEndpoint: Asked to remove
> non-existent executor 8
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
