[spark] branch branch-3.2 updated: [SPARK-36614][CORE][UI] Correct executor loss reason caused by decommission in UI

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-3.2 by this push:
     new 8be53c3  [SPARK-36614][CORE][UI] Correct executor loss reason caused 
by decommission in UI
8be53c3 is described below

commit 8be53c3298c41c749663bc1dbbd5b2d65246c854
Author: yi.wu <yi...@databricks.com>
AuthorDate: Mon Aug 30 09:09:22 2021 -0700

    [SPARK-36614][CORE][UI] Correct executor loss reason caused by decommission 
in UI
    
    ### What changes were proposed in this pull request?
    
    Post the correct executor loss reason to UI.
    
    ### Why are the changes needed?
    
    To show the accurate loss reason.
    
    ### Does this PR introduce _any_ user-facing change?
    
    Yes. Users can see the difference from the UI.
    
    Before:
    <img width="509" alt="WeChataad8d1f27d9f9aa7cf93ced4bcc820e2" 
src="https://user-images.githubusercontent.com/16397174/131341692-6f412607-87b8-405e-822d-0d28f07928da.png";>
    <img width="1138" alt="WeChat13c9f1345a096ff83d193e4e9853b165" 
src="https://user-images.githubusercontent.com/16397174/131341699-f2c9de09-635f-49df-8e27-2495f34276c0.png";>
    
    After:
    
    <img width="599" alt="WeChata4313fa2dbf27bf2dbfaef5c1d4a19cf" 
src="https://user-images.githubusercontent.com/16397174/131341754-e3c93b5d-5252-4006-a4cc-94d76f41303b.png";>
    <img width="1182" alt="WeChat5559d52fd3070ae6c42fe32d56f9dc94" 
src="https://user-images.githubusercontent.com/16397174/131341761-e1e0644f-1e76-49c0-915a-26aad77ec272.png";>
    
    ### How was this patch tested?
    
    Manully tested.
    
    Closes #33868 from Ngone51/fix-executor-remove-reason.
    
    Authored-by: yi.wu <yi...@databricks.com>
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
    (cherry picked from commit ebe7bb62176ac3c29b0c238e411a0dc989371c33)
    Signed-off-by: Dongjoon Hyun <dh...@apple.com>
---
 .../spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala       | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git 
a/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
 
b/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
index 3c121c5..fd68dfa 100644
--- 
a/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
+++ 
b/core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala
@@ -428,8 +428,8 @@ class CoarseGrainedSchedulerBackend(scheduler: 
TaskSchedulerImpl, val rpcEnv: Rp
           totalCoreCount.addAndGet(-executorInfo.totalCores)
           totalRegisteredExecutors.addAndGet(-1)
           scheduler.executorLost(executorId, lossReason)
-          listenerBus.post(
-            SparkListenerExecutorRemoved(System.currentTimeMillis(), 
executorId, reason.toString))
+          listenerBus.post(SparkListenerExecutorRemoved(
+            System.currentTimeMillis(), executorId, lossReason.toString))
         case None =>
           // SPARK-15262: If an executor is still alive even after the 
scheduler has removed
           // its metadata, we may receive a heartbeat from that executor and 
tell its block

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org