shahidki31 opened a new pull request #24398: [SPARK-27468][Core][WEBUI] BlockUpdate replication event shouldn't overwrite storage level description in the UI URL: https://github.com/apache/spark/pull/24398 ## What changes were proposed in this pull request? Test steps to reproduce this: 1) bin/spark-shell local-cluster[2,1,1024] ``` scala> import org.apache.spark.storage.StorageLevel scala> val rdd = sc.parallelize(1 to 10, 1).persist(StorageLevel.MEMORY_ONLY_2) scala> rdd.count ``` Events generated are shown like below ``` event: SparkListenerBlockUpdated(BlockUpdatedInfo(BlockManagerId(1, 10.8.132.160, 65473, None),rdd_0_0,StorageLevel(memory, deserialized, 2 replicas),56,0)) event: SparkListenerBlockUpdated(BlockUpdatedInfo(BlockManagerId(0, 10.8.132.160, 65474, None),rdd_0_0,StorageLevel(memory, deserialized, 1 replicas),56,0)) ``` But in the UI, in the storage tab it displays in the description like, "Memory Deserialized 1x Replicated", even though we have given replication as 2. The root cause is that, the replication block update events will have replication factor 1. Hence in the AppStatusListener class, we overwrite whatever event comes later. If the replication event comes later, then we update replication factor as 1. In the PR, I am fixing from the AppStatusListener class side, as we need to detect if the event is replication or not. Else we need to update the rdd store. ## How was this patch tested? Added UT and Manually tested. Before patch:  After patch: 
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
