[spark] branch branch-2.4 updated: [SPARK-31456][CORE] Fix shutdown hook priority edge cases

[dongjoon]

This is an automated email from the ASF dual-hosted git repository.

dongjoon pushed a commit to branch branch-2.4
in repository https://gitbox.apache.org/repos/asf/spark.git


The following commit(s) were added to refs/heads/branch-2.4 by this push:
     new b28c1fb  [SPARK-31456][CORE] Fix shutdown hook priority edge cases
b28c1fb is described below

commit b28c1fb99f1c0d6252321e4052f47169f43abc75
Author: oleg <o...@nexla.com>
AuthorDate: Mon May 11 13:10:39 2020 -0700

    [SPARK-31456][CORE] Fix shutdown hook priority edge cases
    
    ### What changes were proposed in this pull request?
    Fix application order for shutdown hooks for the priorities of 
Int.MaxValue, Int.MinValue
    
    ### Why are the changes needed?
    The bug causes out-of-order execution of shutdown hooks if their priorities 
were Int.MinValue or Int.MaxValue
    
    ### Does this PR introduce _any_ user-facing change?
    No
    
    ### How was this patch tested?
    Added a test covering the change.
    
    Closes #28494 from oleg-smith/SPARK-31456_shutdown_hook_priority.
    
    Authored-by: oleg <o...@nexla.com>
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
    (cherry picked from commit d7c3e9e53e01011f809b6cb145349ee8a9c5e5f0)
    Signed-off-by: Dongjoon Hyun <dongj...@apache.org>
---
 core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala | 4 +---
 core/src/test/scala/org/apache/spark/util/UtilsSuite.scala          | 6 +++++-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git 
a/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala 
b/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
index b702838..859da72 100644
--- a/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
+++ b/core/src/main/scala/org/apache/spark/util/ShutdownHookManager.scala
@@ -209,9 +209,7 @@ private [util] class SparkShutdownHookManager {
 private class SparkShutdownHook(private val priority: Int, hook: () => Unit)
   extends Comparable[SparkShutdownHook] {
 
-  override def compareTo(other: SparkShutdownHook): Int = {
-    other.priority - priority
-  }
+  override def compareTo(other: SparkShutdownHook): Int = 
other.priority.compareTo(priority)
 
   def run(): Unit = hook()
 
diff --git a/core/src/test/scala/org/apache/spark/util/UtilsSuite.scala 
b/core/src/test/scala/org/apache/spark/util/UtilsSuite.scala
index 8b959ab..00f96b9 100644
--- a/core/src/test/scala/org/apache/spark/util/UtilsSuite.scala
+++ b/core/src/test/scala/org/apache/spark/util/UtilsSuite.scala
@@ -702,10 +702,14 @@ class UtilsSuite extends SparkFunSuite with 
ResetSystemProperties with Logging {
     manager.add(3, () => output += 3)
     manager.add(2, () => output += 2)
     manager.add(4, () => output += 4)
+    manager.add(Int.MinValue, () => output += Int.MinValue)
+    manager.add(Int.MinValue, () => output += Int.MinValue)
+    manager.add(Int.MaxValue, () => output += Int.MaxValue)
+    manager.add(Int.MaxValue, () => output += Int.MaxValue)
     manager.remove(hook1)
 
     manager.runAll()
-    assert(output.toList === List(4, 3, 2))
+    assert(output.toList === List(Int.MaxValue, Int.MaxValue, 4, 3, 2, 
Int.MinValue, Int.MinValue))
   }
 
   test("isInDirectory") {


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org
For additional commands, e-mail: commits-h...@spark.apache.org