Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/6263#discussion_r44029924
--- Diff:
core/src/test/scala/org/apache/spark/storage/StorageStatusListenerSuite.scala
---
@@ -150,4 +158,12 @@ class StorageStatusListenerSuite extends FunSuite {
listener.onUnpersistRDD(SparkListenerUnpersistRDD(1))
assert(listener.executorIdToStorageStatus("big").numBlocks === 0)
}
+
+ test("Killed Executor Entry removed after configurable time") {
+ val localtestconf = new
SparkConf().set(StorageStatusListener.TIME_TO_EXPIRE_KILLED_EXECUTOR,"5s")
+ val listener = new StorageStatusListener(localtestconf)
+ listener.removedExecutorIdToStorageStatus.put("1", new
StorageStatus(null, 50))
+ Thread.sleep(5500)
--- End diff --
I think you can do something in between -- StorageStatusListener can have a
`private[storage]` constructor which takes the ticker, and the public one just
defaults it to the system ticker. Yes, you would not be testing *exactly* the
same behavior, but it tests the important parts.
Sleeping isn't the worst thing in this case -- often it leads to flaky
tests, though I don't think that would be the case here. Still, 5 seconds is
awfully long for this test when it should take a tiny fraction of that, and it
adds up over all the tests.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
