HeartSaVioR commented on a change in pull request #27004: [SPARK-30348][CORE]
Fix flaky test failure on "MasterSuite.SPARK-27510: Master should avoid ..."
URL: https://github.com/apache/spark/pull/27004#discussion_r361342487
##########
File path: core/src/test/scala/org/apache/spark/deploy/master/MasterSuite.scala
##########
@@ -97,13 +97,40 @@ class MockWorker(master: RpcEndpointRef, conf: SparkConf =
new SparkConf) extend
}
}
-class MockExecutorLaunchFailWorker(master: RpcEndpointRef, conf: SparkConf =
new SparkConf)
- extends MockWorker(master, conf) {
+// This class is designed to handle the lifecycle of only one application.
+class MockExecutorLaunchFailWorker(master: Master, conf: SparkConf = new
SparkConf)
+ extends MockWorker(master.self, conf) with Eventually {
+
+ val appRegistered = new CountDownLatch(1)
+ val launchExecutorReceived = new CountDownLatch(1)
+ val appIdsToLaunchExecutor = new mutable.HashSet[String]
var failedCnt = 0
+
override def receive: PartialFunction[Any, Unit] = {
+ case LaunchDriver(driverId, desc, resources_) =>
+ drivers += driverId
+ driverResources(driverId) = resources_.map(r => (r._1,
r._2.addresses.toSet))
+ master.self.send(RegisterApplication(appDesc, newDriver(driverId)))
+
+ // Below code doesn't make driver stuck, as newDriver opens another rpc
endpoint for
+ // handling driver related messages. It guarantees registering
application is done
+ // before handling LaunchExecutor message.
+ eventually(timeout(10.seconds)) {
Review comment:
There're two different event dispatchers in MockExecutorLaunchFailWorker;
'worker' and 'driver' (once it receives `LaunchDriver` message) which we
shouldn't assume messages will be handled sequentially across event
dispatchers, LaunchExecutor and RegisteredApplication in this case.
That's why I just inject verification code here; this would block handling
LaunchExecutor until we verify application registering is done successfully.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
