liupc commented on a change in pull request #23614: [SPARK-26689][CORE]Support
blacklisting bad disk directory and retry in DiskBlockManager
URL: https://github.com/apache/spark/pull/23614#discussion_r252627104
##########
File path:
core/src/test/scala/org/apache/spark/storage/DiskBlockManagerSuite.scala
##########
@@ -91,4 +91,24 @@ class DiskBlockManagerSuite extends SparkFunSuite with
BeforeAndAfterEach with B
for (i <- 0 until numBytes) writer.write(i)
writer.close()
}
+
+ test("test blacklisting bad disk directory") {
+ val blockId = new TestBlockId("1")
+ val hash = Utils.nonNegativeHash(blockId.name)
+ val (badDiskDir, goodDiskDir) = if (hash % 2 == 0) {
Review comment:
@attilapiros good suggestion, I have already update the UT and also fix a
bug: when getFile for block writen before disk corruption, it will return a
different result.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
