dongjoon-hyun commented on code in PR #45661:
URL: https://github.com/apache/spark/pull/45661#discussion_r1535871592
##########
core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala:
##########
@@ -226,6 +226,24 @@ private[spark] class DiskBlockManager(
(blockId, getFile(blockId))
}
+ /** Produces a unique block id and File suitable for storing shuffled
intermediate results
+ * in the input directory. */
+ def createTempShuffleBlockInDir(fileDir: File): (TempShuffleBlockId, File) =
{
+ var blockId = TempShuffleBlockId(UUID.randomUUID())
+ var tmpFile = new File(fileDir, blockId.name)
+ while (tmpFile.exists()) {
+ blockId = TempShuffleBlockId(UUID.randomUUID())
+ tmpFile = new File(fileDir, blockId.name)
Review Comment:
If `fileDir` is invalid, we are in the `Infinite` loop, aren't we? It seems
that we need a safe guard to avoid the infinite loop, @wankunde . Also, please
add a unit test case for the invalid `fileDir` (maybe, nonExist)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
