LuciferYang commented on code in PR #36611:
URL: https://github.com/apache/spark/pull/36611#discussion_r878325387
##########
core/src/main/scala/org/apache/spark/util/Utils.scala:
##########
@@ -308,28 +308,7 @@ private[spark] object Utils extends Logging {
* newly created, and is not marked for automatic deletion.
*/
def createDirectory(root: String, namePrefix: String = "spark"): File = {
- var attempts = 0
- val maxAttempts = MAX_DIR_CREATION_ATTEMPTS
- var dir: File = null
- while (dir == null) {
- attempts += 1
- if (attempts > maxAttempts) {
- throw new IOException("Failed to create a temp directory (under " +
root + ") after " +
- maxAttempts + " attempts!")
- }
- try {
- dir = new File(root, namePrefix + "-" + UUID.randomUUID.toString)
- // SPARK-35907:
- // This could throw more meaningful exception information if directory
creation failed.
- Files.createDirectories(dir.toPath)
- } catch {
- case e @ (_ : IOException | _ : SecurityException) =>
- logError(s"Failed to create directory $dir", e)
- dir = null
- }
- }
-
- dir.getCanonicalFile
+ JavaUtils.createDirectory(root, namePrefix)
Review Comment:
In order to solve
```
Avoid the use of Guava's Files.createTempDir() in Spark code due to
[CVE-2020-8908](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2020-8908)
```
I added two compatible methods in `JavaUtils`.
> Hm, why did we not just implement the logic here? rather than use
JavaUtils? I forget if there was a reason
Code in `network-common` and `network-shuffle` module cannot call
`Utils.createDirectory ` or `Utils. createTempDir` in `core` module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
