attilapiros commented on a change in pull request #35085:
URL: https://github.com/apache/spark/pull/35085#discussion_r831215372
##########
File path: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala
##########
@@ -94,7 +98,16 @@ private[spark] class DiskBlockManager(
} else {
val newDir = new File(localDirs(dirId), "%02x".format(subDirId))
if (!newDir.exists()) {
- Files.createDirectory(newDir.toPath)
+ val path = newDir.toPath
+ Files.createDirectory(path)
+ if (shuffleServiceRemoveShuffleEnabled) {
+ // SPARK-37618: Create dir as group writable so files within can
be deleted by the
+ // shuffle service in a secure setup. This will remove the setgid
bit so files created
+ // within won't be created with the parent folder group.
+ val currentPerms = Files.getPosixFilePermissions(path)
+ currentPerms.add(PosixFilePermission.GROUP_WRITE)
+ Files.setPosixFilePermissions(path, currentPerms)
Review comment:
@Ngone51 If the permissions of dirs needed to be corrected for shuffle
files I expect the same correction is needed for disk persisted RDDs too.
As exactly the same code is running when the directory is created for the
disk persisted RDDs I suggest to extend the conditions of this `if` to cover
that case too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
