mridulm commented on a change in pull request #35085:
URL: https://github.com/apache/spark/pull/35085#discussion_r795954394
##########
File path: core/src/main/scala/org/apache/spark/storage/DiskBlockManager.scala
##########
@@ -94,7 +95,13 @@ private[spark] class DiskBlockManager(
} else {
val newDir = new File(localDirs(dirId), "%02x".format(subDirId))
if (!newDir.exists()) {
- Files.createDirectory(newDir.toPath)
+ // SPARK-37618: Create dir as group writable so files within can be
deleted by the
+ // shuffle service
+ val path = newDir.toPath
+ Files.createDirectory(path)
+ val currentPerms = Files.getPosixFilePermissions(path)
+ currentPerms.add(PosixFilePermission.GROUP_WRITE)
Review comment:
Given we are using this for directories, and not files, the required
functionality should work ?
Did you observe failure when running it ?
On my linux box it still honors if I specify umask of 0027.
```
$ rm -rf t1; umask 0027; mkdir t1 ; chgrp mridul t1; chmod g+s t1; mkdir
-m770 t1/t2; mkdir -m770 t1/t2/t3; cp ~/.bashrc t1/t2/t3/bashrc
$ find t1 | xargs ls -al
-rw-r----- 1 test mridul 3771 Jan 31 12:33 t1/t2/t3/bashrc
t1:
total 12
drwxr-s--- 3 test mridul 4096 Jan 31 12:33 .
drwxr-xr-x 3 test test 4096 Jan 31 12:33 ..
drwxrws--- 3 test mridul 4096 Jan 31 12:33 t2
t1/t2:
total 12
drwxrws--- 3 test mridul 4096 Jan 31 12:33 .
drwxr-s--- 3 test mridul 4096 Jan 31 12:33 ..
drwxrws--- 2 test mridul 4096 Jan 31 12:33 t3
t1/t2/t3:
total 12
drwxrws--- 2 test mridul 4096 Jan 31 12:33 .
drwxrws--- 3 test mridul 4096 Jan 31 12:33 ..
-rw-r----- 1 test mridul 3771 Jan 31 12:33 bashrc
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
