neils-dev opened a new pull request, #4027: URL: https://github.com/apache/ozone/pull/4027
## What changes were proposed in this pull request? An `ofs` tmp directory that is common and shared for all users of `ofs `applications requiring a temporary directory `/tmp`. Use cases include applications such as mapreduce and spark that make use of `/tmp` from the filesystem. Providing a common shared `/tmp` directory for `ofs `allows allows administrators to effectively configure and administer the directory for all users with enforceable access controls and quota policies. Currently, the `ofs://tmp` directory is a virtual directory configured by the admin but not shareable for all users. Instead each user creates their own `tmp `directory for use with `ofs`. Because of the implementation, each `tmp` directory can only be administered by the individual users. Administrators currently are unable to access user `tmp` directories. thus administer the `tmp` directories. As with [HDDS-2929](https://issues.apache.org/jira/browse/HDDS-2929) a virtual `tmp` directory is used by `ofs` users and `ofs` applications using `ofs://tmp`. Instead of each user having their own `tmp` directories in the implementation, here a single `tmp` directory is created from the `tmp` directory mount that is shared for all users. Admins have all access and users can read/write files and only delete files owned by the user. The `ofs` temp directory is configured with access control as follows: with, _**ozone.om.enable.ofs.shared.tmp.dir = true**_ in `ozone-site`. admin is privileged user `testuser2` (admin), regular user is `testuser` (user) in example with admin configuring ofs tmp directory mount for users: ``` ozone sh volume create tmp ozone sh volume setacl -a user:testuser2:rw,user:testuser:a,group:testuser2:rw,group:testuser:a tmp ozone sh bucket create tmp/tmp ozone sh bucket setacl -a user:testuser2:rwlc,user:testuser:a,group:testuser2:rwlc,group:testuser:a tmp/tmp ``` users access the tmp directory as in [HDDS-2929](https://issues.apache.org/jira/browse/HDDS-2929), `ozone fs -put ./NOTICE.txt ofs://om/tmp` ### Sticky-bit behavior ### Configured properly with access control, admin ALL, users R+W+L+C, the common `tmp `directory behaves like the sticky-bit directory of hdfs, [HADOOP-3953](https://issues.apache.org/jira/browse/HADOOP-3953) with files in `tmp` directory writable by all but files only deleted by owner and admin. ### Open Questions Q. What to do with the Trash service on the shared tmp directory, currently executing delete commands with `-skipTrash `option. ## What is the link to the Apache JIRA https://issues.apache.org/jira/browse/HDDS-7570 ## How was this patch tested? Tested by ofs integration tests, acceptance tests and manually tested with dev secure docker cluster. Integration test: `TestRootedOzoneFileSystem#testSharedTmpDir` Acceptance test: `robot --test "Test tmp mount for shared ofs tmp dir" ozone-secure-fs.robot` The test creates virtual `ofs://tmp` directory, sets access control and tests putting files into `tmp` dir with privileged user and regular user. Tests for sticky-bit behavior - all users write, file only deleted by root and owner of file. Manual testing: ``` $ cd hadoop-ozone/dist/target/ozone-1.3.0-SNAPSHOT/compose/ozonesecure add config property, OZONE-SITE.XML_ozone.om.enable.ofs.shared.tmp.dir=true, to docker-config file $ docker-compose up -d --scale datanode=3 $ docker-compose exec scm bash in scm console: # setup tmp mount and access control policies kinit -kt /etc/security/keytabs/testuser.keytab testuser/[email protected] bash-4.2$ ozone sh volume create tmp bash-4.2$ ozone sh volume setacl -a user:testuser2:rw,user:testuser:a,group:testuser2:rw,group:testuser:a tmp bash-4.2$ ozone sh bucket create tmp/tmp bash-4.2$ ozone sh bucket setacl -a user:testuser2:rwlc,user:testuser:a,group:testuser2:rwlc,group:testuser:a tmp/tmp # privileged user write to tmp bash-4.2$ ozone fs -put ./README.md ofs://om/tmp # regular user write to tmp bash-4.2$ kinit -kt /etc/security/keytabs/testuser2.keytab testuser2/[email protected] bash-4.2$ ozone fs -put ./LICENSE.txt ofs://om/tmp # regular user cannot rm README.md from tmp bash-4.2$ ozone fs -rm -skipTrash ofs://om/tmp/README.md ozone fs -rm -skipTrash ofs://om/tmp/LICENSE.txt kinit -kt /etc/security/keytabs/testuser.keytab testuser/[email protected] ozone -fs -rm -r -skipTrash ofs://om/tmp ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
