potiuk commented on issue #17735: URL: https://github.com/apache/airflow/issues/17735#issuecomment-912128633
Yeah. We have not looked at it yet, but I think this problem has a "deployment" proposal rather than "airflow" one. Simply the "other user" that you run should be able to write to the file that is created by Airflow. Maybe we should make it clear in the documentation for "run as user" that the users should be configured this way. Using Linux access right it is not easy to do it in "general" way from Python code in a secure way, without having prior knowledge about the users involved or without being sure that we can use Access Control Lists (which are not available on all file systems/not enabled in all Linux distros, versions etc.). We should make a secure solution that is based on standard POSIX access right scheme (with user, group other) . A solution that I have in mind (and one that is compatible with the way how "Official Airflow Image" is implemented), is to base it on umask and groups. In the official image, umask is set to 0002 - specifically to make any files and folders created, accessible (including write) to the primary group the user belongs: https://github.com/apache/airflow/blob/main/scripts/in_container/prod/entrypoint_prod.sh#L252 - and we have a stric requirement that ANY user that runs airflow should have group 0 ('root') as primary group. The 'airflow' user in the image has group '0' (root) set as default group. This is in order to make the image OpenShift compatible and allow the docker container to run as "any" user (it does not have to be "airflow" as long as it has group set to '0'. More about it here: https://airflow.apache.org/docs/docker-stack/entrypoint.html#allowing-arbitrary-user-to-run-the-container My proposal is - let's make it a pre-requisite that both `airflow` and the 'me` user have both same primary group (might be set to '0' as it is in the docker image) and that airflow sets 'umask` to 0002 allowing anyone from it's primary group to write to any folders/files created by the 'airflow` user. It does not open security issues (all the necessary `root` user permission are user-based not group-based) and since umask will be set only for airflow, it does not also affect he root-created files and folders, but at the same time all the logs/files/folders that airflow created, become writable for `me` user . WDYT? Would it be applicable to your cases? Any concerns you could have with this approach @kcphila @Hrithiksomani ? Why it's not a solution that is "super-generic" I think it is workable and can be applied to your systems even now - without upgrading Airflow even. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
