kcphila edited a comment on issue #17735: URL: https://github.com/apache/airflow/issues/17735#issuecomment-912470857
Hi @potiuk, thanks for the response. However, this would not work and is deeply problematic. This essentially requires us to completely change our entire user and group management policy... so that Airflow can create a temporary file to log error details for a few minutes at a time. Remember that this issue is solely about a temporary file. I just started setting up Airflow to replace existing cron jobs, and so I can't believe this is a realistic expectation. Here's a use case that may be useful. _User1 works with sensitive individual data about children in the foster case system. They need a task to run overnight to pull data from a data system only they have access to and put it somewhere on the filesystem that only their primary work group has access._ _User2 works with sensitive individual credit data. They need a task to run overnight to pull data from a data system only they have access to and put it somewhere on the filesystem that only their primary work group has access. They do not share groups with User1 and should not be able to see anything owned by User1 or their group._ _The data engineer that manages airflow sets up one task to run as User1 and one task to run as User2. Neither User1 nor User2 are or should be managing Airflow, and so they should not be in the airflow group._ Aside from the one temporary file highlighted by my post, Airflow handles the above model just fine, except for this temporary error file. So, I'm looking more closely at the code. The only other times this temporary file are used are to read the contents (line 105, which calls airflow.models.taskinstance.load_error_file, which just reads it once) and to close (which deletes it) (line 178). ..... and @ashb just commented and I see the issue #15947 is indeed the same - and fixed there. Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
