ashb opened a new pull request, #51934: URL: https://github.com/apache/airflow/pull/51934
When we are running normally (without impersonation) then before we fork the supervisor sets up a new socketpair for logging before forking, and then configures structlog in the forked process to send logs over that socket. However sudo by default will close all open file descriptors other than stdin, stdout and stderr, so our logs socket (sockets, and files, are all file descriptors). We could ask people to change their `sudoers` config file to add the [`closefrom_overide`][1] and invoke `sudo -C <logfd>` however many people either might not have access to do this, or might not feel comfortable in making this change. There is however another option to us: On both unix and windows there is the ability to pass _open_ file descriptors (which remember, sockets are file descriptors) between two processes! So what this PR does is introduce a new Request and Response pair, and customize the send+receive code to send a new FD (since we've already closed the child end for normal start up before we knew the task was actually going to run as another user, and we can't get it back, so we just open another) that is configured to receive and handle JSON logs. Relates to #51780 Enough with the words, lets see what it does. ### Logging from a task *Before* <img width="1258" alt="before-logging-in-task" src="https://github.com/user-attachments/assets/0fbb7f1d-1c2b-490f-9cf2-86bf80ff2057"; /> Note the double timestamp and level etc (one formatted nicely by the UI, the other in the log message etc) *After* <img width="1298" alt="after-logging-in-task" src="https://github.com/user-attachments/assets/d771df55-2b3b-498a-a094-87aa91e66d71"; /> The two highlighted sections are the same source in each task run ### Unhandled exception [1]: https://linux.die.net/man/5/sudoers#:~:text=on%20by%20default.-,closefrom_override,is%20off%20by%20default.,-compress_io'%20If%20set Now on to the real "ick". An unhandled exception in an operator/task: *Before* <img width="1534" alt="before-uncaught-exc" src="https://github.com/user-attachments/assets/518c45c7-98f7-49d4-b0b4-a6d2bc9321f5"; /> Ick. Not helped by rich at all here *After* <img width="1168" alt="after-uncaught-exc" src="https://github.com/user-attachments/assets/58484e6e-74f1-4c00-81b3-a1a64cabe295"; /> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
