potiuk commented on PR #41329: URL: https://github.com/apache/airflow/pull/41329#issuecomment-2506166072
> @potiuk Why do we want to send a SIGHUP first? That seems like a very odd signal to send. SIGINT followed by SIGTERM is what Docker/podman/every other container runtime(?) does and feels like a much more normal sequence of signals. Especially given that SIGINT being sent to a python process will raise a KeyboardInterupt signal won't it, where as by default SIGHUP would just kill the process outright (as it doesn't have a default signal handler, so default behaviour is to just exit!) Yes, we could skip SIGHUP, but whatever we do we should clearly document our signal escalation policy. I am not very strong on it because I don't think there is a default and always followed path of escalation of signals. HUP/TERM/INT is what I've been using in the past for all kind of processes, because this is what most processes which have not been "deamon desgined" will normally experience in their lifecycle. Generally speaking docker sends just "TERM/KILL" sequence (without INT) - `docker stop` (but you can change the TERM by `--signal NN` or you can instruct docker to use another signal by declaring STOPSIGNAL in your Dockerfile or when you build or run your container you can override the default `TERM`: https://docs.docker.com/reference/cli/docker/container/stop/ - so they are really not doing signal escalation except very basic one. More on the why I proposed HUP/TERM/INT/KILL over TERM/INT/KILL: * HUP is a signal that is always sent when your terminal disconnects - so if you have any CLI that you run via AIRFLOW, they should be prepared to handle it as the first thing that happens. Most of the daemons will reload configuration when they receive HUP, most of the CLI programs will handle it exactly the way as if the user disconnected - if they are prepared for being disconnected, they will detach the stdin/stdout and stop writing to it basically (and few other things - switching to "non-interactive mode". If not then the programs will stop (and usually in some graceful way if possible). But all the apps generally **should** have a reasonable response to it. Also HUP is something that is not a deliberate action of the user - the idea behind HUP that this is really "what happens when your connection from terminal breakes" I think this is why Docker does not really send HUP, because almost by default does not send HUP is that by default there is no terminal connected to the running app (you have to run your container with `-t` to get the terminal connected. I think airlfow is in a bit different league though - many of the tasks our users run as "tasks" are really "interactive" CLIs wrapped in airflow tasks, rather than long runnign docker processes (which is the default mode of docker). But - not so strong about it (see INT below). * INT is the weak interruption that is sent to non-interactive programs (or those who switched from interactive to non-interactive mode. It's the same which is sent with Ctrl-C, which means that it is deliberate action of the user. And yes - I think what we have here is deliberate action from the "user" ("airflow worker" in this case). So maybe we indeed want to skip the HUP one. * TERM is regular, stronger kill request We could also consider QUIT after TERM * QUIT - quit is the "application misbehaves and we **really** want to quit it. The advantage of using it is that by default in c programs it should produce debuggable core-dump. Which is a good idea I think if the process did not respond in reasonable time for TERM. In case of docker, it makes little sense because generally such core-dump would be lost when container exits, so maybe that's why they are not using it. * KILL - of course, non-ignorable last resort signal So maybe what we really want is `INT/TERM/QUIT/KILL` ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
