Folks, We need to support a use case here at Uber when service processes that don't respect SIGTERM signal and get killed after a default hardcoded preemption timeout of 1 minute during task kill or task restart. That significantly slows down upgrade workflow for such services. We'd like to control this timeout, essentially reducing it to 5-10 seconds.
My current thinking is to expose preemption_wait timeout class ThermosTaskRunner(TaskRunner): .... THERMOS_PREEMPTION_WAIT = Amount(1, Time.MINUTES) in thermos executor flags and set it in DefaultThermosTaskRunnerProvider eventually propagating to all ThermosRunner tasks. A proper fix would be probably something in the line of making this timeout configurable per task config but that would involve changing pystachio thermos schema. Thoughts? -Igor Morozov
