eladkal commented on PR #23861: URL: https://github.com/apache/airflow/pull/23861#issuecomment-1134580999
> Isn't this already achievable using the "Cluster Policy" hooks? I'm wary of "config sprawl" if we add lots of options. Cluster policy has weakness that it's not exposed to the end user (DAG author). When policy applied the DAG author is not aware of it. Even if you add logs in the policy it won't appear in the task log. Furthermore in task details you will see the original value (as defined by DAG author) and not the value changed by the policy. It's very hard to debug/understand something when it changed externally with no indication. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
