karenbraganz commented on PR #61195: URL: https://github.com/apache/airflow/pull/61195#issuecomment-3818176743
I think the logic might be less confusing for users if you allow them to set both `wait_for_completion` and `wait_policy` without raising a ValueError. If `wait_for_completion` is set to True, the code would check `wait_policy`. If `wait_policy` is not set, it defaults to WAIT_FOR_COMPLETE. This shouldn't interfere with your logic too much because you are anyway setting `wait_for_completion` to True on line 711. If you want to use the current logic, I think you should clarify a few things in the param descriptions: 1. For `wait_for_completion` set to True, make it clear that the task will succeed once the cluster starts running and does not wait until termination. Use `wait_policy to change this behavior. 2. For `wait_policy` state that this should not be used unless `wait_for_termination` is set to False. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
