vandonr-amz commented on PR #29522: URL: https://github.com/apache/airflow/pull/29522#issuecomment-1430343979
> @vandonr-amz just and idea not a strong opinion. What if we create separate Operator for multinode Job? `node_overrides` should conflicts with `container_overrides` (current `overrides`) and operator have a different logic for obtain logs, in additional BatchOperator use their own implementation of waiter (separate additional hook 🙄 ), I'm not sure is it would work with this changes or not. I'm not super fan of it, it's the same boto operation behind it, just with a different behavior... And as a user I think I would be surprised if I had to use a different operator for this. I think an operator should be "do a thing" and then the parameters should be about the specifics of how the thing should be done. I don't think lanching a batch job or a multinode batch job are really different "things", but it's up for debate I guess. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
