vandonr-amz commented on PR #29522:
URL: https://github.com/apache/airflow/pull/29522#issuecomment-1476972792
> If I set `CONTAINER_OVERRIDES` to any value rather than `None` (even
`{}`), I've got:
>
> ```
> botocore.errorfactory.ClientException: An error occurred (ClientException)
when calling the SubmitJob operation: Container overrides and node overrides
are mutually exclusive, only one can be set.
> ```
do you mean while still keeping `NODE_OVERRIDES` set ? That'd be normal, and
the error message explains it. If you want to use containers override, you need
to unset the node override.
> If I set `ARRAY_PROPERTIES` than I've got
>
> ```
> botocore.errorfactory.ClientException: An error occurred (ClientException)
when calling the SubmitJob operation: Multinode Array Job not supported.
> ```
Same story here, boto is telling you that you cannot set `NODE_OVERRIDES`
(which implies a multi-nodes job) and `ARRAY_PROPERTIES` at the same time,
though in a less clear way.
I'm not super familiar with batch jobs, but I think the valid combinations
are:
- container_overrides
- container_overrides + array_properties
- node_overrides
I haven't tested the array properties, but container and node overrides both
work well when not mixed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
