Taragolis commented on code in PR #28464:
URL: https://github.com/apache/airflow/pull/28464#discussion_r1052434511
##########
airflow/providers/amazon/aws/hooks/emr.py:
##########
@@ -126,14 +126,21 @@ def create_job_flow(self, job_flow_overrides: dict[str,
Any]) -> dict[str, Any]:
return response
def add_job_flow_steps(
- self, job_flow_id: str, steps: list[dict] | str | None = None,
wait_for_completion: bool = False
+ self,
+ job_flow_id: str,
+ steps: list[dict] | str | None = None,
+ wait_for_completion: bool = False,
+ waiter_delay: int = 5,
+ waiter_max_attempts: int = 100,
Review Comment:
Hmm.. what do you think if by default `waiter_delay` and
`waiter_max_attempts` set to `None` and use in waiter config only if this
values not None?
In this case default values from `botocore` waiters definition file will
use: delay=30, max_attempts=60.
Something like that
```python
waiter.wait(
ClusterId=job_flow_id,
StepId=step_id,
WaiterConfig=trim_none_values(
{
"Delay": waiter_delay,
"MaxAttempts": waiter_max_attempts,
}
),
)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
