jscheffl commented on code in PR #45121:
URL: https://github.com/apache/airflow/pull/45121#discussion_r1895054558
##########
task_sdk/src/airflow/sdk/api/client.py:
##########
@@ -263,6 +267,14 @@ def noop_handler(request: httpx.Request) -> httpx.Response:
return httpx.Response(200, json={"text": "Hello, world!"})
+# Config options for SDK how retries on HTTP requests should be handled
+# Note: Given defaults make attempts after 1, 3, 7, 15, 31seconds, 1:03, 2:07,
3:37 and fails after 5:07min
+# As long as there is no other config facility in SDK we use ENV for the moment
+API_RETRIES = int(os.getenv("AIRFLOW__WORKERS__API_RETRIES", 10))
+API_RETRY_WAIT_MIN = int(os.getenv("AIRFLOW__WORKERS__API_RETRY_WAIT_MIN", 1))
+API_RETRY_WAIT_MAX = int(os.getenv("AIRFLOW__WORKERS__API_RETRY_WAIT_MAX", 90))
+
Review Comment:
Good points. But as in the comments above it is referring to a "proper
config mechanism" which is missing. Would leave it up there.
The values at the end are passed to tenacity, even there in the logic it
seems to be no validation. If you "mess-up" the values then it will use the max
as minimum it seems. Not too bad.
For me it is okay like this, except if @kaxil or @ashb would also enforce me
to make it bullet-proof here. My aim was that it is working by default but that
the defaults can be over-ridden. No "official" config but something that can be
tweaked if you are looking into code.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
