ramitkataria commented on code in PR #76571:
URL: https://github.com/apache/airflow/pull/76571#discussion_r4610810388
##########
task-sdk/src/airflow/sdk/api/client.py:
##########
@@ -1256,11 +1296,18 @@ def noop_handler(request: httpx2.Request) ->
httpx2.Response:
API_SSL_CERT_PATH = conf.get("api", "ssl_cert")
API_SSL_CA_FILE_PATH = conf.get("api", "ssl_ca_file", fallback=None)
API_TIMEOUT = conf.getfloat("workers", "execution_api_timeout")
+# The callback token exchange waits for one answer as long as the usual
retries would wait on timeouts.
+API_CALLBACK_RUN_READ_TIMEOUT = API_TIMEOUT * max(API_RETRIES, 1)
API_CLIENT_SSL_CERT = conf.get("api", "client_ssl_cert", fallback=None)
API_CLIENT_SSL_KEY = conf.get("api", "client_ssl_key", fallback=None)
API_CLIENT_USE_PUBLIC_CERTS = conf.getboolean("api",
"client_use_public_certs", fallback=True)
+def _callback_run_stop() -> stop_base:
+ """Make at least the usual number of attempts, and keep going while the
server is down."""
+ return stop_after_attempt(API_RETRIES) &
stop_after_delay(API_RETRY_WAIT_MAX)
Review Comment:
Following up on
https://github.com/apache/airflow/pull/76571#discussion_r4521372641, one small
thing: the 25 s read timeout matches what the normal retries spend on timeouts,
but read timeouts are retried and `_callback_run_stop()` still needs
`API_RETRIES` attempts. So a server that accepts the connection and never
answers would hold the worker for about 5 x 25 s = 125 s plus backoff. Is that
worst case okay for the stopgap?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]