davlum commented on a change in pull request #6643: [AIRFLOW-6040] Fix
KubernetesJobWatcher Read time out error
URL: https://github.com/apache/airflow/pull/6643#discussion_r350850305
##########
File path: airflow/executors/kubernetes_executor.py
##########
@@ -265,12 +265,12 @@ def run(self):
def _run(self, kube_client, resource_version, worker_uuid, kube_config):
self.log.info(
- 'Event: and now my watch begins starting at resource_version: %s',
- resource_version
- )
+ 'Event: and now my watch begins starting at resource_version: %s, '
+ 'worker_uuid: %s', resource_version, worker_uuid)
watcher = watch.Watch()
- kwargs = {'label_selector': 'airflow-worker={}'.format(worker_uuid)}
+ kwargs = {'label_selector': 'airflow-worker={}'.format(worker_uuid),
+ 'timeout_seconds': 50}
Review comment:
I believe the correct argument name here is `_request_timeout`. I can't link
the generated python API file as it is too large for github, but it's on line
6141 of
https://github.com/kubernetes-client/python/blob/master/kubernetes/client/api/core_v1_api.py.
[This doc link for the is kube_client_request_args
dead](https://github.com/apache/airflow/blob/df35957fd4bab86ca6d2c923899569fb23139f56/airflow/config_templates/default_airflow.cfg#L786).
It also states;
> List of supported params in **kwargs are similar for all core_v1_apis,
hence a single config variable for all apis
I feel like this is the wrong approach, as these setting should be
configurable on a per request basis, but that's another matter and much more
complex. For example [this label_selector
argument](https://github.com/apache/airflow/blob/e51e1c770dad235b3fd8fdc330e44b83df8dcc4a/airflow/executors/kubernetes_executor.py#L273)
would fail if passed to the `create_namespaced_pod` function.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
