o-nikolas commented on code in PR #29147:
URL: https://github.com/apache/airflow/pull/29147#discussion_r1086215221
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -792,28 +793,31 @@ def try_adopt_task_instances(self, tis:
Sequence[TaskInstance]) -> Sequence[Task
}
pod_list = self._list_pods(query_kwargs)
for pod in pod_list:
- self.adopt_launched_task(kube_client, pod, pod_ids)
+ self.adopt_launched_task(kube_client, pod, tis_to_flush_by_key)
self._adopt_completed_pods(kube_client)
- tis_to_flush.extend(pod_ids.values())
+ tis_to_flush.extend(tis_to_flush_by_key.values())
return tis_to_flush
def adopt_launched_task(
- self, kube_client: client.CoreV1Api, pod: k8s.V1Pod, pod_ids:
dict[TaskInstanceKey, k8s.V1Pod]
+ self,
+ kube_client: client.CoreV1Api,
+ pod: k8s.V1Pod,
+ tis_to_flush_by_key: dict[TaskInstanceKey, k8s.V1Pod],
Review Comment:
This method is in the `KubernetesExecutor` class proper so I think squarely
lands in public land :grimacing:
##########
airflow/executors/kubernetes_executor.py:
##########
@@ -353,12 +353,12 @@ def run_next(self, next_job: KubernetesJobType) -> None:
self.run_pod_async(pod, **self.kube_config.kube_client_request_args)
self.log.debug("Kubernetes Job created!")
- def delete_pod(self, pod_id: str, namespace: str) -> None:
- """Deletes POD."""
+ def delete_pod(self, pod_name: str, namespace: str) -> None:
Review Comment:
This is an interesting grey area. We say in the new public interface docs
that executors are public and we even encourage extending them:
> Airflow has a set of Executors that are considered public. You are free to
extend their functionality
But this is not the `KubernetesExecutor` class itself that you're modifying
it's the job watcher singleton (which is indirectly used by the executor class
via the scheduler class, which is itself passed in as a parameter to the
executor).
So I could see this being a private class who's interface we don't guarantee
to be consistent release to release.
I'm interested to hear what others think though.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
