johnhoran commented on code in PR #44357:
URL: https://github.com/apache/airflow/pull/44357#discussion_r1886918542
##########
providers/src/airflow/providers/cncf/kubernetes/callbacks.py:
##########
@@ -83,7 +123,34 @@ def on_pod_completion(*, pod: k8s.V1Pod, client:
client_type, mode: str, **kwarg
pass
@staticmethod
- def on_pod_cleanup(*, pod: k8s.V1Pod, client: client_type, mode: str,
**kwargs):
+ def on_pod_wrapup(
+ *,
+ pod: k8s.V1Pod,
+ client: client_type,
+ mode: str,
+ operator: KubernetesPodOperator,
+ context: Context,
+ **kwargs,
+ ) -> None:
+ """
+ Invoke this callback after all pod completion callbacks but before the
pod is deleted.
+
+ :param pod: the completed pod.
+ :param client: the Kubernetes client that can be used in the callback.
+ :param mode: the current execution mode, it's one of (`sync`, `async`).
+ """
+ pass
Review Comment:
Sorry if I'm not being clear. The way I've been using this so far is that I
have one class that extends off the `KubernetesPodOperatorCallback` and does
the insertion of a sidecar `on_pod_manifest_created`, killing the sidecar
`on_pod_wrapup`, and some code to ensure that the sidecar is only added/killed
once. Then extending off that I have a bunch of other classes that are
responsible for doing some actual work with the sidecar, in my case I pull DBT
artifacts in the `on_pod_completion` method, then kill the pod in
`on_pod_wrapup` before extracting dataset events from DBT artifacts and also
uploading them to S3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
