JzhJay opened a new issue, #58382:
URL: https://github.com/apache/airflow/issues/58382
### Apache Airflow version
Other Airflow 2/3 version (please specify below)
### If "Other Airflow 2/3 version" selected, which one?
2.8.4
### What happened?
Our k8sPodOperator task instances occasionally get stuck in Running state:
`
[2025-11-17, 08:32:48 CST] {pod_manager.py:466} INFO - [base] 20251117
08:32:48.378045 1 INFO _task.py:call_task_standard_function:145$ pid: 1560,
no_buy_list: 229, no_sell_list: 0
[2025-11-17, 08:32:48 CST] {pod_manager.py:466} INFO - [base] 20251117
08:32:48.537806 1 INFO _task.py:call_task_standard_function:145$ pid: 1605,
sid: 42644, black_list_trade_switches: 229
[2025-11-17, 08:32:48 CST] {pod_manager.py:466} INFO - [base] 20251117
08:32:48.677392 1 INFO _task.py:call_task_standard_function:145$ pid: 1605,
sid: 42643, black_list_trade_switches: 229
[2025-11-17, 08:32:52 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:32:57 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:02 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:07 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:12 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:17 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:23 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:28 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:33 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:38 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:43 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:48 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:53 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:33:58 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:03 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:08 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:13 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:18 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:23 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:28 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:33 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:38 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:43 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:48 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:53 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:34:59 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:04 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:09 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:14 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:19 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:24 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:29 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:34 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:39 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:44 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:49 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:54 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:35:59 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:04 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:09 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:14 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:19 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:24 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:29 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:34 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:39 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:45 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:50 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:36:55 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:00 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:05 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:10 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:15 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:20 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:25 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:30 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:35 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:40 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:45 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:48 CST] {pod_manager.py:483} INFO - [base] 20251117
08:32:48.677954 1 INFO _task.py:call_task_standard_function:145$ pid: 1605,
no_buy_list: 229, no_sell_list: 0
[2025-11-17, 08:37:48 CST] {rest.py:235} DEBUG - response body:
{"kind":"Pod","apiVersion":"v1","metadata":{"name":"portfolio-center-archive-task-archive-task-tlsaj6cs","namespace":"airflow-prod","uid":"f450696b-8850-4ff5-b770-11d24e300c0b","resourceVersion":"582326745","creationTimestamp":"2025-11-17T00:31:32Z","labels":{"airflow_kpo_in_cluster":"True","airflow_version":"2.8.4","app":"airflow","dag_id":"exec_platform.wy_algotrading__portfolio_center_0_8_57","env":"prod","kubernetes_pod_operator":"True","run_id":"scheduled__2025-11-13T0700000000-0cce263ff","task_id":"portfolio_center_archive_task_archive_task","try_number":"1"},"annotations":{"cni.projectcalico.org/containerID":"0d9ea05760f2672aff89a53b208c25cdce7f0af1fce297ff03613ee6d4cff7ab","cni.projectcalico.org/podIP":"100.64.15.176/32","cni.projectcalico.org/podIPs":"100.64.15.176/32"},"managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:31:32Z","fieldsType":"FieldsV1",
"fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:airflow_kpo_in_cluster":{},"f:airflow_version":{},"f:app":{},"f:dag_id":{},"f:env":{},"f:kubernetes_pod_operator":{},"f:run_id":{},"f:task_id":{},"f:try_number":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"base\"}":{".":{},"f:args":{},"f:command":{},"f:env":{".":{},"k:{\"name\":\"DAG_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"DATE\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"EXEC_REMOTE_SCRIPT\"}":{".":{},"f:name":{}},"k:{\"name\":\"EXEC_WARNING_LOG\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"FILE_COMMIT_HASH\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"NAME_TO_GREET\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TASK_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TZ\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"WY_EXEC_PLATFORM_MODE\"}":{".":{},"f:name":{},"f:value":{}}},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{".":{},"f:limits":{".":{},
"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}},"f:requests":{".":{},"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/dev/shm\"}":{".":{},"f:mountPath":{},"f:name":{}},"k:{\"mountPath\":\"/etc/group\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/passwd\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/shadow\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/scratch\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:runAsGroup":{},"f:runAsUser":{}},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"group\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"passwd\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},
"f:name":{}},"k:{\"name\":\"scratch\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shadow\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shm\"}":{".":{},"f:emptyDir":{".":{},"f:medium":{}},"f:name":{}}}}}},{"manager":"calico","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:05Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:cni.projectcalico.org/containerID":{},"f:cni.projectcalico.org/podIP":{},"f:cni.projectcalico.org/podIPs":{}}}},"subresource":"status"},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{"
.":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"100.64.15.176\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},"subresource":"status"}]},"spec":{"volumes":[{"name":"shadow","hostPath":{"path":"/etc/shadow","type":"File"}},{"name":"passwd","hostPath":{"path":"/etc/passwd","type":"File"}},{"name":"group","hostPath":{"path":"/etc/group","type":"File"}},{"name":"scratch","hostPath":{"path":"/scratch","type":"Directory"}},{"name":"shm","emptyDir":{"medium":"Memory"}},{"name":"kube-api-access-skkvb","projected":{"sources":[{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}},{"configMap":{"name":"kube-root-ca.crt","items":[{"key":"ca.crt","path":"ca.crt"}]}},{"downwardAPI":{"items":[{"path":"namespace","fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}]}}],"defaultMode":420}}],"containers":[{"name":"base","image":"harbor.wycluster.com/exec_
projects/portfolio_core:0.8.57","command":["/bin/bash","-c"],"args":["echo
\"run task($TASK_ID) in env($WY_EXEC_PLATFORM_MODE) on date($DATE), file hash:
$FILE_COMMIT_HASH\" \u0026\u0026 poetry run bash -c \"python -m
wy_archive_sdk.cmd run --path __init__.py --date 20251114 --variance task
--stage
PRODUCTION\""],"env":[{"name":"NAME_TO_GREET","value":"portfolio_center_archive_task_archive_task"},{"name":"TZ","value":"Asia/Shanghai"},{"name":"DAG_ID","value":"exec_platform.wy_algotrading__portfolio_center_0_8_57"},{"name":"TASK_ID","value":"portfolio_center_archive_task_archive_task"},{"name":"DATE","value":"20251114"},{"name":"WY_EXEC_PLATFORM_MODE","value":"production"},{"name":"EXEC_REMOTE_SCRIPT"},{"name":"FILE_COMMIT_HASH","value":"7efa7d6d520c9c9910b932317b5c0f89f04ad76e"},{"name":"EXEC_WARNING_LOG","value":"true"}],"resources":{"limits":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"},"requests":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"}},"volumeMounts":[{"name":"shad
ow","readOnly":true,"mountPath":"/etc/shadow"},{"name":"passwd","readOnly":true,"mountPath":"/etc/passwd"},{"name":"group","readOnly":true,"mountPath":"/etc/group"},{"name":"scratch","mountPath":"/scratch"},{"name":"shm","mountPath":"/dev/shm"},{"name":"kube-api-access-skkvb","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"datastack-node-114","securityContext":{"runAsUser":3021,"runAsGroup":3021},"imagePullSecrets":[{"name":"image-pull-secret"}],"affinity":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExe
cute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true,"preemptionPolicy":"PreemptLowerPriority"},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"}],"hostIP":"10.20.1.114","podIP":"100.64.15.176","podIPs":[{"ip":"100.64.15.176"}],"startTime":"2025-11-17T00:31:32Z","containerStatuses":[{"name":"base","state":{"running":{"startedAt":"2025-11-17T00:32:06Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"harbor.wycluster.com/exec_projects/portfolio_core:0.8.57","imageID":"docker-pullable://harbor.wycluster.com/exec_projects/portfolio_core@sha256:ea7f016b491fd1af8a
8be4af4617238e1175ae7145a7485240e850ce8dca10b9","containerID":"docker://3a4984cfb6cbe87e78ab7f363626fca2b20afed77420bd1f6aafd7f0f75bb08d","started":true}],"qosClass":"Guaranteed"}}
[2025-11-17, 08:37:49 CST] {rest.py:235} DEBUG - response body:
{"kind":"Pod","apiVersion":"v1","metadata":{"name":"portfolio-center-archive-task-archive-task-tlsaj6cs","namespace":"airflow-prod","uid":"f450696b-8850-4ff5-b770-11d24e300c0b","resourceVersion":"582326745","creationTimestamp":"2025-11-17T00:31:32Z","labels":{"airflow_kpo_in_cluster":"True","airflow_version":"2.8.4","app":"airflow","dag_id":"exec_platform.wy_algotrading__portfolio_center_0_8_57","env":"prod","kubernetes_pod_operator":"True","run_id":"scheduled__2025-11-13T0700000000-0cce263ff","task_id":"portfolio_center_archive_task_archive_task","try_number":"1"},"annotations":{"cni.projectcalico.org/containerID":"0d9ea05760f2672aff89a53b208c25cdce7f0af1fce297ff03613ee6d4cff7ab","cni.projectcalico.org/podIP":"100.64.15.176/32","cni.projectcalico.org/podIPs":"100.64.15.176/32"},"managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:31:32Z","fieldsType":"FieldsV1",
"fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:airflow_kpo_in_cluster":{},"f:airflow_version":{},"f:app":{},"f:dag_id":{},"f:env":{},"f:kubernetes_pod_operator":{},"f:run_id":{},"f:task_id":{},"f:try_number":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"base\"}":{".":{},"f:args":{},"f:command":{},"f:env":{".":{},"k:{\"name\":\"DAG_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"DATE\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"EXEC_REMOTE_SCRIPT\"}":{".":{},"f:name":{}},"k:{\"name\":\"EXEC_WARNING_LOG\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"FILE_COMMIT_HASH\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"NAME_TO_GREET\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TASK_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TZ\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"WY_EXEC_PLATFORM_MODE\"}":{".":{},"f:name":{},"f:value":{}}},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{".":{},"f:limits":{".":{},
"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}},"f:requests":{".":{},"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/dev/shm\"}":{".":{},"f:mountPath":{},"f:name":{}},"k:{\"mountPath\":\"/etc/group\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/passwd\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/shadow\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/scratch\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:runAsGroup":{},"f:runAsUser":{}},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"group\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"passwd\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},
"f:name":{}},"k:{\"name\":\"scratch\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shadow\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shm\"}":{".":{},"f:emptyDir":{".":{},"f:medium":{}},"f:name":{}}}}}},{"manager":"calico","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:05Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:cni.projectcalico.org/containerID":{},"f:cni.projectcalico.org/podIP":{},"f:cni.projectcalico.org/podIPs":{}}}},"subresource":"status"},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{"
.":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"100.64.15.176\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},"subresource":"status"}]},"spec":{"volumes":[{"name":"shadow","hostPath":{"path":"/etc/shadow","type":"File"}},{"name":"passwd","hostPath":{"path":"/etc/passwd","type":"File"}},{"name":"group","hostPath":{"path":"/etc/group","type":"File"}},{"name":"scratch","hostPath":{"path":"/scratch","type":"Directory"}},{"name":"shm","emptyDir":{"medium":"Memory"}},{"name":"kube-api-access-skkvb","projected":{"sources":[{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}},{"configMap":{"name":"kube-root-ca.crt","items":[{"key":"ca.crt","path":"ca.crt"}]}},{"downwardAPI":{"items":[{"path":"namespace","fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}]}}],"defaultMode":420}}],"containers":[{"name":"base","image":"harbor.wycluster.com/exec_
projects/portfolio_core:0.8.57","command":["/bin/bash","-c"],"args":["echo
\"run task($TASK_ID) in env($WY_EXEC_PLATFORM_MODE) on date($DATE), file hash:
$FILE_COMMIT_HASH\" \u0026\u0026 poetry run bash -c \"python -m
wy_archive_sdk.cmd run --path __init__.py --date 20251114 --variance task
--stage
PRODUCTION\""],"env":[{"name":"NAME_TO_GREET","value":"portfolio_center_archive_task_archive_task"},{"name":"TZ","value":"Asia/Shanghai"},{"name":"DAG_ID","value":"exec_platform.wy_algotrading__portfolio_center_0_8_57"},{"name":"TASK_ID","value":"portfolio_center_archive_task_archive_task"},{"name":"DATE","value":"20251114"},{"name":"WY_EXEC_PLATFORM_MODE","value":"production"},{"name":"EXEC_REMOTE_SCRIPT"},{"name":"FILE_COMMIT_HASH","value":"7efa7d6d520c9c9910b932317b5c0f89f04ad76e"},{"name":"EXEC_WARNING_LOG","value":"true"}],"resources":{"limits":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"},"requests":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"}},"volumeMounts":[{"name":"shad
ow","readOnly":true,"mountPath":"/etc/shadow"},{"name":"passwd","readOnly":true,"mountPath":"/etc/passwd"},{"name":"group","readOnly":true,"mountPath":"/etc/group"},{"name":"scratch","mountPath":"/scratch"},{"name":"shm","mountPath":"/dev/shm"},{"name":"kube-api-access-skkvb","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"datastack-node-114","securityContext":{"runAsUser":3021,"runAsGroup":3021},"imagePullSecrets":[{"name":"image-pull-secret"}],"affinity":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExe
cute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true,"preemptionPolicy":"PreemptLowerPriority"},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"}],"hostIP":"10.20.1.114","podIP":"100.64.15.176","podIPs":[{"ip":"100.64.15.176"}],"startTime":"2025-11-17T00:31:32Z","containerStatuses":[{"name":"base","state":{"running":{"startedAt":"2025-11-17T00:32:06Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"harbor.wycluster.com/exec_projects/portfolio_core:0.8.57","imageID":"docker-pullable://harbor.wycluster.com/exec_projects/portfolio_core@sha256:ea7f016b491fd1af8a
8be4af4617238e1175ae7145a7485240e850ce8dca10b9","containerID":"docker://3a4984cfb6cbe87e78ab7f363626fca2b20afed77420bd1f6aafd7f0f75bb08d","started":true}],"qosClass":"Guaranteed"}}
[2025-11-17, 08:37:50 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:37:55 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:00 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:05 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:10 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:15 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:21 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:26 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:31 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:36 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:41 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:46 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:51 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:38:56 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:01 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:06 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:11 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:16 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:21 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:26 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:31 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:36 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:41 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:46 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:52 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:39:57 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:02 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:07 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:12 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:17 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:22 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:27 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:32 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:37 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:42 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:47 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:52 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:40:57 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:02 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:07 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:12 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:17 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:22 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:28 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:33 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:38 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:43 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:48 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:53 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:41:58 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:03 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:08 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:13 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:18 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:23 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:28 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:33 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:38 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:43 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:48 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:49 CST] {pod_manager.py:483} INFO - [base] None
[2025-11-17, 08:42:49 CST] {rest.py:235} DEBUG - response body:
{"kind":"Pod","apiVersion":"v1","metadata":{"name":"portfolio-center-archive-task-archive-task-tlsaj6cs","namespace":"airflow-prod","uid":"f450696b-8850-4ff5-b770-11d24e300c0b","resourceVersion":"582326745","creationTimestamp":"2025-11-17T00:31:32Z","labels":{"airflow_kpo_in_cluster":"True","airflow_version":"2.8.4","app":"airflow","dag_id":"exec_platform.wy_algotrading__portfolio_center_0_8_57","env":"prod","kubernetes_pod_operator":"True","run_id":"scheduled__2025-11-13T0700000000-0cce263ff","task_id":"portfolio_center_archive_task_archive_task","try_number":"1"},"annotations":{"cni.projectcalico.org/containerID":"0d9ea05760f2672aff89a53b208c25cdce7f0af1fce297ff03613ee6d4cff7ab","cni.projectcalico.org/podIP":"100.64.15.176/32","cni.projectcalico.org/podIPs":"100.64.15.176/32"},"managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:31:32Z","fieldsType":"FieldsV1",
"fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:airflow_kpo_in_cluster":{},"f:airflow_version":{},"f:app":{},"f:dag_id":{},"f:env":{},"f:kubernetes_pod_operator":{},"f:run_id":{},"f:task_id":{},"f:try_number":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"base\"}":{".":{},"f:args":{},"f:command":{},"f:env":{".":{},"k:{\"name\":\"DAG_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"DATE\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"EXEC_REMOTE_SCRIPT\"}":{".":{},"f:name":{}},"k:{\"name\":\"EXEC_WARNING_LOG\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"FILE_COMMIT_HASH\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"NAME_TO_GREET\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TASK_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TZ\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"WY_EXEC_PLATFORM_MODE\"}":{".":{},"f:name":{},"f:value":{}}},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{".":{},"f:limits":{".":{},
"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}},"f:requests":{".":{},"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/dev/shm\"}":{".":{},"f:mountPath":{},"f:name":{}},"k:{\"mountPath\":\"/etc/group\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/passwd\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/shadow\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/scratch\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:runAsGroup":{},"f:runAsUser":{}},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"group\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"passwd\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},
"f:name":{}},"k:{\"name\":\"scratch\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shadow\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shm\"}":{".":{},"f:emptyDir":{".":{},"f:medium":{}},"f:name":{}}}}}},{"manager":"calico","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:05Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:cni.projectcalico.org/containerID":{},"f:cni.projectcalico.org/podIP":{},"f:cni.projectcalico.org/podIPs":{}}}},"subresource":"status"},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{"
.":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"100.64.15.176\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},"subresource":"status"}]},"spec":{"volumes":[{"name":"shadow","hostPath":{"path":"/etc/shadow","type":"File"}},{"name":"passwd","hostPath":{"path":"/etc/passwd","type":"File"}},{"name":"group","hostPath":{"path":"/etc/group","type":"File"}},{"name":"scratch","hostPath":{"path":"/scratch","type":"Directory"}},{"name":"shm","emptyDir":{"medium":"Memory"}},{"name":"kube-api-access-skkvb","projected":{"sources":[{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}},{"configMap":{"name":"kube-root-ca.crt","items":[{"key":"ca.crt","path":"ca.crt"}]}},{"downwardAPI":{"items":[{"path":"namespace","fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}]}}],"defaultMode":420}}],"containers":[{"name":"base","image":"harbor.wycluster.com/exec_
projects/portfolio_core:0.8.57","command":["/bin/bash","-c"],"args":["echo
\"run task($TASK_ID) in env($WY_EXEC_PLATFORM_MODE) on date($DATE), file hash:
$FILE_COMMIT_HASH\" \u0026\u0026 poetry run bash -c \"python -m
wy_archive_sdk.cmd run --path __init__.py --date 20251114 --variance task
--stage
PRODUCTION\""],"env":[{"name":"NAME_TO_GREET","value":"portfolio_center_archive_task_archive_task"},{"name":"TZ","value":"Asia/Shanghai"},{"name":"DAG_ID","value":"exec_platform.wy_algotrading__portfolio_center_0_8_57"},{"name":"TASK_ID","value":"portfolio_center_archive_task_archive_task"},{"name":"DATE","value":"20251114"},{"name":"WY_EXEC_PLATFORM_MODE","value":"production"},{"name":"EXEC_REMOTE_SCRIPT"},{"name":"FILE_COMMIT_HASH","value":"7efa7d6d520c9c9910b932317b5c0f89f04ad76e"},{"name":"EXEC_WARNING_LOG","value":"true"}],"resources":{"limits":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"},"requests":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"}},"volumeMounts":[{"name":"shad
ow","readOnly":true,"mountPath":"/etc/shadow"},{"name":"passwd","readOnly":true,"mountPath":"/etc/passwd"},{"name":"group","readOnly":true,"mountPath":"/etc/group"},{"name":"scratch","mountPath":"/scratch"},{"name":"shm","mountPath":"/dev/shm"},{"name":"kube-api-access-skkvb","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"datastack-node-114","securityContext":{"runAsUser":3021,"runAsGroup":3021},"imagePullSecrets":[{"name":"image-pull-secret"}],"affinity":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExe
cute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true,"preemptionPolicy":"PreemptLowerPriority"},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"}],"hostIP":"10.20.1.114","podIP":"100.64.15.176","podIPs":[{"ip":"100.64.15.176"}],"startTime":"2025-11-17T00:31:32Z","containerStatuses":[{"name":"base","state":{"running":{"startedAt":"2025-11-17T00:32:06Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"harbor.wycluster.com/exec_projects/portfolio_core:0.8.57","imageID":"docker-pullable://harbor.wycluster.com/exec_projects/portfolio_core@sha256:ea7f016b491fd1af8a
8be4af4617238e1175ae7145a7485240e850ce8dca10b9","containerID":"docker://3a4984cfb6cbe87e78ab7f363626fca2b20afed77420bd1f6aafd7f0f75bb08d","started":true}],"qosClass":"Guaranteed"}}
[2025-11-17, 08:42:50 CST] {rest.py:235} DEBUG - response body:
{"kind":"Pod","apiVersion":"v1","metadata":{"name":"portfolio-center-archive-task-archive-task-tlsaj6cs","namespace":"airflow-prod","uid":"f450696b-8850-4ff5-b770-11d24e300c0b","resourceVersion":"582326745","creationTimestamp":"2025-11-17T00:31:32Z","labels":{"airflow_kpo_in_cluster":"True","airflow_version":"2.8.4","app":"airflow","dag_id":"exec_platform.wy_algotrading__portfolio_center_0_8_57","env":"prod","kubernetes_pod_operator":"True","run_id":"scheduled__2025-11-13T0700000000-0cce263ff","task_id":"portfolio_center_archive_task_archive_task","try_number":"1"},"annotations":{"cni.projectcalico.org/containerID":"0d9ea05760f2672aff89a53b208c25cdce7f0af1fce297ff03613ee6d4cff7ab","cni.projectcalico.org/podIP":"100.64.15.176/32","cni.projectcalico.org/podIPs":"100.64.15.176/32"},"managedFields":[{"manager":"OpenAPI-Generator","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:31:32Z","fieldsType":"FieldsV1",
"fieldsV1":{"f:metadata":{"f:labels":{".":{},"f:airflow_kpo_in_cluster":{},"f:airflow_version":{},"f:app":{},"f:dag_id":{},"f:env":{},"f:kubernetes_pod_operator":{},"f:run_id":{},"f:task_id":{},"f:try_number":{}}},"f:spec":{"f:affinity":{},"f:containers":{"k:{\"name\":\"base\"}":{".":{},"f:args":{},"f:command":{},"f:env":{".":{},"k:{\"name\":\"DAG_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"DATE\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"EXEC_REMOTE_SCRIPT\"}":{".":{},"f:name":{}},"k:{\"name\":\"EXEC_WARNING_LOG\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"FILE_COMMIT_HASH\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"NAME_TO_GREET\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TASK_ID\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"TZ\"}":{".":{},"f:name":{},"f:value":{}},"k:{\"name\":\"WY_EXEC_PLATFORM_MODE\"}":{".":{},"f:name":{},"f:value":{}}},"f:image":{},"f:imagePullPolicy":{},"f:name":{},"f:resources":{".":{},"f:limits":{".":{},
"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}},"f:requests":{".":{},"f:cpu":{},"f:memory":{},"f:nvidia.com/gpu":{}}},"f:terminationMessagePath":{},"f:terminationMessagePolicy":{},"f:volumeMounts":{".":{},"k:{\"mountPath\":\"/dev/shm\"}":{".":{},"f:mountPath":{},"f:name":{}},"k:{\"mountPath\":\"/etc/group\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/passwd\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/etc/shadow\"}":{".":{},"f:mountPath":{},"f:name":{},"f:readOnly":{}},"k:{\"mountPath\":\"/scratch\"}":{".":{},"f:mountPath":{},"f:name":{}}}}},"f:dnsPolicy":{},"f:enableServiceLinks":{},"f:restartPolicy":{},"f:schedulerName":{},"f:securityContext":{".":{},"f:runAsGroup":{},"f:runAsUser":{}},"f:terminationGracePeriodSeconds":{},"f:volumes":{".":{},"k:{\"name\":\"group\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"passwd\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},
"f:name":{}},"k:{\"name\":\"scratch\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shadow\"}":{".":{},"f:hostPath":{".":{},"f:path":{},"f:type":{}},"f:name":{}},"k:{\"name\":\"shm\"}":{".":{},"f:emptyDir":{".":{},"f:medium":{}},"f:name":{}}}}}},{"manager":"calico","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:05Z","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:cni.projectcalico.org/containerID":{},"f:cni.projectcalico.org/podIP":{},"f:cni.projectcalico.org/podIPs":{}}}},"subresource":"status"},{"manager":"kubelet","operation":"Update","apiVersion":"v1","time":"2025-11-17T00:32:43Z","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:conditions":{"k:{\"type\":\"ContainersReady\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Initialized\"}":{".":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}},"k:{\"type\":\"Ready\"}":{"
.":{},"f:lastProbeTime":{},"f:lastTransitionTime":{},"f:status":{},"f:type":{}}},"f:containerStatuses":{},"f:hostIP":{},"f:phase":{},"f:podIP":{},"f:podIPs":{".":{},"k:{\"ip\":\"100.64.15.176\"}":{".":{},"f:ip":{}}},"f:startTime":{}}},"subresource":"status"}]},"spec":{"volumes":[{"name":"shadow","hostPath":{"path":"/etc/shadow","type":"File"}},{"name":"passwd","hostPath":{"path":"/etc/passwd","type":"File"}},{"name":"group","hostPath":{"path":"/etc/group","type":"File"}},{"name":"scratch","hostPath":{"path":"/scratch","type":"Directory"}},{"name":"shm","emptyDir":{"medium":"Memory"}},{"name":"kube-api-access-skkvb","projected":{"sources":[{"serviceAccountToken":{"expirationSeconds":3607,"path":"token"}},{"configMap":{"name":"kube-root-ca.crt","items":[{"key":"ca.crt","path":"ca.crt"}]}},{"downwardAPI":{"items":[{"path":"namespace","fieldRef":{"apiVersion":"v1","fieldPath":"metadata.namespace"}}]}}],"defaultMode":420}}],"containers":[{"name":"base","image":"harbor.wycluster.com/exec_
projects/portfolio_core:0.8.57","command":["/bin/bash","-c"],"args":["echo
\"run task($TASK_ID) in env($WY_EXEC_PLATFORM_MODE) on date($DATE), file hash:
$FILE_COMMIT_HASH\" \u0026\u0026 poetry run bash -c \"python -m
wy_archive_sdk.cmd run --path __init__.py --date 20251114 --variance task
--stage
PRODUCTION\""],"env":[{"name":"NAME_TO_GREET","value":"portfolio_center_archive_task_archive_task"},{"name":"TZ","value":"Asia/Shanghai"},{"name":"DAG_ID","value":"exec_platform.wy_algotrading__portfolio_center_0_8_57"},{"name":"TASK_ID","value":"portfolio_center_archive_task_archive_task"},{"name":"DATE","value":"20251114"},{"name":"WY_EXEC_PLATFORM_MODE","value":"production"},{"name":"EXEC_REMOTE_SCRIPT"},{"name":"FILE_COMMIT_HASH","value":"7efa7d6d520c9c9910b932317b5c0f89f04ad76e"},{"name":"EXEC_WARNING_LOG","value":"true"}],"resources":{"limits":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"},"requests":{"cpu":"1","memory":"32Gi","nvidia.com/gpu":"0"}},"volumeMounts":[{"name":"shad
ow","readOnly":true,"mountPath":"/etc/shadow"},{"name":"passwd","readOnly":true,"mountPath":"/etc/passwd"},{"name":"group","readOnly":true,"mountPath":"/etc/group"},{"name":"scratch","mountPath":"/scratch"},{"name":"shm","mountPath":"/dev/shm"},{"name":"kube-api-access-skkvb","readOnly":true,"mountPath":"/var/run/secrets/kubernetes.io/serviceaccount"}],"terminationMessagePath":"/dev/termination-log","terminationMessagePolicy":"File","imagePullPolicy":"IfNotPresent"}],"restartPolicy":"Never","terminationGracePeriodSeconds":30,"dnsPolicy":"ClusterFirst","serviceAccountName":"default","serviceAccount":"default","nodeName":"datastack-node-114","securityContext":{"runAsUser":3021,"runAsGroup":3021},"imagePullSecrets":[{"name":"image-pull-secret"}],"affinity":{},"schedulerName":"default-scheduler","tolerations":[{"key":"node.kubernetes.io/not-ready","operator":"Exists","effect":"NoExecute","tolerationSeconds":300},{"key":"node.kubernetes.io/unreachable","operator":"Exists","effect":"NoExe
cute","tolerationSeconds":300}],"priority":0,"enableServiceLinks":true,"preemptionPolicy":"PreemptLowerPriority"},"status":{"phase":"Running","conditions":[{"type":"Initialized","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"},{"type":"Ready","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"ContainersReady","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:32:07Z"},{"type":"PodScheduled","status":"True","lastProbeTime":null,"lastTransitionTime":"2025-11-17T00:31:32Z"}],"hostIP":"10.20.1.114","podIP":"100.64.15.176","podIPs":[{"ip":"100.64.15.176"}],"startTime":"2025-11-17T00:31:32Z","containerStatuses":[{"name":"base","state":{"running":{"startedAt":"2025-11-17T00:32:06Z"}},"lastState":{},"ready":true,"restartCount":0,"image":"harbor.wycluster.com/exec_projects/portfolio_core:0.8.57","imageID":"docker-pullable://harbor.wycluster.com/exec_projects/portfolio_core@sha256:ea7f016b491fd1af8a
8be4af4617238e1175ae7145a7485240e850ce8dca10b9","containerID":"docker://3a4984cfb6cbe87e78ab7f363626fca2b20afed77420bd1f6aafd7f0f75bb08d","started":true}],"qosClass":"Guaranteed"}}
[2025-11-17, 08:42:53 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:42:58 CST] {job.py:209} DEBUG - [heartbeat]
[2025-11-17, 08:43:04 CST] {job.py:209} DEBUG - [heartbeat]
`
### What you think should happen instead?
The task base pod should keep printing logs (instead of [None]) and get
completed quickly (It takes less than 3 mins, normally)
### How to reproduce
This issue can't be reproduced, it happens randomly and is not dedicated to
any specific Dag/Task.
### Operating System
Ubuntu 20.04
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
helm
### Anything else?
Occasionally, while it seems to be much more regularly
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
