curlup opened a new issue, #30400: URL: https://github.com/apache/airflow/issues/30400
### Official Helm Chart version 1.7.0 ### Apache Airflow version 2.4.0 ### Kubernetes Version K3s Kubernetes Version: v1.24.2+k3s2 ### Helm Chart configuration _No response_ ### Docker Image customizations _No response_ ### What happened Same as https://github.com/apache/airflow/issues/12229 ``` [2023-03-31T17:47:08.304+0000] {kubernetes_executor.py:112} ERROR - Unknown error in KubernetesJobWatcher. Failing Traceback (most recent call last): File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py", line 103, in run self.resource_version = self._run( File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py", line 148, in _run for event in list_worker_pods(): File "/home/airflow/.local/lib/python3.10/site-packages/kubernetes/watch/watch.py", line 182, in stream raise client.rest.ApiException( kubernetes.client.exceptions.ApiException: (410) Reason: Expired: too old resource version: 202541353 (202544371) Process KubernetesJobWatcher-3: Traceback (most recent call last): File "/usr/local/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py", line 103, in run self.resource_version = self._run( File "/home/airflow/.local/lib/python3.10/site-packages/airflow/executors/kubernetes_executor.py", line 148, in _run for event in list_worker_pods(): File "/home/airflow/.local/lib/python3.10/site-packages/kubernetes/watch/watch.py", line 182, in stream raise client.rest.ApiException( kubernetes.client.exceptions.ApiException: (410) Reason: Expired: too old resource version: 202541353 (202544371) [2023-03-31T17:47:08.832+0000] {kubernetes_executor.py:291} ERROR - Error while health checking kube watcher process. Process died for unknown reasons ``` ### What you think should happen instead no errors in the logs? ### How to reproduce appears soon after AF-scheduler pod restart ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
