dirrao opened a new issue, #37131: URL: https://github.com/apache/airflow/issues/37131
### Apache Airflow Provider(s) cncf-kubernetes ### Versions of Apache Airflow Providers 7.1.4.0 ### Apache Airflow version 2.8.1 ### Operating System Cent OS 6 ### Deployment Other ### Deployment details Docker based deployment using terraform ### What happened In the Kubernetes executor, Merge Kubernetes Pod objects (base_worker_pod with 3 containers and 1 init container) is taking a significant amount of time (> 500 ms) due to the recursive deep copy. ### What you think should happen instead We are constructing the base_worker_pod, dynamic_pod, and executor_config_pod for every task. So, we don't need to retain these pod objects. We can merge these pod objects in place and return the merged pod. I agree with fact that the immutability brings advantages. Even if we want to retain the pod objects, clone them before calling construct_pod function. ### How to reproduce Profiling the kubernetes executor run_next function. <img width="1361" alt="Screen Shot 2024-02-01 at 5 10 43 PM" src="https://github.com/apache/airflow/assets/39794726/f515f608-d965-4b40-8545-c9bbc86521f5";> ### Anything else _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
