karakanb opened a new issue, #36571: URL: https://github.com/apache/airflow/issues/36571
### Apache Airflow Provider(s) cncf-kubernetes ### Versions of Apache Airflow Providers apache-airflow-providers-amazon==8.14.0 apache-airflow-providers-celery==3.5.1 apache-airflow-providers-cncf-kubernetes==7.12.0 apache-airflow-providers-common-io==1.1.0 apache-airflow-providers-common-sql==1.10.0 apache-airflow-providers-discord==3.5.0 apache-airflow-providers-docker==3.8.2 apache-airflow-providers-elasticsearch==5.3.0 apache-airflow-providers-ftp==3.7.0 apache-airflow-providers-google==10.13.0 apache-airflow-providers-grpc==3.4.0 apache-airflow-providers-hashicorp==3.6.0 apache-airflow-providers-http==4.8.0 apache-airflow-providers-imap==3.5.0 apache-airflow-providers-microsoft-azure==8.4.0 apache-airflow-providers-mysql==5.5.0 apache-airflow-providers-odbc==4.2.0 apache-airflow-providers-openlineage==1.3.0 apache-airflow-providers-postgres==5.9.0 apache-airflow-providers-redis==3.5.0 apache-airflow-providers-sendgrid==3.4.0 apache-airflow-providers-sftp==4.8.0 apache-airflow-providers-slack==8.5.1 apache-airflow-providers-snowflake==5.2.1 apache-airflow-providers-sqlite==3.6.0 apache-airflow-providers-ssh==3.9.0 ### Apache Airflow version 2.8.0 ### Operating System Debian 12 (bookworm) ### Deployment Official Apache Airflow Helm Chart ### Deployment details deployed on digitalocean k8s. I have a custom log format as `'%%(asctime)s | %%(levelname)s | %%(message)s'` ### What happened I have recently upgraded the cncf-kubernetes provider from `7.4.2` to `7.12.0` and started seeing weird log lines being printed out in-between application logs: ``` 2024-01-03T21:31:59.286+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3 2024-01-03T21:32:00.313+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3 2024-01-03T21:32:00.694053856Z 2024-01-03T21:32:00.694107912Z 2024-01-03T21:32:00.694384093Z 2024-01-03T21:32:00.694416033Z 2024-01-03T21:32:00.694514264Z 2024-01-03T21:32:00.694543915Z 2024-01-03T21:32:00.694553314Z 2024-01-03T21:32:00.694557782Z 2024-01-03T21:32:00.694629859Z 2024-01-03T21:32:00.694642682Z 2024-01-03T21:32:01.348+0000 | WARNING | Pod not yet started: my-pod-name-xutbmgq3 ``` this also happens while the application is running: ``` 2024-01-03T21:32:04.814+0000 | INFO | [base] application log line 1 2024-01-03T21:32:05.234+0000 | INFO | [base] application log line 2 2024-01-03T21:32:05.526911055Z 2024-01-03T21:32:05.529+0000 | INFO | [base] application log line 3 2024-01-03T21:32:05.529+0000 | INFO | [base] application log line 4 2024-01-03T21:32:05.654767914Z 2024-01-03T21:32:05.654830820Z 2024-01-03T21:32:05.654857369Z ``` As you can see the log lines have a different format than the actual airflow log format, and they seem to contain no messages. This causes confusion, and for a few cases where I parse the logs it broke my existing code. I have checked through external log collectors as well as via tailing the pod logs and I can confirm that the logs do not come from the application/pod side, and being logged somewhere on the airflow side. ### What you think should happen instead there should be no log lines printed that doesn't respect the logging format. ### How to reproduce Create a pod with KubernetesPodOperator, it happens to me consistently. ### Anything else Every time I run a pod it happens. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
