GitHub user uplsh580 edited a comment on the discussion: S3 Remote Logging
Failure After Upgrading Airflow from 2.9.3 to 2.10.3
# Version Detail Information
## Before Upgrade (Airflow 2.9.3)
```
Apache Airflow
version | 2.9.3
executor | CeleryKubernetesExecutor
task_logging_handler |
airflow.providers.amazon.aws.log.s3_task_handler.S3TaskHandler
sql_alchemy_conn | ...
dags_folder | /opt/airflow/dags
plugins_folder | /opt/airflow/plugins
base_log_folder | /opt/airflow/logs
remote_base_log_folder | s3://airflow-adhoc/logs
System info
OS | Linux
architecture | x86_64
uname | uname_result(system='Linux',
node='airflow-test-webserver-6c6f49dc68-c5f6n',
release='5.10.195-1.20230921.el7.x86_64', version='#1 SMP Thu Sep 21 07:40:57
UTC 2023', machine='x86_64')
locale | ('C', 'UTF-8')
python_version | 3.12.4 (main, Jul 23 2024, 07:23:20) [GCC 12.2.0]
python_location | /home/airflow/.local/bin/python
Tools info
git | NOT AVAILABLE
ssh | OpenSSH_9.2p1 Debian-2+deb12u3, OpenSSL 3.0.15 3 Sep 2024
kubectl | NOT AVAILABLE
gcloud | NOT AVAILABLE
cloud_sql_proxy | NOT AVAILABLE
mysql | mysql Ver 15.1 Distrib 10.11.11-MariaDB, for
debian-linux-gnu (x86_64) using EditLine wrapper
sqlite3 | 3.40.1 2022-12-28 14:03:47
df5c253c0b3dd24916e4ec7cf77d3db5294cc9fd45ae7b9c5e82ad8197f3alt1
psql | psql (PostgreSQL) 16.3 (Debian 16.3-1.pgdg120+1)
Paths info
airflow_home | /opt/airflow
system_path |
/root/bin:/home/airflow/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
python_path |
/home/airflow/.local/bin:/usr/local/lib/python312.zip:/usr/local/lib/python3.12:/usr/local/lib/python3.12/lib-dynload:/home/airflow/.local/lib/python3.12/site-packages:/opt/airflow/dags:/opt/airflow/config:/opt/airflow/plugins
airflow_on_path | True
Providers info
apache-airflow-providers-amazon | 8.25.0
apache-airflow-providers-celery | 3.7.2
apache-airflow-providers-cncf-kubernetes | 8.3.3
apache-airflow-providers-common-io | 1.3.2
apache-airflow-providers-common-sql | 1.14.2
apache-airflow-providers-docker | 3.12.2
apache-airflow-providers-elasticsearch | 5.4.1
apache-airflow-providers-fab | 1.2.2
apache-airflow-providers-ftp | 3.10.0
apache-airflow-providers-google | 10.21.0
apache-airflow-providers-grpc | 3.5.2
apache-airflow-providers-hashicorp | 3.7.1
apache-airflow-providers-http | 4.12.0
apache-airflow-providers-imap | 3.6.1
apache-airflow-providers-microsoft-azure | 10.2.0
apache-airflow-providers-mysql | 5.6.2
apache-airflow-providers-odbc | 4.6.2
apache-airflow-providers-openlineage | 1.9.1
apache-airflow-providers-postgres | 5.11.2
apache-airflow-providers-redis | 3.7.1
apache-airflow-providers-sendgrid | 3.5.1
apache-airflow-providers-sftp | 4.10.2
apache-airflow-providers-slack | 8.7.1
apache-airflow-providers-smtp | 1.7.1
apache-airflow-providers-snowflake | 5.6.0
apache-airflow-providers-sqlite | 3.8.1
apache-airflow-providers-ssh | 3.11.2
```
## After Upgrade (Airflow 2.10.5)
```
Apache Airflow
version | 2.10.5
executor | CeleryKubernetesExecutor
task_logging_handler |
airflow.providers.amazon.aws.log.s3_task_handler.S3TaskHandler
sql_alchemy_conn | ...
dags_folder | /opt/airflow/dags
plugins_folder | /opt/airflow/plugins
base_log_folder | /opt/airflow/logs
remote_base_log_folder | s3://airflow-adhoc/logs
System info
OS | Linux
architecture | x86_64
uname | uname_result(system='Linux',
node='airflow-test-webserver-67cf5f74db-6m728',
release='5.10.195-1.20230921.el7.x86_64', version='#1 SMP Thu Sep 21 07:40:57
UTC 2023', machine='x86_64')
locale | ('C', 'UTF-8')
python_version | 3.12.9 (main, Feb 6 2025, 22:37:05) [GCC 12.2.0]
python_location | /home/airflow/.local/bin/python
Tools info
git | NOT AVAILABLE
ssh | OpenSSH_9.2p1 Debian-2+deb12u4, OpenSSL 3.0.15 3 Sep 2024
kubectl | NOT AVAILABLE
gcloud | NOT AVAILABLE
cloud_sql_proxy | NOT AVAILABLE
mysql | mysql Ver 15.1 Distrib 10.11.11-MariaDB, for
debian-linux-gnu (x86_64) using EditLine wrapper
sqlite3 | 3.40.1 2022-12-28 14:03:47
df5c253c0b3dd24916e4ec7cf77d3db5294cc9fd45ae7b9c5e82ad8197f3alt1
psql | psql (PostgreSQL) 17.2 (Debian 17.2-1.pgdg120+1)
Paths info
airflow_home | /opt/airflow
system_path |
/root/bin:/home/airflow/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
python_path |
/home/airflow/.local/bin:/usr/local/lib/python312.zip:/usr/local/lib/python3.12:/usr/local/lib/python3.12/lib-dynload:/home/airflow/.local/lib/python3.12/site-packages:/opt/airflow/config:/op
| t/airflow/plugins:/opt/airflow/dags
airflow_on_path | True
Providers info
apache-airflow-providers-amazon | 9.2.0
apache-airflow-providers-celery | 3.10.0
apache-airflow-providers-cncf-kubernetes | 10.1.0
apache-airflow-providers-common-compat | 1.3.0
apache-airflow-providers-common-io | 1.5.0
apache-airflow-providers-common-sql | 1.21.0
apache-airflow-providers-docker | 4.0.0
apache-airflow-providers-elasticsearch | 6.0.0
apache-airflow-providers-fab | 1.5.2
apache-airflow-providers-ftp | 3.12.0
apache-airflow-providers-google | 12.0.0
apache-airflow-providers-grpc | 3.7.0
apache-airflow-providers-hashicorp | 4.0.0
apache-airflow-providers-http | 5.0.0
apache-airflow-providers-imap | 3.8.0
apache-airflow-providers-microsoft-azure | 12.0.0
apache-airflow-providers-mysql | 6.0.0
apache-airflow-providers-odbc | 4.9.0
apache-airflow-providers-openlineage | 2.0.0
apache-airflow-providers-postgres | 6.0.0
apache-airflow-providers-redis | 4.0.0
apache-airflow-providers-sendgrid | 4.0.0
apache-airflow-providers-sftp | 5.0.0
apache-airflow-providers-slack | 9.0.0
apache-airflow-providers-smtp | 1.9.0
apache-airflow-providers-snowflake | 6.0.0
apache-airflow-providers-sqlite | 4.0.0
apache-airflow-providers-ssh | 4.0.0
```
GitHub link:
https://github.com/apache/airflow/discussions/51106#discussioncomment-13281939
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
