dougforpres commented on issue #57470: URL: https://github.com/apache/airflow/issues/57470#issuecomment-3464872773
@vikramkoka - I'm working with @arimus and can explain what we're seeing. After the upgrade last-night things started acting weird. It all seemed to be ok until 15:00 (server time) today. That's when I returned to the app after leaving it apparently "ok" last night. When I click log-out I will sometimes get a 500 error with a huge stacktrace in the log indicating a transaction needs to be rolled back. When I am able to log out, and I try to log in, sometimes I'll get a generic "Server Error" page, and other times the "Ooops" page will appear. When I am able to log on, I'll get the home page with errors on it indicating I'm not logged on, then the page will refresh and appear to be ok. Here is a log from the start of the 3.1.1 server yesterday. You can see it all seemed to be ok until 15:00 today (the 29th) when a page request got a mysql timeout... then, when the user (me, as it happens) tried to log on the errors about transactions needing to roll-back started. Apologies if this blob is too big ``` Oct 28 16:39:08 integration-airflow-new.mht python3.13[859034]: INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) Oct 28 16:39:08 integration-airflow-new.mht python3.13[859034]: INFO: Started parent process [859034] Oct 28 16:39:08 integration-airflow-new.mht python3.13[859034]: ____________ _____________ Oct 28 16:39:08 integration-airflow-new.mht python3.13[859034]: ____ |__( )_________ __/__ /________ __ Oct 28 16:39:08 integration-airflow-new.mht python3.13[859034]: ____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / / Oct 28 16:39:08 integration-airflow-new.mht python3.13[859034]: ___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ / Oct 28 16:39:08 integration-airflow-new.mht python3.13[859034]: _/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/ Oct 28 16:39:12 integration-airflow-new.mht python3.13[859126]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/execution_api/routes/__init__.py:23 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:12 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/execution_api/routes/__init__.py:23 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: [2025-10-28T16:39:13.072189Z] {workday.py:41} WARNING - Could not import pandas. Holidays will not be considered. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/execution_api/routes/__init__.py:23 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/execution_api/routes/__init__.py:23 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py:108 RemovedInAirflow4Warning: The airflow.security.permissions module is deprecated; please see https://airflow.apache.org/docs/apache-airflow/stable/security/deprecated_permissions.html Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: /opt/airflow/.local/lib/python3.13/site-packages/flask_limiter/extension.py:324 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859124]: [2025-10-28T16:39:13.553595Z] {workday.py:41} WARNING - Could not import pandas. Holidays will not be considered. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859127]: [2025-10-28T16:39:13.734821Z] {workday.py:41} WARNING - Could not import pandas. Holidays will not be considered. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: [2025-10-28T16:39:13.809361Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py:108 RemovedInAirflow4Warning: The airflow.security.permissions module is deprecated; please see https://airflow.apache.org/docs/apache-airflow/stable/security/deprecated_permissions.html Oct 28 16:39:13 integration-airflow-new.mht python3.13[859125]: [2025-10-28T16:39:13.855514Z] {workday.py:41} WARNING - Could not import pandas. Holidays will not be considered. Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: [2025-10-28T16:39:13.867368Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: [2025-10-28T16:39:13.908115Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859126]: [2025-10-28T16:39:13.963824Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:13 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/flask_limiter/extension.py:324 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend. Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py:108 RemovedInAirflow4Warning: The airflow.security.permissions module is deprecated; please see https://airflow.apache.org/docs/apache-airflow/stable/security/deprecated_permissions.html Oct 28 16:39:14 integration-airflow-new.mht python3.13[859126]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859126]: [2025-10-28T16:39:14.043607Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859126]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859126]: [2025-10-28T16:39:14.096765Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/flask_limiter/extension.py:324 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend. Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py:108 RemovedInAirflow4Warning: The airflow.security.permissions module is deprecated; please see https://airflow.apache.org/docs/apache-airflow/stable/security/deprecated_permissions.html Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/flask_limiter/extension.py:324 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend. Oct 28 16:39:14 integration-airflow-new.mht python3.13[859126]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/extensions/init_jinja_globals.py:43 DeprecationWarning: The expose_hostname option in [webserver] has been moved to the expose_hostname option in [fab] - the old setting has been used, but please update your config. Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: [2025-10-28T16:39:14.300669Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: [2025-10-28T16:39:14.359324Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: [2025-10-28T16:39:14.399801Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: [2025-10-28T16:39:14.402973Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: [2025-10-28T16:39:14.442189Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: [2025-10-28T16:39:14.452262Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: [2025-10-28T16:39:14.481730Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: [2025-10-28T16:39:14.491940Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: [2025-10-28T16:39:14.495844Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: [2025-10-28T16:39:14.523574Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: [2025-10-28T16:39:14.542174Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: [2025-10-28T16:39:14.545462Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: [2025-10-28T16:39:14.585467Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: [2025-10-28T16:39:14.595321Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: [2025-10-28T16:39:14.626452Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: [2025-10-28T16:39:14.647988Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: [2025-10-28T16:39:14.667622Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/extensions/init_jinja_globals.py:43 DeprecationWarning: The expose_hostname option in [webserver] has been moved to the expose_hostname option in [fab] - the old setting has been used, but please update your config. Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: [2025-10-28T16:39:14.723481Z] {init_appbuilder.py:228} WARNING - appbuilder.app is deprecated and will be removed in a future version. Use current_app instead Oct 28 16:39:14 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/extensions/init_jinja_globals.py:43 DeprecationWarning: The expose_hostname option in [webserver] has been moved to the expose_hostname option in [fab] - the old setting has been used, but please update your config. Oct 28 16:39:14 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/extensions/init_jinja_globals.py:43 DeprecationWarning: The expose_hostname option in [webserver] has been moved to the expose_hostname option in [fab] - the old setting has been used, but please update your config. Oct 28 16:39:15 integration-airflow-new.mht python3.13[859126]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/public/__init__.py:36 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:15 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/public/__init__.py:36 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:15 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/public/__init__.py:36 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:15 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/public/__init__.py:36 DeprecationWarning: 'HTTP_422_UNPROCESSABLE_ENTITY' is deprecated. Use 'HTTP_422_UNPROCESSABLE_CONTENT' instead. Oct 28 16:39:18 integration-airflow-new.mht python3.13[859126]: INFO: Started server process [859126] Oct 28 16:39:18 integration-airflow-new.mht python3.13[859126]: INFO: Waiting for application startup. Oct 28 16:39:18 integration-airflow-new.mht python3.13[859126]: INFO: Application startup complete. Oct 28 16:39:18 integration-airflow-new.mht python3.13[859124]: INFO: Started server process [859124] Oct 28 16:39:18 integration-airflow-new.mht python3.13[859124]: INFO: Waiting for application startup. Oct 28 16:39:18 integration-airflow-new.mht python3.13[859124]: INFO: Application startup complete. Oct 28 16:39:19 integration-airflow-new.mht python3.13[859125]: INFO: Started server process [859125] Oct 28 16:39:19 integration-airflow-new.mht python3.13[859125]: INFO: Waiting for application startup. Oct 28 16:39:19 integration-airflow-new.mht python3.13[859125]: INFO: Application startup complete. Oct 28 16:39:19 integration-airflow-new.mht python3.13[859127]: INFO: Started server process [859127] Oct 28 16:39:19 integration-airflow-new.mht python3.13[859127]: INFO: Waiting for application startup. Oct 28 16:39:19 integration-airflow-new.mht python3.13[859127]: INFO: Application startup complete. Oct 28 16:39:24 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:39622 - "GET /api/v2/monitor/health HTTP/1.1" 200 OK Oct 28 16:39:24 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39590 - "GET /ui/dashboard/dag_stats HTTP/1.1" 200 OK Oct 28 16:39:24 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39600 - "GET /api/v2/pools HTTP/1.1" 200 OK Oct 28 16:39:24 integration-airflow-new.mht python3.13[859126]: INFO: 127.0.0.1:39616 - "GET /api/v2/dags/~/dagRuns/~/hitlDetails?state=deferred&response_received=false HTTP/1.1" 200 OK Oct 28 16:39:26 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39630 - "GET /ui/dashboard/historical_metrics_data?start_date=2025-10-27T05%3A42%3A52.587Z HTTP/1.1" 200 OK Oct 28 16:39:27 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39640 - "GET / HTTP/1.1" 200 OK Oct 28 16:39:27 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39656 - "GET /static/assets/index-BjF9k7EU.js HTTP/1.1" 200 OK Oct 28 16:39:28 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39670 - "GET /ui/config HTTP/1.1" 401 Unauthorized Oct 28 16:39:28 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39716 - "GET /static/i18n/locales/en/browse.json HTTP/1.1" 200 OK Oct 28 16:39:28 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:39700 - "GET /static/i18n/locales/en/dashboard.json HTTP/1.1" 200 OK Oct 28 16:39:28 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39686 - "GET /static/i18n/locales/en/dags.json HTTP/1.1" 200 OK Oct 28 16:39:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39738 - "GET /api/v2/auth/login?next=https%3A%2F%2Fintegration-airflow-new.mht%2F HTTP/1.1" 307 Temporary Redirect Oct 28 16:39:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39752 - "GET /ui/config HTTP/1.1" 401 Unauthorized Oct 28 16:39:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39768 - "GET /ui/auth/menus HTTP/1.1" 401 Unauthorized Oct 28 16:39:29 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:39782 - "GET /api/v2/plugins HTTP/1.1" 401 Unauthorized Oct 28 16:39:29 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:39780 - "GET /api/v2/version HTTP/1.1" 200 OK Oct 28 16:39:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39790 - "GET /static/i18n/locales/en/components.json HTTP/1.1" 200 OK Oct 28 16:39:29 integration-airflow-new.mht python3.13[859126]: INFO: 127.0.0.1:39848 - "GET /api/v2/auth/login?next=https%3A%2F%2Fintegration-airflow-new.mht%2F HTTP/1.1" 307 Temporary Redirect Oct 28 16:39:29 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:39854 - "GET /auth/login/?next=https://integration-airflow-new.mht/ HTTP/1.1" 302 Found Oct 28 16:39:30 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:39856 - "GET /auth/ HTTP/1.1" 302 Found Oct 28 16:39:30 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39868 - "GET /static/i18n/locales/en/components.json HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859126]: INFO: 127.0.0.1:39874 - "GET /static/i18n/locales/en/browse.json HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39886 - "GET /static/i18n/locales/en/common.json HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39908 - "GET /static/i18n/locales/en/dags.json HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:39914 - "GET /static/i18n/locales/en/admin.json HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:39894 - "GET / HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:39930 - "GET /static/i18n/locales/en/assets.json HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:39938 - "GET /static/i18n/locales/en/dashboard.json HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:39948 - "GET /ui/auth/menus HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:56910 - "GET /api/v2/version HTTP/1.1" 200 OK Oct 28 16:39:30 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:56918 - "GET /static/i18n/locales/en/hitl.json HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The hide_paused_dags_by_default option in [webserver] has been moved to the hide_paused_dags_by_default option in [api] - the old setting has been used, but please update your config. Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The page_size option in [webserver] has been moved to the page_size option in [api] - the old setting has been used, but please update your config. Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The default_wrap option in [webserver] has been moved to the default_wrap option in [api] - the old setting has been used, but please update your config. Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The auto_refresh_interval option in [webserver] has been moved to the auto_refresh_interval option in [api] - the old setting has been used, but please update your config. Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:56926 - "GET /ui/config HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:56952 - "GET /static/i18n/locales/en/dag.json HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:56932 - "GET /ui/auth/menus HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:56942 - "GET /api/v2/version HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:56974 - "GET /api/v2/plugins/importErrors HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:56962 - "GET /api/v2/importErrors HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:56966 - "GET /ui/dashboard/dag_stats HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:56982 - "GET /ui/dashboard/historical_metrics_data?start_date=2025-10-27T16%3A39%3A31.068Z HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:56996 - "GET /api/v2/assets/events?limit=6&order_by=-timestamp×tamp_gte=2025-10-27T16%3A39%3A31.068Z×tamp_lte=2025-10-28T16%3A39%3A31.068Z HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:57020 - "GET /api/v2/plugins HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:57036 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:57058 - "GET /api/v2/importErrors?limit=15&offset=0 HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:57052 - "GET /api/v2/monitor/health HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:57004 - "GET /ui/dags?limit=10&is_favorite=true HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:57048 - "GET /api/v2/dags/~/dagRuns/~/hitlDetails?state=deferred&response_received=false HTTP/1.1" 200 OK Oct 28 16:39:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:57054 - "GET /api/v2/pools HTTP/1.1" 200 OK Oct 28 16:40:01 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:56330 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:40:19 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:38542 - "GET /api/v2/dagTags?limit=10&offset=0&order_by=name&tag_name_pattern= HTTP/1.1" 200 OK Oct 28 16:40:19 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:38552 - "GET /ui/dags?dag_runs_limit=14&limit=100&offset=0&exclude_stale=true&order_by=dag_display_name HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859126]: INFO: 127.0.0.1:38578 - "GET /ui/next_run_assets/asset_alias_example_alias_consumer_with_no_taskflow HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:38596 - "GET /ui/next_run_assets/asset_consumes_1_and_2 HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:38586 - "GET /ui/next_run_assets/asset_consumes_1 HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:38566 - "GET /ui/next_run_assets/asset_alias_example_alias_consumer HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:38600 - "GET /ui/next_run_assets/asset_consumes_1_never_scheduled HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:38604 - "GET /ui/next_run_assets/asset_consumes_unknown_never_scheduled HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:38606 - "GET /ui/next_run_assets/asset_s3_bucket_consumer HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:38616 - "GET /ui/next_run_assets/asset_s3_bucket_consumer_with_no_taskflow HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:37124 - "GET /ui/next_run_assets/conditional_asset_and_time_based_timetable HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:37138 - "GET /ui/next_run_assets/consume_1_or_2_with_asset_expressions HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:37130 - "GET /ui/next_run_assets/consume_1_and_2_with_asset_expressions HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:37142 - "GET /ui/next_run_assets/consume_1_or_both_2_and_3_with_asset_expressions HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:37154 - "GET /ui/next_run_assets/consumes_asset_decorator HTTP/1.1" 200 OK Oct 28 16:40:20 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:37170 - "GET /ui/next_run_assets/example_asset_with_watchers HTTP/1.1" 200 OK Oct 28 16:40:22 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:37184 - "GET /ui/dashboard/historical_metrics_data?start_date=2025-10-27T16%3A40%3A21.846Z HTTP/1.1" 200 OK Oct 28 16:40:22 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:37174 - "GET /api/v2/assets/events?limit=6&order_by=-timestamp×tamp_gte=2025-10-27T16%3A40%3A21.846Z×tamp_lte=2025-10-28T16%3A40%3A21.846Z HTTP/1.1" 200 OK Oct 28 16:40:27 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/services/public/config.py:29 DeprecationWarning: The expose_config option in [webserver] has been moved to the expose_config option in [api] - the old setting has been used, but please update your config. Oct 28 16:40:27 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/services/public/config.py:33 DeprecationWarning: The expose_config option in [webserver] has been moved to the expose_config option in [api] - the old setting has been used, but please update your config. Oct 28 16:40:27 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:37188 - "GET /api/v2/config HTTP/1.1" 403 Forbidden Oct 28 16:40:33 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:38918 - "GET /api/v2/pools?limit=100&offset=0&order_by=name HTTP/1.1" 200 OK Oct 28 16:40:52 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:59174 - "GET /api/v2/assets/events?limit=6&order_by=-timestamp×tamp_gte=2025-10-27T16%3A40%3A51.859Z×tamp_lte=2025-10-28T16%3A40%3A51.859Z HTTP/1.1" 200 OK Oct 28 16:40:52 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:59186 - "GET /ui/dashboard/historical_metrics_data?start_date=2025-10-27T16%3A40%3A51.859Z HTTP/1.1" 200 OK Oct 28 16:41:22 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:33724 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:41:52 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:35568 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:42:22 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:59920 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:42:52 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:38480 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:43:22 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46440 - "GET /api/v2/assets?limit=100&offset=0&name_pattern= HTTP/1.1" 200 OK Oct 28 16:52:21 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45728 - "GET /ui/dags?dag_runs_limit=1&limit=100&offset=0&exclude_stale=true&paused=false&order_by=dag_display_name HTTP/1.1" 200 OK Oct 28 16:52:22 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The hide_paused_dags_by_default option in [webserver] has been moved to the hide_paused_dags_by_default option in [api] - the old setting has been used, but please update your config. Oct 28 16:52:22 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The page_size option in [webserver] has been moved to the page_size option in [api] - the old setting has been used, but please update your config. Oct 28 16:52:22 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The default_wrap option in [webserver] has been moved to the default_wrap option in [api] - the old setting has been used, but please update your config. Oct 28 16:52:22 integration-airflow-new.mht python3.13[859127]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The auto_refresh_interval option in [webserver] has been moved to the auto_refresh_interval option in [api] - the old setting has been used, but please update your config. Oct 28 16:52:22 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45742 - "GET /ui/config HTTP/1.1" 200 OK Oct 28 16:52:22 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45746 - "GET /ui/dags?is_favorite=true HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45264 - "GET /ui/config HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859126]: INFO: 127.0.0.1:45306 - "GET /api/v2/plugins/importErrors HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45268 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:45328 - "GET /api/v2/assets/events?limit=6&order_by=-timestamp×tamp_gte=2025-10-27T16%3A55%3A31.212Z×tamp_lte=2025-10-28T16%3A55%3A31.212Z HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:45292 - "GET /api/v2/importErrors HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859126]: INFO: 127.0.0.1:45296 - "GET /ui/dashboard/historical_metrics_data?start_date=2025-10-27T16%3A55%3A31.212Z HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:45270 - "GET /ui/dashboard/dag_stats HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45342 - "GET /api/v2/monitor/health HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:45320 - "GET /api/v2/dags/~/dagRuns/~/hitlDetails?state=deferred&response_received=false HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45354 - "GET /api/v2/plugins HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:45366 - "GET /ui/auth/menus HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45382 - "GET /api/v2/importErrors?limit=15&offset=0 HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:45312 - "GET /ui/dags?limit=10&is_favorite=true HTTP/1.1" 200 OK Oct 28 16:55:31 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:45380 - "GET /api/v2/pools HTTP/1.1" 200 OK Oct 28 16:56:01 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:33916 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:56:32 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:45832 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:57:02 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:49524 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:57:32 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:50834 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 16:58:02 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:59562 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:46536 - "GET /api/v2/plugins HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The hide_paused_dags_by_default option in [webserver] has been moved to the hide_paused_dags_by_default option in [api] - the old setting has been used, but please update your config. Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The page_size option in [webserver] has been moved to the page_size option in [api] - the old setting has been used, but please update your config. Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The default_wrap option in [webserver] has been moved to the default_wrap option in [api] - the old setting has been used, but please update your config. Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: /opt/airflow/.local/lib/python3.13/site-packages/airflow/api_fastapi/core_api/routes/ui/config.py:52 DeprecationWarning: The auto_refresh_interval option in [webserver] has been moved to the auto_refresh_interval option in [api] - the old setting has been used, but please update your config. Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:46526 - "GET /ui/config HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46552 - "GET /ui/auth/menus HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:46566 - "GET /api/v2/dags/~/dagRuns?state=running&state=queued HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:46584 - "GET /api/v2/version HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:46600 - "GET /ui/dashboard/dag_stats HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46576 - "GET /api/v2/dags/~/dagRuns/~/hitlDetails?state=deferred&response_received=false HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46614 - "GET /api/v2/importErrors HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46618 - "GET /api/v2/plugins/importErrors HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46630 - "GET /ui/dags?limit=10&is_favorite=true HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:46644 - "GET /api/v2/monitor/health HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46660 - "GET /api/v2/pools HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46670 - "GET /ui/dashboard/historical_metrics_data?start_date=2025-10-27T16%3A55%3A31.212Z HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46684 - "GET /api/v2/assets/events?limit=6&order_by=-timestamp×tamp_gte=2025-10-27T16%3A55%3A31.212Z×tamp_lte=2025-10-28T16%3A55%3A31.212Z HTTP/1.1" 200 OK Oct 28 20:30:06 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:46686 - "GET /api/v2/importErrors?limit=15&offset=0 HTTP/1.1" 200 OK Oct 29 15:00:26 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:51206 - "GET /dags?paused=false HTTP/1.1" 200 OK Oct 29 15:00:27 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:51216 - "GET /static/assets/index-BjF9k7EU.js HTTP/1.1" 200 OK Oct 29 15:00:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:51222 - "GET /ui/config HTTP/1.1" 401 Unauthorized Oct 29 15:00:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:51232 - "GET /static/i18n/locales/en/common.json HTTP/1.1" 200 OK Oct 29 15:00:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:51236 - "GET /api/v2/auth/login?next=https%3A%2F%2F172.23.4.38%2Fdags%3Fpaused%3Dfalse HTTP/1.1" 307 Temporary Redirect Oct 29 15:00:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:51246 - "GET /static/i18n/locales/en/components.json HTTP/1.1" 200 OK Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: [2025-10-29T15:00:29.482378Z] {app.py:1744} ERROR - Exception on /login/ [GET] Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: Traceback (most recent call last): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: self.dialect.do_execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: cursor, str_statement, effective_parameters, context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: cursor.execute(statement, parameters) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 179, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: res = self._query(mogrified_query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 330, in _query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: db.query(q) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/connections.py", line 280, in query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: _mysql.connection.query(self, query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: MySQLdb.OperationalError: (4031, 'The client was disconnected by the server because of inactivity. See wait_timeout and interactive_timeout for configuring this behavior.') Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: The above exception was the direct cause of the following exception: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: Traceback (most recent call last): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2529, in wsgi_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: response = self.full_dispatch_request() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 1825, in full_dispatch_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: rv = self.handle_user_exception(e) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 1821, in full_dispatch_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: rv = self.preprocess_request() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2313, in preprocess_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: rv = self.ensure_sync(before_func)() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/security_manager.py", line 57, in before_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: g.user = get_auth_manager().get_user() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py", line 278, in get_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: if current_user.is_anonymous and getattr(g, "user", None) is not None and not g.user.is_anonymous: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/werkzeug/local.py", line 316, in __get__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: obj = instance._get_current_object() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/werkzeug/local.py", line 520, in _get_current_object Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return get_name(local()) # type: ignore Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 25, in <lambda> Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: current_user = LocalProxy(lambda: _get_user()) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 370, in _get_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: current_app.login_manager._load_user() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/login_manager.py", line 364, in _load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: user = self._user_callback(user_id) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1384, in load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: user = self.get_user_by_id(int(pk)) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1390, in get_user_by_id Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return self.session.get(self.user_model, pk) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 3680, in get Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return self._get_impl( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: entity, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ...<7 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 3859, in _get_impl Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return db_load_fn( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ...<5 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/loading.py", line 695, in load_on_pk_identity Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: session.execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: q, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ...<2 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 2351, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return self._execute_internal( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: statement, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ...<4 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: _add_event=_add_event, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: result: Result[Any] = compile_state_cls.orm_execute_statement( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ...<4 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: conn, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: result = conn.execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: statement, params or {}, execution_options=execution_options Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return meth( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: distilled_parameters, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: execution_options or NO_OPTIONS, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return connection._execute_clauseelement( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: self, distilled_params, execution_options Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ret = self._execute_context( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: dialect, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ...<8 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: cache_hit=cache_hit, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: return self._exec_single_context( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: dialect, context, statement, parameters Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: self._handle_dbapi_exception( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: e, str_statement, effective_parameters, cursor, context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: raise sqlalchemy_exception.with_traceback(exc_info[2]) from e Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: self.dialect.do_execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: cursor, str_statement, effective_parameters, context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: cursor.execute(statement, parameters) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 179, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: res = self._query(mogrified_query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 330, in _query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: db.query(q) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/connections.py", line 280, in query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: _mysql.connection.query(self, query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (4031, 'The client was disconnected by the server because of inactivity. See wait_timeout and interactive_timeout for configuring this behavior.') Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: [SQL: SELECT ab_user.id AS ab_user_id, ab_user.first_name AS ab_user_first_name, ab_user.last_name AS ab_user_last_name, ab_user.username AS ab_user_username, ab_user.password AS ab_user_password, ab_user.active AS ab_user_active, ab_user.email AS ab_user_email, ab_user.last_login AS ab_user_last_login, ab_user.login_count AS ab_user_login_count, ab_user.fail_login_count AS ab_user_fail_login_count, ab_user.created_on AS ab_user_created_on, ab_user.changed_on AS ab_user_changed_on, ab_user.created_by_fk AS ab_user_created_by_fk, ab_user.changed_by_fk AS ab_user_changed_by_fk Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: FROM ab_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: WHERE ab_user.id = %s] Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: [parameters: (1,)] Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: (Background on this error at: https://sqlalche.me/e/20/e3q8) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: INFO: 127.0.0.1:51254 - "GET /auth/login/?next=https://172.23.4.38/dags?paused=false HTTP/1.1" 500 Internal Server Error Oct 29 15:00:29 integration-airflow-new.mht python3.13[859124]: INFO: 127.0.0.1:51266 - "GET /static/i18n/locales/en/hitl.json HTTP/1.1" 200 OK Oct 29 15:00:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:51278 - "GET /static/i18n/locales/en/dashboard.json HTTP/1.1" 200 OK Oct 29 15:00:29 integration-airflow-new.mht python3.13[859127]: INFO: 127.0.0.1:51290 - "GET /static/i18n/locales/en/admin.json HTTP/1.1" 200 OK Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: ERROR: Exception in ASGI application Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: + Exception Group Traceback (most recent call last): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | result = await app( # type: ignore[func-returns-value] Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self.scope, self.receive, self.send Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/fastapi/applications.py", line 1082, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await super().__call__(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/applications.py", line 113, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.middleware_stack(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 186, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | raise exc Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 164, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.app(scope, receive, _send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/gzip.py", line 29, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await responder(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/gzip.py", line 130, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await super().__call__(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/gzip.py", line 46, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.app(scope, receive, self.send_with_compression) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/cors.py", line 85, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.app(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 63, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | raise exc Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await app(scope, receive, sender) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/routing.py", line 716, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.middleware_stack(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/routing.py", line 736, in app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await route.handle(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/routing.py", line 462, in handle Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.app(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/fastapi/applications.py", line 1082, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await super().__call__(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/applications.py", line 113, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.middleware_stack(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 186, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | raise exc Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/errors.py", line 164, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.app(scope, receive, _send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/exceptions.py", line 63, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | raise exc Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await app(scope, receive, sender) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/routing.py", line 716, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.middleware_stack(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/routing.py", line 736, in app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await route.handle(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/routing.py", line 462, in handle Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await self.app(scope, receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/wsgi.py", line 81, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await responder(receive, send) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/wsgi.py", line 106, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | async with anyio.create_task_group() as task_group: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 781, in __aexit__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | raise BaseExceptionGroup( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | "unhandled errors in a TaskGroup", self._exceptions Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) from None Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: +-+---------------- 1 ---------------- Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Traceback (most recent call last): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self.dialect.do_execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | cursor, str_statement, effective_parameters, context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | cursor.execute(statement, parameters) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 179, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | res = self._query(mogrified_query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 330, in _query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | db.query(q) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/connections.py", line 280, in query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | _mysql.connection.query(self, query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | MySQLdb.OperationalError: (4031, 'The client was disconnected by the server because of inactivity. See wait_timeout and interactive_timeout for configuring this behavior.') Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | The above exception was the direct cause of the following exception: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Traceback (most recent call last): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2529, in wsgi_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | response = self.full_dispatch_request() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 1825, in full_dispatch_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | rv = self.handle_user_exception(e) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 1821, in full_dispatch_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | rv = self.preprocess_request() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2313, in preprocess_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | rv = self.ensure_sync(before_func)() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/security_manager.py", line 57, in before_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | g.user = get_auth_manager().get_user() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py", line 278, in get_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | if current_user.is_anonymous and getattr(g, "user", None) is not None and not g.user.is_anonymous: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/werkzeug/local.py", line 316, in __get__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | obj = instance._get_current_object() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/werkzeug/local.py", line 520, in _get_current_object Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return get_name(local()) # type: ignore Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 25, in <lambda> Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | current_user = LocalProxy(lambda: _get_user()) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 370, in _get_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | current_app.login_manager._load_user() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/login_manager.py", line 364, in _load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | user = self._user_callback(user_id) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1384, in load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | user = self.get_user_by_id(int(pk)) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1390, in get_user_by_id Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self.session.get(self.user_model, pk) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 3680, in get Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self._get_impl( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | entity, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | db.query(q) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/connections.py", line 280, in query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | _mysql.connection.query(self, query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | MySQLdb.OperationalError: (4031, 'The client was disconnected by the server because of inactivity. See wait_timeout and interactive_timeout for configuring this behavior.') Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | The above exception was the direct cause of the following exception: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Traceback (most recent call last): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2529, in wsgi_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | response = self.full_dispatch_request() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 1825, in full_dispatch_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | rv = self.handle_user_exception(e) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 1821, in full_dispatch_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | rv = self.preprocess_request() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2313, in preprocess_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | rv = self.ensure_sync(before_func)() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/security_manager.py", line 57, in before_request Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | g.user = get_auth_manager().get_user() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py", line 278, in get_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | if current_user.is_anonymous and getattr(g, "user", None) is not None and not g.user.is_anonymous: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/werkzeug/local.py", line 316, in __get__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | obj = instance._get_current_object() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/werkzeug/local.py", line 520, in _get_current_object Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return get_name(local()) # type: ignore Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 25, in <lambda> Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | current_user = LocalProxy(lambda: _get_user()) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 370, in _get_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | current_app.login_manager._load_user() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/login_manager.py", line 364, in _load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | user = self._user_callback(user_id) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1384, in load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | user = self.get_user_by_id(int(pk)) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1390, in get_user_by_id Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self.session.get(self.user_model, pk) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 3680, in get Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self._get_impl( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | entity, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<7 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 3859, in _get_impl Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return db_load_fn( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<5 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/loading.py", line 695, in load_on_pk_identity Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | session.execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | q, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<2 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 2351, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self._execute_internal( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | statement, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<4 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | _add_event=_add_event, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | result: Result[Any] = compile_state_cls.orm_execute_statement( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<4 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | conn, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | result = conn.execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | statement, params or {}, execution_options=execution_options Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return meth( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | distilled_parameters, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | execution_options or NO_OPTIONS, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return connection._execute_clauseelement( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, distilled_params, execution_options Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ret = self._execute_context( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | dialect, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<8 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | cache_hit=cache_hit, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self._exec_single_context( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | dialect, context, statement, parameters Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self._handle_dbapi_exception( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | e, str_statement, effective_parameters, cursor, context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 2355, in _handle_dbapi_exception Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | raise sqlalchemy_exception.with_traceback(exc_info[2]) from e Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self.dialect.do_execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | cursor, str_statement, effective_parameters, context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/default.py", line 951, in do_execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | cursor.execute(statement, parameters) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 179, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | res = self._query(mogrified_query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/cursors.py", line 330, in _query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | db.query(q) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/MySQLdb/connections.py", line 280, in query Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | _mysql.connection.query(self, query) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (4031, 'The client was disconnected by the server because of inactivity. See wait_timeout and interactive_timeout for configuring this behavior.') Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | [SQL: SELECT ab_user.id AS ab_user_id, ab_user.first_name AS ab_user_first_name, ab_user.last_name AS ab_user_last_name, ab_user.username AS ab_user_username, ab_user.password AS ab_user_password, ab_user.active AS ab_user_active, ab_user.email AS ab_user_email, ab_user.last_login AS ab_user_last_login, ab_user.login_count AS ab_user_login_count, ab_user.fail_login_count AS ab_user_fail_login_count, ab_user.created_on AS ab_user_created_on, ab_user.changed_on AS ab_user_changed_on, ab_user.created_by_fk AS ab_user_created_by_fk, ab_user.changed_by_fk AS ab_user_changed_by_fk Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | FROM ab_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | WHERE ab_user.id = %s] Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | [parameters: (1,)] Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | (Background on this error at: https://sqlalche.me/e/20/e3q8) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | During handling of the above exception, another exception occurred: Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | Traceback (most recent call last): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/wsgi.py", line 109, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | await anyio.to_thread.run_sync(self.wsgi, environ, self.start_response) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/anyio/to_thread.py", line 56, in run_sync Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return await get_async_backend().run_sync_in_worker_thread( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | func, args, abandon_on_cancel=abandon_on_cancel, limiter=limiter Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 2485, in run_sync_in_worker_thread Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return await future Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 976, in run Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | result = context.run(func, *args) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/starlette/middleware/wsgi.py", line 147, in wsgi Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | for chunk in self.app(environ, start_response): Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2552, in __call__ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self.wsgi_app(environ, start_response) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 2532, in wsgi_app Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | response = self.handle_exception(e) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 1727, in handle_exception Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | server_error = self.ensure_sync(handler)(server_error) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/www/views.py", line 82, in show_traceback Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return render_template("airflow/traceback.html"), 500 Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/templating.py", line 147, in render_template Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return _render(app, template, context) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/templating.py", line 128, in _render Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | app.update_template_context(context) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask/app.py", line 997, in update_template_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | context.update(func()) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 405, in _user_context_processor Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return dict(current_user=_get_user()) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/utils.py", line 370, in _get_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | current_app.login_manager._load_user() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/flask_login/login_manager.py", line 364, in _load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | user = self._user_callback(user_id) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1384, in load_user Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | user = self.get_user_by_id(int(pk)) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/airflow/providers/fab/auth_manager/security_manager/override.py", line 1390, in get_user_by_id Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self.session.get(self.user_model, pk) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 3680, in get Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self._get_impl( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | entity, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<7 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 3859, in _get_impl Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return db_load_fn( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<5 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/loading.py", line 695, in load_on_pk_identity Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | session.execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | q, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<2 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | bind_arguments=bind_arguments, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 2351, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return self._execute_internal( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | statement, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<4 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | _add_event=_add_event, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | result: Result[Any] = compile_state_cls.orm_execute_statement( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<4 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | conn, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | result = conn.execute( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | statement, params or {}, execution_options=execution_options Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1419, in execute Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return meth( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | distilled_parameters, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | execution_options or NO_OPTIONS, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/sql/elements.py", line 526, in _execute_on_connection Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | return connection._execute_clauseelement( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self, distilled_params, execution_options Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ret = self._execute_context( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | dialect, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<8 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | cache_hit=cache_hit, Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 1813, in _execute_context Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | conn = self._revalidate_connection() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 673, in _revalidate_connection Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | self._invalid_transaction() Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ~~~~~~~~~~~~~~~~~~~~~~~~~^^ Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | File "/opt/airflow/.local/lib/python3.13/site-packages/sqlalchemy/engine/base.py", line 663, in _invalid_transaction Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | raise exc.PendingRollbackError( Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ...<4 lines>... Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | ) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: | sqlalchemy.exc.PendingRollbackError: Can't reconnect until invalid transaction is rolled back. Please rollback() fully before proceeding (Background on this error at: https://sqlalche.me/e/20/8s2b) Oct 29 15:00:29 integration-airflow-new.mht python3.13[859125]: +------------------------------------ ``` We got frustrated and blew away the database using the `airflow db reset` command - it deleted the db and kicked off the migrate. _Every time_ the migration will get to this point and freeze: (we have mysql query logging enabled) ``` 2025-10-29T22:39:59.025609Z 3264 Query SET time_zone = '+00:00' 2025-10-29T22:39:59.026833Z 3264 Query DESCRIBE `airflow`.`alembic_version_fab` 2025-10-29T22:39:59.027421Z 3264 Query DESCRIBE `airflow`.`alembic_version_fab` 2025-10-29T22:39:59.028196Z 3264 Query CREATE TABLE alembic_version_fab ( version_num VARCHAR(32) NOT NULL, CONSTRAINT alembic_version_fab_pkc PRIMARY KEY (version_num) ) ``` When we eventually kill the migration, the slow query spits out an error indicating _that_ query took "however long" to complete. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
