anavrotski opened a new issue, #63671: URL: https://github.com/apache/airflow/issues/63671
### Apache Airflow Provider(s) fab ### Versions of Apache Airflow Providers apache-airflow-providers-fab 3.4.0 ### Apache Airflow version 3.1.8 ### Operating System linux ### Deployment Official Apache Airflow Helm Chart ### Deployment details Airflow - AWS EKS K8s cluster, database - MySQL AWS RDS. ### What happened After several hours, Airflow UI becomes unavailable, and an exception is raised. Confirmed in two different environments (and different database servers), so it is not a db problem. ``` api-server await super().__call__(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__ api-server await self.middleware_stack(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__ api-server raise exc api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__ api-server await self.app(scope, receive, _send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__ api-server await self.app(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 29, in __call__ api-server await responder(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 130, in __call__ api-server await super().__call__(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 46, in __call__ api-server await self.app(scope, receive, self.send_with_compression) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 182, in __call__ api-server with recv_stream, send_stream, collapse_excgroups(): api-server ^^^^^^^^^^^^^^^^^^^^ api-server File "/usr/python/lib/python3.12/contextlib.py", line 158, in __exit__ api-server self.gen.throw(value) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_utils.py", line 85, in collapse_excgroups api-server raise exc api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 184, in __call__ api-server response = await self.dispatch_func(request, call_next) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py", line 49, in dispatch api-server new_user, current_user = await self._refresh_user(current_token) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py", line 81, in _refresh_user api-server user = await resolve_user_from_token(current_token) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/core_api/security.py", line 103, in resolve_user_from_token api-server return await get_auth_manager().get_user_from_token(token_str) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py", line 111, in get_user_from_token api-server return self.deserialize_user(payload) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/cachetools/_cachedmethod.py", line 380, in __call__ api-server return wrapper(self._obj, *args, **kwargs) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/cachetools/_cachedmethod.py", line 363, in wrapper api-server v = method(self, *args, **kwargs) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py", line 296, in deserialize_user api-server return self.session.scalars(select(User).where(User.id == int(token["sub"]))).one() api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 1891, in scalars api-server return self._proxied.scalars( api-server ^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars api-server return self._execute_internal( api-server ^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal api-server result: Result[Any] = compile_state_cls.orm_execute_statement( api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement api-server result = conn.execute( api-server ^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute api-server return meth( api-server ^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 527, in _execute_on_connection api-server return connection._execute_clauseelement( api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement api-server ret = self._execute_context( api-server ^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context api-server return self._exec_single_context( api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context api-server self._handle_dbapi_exception( api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception api-server raise sqlalchemy_exception.with_traceback(exc_info[2]) from e api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context api-server self.dialect.do_execute( api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute api-server cursor.execute(statement, parameters) api-server File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute api-server res = self._query(mogrified_query) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 330, in _query api-server db.query(q) api-server File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/connections.py", line 286, in query api-server _mysql.connection.query(self, query) api-server sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'Server has gone away') api-server [SQL: SELECT ab_user.id, ab_user.first_name, ab_user.last_name, ab_user.username, ab_user.password, ab_user.active, ab_user.email, ab_user.last_login, ab_user.login_count, ab_user.fail_login_count, ab_user.created_on, ab_user.changed_on, ab_user.created_by_fk, ab_user.changed_by_fk api-server FROM ab_user api-server WHERE ab_user.id = %s] api-server [parameters: (2,)] api-server (Background on this error at: https://sqlalche.me/e/20/e3q8) ``` ### What you think should happen instead _No response_ ### How to reproduce Deploy Airflow 3.1.8 with fab provider v3.4.0 and MySQL as Airflow's metadata database. ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
