GitHub user anavrotski created a discussion: FAB Provider - 'Server has gone away'
### Apache Airflow Provider(s) fab ### Versions of Apache Airflow Providers apache-airflow-providers-fab 3.4.0 ### Apache Airflow version 3.1.8 ### Operating System linux ### Deployment Official Apache Airflow Helm Chart ### Deployment details Airflow - AWS EKS K8s cluster, database - MySQL AWS RDS. ### What happened After several hours, Airflow UI becomes unavailable, and an exception is raised. Confirmed in two different environments (and different database servers), so it is not a db problem. ``` api-server await super().__call__(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/applications.py", line 113, in __call__ api-server await self.middleware_stack(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 186, in __call__ api-server raise exc api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/errors.py", line 164, in __call__ api-server await self.app(scope, receive, _send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__ api-server await self.app(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 29, in __call__ api-server await responder(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 130, in __call__ api-server await super().__call__(scope, receive, send) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/gzip.py", line 46, in __call__ api-server await self.app(scope, receive, self.send_with_compression) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 182, in __call__ api-server with recv_stream, send_stream, collapse_excgroups(): api-server ^^^^^^^^^^^^^^^^^^^^ api-server File "/usr/python/lib/python3.12/contextlib.py", line 158, in __exit__ api-server self.gen.throw(value) api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/_utils.py", line 85, in collapse_excgroups api-server raise exc api-server File "/home/airflow/.local/lib/python3.12/site-packages/starlette/middleware/base.py", line 184, in __call__ api-server response = await self.dispatch_func(request, call_next) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py", line 49, in dispatch api-server new_user, current_user = await self._refresh_user(current_token) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py", line 81, in _refresh_user api-server user = await resolve_user_from_token(current_token) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/core_api/security.py", line 103, in resolve_user_from_token api-server return await get_auth_manager().get_user_from_token(token_str) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/api_fastapi/auth/managers/base_auth_manager.py", line 111, in get_user_from_token api-server return self.deserialize_user(payload) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/cachetools/_cachedmethod.py", line 380, in __call__ api-server return wrapper(self._obj, *args, **kwargs) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/cachetools/_cachedmethod.py", line 363, in wrapper api-server v = method(self, *args, **kwargs) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/airflow/providers/fab/auth_manager/fab_auth_manager.py", line 296, in deserialize_user api-server return self.session.scalars(select(User).where(User.id == int(token["sub"]))).one() api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/scoping.py", line 1891, in scalars api-server return self._proxied.scalars( api-server ^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2459, in scalars api-server return self._execute_internal( api-server ^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/session.py", line 2249, in _execute_internal api-server result: Result[Any] = compile_state_cls.orm_execute_statement( api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/orm/context.py", line 306, in orm_execute_statement api-server result = conn.execute( api-server ^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1419, in execute api-server return meth( api-server ^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 527, in _execute_on_connection api-server return connection._execute_clauseelement( api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1641, in _execute_clauseelement api-server ret = self._execute_context( api-server ^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context api-server return self._exec_single_context( api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1986, in _exec_single_context api-server self._handle_dbapi_exception( api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2363, in _handle_dbapi_exception api-server raise sqlalchemy_exception.with_traceback(exc_info[2]) from e api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1967, in _exec_single_context api-server self.dialect.do_execute( api-server File "/home/airflow/.local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 952, in do_execute api-server cursor.execute(statement, parameters) api-server File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 179, in execute api-server res = self._query(mogrified_query) api-server ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ api-server File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/cursors.py", line 330, in _query api-server db.query(q) api-server File "/home/airflow/.local/lib/python3.12/site-packages/MySQLdb/connections.py", line 286, in query api-server _mysql.connection.query(self, query) api-server sqlalchemy.exc.OperationalError: (MySQLdb.OperationalError) (2006, 'Server has gone away') api-server [SQL: SELECT ab_user.id, ab_user.first_name, ab_user.last_name, ab_user.username, ab_user.password, ab_user.active, ab_user.email, ab_user.last_login, ab_user.login_count, ab_user.fail_login_count, ab_user.created_on, ab_user.changed_on, ab_user.created_by_fk, ab_user.changed_by_fk api-server FROM ab_user api-server WHERE ab_user.id = %s] api-server [parameters: (2,)] api-server (Background on this error at: https://sqlalche.me/e/20/e3q8) ``` ### What you think should happen instead _No response_ ### How to reproduce Deploy Airflow 3.1.8 with fab provider v3.4.0 and MySQL as Airflow's metadata database. ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) GitHub link: https://github.com/apache/airflow/discussions/63718 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
