geoffo-dev opened a new issue, #36796: URL: https://github.com/apache/airflow/issues/36796
### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7.3 ### What happened? Hello, I am trying to integrate keycloak with airflow which I have managed successfully in the past however I am currently struggling - possibly due to it being a newer version of airflow where I understand certain authentication mechanisms have changed. Having followed the [guide](https://airflow.apache.org/docs/apache-airflow/2.7.3/security/webserver.html) have the following `webserver_config.py` file: ```python from airflow.www.security import AirflowSecurityManager from flask_appbuilder.security.manager import AUTH_OAUTH import os AUTH_TYPE = AUTH_OAUTH AUTH_ROLES_SYNC_AT_LOGIN = True AUTH_USER_REGISTRATION = True AUTH_ROLE_ADMIN = os.environ['KEYCLOAK_ADMIN_ROLE_NAME'] AUTH_ROLE_OP = os.environ['KEYCLOAK_OP_ROLE_NAME'] AUTH_ROLE_USER = os.environ['KEYCLOAK_USER_ROLE_NAME'] AUTH_ROLE_VIEWER = os.environ['KEYCLOAK_VIEWER_ROLE_NAME'] AUTH_ROLE_PUBLIC = os.environ['KEYCLOAK_PUBLIC_ROLE_NAME'] # Will allow user self registration AUTH_USER_REGISTRATION = True AUTH_USER_REGISTRATION_ROLE = "Public" AUTH_ROLES_SYNC_AT_LOGIN = True AUTH_ROLES_MAPPING = { "Admin": [AUTH_ROLE_ADMIN], "Op": [AUTH_ROLE_OP], "User": [AUTH_ROLE_USER], "Viewer": [AUTH_ROLE_VIEWER], "Public": [AUTH_ROLE_PUBLIC], } PROVIDER_NAME = 'keycloak' CLIENT_ID = os.environ['KEYCLOAK_CLIENT_ID'] # Get the OIDC SECRET with open('/secret/oidc-secret/client-secret', 'r') as file: CLIENT_SECRET = file.read().rstrip() OIDC_ISSUER = os.environ['OIDC_ISSUER_URL'] OIDC_BASE_URL = "{oidc_issuer}/protocol/openid-connect".format(oidc_issuer=OIDC_ISSUER) OIDC_TOKEN_URL = "{oidc_base_url}/token".format(oidc_base_url=OIDC_BASE_URL) OIDC_AUTH_URL = "{oidc_base_url}/auth".format(oidc_base_url=OIDC_BASE_URL) OAUTH_PROVIDERS = [{ 'name': PROVIDER_NAME, 'token_key':'access_token', 'icon':'fa-globe', 'remote_app': { 'client_id': CLIENT_ID, 'client_secret': CLIENT_SECRET, 'api_base_url': OIDC_BASE_URL, 'client_kwargs': { 'scope': 'email profile' }, 'access_token_url': OIDC_TOKEN_URL, 'authorize_url': OIDC_AUTH_URL, 'request_token_url': None, } }] # Make sure to replace this with your own implementation of AirflowSecurityManager class SECURITY_MANAGER_CLASS = KeycloakAuthorizer ``` and the following `keycloakAuthorizer.py` which should be in the same path: ```python from airflow.www.security import AirflowSecurityManager import logging from typing import Any, List, Union import os # Create some logging log = logging.getLogger(__name__) log.setLevel(os.getenv("AIRFLOW__LOGGING__FAB_LOGGING_LEVEL", "INFO")) class KeycloakAuthorizer(AirflowSecurityManager): CLIENT_ID = os.environ['KEYCLOAK_CLIENT_ID'] def get_oauth_user_info(self, provider: str, resp: Any) -> dict[str, Union[str, list[str]]]: remote_app = self.appbuilder.sm.oauth_remotes[provider] me = remote_app.get("user") roles = me["resource_access"][CLIENT_ID]["roles"] if len(roles) < 1: roles = ["public"] userinfo = { "username": me.get("preferred_username"), "email": me.get("email"), "first_name": me.get("given_name"), "last_name": me.get("family_name"), "role_keys": roles, } log.debug(f"User info from Keycloak: {userinfo}\Role info from Keycloak: {roles}") return userinfo ``` However when the webserver launches, there is no option to login via keycloak - only standard username and password. The only error I can see that might be relevant when the webserver loads which relates to the google package (for what I believe is required for keycloak to work): ``` /home/airflow/.local/lib/python3.11/site-packages/flask_limiter/extension.py:336 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend. [2024-01-15 15:53:42 +0000] [13] [INFO] Starting gunicorn 21.2.0 [2024-01-15T15:53:49.840+0000] {providers_manager.py:271} INFO - Optional provider feature disabled when importing 'airflow.providers.google.leveldb.hooks.leveldb.LevelDBHook' from 'apache-airflow-providers-google' package /home/airflow/.local/lib/python3.11/site-packages/snowflake/connector/options.py:103 UserWarning: You have an incompatible version of 'pyarrow' installed (11.0.0), please install a version that adheres to: 'pyarrow<10.1.0,>=10.0.1; extra == "pandas"' [2024-01-15 15:53:53 +0000] [13] [INFO] Listening at: http://0.0.0.0:8080 (13) [2024-01-15 15:53:53 +0000] [13] [INFO] Using worker: sync [2024-01-15 15:53:53 +0000] [25] [INFO] Booting worker with pid: 25 [2024-01-15 15:53:53 +0000] [26] [INFO] Booting worker with pid: 26 [2024-01-15 15:53:53 +0000] [27] [INFO] Booting worker with pid: 27 [2024-01-15 15:53:53 +0000] [28] [INFO] Booting worker with pid: 28 10.38.0.0 - - [15/Jan/2024:15:53:59 +0000] "GET /health HTTP/1.1" 200 318 "-" "kube-probe/1.28+" 10.38.0.0 - - [15/Jan/2024:15:53:59 +0000] "GET /health HTTP/1.1" 200 318 "-" "kube-probe/1.28+" ``` I have tried version 2.8.0 as well, but due to #36702 - this does not work. I am at a bit of a loss as to how to resolve, I have tried a number of images, but not certain if this is supported anymore or if there is another way to resolve. ### What you think should happen instead? User should be presented with a login option with keycloak ### How to reproduce Have described above ### Operating System Tags 2.7.3-python3.11 and 2.7.3 in docker ### Versions of Apache Airflow Providers apache-airflow-providers-amazon==8.10.0 apache-airflow-providers-celery==3.4.1 apache-airflow-providers-cncf-kubernetes==7.8.0 apache-airflow-providers-common-sql==1.8.0 apache-airflow-providers-daskexecutor==1.1.0 apache-airflow-providers-docker==3.8.0 apache-airflow-providers-elasticsearch==5.1.0 apache-airflow-providers-ftp==3.6.0 apache-airflow-providers-google==10.11.0 apache-airflow-providers-grpc==3.3.0 apache-airflow-providers-hashicorp==3.5.0 apache-airflow-providers-http==4.6.0 apache-airflow-providers-imap==3.4.0 apache-airflow-providers-microsoft-azure==8.1.0 apache-airflow-providers-mysql==5.4.0 apache-airflow-providers-odbc==4.1.0 apache-airflow-providers-openlineage==1.2.0 apache-airflow-providers-postgres==5.7.1 apache-airflow-providers-redis==3.4.0 apache-airflow-providers-sendgrid==3.3.0 apache-airflow-providers-sftp==4.7.0 apache-airflow-providers-slack==8.3.0 apache-airflow-providers-snowflake==5.1.0 apache-airflow-providers-sqlite==3.5.0 apache-airflow-providers-ssh==3.8.1 ### Deployment Official Apache Airflow Helm Chart ### Deployment details Deploying on EKS v1.28.3 Helm Version v3.13.1 Helm Chart Version 1.11.0 ### Anything else? Would be happy to support updating a PR, but not really sure where to start! ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
