potiuk commented on PR #37638:
URL: https://github.com/apache/airflow/pull/37638#issuecomment-2056627808
Hello @RobbeSneyders : I think we need your expert help here.
We are quite far with making all our tests work with the new Connexion 3
(basically - they all work and we found and fxed quite a number of
incompatibilities we found) - but we are at the stage of reviewing the way how
we solve some of the problems and how we integrate connexion with the
production setting and we likely need to make sure we do the right thing when
starting Airflow. Some of your guidance and help here would be more than
welcome.
The last remaining issue we have right now is that when we run now is that
we are getting rather cryptic error when running Airflow from a package that
signals that likely the way we run gunicorn with uvicorn is somewhat wrong, so
your help would be greatly appreciated here.
This is the error we have currently when we try to run any http request when
running
```
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.8/site-packages/uvicorn/protocols/http/httptools_impl.py",
line 411, in run_asgi
result = await app( # type: ignore[func-returns-value]
File
"/home/airflow/.local/lib/python3.8/site-packages/uvicorn/middleware/proxy_headers.py",
line 69, in __call__
return await self.app(scope, receive, send)
File
"/home/airflow/.local/lib/python3.8/site-packages/connexion/apps/abstract.py",
line 284, in __call__
return await self.middleware(scope, receive, send)
File
"/home/airflow/.local/lib/python3.8/site-packages/connexion/middleware/main.py",
line 497, in __call__
self.app, self.middleware_stack = self._build_middleware_stack()
File
"/home/airflow/.local/lib/python3.8/site-packages/connexion/middleware/main.py",
line 338, in _build_middleware_stack
app.add_api(
File
"/home/airflow/.local/lib/python3.8/site-packages/connexion/apps/flask.py",
line 141, in add_api
self.app.register_blueprint(api.blueprint)
File "/home/airflow/.local/lib/python3.8/site-packages/flask/scaffold.py",
line 50, in wrapper_func
return f(self, *args, **kwargs)
File "/home/airflow/.local/lib/python3.8/site-packages/flask/app.py", line
1299, in register_blueprint
blueprint.register(self, options)
File
"/home/airflow/.local/lib/python3.8/site-packages/flask/blueprints.py", line
384, in register
raise ValueError(
ValueError: The name '/api/v1' is already registered for a different
blueprint. Use 'name=' to provide a unique name.
127.0.0.1:57988 - "GET /test HTTP/1.1" 500
````
I tried to debug it and see what could be the cause, but I have no clue
currently.
Just to give some more information on how we are running webserver:
```
/home/airflow/.local/bin/python -m gunicorn --workers 1 --worker-class
uvicorn.workers.UvicornWorker --timeout 120 --bind 0.0.0.0:8080 --name
airflow-webserver --pid /opt/airflow/airflow-webserver.pid --config
python:airflow.www.gunicorn_config --access-logfile - --error-logfile -
"airflow.www.app:cached_app()"
```
The interesting thing is that it works fine when we run airflow in our CI
image, while it does not work in the PROD one and so far I cannot pin-point the
root cause of the difference between the two. In the first one airflow is
installed from sources in editable mode essentially, in the second one, it is
installed from packages.
The only clue/difference I see between the two is the PROD one produces
extra "ASGI `livespan` protocol appears unsupported`:
```
[2024-04-15T10:49:50.933+0000] {swagger_ui.py:80} INFO - Adding spec json:
/api/v1/openapi.json
[2024-04-15 10:49:50 +0000] [857] [INFO] ASGI 'lifespan' protocol appears
unsupported.
```
I'd appreciate any insights there - happy to explain/ show what we are doing
there, but the most important thing is here:
https://github.com/apache/airflow/pull/37638/files#diff-c41a5d9e7ffad8150a345beb895d2ca3736de6c3cf9cac4ad093fbcf4c8f36f2R75
where we changed how we initialize the app
Basically what we do there - we follow the earlier advices of yours:
* we initialize connexion via connexion's FlaskApp and register APIs via
`add_api`
* then we use the Flask App created by connexion and we register all our web
components
* we use uvicorn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
