GitHub user saaifali added a comment to the discussion: SSL:
CERTIFICATE_VERIFY_FAILED in airflow-worker for certificates with internal CA
I am facing a similar issue here.
Using Airflow 3.1.1
After Debugging and going through the Airflow code I have noticed the following
which I will try my best to explain here :
When creating an HTTP Client in `airflow.sdk.api.client.py` lines 829 - 832 :
```
ctx = ssl.create_default_context(cafile=certifi.where())
if API_SSL_CERT_PATH:
ctx.load_verify_locations(API_SSL_CERT_PATH)
kwargs["verify"] = ctx
```
Here as we can see, a default Context gets created using `certifi.where()`
which returns : '/etc/ssl/certs/ca-certificates.crt'
It also allows us to load another context into it from the configuration var :
**API_SSL_CERT_PATH** which is the `ssl_cert` config in [api] in airflow.cfg
The problem is that there are __two completely different SSL configurations in
Airflow that serve different purposes, and they're being confused:
## 1. __Server-side SSL Configuration__
__Location__: `airflow/cli/commands/api_server_command.py` in the
`_get_ssl_cert_and_key_filepaths()` function (lines 147-160)
__Purpose__: This is for the __Airflow API server__ to serve HTTPS connections
(server-side SSL)
__Configuration__:
- `[api] ssl_cert` - Server certificate file
- `[api] ssl_key` - Server private key file
__The Restriction__: Lines 155-158 enforce that __both__ ssl_cert AND ssl_key
must be provided together:
This makes sense because a web server needs both a certificate AND its private
key to serve HTTPS.
Now what I want is Client Side verification of the URL which uses the same
`ssl_cert` configuration from above. Which does not really have a `ssl_key`
The client-side SSL verification code is incorrectly reusing the server-side
SSL certificate configuration. The API_SSL_CERT_PATH in client.py is reading
from [api] ssl_cert, which is intended for server certificates, not CA
certificates for client verification.
hence the `ssl_cert` has two completely different use cases. And it would be
good to move the second use case to use a different config value like
`ssl_ca_cert` or something similar.
Currently all we can do is patch or replace the file returned by
`certifi.where()`.
I dont know if my analysis is completely accurate here and I would love to
contribute and help fix this up if there is a trust in this solution.
:)
GitHub link:
https://github.com/apache/airflow/discussions/56590#discussioncomment-14828016
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
