nicolamarangoni opened a new issue, #25377:
URL: https://github.com/apache/airflow/issues/25377
### Apache Airflow version
2.3.3 (latest released)
### What happened
After upgrading to alpine 3.16.1, Python 3.10 and AirFlow 2.3.3, the
JenkinsJobTriggerOperator cannot connect to Jenkins because it cannot verify
the SSL certificate:
```
[2022-07-28, 16:37:34 CEST] {base.py:68} INFO - Using connection ID
'jenkins' for task execution.
[2022-07-28, 16:37:34 CEST] {warnings.py:109} WARNING -
/usr/lib/python3.10/site-packages/airflow/models/connection.py:294:
DeprecationWarning: Encountered non-JSON in `extra` field for connection
'jenkins'. Support for non-JSON `extra` will be removed in Airflow 3.0
self._validate_extra(extra_val, self.conn_id)
[2022-07-28, 16:37:34 CEST] {jenkins.py:46} INFO - Trying to connect to
[https://jenkins.pharos.pke.fhm.de:443](https://jenkins.pharos.pke.fhm.de/)
[2022-07-28, 16:37:34 CEST] {taskinstance.py:1909} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/usr/lib/python3.10/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py",
line 197, in execute
jenkins_response = self.build_job(jenkins_server, self.parameters)
File
"/usr/lib/python3.10/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py",
line 133, in build_job
return jenkins_request_with_headers(jenkins_server, request)
File
"/usr/lib/python3.10/site-packages/airflow/providers/jenkins/operators/jenkins_job_trigger.py",
line 51, in jenkins_request_with_headers
response = jenkins_server.jenkins_request(req)
File "/usr/lib/python3.10/site-packages/jenkins/__init__.py", line 571, in
jenkins_request
self._maybe_add_auth()
File "/usr/lib/python3.10/site-packages/jenkins/__init__.py", line 410, in
_maybe_add_auth
raise JenkinsException(
jenkins.JenkinsException: Unable to authenticate with any scheme:
auth(kerberos) HTTPSConnectionPool(host='jenkins.pharos.pke.fhm.de',
port=443): Max retries exceeded with url: /api/json (Caused by
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: unable to get local issuer certificate
(_ssl.c:997)')))
auth(basic) HTTPSConnectionPool(host='jenkins.pharos.pke.fhm.de', port=443):
Max retries exceeded with url: /api/json (Caused by
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: unable to get local issuer certificate
(_ssl.c:997)')))
[2022-07-28, 16:37:34 CEST] {taskinstance.py:1415} INFO - Marking task as
FAILED. dag_id=test_start_jenkins_job, task_id=jenkins_job,
execution_date=20220728T123730, start_date=20220728T123733,
end_date=20220728T123734
[2022-07-28, 16:37:34 CEST] {standard_task_runner.py:92} ERROR - Failed to
execute job 38 for task jenkins_job (Unable to authenticate with any scheme:
auth(kerberos) HTTPSConnectionPool(host='jenkins.pharos.pke.fhm.de',
port=443): Max retries exceeded with url: /api/json (Caused by
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: unable to get local issuer certificate
(_ssl.c:997)')))
auth(basic) HTTPSConnectionPool(host='jenkins.pharos.pke.fhm.de', port=443):
Max retries exceeded with url: /api/json (Caused by
SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed: unable to get local issuer certificate
(_ssl.c:997)'))); 107)
[2022-07-28, 16:37:34 CEST] {local_task_job.py:156} INFO - Task exited with
return code 1
```
### What you think should happen instead
The JenkinsJobTriggerOperator should successfully connect to Jenkins over
https
### How to reproduce
Write a DAG containing a task using the JenkinsJobTriggerOperator to connect
to a Jenkins server over https
### Operating System
Alpine Linux 3.16.1
### Versions of Apache Airflow Providers
```
apache-airflow-providers-apache-hdfs==3.0.1
apache-airflow-providers-celery==3.0.0
apache-airflow-providers-cncf-kubernetes==4.2.0
apache-airflow-providers-common-sql==1.0.0
apache-airflow-providers-datadog==3.0.0
apache-airflow-providers-exasol==2.1.3
apache-airflow-providers-ftp==3.1.0
apache-airflow-providers-http==4.0.0
apache-airflow-providers-imap==3.0.0
apache-airflow-providers-jenkins==3.0.0
apache-airflow-providers-microsoft-mssql==3.1.0
apache-airflow-providers-odbc==3.1.0
apache-airflow-providers-oracle==3.1.0
apache-airflow-providers-postgres==5.1.0
apache-airflow-providers-redis==3.0.0
apache-airflow-providers-slack==5.1.0
apache-airflow-providers-sqlite==3.1.0
apache-airflow-providers-ssh==3.1.0
```
### Deployment
Other 3rd-party Helm chart
### Deployment details
One Pod on Kubernetes containing the following containers
1 Container for the webserver service
1 Container for the scheduler service
1 Container for the dag-processor service
1 Container for the flower service
1 Container for the redis service
2 or 3 containers for the celery workers services
Due to a previous issue crashing the scheduler with the message UNEXPECTED
COMMIT - THIS WILL BREAK HA LOCKS, we substitute scheduler_job.py with the file
https://raw.githubusercontent.com/tanelk/airflow/a4b22932e5ac9c2b6f37c8c58345eee0f63cae09/airflow/jobs/scheduler_job.py.
### Anything else
The jenkins operator could previously successfully connect to Jenkins over
http.
We build our image on an alpine base image provided by our kubernetes admins
that already contains the company ssl-certificates needed to interact with the
several company's web services
It looks like the new setup (Alpine 3.16.1 + python 3.10.x + AirFlow 2.3.3)
isn't able to use the ssl certificates already provided in the os in the file
`/etc/ssl/certs/ca-certificates.crt`.
However python 3.10.x is the suspect not AirFlow itself.
Is there a way to tell the operator to use the certificates in a specific
file?
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
