edglobo-hmedeiros opened a new issue, #27850:
URL: https://github.com/apache/airflow/issues/27850
### Apache Airflow Provider(s)
google
### Versions of Apache Airflow Providers
apache-airflow-providers-google==6.8.0
### Apache Airflow version
2.3.0
### Operating System
Debian GNU/Linux 11 (bullseye)
### Deployment
Composer
### Deployment details
docker-compose
### What happened
I am getting an SSL error everytime i try to execute an upload to Google
Cloud Storage using GCS Hook.
I am executing the bellow code:
```python
from airflow.providers.google.cloud.operators.bigquery import BigQueryHook
from airflow.providers.google.cloud.operators.gcs import GCSHook
### removed for brevity
def upload_df_to_gcs():
bigquery = BigQueryHook(gcp_conn_id='BQ_CONN_ID', use_legacy_sql=False)
storage = GCSHook(gcp_conn_id='GCS_CONN_ID')
df = bigquery.get_pandas_df(sql='REMOVED FOR BREVITY')
csv = df.to_csv(sep=';', encoding='UTF-8', index=False)
print('Saving at Cloud Storage...')
storage.upload(
bucket_name='BUCKET_NAME',
object_name='mydataframe.csv',
data=csv,
mime_type='text/csv'
)
```
But everytime it throws an instance of: `ssl.SSLCertVerificationError`
```
[2022-11-23, 02:21:39 UTC] {logging_mixin.py:115} INFO - Saving at Cloud
Storage...
[2022-11-23, 02:21:40 UTC] {taskinstance.py:1889} ERROR - Task failed with
exception
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/connectionpool.py",
line 710, in urlopen
chunked=chunked,
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/connectionpool.py",
line 386, in _make_request
self._validate_conn(conn)
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/connectionpool.py",
line 1040, in _validate_conn
conn.connect()
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/connection.py", line
424, in connect
tls_in_tls=tls_in_tls,
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line
450, in ssl_wrap_socket
sock, context, tls_in_tls, server_hostname=server_hostname
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/util/ssl_.py", line
493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/local/lib/python3.7/ssl.py", line 423, in wrap_socket
session=session
File "/usr/local/lib/python3.7/ssl.py", line 870, in _create
self.do_handshake()
File "/usr/local/lib/python3.7/ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate
verify failed: self signed certificate in certificate chain (_ssl.c:1091)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/requests/adapters.py", line
450, in send
timeout=timeout
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/connectionpool.py",
line 786, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File
"/home/airflow/.local/lib/python3.7/site-packages/urllib3/util/retry.py", line
592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError:
HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries
exceeded with url: /upload/storage/v1/b/bi-py-***/o?uploadType=multipart
(Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate
in certificate chain (_ssl.c:1091)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/python.py",
line 171, in execute
return_value = self.execute_callable()
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/operators/python.py",
line 189, in execute_callable
return self.python_callable(*self.op_args, **self.op_kwargs)
File "/opt/airflow/dags/gcom_globoid.py", line 64, in selecionar_globoids
mime_type='text/csv'
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/google/cloud/hooks/gcs.py",
line 537, in upload
_call_with_retry(partial(blob.upload_from_string, data,
content_type=mime_type, timeout=timeout))
File
"/home/airflow/.local/lib/python3.7/site-packages/airflow/providers/google/cloud/hooks/gcs.py",
line 477, in _call_with_retry
f()
File
"/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py",
line 2867, in upload_from_string
retry=retry,
File
"/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py",
line 2594, in upload_from_file
retry=retry,
File
"/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py",
line 2396, in _do_upload
retry=retry,
File
"/home/airflow/.local/lib/python3.7/site-packages/google/cloud/storage/blob.py",
line 1917, in _do_multipart_upload
transport, data, object_metadata, content_type, timeout=timeout
File
"/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/requests/upload.py",
line 154, in transmit
retriable_request, self._get_status_code, self._retry_strategy
File
"/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/requests/_request_helpers.py",
line 170, in wait_and_retry
raise error
File
"/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/requests/_request_helpers.py",
line 147, in wait_and_retry
response = func()
File
"/home/airflow/.local/lib/python3.7/site-packages/google/resumable_media/requests/upload.py",
line 146, in retriable_request
method, url, data=payload, headers=headers, timeout=timeout
File
"/home/airflow/.local/lib/python3.7/site-packages/google/auth/transport/requests.py",
line 496, in request
**kwargs
File
"/home/airflow/.local/lib/python3.7/site-packages/requests/sessions.py", line
529, in request
resp = self.send(prep, **send_kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/requests/sessions.py", line
645, in send
r = adapter.send(request, **kwargs)
File
"/home/airflow/.local/lib/python3.7/site-packages/requests/adapters.py", line
517, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError:
HTTPSConnectionPool(host='storage.googleapis.com', port=443): Max retries
exceeded with url: /upload/storage/v1/b/bi-py-***/o?uploadType=multipart
(Caused by SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate
in certificate chain (_ssl.c:1091)')))
```
### What you think should happen instead
_No response_
### How to reproduce
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
