James Meickle created AIRFLOW-3228:
--------------------------------------
Summary: Airflow leaks Kubernetes credentials on exceptions
Key: AIRFLOW-3228
URL: https://issues.apache.org/jira/browse/AIRFLOW-3228
Project: Apache Airflow
Issue Type: Bug
Components: kubernetes
Affects Versions: 1.10.0
Reporter: James Meickle
I have a Kubernetes integration with Airflow using service account tokens,
which are equivalent to passwords in risk/scope. We had an issue where one of
our tokens had an appended newline, rendering it invalid. This led to the
header leaking into the logs:
{{[2018-10-17 20:30:44,355] {{models.py:1736}} ERROR - Invalid header value
b'Bearer MY_KUBERNETES_TOKEN_HERE'
Traceback (most recent call last):
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/airflow/models.py",
line 1633, in _run_raw_task
result = task_copy.execute(context=context)
File
"/home/airflow/src/plugins/moneytree/moneytree/operators/qbernetes_operators.py",
line 331, in execute
get_logs=self.get_logs)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/airflow/contrib/kubernetes/pod_launcher.py",
line 71, in run_pod
resp = self.run_pod_async(pod)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/airflow/contrib/kubernetes/pod_launcher.py",
line 55, in run_pod_async
resp = self._client.create_namespaced_pod(body=req, namespace=pod.namespace)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py",
line 6057, in create_namespaced_pod
(data) = self.create_namespaced_pod_with_http_info(namespace, body,
**kwargs)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py",
line 6142, in create_namespaced_pod_with_http_info
collection_formats=collection_formats)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/kubernetes/client/api_client.py",
line 321, in call_api
_return_http_data_only, collection_formats, _preload_content,
_request_timeout)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/kubernetes/client/api_client.py",
line 155, in __call_api
_request_timeout=_request_timeout)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/kubernetes/client/api_client.py",
line 364, in request
body=body)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/kubernetes/client/rest.py",
line 266, in POST
body=body)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/kubernetes/client/rest.py",
line 166, in request
headers=headers)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/urllib3/request.py",
line 72, in request
**urlopen_kw)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/urllib3/request.py",
line 150, in request_encode_body
return self.urlopen(method, url, **extra_kw)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/urllib3/poolmanager.py",
line 322, in urlopen
response = conn.urlopen(method, u.request_uri, **kw)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/urllib3/connectionpool.py",
line 600, in urlopen
chunked=chunked)
File
"/home/airflow/virtualenvs/airflow/lib/python3.5/site-packages/urllib3/connectionpool.py",
line 354, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.5/http/client.py", line 1106, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.5/http/client.py", line 1146, in _send_request
self.putheader(hdr, value)
File "/usr/lib/python3.5/http/client.py", line 1083, in putheader
raise ValueError('Invalid header value %r' % (values[i],))
ValueError: Invalid header value b'Bearer MY_KUBERNETES_TOKEN_HERE'}}
We should catch these errors and re-raise them without the secret value, since
this isn't suitable for a production application.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
