[
https://issues.apache.org/jira/browse/AIRFLOW-4991?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Greg Ferrar updated AIRFLOW-4991:
---------------------------------
Description:
Using KubernetesOperator with EKS, tasks that run more than 15 minutes result
in a `401 Unauthenticated` error after the worker script successfully completes.
{code:java}
[2019-07-10 20:46:32,509] {__init__.py:1580} ERROR - (401)
Reason: Unauthorized
HTTP response headers: HTTPHeaderDict({'Audit-Id':
'0587c262-eec3-4d20-a2ce-925e1bdd51eb', 'Content-Type': 'application/json',
'Date': 'Wed, 10 Jul 2019 20:46:32 GMT', 'Content-Length': '129'})
{code}
This is due to EKS having a 15-minute timeout on its authentication token.
Solution is to re-authenticate with EKS at least every fifteen minutes, or
maybe just at the end of the job.
was:
Using KubernetesOperator with EKS, tasks that run more than 15 minutes result
in a `401 Unauthenticated` error after the worker script successfully completes.
This is due to EKS having a 15-minute timeout on its authentication token.
Solution is to re-authenticate with EKS at least every fifteen minutes, or
maybe just at the end of the job.
> 401 error when EKS task runs longer than 15 minutes
> ---------------------------------------------------
>
> Key: AIRFLOW-4991
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4991
> Project: Apache Airflow
> Issue Type: Bug
> Components: authentication
> Affects Versions: 1.10.3
> Reporter: Greg Ferrar
> Priority: Minor
>
> Using KubernetesOperator with EKS, tasks that run more than 15 minutes result
> in a `401 Unauthenticated` error after the worker script successfully
> completes.
> {code:java}
> [2019-07-10 20:46:32,509] {__init__.py:1580} ERROR - (401)
> Reason: Unauthorized
> HTTP response headers: HTTPHeaderDict({'Audit-Id':
> '0587c262-eec3-4d20-a2ce-925e1bdd51eb', 'Content-Type': 'application/json',
> 'Date': 'Wed, 10 Jul 2019 20:46:32 GMT', 'Content-Length': '129'})
> {code}
> This is due to EKS having a 15-minute timeout on its authentication token.
> Solution is to re-authenticate with EKS at least every fifteen minutes, or
> maybe just at the end of the job.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
