lgov opened a new issue #11169:
URL: https://github.com/apache/airflow/issues/11169
**Apache Airflow version**:
v1-10-test branch
**Kubernetes version (if you are using kubernetes)**
v1.19.2
**Environment**:
- **Cloud provider or hardware configuration**: Standalone PC
- **OS** (e.g. from /etc/os-release): Ubuntu 18.04.5 LTS
- **Kernel** (e.g. `uname -a`): GNU/Linux 4.15.0-91-generic x86_64
- **Install tools**: Docker version 19.03.12
- **Others**:
The airflow worker is running in a docker container with an image derived
from `apache/airflow:latest` but the stable apache-airflow package replaced by
current v1-10-test branch.
**What happened**:
We use KubernetesPodOperator in a dag, but when triggering it it always
immediately result in an exception.
Folder structure:
dags/basic_pod_task.py
dags/k8s/basic_pod.yaml
Relevant extract of dags/basic_pod_task.py:
```
with dag:
t2 = KubernetesPodOperator(
is_delete_operator_pod=True,
in_cluster=False,
cluster_context="kubernetes-admin@kubernetes",
task_id="task-two",
get_logs=True,
pod_template_file="k8s/basic_pod.yaml"
)
```
The yaml file is taken from the airflow repo:
tests/kubernetes/basic_pod.yaml:
```
apiVersion: v1
kind: Pod
metadata:
namespace: default
name: test-pod
spec:
containers:
- name: base
image: perl
command: ["/bin/bash"]
args: ["-c", 'echo {\"hello\" : \"world\"} | cat >
/airflow/xcom/return.json']
restartPolicy: Never
```
Error message:
```
[2020-09-26 15:25:18,659] {taskinstance.py:901} INFO - Executing
<Task(KubernetesPodOperator): task-two> on 2020-09-26T15:24:53.914308+00:00
[2020-09-26 15:25:18,661] {standard_task_runner.py:54} INFO - Started
process 423 to run task
[2020-09-26 15:25:18,676] {standard_task_runner.py:77} INFO - Running:
['airflow', 'run', 'basic_test_pod', 'task-two',
'2020-09-26T15:24:53.914308+00:00', '--job_id', '93', '--pool', 'default_pool',
'--raw', '-sd', 'DAGS_FOLDER/basic_pod_task.py', '--cfg_path',
'/tmp/tmpa70_g_m7']
[2020-09-26 15:25:18,677] {standard_task_runner.py:78} INFO - Job 93:
Subtask task-two
[2020-09-26 15:25:18,677] {cli_action_loggers.py:68} DEBUG - Calling
callbacks: [<function default_action_log at 0x7f651c1be6a8>]
[2020-09-26 15:25:18,683] {settings.py:233} DEBUG - Setting up DB connection
pool (PID 423)
[2020-09-26 15:25:18,683] {settings.py:241} DEBUG -
settings.configure_orm(): Using NullPool
[2020-09-26 15:25:18,698] {logging_mixin.py:112} INFO - Running
<TaskInstance: basic_test_pod.task-two 2020-09-26T15:24:53.914308+00:00
[running]> on host 9b5a0a6250ec
[2020-09-26 15:25:18,711] {__init__.py:101} DEBUG - Preparing lineage inlets
and outlets
[2020-09-26 15:25:18,711] {__init__.py:137} DEBUG - inlets: [], outlets: []
[2020-09-26 15:25:18,720] {taskinstance.py:1150} ERROR - 'NoneType' object
has no attribute 'name'
Traceback (most recent call last):
File "/opt/airflow/airflow/airflow/models/taskinstance.py", line 984, in
_run_raw_task
result = task_copy.execute(context=context)
File
"/opt/airflow/airflow/airflow/contrib/operators/kubernetes_pod_operator.py",
line 280, in execute
self.pod = self.create_pod_request_obj()
File
"/opt/airflow/airflow/airflow/contrib/operators/kubernetes_pod_operator.py",
line 384, in create_pod_request_obj
pod=self.full_pod_spec,
File "/opt/airflow/airflow/airflow/kubernetes/pod_generator.py", line 289,
in gen_pod
result.metadata.name = self.make_unique_pod_id(result.metadata.name)
AttributeError: 'NoneType' object has no attribute 'name'
[2020-09-26 15:25:18,721] {taskinstance.py:1194} INFO - Marking task as
FAILED. dag_id=basic_test_pod, task_id=task-two,
execution_date=20200926T152453, start_date=20200926T152518,
end_date=20200926T152518
```
**What you expected to happen**:
I expect that k8s to start up the pod, but we don't get even that far.
<!-- What do you think went wrong? -->
I'm guessing that the yaml file is not found, because when I remove the yaml
file I get exactly the same error.
The pod starts when I use `kubectl apply -f k8s//basic_pod.yaml `
Specifying the path as absolute path doesn't solve the problem.
It's not clear what parent path is used for the `pod_template_file`
parameter, but if this is indeed the root cause the exception logging could be
more clear.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
