cris-b opened a new issue #13023:
URL: https://github.com/apache/airflow/issues/13023
**Apache Airflow version**: 1.10.14
**Kubernetes version (if you are using kubernetes)** (use `kubectl
version`): 1.19.3
**Environment**:
- **Cloud provider or hardware configuration**: Azure
- **OS** (e.g. from /etc/os-release): RHEL8
**What happened**:
In my executor_config I had an env_from. I also wanted to pass some env as
secrets but cant do that directly in executor_config so had to use the
AIRFLOW__KUBERNETES__ENV_FROM_SECRET_REF variable.
This led to a crash when trying to create the pod:
Traceback (most recent call last):
File
"/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line
1382, in _execute
self._execute_helper()
File
"/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line
1453, in _execute_helper
if not
self._validate_and_run_task_instances(simple_dag_bag=simple_dag_bag):
File
"/usr/local/lib/python3.7/site-packages/airflow/jobs/scheduler_job.py", line
1515, in _validate_and_run_task_instances
self.executor.heartbeat()
File
"/usr/local/lib/python3.7/site-packages/airflow/executors/base_executor.py",
line 134, in heartbeat
self.sync()
File
"/usr/local/lib/python3.7/site-packages/airflow/executors/kubernetes_executor.py",
line 832, in sync
self.kube_scheduler.run_next(task)
File
"/usr/local/lib/python3.7/site-packages/airflow/executors/kubernetes_executor.py",
line 447, in run_next
base_worker_pod=self.worker_configuration_pod
File
"/usr/local/lib/python3.7/site-packages/airflow/kubernetes/pod_generator.py",
line 564, in construct_pod
return reduce(PodGenerator.reconcile_pods, pod_list)
File
"/usr/local/lib/python3.7/site-packages/airflow/kubernetes/pod_generator.py",
line 429, in reconcile_pods
client_pod_cp.spec = PodGenerator.reconcile_specs(base_pod.spec,
client_pod_cp.spec)
File
"/usr/local/lib/python3.7/site-packages/airflow/kubernetes/pod_generator.py",
line 476, in reconcile_specs
base_spec.containers, client_spec.containers
File
"/usr/local/lib/python3.7/site-packages/airflow/kubernetes/pod_generator.py",
line 509, in reconcile_containers
client_container = extend_object_field(base_container, client_container,
'env_from')
File
"/usr/local/lib/python3.7/site-packages/airflow/kubernetes/pod_generator.py",
line 647, in extend_object_field
base_obj_set = _get_dict_from_list(base_obj_field, field_to_merge)
File
"/usr/local/lib/python3.7/site-packages/airflow/kubernetes/pod_generator.py",
line 676, in _get_dict_from_list
result[getattr(obj, field_to_merge)] = obj
AttributeError: 'V1EnvFromSource' object has no attribute 'name'
**What you expected to happen**:
Pod created successfully
**How to reproduce it**:
Define a pod in executor_config (or pod_template) and additionally use
AIRFLOW__KUBERNETES__ENV_FROM_SECRET_REF or
AIRFLOW__KUBERNETES__ENV_FROM_CONFIGMAP_REF
As long as both the base config and the template contain envFrom/env_from.
**Anything else we need to know**:
This is caused by PodGenerator.extend_object_field trying to merge on 'name'
but V1EnvFromSource don't have those, they either specify config_map_ref or
secret_ref.
<!--
How often does this problem occur? Once? Every time etc?
Every time.
-->
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
