macdonald-keith-vmware opened a new issue #18380:
URL: https://github.com/apache/airflow/issues/18380
### Apache Airflow version
2.1.2
### Operating System
Ubuntu 20.04.3 LTS
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other 3rd-party Helm chart
### Deployment details
Helm, Bitnami charts, MicroK8 running on Ubuntu cluster.
### What happened
When running multiple tasks that use the same pod-template yaml file,
[2021-09-20 15:50:02,605] {pod_launcher.py:95} ERROR - Exception when
attempting to create Namespaced Pod: {
...
File
"/opt/bitnami/airflow/venv/lib/python3.8/site-packages/kubernetes/client/rest.py",
line 231, in request
raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (409)
Reason: Conflict
HTTP response headers: HTTPHeaderDict({'Cache-Control': 'no-cache, private',
'Content-Type': 'application/json', 'Date': 'Mon, 20 Sep 2021 15:50:02 GMT',
'Content-Length': '204'})
HTTP response body:
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods
\"k8s-small-pod\" already
exists","reason":"AlreadyExists","details":{"name":"k8s-small-pod","kind":"pods"},"code":409}
### What you expected to happen
The pod name in the template file should be overridden at runtime with a
unique name.
### How to reproduce
1. Create a pod template small_pod.yaml
`apiVersion: v1
kind: Pod
metadata:
name: k8s-small-pod
spec:
containers:
- name: small-pod
imagePullPolicy: IfNotPresent
args: []
command: []
env:
- name: GIT_SSL_NO_VERIFY
value: "true"
envFrom: []
image: dummy_image
imagePullPolicy: IfNotPresent
name: base
ports: []
restartPolicy: Never`
2. Create two simple dags that use the same template
`passing = KubernetesPodOperator(namespace='airflow',
image="python:3.6",
cmds=["python","-c"],
arguments=["print('hello world')"],
labels={"foo": "bar"},
name="passing-test",
task_id="passing-task",
pod_template_file="./templates/small_pod.yaml",
is_delete_operator_pod=True,
get_logs=True,
dag=dag
)`
3. Run the dags at the same time
### Anything else
Occurs every time both pod operators are executing at the same time.
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
