arjunanan6 commented on PR #28394:
URL: https://github.com/apache/airflow/pull/28394#issuecomment-1386603496
@snjypl That solved the previous issue, and the execution is attempted now.
However, manual runs still fail because the service account gets rejected from
creating a pod for the execution:
```
[2023-01-18T07:20:26.236+0000] {kubernetes_executor.py:527} INFO - Start
Kubernetes executor
[2023-01-18T07:20:26.261+0000] {kubernetes_executor.py:130} INFO - Event:
and now my watch begins starting at resource_version: 0
[2023-01-18T07:20:26.395+0000] {kubernetes_executor.py:476} INFO - Found 0
queued task instances
[2023-01-18T07:20:26.438+0000] {base_executor.py:95} INFO - Adding to queue:
['airflow', 'tasks', 'run', 'x-y-z', 'X-Y',
'scheduled__2023-01-18T07:00:00+00:00', '--ignore
-all-dependencies', '--ignore-dependencies', '--local', '--pool',
'default_pool', '--subdir', 'DAGS_FOLDER/airflow-dags-sap/X-Y/x-y-z.py']
[2023-01-18T07:20:26.438+0000] {base_executor.py:215} INFO - task
TaskInstanceKey(dag_id='x-y-z', task_id='x-y-z',
run_id='scheduled__2023-01-18T07:00:00+00:00', try_number
=3, map_index=-1) is still running
[2023-01-18T07:20:26.508+0000] {kubernetes_executor.py:339} INFO - Creating
kubernetes pod for job is TaskInstanceKey(dag_id='x-y-z', task_id='X-Y',
run_id='scheduled__20
23-01-18T07:00:00+00:00', try_number=3, map_index=-1), with pod name
x-y-z-78e3092210f94420bb0e98a744969f29
[2023-01-18T07:20:26.538+0000] {kubernetes_executor.py:274} ERROR -
Exception when attempting to create Namespaced Pod: {
.
.
.
kubernetes.client.exceptions.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Audit-Id':
'ddc99dcf-9d70-4f88-8c7c-77f543879844', 'Cache-Control': 'no-cache, private',
'Content-Type': 'application/json', 'X-Content-Type-
Options': 'nosniff', 'X-Kubernetes-Pf-Flowschema-Uid':
'e7834783-2050-421a-b99e-0615f85f6e92', 'X-Kubernetes-Pf-Prioritylevel-Uid':
'e9e2e589-5d4c-442b-8568-f7bfbdbfaafd', 'Date': '
Wed, 18 Jan 2023 07:20:26 GMT', 'Content-Length': '315'})
HTTP response body:
{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods
is forbidden: User \"system:serviceaccount:airflow-test-ns:airflow-test-webse
rver\" cannot create resource \"pods\" in API group \"\" in the namespace
\"airflow-test-ns\"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
```
Which is strange, because this SA is allowed to create pods, and other tasks
are being executed by the same service account. I trimmed out the pod
definition, but nothing looks particularly out of place there. Any idea why
this fails only on a manual run attempt?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
