devdattakulkarni commented on issue #27358:
URL: https://github.com/apache/airflow/issues/27358#issuecomment-1569995381
@jedcunningham Hello. I stumbled upon this issue while debugging an error. I
watched the video you mentioned above and could not find an answer so I thought
of asking here. I hope it's okay.
So I have a K8s executor with a custom image. I trigger the dag from Airflow
UI by passing in custom parameters using the "trigger DAG w/config" option. I
understand these parameters will be accessible to the task via the dag_run
dictionary. But I am not able to access the dag_run dictionary. Below is a
brief snippet of my task definition.
`
my_executor_config = {
"pod_override": k8s.V1Pod(
spec=k8s.V1PodSpec(
containers=[
k8s.V1Container(
name="base",
image="custom-image",
command=["python3","count_exposure_points.py",
{{ dag_run.conf['bucket'] }}, {{ dag_run
.conf['prefix'] }}]
@task(executor_config=my_executor_config)
def my_task():
print_stuff()
my_task
`
I tried dereferencing dag_run as "{{ dag_run }}" (with double quotes) -- but
this just passes the string "{{ dag_run }}" to the command. Without the double
quotes, DAG fails to load with "Nameerror dag_run is an unknown name".
Any pointers/suggestions on how to access dag_run in a KubernetesExecutor
will be very helpful.
Thank you!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
