vatsrahul1001 commented on issue #40788:
URL: https://github.com/apache/airflow/issues/40788#issuecomment-2230039846
@boraberke
> Hi @vatsrahul1001,
>
> `job_id` field of `DatabricksRunNowOperator` is not a templated field
which might be the cause of the issue. Before #40471 constructor added `job_id`
inside the `json` parameter before rendering the templated field which
supported job_id as a template field.
>
> A workaround for this could be setting json parameter as
>
> ```
> json={"job_id": "{{ task_instance.xcom_pull(task_ids='submit_run',
dag_id='example_async_databricks', key='job_id') }}"},
> ```
>
> instead of specifying explicit `job_id` parameter:
>
> ```
> job_id="{{ task_instance.xcom_pull(task_ids='submit_run',
dag_id='example_async_databricks', key='job_id') }}",
> ```
>
> I do not have test environment for Databricks to test if my assumption
works. Let me know if this fixes the problem.
I tried using templating json param as expected, however, existing example
DAG using named param with templating should not break by this new change. As
mentioned [here]
(https://github.com/apache/airflow/blob/main/airflow/providers/databricks/operators/databricks.py#L643-L656)
using only named param instead of json is very common in use.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
