moomindani opened a new pull request, #66613:
URL: https://github.com/apache/airflow/pull/66613
The Databricks operators currently require job-level parameters to be
hardcoded inside `json`. This forwards the operator's `self.params` (Airflow
Dag / task / dag_run.conf params) into the corresponding Databricks parameter
slot when the user has not explicitly populated it:
* `DatabricksCreateJobsOperator` -> top-level `parameters` list (`[{"name":
k, "default": v}, ...]`).
* `DatabricksRunNowOperator` -> top-level `job_parameters` dict.
* `DatabricksSubmitRunOperator` -> dict-shaped per-task fields:
`notebook_task.base_parameters`, `python_wheel_task.named_parameters`,
`sql_task.parameters`, `run_job_task.job_parameters`. Tasks whose only
parameter slot is `List[str]` (`spark_jar_task`, `spark_python_task`,
`spark_submit_task`) are skipped because there is no canonical mapping from a
key/value dict to positional CLI arguments.
The injection only fires when the corresponding slot is empty, so users who
explicitly pass parameters in `json` keep their existing behaviour.
Builds on @SubhamSinghal's earlier work in #39007 (closed as stale). Picks
up @dirrao's and @Lee-W's review feedback (list-comprehension refactor) and
@galafis's request to extend the feature to RunNow / SubmitRun.
closes: #39002
---
##### Was generative AI tooling used to co-author this PR?
- [X] Yes — Claude Code (Opus 4.7)
Generated-by: Claude Code (Opus 4.7) following [the
guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)
---
* Read the **[Pull Request
Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pull-request-guidelines)**
for more information. Note: commit author/co-author name and email in commits
become permanently public when merged.
* For fundamental code changes, an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvement+Proposals))
is needed.
* When adding dependency, check compliance with the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
* For significant user-facing changes create newsfragment:
`{pr_number}.significant.rst`, in
[airflow-core/newsfragments](https://github.com/apache/airflow/tree/main/airflow-core/newsfragments).
You can add this file in a follow-up commit after the PR is created so you
know the PR number.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
