galafis commented on issue #39002: URL: https://github.com/apache/airflow/issues/39002#issuecomment-4068305231
I'd like to take this on. Noticed the previous PR #39007 was closed, and the issue was recently re-labeled as good first issue. From what I understand, the goal is to allow Airflow DAG params (e.g., from `dag_run.conf`) to be passed dynamically as job parameters to Databricks spark jobs, instead of having them hardcoded in the JSON config. I'll look into the `DatabricksSubmitRunOperator` and `DatabricksRunNowOperator` to understand how `json` / `spark_submit_params` are currently handled, and figure out the best way to inject Airflow params via `template_fields` or an explicit mapping parameter. Will also add tests covering the new behavior. I'll open a PR soon. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
