HariGS-DB opened a new issue, #45138:
URL: https://github.com/apache/airflow/issues/45138
### Description
The current Databricks operator to run jobs (DatabricksNotebookOperator,
DatabricksTaskOperator and DatabricksWorkflowTaskGroupOperator) doesn't support
serverless. All the above operators expect either an existing_cluster_id or
job_cluster_key, whereas neither should be passed for serverless workflow. The
work required is :
- add a named parameter to support environments which are used for
serverless
- add a named_parameter called is_serverless as boolean. This flag will
make existing_cluster_id and job_cluster_key optional and submit a serverless
job
- Update doc to reflect the new addition
### Use case/motivation
This change will allow users to submit a Databricks serverless job for
DatabricksNotebookOperator, DatabricksTaskOperator and
DatabricksWorkflowTaskGroupOperator which is currently supported for only
DatabricksSubmitRunOperator
### Related issues
_No response_
### Are you willing to submit a PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
