MihaixD opened a new issue, #27837:
URL: https://github.com/apache/airflow/issues/27837
### Apache Airflow Provider(s)
databricks
### Versions of Apache Airflow Providers
apache-airflow-providers-databricks==3.3.0
### Apache Airflow version
2.4.2
### Operating System
Mac OS
### Deployment
Virtualenv installation
### Deployment details
Virtualenv deployment with Python 3.10
### What happened
Submitting a Databricks job run with the deferrable version
(`DatabricksRunNowDeferrableOperator`) does not actually fill the `job_id` and
the Databricks API responds with an HTTP 400 bad request - attempting to run a
job (POST `https://<databricks-instance>/api/2.1/jobs/run-now`) without an ID
specidied.
### What you think should happen instead
The deferrable version (`DatabricksRunNowDeferrableOperator`) should
maintain the behavior of the parent class (`DatabricksRunNowOperator`) and use
the `job_name` to find the `job_id`.
The following logic is missing in the deferrable version:
```
# Sample from the DatabricksRunNowOperator#execute
hook = self._hook
if "job_name" in self.json:
job_id = hook.find_job_id_by_name(self.json["job_name"])
if job_id is None:
raise AirflowException(f"Job ID for job name
{self.json['job_name']} can not be found")
self.json["job_id"] = job_id
del self.json["job_name"]
```
### How to reproduce
To reproduce, use a deferrable run now operator with the job name as an
argument in an airflow task:
```
from airflow.providers.databricks.operators.databricks import
DatabricksRunNowDeferrableOperator
DatabricksRunNowDeferrableOperator(
job_name='some-name',
# Other args
)
```
### Anything else
The problem occurs at every call.
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
