Lee-W commented on code in PR #32221:
URL: https://github.com/apache/airflow/pull/32221#discussion_r1353872508
##########
generated/provider_dependencies.json:
##########
@@ -299,6 +299,7 @@
"aiohttp>=3.6.3, <4",
"apache-airflow-providers-common-sql>=1.5.0",
"apache-airflow>=2.4.0",
+ "databricks-sdk>=0.1.11, <1.0.0",
Review Comment:
Is there a reason we need/want to pin it to `<1.0.0`?
##########
airflow/providers/databricks/operators/databricks.py:
##########
@@ -160,6 +162,142 @@ def get_link(
return XCom.get_value(key=XCOM_RUN_PAGE_URL_KEY, ti_key=ti_key)
+class DatabricksJobsCreateOperator(BaseOperator):
+ """
+ Creates (or resets) a Databricks job using the
+ `api/2.1/jobs/create
+
<https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsCreate>`_
+ (or `api/2.1/jobs/reset
+
<https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsReset>`_)
+ API endpoint.
+
+ .. seealso::
+
https://docs.databricks.com/dev-tools/api/latest/jobs.html#operation/JobsCreate
+
+ :param json: A JSON object containing API parameters which will be passed
+ directly to the ``api/2.1/jobs/create`` endpoint. The other named
parameters
+ (i.e. ``name``, ``tags``, ``tasks``, etc.) to this operator will
+ be merged with this json dictionary if they are provided.
+ If there are conflicts during the merge, the named parameters will
+ take precedence and override the top level json keys. (templated)
+
+ .. seealso::
+ For more information about templating see
:ref:`concepts:jinja-templating`.
+ :param name: An optional name for the job.
+ :param tags: A map of tags associated with the job.
+ :param tasks: A list of task specifications to be executed by this job.
Review Comment:
Yep, but I thought type annotation was enough? not a blocker, though
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
