hussein-awala commented on code in PR #32903:
URL: https://github.com/apache/airflow/pull/32903#discussion_r1314011396
##########
airflow/providers/databricks/hooks/databricks.py:
##########
@@ -215,6 +216,67 @@ def find_job_id_by_name(self, job_name: str) -> int | None:
else:
return matching_jobs[0]["job_id"]
+ def list_pipelines(
+ self, max_results: int = 25, pipeline_name: str | None = None,
notebook_path: str | None = None
+ ) -> list[dict[str, Any]]:
+ """
+ Lists the pipelines in Databricks Delta Live Tables.
+
+ :param max_results: The limit/batch size used to retrieve pipelines.
Review Comment:
The name is confusing, I wonder if we can change it.
##########
airflow/providers/databricks/operators/databricks.py:
##########
@@ -384,6 +386,11 @@ def _get_hook(self, caller: str) -> DatabricksHook:
)
def execute(self, context: Context):
+ if "pipeline_task" in self.json and
self.json["pipeline_task"].get("pipeline_id") is None:
Review Comment:
You need to add an extra check on `pipeline_name`:
```suggestion
if (
"pipeline_task" in self.json
and self.json["pipeline_task"].get("pipeline_id") is None
and self.json["pipeline_task"].get("pipeline_name")
):
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
