John Dyer created AIRFLOW-5861:
----------------------------------
Summary: DatabricksRunNowOperator enforcing argument that should
be able to be empty
Key: AIRFLOW-5861
URL: https://issues.apache.org/jira/browse/AIRFLOW-5861
Project: Apache Airflow
Issue Type: Bug
Components: operators
Affects Versions: 1.10.6
Reporter: John Dyer
It looks like /airflow/utils/decorators.py: apply_defaults makes sure that
arguments without a default value are enforced. In the
DatabricksRunNowOperator, the documentation specifies:
{noformat}
There are two ways to instantiate this operator.
In the first way, you can take the JSON payload that you typically use
to call the ``api/2.0/jobs/run-now`` endpoint and pass it directly
to our ``DatabricksRunNowOperator`` through the ``json`` parameter.
For example ::
json = {
"job_id": 42,
"notebook_params": {
"dry-run": "true",
"oldest-time-to-consider": "1457570074236"
}
}
notebook_run = DatabricksRunNowOperator(task_id='notebook_run',
json=json){noformat}
But, when we run it that way, we get:
{noformat}
Broken DAG: [/home/airflow-user/my-awesome-dag.py] Argument ['job_id'] is
required{noformat}
So, my thought to fix this is to provide a default for job_id as None, here:
[https://github.com/apache/airflow/blob/v1-10-stable/airflow/contrib/operators/databricks_operator.py#L433]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
