[ 
https://issues.apache.org/jira/browse/AIRFLOW-2964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jarek Potiuk closed AIRFLOW-2964.
---------------------------------
    Resolution: Won't Fix

I am closing some old issues that are not relevant any more. Please let me know 
if you want to reopen it.

> Lazy generation of the job description
> --------------------------------------
>
>                 Key: AIRFLOW-2964
>                 URL: https://issues.apache.org/jira/browse/AIRFLOW-2964
>             Project: Apache Airflow
>          Issue Type: Improvement
>          Components: contrib, operators
>    Affects Versions: 1.9.0, 2.0.0
>            Reporter: Victor Jimenez
>            Priority: Major
>
> When instantiating a {{DatabricksSubmitRunOperator}} users need to pass the 
> description of the job that will later be executed on Databricks.
> The job description is only needed at execution time (when the hook is 
> called). However, the {{json}} parameter must already have the full job 
> description when constructing the operator. This may present a problem if 
> computing the job description needs to execute expensive operations (e.g., 
> querying a database). The expensive operation will be invoked every single 
> time the DAG is reprocessed (which may happen quite frequently).
> It would be good to have an equivalent mechanism to the {{python_callable}} 
> parameter in the {{PythonOperator}}. In this way, users could pass a function 
> that would generate the job description only when the operator is actually 
> executed.
> [~andrewmchen] Is there any other way to do this? If not, does this sound 
> reasonable? I can create a PR implementing this proposal.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Reply via email to