[
https://issues.apache.org/jira/browse/AIRFLOW-2016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16470746#comment-16470746
]
Kevin McHale commented on AIRFLOW-2016:
---------------------------------------
hi [~DanSedov] I recently tried using these operators (or, at least, the
instantiate-inline variant) and I received errors due to the {{template_id}}
field being missing. I think that the issue comes from the airflow
{{template_fields}} settings that are set on [this
line|https://github.com/apache/incubator-airflow/blob/master/airflow/contrib/operators/dataproc_operator.py#L1034]
in the base class. Airflow's jinja templater tries to populate these fields
on the derived classes, but {{template_id}} does not apply for the
instantiate-inline operator, and likewise {{template}} should not apply for the
instantiate operator, so an error is thrown.
I have put together a quick fix that looks like
[this|https://github.com/apache/incubator-airflow/compare/master...mchalek:mchalek-fix-inline-workflow-template-operator]
and I am running that successfully on our Airflow deployments at Etsy, but I
just wanted to check with you before submitting it to make sure I'm not missing
something? In particular we are using Airflow 1.9.0 and I suppose it's
possible that the jinja templater behaves differently in the latest airflow
versions?
> Add support for Dataproc Workflow Templates
> -------------------------------------------
>
> Key: AIRFLOW-2016
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2016
> Project: Apache Airflow
> Issue Type: Improvement
> Components: contrib
> Reporter: Dan Sedov
> Assignee: Dan Sedov
> Priority: Minor
> Fix For: 1.10.0
>
>
> Add new operators to support instantiation of Google Cloud Dataproc Workflow
> Templates.
> See:
> https://cloud.google.com/dataproc/docs/reference/rest/v1beta2/projects.regions.workflowTemplates
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
