[
https://issues.apache.org/jira/browse/AIRFLOW-1504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16127570#comment-16127570
]
ASF subversion and git services commented on AIRFLOW-1504:
----------------------------------------------------------
Commit 1cd6c4b0e8d73426be49106eaab75ace1bf4a4bf in incubator-airflow's branch
refs/heads/master from [~TrevorEdwards]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=1cd6c4b ]
[AIRFLOW-1504] Log dataproc cluster name
Closes #2517 from
TrevorEdwards/dataproc_log_clustername
> Log Cluster Name on Dataproc Operator When Executed
> ---------------------------------------------------
>
> Key: AIRFLOW-1504
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1504
> Project: Apache Airflow
> Issue Type: Improvement
> Reporter: Trevor Edwards
> Assignee: Trevor Edwards
> Priority: Minor
> Attachments: dataproc_cluster_logging.py
>
>
> Since Jinja template substitution happens only just before execution of a
> task instance, it is not possible to inspect the cluster name populated for a
> Dataproc operator. This makes it hard to debug issues such as using ts_nodash
> as part of the clustername, as printing when the DAG is read would give the
> output:
> Cluster name: smoke-cluster-{{ ts_nodash }}
> then the operator would execute and the user receives the error:
> <HttpError 400 when requesting ... returned "clusterName must be a match of
> regex '(?:[a-z](?:[-a-z0-9]{0,49}[a-z0-9])?).'">
> but the user has no idea what cluster name was provided to the API.
> See attached example pipeline.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
