Kamil Bregula created AIRFLOW-4771:
--------------------------------------
Summary: Initialization of hook in the GCP operator constructors
Key: AIRFLOW-4771
URL: https://issues.apache.org/jira/browse/AIRFLOW-4771
Project: Apache Airflow
Issue Type: Bug
Components: gcp
Affects Versions: 1.10.3
Reporter: Kamil Bregula
Hello,
This affects following operators:
* BigtableInstanceCreateOperator
* BigtableInstanceDeleteOperator
* BigtableTableCreateOperator
* BigtableTableDeleteOperator
* BigtableClusterUpdateOperator
* BigtableTableWaitForReplicationSensor
* GceBaseOperator
* GcfFunctionDeployOperator
* GcfFunctionDeleteOperator
* CloudSpannerInstanceDeployOperator
* CloudSpannerInstanceDeleteOperator
* CloudSpannerInstanceDatabaseQueryOperator
* CloudSpannerInstanceDatabaseDeployOperator
* CloudSpannerInstanceDatabaseUpdateOperator
* CloudSpannerInstanceDatabaseDeleteOperator
* CloudSqlBaseOperator
* CloudVisionProductSetCreateOperator
* and more...
This is not good practice for two reasons:
* The gcp_conn_id parameter can not be a Jinja template. Templates are
resolved before calling the execute method.
* Hook initialization calls the database connection, because In the
GoogleCloudBaseHook class, we have a code snippet:
{code:java}
def __init__(self, gcp_conn_id='google_cloud_default', delegate_to=None):
self.gcp_conn_id = gcp_conn_id
self.delegate_to = delegate_to
self.extras = self.get_connection(self.gcp_conn_id).extra_dejson
{code}
self.get_connection triggers a database query.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
