[
https://issues.apache.org/jira/browse/AIRFLOW-5768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Standish updated AIRFLOW-5768:
-------------------------------------
Description:
GCP cloud sql operator creates dynamically an ephemeral Connection object. It
persists to metastore during execution and deletes afterward.
This behavior has negative impact on our ability to refactor creds management.
By not persisting to database, we can also remove some complexity re ensuring
connection is deleted in event of failure, and the tests that go along with
that.
This change requires that we add optional param `connection` to both MySqlHook
and PostgresHook.
was:
GCP cloud sql operator creates dynamically an ephemeral Connection object. It
persists to metastore during execution and deletes afterward.
This behavior has negative impact on our ability to refactor creds management.
By not persisting to database, we can also remove some complexity re ensuring
connection is deleted in event of failure, and the tests that go along with
that.
It does require that we add optional param `connection` to both MySqlHook and
PostgresHook.
> Google Cloud SQL - Don't store ephemeral connection object to database
> ----------------------------------------------------------------------
>
> Key: AIRFLOW-5768
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5768
> Project: Apache Airflow
> Issue Type: Improvement
> Components: gcp
> Affects Versions: 1.10.5
> Reporter: Daniel Standish
> Assignee: Daniel Standish
> Priority: Major
>
> GCP cloud sql operator creates dynamically an ephemeral Connection object.
> It persists to metastore during execution and deletes afterward.
> This behavior has negative impact on our ability to refactor creds
> management.
> By not persisting to database, we can also remove some complexity re ensuring
> connection is deleted in event of failure, and the tests that go along with
> that.
> This change requires that we add optional param `connection` to both
> MySqlHook and PostgresHook.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
