ying-w opened a new pull request, #38169: URL: https://github.com/apache/airflow/pull/38169
closes: https://github.com/apache/airflow/issues/33400 Currently `scopes` for google libraries are taken from connection parameters, this PR creates an additional parameter `impersonation_scopes` that will be used in situations when `impersonation_chain` is being used. The flow to get credentials for gcp providers goes something like this 1. A library like bigquery will call get_credentials() https://github.com/apache/airflow/blob/00cd44e4ccb4a7d6406e75d3cf8f80277545d5d2/airflow/providers/google/cloud/hooks/bigquery.py#L163 2. This is provided by GoogleBaseHook https://github.com/apache/airflow/blob/00cd44e4ccb4a7d6406e75d3cf8f80277545d5d2/airflow/providers/google/common/hooks/base_google.py#L307 3. This then makes a call to `self.scopes()` https://github.com/apache/airflow/blob/00cd44e4ccb4a7d6406e75d3cf8f80277545d5d2/airflow/providers/google/common/hooks/base_google.py#L290 4. scopes will then return scopes from connection extras or default (in `_get_scopes()`) https://github.com/apache/airflow/blob/00cd44e4ccb4a7d6406e75d3cf8f80277545d5d2/airflow/providers/google/common/hooks/base_google.py#L414 Possible alternative solutions 1. Override `scopes()` definition in custom hook - as detailed in above issue 2. Introduce a parameter called `scopes` - it would be confusing what takes precedence if there is scopes in parameter and connection I would need some help testing and bringing this over the finish line Should be able to do something like ```py bq_hook = BigQueryHook( gcp_conn_id="my_con", impersonation_chain="[email protected]", use_legacy_sql=False, location="us", api_resource_configs={"query": {"useQueryCache": False, "priority": "BATCH"}}, scopes = ( "https://www.googleapis.com/auth/cloud-platform";, "https://www.googleapis.com/auth/drive";, ), ) bq_hook.scopes # check that it got set ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
