adamides opened a new issue, #56311: URL: https://github.com/apache/airflow/issues/56311
### Description Currently, Airflow’s Google Cloud provider does not allow users to specify a quota project/billing project when using Google services (such as Dataflow, BigQuery, GCS, etc) via hooks/operators. Google Cloud APIs require the x-goog-user-project header to be set when the quota project is different from the project_id. There is no way to set this in Airflow via connection or operator arguments. _Setting the project_id does not work when the service account is from a different project_ ### Use case/motivation Many organizations use shared service accounts that cannot be billed directly and require a separate billing project. Some workflows require explicit quota/billing project overrides depending on how APIs are enabled and/or where service accounts are maintained. Current workaround requires subclassing operators/hooks, copying code, etc. Just like how the project_id can be overridden when creating a google cloud connection, and in the Operator itself. There should also be an option to specify the quota project. In addition to the project_id, one could specify quota_project_id in the Operator. Ultimately this results in an HTTP call against the GCP REST API which supports this as the "x-goog-user-project" header. The google auth library for python also supports with_quota_project which returns a copy of the credentials containing the modified quota project. ### Related issues _No response_ ### Are you willing to submit a PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
