Ankurdeewan opened a new pull request, #56324:
URL: https://github.com/apache/airflow/pull/56324
Add quota/billing project support for Google Cloud providers
Currently, Airflow's Google Cloud providers lack support for specifying a
quota/billing project when using Google services. This PR adds the ability to
configure a separate project for API quota and billing purposes, which is
particularly useful for organizations using shared service accounts.
Changes:
- Added quota_project_id parameter to GoogleBaseHook
- Added support for quota project configuration via connection extras
- Implemented proper validation of quota project IDs
- Added comprehensive error handling with helpful messages
- Updated UI with tooltips and descriptions
- Added documentation and examples
The quota project can be specified in two ways:
1. Via connection configuration (in extras):
"quota_project_id": "billing-project-123"
2. Via operator parameter:
task = BigQueryExecuteQueryOperator(
quota_project_id="billing-project-123",
...
)
The implementation uses Google Auth library's with_quota_project() method to
properly set the x-goog-user-project header required by Google Cloud APIs.
Testing:
- Added unit tests for parameter and connection-based configuration
- Added validation tests
- Tested with actual Google Cloud services
- Verified backward compatibility
Documentation has been updated to reflect the new functionality.
Fixes #[56331]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
