shahar1 commented on issue #31126: URL: https://github.com/apache/beam/issues/31126#issuecomment-2084510959
> @shahar1 this level of customization might make sense. > > Let's explore your specific concern for a moment, I might have others, but imagine worth understanding your needs/usecase: > > What Quotas are getting hit that are problematic? Or, what are the specific billing charges you are looking to attribute elsewhere? Your are running on Dataflow? Or other? [ not critical, but curious ] What read method? [ BQ Storage Read API? ] You want to run the compute in one GCP project, but use BQ from another? If this unloads, writes to GCS and then into Dataflow [ that is another way that can occur ], do you intend to specify which project [ bucket within ] that the data is written? Thank for your response! Here are the answers for your questions: 1. I'd like to attribute the queries execution to another project. In my case, the BigQuery is on project A, and `beam.io.ReadFromBigQuery` runs on project B - I'd like to bill project B for the queries (for that matter it could also be project C). 2. We use Dataflow and direct runner (when implementing, it should better a be a general solution and not Dataflow specific). 3. I use both methods - if I'm not wrong, in both cases you could set the `quota_project_id` via `ClientOptions`. 5. Yup, you got the idea correctly :) As for #30747 - it is related, but there might be some changes in implementation as GCS is project's resource rather than a service. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
