lostluck commented on PR #33236: URL: https://github.com/apache/beam/pull/33236#issuecomment-2540092061
If it were me, I'd add a function to the GCS package that sets the value when the fs is constructed. eg. `SetRequesterBillingProject(project string)` Then do something like the harness opts do for the side input cache: Defines and installs the hook URN. https://github.com/apache/beam/blob/de5e8ebcee290a00036c868a1f444018bd445506/sdks/go/pkg/beam/core/runtime/harness/cache_hooks.go#L50 Has a method to be called at before pipeline submission to enable the hook, and call `SetRequesterBillingProject` with the value stored in the hook. Note, it just needs to be an arbitrary string (which could then be any serialized data that you like). This gets stored as a pipeline option, so it can be retrieved worker side. https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/util/harnessopts/cache.go This would affect the whole pipeline (just as this change does), but if a pipeline author wants to do trickier bits where the billing requester is changing, then we can figure out other options. Ultimately, the goal is to enable most uses, but not all uses. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
