Thanks, Chuck for looking into this. We explored switching to google-cloud-* python libraries for Dataflow runner purposes, and encountered several issues related to dependency management of these libraries in Google-internal repository, which were difficult to address just in Beam plane without vendroing these dependencies. We are currently working to address these issues internally before coming back to this effort. I would not recommend switching these libraries on master branch without commitment/help from Google Dataflow runner maintainers at this time, although we can certainly explore what external (Beam) changes will be needed.
On Thu, Jun 10, 2021 at 3:55 PM Luke Cwik <lc...@google.com> wrote: > I did something very similar during the Dataflow Java 1.x to Beam Java 2.x > migration. The work boiled down to: > * swapping to a different library to get the application default > credentials (including fixing upstream bugs at Google and improving some > documentation) > * swapping existing API calls to use the new credentials object (was easy > since there was a trivial wrapper object that allowed you to convert new > credentials object into the old type that some API client libraries only > supported) > * a bunch of documentation and trivial plumbing issues > > On Fri, May 14, 2021 at 5:33 PM Ahmet Altay <al...@google.com> wrote: > >> +Valentyn Tymofieiev <valen...@google.com> might have an idea. >> >> On Mon, May 3, 2021 at 4:12 PM Chuck Yang <chuck.y...@getcruise.com> >> wrote: >> >>> Hi Beam devs, >>> >>> I saw there has been some previous discussion [1][2] around removing >>> the deprecated oauth2client dependency and using the supported >>> google-auth dependency instead. A portion of this work seems to >>> involve migrating off of google-apitools since this call [3] is not >>> supported by credentials objects emitted by google-auth. >>> >>> Does anyone have any experience/insights on how much work migrating >>> off of oauth2client would involve? I might be able to help out but >>> wanted to see a) if anyone is already looking at this and b) if there >>> are any hidden obstacles beyond needing to move from google-apitools >>> to the google-cloud-* libraries. Any pointers are appreciated! >>> >>> We're interested in this migration because of the need to use custom >>> token URIs for issuing service account tokens--it's supported by >>> google-auth but not oauth2client. >>> >>> [1] https://issues.apache.org/jira/browse/BEAM-7352 >>> [2] https://github.com/google/apitools/issues/225#issuecomment-434884589 >>> [3] >>> https://github.com/google/apitools/blob/v0.5.31/apitools/base/py/base_api.py#L266 >>> >>> Thanks! >>> Chuck >>> >>> -- >>> >>> >>> *Confidentiality Note:* We care about protecting our proprietary >>> information, confidential material, and trade secrets. This message may >>> contain some or all of those things. Cruise will suffer material harm if >>> anyone other than the intended recipient disseminates or takes any >>> action >>> based on this message. If you have received this message (including any >>> attachments) in error, please delete it immediately and notify the >>> sender >>> promptly. >>> >>
