[
https://issues.apache.org/jira/browse/BEAM-11780?focusedWorklogId=550501&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-550501
]
ASF GitHub Bot logged work on BEAM-11780:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Feb/21 00:01
Start Date: 10/Feb/21 00:01
Worklog Time Spent: 10m
Work Description: y1chi edited a comment on pull request #13933:
URL: https://github.com/apache/beam/pull/13933#issuecomment-776327965
> 1. What is the process to update or regenerate the GCB code? How does one
find out What version of GCB used?
we can use
```
pip install google-apitools
gen_client --discovery_url=cloudbuild.v1 --overwrite
--outdir=third_party/py/apache_beam/runners/dataflow/internal/clients/cloudbuild
--root_package=. client
```
to generate the client.
I think the version is only bind to the api url version, in this case we are
using cloudbuild v1
https://github.com/apache/beam/blob/875e01e45d5bffbb06bde2b92c60c9dec1c383dd/sdks/python/apache_beam/runners/dataflow/internal/clients/cloudbuild/cloudbuild_v1_client.py#L40.
If client is updated it will become v1b1, v1b2 etc.
> 2. How do we account for GCB dependencies since top-level library is no
longer installed ? I am seeing following deps:
>
> ```
> google-cloud-build==2.0.0
> - google-api-core [required: >=1.22.0,<2.0.0dev, installed: 1.26.0]
> - libcst [required: >=0.2.5, installed: 0.3.17]
> - proto-plus [required: >=0.4.0, installed: 1.13.0]
> ```
I think if they are solely dependencies for google-cloud-build they won't be
needed anymore? Do we need some special handling?
>
> It seems strange that we are modifying the external library for internal
testing purposes. Can these or similar changes be done upon import instead?
It will add a lot of hacks into the importing process and I'm guessing we
were avoiding that before since there are also other gcp client vendored:
- apache_beam/runners/dataflow/internal/clients/dataflow
- apache_beam/io/gcp/internal/clients/bigquery
- apache_beam/io/gcp/internal/clients/storage
I think it's desirable to use the third_party libraries, but it seems to
cause a lot of frictions since the dependencies are often out of sync and have
different hacks with copybara internally.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 550501)
Time Spent: 1h 40m (was: 1.5h)
> Use vendor cloud build client library so that cloudbuild-python dependency
> can be removed
> -----------------------------------------------------------------------------------------
>
> Key: BEAM-11780
> URL: https://issues.apache.org/jira/browse/BEAM-11780
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow, sdk-py-core
> Reporter: Yichi Zhang
> Assignee: Yichi Zhang
> Priority: P2
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This will make dataflow internal testing with prebuild sdk container workflow
> much easier.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
