The GCR repository can be configured with public pull access, which I think will be required to use the container.
On Thu, Dec 21, 2017 at 2:34 AM, David Sabater Dinter < [email protected]> wrote: > +1 > Hi, > It makes sense to use GCR (locality with GCP services and works like any > other container repository), only caveat being that the images will be > private, in case anyone requires to debug locally will need access to pull > the image or build locally and push. > I agree getting closer to (a) is preferable assuming the build time > doesn't increase dramatically in the post commit process. > > On Thu, Dec 21, 2017 at 1:59 AM Henning Rohde <[email protected]> wrote: > >> +1 >> >> It would be great to be able to test this aspect of portability. For >> testing purposes, I think whatever container registry is convenient to use >> for distribution is fine. >> >> Regarding frequency, I think we should consider something closer to (a). >> The container images -- although usually quite stable -- are part of the >> SDK at that commit and are not guaranteed to work with any other version. >> Breaking changes in their interaction would cause confusion and create >> noise. Any local tests can also in theory just build the container images >> directly and not use any registry, so it might make sense to set up the >> tests so that pushing occurs less frequently then building. >> >> Henning >> >> >> >> On Wed, Dec 20, 2017 at 3:10 PM, Ahmet Altay <[email protected]> wrote: >> >>> Hi all, >>> >>> After some recent changes (e.g. [1]) we have a feasible container that >>> we can use to test Python SDK on portability framework. Until now we were >>> using Google provided container images for testing and for the released >>> product. We can gradually move away from that (at least partially) for >>> Python SDK. >>> >>> I would like to propose building containers for testing purposes only >>> and pushing them to gcr.io as part of jenkins jobs. I would like to >>> clarify two points with the team first: >>> >>> 1. Use of GCR, I am proposing it for a few reasons: >>> - Beam's jenkins workers run on GCP, and it would be easy to push them >>> to gcr from there. >>> - If we use another service (perhaps with a free tier for open source >>> projects) we might be overusing it by pushing/pulling from our daily tests. >>> - This is similar to how we stage some artifacts to GCS as part of the >>> testing process. >>> >>> 2. Frequency of building and pushing containers >>> >>> a. We can run it at every PR, by integrating with python post commit >>> tests. >>> b. We can run it daily, by having a new Jenkins job. >>> c. We can run it manually, by having a parameterized Jenkins job that >>> can build and push a new container from a tag/commit. Given that we >>> infrequently change container code, I would suggest choosing this option. >>> >>> What do you think about this? To be clear, this is just a proposal about >>> the testing environment. I am not suggesting anything about the release >>> artifacts. >>> >>> Thank you, >>> Ahmet >>> >>> [1] https://github.com/apache/beam/pull/4286 >>> >> >>
