Thanks for working on this. Do you happen to have publicly accessible snapshots published for your testing currently (even when the final location isn't sorted out)?
I would like to use a 2.16 based Python SDK image for working on my downstream project, but could not find anything in gcr.io/apache-beam-testing/beam/sdks/rc/snapshot Thanks, Thomas On Fri, Aug 30, 2019 at 10:56 AM Valentyn Tymofieiev <[email protected]> wrote: > On Tue, Aug 27, 2019 at 3:35 PM Hannah Jiang <[email protected]> > wrote: > >> Hi team >> >> I am working on improving docker container support for Beam. We would >> like to publish prebuilt containers for each release version and daily >> snapshot. Current work focuses on release images only and it would be part >> of the release process. >> >> The release images will be pushed to GCR which is publicly >> accessible(pullable). We will use the following locations. >> *Repository*: gcr.io/beam >> *Project*: apache-beam-testing >> More details, including naming and tagging scheme, can be found at wiki >> <https://cwiki.apache.org/confluence/display/BEAM/%5BWIP%5D+SDKHarness+Container+Image+Release+Process> >> which >> is written by several contributors. >> >> I would like to discuss these two questions. >> *1. How many tests do we need to run before pushing images to gcr*? >> Publishing artifacts is the last step of the release process, so at this >> moment, we already verified all codebase. In addition, many Jenkins tests >> use containers, so it is already verified several times. Do we need to run >> it again? >> > > In a docker repository, one container image can have multiple tags. One > possibility is that on the last step of the release process, after > sufficient testing, we place a production tag on an image that was already > pushed with a dev tag. > > For example a dev tag may look like: > gcr.io/apache-beam/python37:2.16.0-RC4, and production tag may look like: > gcr.io/apache-beam/python37:2.16.0 and both will refer to the same image > at the end. > > We should also plan what the process of updating the container image will > look like, if we need to release the image with additional changes, and how > we will test these changes before the final push (or placing production > tag). > > >> >> *2. How many tests do we need to run to validate pushed images?* >> When we push the images, we assume the images would work and pass all the >> tests. After pushing, we should confirm the images are pullable and >> useable. I suggest we run several tests on dataflow with each pushed image. >> What do you think? >> > > I think it makes sense to do - Beam runners that use SDK container images > should have some continuously running tests, which periodically check that > all supported images are pullable and still compatible with the runner. > > This work can be refined later as we explore more during our release >> process. >> Please comment or edit the wiki page or reply to this email with your >> opinions. >> >> Thanks, >> Hannah >> >
